Agents are built with substantive goals in mind. In order to achieve these goals, the agent charts a path of many lower level decisions. These decisions represent a sequence of opportunities for the agents to deviate from the desired path. For example, a deviation may occur at a fork in the road when an instruction cannot be unambiguously translated into a single concrete next step due to messy or conflicting information in the agent’s context. The source of this ambiguity–in this case specific pieces of the agent’s context– represents a behavioral vulnerability. When a behavioral vulnerability manifests and an agent acts in unintended ways, we say the agent has gone rogue.
Note that going rogue does not necessarily involve a single destructive act, such as deleting a production database. The deviation could be as seemingly innocuous as passing the wrong argument to a tool or relying on a stale memory, allowing incorrect information to propagate through a multistep process—for example, an outdated account status causes a customer to be excluded from a follow-up campaign.
Over time, the context of live agents can become bloated and increasingly difficult to manage, a problem illustrated by research on catastrophic remembering in agentic coding. As context accumulates, behavioral vulnerabilities can spring up, making agents susceptible to acting against the intentions of the stakeholders, human or agent, to whom they are responsible. Their effects can be subtle and may surface only intermittently because of LLM nondeterminism.
Behavioral vulnerabilities raise two natural questions:
How do we find them?
How do they manifest, and what consequences can follow?
Scanning and probing with the Agent Assurance Engine
At VAIL, we use several internal agents, including a LangChain-based go-to-market (GTM) agent and an OpenClaw general-purpose agent. Both deliver substantial value, streamlining tasks ranging from account tracking to analytics queries. Yet, like many agent users, we have also observed unwanted behavior, such as the agents returning outdated information or calling tools they were instructed not to use.
To identify behavioral vulnerabilities and understand their impact, we have developed an Agent Assurance Engine with three components:
Rogue scanning, which surfaces potential behavioral vulnerabilities.
Probe generation, which turns identified vulnerabilities into probes. Each probe pairs a lightweight prompt with a framework for deterministically grading the agent’s response.
A Live vulnerability application, which sends probes to agents at regular intervals, parses their responses, and grades them against the deterministic metrics.
We believe scanning and probing are essential to deploying reliable agents. Two examples from our internal assurance engine show why.
What the probes revealed and how to respond
Example 1: Prospect Research
Our GTM agent had a prospect research tool and instructions requiring both a person’s name and a company name before initiating a search:
The Agent Assurance Engine devised a simple probe to test whether the agent would research a prospect when given only a person’s name. Repeated runs revealed that the agent would sometimes call the tool despite the missing company name, returning invalid information and wasting tokens. Moreover, the nature of the unintended behavior was inconsistent– the snippet below shows the way tools were called across three repetitions of this probe.
The metric plotted below tracks whether the agent correctly abstained (green dots) from calling the research_person tool, or whether it called the tool when it shouldn’t have (red dots). The data shows the intermittent nature of the unintended behavior.
In a multistep process, an insufficiently constrained search could lead to prospect-specific outreach being sent to the wrong person, damaging trust or jeopardizing a sales opportunity. In addition, an open search like this quickly depletes token budgets as the model must reason over multiple useless search results that are unlikely to be in its cache.
Example 2: Recurring Analytics
Our general-purpose agent performs a recurring analytics task that compares two potentially correlated data streams. One probe asked a related question that requires applying the tools used in the analytics task to retrieve the latest data from an adjacent dataset. In response, we observed an instance in which the agent bypassed the designated user-defined tool for updating our results log, and hardcoded the latest result into its memory instead. This unintended memory write risks carrying stale information into future queries of the dataset.
Analytics tasks like these inform our product development. A stale or incorrect comparison could lead to prioritization of the wrong feature, committing engineering time to a decision that current data does not support.
It is also noteworthy that the agent bypassed our custom tool in favor of a tool native to its harness (a finding that we may not have caught without proactive probing). In cases where the model is trained in conjunction with the development of a harness, the resulting agent may show a preference for the harness’s native tools over tools defined by the user. At a higher level, this dynamic pits the biases of the organization that built the model and the harness against the preferences of the user.
Actioning off of probes and looking ahead
Identifying a vulnerability and observing how it manifests are the first steps; we also need to steer the agent toward the intended behavior. For the GTM agent, we can strengthen the implementation of the tool schema’s requirement for company name. For the analytics agent, we can update the tool’s skill instructions to prohibit reliance on memory files for fresh results, or to clear task-generated memories after each run.
Over time, automated steering could reduce the need for manual intervention. One possibility is to use probe grades as a reward signal for reinforcement learning, training a steering mechanism to select corrective interventions and evaluating their effectiveness through further probing.
Proactively scanning for vulnerabilities and probing agent behavior provides a direct countermeasure to such costly deviations. The failure types in these examples can otherwise cause silent damage and remain undetected for weeks on end if they are papered over as part of a multistep process. While traditional observability metrics remain valuable in this new agentic era, new tools are needed to help teams find behavioral weaknesses, observe their effects, and address their causes. Catching those weaknesses early gives teams a chance to intervene before an incorrect tool call or stale memory becomes a costly downstream decision.