VAIL — Agent Assurance Engine

Peace of mind that your agents are working.

Your business now runs on models, APIs, and agents operated by someone else, and changing without notice. The Agent Assurance Engine watches your endpoints continuously from your network, confirms they're working the way you need them to, tells you the moment something changes, and what to do next. The probes it sends are built in the shape of your own applications and agents, so what it checks is what you actually depend on.

Learn more See how it works
How it works

In the background, never in the way.

The Agent Assurance Engine is a lightweight service that runs on your terms. It only needs permission to reach the same AI services your company already uses and it does not access sensitive production traffic. Your IP stays yours, guaranteed.

YOUR NETWORK · INSIDE THE FIREWALL Agents & apps Codex · Claude Code · OpenClaw AI Gateway / Router Databricks · Vercel · Cloudflare Agent Assurance Engine Your telemetry & SIEM OTel · Prometheus · Splunk · Datadog findings & alerts steer routing & policy steer agents PRODUCTION ENDPOINTS api.openai.com/v1 OpenAI · Anthropic · Google api.fireworks.ai/inference/v1 Fireworks · Together AI · Bedrock api.salesforce.com/einstein Salesforce · ServiceNow · Zendesk vllm-gpu01.internal.corp:8000 vLLM · SGLang · Ollama production traffic agent-shaped probes · no customer data
Running in minutes
A single container for Docker or Kubernetes, pointed at the services and agents you want watched. Prefer zero upkeep? VAIL runs it for you as a managed service.
Inside your walls
Checks come from your own network, under the security rules you already enforce. Nothing new is exposed to the internet, and no customer data is ever read.
Works with your tools
Findings land in the dashboards and alerting your teams already watch — Datadog, Grafana, Splunk, Slack, PagerDuty — not another console someone has to remember to check.

Out of the request path, but not out of the loop. What the Engine finds can feed straight into the routing decisions and governance policies your AI gateway or router enforces, so traffic shifts and access tightens the moment something changes.

What we send

Probes shaped like your agents.

An endpoint can pass every generic health check and still be wrong for you. What matters is whether it still does the specific things your applications and agents need: the tools they call, the formats they parse, the multi-step work they carry out. So those are the things the Engine tests, with probes built in the shape of your own workloads.

The usual approach

A generic check asks: is it up?

Uptime monitors ping the endpoint and time the response. Public benchmarks score the model on tasks it was likely tuned for. Both can report perfect health while the endpoint quietly stops doing the one thing your agent depends on.

You find out from a failed workflow, an unhappy customer, or a support ticket, days after the behavior actually changed.

Tells you it works for someone
The Engine's approach

An agent-shaped probe asks: is it still right for us?

We map how each important workload actually uses the model: its tool schemas, prompt patterns, output contracts, context sizes, and what a correct result looks like. Each one becomes a probe with that same shape.

Running on a schedule against every endpoint that workload can reach, it answers the question a generic check cannot: would our production agents still succeed right now?

Tells you it works for you
Worked example · one workload, four probes runs in seconds
The production workload
support-triage-agent

Reads an incoming support ticket, looks up the customer, searches the knowledge base, and returns a structured triage record for the queue. Runs tens of thousands of times a day.

What it depends on
  • Two tools, called in order: lookup_customer, then search_kb
  • A strict JSON result: severity, category, suggested reply
  • Eight to twelve thousand tokens of context per call
  • Escalating when a ticket is ambiguous, never inventing a customer ID
The probes it becomes
tool-selection
A synthetic ticket that clearly needs a customer record. Does the model call lookup_customer first, with well-formed arguments?
Pass · correct tool, valid arguments
schema-conformance
A routine ticket. Does the reply parse against the triage schema, with severity inside the allowed set?
Pass · parses clean, severity in range
context-recall
A twelve-thousand-token ticket thread with the deciding detail buried in the middle. Is the category still right?
Pass · correct category
escalation-guardrail
A ticket missing its account number. Does the agent escalate instead of inventing one?
Pass · escalates, no fabricated ID
Four small probes, a few seconds each, run against every endpoint this agent can reach. When tool selection starts slipping on your primary provider at 02:00, the platform team knows at 02:05, not when the support queue backs up at 09:00.

A probe suite that mirrors production, and keeps up with it.

01
Map the shape
We capture how each important workload calls the model, and what a good answer looks like. No code changes.
02
Build the suite
Each workload becomes a probe exercising the same tools, formats, and steps on synthetic inputs.
03
Run everywhere
Against every endpoint that workload can reach: your provider, your fallbacks, your own deployments.
04
Keep it in step
When a workload changes, its probe changes with it, so coverage never drifts from production.

The Engine copies the shape of your workloads, never their contents. Probes reproduce the tools, formats, and steps your agents rely on and run entirely on synthetic inputs: no customer data, and no production traffic touched.

What it watches

The concerns that grow with every agent you launch.

Every quarter, agents take on more of the work your business runs on — and the stakes of one quietly misbehaving grow with them. The Engine keeps answering the two questions that matter most, around the clock, and delivers each answer to the team that needs it.

Is your agent behaving the way you expect?

When an AI workflow starts failing, the first question is always "is it us or them?" Using probes shaped like your own workloads, the Engine tracks the specific behaviors your agents rely on: using tools correctly, returning well-formed responses, and following your instructions. Your team can tell in minutes whether a problem lives in your code or on the provider's side, instead of losing days to debugging the wrong thing.

Answers Is the endpoint behaving consistently for agents? · Us or them?

Are your agents still on course?

Agents evolve: their instructions, memory, and tools change over time, and their behavior shifts with them. The Engine keeps a running picture of how each agent is behaving, so a slow drift off course shows up as a trend you can act on early, not a surprise you discover after it's already caused damage.

Answers Are our agents still on course? · What changed their behavior?

When something changes, the right person already knows — and knows what to do.

AI platform lead · Platform engineering
Incident reports, before they're incidents

The Engine surfaces Lightweight probes, designed to match the shape of your own applications and agents, run continuously against each endpoint. When responses start breaking the patterns your workloads depend on, a PagerDuty alert fires — faster and cheaper than finding out in production.

The action The platform lead sees exactly which applications are about to be affected and acts before they fail — a fix shipped on their schedule, not a post-mortem written after the outage.

Engineering lead · Agent & app teams
Tool calling starts to slip

The Engine surfaces Consistency probes show tool-call success down 5% over seven hours — on the provider's side, not in your code. The trend lands on the team's Datadog dashboard.

The action The lead skips the internal fire drill, files a provider ticket with the evidence attached, and shifts the affected agent to a backup model in the meantime.

AI governance lead · Security & compliance
A coding agent drifts off course

The Engine surfaces Trajectory tracking shows an agent's behavior trending away from its baseline. A Slack alert connects the drift to the model change that started it.

The action The governance lead tightens the agent's access policy and quarantines the endpoint pending re-approval — with the whole timeline already logged for auditors.

See it working

The same capabilities, already proving themselves.

You don't have to take the Engine's approach on faith. The kinds of information it collects are already on display — in public, and with partners — so you can judge them for yourself before any conversation.

Cisco partnership

Provenance Explorer

VAIL provides the model identity and similarity matching behind Cisco's Provenance Explorer — a resource for compliance teams evaluating models, powered by the Cisco AI Security Framework. It's the same matching the Engine uses to confirm which model is really answering an endpoint, and how close it is to the one you approved.

Live · Updated hourly

Stability Arena

Our public dashboard demonstrating how the Engine measures endpoint stability — including quiet changes and provider-to-provider differences in behavior that ripple into application and agent workflows. What the Arena does in public, the Engine does for your services, privately, inside your own network.

Open Stability Arena
Peer-reviewed research

Built on published methods

The Engine's checks aren't a black box either. The methods for verifying model identity and detecting change, and for tracking agent behavior over time, are published, peer-reviewed research — presented at ACM CAIS 2026 and ICML 2026.

Endpoint stability paper Agent trajectories paper

Stop wondering whether your AI changed. Know.

Tell us which models, providers, and agents your business depends on. We'll show you what the Engine would watch from day one — and how the answers show up in your own dashboards.

Nothing to install in your apps. No customer traffic touched. Checks run from inside your own network — or from VAIL's, if you prefer the managed service.