TLDR: Effective agentic workflows depend on strong tool calling, the technical functionality that equips agents with specialized capabilities and applications. Unstable tool calling leads to unpredictable, inaccurate, and potentially dangerous agent behavior, which is more expensive to fix reactively. We discovered that tool calling accuracy and formats are inconsistent across different model-inference provider pairs, and we explored the sources of these inconsistencies. Proactive monitoring and evaluation of these instabilities can help developers make corrections in real-time, which maintains the health, accuracy, and effectiveness of agentic workflows.
Image credit: Machine Learning Mastery, 7th January 2026,
Understanding Tool Calling & Agentic Workflows
Tool calling is the mechanism that supercharges agents to take specialized actions across applications. Customer support agents need tools to access order history, process refunds, and more. If a military defense support agent uses the wrong tool, it may access the wrong data and make an incorrect decision.
However, unstable tool calling can accelerate agent drift, unpredictability, inaccuracy, and diminish safety in agentic workflows. Failures and lack of precision in agentic tool calling can lead to bad data records and/or unintended data exposure, which is exceptionally difficult to untangle, and can lead to context pollution. In multi-turn agentic workflows, which are becoming increasingly more common, these errors compound exponentially. Volatility in tool calling behavior is financially, logistically, and potentially geopolitically expensive to correct.
Multiplicative drift in quality output, amplification factor k > 1 when downstream agents magnify upstream mistakes.
\begin{equation}
Q_n = Q_0 \, (1 - k\varepsilon)^{n}
\end{equation}Q_n: remaining quality after n agent steps
Q_0: Baseline quality, a starting score
k: Amplification factor, how much a downstream agent magnifies upstream mistake
e: Per step error rate
n: number of sequential agent steps
Model eval platforms like Artificial Analysis and BenchLLM excel at rating model’s intelligence, latency, tool use, and token cost per task, but are limited to closed, controlled environments that are relatively predictable. Real-life workflows introduce complexities and unique vulnerabilities for agentic tool calling that benchmarks cannot capture. Tool schemas and structures vary across enterprises; intensity and frequency of requests oscillate across users and applications. A customer support agent will have distinct tool use compared to a platform security agent, even if both are powered by GLM 5.2 on Baseten. Models are non-deterministic, and their performance fluctuates across different inference providers, adding another layer of unpredictability. Model behavior can also change throughout the day, depending on the inference load on hardware. Additionally, tool calling formats are not necessarily universal. Some models make a model-native tool call, which the inference provider is responsible for translating into a more common JSON format.
Our Findings With Tool Calling Failure
To evaluate tool calling consistency, we designed a suite of 15 probes that ping various models on different inference providers and collect data on the accuracy of tool and argument selection, syntax, latency, and more. We categorize failures into two major categories: serving-based fault or model-based fault. A serving-based error occurs when a model-native tool call is prescribed, but it was not translated into the JSON format expected by the API contract. Model-based faults occur when there are inaccurate tool or argument selections, or invalid model-native tool call syntax.
Below are some results from the Gemma 4 and LFM2 models examples illustrating our error attribution:
Gemma 4 produced the following on Novita:
<tool_call>get_favorite_singer{}</tool_call> on Bedrock and 触call:get_favorite_singer{}<tool_call|> Yet the same model with the same probe produced a robust JSON formatted tool call on TogetherAI:
{”id”: “chatcmpl-tool-8aee59d750b3bee2”, “type”: “function”, “function”: {”name”: “get_favorite_singer”, “arguments”: “{}”}}LFM2-8B-A1B produced the model native tool call below, which the inference provider did not correctly translate due to the “-0.5” argument. The JSON format determines which tool and arguments are called.
<|tool_call_start|>[get_square(number=-0.5), get_least_favorite_singer(), get_square(number=-0.5), get_favorite_singer()]<|tool_call_end|>All values in percentages, as of August 27th, 2026
The error rates on both Baseten and TogetherAI are highest from 8am-10am PST and lowest during 2am-4am PST, potentially indicating volatile model performance is volatile during peak business hours when usage loads on hardware is high.
LFM and similar models are built to be run locally, so it is possible that the observed incorrectness is due to a serving-stack mismatch. However, smaller models are increasingly used as cloud sub-agents for long-horizon tasks, many of which are complicated tasks with high stakes outcomes. In these situations, tool calling precision at scale is key for agent functionality and credibility.
To establish a control, we ran the models locally with Ollama, isolating its behavior from provider-specific serving decisions to evaluate raw model configurations. In these instances the total error rate drops to 0.53%, nearly eliminating the failures observed on Baseten and TogetherAI. This significant difference suggests that inference provider customizations of both the raw model (e.g. specific quantization techniques) and the serving framework are at the root of these tool calling errors. AI accuracy is dependent on the raw model itself, and the way it is served.
Gemma 4 31B error rates vary greatly across inference providers. 100% of these errors, across all inference providers, are model-based faults. In other words, model-inference provider pairs have a wide range in error rates and high variance in distribution of error types. Traditional benchmarks miss these nuances in model performance that occur in live deployments.
All values as percentages (%) as of August 27th, 2026
*On Bedrock, the vast majority (98%) of wrong tool call function errors were due to selecting the wrong function versus selecting no function at all.
Different error types require different remedies; for example, understanding why the model is choosing the wrong function is a different problem than correcting the way the model’s raw output is parsed.
Limits of Self Correction
These discrepancies are the source of many systematic agentic failures. Using a fallback endpoint does not guarantee consistent behavior, nor does using an alternate model on the same inference provider, because each inference provider serves models distinctly. The performance of each model-provider pair is unique, therefore, agents powered by distinct pairs have distinct vulnerabilities.
Some agents and agent harnesses have capabilities to self correct. Often, this requires retrying several tools until the correct one sticks. This exponentially increases token usage and latency, making it a bandaid solution to a structural problem. In complex agentic systems that require sophisticated tool calling, even a 1% error rate can compound in the system and lead to drastically poor agent utility.
Successful agentic workflows cannot be built without strong tool calling, because it is the core functionality that allows agents to take actions. Agentic workflows are adaptive environments where tool schemas are continuously revised and restructured. Understanding points of failure in the system improves AI safety and effectiveness for all contributors in the AI stack. Proactive monitoring can alert users when there is a higher risk of failure, and help users make more informed decisions for agent steering. Tracing unexpected behavior in agentic systems is the core of strengthening AI safety, usefulness, and validity.