Fabricated Identifier Hallucinations in Tool-Calling Agents

A fabricated identifier is a key that unlocks a door in a system that has no idea the key is fake, and once the door opens, the agent walks through it with full confidence.

Why fabricated identifiers are a distinct failure class

Text hallucination produces a false sentence. A reader can check it against a source, catch the error, and move on without consequence to anything but the reader's trust. Fabricated identifier hallucination works differently: an agent invents an order ID, an employee number, or a policy reference that does not exist, hands it to a tool, and the tool runs. The identifier is the bridge between what the agent intends to do and the state of a real system. A wrong word in a paragraph is a wrong word. A wrong order ID passed to a refund tool is a wrong key into a live account, and the system on the other end has no way of knowing the key was never issued.

This distinction is visible in how researchers categorize agent failures. That gap reflects a basic fact about what a tool call is: a statement can be wrong and sit there, inert, waiting to be fact-checked. An action has already happened. The consequences of a fabricated identifier do not live in the next token the model generates. They live in whatever system received the call, which makes this a problem of state, not of text, and it needs its own way of being watched for.

The mechanism: why the tool running is what makes this dangerous

The sequence is short and, in isolation, each step looks ordinary. The agent fabricates an identifier, perhaps because the real one was never retrieved, perhaps because the model's confidence outran its grounding. It calls a real tool with that identifier as an argument. The tool, having no reason to doubt the input, executes against a real system: a customer database, a ticketing platform, a payment processor. The tool returns a result. And because the result looks like the kind of thing a successful call returns, the agent proceeds as if everything upstream was correct.

From the infrastructure's point of view, nothing went wrong. The call returned 200. Latency sat in the normal range. No exception fired. Every alert built to catch a crash, a timeout, or a dropped connection stays exactly where it was, silent, because none of those things happened. What happened instead is quieter and worse: the tool may have returned a record that is real but attached to the wrong entity. A policy window that genuinely exists, just for a different transaction. A tracking label that is valid, just for someone else's order. The data is not corrupted. The reference to it is.

This is what separates a fabricated identifier from a crash. A crash stops the agent in its tracks, and someone eventually notices the thing that didn't run. A successful call built on a fabricated identifier does not stop anything. It redirects the agent onto a trajectory that is false from that point forward, and because the agent has no internal signal that anything diverged, it will treat every subsequent step as though the ground beneath it is solid. It will defend that ground. It will build on it. From the infrastructure perspective, the call returned 200, latency was normal, no exception was raised, and every conventional alert stayed silent. Production monitoring for AI agents has to work differently from traditional error detection for exactly this reason: platforms like Lemma audit traces not against whether something crashed, but against what the agent was actually instructed to do, which is the only way to surface a divergence that standard observability was never built to see.

How a fabricated ID compounds across a multi-step agent workflow

Once a fabricated identifier enters an agent's working context, it does not stay contained to the step that produced it. Every later step that reads from or writes to that context inherits the error without any signal that the ground has shifted. A wrong customer ID handed to a refund tool gets passed again to a notification tool, and again to a compliance log. That is three separate wrong actions, and each of them looks locally correct because each tool did what it was asked to do with the input it was given.

In systems built on multiple cooperating agents, the exposure widens further. A fabricated ID written into shared memory propagates to every other agent that later queries that memory, and the audit trail across the whole system can look perfectly clean even though the underlying decision was wrong from the first step. The math behind this is not additive, it compounds: a per-step hallucination rate that looks modest on its own produces a trajectory failure rate that grows substantially once a workflow runs five or more steps, a property AgentHallu's own results reflect. And because tool-use hallucinations are, per that same benchmark, the hardest category to detect automatically, this compounding mostly happens without anyone watching it happen. AgentHallu's 693 trajectories, spanning 7 frameworks and 5 domains, point to the same conclusion from a different angle: the failure mode is not rare or framework-specific, and detection lags behind it everywhere it was tested.

The clearest illustration of what this looks like in practice comes from a documented production run: nine days in production, one ticket in fourteen handled incorrectly, and not a single dashboard flagged it. The failure was invisible for the entire run because nothing in the monitoring stack was built to ask the question that would have caught it.

Stronger reasoning models make identifier fabrication more likely, not less

The obvious response to all of this is to assume a smarter model will simply make fewer mistakes. That assumption does not hold. A paper, "The Reasoning Trap: How Enhancing LLM Reasoning Amplifies Tool Hallucination," presented at ICLR 2026, found that the same reinforcement learning used to sharpen a model's reasoning also raises its rate of hallucinated tool calls. Task performance goes up. The rate at which the model invents a tool call instead of admitting it cannot complete the task goes up too.

The cause is structural. Reasoning-focused RL training disproportionately collapses the representations a model relies on for tool reliability, and the paper traces this collapse to the late layers of the network, the same layers that would otherwise be positioned to restrain a bad call before it goes out. Training the model to reason harder wears down the part of the network built to say no. The researchers tested two of the standard fixes, better prompting and direct preference optimization, and found that neither closed the gap. They describe the result as a reliability-capability trade-off: the two properties move in opposite directions as training intensifies, and one cannot simply be dialed up to compensate for the other.

For a team evaluating which model to put in front of a live tool, this closes off a response that otherwise seems like the easy one. Upgrading to a more capable reasoning model does not make identifier fabrication less likely. It can make it more likely, and that failure mode exists in every model currently available for production use, not in some future version that hasn't shipped yet. Waiting on a training-time fix is not a plan a production system can run on today.

The external threat surface: when fabricated identifiers are injected, not generated

Not every fabricated identifier originates from the model's own uncertainty. Some are planted. The Model Context Protocol, now widely used to connect agents to external tools, introduces an attack surface where malicious instructions hidden inside a tool's description or metadata can cause an agent to fabricate or swap in an identifier before the tool call ever runs. The agent's own instruction-following ability, the trait that makes it useful, becomes the mechanism an attacker exploits.

A benchmark called MCPTox, built on over 45 real-world MCP servers across 8 application domains, tested this directly and found that more capable models are often more susceptible to these tool-poisoning attacks, precisely because the attack works by exploiting superior instruction-following. The pattern runs parallel to the Reasoning Trap finding: a more capable model is more exposed, and here the exposure concerns security. The resulting failure looks identical to accidental fabrication from the outside. The tool runs, it returns a result, the agent carries on, and nothing in the surrounding infrastructure raises a flag. Work on defenses such as MindGuard has started to address this by inspecting an agent's internal decision process rather than trusting its final output, but the underlying lesson holds regardless of whether the fabrication was accidental or induced: a trace has to be checked against what the agent was supposed to do, not merely whether the call it made came back clean.

Traditional error monitoring and standard evals cannot see this failure

Every escape route a team might reach for here closes the same way. HTTP 200 with normal latency is the standard, near-universal signal that conventional monitoring treats as proof of health, and a fabricated identifier failure produces exactly that signal, every single time it occurs. There is no distinguishing mark at the infrastructure layer.

Log-level observability does not close the gap either. A log can show that a tool was called and that a result came back, but it cannot show whether the identifier inside that call was ever real. One documented case involved a practitioner with full access to the logs, the original prompt, and the agent's final response, and still no visibility into sixteen tool calls, three retries, one silent fallback, and two context truncations buried in between. The information that would have revealed the fabrication was never captured in a form anyone could inspect.

Standard evaluation suites built around scoring final answers miss this type of failure almost by design, because they are not looking at the steps in between. AgentHallu found that even its best-performing models stayed well below majority accuracy at localizing the specific step where a hallucination occurred on expert-annotated traces. The TRAIL benchmark, built by Patronus AI, found something similar from a different angle: on 148 expert-annotated traces, the best model scored only 11% joint accuracy across error category and location combined. Aggregate metrics can mask the problem further. One documented case showed accuracy holding flat across five evaluation windows in a row while output diversity collapsed substantially underneath that flat line. The evaluation itself handed out a false all-clear while the system's actual behavior was deteriorating. A trajectory-level evaluation framework published in 2026 was built specifically to address this blind spot: it targets phantom tool-call claims, missing mandatory tool calls, and policy drift, all behaviors that stay invisible if a team looks only at the final answer and become visible only once scores are tied back to the full trajectory and the turn-by-turn evidence inside it.

Auditing intent against action at the trace level

A monitoring system needs to answer whether the identifier the agent used was consistent with what the agent's own instructions actually authorized it to use. Answering that question requires keeping a record of intent alongside a record of execution, not just the execution itself.

Span-level tracing, built on OpenTelemetry, gives teams the standardized structure every tool call, every argument passed into it, and every value returned can be captured as a discrete span and checked against a model of what the agent was supposed to be doing at that point. That structure only becomes useful at scale if the deviations it surfaces are grouped automatically; no team can read every trace from every run by hand. What matters is catching the pattern, a particular class of identifier getting fabricated repeatedly, a specific tool argument drifting in a consistent direction, and turning that pattern into something actionable. Expecting a team to predefine every failure mode in advance does not work either, since fabrication patterns tend to emerge from live production traffic that no one anticipated at design time. A detection system has to be able to infer what the agent was supposed to do directly from the agent's own instructions, rather than relying on a checklist written before the system ever ran in production.

Corruption that has propagated across five or more tool calls is close to invisible to conventional monitoring. Catching these failures at scale depends on automatically grouping recurring patterns across traces. Tools built for this purpose can surface the moment an agent's actions diverge from its instructions, giving a team the chance to catch a compounding trajectory before it ever reaches a user. And the fix needs to land where developers already work: alerting through Slack and integrating directly with coding agents means a detected fabrication pattern moves straight into remediation instead of requiring a separate trip to a monitoring console. Lemma's approach, auditing every trace against the agent's own instructions, grouping deviations automatically, alerting in Slack, and plugging into coding agents directly, is built around this exact failure class, and it supports the frameworks teams are already running, including Vercel AI SDK, OpenAI Agents SDK, LangGraph, and Azure.

Building a continuous reliability practice around identifier fabrication, not a one-time fix

Fabricated identifier failures emerge from live production traffic, and they shift as models get updated and prompts get rewritten; no pre-deployment checklist can close the problem once and leave it closed. A reliable practice needs three connected stages working together: pre-deploy evaluation run against golden datasets with gates built into CI, production observability that captures live traces and scores as the agent runs, and regression debugging that takes a failed production trace and turns it directly into a test case the next evaluation run will check.

When the monitoring layer and the evaluation layer don't share data, the evaluation dataset goes stale the day the system ships. A fabrication pattern that first appears in week three of production will never be caught by an eval suite written before the system went live, because nothing fed that pattern back into the suite. Practitioners working on this problem have proposed concrete thresholds as a starting point: tool failures held at or below 2% per run, and a ceiling on how far the median number of steps in a trace is allowed to climb, both of which serve as proxy signals for identifier fabrication drift. A 2026 taxonomy called ToolFailBench, covering Tool-Skip, Result-Ignore, Output-Fabrication, and Unnecessary-Tool-Use, gives teams a structured way to categorize whatever failures show up in production and turn each category into a targeted regression case.

It is about building a loop where every production failure narrows the distance between what the agent was supposed to do and what it actually does the next time it runs, since reaching zero hallucinations before a system ships is not a bar any current model clears. A failure class that compounds silently, step after step, without tripping a single conventional alert, cannot be closed by a one-time audit before launch. It can only be closed by a system that keeps checking, continuously, for as long as the agent keeps running.