CHARM: The Framework Tackling Hallucinations That Silently Compound Across Agentic RAG Pipelines

The Hidden Failure Mode in Your Agentic RAG Pipeline

Retrieval-Augmented Generation (RAG) was supposed to be the cure for hallucination: ground the model in retrieved facts and it can't make things up. That logic holds reasonably well for simple, one-shot pipelines. But as teams push into agentic RAG - where an LLM plans, retrieves, reasons, retrieves again, and synthesizes across many steps - a new and subtler failure mode has emerged. A preprint posted to arXiv on June 3, 2026 gives that failure mode a name and, more importantly, a solution.

Multi-step agentic RAG pipelines remain vulnerable to a class of failure that existing hallucination detection mechanisms systematically miss: cascading hallucination, where errors introduced at early pipeline stages propagate and amplify across successive reasoning steps, producing confident but factually incorrect final outputs 1. The paper formalizes this as a distinct failure mode, presents a four-type taxonomy of cascade patterns, and introduces CHARM (Cascading Hallucination Aware Resolution and Mitigation), an architectural framework for detecting and interrupting error propagation in multi-step reasoning pipelines 1.

Why Existing Defenses Fall Short

The problem isn't that the field has ignored hallucination - it's that prior defenses were designed for a simpler world.

Frameworks like RAGAS evaluate retrieval-stage errors but are blind to what happens afterward: in CHARM's evaluation, RAGAS achieved only a 41.7% cascade detection rate (CDR) because it entirely missed inference and confidence inflation cascades, which occur after the retrieval stage it monitors 2.

Consistency-based approaches fare no better. Because downstream reasoning steps are coherent relative to a corrupted intermediate context, LLM self-correction suffers from severe confirmation bias - in head-to-head testing, self-correction achieved just a 12.8% cascade detection rate 2. The agent reinforces the cascade because the downstream reasoning appears logically sound relative to its corrupted memory.

This is the crux of why cascading hallucination is so dangerous. As the companion SoK survey on agentic RAG notes, "Agentic systems are inherently susceptible to cascading reasoning failures," and in a multi-step workflow, a minor hallucination or incorrect tool invocation early in the execution graph can propagate, leading to systemic failure 3. The model doesn't know it went wrong - and neither do the standard guardrails watching over it.

A parallel strand of recent research reinforces the point. A May 2026 paper from IBM and Columbia University researchers introducing the Trajel framework for auditing trajectory-level hallucinations in multi-agent industrial workflows found that most hallucination benchmarks still evaluate only the final output, missing failures that originate in intermediate Thought-Action-Observation steps - and that nearly half of hallucinated trajectories involve multiple hallucination types simultaneously 4. The compounding nature of these failures is precisely what CHARM is designed to intercept.

How CHARM Works

CHARM is designed as an architectural overlay - it runs alongside an existing agentic RAG pipeline rather than replacing it. Its four components - stage-level fact verification (SFV), cross-stage consistency tracking (CSCT), confidence propagation monitoring (CPM), and cascade resolution triggering (CRT) - operate alongside standard agentic RAG pipelines without requiring architectural replacement 1.

Each component targets a different vector of failure:

• Stage-Level Fact Verification (SFV) checks factual accuracy at every discrete reasoning step, not just the final answer.
• Cross-Stage Consistency Tracking (CSCT) flags when a claim made in a later reasoning step contradicts or drifts from what was established earlier.
• Confidence Propagation Monitoring (CPM) watches for anomalous confidence inflation - a telltale sign that the model has committed to a false premise and is doubling down.
• Cascade Resolution Triggering (CRT) is the enforcement layer, aggregating signals from SFV, CSCT, and CPM using a weighted voting scheme that reflects the relative reliability of each signal source 2.

When the CRT fires, it differentiates its response by severity: on a low-confidence cascade, it triggers lightweight automated mitigation such as re-retrieval; on a high-confidence cascade - where the system is confidently hallucinating a compounding error - it halts execution and routes the trajectory to a human approval checkpoint 1. Crucially, unlike SelfCheckGPT or RAGAS, CHARM's detection components require no additional LLM calls 2. Early cascade detection at an average stage of 2.1 also halts the pipeline before later, expensive stages execute, meaning end-to-end overhead is substantially lower than the raw per-stage figure suggests 2.

The Results

CHARM was evaluated on HotpotQA, MuSiQue, 2WikiMultiHopQA, and a custom adversarial dataset across LangChain agentic pipeline configurations 1. The headline numbers are strong: an 89.4% cascade detection rate with a 5.3% false positive rate and 215 ms ± 18 ms average latency overhead per stage, achieving an error propagation reduction of 82.1%, compared to just 18.5% for output-level detectors 1. Component ablations confirm that each detection module contributes meaningfully to overall cascade coverage 1.

The Broader Context: Agentic RAG Is Growing Up

CHARM arrives at a moment when the agentic RAG pattern is moving from demo to production. As one Towards Data Science analysis of agentic RAG observes, "Retrieval becomes a control loop: retrieve, reason, decide, then retrieve again or stop" 5 - that loop is what makes it powerful for complex queries, and it is exactly what makes it dangerous in production. Every iteration is a new opportunity for the agent to make a bad decision, and bad decisions compound.

This compounding dynamic is increasingly well-documented. A separate Towards Data Science piece on agentic RAG failure modes catalogs retrieval thrash, tool storms, and context bloat as the dominant production failure patterns 6. Each can feed into the kind of corrupted intermediate state that CHARM is designed to detect. Meanwhile, the broader SoK survey on agentic RAG identifies compounding hallucination propagation, memory poisoning, retrieval misalignment, and cascading tool-execution vulnerabilities as the central systemic risks in autonomous loops 3.

Why It Matters

For engineers building on top of LangGraph, LlamaIndex Agents, or any custom multi-step RAG orchestration, CHARM represents a practical conceptual blueprint, not just another benchmark. The four-component design is modular: teams could integrate SFV and CSCT alone if the primary production concern is factual drift, and layer in CPM later. The CRT's weighted voting also makes the system interpretable - when it fires, the audit log records which signal tripped it, feeding directly into compliance logging for enterprise review 2.

More broadly, the paper signals a maturation in how the community thinks about agentic reliability. The single-step hallucination framing - did the model make something up in this response? - is insufficient for systems that reason across a dozen tool calls and retrieval hops. Cascading hallucination requires a different mental model: treating the pipeline's intermediate state as a potential source of corruption that must be actively tracked, not just the final output that must be verified.