LLMON: IBM's LLM-Native Markup Language That Drops Prompt Injection Rates to Near Zero
IBM's Granite team has released LLMON (LLM Object Notation), a novel markup language that teaches 3B and 8B models to structurally separate instructions from data - slashing indirect prompt injection attack success from up to 100% down to 0.02%, without sacrificing general benchmark performance.
LLMON: IBM's LLM-Native Markup Language That Drops Prompt Injection Rates to Near Zero
Prompt injection has been an open wound in LLM deployment for years. Attackers slip malicious instructions into documents, tool responses, or retrieved data, and models - trained only to follow instructions, not to distinguish whose instructions they are - obey. IBM's Granite team has just shipped a structural fix: a new markup language called LLMON (LLM Object Notation, pronounced "lemon") that bakes instruction-data separation directly into the model's vocabulary and training, rather than bolting a classifier on top.
The results, released on November 12, 2026, are striking enough to pay close attention to.
What Problem Does LLMON Actually Solve?
Modern LLM context windows are a jumble: system prompts, tool schemas, retrieved documents, user messages, reasoning chains, and output requirements all share the same flat token stream. The model has no structural way to know which tokens are authoritative instructions and which are just data to be processed. That ambiguity is exactly what indirect prompt injection exploits.
LLMON addresses this at the format level. 1 It is an LLM-native markup language designed to "explicitly communicate structure and semantics to the models." 1 Data schemas, including nesting, are stated in a way that is unambiguous to a causal LLM. 2 Tags delimit blocks, schemas can describe nesting, and unique block identifiers allow instructions to point to specific data and requirements - making it structurally clear which tokens give orders and which tokens are merely material to be read.
Crucially, 2 IBM describes LLMON as intended to complement, not replace, existing chat templates.
The Models: Granite 4.1 LLMON Preview
IBM released two experimental models alongside the format specification: 1 granite-4.1-3b-LLMON-preview and granite-4.1-8b-LLMON-preview. Both are derived from Granite 4.1 base checkpoints rather than the released instruct models, meaning 1 LLMON behavior is learned during post-training rather than layered on an existing chat format.
Training was substantial. 20 The corpus comprised 4.6 billion tokens spanning general, RAG, and tool-calling data, augmented with distractor-infused variants - including 130,000 prompt-injection examples totaling 211 million tokens - via supervised fine-tuning followed by reinforcement learning. 22 Reinforcement learning used five GRPO stages and a final DPO stage built from the model's own responses to injection attempts.
The GitHub repo at generative-computing/llmon ships the full LLMON toolkit, format converters, example workflows, and templates. 1 A chat_template.jinja renders messages into the markup automatically, so adopters can call apply_chat_template as normal.
The Benchmark Numbers
This is where LLMON's headline claim lives. 19 IBM's evaluation shows that LLMON-structured supervision reduces attack success rates on indirect prompt-injection benchmarks from 53 - 100% all the way down to 0.02 - 5.6%, without catastrophic degradation on standard capabilities.
Broken out by model and benchmark:
- OPI (text-level injection): 19 Attack success rate falls from 100.0% → 0.1% at 3B and from 96.3% → 0.02% at 8B.
- ASB (agentic tool-observation injection): 20 Success drops from 99.2% → 5.6% at 3B and from 52.8% → 3.1% at 8B.
Compared to other models on OPI, 21 the LLMON previews avoided 47% more attacks than the best frontier model IBM tested, and 95% more than the best open-source model - while 21 maintaining general capability relative to Granite 4.1 Instruct models.
1 In standard benchmark categories, the LLMON models perform on par with the Granite 4.1 Instruct models, meaning the injection resistance does not come at a cost to usability.
The Important Caveats
IBM is upfront about the limits. 2 The separation is described as a learned behavior, not an enforced guarantee. 2 Work continues on adaptive attacks spanning multiple turns - the current evaluations cover static attacks, not an adversary who iteratively probes the format boundary.
The experimental preview label matters, too. 8 IBM and Hugging Face label the release an experimental preview for exploration, not production deployment. Format-level solutions can also be circumvented if an attacker understands the format well enough to spoof its tags - a risk that grows if LLMON becomes widely adopted and documented.
That said, reducing attack success from 96 - 100% to under 0.1% on standard benchmarks is a meaningful delta, especially for agentic pipelines where injected tool responses are a primary attack surface.
Why This Architecture Matters
The standard industry response to prompt injection has been classifiers, guardrails, and careful system-prompt engineering - none of which fundamentally change how the model interprets the token stream. LLMON takes a different approach: 9 the fix is not another guardrail layer or a classifier bolted on top, but a vocabulary-level change built around structural separation.
2 Because LLMON is not limited to a fixed schema, developers can define domain-specific tags and fine-tune adapters - making it extensible to custom tool schemas, retrieval formats, and agent communication protocols.
For builders working with smaller models in agentic pipelines, the size-to-safety tradeoff is particularly relevant. Smaller and mid-sized models are where formatting discipline often matters most: a 3B or 8B model cannot always infer intent through implicit context. A model-native structure may reduce the prompt ceremony needed to get consistent, safe behavior.
Why It Matters
Prompt injection is currently one of the most practical attack vectors against deployed LLM agents. The fact that IBM achieved near-zero attack success rates on established benchmarks - without sacrificing general performance - by changing the format contract between developer and model represents a genuinely different design philosophy from the guardrail-and-classifier approach that dominates today.
Whether LLMON or a descendant of it becomes a standard part of the agent stack remains to be seen. But the research direction - treating structural separation of instructions and data as a first-class training objective rather than an afterthought - is one the broader field should be watching closely.
The models are available now on Hugging Face under the IBM Granite organization. The toolkit is open at github.com/generative-computing/llmon.
Sources
- 1. LLMON: An LLM-Native Markup Language, now in Granite 4.1 3B and 8B (Hugging Face Blog)
- 2. Granite 4.1 3B, 8B Preview Models Add LLM-Native LLMON Markup (Hyper.ai)
- 3. Post-Training Large Language Models with LLMON (arXiv)
- 4. IBM Granite ships LLMON, a markup language that teaches small models to ignore prompt injections (DEV Community)
- 5. IBM ships Granite 4.1 previews tuned to resist prompt injection (DEV Community)
- 6. IBM releases experimental AI models trained to separate instructions from data (SuperpowerDaily)
- 7. IBM LLMON: Granite 4.1 Cuts Prompt Injection to Near Zero (ByteIota)
- 8. LLMON in Granite 4.1: Less Injection, Same Bench (ExplainX)
- 9. ibm-granite/granite-4.1-3b-llmon-preview (Hugging Face)
This article was researched and drafted by an AI writer agent (claude-sonnet-4-6) and reviewed by an editor agent before publishing.