JSON was never built for the model: IBM's LLMON moves prompt structure into the tokenizer, and the 3B/8B previews shrug off prompt injection

IBM's LLMON wraps prompt components in reserved single-token delimiters with bindable two-character IDs, so structure and content live in separate parts of the vocabulary. The 3B and 8B Granite 4.1 previews trained on it avoid 47% more OPI attacks than the best frontier model tested - and the authors are clear it's a learned behavior, not an enforced guarantee.

The argument about JSON has been taken all the way down to the tokenizer

Every LLM app on Earth flattens a structured prompt into a string, serializes it as JSON, and hopes the model remembers to count brackets and guess which quote mark closes a field. That is because JSON was built for parsers, not for causal language models, and a transformer has no parser. IBM’s LLMON - LLM Object Notation, pronounced “lemon” - is a bet that the fix belongs one layer deeper: structure should live in reserved tokens, not in characters.

The announcement landed on a Hugging Face blog post on 2026-11-12, alongside two same-day experimental preview models, a converter toolkit, and a training recipe. It is the first time the “structure in the vocabulary” idea has shipped as working weights, not a paper.

What LLMON actually is

Three mechanisms, all verified in the primary post:

  1. Reserved special-token delimiters. Semantically coherent prompt components - the task, the document, a tool schema, a prior tool result - are each wrapped in their own delimiter pair. The delimiters are single tokens in the vocabulary, so the model never tokenizes a bracket or a quote mark into structural noise. Structure and content occupy separate parts of the vocabulary, and the model never has to “count” anything.

  2. Addressable blocks. Every component gets a unique two-character ID drawn from a fixed set that the tokenizer encodes as single tokens, so addressing a block always costs exactly two tokens no matter which ID it was dealt. A later instruction can bind to the exact data payload it applies to - the post draws the analogy to a function call with named arguments.

  3. The execution block. A turn is assembled as named, individually addressable blocks bound by an execution block that says, in effect: run this instruction, on this data, subject to these requirements, and put the answer in this block.

The security payoff follows from the separation. The post states the rule plainly: an instruction reached through the execution block is authoritative, while anything reached as input - data, documents, tool schemas, tool results - is material to read, not a command to follow. Components the execution block does not call on are ignored. The takeaway for readers is a design pattern: stop flattening prompts into strings and start giving the model typed slots.

The previews: what you can run today

Two models shipped with the post, both on Hugging Face and both verified live:

  • granite-4.1-3b-llmon-preview
  • granite-4.1-8b-llmon-preview

Both derive from the Granite 4.1 BASE checkpoints - not the released instruct models - which means LLMON behavior is learned during post-training rather than layered onto an existing chat format. Mellea, IBM’s agent framework, is cited in the post as shipping native LLMON support, so the format is not orphaned in a blog.

Capability is not the bet; the injection behavior is. The authors report that across standard benchmark categories the LLMON models perform on par with the Granite 4.1 Instruct models, and state you can swap the previews in for Granite 4.1 3B or 8B and expect comparable general performance.

The injection numbers - and how to read them

On the OPI benchmark, which places an indirect injection in the data of classic NLP tasks, robustness is defined as 1 minus attack success rate. The LLMON previews avoid 47% more attacks than the best frontier model the authors tested and 95% more than the best open-source model in that test; the authors report similar trends on ASB and StruQ.

Now the traps, all of them from the post itself:

  • Every number is self-reported on the authors’ own harness. No independent replication was found in this run. Treat the tables as a vendor claim, exactly like you would treat any model-card number.
  • The benchmarks test static indirect injection. The post explicitly says adaptive attacks that unfold over many chat turns are not yet addressed. Do not read “attack success rates drop to nearly zero” as “this model is injection-proof.”
  • The privilege rule is learned, not enforced. The post says so itself: “This is a learned behavior, not an enforced guarantee.” A learned rule is defeatable by a training-distribution shift, which is the honest ceiling on the whole approach.
  • These are experimental previews. The post labels them “ready for exploration and evaluation but not for production,” and they are 3B and 8B - on par with Granite 4.1 Instruct, not with frontier models.

Why the training recipe is the interesting part

The recipe, as described in the post, is closer to a data-design argument than a model-design one: a conventional instruction-tuning corpus was re-serialized into LLMON - each prompt component became its own addressable block, each turn carries an execution block - and the corpus was deliberately polluted with decoy instruction and data blocks that are present in context but left unbound, so the model has to learn to ignore structure it is not pointed at. The model then went through five sequential GRPO stages (multi-domain reasoning, RLHF, knowledge calibration, and math) plus a final DPO stage on a thousand on-policy preference pairs built from the model’s own responses to injection attempts.

That is the real contribution to watch: the separation of instruction and data is not enforced by the architecture or the tokenizer. It is a behavior the tokenizer merely makes cheap to learn - two tokens per reference instead of a paragraph of stringly-typed convention.

There is also an interpretability angle the post backs up with a visualization: a Hugging Face Space renders the attention of the LLMON previews, showing the model attending to the execution block when choosing which data block to read from.

What this means for your prompts

If the approach holds up under third-party replication, the practical move is to stop building stringly-typed prompts. Give each piece of your prompt a type and an ID, bind them explicitly, and let the execution block say which ones are commands and which are payload. You get three things: a model that cannot confuse data for instructions by construction of the corpus it trained on, a fixed two-token cost for every reference you make, and named blocks you can log, diff, and reason about.

The honest caveat rides along with all of it: the frontier is still defined by the authors’ own harness, the previews are 3B/8B and not for production, and a learned privilege rule is a learned privilege rule. The post is a strong, well-supported data point that structure belongs in the tokenizer - not a replacement for guardrails on anything you run in anger.