Nokia AnyJev — A Training-Free Decision Layer That Cuts Classifier Flip-Rate From 23% to 7.3% With Zero Labels

Nokia's Sunnyvale applied research team open-sourced AnyJev on Sept 23 under Apache-2.0: a training-free layer that turns any open LLM into a calibrated classifier. The raw argmax flip-rate on option reorder drops from 23.0% to 7.3%. A closed-form linear head at 2/3 model depth ties a fine-tuned 421M classifier with 100KB of weights.

September 25, 2026 – Nokia’s Sunnyvale applied research team shipped AnyJev on September 23 under Apache-2.0, on PyPI, with transformers and vLLM backends. The package is a training-free decision layer that turns any open LLM into a calibrated classifier. The lede is one number: reading raw argmax over option tokens flips its answer 23.0% of the time when the option order is reversed. AnyJev L0 drops that to 7.3% with zero labels, no fine-tuning, and no weight changes to the underlying model.

The 23.0% figure is the position-bias artifact that any production LLM-classifier user has hit. The model is reading the order of the options, not the content of the prompt. Reversing the option list flips a quarter of the answers. AnyJev L0 fixes that with cyclic-shift marginalisation and batch prior correction – two operations that take milliseconds and need no labels. The fix is not a new model. It is a better way to read the logits the model already produces.

What the numbers look like on BANKING77

AnyJev’s repo publishes a full ablation across Qwen3 (1.7B / 4B / 8B / 30B-A3B / 32B), OLMo, Granite, Phi, and Mistral. The BANKING77 20-way intent-classification benchmark is the cleanest single-dataset view. On Qwen3-8B:

Method Auto-decidable at <=5% error ECE (calibration error) Labels needed
Raw argmax 7.7% 0.240 0
AnyJev L0 (cyclic-shift marginalisation) 46.3% 0.184 0
AnyJev L1 (L0 + temperature scaling) 52.0% 0.095 100-500

The 7.7% auto-decidable floor on raw argmax is the production failure mode. Eight percent of decisions can be made without a human in the loop. Adding L0 with zero labels takes that to 46.3% – a 6x jump in auto-decidable traffic with no training data and no model changes. Adding temperature scaling on top with 100-500 labels takes it to 52.0% and cuts the calibration error from 0.240 to 0.095. Calibration error is the metric that matters when you are routing decisions based on confidence thresholds – a 60% reduction is the difference between “act on the model’s call” and “always escalate to a human.”

The flip-rate story generalises across model sizes. The 23.0% raw / 7.3% AnyJev L0 contrast is reported on the full ablation table, not just Qwen3-8B. OLMo and Granite show the same shape; Phi and Mistral show slightly different baselines but the same L0 fix lands in the 6-9% range.

L2 is the contrarian punchline

L0 is the headline fix. L2 is the part of the paper that nobody is going to believe until they read it. L2 is a closed-form linear head – no gradients, no weight changes to the base model – fitted on the hidden state at a fixed block about two-thirds of the way through the model. The head is solved in seconds on CPU, weights around 100KB per head, and is fitted on 100-300 labels.

On the Jev evaluation harness, a 1.7B model with the L2 head at 64% depth hits 0.730 accuracy. A 4B model with L2 at 67% depth ties a fine-tuned 421M Laya classifier at 0.786. The 421M Laya is the production classifier AnyJev is being benchmarked against. A 4B open model with 100KB of additional weights ties a fine-tuned classifier ten times its parameter count.

The architectural read is that decision-relevant information is concentrated in the middle layers of an autoregressive transformer, and a single linear projection is enough to read it. The standard “fine-tune the whole model on labels” pattern that the industry has run for three years turns out to be doing the same job as 100KB of weights fitted in closed form on top of the hidden state. The cost difference is two orders of magnitude on labels and several orders of magnitude on compute.

Apache-2.0, PyPI, two backends

AnyJev ships as pip install "anyjev[hf]" for the transformers backend, with a vLLM backend that uses prefix caching. On the vLLM backend, L0’s K prefetches per decision are batch-friendly – roughly 0.25 seconds per decision at batch 32 with K=20 options on a single H100. For a 20-way classifier with prefix caching warmed up, the per-decision cost is in the same band as a single forward pass on a 7B model. The Apache-2.0 license means it can ship inside a proprietary product without source disclosure. There is no SaaS, no API, no telemetry requirement.

The supported model families – Qwen3 (1.7B / 4B / 8B / 30B-A3B / 32B), OLMo, Granite, Phi, Mistral – cover the bulk of the open-weight landscape. AnyJev is not tied to a single model family. The decision layer is model-agnostic; the only requirement is that the model expose its hidden states at a chosen block.

Nokia isn’t a typical AI vendor

The authors are Jiamu Zhang, Tianze Yang, and Liang Wu at Nokia Sunnyvale applied research, plus Yucheng Shi at Tencent Hunyuan. The contrarian read on this release is that it tells you what “Nokia applied research shipping Apache-2.0 infra” looks like. Nokia does not sell an AI platform. Nokia sells network infrastructure. Shipping an open-source, training-free classifier layer under Apache-2.0 with a paper that benchmarks against every open-weight family is not the move a vendor with a competing product makes.

The licensing and vendor-neutrality angle matters for production adoption. AnyJev can be deployed inside a regulated environment without a vendor relationship. There is no rate limit, no API key, no telemetry endpoint. For procurement teams that have been burned by lock-in, the “Nokia-applied-research + Apache-2.0” provenance is the part that gets this past the security review. The technical merit is what gets the first deployment.

Jev is the proprietary upstream

AnyJev is a training-free open port of Jev, the proprietary System One model from TypeSafe AI (announced 2026-09-19). Jev ships a typed-decision interface – choice, noul, score – on top of a fine-tuned base model. AnyJev borrows the interface and ships the same accuracy band with no fine-tuning. The framing is “any causal LM can be a System One model if you read its logits correctly.” TypeSafe’s commercial play is the fine-tuned model; Nokia’s open play is the training-free layer. They are competing on the “wrap any causal LM” pattern – one monetises the fine-tuning, the other monetises the layer.

The architectural implication is that the decision layer is becoming a standard component. The model is a commodity. The wrapper is the value. AnyJev joins a category that already includes tools like vLLM, Outlines, and Guidance – each of which sits on top of an open model and adds a capability the model alone does not provide. The category is large enough now that “decision layer” is a recognisable product surface.

What this changes for production

For teams running an LLM classifier today, the immediate move is to instrument the option-order flip rate. Run a sample of decisions with the option list reversed and measure how often the answer changes. If the flip rate is in the 15-25% range, you are paying the position-bias tax on every decision and you have a 6x auto-decidable uplift available with zero labels. L0 is a drop-in replacement for the argmax read-out – it does not require retraining the underlying model.

For teams that have already fine-tuned a small classifier head, the L2 result is the harder question. A 4B open model with 100KB of weights ties a fine-tuned 421M classifier. The fine-tuned classifier wins on latency and per-decision cost – the fine-tuned model is smaller and faster than a 4B base. The 4B-with-L2 wins on coverage: it can do any classification task with 100-300 labels, not just the task it was fine-tuned on. The trade-off is “specialised and cheap” versus “general and a bit slower.” Most production classifiers are specialised enough that the fine-tuned path still wins on cost.

For open-weight adoption broadly, AnyJev is another data point that the model layer is becoming commoditised. The decision layer – like the routing layer (see TopClanker’s August 18 piece on the Stripe-OpenRouter acquisition) – is where the value is moving. A 4B model with AnyJev L2 ties a fine-tuned classifier; the layer above the model is what gets you production-grade decisions.

The practical takeaway: instrument the flip rate, run L0 against your production traffic for a week, and budget the calibration-error reduction against your human-in-the-loop costs. If the numbers hold on your workload, L1 is a 100-500 label spend that pays for itself the first time your confidence-threshold routing works correctly.

Sources