neural / flexibility-trap
No slider position satisfies both problems.
That's the flexibility trap.
Diffusion language models can generate tokens in any order. Theoretically, this makes them more powerful than autoregressive models. In practice, the flexibility creates a trap: parallel generation excels at constraint satisfaction but fails at sequential reasoning. The model must choose which intelligence to deploy — and no single choice works for everything.
interactive — the flexibility trap
drag the ARness slider from parallel (0) to sequential (1) · watch two problem types respond inversely · the combined score never exceeds ~67% · try the oscillate button to see the trap in motion
Dream 7B Sudoku
81%
vs 21% AR
Diffusion dominates constraint satisfaction. Global constraint checking in parallel.
JustGRPO
89.1%
GSM8K
Reasoning recovered by intentionally abandoning arbitrary order.
ARness
high
empirical
dLLMs trained on CoT collapse toward left-to-right generation despite parallel architecture.
Max combined
~67%
theoretical ceiling
No single ARness value maximizes both constraint and reasoning performance.
The trap isn't a bug — it's a reflection of two fundamentally different kinds of cognitive work. Understanding the difference is the first step toward architectures that can navigate between them.
Two kinds of intelligence
Constraint satisfaction — Sudoku, scheduling, logistics — requires seeing the whole board at once. Every cell constrains every other cell simultaneously. Sequential processing can't check constraints it hasn't reached yet. Chain-of-thought reasoning — math proofs, logical deduction — requires each step to build on the previous. The result of step 1 IS the input of step 2. Parallel processing can't use results that don't exist yet.
The ARness collapse
Current dLLMs are trained on text that is inherently sequential — human writing flows left to right. Chain-of-Thought supervision reinforces this. Measured ARness across LLaDA and Dream remains stubbornly high. The models learn parallel architecture but fall back to sequential behavior because that's what the training data demands. The flexibility is theoretical, not practical.
The escape: don't choose
JustGRPO abandons arbitrary order for reasoning — standard sequential GRPO, 89.1% on GSM8K. TiDAR uses hybrid attention — causal prefix for sequential reasoning, diffusion blocks for parallel drafting, in a single forward pass. Block Diffusion generates blocks autoregressively, tokens within blocks via diffusion. The answer isn't one strategy — it's knowing when to use which.
The Flexibility Trap
Ni et al. · Jan 2026finding
Arbitrary generation order narrows rather than expands reasoning boundaries. dLLMs exploit flexibility to bypass high-uncertainty tokens, causing premature solution space collapse.
result
JustGRPO: 89.1% on GSM8K by intentionally forgoing arbitrary order.
Why Diffusion Language Models Struggle with Truly Parallel Decoding
Multiple labs · Feb 2026finding
Across LLaDA and Dream, ARness remains high — models collapse into quasi-left-to-right patterns. Training on CoT data further increases sequential behavior.
result
NAP (Non-Autoregressive Parallel) improves genuine parallel performance through data curation.
Dream 7B: Diffusion Large Language Models
HKU NLP + Huawei Noah's Ark · 2025finding
Diffusion models fundamentally outperform AR on constraint satisfaction: Sudoku 81% vs 21%, Countdown 16.0 vs 6.2, Trip planning 17.8 vs 3.6.
result
First diffusion LLM to consistently surpass AR baselines on planning tasks at scale.
The flexibility trap teaches us something fundamental: intelligence isn't one thing. Some problems need you to see everything at once. Others need you to think step by step. The best architectures — TiDAR, Block Diffusion, JustGRPO — don't pick one mode. They learn when to switch.
This is the real frontier of diffusion language models. Not just "think in parallel" or "think sequentially" but knowing which problems demand which mode — and transitioning between them within a single forward pass. The models that solve the flexibility trap won't just be faster. They'll be genuinely smarter.