Standard LLMs predict the next token. Agent models built on top decide the next action. Last week Qwen published something that does neither of those things as the primary objective — it predicts what the environment returns after an action.
They call it a Language World Model (LWM). The newly released Qwen-AgentWorld-35B is trained to answer a specific question: "If the agent runs this shell command, what does the terminal print back?"
What a Language World Model actually is
The core shift is in the prediction target. A regular agent model looks like:
state → action
A world model adds a second loop:
(context, action) → next_observation
Formally: ô_ = f_θ(c, o_≤t, a_≤t) — given the full trajectory so far and the action taken, predict the observation that comes back from the environment.
For a coding agent, "observation" is the terminal output. For a web agent, it's the page state after a click. For an API agent, it's the JSON response body.
This solves two distinct problems:
Simulated training environments. Real-world agent RL is expensive — you need sandboxed VMs, disposable databases, or live browser sessions. With a world model acting as a proxy environment, you run RL entirely in-model:
for step in trajectory:
action = agent.decide(state)
# LWM replaces the real environment
observation = lwm.predict(context, action)
reward = verifier(observation)
agent.update(reward)
Qwen's results show agents trained purely on simulated trajectories outperforming those trained only on real environments. That's a significant result.
Planning before committing. An agent can mentally simulate a few steps ahead before picking the action with the best predicted outcome. Think of it as reflection oriented toward the future rather than the past.
The training pipeline
Three stages, and the middle one is worth stealing:
CPT (Continual Pre-Training): 10M+ environment trajectories. Turns are weighted by novelty — if an observation just echoes the action back, it gets 5% loss weight. Only genuinely informative state transitions count fully. This is a cleaner signal than treating all turns equally.
SFT (Supervised Fine-Tuning): Activates explicit next-state prediction. Qwen ran 12 parallel prompt optimisation passes they call "AutoResearch" — basically automated prompt engineering that analyses failure modes and generates diverse template variants. Worth stealing this idea for your own evals.
RL (Reinforcement Learning): Hybrid reward: 90% LLM judge scoring across Format/Factuality/Consistency/Realism/Quality, plus 10% rule-based binary verifier. The binary anchor stops the LLM judge from drifting into "everything is fine" territory — a real problem with pure LLM-as-judge setups.
One gotcha they found: multi-turn RL causes reward collapse. Fix is restricting each trajectory to exactly one prediction target per RL update. Narrow fix, but worth knowing if you try to replicate this.
Numbers worth noting
The 35B-A3B model (35B total parameters, 3B active via MoE routing) sits at 56.39 on AgentWorldBench — edging out Claude Sonnet 4.6 at 56.04. The larger 397B-A17B scores 58.71, nudging past GPT-5.4's 58.25.
These are world modelling scores — how accurately the model predicts what a real environment returns, evaluated against 2,170 samples from real frontier-model interactions. That's different from agent task completion, but accurate environment prediction is a prerequisite for good planning, so the correlation should be there.
The model covers seven domains in a single set of weights: MCP, Search, Terminal, SWE, Android, Web, OS. That generalisation is probably the most interesting result — one model learning to simulate all of them rather than needing domain-specific variants.
What I'd build with this
Agent testing harness without real infra. If you're building a coding agent that touches files and runs shell commands, you currently need Docker containers or disposable VMs for safe RL. Swap those out for an LWM as the environment proxy. Cuts infrastructure costs and lets you inject failure modes that are hard to reproduce reliably in real environments — network timeouts, partial writes, permission errors.
Pre-action confidence scoring. Before an agent executes something destructive (deleting a file, sending an email, calling a payment API), run the planned action through an LWM to predict the outcome. If the predicted state diverges significantly from expected, flag it for human review. Cheap safety layer that doesn't require running the action for real.
Synthetic trajectory generation for fine-tuning. Collecting real agent trajectories is slow and expensive. Use an LWM to generate plausible environment responses for thousands of (action, observation) pairs, then use those synthetic trajectories to fine-tune a smaller, faster downstream agent. Same logic as using a big model to generate training data for a small model — just applied to environment simulation.
The "Language World Model" framing is going to matter. The idea that an agent should model its environment — not just its next action — makes intuitive sense, and the open weight release means the research community can actually build on it. I'd expect this architecture to show up in agent frameworks within the next few months.
Weights on HuggingFace: Qwen/Qwen-AgentWorld-35B-A3B. Paper: arxiv.org/abs/2606.24597.