INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models

Train a world model to answer the control query it will receive at deployment.

End-to-end JEPA world modeling for goal-conditioned robot control without broad action search.

Junhan Sun^1,4  ·  Hao Zhao^2,4,†  ·  Guofeng Zhang^1,3,†

¹State Key Laboratory of CAD&CG, Zhejiang University
²Institute for AI Industry Research (AIR), Tsinghua University
³InSpatio  ·  ⁴RoboParty Lab
^†Corresponding authors

Why INTACT?  ·  Method Notes  ·  Audited Results  ·  Reproducibility  ·  中文

Project Film

Watch the 61.9-second project film in 1080p  ·  Download the share edition with project QR  ·  Open the interactive project page

A strong representation keeps the information that matters intact.
INTACT does exactly that, turning LeWM into a stronger world model.

Why INTACT?

We introduce INTACT (INtent-To-ACTion), an end-to-end JEPA that turns action-labeled, reward-free trajectories into a deployable intent-to-action interface. The name captures both the structure we impose and the information we preserve:

• Isomorphic between predictor graphs. Local and goal motion-intent calls use the same four-slot input grammar and the same parameters.
• Isomorphic between supported families. Local and goal intent families correspond through the action-law semantics induced by that shared predictor, not through pointwise latent equality.
• Intact from RGB evidence to latent intent. End-to-end action gradients retain action-effective visual information while suppressing nuisance that is unrelated to motion intent.
• Intact from intent families to action-law families. The shared predictor preserves the supported family correspondence all the way to direct action readout.

TL;DR

Forward world models answer "what will happen if I execute this action?" Yet deployment asks the inverse question: "which action realizes this intent?" CEM and MPPI answer it by numerically searching over many candidates, leaving training and inference without a learned semantic correspondence. The result is often a predictor plus an action searcher, rather than a self-consistent intent-action model.

INTACT learns that missing correspondence end to end. One conditional operator maps both observed physical change and deployable goal intent to an action law. Its conditional mean is a zero-search controller; sampling is retained only for diversity or optional local verification. No frozen encoder, extra policy-training stage, or globally linear latent dynamics is required.

1 epoch training  ·  95.33% Direct macro  ·  0 search  ·  2.9-5.5 ms latency

89.39% Shared E5 Direct  ·  96.86% Guarded  ·  23.44x fewer candidates

Core Insight: One Input Grammar, Two Intent Instances

INTACT has one predictor input form. For either intent instance $m_t$, it uses

$$ x_t(m_t)=\big[z_t,m_t,z_t\odot m_t,A(a_{t-1})\big], \qquad G_\eta\left(x_t(m_t)\right) =p_\eta(a_t\mid x_t(m_t)). $$

Only the value and gradient role of $m_t$ change:

$$ m_t^{\mathrm{local}}=z_{t+1}-z_t, \qquad m_t^{\mathrm{goal}}=\mathrm{sg}(z_g)-z_t. $$

The local instance uses a realized successor to ground which physical change produced the demonstrated action. The goal instance presents the same operator with the intent available before acting. Both come from the same demonstration and are supervised against the same correct $a_t$, but each supervised conditional remains one triplet $(z_t,m_t,a_t)$ with one endpoint and one proper NLL:

$$ \mathcal L_{\mathrm{I2A}} =\lambda_{\mathrm{local}}[-\log p_\eta(a_t\mid x_t(m_t^{\mathrm{local}}))] +\lambda_{\mathrm{goal}}[-\log p_\eta(a_t\mid x_t(m_t^{\mathrm{goal}}))]. $$

There is no direct loss between the two endpoints or their latent displacements. Instead, the shared likelihood creates a conditional action quotient: at a fixed state, intents are equivalent when they induce the same expert action law. Under a task-appropriate tolerance, nearby predictions $\hat a_t^{(1)}$, $\hat a_t^{(2)}$, and the demonstrated $a_t$ can therefore belong to the same action-equivalence neighborhood. This distributional view makes direct control less sensitive to small prediction errors and helps limit closed-loop drift, while the forward JEPA keeps the richer world information needed for prediction.

Method

The physical successor remains attached to ground reachability; the future goal is a stop-gradient deployment anchor. INTACT aligns the two supported condition families through the actions they induce, without matching their endpoints, imposing globally linear latent dynamics, freezing the encoder, or adding a phase-2 controller. The goal likelihood alone is a goal-conditioned imitation objective; full INTACT is its shared, end-to-end coupling with the attached physical likelihood and forward JEPA.

INTACT retains the forward JEPA and adds one action-law predictor with a matched input grammar for physical and deployable intents. The key construction is not two unrelated auxiliary losses: both calls share parameters and a proper action likelihood, while their upstream gradient routes remain deliberately asymmetric.

Component	Role
Forward predictor	Preserves latent dynamics, contacts, topology, and visual information needed for rollout.
Physical intent call	Uses the observed successor with attached gradients to preserve action-recoverable change.
Goal intent call	Uses a detached future goal to train the same interface on a condition available before acting.
Matched interaction	Pairs first-order intent $m_t$ with the state-intent feature $z_t\odot m_t$.
Direct controller	Emits an action chunk with no candidate search or terminal latent-cost call.
Guarded local verification	Optionally refines the coherent Direct plan with a small local CEM budget.

The four-domain model shares one visual encoder and keeps lightweight, task-specific forward/action heads:

See Method Notes for the statistical construction, gradient contract, and the distinction from inverse dynamics, goal-conditioned behavior cloning, and post-hoc sampling priors.

Results

Task-Specific, One Epoch

The matched task-specific setting trains three models per task. Each checkpoint is evaluated with three seeds and 100 episodes per seed on the official LeWM protocol.

One epoch of goal-displacement INTACT reaches 95.33 +/- 0.58% Direct macro SR with no candidate search. The same checkpoints reach 96.86 +/- 0.38% with 384-sequence local verification. Published LeWM numbers use its separate 10-epoch CEM protocol and serve only as landscape context, not paired significance controls. Exact task values and the full matched inference matrix remain available in Audited Results.

One Shared Encoder

At epoch 5, Goal-displacement INTACT reaches 89.39 +/- 0.77% Direct macro SR with one encoder shared across all four visual domains. The matched shared LeWM baseline reaches 66.17 +/- 2.67% with CEM 300x30. With all INTACT action heads disabled, pure-CEM macro still rises to 70.08 +/- 1.13%, separating representation shaping from direct action readout.

Theory Meets Measurement

At fixed state $z$, INTACT treats two endpoint conditions as equivalent when they induce the same expert action law:

$$ y\sim_z y' \iff p_E^\star(a\mid z,y)=p_E^\star(a\mid z,y'). $$

This conditional action quotient predicts that control should track the relation between predicted and expert action-law families, not task clustering or latent rank alone. Across 45 eligible shared-encoder checkpoints, predicted-expert kNN overlap correlates with Direct SR at r = 0.954 and linear CKA at r = 0.897; pointwise action $R^2$ is weaker at r = 0.815.

Paper, Website, and Code

This repository is currently a public research preview. The method record, audited result tables, attribution, and reproducibility contract are organized here first. Training/evaluation source, anonymous submission artifacts, and checkpoints will be released in stages after their licensing, provenance, and double-blind review checks are complete.

Artifact	Status
Method and result documentation	Available in this repository
Reproducibility contract	Available in docs/REPRODUCIBILITY.md
Training and evaluation code	Release qualification in progress
Configurations and checkpoint manifests	Release qualification in progress
Model checkpoints	Planned
Paper	arXiv preprint 2607.26056 (2026-07-28)
Project website	zju3dv.github.io/INTACT-JEPA

The detailed release gates are tracked in Release Plan.

Citation

An arXiv preprint is available at arXiv:2607.26056. Please prefer the machine-readable CITATION.cff record.

@misc{sun2026intact,
  title         = {INTACT: Isomorphic Intent-to-Action Learning for Search-Free World Models},
  author        = {Sun, Junhan and Zhao, Hao and Zhang, Guofeng},
  year          = {2026},
  eprint        = {2607.26056},
  archivePrefix = {arXiv},
  url           = {https://arxiv.org/abs/2607.26056}
}

Acknowledgements

INTACT builds on the LeWM and stable-worldmodel research ecosystem. Evaluation is reported on the official LeWM protocol and, where explicitly marked, the separate CLEAR-LeWM evaluator. See NOTICE.md for provenance and licensing boundaries.

Questions: luoliibaqi4747@gmail.com