zero residual target

[Kguo-cs]

In the code, the state is processed by the rollout_k_disks, so the residual target will be always 0. Why do you predict it using a head?

[jiangchaokang]

Thanks for the question — this is a good point.
I sincerely apologize for the delayed response.

The residual would indeed be zero if it were recomputed from the post-rollout_k_disks states, because those states are generated by applying the selected K-disks anchor. However, that is not the residual target used by the refinement head.

In rollout_k_disks, we keep two different notions of state:

1. states[:, t]: the anchor-rolled processed state used as the current rollout state.
2. agent_states[:, t + 1]: the original/raw GT next state.

At each step, we compute the local transition from the processed current state to the raw GT next state, select the closest K-disks anchor, and store

residual_target = local_transition(processed_state_t -> raw_gt_state_{t+1}) - anchor_delta

with yaw normalized. Only after this residual is stored do we update states[:, t+1] using the anchor alone.

So if one recomputes

local_transition(processed_state_t -> processed_state_{t+1}) - anchor_delta

then yes, that value is zero by construction. But the supervised target for the refinement head is the precomputed residual_targets, which are computed against the raw GT next state before overwriting the next processed state.

This is intentional: the discrete K-disks token provides a coarse anchor action, while the residual head predicts a bounded continuous local SE(2) correction. This lets DeltaSim keep a stable discrete action space while recovering sub-token motion and reducing quantization / rollout drift.

In the current code, the loss uses data["agent"].residual_targets from preprocessing rather than recomputing residuals from the processed states. If you observe all-zero residuals, it is likely due to stale DeltaSim preprocessing files generated with an older contract. Please delete the DeltaSim preprocess directory and rerun preprocessing so that residual_targets, transition_valid, and the vocabulary hash are regenerated consistently.