KV Cache update strategy on environments with kinematic constraints

[chuanye-Wang]

Hi LingBot-VA Team,
Thanks for your great work!

I am evaluating lingbot-va on a new simulator that introduces strict kinematic constraints and physical boundaries. Unlike RoboTwin, the predicted action in this environment might not always be executed perfectly ($a_{t, \text{actual}} \neq a_{t, \text{predicted}}$).

My question is: When updating the KV Cache for the next step, which action should we append to the history?

1. Predicted Action ($a_{t, \text{predicted}}$): Maintains autoregressive consistency, but may cause misalignment with the next visual observation if the action wasn't fully executed.
2. Actual Action ($a_{t, \text{actual}}$): Aligns with the physical world state, but might introduce a distribution shift for the model.

Which approach do you recommend or use internally for physically constrained environments?

Thanks!