Question about memory dimension of Robodojo

[tuanziqiqi]

Hello, I would like to inquire as to why EventVLA seems to have made little improvement in the memory dimension of Robodojo, compared to some of the previous models without memory capabilities.

[JLUbaijh2323]

I have the same question. I'm also curious about what keyframe GT is used to train EventVLA on RoboDojo. Could the difference in performance be caused by different keyframe selection strategies?

[zhangzhengtu]

Thank you for raising this question.

For RoboDojo, our understanding is that most tasks do not rely on dense manually annotated keyframe labels. In many cases, the memory-relevant information can be sufficiently provided by visual anchors, while only a small subset of tasks involve explicit keyframe annotations.

Regarding the relatively limited improvement of EventVLA on the memory dimension, our current understanding is that this may not be mainly caused by the keyframe selection strategy itself. RoboDojo requires strong low-level manipulation and stable execution ability, while EventVLA has not been pretrained on large-scale real-world robot data or simulated trajectory data. Therefore, although the memory mechanism can provide useful historical visual evidence, the final performance may still be constrained by the model’s basic manipulation capability, which limits the overall gain on the memory dimension.

In addition, we believe that the memory used in DM05 can also be understood as a form of keyframe-based memory, although the details of how their keyframes are selected are not fully clear to us. From this perspective, we still consider keyframe-based memory to be an efficient and reasonable design. Different keyframe selection strategies may indeed affect the final performance, and this is also an important direction for further analysis and comparison.

Thanks again for your attention and discussion.