Questions about hyperparameter settings for reproducing Octo

[MasterXiong]

Dear authors,

Thanks for open-sourcing this great VLA project! I'm trying to reproduce the Octo pre-training code and have some questions about the hyperparameter settings that hope you can help with. Thanks!

1. How does the batch size influence model performance based on your experiments before? As I need to reduce the training batch size from 2048 as reported in the Octo paper to 128 due to GPU memory limits. And the results I got seem to be much worse than the pre-trained model you uploaded by evaluating on the SIMPLER benchmark.
2. I notice that you mentioned in the report that you use a weight decay coefficient of 0.1 during training. This value seems to be very large compared to commonly used L2 regularization term. I was wondering that why do you choose such a large value? Does it work better than using a smaller value?

Thanks for your help!