TPU multi-worker(pod) training

[DimensionSTP]

Hi, I’m currently using a TPU v4-64 pod, and I encountered an issue when trying to run multi-worker training llama with the example provided for TPU v5-8. Each worker seems to run independently instead of syncing properly during training. Could you provide an example specifically for TPU pod multi-worker (e.g., TPU v4-64) training where the entire pod is used as a single unit?