[SONIC deployment] Live PSI0 policy output remains nearly stationary

[waiyc]

System setup
Robot: Unitree G1 EDU
Hands: BrainCo three-finger hands
Policy: Fine-tuned PSI0
Checkpoint step: 15,000

Description

I collected approximately 50 episodes of an upper-body snack-picking task while the robot was operating in SONIC full-body-control mode.
Although the demonstrated task mainly involved the arms and BrainCo hands, the recorded 64D SONIC latent actions also included whole-body stabilization and lower-body control.

Then i deployed this fine-tuned PSI0 (tested 5k, 10k ,15k) checkpoint on a Unitree G1 with BrainCo three-finger hands.
However, after activating the robot, it remains standing and does not respond meaningfully to changes in the scene.

Could this cause the dataset to be dominated by standing and balance-related tokens, leading the fine-tuned policy to repeatedly predict a nearly stationary latent action?

---

[ZMQEndpointInterface] *** Starting ZMQ processing ***
[ZMQEndpointInterface] Protocol version: 4
[ZMQEndpointInterface] Protocol v4: Received 64D token (latent action), tokens[0]=-0.0000
[ZMQEndpointInterface] Protocol v4: Left hand joints set: [0.0078, -0.0469, -0.0469, 0.0000, 0.0156, 0.0098, 0.0039]
[ZMQEndpointInterface] Protocol v4: Right hand joints set: [0.0039, -0.0430, -0.0430, 0.0000, 0.0039, -0.0039, 0.0059]
[Token Flow] Copied external tokens, first 8=[-0.0000, 0.1875, 0.0625, -0.0625, 0.0625, -0.1875, 0.3750, 0.1250]
Loop timing - LowState age: 0.6960ms, Streaming data mean delay: 0.0000ms, Streaming data std delay: 0.0000ms, IMU age: 0.6350ms, Obs: 86us, Policy: 334us, Obs 2 Motor Command: 420us, Post processing: 15us | HandCloseRatio: 1.0000
[ExternalTokenDebug] has=1 size=64 expected=64 initial_encoder_mode=0 using_encoder=0

═══════════════════════════════════════════════════════════════════════
DEPLOYMENT CONFIGURATION
═══════════════════════════════════════════════════════════════════════

Environment: real
Network Interface: enp130s0
Decoder Model: policy/release/model_decoder.onnx
Encoder Model: policy/release/model_encoder.onnx
Motion Data: reference/example/
Obs Config: policy/release/observation_config.yaml
Planner: planner/target_vel/V2/planner_sonic.onnx
Input Type: zmq
Output Type: all
ZMQ Host: localhost
Default Motion: neutral_kick_R_001__A543

═══════════════════════════════════════════════════════════════════════

[Yewandou7]

Hi @waiyc,

For reference, we have fine-tuned PSI0 on ~140 episodes across two tasks and deployed it successfully, so the overall pipeline should work; our wandb curves are attached below.

From your log, it looks like the SONIC external token mode is enabled correctly, but please double-check it by running a replay script to confirm the whole-body control tracks well.

To your question: even though the 64D latent encodes whole-body actions, it can still capture the upper-body motion well. However, if there are too many stationary segments in your dataset, the model can indeed be biased toward predicting stationary tokens.

On the model/data side, a few things to check:

1. Make sure your dataset does not contain too many stationary segments. Depending on the task complexity, you may also need to record more episodes.
2. Run an open-loop inference test: feed the observations of one recorded episode to your fine-tuned model and check whether its predicted actions match the recorded ones.
3. Check whether the state assembly at deployment is identical to training (joint ordering, offsets, and how the hand joints are appended).

Thanks,
Weiduo

[lebron123zc]

Hi @waiyc,

For reference, we have fine-tuned PSI0 on ~140 episodes across two tasks and deployed it successfully, so the overall pipeline should work; our wandb curves are attached below.

From your log, it looks like the SONIC external token mode is enabled correctly, but please double-check it by running a replay script to confirm the whole-body control tracks well.

To your question: even though the 64D latent encodes whole-body actions, it can still capture the upper-body motion well. However, if there are too many stationary segments in your dataset, the model can indeed be biased toward predicting stationary tokens.

On the model/data side, a few things to check:

1. Make sure your dataset does not contain too many stationary segments. Depending on the task complexity, you may also need to record more episodes.
2. Run an open-loop inference test: feed the observations of one recorded episode to your fine-tuned model and check whether its predicted actions match the recorded ones.
3. Check whether the state assembly at deployment is identical to training (joint ordering, offsets, and how the hand joints are appended).

Thanks, Weiduo

For reference, we have fine-tuned PSI0 on ~140 episodes across two tasks and deployed it successfully, so the overall pipeline should work; our wandb curves are attached below.请问这是使用sonic采集的140条数据吗？

[waiyc]

I would like to confirm the recommended PSI0 SONIC fine-tuning configuration for my laptop:

• Single NVIDIA RTX 5090 Laptop GPU with 24 GB VRAM
• 33 GB system RAM
• 24-core CPU

My current fine-tuning configuration is:

export CUDA_VISIBLE_DEVICES=0
export OMP_NUM_THREADS=8

--train.mixed_precision=bf16
--train.train_batch_size=16
--train.gradient_accumulation_steps=1
--train.learning_rate=1e-4
--train.max_training_steps=40000
--train.warmup_steps=1000
--train.checkpointing_steps=5000
--train.max_checkpoints_to_keep=10
# ...
--model.no-tune-vlm
--model.action-chunk-size=30
--model.action-exec-horizon=30
--model.action-dim=78
--model.odim=43

Default configuration

My understanding is that the default fine-tuning script uses a per-GPU batch size of 16 with one gradient-accumulation step across eight GPUs, resulting in an effective global batch size of 128. Please correct me if this interpretation is incorrect.

Issue 1: Preserving global batch size on one GPU

To preserve the global batch size of 128 on my single RTX 5090, I tried:

--train.train_batch_size=16
--train.gradient_accumulation_steps=8

However, SonicTrainer failed with:

AttributeError: 'SonicTrainer' object has no attribute '_grad_norm_act'

Issue 2: High system-RAM usage

A five-step smoke test with batch size 16 and accumulation 1 completed successfully, including forward/backward passes, optimizer updates and checkpoint saving.

However, larger microbatches caused high system-RAM usage and swapping. With batch size 64, memory usage reached approximately 29.5 GB out of 33 GB.

Could you advise how the default PSI0 SONIC fine-tuning script should be configured for reliable single-GPU training on this laptop, particularly regarding:

• Microbatch size
• Gradient accumulation
• Effective global batch size
• DataLoader worker count
• OpenMP threads
• Checkpoint frequency and retention

I would also appreciate guidance on the correct fix for the _grad_norm_act error when gradient accumulation is greater than one.

[songlin]

@waiyc I should have the bug fixed. Please pull the latest code. Let me know if it still does not work.

Regarding the RAM requirements, try using a smaller batch size on a 5090. I would try a training batch size (device batch size) of 16 with 8 gradient accumulation steps. Also, try reducing num_workers (eg., 2) in the Trainer to avoid RAM OOM.

[waiyc]

RTC inference randomly stalls and G1 reports “robot data late”

I recollected a simple swipe away object on a table dataset and tested checkpoints at 5k, 10k and 15k steps.
All checkpoints can initially control the robot, but inference randomly stalls—sometimes halfway through the motion.

Before the stall, the robot receives protocol-v4 actions correctly:

[ZMQEndpointInterface] *** Starting ZMQ processing ***
[ZMQEndpointInterface] Protocol version: 4
[ZMQEndpointInterface] Protocol v4: Received 64D token (latent action), tokens[0]=-0.0625
[ZMQEndpointInterface] Protocol v4: Left hand joints set: [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
[ZMQEndpointInterface] Protocol v4: Right hand joints set: [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
[ZMQEndpointInterface] Received ZMQ message - topic: 'pose', protocol_version: 4, num_fields: 3, total_size: 3 buffers
[ZMQEndpointInterface] *** Starting ZMQ processing ***
[ZMQEndpointInterface] Protocol version: 4
[ZMQEndpointInterface] Protocol v4: Received 64D token (latent action), tokens[0]=-0.1250
[ZMQEndpointInterface] Protocol v4: Left hand joints set: [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
[ZMQEndpointInterface] Protocol v4: Right hand joints set: [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]

The RTC client initially receives actions:

[client] Received action, version=3592, shape=(1, 78)
[client] send interval: 0.016s
[client] send interval: 0.015s
[client] recv_action interval: 0.030s
[client] Received action, version=3593, shape=(1, 78)
[client] send interval: 0.015s
[client] send interval: 0.015s
[client] recv_action interval: 0.031s
[client] Received action, version=3594, shape=(1, 78)

At the stall,

the client continues sending observations approximately every 15 ms, but stops receiving new actions.
The robot then repeatedly uses the last received token. Around the same time, the G1 sometimes announces:

[client] send interval: 0.017s
[client] send interval: 0.015s
[client] send interval: 0.015s
[client] send interval: 0.015s

The robot controller may also print Reset to frame 0 :

[ZMQEndpointInterface] *** Starting ZMQ processing ***
[ZMQEndpointInterface] Protocol version: 4
[ZMQEndpointInterface] Protocol v4: Received 64D token (latent action), tokens[0]=-0.1250
[ZMQEndpointInterface] Protocol v4: Left hand joints set: [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
[ZMQEndpointInterface] Protocol v4: Right hand joints set: [0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000, 0.0000]
[Token Flow] Copied external tokens, tokens[0]=-0.1250
Loop timing - LowState age: 1.1790ms, Streaming data mean delay: 0.0000ms, Streaming data std delay: 0.0000ms, IMU age: 1.2090ms, Obs: 52us, Policy: 296us, Obs 2 Motor Command: 349us, Post processing: 24us | HandCloseRatio: 1.0000
[Token Flow] Copied external tokens, tokens[0]=-0.1250
______________________________________________________
Motion index: 5 : neutral_kick_R_001__A543 completed.
______________________________________________________
Reset to frame 0
Reset heading state to 0.9780, -0.0048, -0.0162, -0.2077
Reset delta heading to 0
Reset init reference data root rotation to current frame: 0.7034, -0.0191, 0.0131, 0.7104
**********************************************************
Reference motion name: neutral_kick_R_001__A543
**********************************************************

---

Observations and testing

• During testing, the G1 occasionally announces “robot data late.” I am unsure whether this warning is related to, or indirectly causing, the inference stall.
• The inference stall issue occurs at random points across multiple checkpoints (5k, 10k and 15k). This suggests it is more likely a runtime or communication issue than a checkpoint-quality issue.

Could you please advise:

1. I can only run the robot deployment successfully with --input-type zmq_manager; the default input type does not work in my setup. Is zmq_manager the correct input type for PSI0 RTC deployment?
2. Have you encountered the G1 warning “robot data late” during RTC deployment? Can this warning cause the controller to enter a hold or fallback state?
3. Could the mismatch between the client’s observation send rate and the server’s inference rate create an unbounded WebSocket queue and eventually cause the pipeline to stall?
4. Is there an officially recommended timeout, backpressure or reconnection mechanism for the SONIC RTC client?

[songlin]

Hi,

1. We just pushed a fix that changes --input-type from zmq_manager to zmq. Can you try again?
2. No, we don't. Can you check the network?
3. Yes, it's possible.
4. Currently, the SONIC RTC client does not support reconnection. The best way is to avoid network issues by using a good-quality router.

[lihuawei-topsun]

The new logs plus the current RTC client code point to a specific fail-open condition that is worth separating from Wi-Fi quality.

At commit f4ff131, _on_message() writes the newest action into the global buffer, but it does not store a receive timestamp or version with it:

Psi0/real/deploy/psi-inference_rtc.py

Lines 149 to 164 in f4ff131

	def _on_message(self, ws, message):
	"""Receive message from server (runs in WebSocketApp thread)"""
	interval = time.time() - self.start_time
	self.start_time = time.time()
	print(f"[client] recv_action interval: {interval} seconds")
	try:
	data = json.loads(message)
	action_data = data.get("action")
	version = data.get("version", -1)
	if action_data is not None:
	action = convert_numpy_in_dict(action_data, numpy_deserialize)
	if isinstance(action, np.ndarray):
	self.execute_action(action)
	print(f"[client] Received action, version={version}")

. The control loop then reads that same buffer at FREQ_CTRL, and the action is never consumed or cleared:

Psi0/real/deploy/psi-inference_rtc.py

Lines 272 to 342 in f4ff131

	def apply_action_from_buffer(last_pd_target):
	current_lr_arm_q, current_lr_arm_dq = master.get_robot_data()
	have_vla = False
	with pred_action_lock:
	action = pred_action_buffer["actions"]
	if action is not None:
	have_vla = True
	action = action[0]
	arm_cmd = None
	hand_cmd = None
	if have_vla:
	if action.shape[0] < 36:
	print("[CTRL] Invalid action shape:", action.shape)
	else:
	vx = action[32]
	vy = action[33]
	vyaw = action[34]
	# dyaw = action[35]
	target_yaw = action[35]
	vx = 0.6 if vx > 0.25 else 0
	vy = 0 if abs(vy) < 0.3 else 0.5 * (1 if vy > 0 else -1)
	rpyh = action[28:32]
	arm_cmd = action[14:28]
	hand_cmd = action[:14]
	master.torso_roll = rpyh[0]
	master.torso_pitch = rpyh[1]
	master.torso_yaw = rpyh[2]
	master.torso_height = rpyh[3]
	print("torso_roll, pitch, yaw, height:", master.torso_roll, master.torso_pitch, master.torso_yaw, master.torso_height)
	master.vx = vx
	master.vy = vy
	master.vyaw = vyaw
	master.target_yaw = target_yaw
	master.prev_torso_roll = master.torso_roll
	master.prev_torso_pitch = master.torso_pitch
	master.prev_torso_yaw = master.torso_yaw
	master.prev_torso_height = master.torso_height
	master.prev_vx = master.vx
	master.prev_vy = master.vy
	master.prev_vyaw = master.vyaw
	master.prev_target_yaw = master.target_yaw
	master.prev_arm = arm_cmd
	master.prev_hand = hand_cmd
	if not have_vla:
	master.torso_roll = master.prev_torso_roll
	master.torso_pitch = master.prev_torso_pitch
	master.torso_yaw = master.prev_torso_yaw
	master.torso_height = master.prev_torso_height
	master.vx = master.prev_vx
	master.vy = 0
	master.vyaw = master.prev_vyaw
	master.target_yaw = master.prev_target_yaw

and

Psi0/real/deploy/psi-inference_rtc.py

Lines 380 to 388 in f4ff131

	def control_loop_thread():
	dt = 1.0 / FREQ_CTRL
	last_pd_target = None
	while running.is_set() and not kill_event.is_set():
	try:
	last_pd_target = apply_action_from_buffer(last_pd_target)
	except Exception as e:
	print("[CTRL] loop error:", e)
	time.sleep(dt)

.

So if the WebSocket stays half-open while responses stop, the controller can keep replaying the last VLA action. _on_close() clears running, but a silent receive stall may never produce a close event. This matches the reported observation: sends continue, receives stop, and the last token remains active.

I would add a fail-closed action-freshness contract before adding automatic reconnection:

1. Store {action, server_version, received_at_monotonic} atomically.
2. In the control loop, reject an action once its age exceeds a configurable budget based on the expected inference period (for example, a small multiple of the normal receive interval rather than a hard-coded long timeout).
3. On expiry, command zero base velocity and enter the controller-defined whole-body hold/standing path. Do not continue replaying the stale arm/base target.
4. Add an observation sequence number and have the server echo the source sequence with each action. This distinguishes inference latency/backlog from packet loss and prevents an old action from being accepted after reconnect.
5. Bound observations to one in flight or a latest-only slot. Reconnection should use backoff and should not re-arm motion until a fresh action passes the version/sequence check.

A useful hardware-in-the-loop acceptance matrix would be: normal operation, injected 100/250/500 ms response delay, silent response drop while the socket remains open, and a hard disconnect during non-zero motion. For every case, record action age, observation/action sequence, commanded base velocity and the transition into hold. The pass condition is deterministic hold before the configured stale-action budget, with no pre-disconnect action replayed after reconnection.

I work on independent real-hardware G1/Go2-W integration. If the maintainers want one narrow recorded fault-injection cell, I can help validate the action-freshness/hold boundary on hardware once the expected safe-hold API and timeout budget are agreed. I have not run PSI0 on my hardware yet, so this is a code-and-log diagnosis rather than a claim that the patch is already validated.

[waiyc]

Follow-up: later checkpoints degrade on small single-task datasets

After resolving the earlier training issue, I fine-tuned PSI0 seperately on several simple pick and place task with small SONIC datasets, including snack-pack picking, tissue-pack picking, and paper-cup picking.
Each dataset contains approximately 40–50 episodes, with each episode lasting around 20 seconds.
Every episode begins and end in the default SONIC teleoperation WBC starting pose (arms resting naturally at its sides, with the hands pointing downward).

Across these tasks, the 5k checkpoint is consistently the most responsive and robust.
Performance progressively degrades at 10k–40k steps, and the 40k checkpoint can become almost stationary.

Current fine tune configuration with a laptop RTX5090 GPU:

--train.mixed_precision=bf16
--train.train_batch_size=64
--train.gradient_accumulation_steps=2
--train.max_training_steps=40000
--train.checkpointing_steps=5000
--train.max_checkpoints_to_keep=10
--train.num_workers=2

I also tested batch size 16 with 8 accumulation steps and observed similar performance.
Could this indicate overfitting or stationary-action bias because the datasets contain too few episodes?
For a small single-task dataset, should I:

1. Reduce the effective batch size?
2. Reduce the learning rate from 1e-4?
3. Limit training to approximately 5k–10k steps?
4. Select checkpoints using an action-magnitude or open-loop validation metric rather than the final training loss?

Is there a recommended episode count and fine-tuning configuration for small SONIC datasets?