feat: Add protocol-driven GR00T inference compatibility

[eraykeskinmac]

Summary

Add multi-protocol support for the GR00T policy client, enabling communication with both Gr00tSimPolicyWrapper servers (Isaac-GR00T N1.6+) and direct policy servers (N1.5).

Problem

Isaac-GR00T N1.6 changed the server-side inference protocol: video observations require a temporal dimension (B,T,H,W,C), state values must be float32, requests must be wrapped in {"observation": obs}, and action responses include a batch dimension. The existing client only supports the older format, causing inference failures with newer servers.

Approach

Data configs are unchanged. The original key mappings (video, state, action, language) work with both protocols. Only the transport-level formatting differs — this is handled by a new PROTOCOLS registry:

Protocol	Video	State dtype	Request	Response	Used by
sim_wrapper	5D (B,T,H,W,C)	float32	{"observation": obs}	(B,T,dim)	N1.6+ SimPolicyWrapper
direct	4D (B,H,W,C)	float64	flat dict	(T,dim)	N1.5 direct policy

Adding future protocols requires only a new entry in PROTOCOLS — no changes to Gr00tPolicy or data configs.

Changes

File	What changed
data_config.py	Add PROTOCOLS registry. Add protocol field to libero configs. Original config keys untouched.
__init__.py	Single _build_observation() adapts to protocol. Batch dim stripping for responses.
client.py	Protocol-aware request wrapping with auto-detection fallback.
tests/test_groot_n1d6.py	26 tests: config integrity, observation formatting, action conversion, protocol selection.

Backward compatibility

• All original data configs preserved exactly (verified by test_original_configs_unchanged)
• groot_version="n1d6" kwarg still works (maps to protocol="sim_wrapper")
• Configs without protocol field default to auto-detection
• All 11 existing tests pass unchanged

Usage

# Protocol auto-resolved from config (recommended)
policy = create_policy(provider="groot", data_config="libero", host="localhost", port=5556)

# Explicit override
policy = create_policy(provider="groot", data_config="libero:sim_wrapper", ...)
policy = create_policy(provider="groot", data_config="libero:direct", ...)

Test results

• 37 tests pass (26 new + 11 existing, zero regressions)
• Lint clean (black, isort, flake8)
• Real GR00T N1.6 server on AWS g5.xlarge (A10G 24GB GPU):
  • 3/3 Libero spatial tasks completed with reward=1.0
  • 0xAnkitSingh/GR00T-N1.6-LIBERO checkpoint

[eraykeskinmac]

@cagataycali Hey! Could you review this PR when you get a chance? 🙏

[yinsong1986]

PR #5 — feat: Add protocol-driven GR00T inference compatibility

Summary: Adds a two-protocol dispatch layer (sim_wrapper / direct) to the GR00T policy client so it can communicate with both Isaac-GR00T N1.5 and N1.6 servers without changing data configs.

---

🟡 Suggestions (non-blocking)

• [__init__.py / _map_libero_state] The original code had an explicit else branch that zeroed out all state keys when robot0_eef_pos/robot0_eef_quat were missing. The new version drops it — if those keys are absent, state entries are never added to obs. Worth restoring as a defensive fallback.

• [client.py / get_action auto-detect] _request never raises on a malformed response, so the except RuntimeError fallback to direct will never trigger in practice. Consider validating the response shape/structure and falling back on that instead.

• [__init__.py / _create_fallback_actions] Fallback now returns np.zeros((h, 1)) for every key regardless of actual action dimensionality. Worth restoring per-key dim inference from key names (joint_pos→7, eef_pos→3, etc.).

• [__init__.py / _add_video_dims] reshape(1, 1, *image.shape) assumes input is always (H, W, C). An assert image.ndim == 3 before the reshape would catch edge cases early.

• [data_config.py / _lookup] Fuzzy libero match + protocol override precedence is implicit — a short comment would help future readers.

• [tests/test_groot_n1d6.py] Consider mocking GR00TClient in tests that instantiate Gr00tPolicy to make isolation explicit.

---

✅ Verdict

Needs changes (minor) — solid, well-structured work with good test coverage and clean backward compatibility. A few small defensive gaps worth addressing.

[yinsong1986]

Creating environment: default
Installing project in development mode
Checking dependencies
Syncing dependencies
cmd [1] | black --check strands_robots_sim tests
would reformat /home/runner/work/robots-sim/robots-sim/strands_robots_sim/policies/groot/client.py
would reformat /home/runner/work/robots-sim/robots-sim/strands_robots_sim/policies/groot/__init__.py
would reformat /home/runner/work/robots-sim/robots-sim/tests/test_groot_n1d6.py

FYI: @eraykeskinmac failing the lint test. Thanks!