Skip to content

Speed up evaluation with lockstep execution and sparse Path Tracing - #2

Open
little-veg wants to merge 2 commits into
FluxVLA:mainfrom
little-veg:main
Open

Speed up evaluation with lockstep execution and sparse Path Tracing#2
little-veg wants to merge 2 commits into
FluxVLA:mainfrom
little-veg:main

Conversation

@little-veg

Copy link
Copy Markdown

Summary

This PR speeds up FluxBisim evaluation by removing fixed wall-clock action pacing and adding an opt-in sparse Path Tracing mode.

Changes

  • Execute action chunks in fast lockstep mode without the previous fixed 10 Hz delay.
  • Add --realtime and --control-rate to preserve wall-clock-paced execution when needed.
  • Add chunk completion synchronization between FluxBisim and FluxVLA.
  • Add opt-in Path Tracing through FLUXBISIM_RENDERER=pathtracing.
  • Add configurable Path Tracing SPP, denoising, and reset warm-up frames.
  • Add --render-interval to skip intermediate renders while still advancing every physics action.
  • Reset Path Tracing accumulation before boundary renders.
  • Verify that every camera advances before publishing a boundary observation.
  • Preserve the existing RaytracedLighting + DLSS behavior by default.
  • Document the new evaluation modes and configuration options.

Sparse rendering is restricted to fast, headless Path Tracing with image saving disabled. The render interval must divide the action chunk size.

Usage

Render only the final action of each 32-action policy chunk:

FLUXBISIM_RENDERER=pathtracing \
FLUXBISIM_PT_SPP=4 \
PYTHON_BIN=/path/to/isaac-sim/python.sh \
  ./fluxbisim_benchmark.sh pick_place_banana \
  --render-interval 32

## Validation

- 6 renderer and scheduling unit tests passed.
- flake8 passed.
- ruff passed.
- Isaac Sim startup and ROS bridge initialization passed with:
    - Path Tracing
    - SPP 4
    - render interval 32

- Preliminary banana benchmark:
    - 10/10 successful episodes
    - 100% observed success rate
    - 72.78 average action steps/s
    - no missing, duplicate, malformed, or unpaired action messages
    - no sparse camera synchronization failures

The 10-episode benchmark is intended as a regression smoke test. A larger evaluation is still recommended before making production success-rate claims.

## Integration note

The fast lockstep path requires the corresponding FluxVLA operator changes that publish indexed action chunks and wait for the matching chunk_done acknowledgement.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants