Skip to content

[FEATURE] Engine-owned visual vertex buffer (set_vverts / get_vverts API)#2776

Open
duburcqa wants to merge 5 commits into
Genesis-Embodied-AI:mainfrom
duburcqa:vverts-consolidation
Open

[FEATURE] Engine-owned visual vertex buffer (set_vverts / get_vverts API)#2776
duburcqa wants to merge 5 commits into
Genesis-Embodied-AI:mainfrom
duburcqa:vverts-consolidation

Conversation

@duburcqa
Copy link
Copy Markdown
Collaborator

Summary

Adds vverts_state to KinematicSolver as the single engine-side ground truth for per-frame world-space visual vertex positions. Surfaces set_vverts / get_vverts at solver, entity, and vgeom levels. Rasterizer picks up user overrides via a lazy migration to per-env vertex-buffer rendering; collision/sdf rendering is unchanged.

This is the renderer half. PR #2769 (visual-mesh raycasting) rebases on top of this — its kernel_copy_custom_vverts and parallel _custom_vverts buffer go away because vverts_state is now the engine-level buffer everyone reads.

Supersedes the closed #2768 by relocating the buffer from KinematicEntity to the solver.

Design

  • vverts_state.pos: gs.qd_vec3, (n_vverts_, B) — populated by kernel_update_all_vverts (vgeom-pose × init_pos) in KinematicSolver.update_vgeoms.
  • vverts_info.is_custom: gs.qd_int — flips to 1 for entries written by set_vverts. FK skips them so user data survives step().
  • KinematicOptions.batch_vverts_info: bool = False (mirrors batch_links_info / batch_dofs_info / batch_joints_info). When True, the whole vverts_info struct gains a B dimension; without it, partial envs_idx on set_vverts raises.
  • set_vverts(verts, envs_idx=None) / get_vverts(envs_idx=None) at KinematicSolver, KinematicEntity, Vgeom. vverts=None clears. Plane entities are refused.
  • Rasterizer: non-custom entities keep the existing instancing path. Entities with any custom vvert migrate to per-env pyrender nodes that re-upload from vverts_state.pos each frame. Clear migrates back; the restored render is byte-identical to baseline.
  • Zero-copy on set_vverts when supported; warns once per build on non-zero-copy backends and uses a fallback kernel.

Test plan

  • tests/test_render.py::test_set_vverts[False-RASTERIZER] and [True-RASTERIZER] cover the API + render path end-to-end (round-trip, broadcast forms, get-returns-copy, vgeom-level write, Plane refusal, migration back, partial-envs error for unbatched, mixed user/FK envs for batched).
  • Reviewers verify the vverts_info batching change doesn't break existing consumers (only one in genesis/vis/batch_renderer.py; adjusted to take env-0 slice when batched).

Relationship to other PRs

@duburcqa duburcqa requested a review from YilingQiao as a code owner May 12, 2026 20:09
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c6587a54c

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread genesis/vis/rasterizer_context.py Outdated
Comment on lines +517 to +518
return bool(is_custom[entity.vvert_start, self.rendered_envs_idx].any())
return bool(is_custom[entity.vvert_start])
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Detect custom vverts across the full entity range

_entity_has_custom_vverts only inspects entity.vvert_start, so it misses overrides written to any later slice (e.g., RigidVisGeom.set_vverts on a non-first vgeom). In that case update_rigid never migrates the entity to the per-env vertex-upload path, and the custom vertices are ignored in rendering/sensors even though set_vverts succeeded. This regresses correctness for multi-vgeom entities where users override a subset that does not include the first vertex.

Useful? React with 👍 / 👎.

@duburcqa duburcqa force-pushed the vverts-consolidation branch 2 times, most recently from 56f26e4 to be93ac4 Compare May 12, 2026 22:25
@duburcqa duburcqa force-pushed the vverts-consolidation branch from be93ac4 to d63fdd4 Compare May 12, 2026 22:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant