Skip to content

Extract shared runner I/O utilities#384

Merged
jarcherNV merged 5 commits into
mainfrom
dev/jarcher/common-refactor
Jul 20, 2026
Merged

Extract shared runner I/O utilities#384
jarcherNV merged 5 commits into
mainfrom
dev/jarcher/common-refactor

Conversation

@jarcherNV

@jarcherNV jarcherNV commented Jul 18, 2026

Copy link
Copy Markdown
Collaborator

Extract shared runner I/O utilities

Add flashdreams.infra.runner_io for common prompt resolution, output paths,
stats JSON, image/video loading, normalization, and MP4 writing with lazy
cv2/mediapy imports.

Migrate matching public runners to use the shared helpers while preserving
model-specific download, crop, camera, and writer behavior. Add CPU tests for
the new helper contracts.

Add flashdreams.infra.runner_io for common prompt resolution, output paths,
stats JSON, image/video loading, normalization, and MP4 writing with lazy
cv2/mediapy imports.

Migrate matching public runners to use the shared helpers while preserving
model-specific download, crop, camera, and writer behavior. Add CPU tests for
the new helper contracts.
@copy-pr-bot

copy-pr-bot Bot commented Jul 18, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@jarcherNV jarcherNV self-assigned this Jul 18, 2026
@greptile-apps

greptile-apps Bot commented Jul 18, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR extracts a new flashdreams.infra.runner_io shared module consolidating prompt resolution, output path construction, stats JSON writing, image/video loading, normalization, and MP4 writing — replacing duplicated in-runner helpers across eight integration runners. All callers are migrated to the shared API while model-specific logic (crop, camera, custom writers) is preserved in-place.

  • runner_io.py introduces the new utility surface with lazy cv2/mediapy imports, explicit ValueError guards (replacing the old assert-based validation), and a .detach().cpu().float() chain in video_tensor_to_uint8 that handles CUDA tensors correctly.
  • Eight integration runners drop their local I/O helpers in favour of the shared equivalents; rearrange/einops, json, cv2, and mediapy are removed from their top-level imports where they were only used in the removed helpers.
  • CPU-only test suite covers all three VideoTensorLayout branches, the prompt resolver, the download pass-through, stats JSON format, and the first-frame resize pipeline.

Confidence Score: 5/5

Safe to merge — the extraction is a straightforward refactor that removes duplicated I/O helpers without altering any model-specific logic, download, crop, or camera behaviour.

All eight migrated runners preserve their original tensor layouts, interpolation modes, dtype placement, and file-naming conventions. The previous concerns about assert-based validation, missing .cpu() calls on CUDA tensors, and the untested thwc layout branch are all addressed in this PR. No regressions were found during cross-referencing of old and new code paths.

No files require special attention.

Important Files Changed

Filename Overview
flashdreams/flashdreams/infra/runner_io.py New shared I/O utility module; all three VideoTensorLayout paths, prompt validation, and download routing are correct. Previous assert-based guards replaced with ValueError; .detach().cpu() added to video_tensor_to_uint8.
flashdreams/tests/test_runner_io.py Comprehensive CPU test suite covering all three tensor layout branches (thwc now included), prompt validation, URL download, stats JSON format, lazy-import paths, and interpolation round-trip.
integrations/omnidreams/omnidreams/runner.py Migrated _load_first_frame, _load_video, and _write_video to shared helpers; rearrange-to-thwc canvas path correctly uses layout='thwc'; einops import retained for remaining multi-view rearrange calls.
integrations/flashvsr/flashvsr/runner.py Video loading replaced with rgb_video_to_normalized_tensor + manual permute/unsqueeze producing identical [1,C,T,H,W]; bcthw write_video_tensor call matches postprocess_output_layout='bcthw'. No remaining rearrange usages after import removal.
integrations/lingbot/lingbot/runner.py load_first_frame_tensor preserves bicubic interpolation (interpolation='cubic') and bfloat16 dtype from old _load_first_frame; write_video_tensor with layout='tchw' is equivalent to the old rearrange+_write_video pair.
integrations/causal_forcing/causal_forcing/runner.py I2V path now uses load_first_frame_tensor via resolve_input_path; default interpolation (INTER_LINEAR) matches the old cv2.resize call; validator=read_image_rgb provides equivalent image validation during URL downloads.
integrations/cosmos_predict2/cosmos_predict2/runner.py Mirrors causal_forcing migration exactly; T2V and I2V paths both correctly updated with preserved behaviour.
integrations/wan21/wan21/runner.py Same migration pattern as causal_forcing; I2V image load and T2V video write correctly use shared helpers.
integrations/hy_worldplay/hy_worldplay/runner.py Prompt, output dir, artifact path, and stats helpers migrated; custom _write_mp4 intentionally retained for model-specific MP4 writing behaviour.
integrations/fastvideo_causal_wan22/fastvideo_causal_wan22/runner.py T2V-only runner correctly migrated; write_video_tensor with layout='tchw' matches old rearrange pattern.
integrations/self_forcing/self_forcing/runner.py Prompt resolution, video write, and stats write correctly migrated to shared helpers with no behavioural changes.
flashdreams/flashdreams/recipes/template/runner.py Minimal change: mkdir replaced with ensure_output_dir and f-string path replaced with runner_artifact_path; behaviour preserved.

Reviews (5): Last reviewed commit: "Migrate remaining runners to shared runn..." | Re-trigger Greptile

Comment thread flashdreams/flashdreams/infra/runner_io.py Outdated
Comment thread integrations/omnidreams/omnidreams/runner.py
Comment thread integrations/omnidreams/omnidreams/runner.py Outdated
Comment thread flashdreams/flashdreams/infra/runner_io.py Outdated
Comment thread flashdreams/tests/test_runner_io.py
Make prompt validation explicit, move video tensors to CPU before NumPy
conversion, reuse the shared runner install hint in OmniDreams, and add
coverage for empty prompts and THWC video conversion.
Comment thread flashdreams/flashdreams/infra/runner_io.py
Replace remaining assert-based runner I/O guards with explicit ValueError
checks so invalid empty-video and multi-batch BCTHW inputs fail reliably
under optimized Python. Add focused tests for both validation paths.
@jarcherNV jarcherNV changed the title Add shared realtime inference utilities across demos Extract shared runner I/O utilities Jul 19, 2026
@jarcherNV

Copy link
Copy Markdown
Collaborator Author

/ok to test 101de2c

Use setattr when monkeypatching fake media and cv2 modules in the runner
I/O tests so ty does not flag unresolved ModuleType attributes. Also
apply Ruff formatting to the OmniDreams runner.
@jarcherNV

Copy link
Copy Markdown
Collaborator Author

/ok to test 880d96d

Add shared helpers for cached input-asset resolution and lazy video FPS
probing, then migrate public runner call sites to use them directly.

This removes duplicated local/URL path resolution from Wan, Causal
Forcing, Cosmos Predict2, and FlashVSR, moves FlashVSR video loading and
metadata probing onto shared runner I/O, and routes the remaining simple
runner output path creation through the shared helpers.
@jarcherNV

Copy link
Copy Markdown
Collaborator Author

/ok to test e7688b4

"""Write a ``[-1, 1]`` video tensor as an MP4 and return the path."""
media = _import_mediapy("Writing videos", install_hint=install_hint)
path = Path(path)
media.write_video(str(path), video_tensor_to_uint8(video, layout=layout), fps=fps)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not for this PR, but we should provide a stream_video_tensor_to_file function to avoid OOM for long video output by appending the tensor to the output during the rollout, rather than dumping in the end at once.

resolve_prompt_value,
runner_artifact_path,
write_runner_stats,
write_video_tensor,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: instead of importing all the functions, maybe better organize them in a single class?

@gtong-nv gtong-nv left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
Left a few comments but OK to address in the future.

@jarcherNV

Copy link
Copy Markdown
Collaborator Author

Thanks for the comments. I have added them to my TODO list.

@jarcherNV
jarcherNV added this pull request to the merge queue Jul 20, 2026
Merged via the queue into main with commit 1cc00f1 Jul 20, 2026
8 checks passed
@jarcherNV
jarcherNV deleted the dev/jarcher/common-refactor branch July 20, 2026 20:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants