Add OBS Studio Workflow blocks: OBS Connection, OBS Action, Detections Class Router, RTSP Stream Watcher

[Tyler-Odenthal]

Summary

Adds Workflow blocks that let a Workflow drive OBS Studio over obs-websocket, plus two supporting blocks that make the "when the model sees X, do Y in OBS" pattern composable without custom code:

Block	Type	Purpose
OBS Connection roboflow_core/obs_connection@v1	sink	Holds the websocket address; can discover the local OBS password from OBS's own config; emits an obs_connection other blocks consume
OBS Action roboflow_core/obs_action@v1	sink	One OBS operation per run: set_scene, set_source_visibility, set_text, toggle_filter, trigger_hotkey, set_source_transform, move_source_to_detection, start/stop virtual camera, start/stop recording
Detections Class Router roboflow_core/detections_class_router@v1	formatter	Maps the most confident detection's class to a value via a {class: value} table (e.g. {"cup": "Coffee", "cat": "Cat"}); empty → default / None
RTSP Stream Watcher roboflow_core/rtsp_stream_watcher@v1	transformation	Watches a second RTSP camera from inside a Workflow: background reader, optional background model on a check interval, emits latest frame + predictions; costs the primary video path no frame rate

New kind: obs_connection.

Why

Customers asked for a way to react to what a model sees by switching OBS scenes, moving overlays, or toggling filters during a stream or call. Today that needs a script outside Workflows. With these blocks the whole thing is a Workflow: webcam → detector → Class Router → OBS Action(set_scene), with extra cameras via the RTSP Stream Watcher and First Non Empty Or Default for priority.

Design notes

• Connection ≠ credentials. The connection descriptor passed between blocks carries host/port/timeout only. Passwords live in a process-local registry; the obsws INFO logger that prints them is raised to WARNING.
• Password discovery reads OBS's obs-websocket/config.json (macOS / Windows / Linux / Flatpak paths) when OBS is local, so the common case needs no password field. It is only attempted when ALLOW_WORKFLOW_BLOCKS_ACCESSING_LOCAL_STORAGE permits blocks to read the host filesystem (the same gate the Local File sink honours); with it off the block never touches the disk and says so in message. The block's message always reports where the credential came from, never the credential.
• One pooled websocket per (host, port) with a per-connection lock; one reconnect on transport errors, fail-fast on OBS application errors.
• OBS Action is per-frame safe: skip_if_unchanged (default on) dedups identical calls; a None target (nothing routed this frame) is a skip, not an error; scene-item ids and canvas size are cached; failures are logged and reported in message, never raised into the pipeline. Unknown scene/source names report the available names.
• RTSP Stream Watcher runs its model off the request path. The reader thread drains the stream continuously, samples every check_interval_seconds, and runs model_id on the sample itself through the ModelsProvider port; run() returns the latest detections instantly. Measured on an M5 Max: workflow throughput identical with 0 or 2 extra cameras (4.1 fps both), vs 2.9 fps when camera detectors sat on the request path. Readers are daemon threads shared per (URL, model), reconnect on failure, stop when idle, and strip credentials from logs.
• Both prediction representations: the Class Router, OBS Action and RTSP Stream Watcher read detections through small accessors that accept supervision.Detections or the tensor-native inference_models Detections (ENABLE_TENSOR_DATA_REPRESENTATION), and the watcher emits whichever the workflow runs with; the integration tests pass in both CI variants.
• Workflows-package hygiene: no imports from the inference server package and no direct environment reads, per the decontamination lint and the isolation probe.
• Runtime restrictions: OBS blocks and the watcher are HARD-restricted on Hosted Serverless / Dedicated Deployments (they need LAN access to OBS or the camera), matching the ONVIF sink; OBS blocks also refuse REMOTE step execution. OBS Connection honours disable_sinks.

Testing

• Unit: tests/workflows/unit_tests/core_steps/sinks/test_obs.py (60) and .../transformations/test_rtsp_stream_watcher.py (12), fake OBS clients / fake captures, no network.
• Integration: tests/workflows/integration_tests/execution/test_workflow_with_obs_blocks.py runs two full workflows through the Execution Engine (scene switch on a detected class incl. dedup and the hold-on-nothing case; RTSP watcher with a background model feeding a Class Router) and registers both in the workflows gallery.
• Live-verified against OBS Studio and two RTSP cameras: scene switching on detected classes, GIF overlays following bounding boxes, camera-triggered switching with the watcher's background model, password discovery, dedup, and error messages for missing scenes/sources.

Dependencies

obsws-python~=1.8.0 added to requirements/_requirements.txt.

Not in this PR

Keypoint-driven avatar actions were prototyped and removed to keep the initial release small.

🤖 Generated with Claude Code

[github-actions]

👋 Thanks for the pull request! Here is how automated Claude review works here, so you spend credits (and reviewer time) wisely.

🚧 Right now this is a draft, so automated Claude review is paused — nothing is being spent yet. Mark it Ready for review to trigger it.

Warning

💸 The Claude reviewer bills in credits, not vibes

Automated review spins up a real agent that reads real code and spends real credits on every pass. It is glad to help — but it is not a rubber duck, a linter you poke in a loop, or a substitute for reading the contributing guide. Treat it like an expensive senior reviewer whose time you booked, and show up prepared.

Draft when unsure, Ready when you mean it:

• 🌱 Not sure the PR is in good shape yet? Keep it (or set it back) as a draft — drafts pause review, so you can push and iterate without burning credits on a moving target.
• 💪 Feel strong about the contents? Mark it Ready for review and the reviewer will take a look.

However you get there, arrive prepared:

• 🧱 Bring a SOLID, thorough PR. Point your local agent at our skills/ to tune it to our guidelines first — or, if you are one of those fabled carbon-based contributors, read them yourself. A half-baked diff costs exactly the same to review as a finished one.
• ✅ Resolve every comment before you re-request review. Re-requesting with threads still open means paying twice for the same conversation.
• 🔁 Do not use CI review as an inner loop for a local agent. The reviewer is not a step-by-step debugger — do the unfolding locally and arrive with the answer, not the search.
• 🙋 If something looks off, ask a human. One question to a maintainer is cheaper and faster than three rounds of agent re-review chasing a misread.

Reviews are not free. A draft costs nothing to review; a Ready PR is a promise that it is worth reviewing.

• Prefer to skip automated review entirely? Add the skip-claude-review label.