Repository navigation
perf(editor): stream short Studio Sound cuts with bounded history - #3254
Merged
richiemcilroy merged 1 commit intoOct 25, 2026
Merged
richiemcilroy merged 1 commit into
richiemcilroy merged 1 commit into
Conversation
Contributor
|
Someone is attempting to deploy a commit to the Cap Software Inc Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Short forward cuts with Studio Sound currently fall back to decoding every recording track into full PCM before MP4 export can start. This avoids the preroll read-back error fixed by #2812, but makes a short edit of a long recording wait for a full-track allocation/decode.
Keep a bounded history for enhanced microphone reads so the voice enhancer can warm up after those cuts while its decoder continues forward. Ordinary tracks request zero history. The original sample arithmetic, gain/stereo handling, enhancer reset rules, cancellation and full-file validation remain in use. Backward source timelines, retiming, music and unsupported transition shapes retain their existing eligibility rules.
This implements the bounded-history extension suggested in review. #3229 adds end-to-end regression coverage for short cuts; this extends the streaming implementation itself.
Complete MP4 measurements
Synthetic one-hour recording edited to eight seconds with a 100 ms forward gap; two four-second clips. The benchmark follows the GPUI app's
ExporterBuilder::build_for_mp4→Mp4ExportSettings::export_preparedpath, renders and encodes the complete video, mixes Studio Sound and validates the remaining source audio before successful completion.Complete exports take 12.4–23.8% less time, first rendered-frame progress arrives 76.7–91.5% sooner, and process peak RSS is 84.8–85.5% lower. The AAC case removes about 1.9 seconds of initial waiting. These are standalone exporter-process measurements; they do not include an already-open editor's retained audio or establish total desktop-app memory savings.
Five alternating independent processes per revision/source/profile/case, with warm filesystem and shader caches. All 120 MP4 exports have matching decoded video and float-PCM SHA-256 hashes within each workload, across baseline/head and profiles. Timers include actual builder preparation, rendering, audio processing/validation, encoding and final output completion. First progress is the callback after a rendered frame is received, before forwarding it to the encoder; it is not UI window presentation. Metric columns are independent process medians.
The
export_preparedphase alone is 9–100% slower in the newly eligible cases because full-file validation moves into that phase; the reported total includes preparation on both revisions. Validation is still performed on the entire source. The change does not promise constant total export time independent of source length.Already-streaming exact splits and one-second gaps are controls. Their total times range from 2.5% faster to 1.3% slower; worst slowdown is WAV/size, 1.724 → 1.746 s. Peak RSS control changes are at most about 1.7 MiB. The largest preparation-control slowdown is 5.8 ms. No control speedup is claimed.
Linux x86_64, AMD EPYC 9V74, two-CPU quota, affinity CPUs 0,1; Rust 1.88.0, FFmpeg 7.1.5, Mesa 25.0.7 Lavapipe with two rasterizer threads. Root release uses opt-level=s/thin LTO/one codegen unit.
editor3overrides only cap-editor to level 3; it is not a full GPUI shipping-profile or physical-GPU measurement. Source video is generated 320×180 H.264 at 1 fps; output is 640×360 at 15 fps. Microphone inputs are stereo 48 kHz PCM16 WAV or AAC-LC/MP4 at 96 kb/s, generated from deterministic tones. There is no camera/cursor drawing, music or caption track.Baseline
0983901353c759f4857520afdfdc5fe12b2ff19a. Raw repeated samples, saved binaries, comparison JSON and hashes are retained locally outside the production diff.Correctness and bounds
Invalid sequential export audio windowon the baseline, then compares complete sample bits against the existing in-memory renderer after repeated short cuts. It exercises all isolation tiers, mono/stereo microphones, positive/negative offsets, source switches, one-sample and non-aligned requests, and source exhaustion.Veccapacity and other decoder/model allocations are separate from this sample-length bound.Validation
cargo fmt --all --checkand editor/export all-target checks pass.--no-deps.mp4.rs:2120; the same code is present on the baseline. No lint suppression or unrelated cleanup is included.Reproduction
Create a fresh fixture directory; the example writes its project config/metadata. Copy the new example onto the baseline, build it on both revisions and save the binaries separately:
cargo build -p cap-export --release --locked --example studio-sound-cut-benchmark cargo build -p cap-export --release --locked --example studio-sound-cut-benchmark \ --config profile.release.package.cap-editor.opt-level=3 mkdir -p /tmp/cap-short-cut ffmpeg -f lavfi -i testsrc2=size=320x180:rate=1 -t 3600 \ -c:v libx264 -preset ultrafast -threads 2 -g 1 -pix_fmt yuv420p \ /tmp/cap-short-cut/display.mp4 ffmpeg -f lavfi \ -i 'aevalsrc=0.14*sin(2*PI*180*t)+0.04*sin(2*PI*817*t)|0.11*sin(2*PI*240*t)+0.03*sin(2*PI*1103*t):s=48000' \ -t 3600 -c:a pcm_s16le /tmp/cap-short-cut/mic.wav taskset -c 0,1 /path/to/saved/studio-sound-cut-benchmark \ /tmp/cap-short-cut /tmp/cap-short-cut/result.mp4For AAC, encode the generated WAV to
mic.m4awith-c:a aac -b:a 96k, then expose that file as the fixture'smic.wavpath (for example, with a symlink in a separate fixture directory). FFmpeg detects the container from its contents. The saved benchmark binaries are identical for both source formats. SetCAP_EXPORT_CUT_GAP=0or1for the already-streaming controls.Use the contributor guide's native environment. To reproduce the headless measurements, set
VK_DRIVER_FILESto a Lavapipe ICD,CAP_RENDER_FORCE_SOFTWARE_ADAPTER=1,LP_NUM_THREADS=2, and a writableXDG_RUNTIME_DIR. Alternate five baseline/head processes. The example emits preparation/first-progress/total timings and LinuxVmHWM. Compare decoded video (-map 0:v:0 -pix_fmt rgba) and PCM (-map 0:a:0 -c:a pcm_f32le) using FFmpeg's SHA-256 hash muxer, outside timing.Risk is confined to retaining and addressing a bounded decoded-sample prefix, and enabling the existing streaming pipeline for short forward cuts. There is no new public API, dependency, unsafe code, persistent cache, neural model or audio-format change.


Confidence Score: 5/5
The PR appears safe to merge; no actionable issues were found.Summary
Enables streaming audio for short forward cuts with Studio Sound by retaining a bounded prefix of decoded samples.
No actionable issues found. Review used code inspection; tests and benchmark measurements were not rerun.
Reviews (1) · Last reviewed commit: "perf(editor): stream short Studio Sound ..." · Reviewed by Greptile