feat: record PC audio during online meetings - #20
Merged
Conversation
Covers issue #18: the meeting recorder captures the microphone only, so remote participants never reach the transcript. Design mirrors OBS Studio's audio mixer: WASAPI loopback capture (already supported by cpal 0.15.3) plus a timestamp-based timeline mixer, so silence on the system source becomes silence on the timeline instead of drift. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mixing two capture streams by arrival order breaks as soon as system audio joins: WASAPI loopback delivers no packets while nothing plays, so every silence would shrink and the system source would drift ahead of the microphone. Address the timeline by capture timestamp instead, the way OBS Studio's audio mixer does, so a gap becomes silence and stays put. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An online meeting now opens a second cpal stream on the render endpoint, which cpal turns into a WASAPI loopback client, and both streams mix onto the shared timeline. Remote participants finally reach the transcript. Also honours settings.audio_device, which the Settings UI has been writing since forever while audio.rs always opened the default microphone, and adds settings.system_audio_device for picking which output to capture. Dictation stays microphone-only even while a meeting runs, so a dictated sentence can never pick up whatever the machine is playing. Refs #18 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Record & Transcribe now asks whether the meeting is in-person or online. Online opens the loopback capture, adds a PC level meter beside the microphone meter, and warns when no PC audio has arrived for 30 seconds — the failure you would otherwise only discover after the meeting. Settings gains a system audio device dropdown for when you switch between headphones and speakers. Outside the desktop app the online option is disabled: browser mode would need getDisplayMedia and a share prompt. Refs #18 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Renders a quiet 440 Hz tone on the default output and asserts the loopback stream picks it up and reaches the mixed timeline. Ignored by default since it needs audio hardware: cargo test --lib -- --ignored loopback Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #18.
The problem
Recording a meeting captured the default microphone and nothing else, so a Teams,
Zoom or Meet call produced a transcript containing only the local user. The
workaround was recording the call in OBS Studio first and transcribing the file
afterwards, which makes the built-in recorder pointless.
Two smaller defects sat in the same code path:
settings.audio_devicewas written by the Settings UI and stored, butaudio.rsnever read it — it always opened
host.default_input_device(). The dropdown didnothing.
was over.
What changed
Record & Transcribe now asks whether the meeting is in-person (microphone
only) or online (microphone plus PC audio). Online opens a second cpal stream on
the render endpoint, which cpal turns into a WASAPI loopback client — it sets
AUDCLNT_STREAMFLAGS_LOOPBACKitself, so no new dependency was needed.Dictation via the global hotkey stays microphone-only even while a meeting is
running, so a dictated sentence can never pick up whatever the machine is playing.
Why a timeline mixer
Appending samples as they arrive does not survive system audio. WASAPI delivers no
packets at all while nothing is playing, so every silent stretch would shrink and
the system source would slide ever further ahead of the microphone — badly out of
sync well before an hour-long meeting ends, and overlapping speech is exactly what
Whisper handles worst.
OBS Studio solves this in its mixer rather than its capture, and this PR does the
same.
TimelineMixer(new,src-tauri/src/mixer.rs) addresses one shared 16 kHztimeline by capture timestamp: gaps become silence, overlapping sources are summed
and clamped. Timestamps come from
InputCallbackInfo::timestamp().capture, whichon WASAPI is an absolute QPC value, so both streams share one clock. The first
microphone packet anchors position zero.
take()leaves a 300 ms tail so a packetfrom the slower stream still lands in place when the recorder pulls a segment.
Also in this PR
settings.audio_deviceis finally honoured, for meetings and dictation alike.settings.system_audio_deviceplus a Settings → Audio → System audiodevice dropdown, for switching between headphones and speakers.
arrived for 30 seconds — the failure you would otherwise only discover afterwards.
instead of failing the recording.
Failure handling
A meeting should never be lost to an audio error.
Testing
16 unit tests, all passing (
cargo test --lib): the mixer's gap-to-silence,out-of-order writes, summing and clamping, the drain margin, stale-write rejection;
plus device resolution and the downmix/resample helper.
One opt-in hardware test verifies the real thing on Windows — it renders a quiet
440 Hz tone on the default output and asserts the loopback stream picks it up and
reaches the mixed timeline:
Verified passing on Windows 11.
npm run buildandtsc --noEmitare clean.Not in this PR
Keeping microphone and system audio as separate tracks for speaker attribution
("me" versus "the others"). Attractive for minutes, but it doubles transcription
time and deserves its own design.
Design and plan:
docs/superpowers/specs/2026-08-28-system-audio-capture-design.md,docs/superpowers/plans/2026-08-28-system-audio-capture.md.🤖 Generated with Claude Code