Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor PCM audio utilities and add support for visualizers #119

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

mnemitz
Copy link
Contributor

@mnemitz mnemitz commented Feb 26, 2025

Package changes

⚠️ This is a breaking change to the following packages:

  • @speechmatics/browser-audio-input
    • Update PCMRecorder class to clean up the AudioWorkletNode, rather than assuming the AudioContext will be closed.
    • Automatically resume audio context when startRecording is called, throwing a specific error of this fails.
  • @speechmatics/browser-audio-input-react
    • Split into two hooks: usePCMAudioRecorder and usePCMAudioRecorderContext (first renders inline, second uses the first as a value to a context provider)
    • AudioContext must be supplied to usePCMAudioRecorder or PCMAudioRecorderProvider, rather than passed to startRecording
  • @speechmatics/web-pcm-player-react
    • Supply extra hook usePCMAudioPlayerContext, plus context and provider for audio player (useful for sharing player across the app)
    • Audio context must be supplied to either usePCMAudioPlayer() or PCMAudioPlayerProvider

Examples changes

NextJS Flow:

  • Add audio visualizer component, use for both agent and speaker transcript blocks
  • Update dependencies, including React types (which were mistakenly at 18 instead of 19)
Screenshot 2025-02-26 at 14 06 47

NextJS real-time

  • Update dependencies, including React types (which were mistakenly at 18 instead of 19)

@mnemitz mnemitz changed the title DRAFT: Refactor PCM player/recorder packages to follow similar structure Refactor PCM audio utilities and add support for visualizers Feb 26, 2025
@mnemitz mnemitz changed the title Refactor PCM audio utilities and add support for visualizers DRAFT: Refactor PCM audio utilities and add support for visualizers Feb 26, 2025
@mnemitz mnemitz changed the title DRAFT: Refactor PCM audio utilities and add support for visualizers Refactor PCM audio utilities and add support for visualizers Feb 26, 2025
Copy link
Contributor

@TudorCRL TudorCRL left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SDK code lgtm! I think there might be an issue with the useHydrated hook in the examples which means that a valid audioContext is never returned. Maybe we can have a chat about this offline?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

Successfully merging this pull request may close these issues.

2 participants