Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add event to receive raw PCM16 Audio #57

Open
AnasMations opened this issue Aug 7, 2024 · 3 comments
Open

Add event to receive raw PCM16 Audio #57

AnasMations opened this issue Aug 7, 2024 · 3 comments

Comments

@AnasMations
Copy link

We are adding a face to Vapi's AI agent but there is a limitation:
We want to be able to receive raw PCM16 audio bytes to pass that to our facial animation model.

@Mhal007
Copy link

Mhal007 commented Aug 23, 2024

We have a similar need actually. We are not displaying a face, but we'd also like to be able to show an animation of sorts to visualize the agent speaking.

@nikhilro
Copy link
Contributor

 public getDailyCallObject(): DailyCall | null {
    return this.call;
  }

we expose vapi.getDailyCallObject() now. to unblock yourself, recommend getting the raw pcm from there

@austinw1995
Copy link

I also found a workaround to query the DOM for audio elements that are playing live media streams through the quickstart-react webpage, then for each stream, I created an AudioContext and connect it to a ScriptProcessorNode to capture raw PCM audio samples. Note that the audio capture itself isn’t handled by an external library, it uses the browser’s native Web Audio API (with an AudioContext and ScriptProcessorNode).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants