Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Updated intro #1591

Merged
merged 1 commit into from
Oct 1, 2024
Merged

Updated intro #1591

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 12 additions & 3 deletions shared/open-ai-integration/quickstart.mdx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Integrating Agora's real-time audio communication capabilities with OpenAI's lan

The `RealtimeKitAgent` class manages the integration by connecting to an Agora channel for real-time audio streaming and to OpenAI's API for processing audio input and generating AI-driven responses. Audio frames captured from the Agora channel are streamed to OpenAI's API, where the AI processes the input. The API responses, which include transcribed text and synthesized voice output, are then delivered back to the Agora channel.

The code sets up tools that can be executed locally or passed through the API. This allows the AI to perform specific tasks, such as retrieving data from external sources. The agent processes various message types from OpenAI, such as audio responses, transcription updates, and error messages, and sends them to users through the Agora audio channel, facilitating continuous interaction.
The code sets up tools that can be executed locally or passed through the API, allowing the AI to perform specific tasks, such as retrieving data from external sources. The agent processes various message types from OpenAI, including audio responses, transcription updates, and error messages, and sends them to users through the Agora audio channel, facilitating continuous interaction.

The following figure illustrates the integration topology:

Expand All @@ -24,9 +24,18 @@ The following figure illustrates the integration topology:
sudo apt install ffmpeg
```

## Set up the project
## Getting Started

This guide walks you through the core elements of the [Agora Conversational AI Demo](https://github.com/AgoraIO/openai-realtime-python) integrating Agora's Python SDK with OpenAI's Realtime API.

This guide walks you through the core elements of the [Agora Conversational AI Demo](https://github.com/AgoraIO/agora-openai-converse) integrating Agora's Python SDK with OpenAI's Realtime API:
If you’d prefer to skip the step-by-step guide and explore the demo project, clone the repository and follow the steps in the README to get started.

```bash
git clone https://github.com/AgoraIO/openai-realtime-python.git
cd agora-openai-converse
```

## Set up the project

1. Create a new folder for the project:

Expand Down
Loading