feat: add twilio plugin implementation Closes #1631 #2139
Conversation
|
@Freytes current code doesn't need ElevenLabs or Deepgram services. |
If I can get a demo of this working I would recommend that you completed the bounty |
Hi, demo is here: https://www.boolkeys.com/eliza/plugin-twilio/DEMO-plugin-twilio.mov |
@boolkeys great demo! Jin will be in touch for bounty payment |
|
would like to mention that this plugin does not seem functional... i cant find the code for the actions? |
This comment was marked as resolved.
This comment was marked as resolved.
|
You can add a service and avoid the ngrok. Also the actions should be implemented since is a framework and people will use your tool as building block. |
This comment was marked as resolved.
This comment was marked as resolved.
boolkeys
force-pushed
the
feature/plugin-twilio
branch
from
ceedaf2 to
bf6c95c
Compare
|
@boolkeys is it probable to use a custom voice if support by Twilio? |
Yes, Twilio supports custom voices through Amazon Polly. I updated the code to support custom Polly voices. |
2 options for the webhook: ngrok for local (https needed by twilio) or custom domain name). |
|
@boolkeys I thought Twilio has an automatic ElevenLabs integration, I am just thinking of other options aside from Amazon |
boolkeys
changed the title
I will integrate elevenlabs as an option (Amazon does not require api key) |
Can we discuss? Please reach out to me on discord, I have reached out to you |
|
I have tested everything and reviewed your code. Everything seems to be in order, and it's ready to proceed to the main branch. |
great! did you enjoy the experience? |
Head branch was pushed to by a user without write access
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the You can disable this status message by setting the WalkthroughThe pull request introduces a comprehensive Twilio plugin for ElizaOS, enabling SMS and voice call capabilities. The implementation includes services for sending SMS, handling voice calls, text-to-speech conversion using ElevenLabs, and webhook management. The plugin supports initializing calls, generating responses, and managing conversation memory across different interaction types. Changes
Assessment against linked issues
The pull request substantially meets the core requirements for Twilio text and voice integration, with a robust implementation of SMS and voice call capabilities. The ElevenLabs integration for text-to-speech is well-implemented. However, the speech-to-text component and video demonstration are not explicitly addressed in the current PR. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 25
🧹 Nitpick comments (20)
packages/plugin-twilio/src/services/twilio.ts (2)
60-60: Cache environment variables to improve performance and consistencyAccessing
process.envmultiple times can lead to performance issues and potential inconsistencies if environment variables change at runtime. Cache the environment variables during initialization.Modify the constructor to cache
TWILIO_PHONE_NUMBER:private initialized: boolean = false; + private twilioPhoneNumber: string; constructor() { const accountSid = process.env.TWILIO_ACCOUNT_SID; const authToken = process.env.TWILIO_AUTH_TOKEN; + this.twilioPhoneNumber = process.env.TWILIO_PHONE_NUMBER || ''; if (!accountSid || !authToken || !this.twilioPhoneNumber) { throw new Error('Missing required Twilio credentials'); } // ... }Update
isInitialized()andsendSms()methods:public isInitialized(): boolean { - return this.initialized && !!process.env.TWILIO_PHONE_NUMBER; + return this.initialized && !!this.twilioPhoneNumber; } async sendSms(params: { to: string; body: string }) { if (!this.isInitialized()) { throw new Error('Twilio service not properly initialized'); } return this.client.messages.create({ to: params.to, - from: process.env.TWILIO_PHONE_NUMBER, + from: this.twilioPhoneNumber, body: params.body }); }Also applies to: 92-92, 98-98
104-104: Delay service initialization to avoid potential errorsExporting an instance of
TwilioServiceupon module import can cause issues if the environment variables are not set yet. Consider delaying the initialization until the service is actually needed.Change the export to:
- export const twilioService = TwilioService.getInstance(); + export const getTwilioService = () => TwilioService.getInstance();Update imports where
twilioServiceis used:- import { twilioService } from '../services/twilio.js'; + import { getTwilioService } from '../services/twilio.js'; - twilioService.sendSms(params); + getTwilioService().sendSms(params);packages/plugin-twilio/src/actions/call.ts (2)
6-6: Reuse theCALL_PATTERNregex to avoid duplicationThe regular expression
CALL_PATTERNis defined but not reused in thevalidateandhandlermethods. Use the definedCALL_PATTERNto ensure consistency and maintainability.Update the
validatemethod:validate: async (runtime, message) => { const text = message.content.text; - const phoneMatch = text.match(/(?:call|dial|phone|reach|contact) (\+\d{10,15}) (?:and|to)? (?:tell|say) (?:them|about|that)? (.*)/i); + const phoneMatch = text.match(CALL_PATTERN); return !!phoneMatch; },Ensure the
handlermethod also usesCALL_PATTERNconsistently.Also applies to: 19-19, 26-26
74-90: Improve error handling by checking error types instead of messagesRelying on error message content is fragile and can break if the messages change. Use specific error types or codes provided by the Twilio API.
Modify the error handling block to check for error properties:
if (error instanceof Error) { - const errorMessage = error.message.toLowerCase(); - if (errorMessage.includes('invalid') || errorMessage.includes('not a valid phone number')) { + if (error.code === 21211) { // Twilio error code for invalid phone number return { success: false, message: 'Invalid phone number format. Please use international format (e.g., +1234567890)' }; } - if (errorMessage.includes('permission')) { + else if (error.code === 20003) { // Twilio error code for permissions error return { success: false, message: "Sorry, I don't have permission to call this number. It might need to be verified first." }; } }packages/plugin-twilio/src/services/elevenlabs.ts (3)
76-80: Handle missing API key appropriately in initializationCurrently, if the
ELEVENLABS_XI_API_KEYenvironment variable is missing, the service logs a warning but doesn't update theinitializedstatus. Explicitly setthis.initializedtofalseto reflect the service state accurately.Update the
initialize()method:if (!apiKey) { SafeLogger.warn('ELEVENLABS_XI_API_KEY not set - ElevenLabs features will be disabled'); + this.initialized = false; return; }
164-167: Avoid checking environment variables during runtimeFrequently accessing
process.envcan lead to performance issues and inconsistency. Consider caching the API key during initialization and monitoring changes through configuration management.Remove the runtime check for
process.env.ELEVENLABS_XI_API_KEY:- const currentApiKey = process.env.ELEVENLABS_XI_API_KEY; - if (currentApiKey && currentApiKey !== this.apiKey && !this.isReinitializing) { + if (!this.apiKey && !this.isReinitializing) { await this.reinitialize(); }Ensure the API key is cached during initialization:
- this.apiKey = null; + this.apiKey = apiKey;
241-243: Avoid logging sensitive error informationLogging the entire error object might expose sensitive information. Log only necessary details to prevent potential security risks.
Modify the error logging:
- SafeLogger.error('❌ Audio generation failed:', error); + SafeLogger.error('❌ Audio generation failed:', error.message);packages/plugin-twilio/src/services/webhook.ts (2)
233-271: Utilize 'findAvailablePort' or remove unused methodThe
findAvailablePortmethod is defined but never used, while the port is hardcoded in theinitializemethod. Either modifyinitializeto usefindAvailablePortfor dynamic port selection or remove the unused method to clean up the code.Also applies to: 291-298
529-532: Use a reliable method to determine the project rootUsing
process.cwd().replace(/\/agent$/, '')may not accurately resolve the project root in all environments. Consider using a module likeapp-root-pathorimport.meta.urlfor a more robust solution.packages/plugin-twilio/src/actions/index.ts (1)
5-6: Simplify the comment or remove itThe comment on line 5 states, "Export as an array of actions, not an object," which may be unnecessary as the code is self-explanatory. Consider simplifying or removing the comment to improve code readability.
packages/plugin-twilio/src/types/actions.ts (2)
1-5: Enhance error handling in ActionResult interface.Consider adding
error?: Error | stringto capture detailed error information whensuccessis false.export interface ActionResult { success: boolean; callSid?: string; message?: string; + error?: Error | string; }
7-14: Add validation constraints to phone number parameters.Consider adding regex pattern validation for phone numbers using string literal types or documentation.
export interface CallVoiceParams { - phoneNumber: string; + phoneNumber: `+${number}`; // E.164 format message: string; } export interface VoiceConversationParams { - phoneNumber: string; + phoneNumber: `+${number}`; // E.164 format }packages/plugin-twilio/vitest.config.ts (1)
9-18: Add coverage thresholds.Set minimum coverage thresholds to maintain code quality standards.
coverage: { provider: 'v8', reporter: ['text', 'json', 'html'], + thresholds: { + statements: 80, + branches: 80, + functions: 80, + lines: 80 + }, exclude: [ 'node_modules/**', 'dist/**', '**/*.d.ts', 'src/tests/**' ] }packages/plugin-twilio/src/types/voice.ts (1)
6-10: Consider narrowing timestamp type for better type safetyThe
timestampfield could benefit from a more specific type likeISO8601StringorDate.- timestamp: string; + timestamp: ISO8601String; // Add type: type ISO8601String = string;vitest.config.ts (1)
6-21: Add timeout and memory limits to test configurationPrevent hanging tests and memory leaks by adding appropriate limits.
test: { globals: true, environment: 'node', + testTimeout: 10000, + poolOptions: { + threads: { + singleThread: true + } + }, + maxConcurrency: 1, include: ['**/*.test.ts', '**/*.spec.ts'],packages/plugin-twilio/scripts/build.ts (1)
18-38: Optimize build configuration.Add build optimizations for production use.
await build({ entryPoints, outdir: 'dist', platform: 'node', format: 'esm', target: 'node18', bundle: true, sourcemap: true, + minify: process.env.NODE_ENV === 'production', + treeShaking: true, external: [ '@elizaos/core', 'twilio', 'express', 'uuid' ], logLevel: 'info', mainFields: ['module', 'main'], + metafile: true, banner: { js: '// @ts-check\n' }, outExtension: { '.js': '.js' } }); + +// Analyze bundle if in development +if (process.env.NODE_ENV !== 'production') { + const analysis = await analyzeMetafile((await build).metafile); + console.log('📊 Bundle analysis:', analysis); +}packages/plugin-twilio/src/actions/sms.ts (1)
45-64: Enhance error handling with rate limit detection.Add specific handling for Twilio rate limit errors to provide better user feedback.
if (error instanceof Error) { const errorMessage = error.message.toLowerCase(); + if (errorMessage.includes('rate limit')) { + return { + success: false, + message: 'Too many messages sent. Please try again in a few minutes.' + }; + } if (errorMessage.includes('invalid') || errorMessage.includes('not a valid phone number')) {packages/plugin-twilio/src/utils/logger.ts (1)
42-46: Secure debug mode configuration.Consider using a more specific environment variable name to avoid conflicts.
- if (this.debugMode || process.env.DEBUG) { + if (this.debugMode || process.env.TWILIO_PLUGIN_DEBUG) {packages/plugin-twilio/README.md (2)
68-71: Add language specifiers to code blocksAdd language specifiers to enable proper syntax highlighting:
-``` +```bashAlso applies to: 75-78
🧰 Tools
🪛 Markdownlint (0.37.0)
68-68: null
Fenced code blocks should have a language specified(MD040, fenced-code-language)
51-59: Document voice configuration parametersAdd descriptions and valid ranges for:
- stability
- similarityBoost
- style
- useSpeakerBoost
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (36)
.gitignore(2 hunks)characters/c3po.character.json(0 hunks)characters/trump.character.json(0 hunks)package.json(1 hunks)packages/core/tsconfig.json(2 hunks)packages/core/types/index.d.ts(1 hunks)packages/plugin-twilio/.env.example(1 hunks)packages/plugin-twilio/.gitignore(1 hunks)packages/plugin-twilio/README.md(1 hunks)packages/plugin-twilio/package.json(1 hunks)packages/plugin-twilio/scripts/build.ts(1 hunks)packages/plugin-twilio/scripts/tsconfig.json(1 hunks)packages/plugin-twilio/src/actions/call.ts(1 hunks)packages/plugin-twilio/src/actions/index.ts(1 hunks)packages/plugin-twilio/src/actions/sms.ts(1 hunks)packages/plugin-twilio/src/index.ts(1 hunks)packages/plugin-twilio/src/plugin.ts(1 hunks)packages/plugin-twilio/src/routes/voice.ts(1 hunks)packages/plugin-twilio/src/services/elevenlabs.ts(1 hunks)packages/plugin-twilio/src/services/sms/handler.ts(1 hunks)packages/plugin-twilio/src/services/twilio.ts(1 hunks)packages/plugin-twilio/src/services/voice/handler.ts(1 hunks)packages/plugin-twilio/src/services/voice/memory.ts(1 hunks)packages/plugin-twilio/src/services/voice/tts.ts(1 hunks)packages/plugin-twilio/src/services/webhook.ts(1 hunks)packages/plugin-twilio/src/types/actions.ts(1 hunks)packages/plugin-twilio/src/types/service.ts(1 hunks)packages/plugin-twilio/src/types/voice.ts(1 hunks)packages/plugin-twilio/src/utils/audioHandler.ts(1 hunks)packages/plugin-twilio/src/utils/logger.ts(1 hunks)packages/plugin-twilio/src/utils/voiceSettingsParser.ts(1 hunks)packages/plugin-twilio/tsconfig.json(1 hunks)packages/plugin-twilio/vitest.config.ts(1 hunks)src/actions/index.ts(1 hunks)tsconfig.json(1 hunks)vitest.config.ts(1 hunks)
💤 Files with no reviewable changes (2)
- characters/c3po.character.json
- characters/trump.character.json
✅ Files skipped from review due to trivial changes (8)
- tsconfig.json
- packages/plugin-twilio/.gitignore
- packages/core/tsconfig.json
- src/actions/index.ts
- packages/plugin-twilio/scripts/tsconfig.json
- packages/plugin-twilio/tsconfig.json
- packages/plugin-twilio/package.json
- packages/plugin-twilio/.env.example
🧰 Additional context used
🪛 Markdownlint (0.37.0)
packages/plugin-twilio/README.md
68-68: null
Fenced code blocks should have a language specified
(MD040, fenced-code-language)
75-75: null
Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔇 Additional comments (4)
packages/plugin-twilio/src/index.ts (1)
1-18: Well-organized exports structure!Clean barrel file implementation with logical grouping of exports.
packages/plugin-twilio/src/types/voice.ts (1)
24-26: Add input validation for text-to-speech parametersThe
textToSpeechmethod should validate text length to prevent excessive API usage and potential DOS attacks..gitignore (1)
5-5: LGTM!The additions to .gitignore are appropriate and follow security best practices by excluding sensitive files.
Also applies to: 68-70
package.json (1)
67-68: Dependencies look good!The workspace dependency is correctly configured for the Twilio plugin.
boolkeys
force-pushed
the
feature/plugin-twilio
branch
from
e92e732 to
90b42bf
Compare
The pull request substantially meets the core requirements for Twilio text and voice integration, with a robust implementation of SMS and voice call capabilities. The ElevenLabs integration for text-to-speech is well-implemented. However, the speech-to-text component and video demonstration are not explicitly addressed in the current PR. This plugin uses Twilio's built-in speech recognition capabilities for voice interactions. Demo here: https://www.boolkeys.com/eliza/plugin-twilio/DEMO-plugin-twilio.mov |
boolkeys
force-pushed
the
feature/plugin-twilio
branch
from
339bad9 to
9e54736
Compare
boolkeys
force-pushed
the
feature/plugin-twilio
branch
from
9e54736 to
4240c65
Compare
feat: add twilio plugin with ElevenLabs voice integration
Relates to & Closes #1631
Risks
Low - This plugin:
Background
What does this PR do?
smsandcallWhat kind of change is this?
Feature (non-breaking change which adds functionality)
Documentation
Added comprehensive documentation including:
Testing
Where should a reviewer start?
src/actions/- SMS and call actionssrc/services/voice/handler.ts- Voice call handlingsrc/services/webhook.ts- Webhook implementationTesting steps
TWILIO_ACCOUNT_SID=your_sid
TWILIO_AUTH_TOKEN=your_token
TWILIO_PHONE_NUMBER=your_number
TWILIO_CHARACTER=aigyver.character.json
WEBHOOK_PORT=3004
WEBHOOK_BASE_URL=your_webhook_url
ELEVENLABS_XI_API_KEY=your_key
Send SMS to +1234567890 saying Hello world!
Call +1234567890 and tell them about AI technology
Or with custom Polly voice:
{ "settings": { "voice": { "custom": "Polly.Aria-Neural" } } }Video demo
https://www.boolkeys.com/eliza/plugin-twilio/DEMO-plugin-twilio.mov
Deploy Notes
Key Features
Discord: @boolkeys
Summary by CodeRabbit
Release Notes: Twilio Plugin Update
New Features
Improvements
Configuration Updates
Removed