Email Tool Integration for ElevenLabs Agents

Overview

The Email Tool Integration allows ElevenLabs AI agents to search, read, and draft emails during conversations with users. This feature is Enterprise plan only and uses IMAP/SMTP for broad email provider compatibility.

Related GitHub Issue: #376 - Implement Real-time Agent Tool for Email Integration

Future Enhancements:

• Gmail OAuth2 integration (#379)
• Microsoft 365/Outlook OAuth2 integration (#380)

Features

1. Search Emails: Search across inbox, sent, and archived emails
2. Read Emails: Retrieve full email content by ID
3. Create Drafts: Create new email drafts for user review
4. Create Reply Drafts: Draft replies to existing emails

Architecture

Database Tables

• webhook_tools - Tool registration (tool_name: ‘email’)
• email_credentials - Stores IMAP/SMTP credentials per user/agent (encrypted)
• email_search_cache - Optional caching for search results
• webhook_logs - Tracks all email tool invocations

API Endpoints

Credentials Management

• POST /api/email-credentials - Save IMAP/SMTP credentials (requires auth token)
• GET /api/email-credentials/status - Check connection status (requires auth token)
• POST /api/email-credentials/test - Test IMAP connection (requires auth token)
• DELETE /api/email-credentials - Delete credentials (requires auth token)

Email Operations

• POST /api/email-tool/webhook - Main webhook endpoint called by ElevenLabs agents

Setup Instructions

1. Install Required Packages

The following packages are required and should already be installed:

npm install imap mailparser nodemailer

2. Database Migration

Apply both migrations:

# Initial schema
sudo mysql -D triluna_production < /var/www/triluna.app/webapp/server/migrations/2025-10-11_email-tool-oauth-storage.sql

# IMAP/SMTP update
sudo mysql -D triluna_production < /var/www/triluna.app/webapp/server/migrations/2025-10-11_update-email-credentials-for-imap.sql

3. Server Restart

After installing packages, restart the server:

cd /var/www/triluna.app/webapp/server
pm2 restart rusuai-server

Usage

For End Users (Enterprise Plan Only)

1. Configure IMAP/SMTP Credentials:

   • Navigate to Dashboard → Agent Settings
   • Enter email server settings:
     • IMAP Server: mail.example.com
     • IMAP Port: 993 (SSL/TLS) or 143 (STARTTLS)
     • SMTP Server: mail.example.com
     • SMTP Port: 587 (STARTTLS) or 25 (plain)
     • Email Address: user@example.com
     • Username: Usually same as email address
     • Password: Email account password
   • Click “Test Connection” to verify settings
   • Save credentials

2. Configure Agent to Use Email Tool:

   • In ElevenLabs dashboard, edit agent configuration
   • Add email tool to agent’s available tools
   • The tool will be available during conversations

Example: TriLuna Mail Server Configuration

For testing with triluna.app mail server:

IMAP Server: mail.triluna.app
IMAP Port: 993 (SSL/TLS)
SMTP Server: mail.triluna.app
SMTP Port: 587 (STARTTLS)
Username: your-username@triluna.app
Password: (your mail password)

For Agents (Tool Parameters)

The email tool accepts these parameters:

Search Emails

{
  "action": "search_emails",
  "search_query": "from:john@example.com subject:meeting"
}

Returns up to 5 matching emails with preview.

Read Email

{
  "action": "read_email",
  "email_id": "18f1234567890abc"
}

Returns full email content.

Create Draft

{
  "action": "create_draft",
  "draft_to": "recipient@example.com",
  "draft_subject": "Meeting Follow-up",
  "draft_body": "Hi,\n\nThanks for the meeting today..."
}

Creates a draft in user’s Gmail drafts folder.

Create Reply Draft

{
  "action": "create_reply_draft",
  "reply_to_id": "18f1234567890abc",
  "draft_body": "Thanks for your email. I'll get back to you soon."
}

Creates a reply draft to an existing email.

Security & Compliance

Credential Storage

• Passwords are stored in email_credentials table
• Credentials are encrypted at rest (consider adding encryption layer)
• Connection status tracked (success/failed)
• Credentials are user-specific or agent-specific

Enterprise-Only Enforcement

• Middleware checks user’s subscription plan
• Returns 403 error for non-Enterprise users
• Error message includes upgrade link

Audit Logging

• All credential saves/deletions are logged in audit_log
• All email tool invocations are logged in webhook_logs
• Includes timestamps, user IDs, and actions performed
• Connection test results tracked

IMAP/SMTP Protocols

Supported IMAP Features

• Search emails using IMAP SEARCH command
• Fetch email headers and body
• Save drafts to Drafts folder
• Support for STARTTLS and SSL/TLS connections

Supported SMTP Features

• Send email drafts (future enhancement)
• STARTTLS and SSL/TLS support
• Authentication via username/password

Error Handling

Common Errors

“Email integration is only available for Enterprise plan users”

• User needs to upgrade to Enterprise plan
• Provide link to pricing page

“Email account not configured”

• User needs to enter IMAP/SMTP settings
• Provide link to settings page

“IMAP connection test failed”

• Verify server address and port are correct
• Check username and password
• Ensure firewall allows IMAP/SMTP connections
• Try different ports (993 vs 143 for IMAP, 587 vs 25 for SMTP)

“Agent not found”

• Invalid agent_id provided in request
• Check agent exists in database

Testing

Test Credentials Setup

1. Use an Enterprise user account
2. Call POST /api/email-credentials with credentials:

{
  "email_address": "test@triluna.app",
  "email_username": "test@triluna.app",
  "email_password": "password",
  "imap_host": "mail.triluna.app",
  "imap_port": 993,
  "imap_secure": true,
  "smtp_host": "mail.triluna.app",
  "smtp_port": 587,
  "smtp_secure": false
}

3. Verify connection test passes
4. Check email_credentials table for stored credentials

Test Email Tool Webhook

curl -X POST https://api.triluna.app/api/email-tool/webhook \
  -H "Content-Type: application/json" \
  -d '{
    "agent_id": "your-agent-id",
    "conversation_id": "test-conv-123",
    "action": "search_emails",
    "search_query": "from:me"
  }'

Monitoring

Webhook Logs

Query webhook execution logs:

SELECT * FROM webhook_logs
WHERE tool_id = (SELECT id FROM webhook_tools WHERE tool_name = 'email')
ORDER BY created_at DESC
LIMIT 20;

Credentials Status

Check credentials and connection status:

SELECT user_id, email_address, connection_status,
  last_connection_test, last_used_at, connection_error
FROM email_credentials
WHERE email_provider = 'imap'
ORDER BY last_used_at DESC;

Future Enhancements

• Gmail OAuth2 integration (Issue #379) - Better security, no password storage
• Microsoft 365/Outlook OAuth2 integration (Issue #380)
• Email sending (not just drafts)
• Attachment handling (upload/download)
• Advanced search filters (date ranges, flags, folders)
• Email categorization and AI summarization
• Integration with CRM systems
• Encryption layer for stored passwords

Support

For issues or questions:

• GitHub Issues: https://github.com/edmiidz/RusuAI/issues
• Documentation: /var/www/triluna.app/docs/

---

Last Updated: 2025-10-11 Status: Pilot (Enterprise users only)