Conversational AI

Audio-based conversational AI for interviews and free-form conversations. Real-time voice conversations via WebRTC audio.

Privacy-first design

CanaryLLM stores only template configuration and session metadata (duration, status, cost). No transcripts, participant data, or conversation content is stored. All content data is relayed to your own webhook URL.

How it works

Create a conversation template with system prompt, voice settings, and optional interview questions
CanaryLLM provisions the voice agent
Start a session to get a signed URL for WebRTC connection
Connect from the browser using the CanaryLLM client SDK
During the conversation, CanaryLLM webhooks relay data to your webhook URL

Templates

Create Template

bash

curl -X POST /api/convagents/templates \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "Customer Interview",
    "type": "interview",
    "systemPrompt": "You are a friendly interviewer conducting a customer feedback session.",
    "firstMessage": "Hello! Thank you for joining. I have a few questions for you.",
    "voiceId": "JBFqnCBsd6RMkjVDRZzb",
    "voiceSettings": { "stability": 0.3, "speed": 1.0, "similarityBoost": 0.8 },
    "language": "en",
    "maxDurationSeconds": 600,
    "tag": "my_project",
    "clientWebhookUrl": "https://your-app.com/webhook/conversation",
    "webhookSecret": "your-secret-key",
    "tools": [
      { "name": "end_call" },
      { "name": "language_detection", "description": "Switch language when user speaks another language" }
    ],
    "questions": [
      { "question": "How did you hear about our product?", "isRequired": true },
      { "question": "What feature do you use the most?", "context": "Focus on daily usage" },
      { "question": "Any suggestions for improvement?", "isRequired": false }
    ]
  }'

Parameter	Type	Required	Description
`name`	string	Yes	Template name
`type`	string	Yes	`"interview"` or `"conversation"`
`systemPrompt`	string	Yes	System prompt for the AI agent
`firstMessage`	string	No	Initial greeting message
`voiceId`	string	No	Voice ID (see Voice Browser)
`voiceModel`	string	No	Voice model
`voiceSettings`	object	No	TTS voice tuning: `stability` (0–1, lower = more expressive, renders audio tags like `[laughs]` stronger), `speed` (0.7–1.2), `similarityBoost` (0–1)
`language`	string	No	Language code (default: `en`)
`llmProvider`	string	No	AI provider (e.g. `anthropic`, `openai`). Required with `llmModel`
`llmModel`	string	No	AI model (e.g. `claude-sonnet-4-5`). Required with `llmProvider`
`tag`	string	No	Custom tag for grouping usage/billing (max 100 chars)
`clientWebhookUrl`	string	No	Your webhook URL for data relay. Must be a public http(s) URL; URLs that resolve to private or internal addresses are rejected
`webhookSecret`	string	No	Secret for webhook auth (sent as `X-Webhook-Secret`). Auto-generated and returned in the create response if omitted. Webhook calls without it get a 401, so store it
`maxDurationSeconds`	number	No	Max duration in seconds (default: 600)
`tools`	array	No	System tools the agent can use. Default: `[{name: "end_call"}]`. Available: `end_call`, `language_detection`, `transfer_to_agent`, `transfer_to_number`, `skip_turn`, `play_keypad_touch_tone`, `voicemail_detection`
`questions`	array	No	Interview questions (for type `"interview"`)

Other Template Endpoints

Method	Endpoint	Description
`GET`	`/api/convagents/templates`	List all templates
`GET`	`/api/convagents/templates/:id`	Get template details
`PUT`	`/api/convagents/templates/:id`	Update template (auto-syncs voice agent)
`DELETE`	`/api/convagents/templates/:id`	Delete template (removes voice agent)

Sessions

Start a Session

Create a session to get a signed URL for WebRTC connection. The signed URL expires after 15 minutes.

bash

curl -X POST /api/convagents/sessions \
  -H "Authorization: Bearer YOUR_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "templateId": 1 }'

Response

json

{
  "success": true,
  "data": {
    "session": {
      "id": 1,
      "templateId": 1,
      "status": "pending",
      "createdAt": "2026-02-11T12:00:00.000Z"
    },
    "signedUrl": "wss://...",
    "expiresIn": 900
  }
}

Other Session Endpoints

Method	Endpoint	Description
`GET`	`/api/convagents/sessions`	List sessions (metadata only)
`GET`	`/api/convagents/sessions/:id`	Get session details (no transcript)

Webhook Relay

During a conversation, CanaryLLM relays events to your clientWebhookUrl with the X-Webhook-Secret header.

Webhook authentication

Webhook endpoints require the X-Webhook-Secret header matching the template's webhookSecret. Requests without a valid secret receive a 401 response.

Webhook Payload

json

{
  "event": "event | context | complete",
  "sessionId": 1,
  "templateId": 1,
  "templateName": "Customer Interview",
  "timestamp": "2026-02-11T12:05:00.000Z",
  "data": {
    // Event-specific data
  }
}

Webhook	Endpoint	Description
`context`	`POST /api/convagents/webhook/context`	Agent requests context. Your response is forwarded back.
`event`	`POST /api/convagents/webhook/event`	Mid-conversation events (responses, sections completed)
`complete`	`POST /api/convagents/webhook/complete`	Conversation ended. Session status updated, transcript relayed.

Privacy & data handling

When a conversation ends, ElevenLabs sends the full transcript and post-call analysis to CanaryLLM via webhook. CanaryLLM relays that payload to the template's clientWebhookUrl and immediately discards it — no transcript or conversation content is written to CanaryLLM's database. Only session metadata (status, duration, cost) is persisted.

Authentication: every relay call includes the per-template X-Webhook-Secret header. Your endpoint must be HTTPS and must validate this header before processing the payload.
Voice processing: ElevenLabs (US, SCCs) handles the live voice conversation. See the Data Residency section in Providers & Models for the full transfer basis.

No retry, no recovery

The relay is fire-and-forget: there is no retry queue and no dead-letter store. If your webhook endpoint is unreachable when a conversation completes, the transcript is not delivered and cannot be retrieved from CanaryLLM — it was never stored here. Treat your webhook endpoint as the system of record and make it reliable.

Pricing

Conversational AI sessions are billed per minute of conversation. When a session completes, the cost is automatically calculated based on duration and added to the standard billing pipeline.

Component	Rate	Details
`Billing`	Per minute	Automatically calculated when session completes

Session costs appear in your usage data: GET /api/llm/usage and GET /api/portal/usage/daily.

Client-Side Integration

Use the @11labs/client SDK (or @11labs/react for React) to connect from the browser.

typescript

import { Conversation } from '@11labs/client';

// 1. Get signed URL from CanaryLLM
const res = await fetch('/api/convagents/sessions', {
  method: 'POST',
  headers: {
    'Authorization': 'Bearer YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({ templateId: 1 }),
});
const { data } = await res.json();

// 2. Start WebRTC conversation
const conversation = await Conversation.startSession({
  signedUrl: data.signedUrl,
  onMessage: (msg) => console.log('Agent:', msg),
  onDisconnect: () => console.log('Conversation ended'),
});

// 3. End conversation when done
await conversation.endSession();