anthropics--knowledge-work-plugins/partner-built/zoom-plugin/skills/rtms/examples/rtms-bot.md

RTMS Bot (Real-Time Media Streams)

Build resilient RTMS bots that access meeting audio, video, transcription, screen share, and chat without joining as a visible participant.

Overview

RTMS bots are invisible read-only services that subscribe to meeting media streams via WebSockets. They do NOT appear in the participant list.

Use this approach when:

• You only need to observe/transcribe (no interaction needed)
• You want invisible operation
• You're processing external meetings (with permission)
• You want minimal resource usage

Alternative: See Meeting SDK Bot (Linux) for visible participant bots with full meeting control.

Architecture

┌─────────────────────────────────────────────────────────────────────┐
│                        RTMS BOT FLOW                                 │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│ 1. Trigger RTMS: REST API or In-Meeting Start                       │
│    └── POST /meetings/{meetingId}/rtms                              │
│    └── Or: Start RTMS manually from Zoom client                     │
└────────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│ 2. Wait for Webhook: meeting.rtms_started                           │
│    └── Zoom sends signaling + media WebSocket URLs                  │
│    └── No webhook = RTMS unavailable (no polling fallback)          │
└────────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│ 3. Connect: Signaling WebSocket (Handshake with HMAC)               │
│    └── Generate HMAC-SHA256 signature                               │
│    └── Send handshake message                                       │
│    └── Receive session confirmation                                 │
└────────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│ 4. Connect: Media WebSocket (Subscribe to Streams)                  │
│    └── Subscribe to: audio, video, transcription, share, chat       │
│    └── Send keep-alive pings                                        │
└────────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│ 5. Process Media Data                                               │
│    └── Audio: Opus/PCM streams per speaker                          │
│    └── Video: H.264 encoded frames                                  │
│    └── Transcription: Real-time text with speaker labels            │
└────────────────────────────┬────────────────────────────────────────┘
                             │
                             ▼
┌─────────────────────────────────────────────────────────────────────┐
│ 6. Mid-Stream: Connection Monitoring                                │
│    └── Detect WebSocket close → Exponential backoff retry           │
│    └── Stop after N reconnection attempts                           │
└─────────────────────────────────────────────────────────────────────┘

Skills Required

Skill	Purpose
zoom-rest-api	Trigger RTMS start (optional - can also start manually)
rtms	WebSocket connection, media processing
webhooks	Receive meeting.rtms_started event

Prerequisites

• Zoom app with RTMS features enabled
• Webhook endpoint (HTTPS, publicly accessible)
• Event subscriptions: meeting.rtms_started, meeting.rtms_stopped
• Scopes: meeting:read:admin, meeting:write:admin (if triggering via API)
• RTMS SDK or native WebSocket implementation

Configuration

Retry Parameters (Customizable)

// config.js or environment variables
const rtmsConfig = {
    // WebSocket connection (initial)
    connection_timeout_ms: 10000,        // Handshake timeout (default: 10s)
    connection_max_attempts: 5,          // Max connection attempts (default: 5)
    connection_retry_delay_ms: 5000,     // Constant retry: 5s (default: 5s)
    
    // Mid-stream reconnection (network failures)
    reconnect_max_attempts: 3,           // Max reconnection attempts (default: 3)
    reconnect_base_delay_ms: 2000,       // Initial delay: 2s (default: 2s)
    // Exponential backoff: 2s, 4s, 8s...
    
    // Keep-alive ping
    keepalive_interval_ms: 5000,         // Send ping every 5s (default: 5s, min: 3s)
    keepalive_timeout_ms: 15000,         // Expect pong within 15s (default: 15s)
    
    // Webhook wait timeout
    webhook_wait_timeout_ms: 300000      // Wait 5min for webhook (default: 5min)
};

// Load from environment variables (recommended for production)
function loadConfig() {
    return {
        connection_timeout_ms: 
            parseInt(process.env.RTMS_CONNECTION_TIMEOUT_MS) || 10000,
        connection_max_attempts: 
            parseInt(process.env.RTMS_CONNECTION_MAX_ATTEMPTS) || 5,
        connection_retry_delay_ms: 
            parseInt(process.env.RTMS_CONNECTION_RETRY_DELAY_MS) || 5000,
        reconnect_max_attempts: 
            parseInt(process.env.RTMS_RECONNECT_MAX_ATTEMPTS) || 3,
        reconnect_base_delay_ms: 
            parseInt(process.env.RTMS_RECONNECT_BASE_DELAY_MS) || 2000,
        keepalive_interval_ms: 
            Math.max(parseInt(process.env.RTMS_KEEPALIVE_INTERVAL_MS) || 5000, 3000),
        keepalive_timeout_ms: 
            parseInt(process.env.RTMS_KEEPALIVE_TIMEOUT_MS) || 15000,
        webhook_wait_timeout_ms: 
            parseInt(process.env.RTMS_WEBHOOK_WAIT_TIMEOUT_MS) || 300000
    };
}

Customization Guide

Parameter	Default	When to Increase	When to Decrease
connection_max_attempts	5	Slow/congested networks	Fast failure detection needed
connection_retry_delay_ms	5000 (5s)	High network latency	Local network, low latency
reconnect_max_attempts	3	Critical meetings, unstable network	Cost-sensitive, batch processing
reconnect_base_delay_ms	2000 (2s)	International connections	Local network
keepalive_interval_ms	5000 (5s)	Aggressive connection monitoring	Reduce bandwidth overhead
webhook_wait_timeout_ms	300000 (5min)	Meetings may start late	Fast failure detection

Recommended Ranges:

• Connection attempts: 3-10
• Connection retry delay: 2s-15s
• Reconnect attempts: 2-5
• Reconnect base delay: 1s-5s
• Keep-alive interval: 3s-30s (min: 3s per Zoom docs)

Examples:

# High-priority production bot (aggressive)
export RTMS_CONNECTION_MAX_ATTEMPTS=10
export RTMS_CONNECTION_RETRY_DELAY_MS=3000      # 3s
export RTMS_RECONNECT_MAX_ATTEMPTS=5
export RTMS_RECONNECT_BASE_DELAY_MS=1000        # 1s
export RTMS_KEEPALIVE_INTERVAL_MS=3000          # 3s (minimum)

# Cost-sensitive batch processing (conservative)
export RTMS_CONNECTION_MAX_ATTEMPTS=3
export RTMS_CONNECTION_RETRY_DELAY_MS=10000     # 10s
export RTMS_RECONNECT_MAX_ATTEMPTS=2
export RTMS_RECONNECT_BASE_DELAY_MS=5000        # 5s
export RTMS_KEEPALIVE_INTERVAL_MS=15000         # 15s

# Development/testing (fail fast)
export RTMS_CONNECTION_MAX_ATTEMPTS=2
export RTMS_CONNECTION_RETRY_DELAY_MS=2000      # 2s
export RTMS_RECONNECT_MAX_ATTEMPTS=1
export RTMS_RECONNECT_BASE_DELAY_MS=1000        # 1s
export RTMS_WEBHOOK_WAIT_TIMEOUT_MS=60000       # 1min

Step 1: Trigger RTMS (Optional - REST API)

You can start RTMS programmatically or manually from the Zoom client.

Option A: REST API Trigger

# Start RTMS for a meeting
curl -X POST "https://api.zoom.us/v2/meetings/{meetingId}/rtms" \
  -H "Authorization: Bearer YOUR_ACCESS_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "meeting"
  }'

Response:

{
  "rtms_id": "abc123def456",
  "status": "starting"
}

Option B: Manual Start (In-Meeting)

Host clicks Apps → Your RTMS App → Start RTMS

Both trigger the same webhook → meeting.rtms_started

Step 2: Wait for Webhook (Required)

CRITICAL: RTMS requires a webhook. There is NO polling alternative. If webhook doesn't arrive, RTMS is unavailable.

Webhook Handler

const express = require('express');
const crypto = require('crypto');
const app = express();

const config = loadConfig();
const pendingConnections = new Map();  // Track webhook waiters

app.post('/webhook', express.json(), async (req, res) => {
    // 1. Verify webhook signature
    const signature = req.headers['x-zm-signature'];
    const timestamp = req.headers['x-zm-request-timestamp'];
    
    if (!verifyWebhookSignature(req.body, signature, timestamp)) {
        return res.status(403).send('Invalid signature');
    }
    
    // 2. Respond immediately (Zoom expects 200 within 3s)
    res.status(200).send();
    
    // 3. Process webhook asynchronously
    const event = req.body;
    
    if (event.event === 'meeting.rtms_started') {
        console.log('[WEBHOOK] RTMS started for meeting:', event.payload.object.uuid);
        
        const rtmsInfo = {
            meetingUuid: event.payload.object.uuid,
            signalingUrl: event.payload.object.signaling_url,
            mediaUrl: event.payload.object.media_url,
            sessionKey: event.payload.object.session_key
        };
        
        // Notify waiting connection
        const waiter = pendingConnections.get(rtmsInfo.meetingUuid);
        if (waiter) {
            waiter.resolve(rtmsInfo);
            pendingConnections.delete(rtmsInfo.meetingUuid);
        } else {
            // No waiter - proactive start
            connectToRTMS(rtmsInfo);
        }
    }
});

function verifyWebhookSignature(body, signature, timestamp) {
    const message = `v0:${timestamp}:${JSON.stringify(body)}`;
    const hmac = crypto.createHmac('sha256', WEBHOOK_SECRET_TOKEN);
    const computed = 'v0=' + hmac.update(message).digest('hex');
    return crypto.timingSafeEqual(
        Buffer.from(signature),
        Buffer.from(computed)
    );
}

Wait for Webhook (with Timeout)

async function waitForRTMSWebhook(meetingUuid) {
    return new Promise((resolve, reject) => {
        const timeoutId = setTimeout(() => {
            pendingConnections.delete(meetingUuid);
            reject(new Error(
                `RTMS webhook not received within ${config.webhook_wait_timeout_ms}ms. ` +
                `Possible causes: ` +
                `(1) Meeting hasn't started, ` +
                `(2) RTMS not enabled for this meeting, ` +
                `(3) Webhook endpoint unreachable.`
            ));
        }, config.webhook_wait_timeout_ms);
        
        pendingConnections.set(meetingUuid, {
            resolve: (rtmsInfo) => {
                clearTimeout(timeoutId);
                resolve(rtmsInfo);
            },
            reject
        });
    });
}

// Usage
try {
    console.log('[RTMS] Waiting for webhook...');
    const rtmsInfo = await waitForRTMSWebhook(MEETING_UUID);
    console.log('[RTMS] Webhook received, connecting...');
    await connectToRTMS(rtmsInfo);
} catch (error) {
    console.error('[RTMS] ABORT:', error.message);
}

Error if no webhook: ABORT. No webhook = RTMS unavailable. No polling alternative.

Step 3: Connect to Signaling WebSocket

Connection with Retry

const WebSocket = require('ws');

async function connectSignalingWithRetry(signalingUrl, sessionKey) {
    for (let attempt = 1; attempt <= config.connection_max_attempts; attempt++) {
        console.log(`[SIGNALING] Attempt ${attempt}/${config.connection_max_attempts}`);
        
        try {
            const ws = await connectSignalingSocket(signalingUrl, sessionKey);
            console.log('[SIGNALING] Connected successfully');
            return ws;
        } catch (error) {
            console.error(`[SIGNALING] Attempt ${attempt} failed:`, error.message);
            
            if (attempt < config.connection_max_attempts) {
                const delay = config.connection_retry_delay_ms;
                console.log(`[SIGNALING] Retrying in ${delay}ms...`);
                await sleep(delay);
            }
        }
    }
    
    throw new Error(
        `Failed to connect signaling WebSocket after ${config.connection_max_attempts} attempts`
    );
}

function connectSignalingSocket(signalingUrl, sessionKey) {
    return new Promise((resolve, reject) => {
        const ws = new WebSocket(signalingUrl);
        const timeoutId = setTimeout(() => {
            ws.close();
            reject(new Error('Signaling connection timeout'));
        }, config.connection_timeout_ms);
        
        ws.on('open', () => {
            console.log('[SIGNALING] WebSocket opened, sending handshake...');
            
            // Generate HMAC signature
            const timestamp = Date.now();
            const message = `${timestamp}:${sessionKey}`;
            const signature = crypto
                .createHmac('sha256', WEBHOOK_SECRET_TOKEN)
                .update(message)
                .digest('hex');
            
            // Send handshake
            ws.send(JSON.stringify({
                type: 'handshake',
                timestamp,
                signature
            }));
        });
        
        ws.on('message', (data) => {
            const msg = JSON.parse(data);
            
            if (msg.type === 'handshake_response') {
                clearTimeout(timeoutId);
                
                if (msg.status === 'success') {
                    console.log('[SIGNALING] Handshake successful');
                    resolve(ws);
                } else {
                    ws.close();
                    reject(new Error(`Handshake failed: ${msg.error}`));
                }
            }
        });
        
        ws.on('error', (error) => {
            clearTimeout(timeoutId);
            reject(error);
        });
        
        ws.on('close', (code, reason) => {
            clearTimeout(timeoutId);
            reject(new Error(`Connection closed: ${code} ${reason}`));
        });
    });
}

Step 4: Connect to Media WebSocket

async function connectMediaWithRetry(mediaUrl, signalingWs) {
    for (let attempt = 1; attempt <= config.connection_max_attempts; attempt++) {
        console.log(`[MEDIA] Attempt ${attempt}/${config.connection_max_attempts}`);
        
        try {
            const ws = await connectMediaSocket(mediaUrl);
            console.log('[MEDIA] Connected successfully');
            subscribeToStreams(ws);
            setupKeepAlive(ws);
            return ws;
        } catch (error) {
            console.error(`[MEDIA] Attempt ${attempt} failed:`, error.message);
            
            if (attempt < config.connection_max_attempts) {
                const delay = config.connection_retry_delay_ms;
                console.log(`[MEDIA] Retrying in ${delay}ms...`);
                await sleep(delay);
            }
        }
    }
    
    throw new Error(
        `Failed to connect media WebSocket after ${config.connection_max_attempts} attempts`
    );
}

function subscribeToStreams(mediaWs) {
    // Subscribe to all available streams
    mediaWs.send(JSON.stringify({
        type: 'subscribe',
        streams: ['audio', 'video', 'transcription', 'share', 'chat']
    }));
    
    console.log('[MEDIA] Subscribed to: audio, video, transcription, share, chat');
}

Step 5: Keep-Alive Management

function setupKeepAlive(ws) {
    let lastPongReceived = Date.now();
    let keepAliveInterval;
    let timeoutCheck;
    
    // Send ping periodically
    keepAliveInterval = setInterval(() => {
        if (ws.readyState === WebSocket.OPEN) {
            ws.ping();
            console.log('[KEEPALIVE] Ping sent');
        }
    }, config.keepalive_interval_ms);
    
    // Check for pong timeout
    timeoutCheck = setInterval(() => {
        const timeSinceLastPong = Date.now() - lastPongReceived;
        
        if (timeSinceLastPong > config.keepalive_timeout_ms) {
            console.error('[KEEPALIVE] Pong timeout, closing connection');
            clearInterval(keepAliveInterval);
            clearInterval(timeoutCheck);
            ws.close(1000, 'Keep-alive timeout');
        }
    }, 1000);
    
    ws.on('pong', () => {
        lastPongReceived = Date.now();
        console.log('[KEEPALIVE] Pong received');
    });
    
    ws.on('close', () => {
        clearInterval(keepAliveInterval);
        clearInterval(timeoutCheck);
    });
}

Step 6: Mid-Stream Reconnection

class ResilientRTMSConnection {
    constructor(rtmsInfo, config) {
        this.rtmsInfo = rtmsInfo;
        this.config = config;
        this.reconnectionAttempt = 0;
        this.signalingWs = null;
        this.mediaWs = null;
    }
    
    async connect() {
        try {
            this.signalingWs = await connectSignalingWithRetry(
                this.rtmsInfo.signalingUrl,
                this.rtmsInfo.sessionKey
            );
            
            this.mediaWs = await connectMediaWithRetry(
                this.rtmsInfo.mediaUrl,
                this.signalingWs
            );
            
            this.setupReconnectionHandlers();
            
        } catch (error) {
            console.error('[RTMS] Initial connection failed:', error);
            throw error;
        }
    }
    
    setupReconnectionHandlers() {
        const handleDisconnection = (wsType) => async (code, reason) => {
            console.error(`[${wsType}] Disconnected: ${code} ${reason}`);
            
            this.reconnectionAttempt++;
            
            if (this.reconnectionAttempt > this.config.reconnect_max_attempts) {
                console.error(
                    `[RECONNECT] Giving up after ${this.reconnectionAttempt} attempts`
                );
                this.cleanup();
                return;
            }
            
            // Exponential backoff: 2s, 4s, 8s...
            const delay = this.config.reconnect_base_delay_ms 
                          * Math.pow(2, this.reconnectionAttempt - 1);
            
            console.log(
                `[RECONNECT] Attempt ${this.reconnectionAttempt}/` +
                `${this.config.reconnect_max_attempts} in ${delay}ms...`
            );
            
            await sleep(delay);
            
            try {
                await this.connect();
                console.log('[RECONNECT] Successfully reconnected');
                this.reconnectionAttempt = 0;  // Reset counter
            } catch (error) {
                console.error('[RECONNECT] Failed:', error.message);
                // Handler will be called again if connection fails
            }
        };
        
        this.signalingWs.on('close', handleDisconnection('SIGNALING'));
        this.mediaWs.on('close', handleDisconnection('MEDIA'));
        
        this.signalingWs.on('error', (error) => {
            console.error('[SIGNALING] Error:', error.message);
        });
        
        this.mediaWs.on('error', (error) => {
            console.error('[MEDIA] Error:', error.message);
        });
    }
    
    cleanup() {
        if (this.signalingWs) this.signalingWs.close();
        if (this.mediaWs) this.mediaWs.close();
    }
}

Customizing Reconnection Behavior

// Example: Capped exponential backoff (max 30s)
const delay = Math.min(
    config.reconnect_base_delay_ms * Math.pow(2, reconnectionAttempt - 1),
    30000  // Cap at 30s
);

// Example: Linear backoff instead of exponential
const delay = config.reconnect_base_delay_ms * reconnectionAttempt;

// Example: Jittered backoff (avoid thundering herd)
const baseDelay = config.reconnect_base_delay_ms * Math.pow(2, reconnectionAttempt - 1);
const jitter = Math.random() * 1000;  // Random 0-1000ms
const delay = baseDelay + jitter;

Complete Resilient Bot Example

const config = loadConfig();

async function main() {
    try {
        // 1. Optional: Trigger RTMS via REST API
        console.log('[RTMS] Triggering RTMS start...');
        await triggerRTMSStart(MEETING_ID);
        
        // 2. Wait for webhook
        console.log('[RTMS] Waiting for meeting.rtms_started webhook...');
        const rtmsInfo = await waitForRTMSWebhook(MEETING_UUID);
        
        // 3. Connect with resilience
        const rtms = new ResilientRTMSConnection(rtmsInfo, config);
        await rtms.connect();
        
        console.log('[RTMS] Bot is running, processing streams...');
        
        // 4. Process media data
        rtms.mediaWs.on('message', (data) => {
            const frame = parseMediaFrame(data);
            processMediaFrame(frame);
        });
        
        // 5. Handle graceful shutdown
        process.on('SIGINT', () => {
            console.log('[RTMS] Shutting down...');
            rtms.cleanup();
            process.exit(0);
        });
        
    } catch (error) {
        console.error('[RTMS] ABORT:', error.message);
        process.exit(1);
    }
}

main();

Comparison: RTMS Bot vs Meeting SDK Bot

Aspect	RTMS Bot	Meeting SDK Bot
Visibility	Invisible (read-only service)	Visible participant
Authentication	REST API trigger + webhook	JWT + OBF token
Join Dependency	No dependency on participants	Owner must be present
Retry Logic	Not applicable (webhook-based)	Required (owner presence)
Media Access	Audio/video/text/share/chat via WebSocket	Raw audio/video/share via SDK
Recording Control	None (read-only)	Full (local, cloud, raw)
Interaction	Cannot interact	Can send chat, reactions
Resource Usage	Lower (WebSocket only)	Higher (full SDK)
Use Case	Passive transcription, analytics	Interactive bots, recording, moderation

Choose RTMS Bot when:

• You only need to observe/transcribe
• You want minimal resource usage
• You prefer invisible operation
• You're processing external meetings (with permission)

Choose Meeting SDK Bot when:

• You need to interact with the meeting (chat, reactions)
• You need local recording control
• You want to be visible in participant list
• You're processing your own meetings

Troubleshooting

Webhook Never Arrives

Symptom: waitForRTMSWebhook() times out

Solution:

1. Verify webhook endpoint is HTTPS and publicly accessible
2. Check Event Subscriptions in Zoom Marketplace: meeting.rtms_started enabled
3. Verify RTMS was actually started (check Zoom client or REST API response)
4. Increase webhook_wait_timeout_ms if meeting starts later than expected
5. Test webhook delivery: curl -X POST YOUR_WEBHOOK_URL

Signaling Handshake Fails

Symptom: Connection closes immediately after handshake

Solution:

1. Verify HMAC signature generation matches Zoom docs
2. Check timestamp is current (not stale)
3. Verify WEBHOOK_SECRET_TOKEN matches Zoom Marketplace config
4. Check signaling URL hasn't expired (short TTL)

Keep-Alive Timeout

Symptom: Connection closes with "Keep-alive timeout"

Solution:

1. Network congestion - increase keepalive_timeout_ms
2. Server overloaded - increase keepalive_interval_ms
3. Verify ping/pong implementation is correct
4. Check firewall/proxy not blocking WebSocket pings

Frequent Reconnections

Symptom: Bot reconnects multiple times, then gives up

Solution:

1. Increase reconnect_max_attempts (e.g., 5 instead of 3)
2. Increase reconnect_base_delay_ms if network is slow
3. Monitor server resources (CPU/memory/network)
4. Check for rate limiting (too many connection attempts)

Resources

• RTMS Docs: https://developers.zoom.us/docs/rtms/
• RTMS WebSocket Guide: https://developers.zoom.us/docs/api/websockets/
• RTMS SDK: https://github.com/zoom/rtms
• Webhook Reference: ../references/webhooks.md
• Connection Architecture: ../concepts/connection-architecture.md
• Meeting SDK Bot Alternative: Meeting SDK Bot (Linux)