Call Recording and Transcription

This document provides an overview of the tools and storage infrastructure used for recording calls, generating transcripts, and protecting sensitive information.

Overview

echo automatically records all voice calls and generates anonymized transcripts. Both recordings and transcripts are stored securely in Azure Blob Storage for later retrieval and analysis.

Tools and Technologies

Azure Blob Storage

All recordings and transcripts are stored in Azure Blob Storage with agent-level isolation for multitenancy.

• Storage Account: echodev
• Container: calls (configured via environment variables)
• Structure: Organized by agent ID for complete tenant isolation
• Access: Secured with account keys

LiveKit

Egress (Recording)

Call recordings are handled by LiveKit's egress system, which captures audio from the room and uploads it directly to Azure Blob Storage.

• Format: OGG audio files
• Quality: Audio-only recordings
• Upload: Real-time streaming during the call

AgentSession (Transcription)

Transcripts are automatically generated by LiveKit's AgentSession, which tracks the conversation history between the user and the AI agent.

• Capture: Real-time conversation tracking
• Format: Structured message history with roles (user/assistant)
• Storage: Conversation saved via session.history.to_dict()

PII Anonymization

Transcripts are automatically anonymized to protect sensitive information using industry-standard tools.

Presidio

Presidio is Microsoft's open-source framework for detecting and anonymizing personally identifiable information (PII).

• Analyzer: Detects PII entities in text
• Anonymizer: Replaces PII with anonymized placeholders
• Language Support: Multiple languages including English and French

spaCy

spaCy is used as the NLP engine for entity recognition.

• Models:
  • en_core_web_sm - English language model (small)
  • fr_core_news_sm - French language model (small)
• Purpose: Named entity recognition for PII detection
• Integration: Works seamlessly with Presidio

Storage Structure

All files are organized within the calls container with agent-level isolation:

calls/                                    (Azure container)
└── {agent_id}/                          (Agent UUID - tenant isolation)
    ├── recordings/                      (All recordings for this agent)
    │   └── {room_name}/                (Call session folder)
    │       └── {room_name}-{timestamp}.ogg
    └── transcripts/                     (All transcripts for this agent)
        └── {room_name}/                (Call session folder)
            └── transcript-{timestamp}.json

Recordings

Path structure:

{agent_id}/recordings/{room_name}/{room_name}-{timestamp}.ogg

Example:

d4e5f6a7-b8c9-0d1e-2f3a-4b5c6d7e8f9a/recordings/call-_+573134568674_7G4bKHGW4yJJ/call-_+573134568674_7G4bKHGW4yJJ-20251110T121532.ogg

Full Azure URL:

https://echodev.blob.core.windows.net/calls/d4e5f6a7-b8c9-0d1e-2f3a-4b5c6d7e8f9a/recordings/call-_+573134568674_7G4bKHGW4yJJ/call-_+573134568674_7G4bKHGW4yJJ-20251110T121532.ogg

• Agent ID: UUID for tenant isolation
• Room Name: Unique identifier for each call session
• Timestamp: ISO 8601 format (YYYYMMDDTHHMMSS)
• Format: OGG audio file

Transcripts

Path structure:

{agent_id}/transcripts/{room_name}/transcript-{timestamp}.json

Example:

d4e5f6a7-b8c9-0d1e-2f3a-4b5c6d7e8f9a/transcripts/call-_+573134568674_7G4bKHGW4yJJ/transcript-20251110_121532.json

Full Azure URL:

https://echodev.blob.core.windows.net/calls/d4e5f6a7-b8c9-0d1e-2f3a-4b5c6d7e8f9a/transcripts/call-_+573134568674_7G4bKHGW4yJJ/transcript-20251110_121532.json

• Agent ID: Same UUID as recordings (tenant isolation)
• Room Name: Same as recording for easy correlation
• Timestamp: Format YYYYMMDD_HHMMSS
• Format: JSON file with conversation and metadata

Multitenancy Benefits

This structure provides:

• Complete Isolation: Each agent's data is in a separate folder
• Security: Agent ID is a UUID (not exposing phone numbers in paths)
• Access Control: Azure RBAC can grant access per agent
• Easy Management: Export, backup, or delete all data for one agent
• Scalability: Add more data types per agent without restructuring

Transcript Structure

The JSON transcript file contains:

{
  "room_name": "call-_+573134568674_7G4bKHGW4yJJ",
  "agent_name": "Support Agent",
  "phone_number": "+16182988373",
  "timestamp": "20251110_121532",
  "transcript": {
    "messages": [
      {
        "role": "user",
        "content": "Hello, I need help with my account"
      },
      {
        "role": "assistant",
        "content": "Hello! I'd be happy to help you. What issue are you experiencing?"
      }
    ]
  }
}

Note: All PII (phone numbers, names, addresses, etc.) in the conversation content is automatically anonymized before storage.

PII Detection

The following types of personally identifiable information are detected and anonymized:

• Phone Numbers: Replaced with [PHONE_NUMBER]
• Email Addresses: Replaced with [EMAIL_ADDRESS]
• Names: Replaced with [PERSON]
• Addresses: Replaced with [LOCATION]
• Credit Card Numbers: Replaced with [CREDIT_CARD]
• Dates of Birth: Replaced with [DATE_TIME]

Database Tracking

Each call is tracked in the database with references to both files:

• Call Record: Stores metadata about the call
• Transcription Path: URL to the transcript in Azure Blob Storage
• Recording Path: URL to the audio recording in Azure Blob Storage
• Timestamps: Start and end times of the call

This allows easy retrieval of both recordings and transcripts for a specific call.

Configuration

Azure Storage (Environment Variables)

Azure Blob Storage is configured via environment variables in the agent:

AZURE_STORAGE_ACCOUNT_NAME=echodev
AZURE_STORAGE_ACCOUNT_KEY=<your-key>
AZURE_STORAGE_CONTAINER_NAME=<container-name>

PII Anonymization (Constants File)

PII anonymization is configured in the constants.py file:

# packages/agent/src/constants.py
ENABLE_PII_ANONYMIZATION = True

Set to False to disable PII anonymization (not recommended for production).

Access and Retrieval

Both recordings and transcripts can be accessed via:

1. Direct URL: Using the Azure Blob Storage URL stored in the database
2. API: Through the echo backend API endpoints
3. Azure Portal: For administrative access

All files are securely stored and require proper authentication to access.