Elevenlabs has released Conversational AI 2.0, an updated system that allows its agents to handle speech and text simultaneously for more fluid interactions.
The update introduces a new conversation model that, according to Elevenlabs, analyzes conversational signals like pauses and filler sounds ("um," "ah") in real time. The system uses these cues to determine when to speak or stay silent. For example, if a user pauses to look something up, the agent waits rather than responding immediately.
In earlier versions, users had to switch languages manually. The updated platform now detects the spoken language automatically and adjusts its responses accordingly, which allows users to move between languages during a single interaction.
The system also supports multimodal communication. Users and agents can interact through voice, text, or both at once. According to Elevenlabs, this helps maintain continuity when users switch between speaking and typing.
Voice agents powered by RAG for company-specific knowledge
One of the more substantial updates is the integration of Retrieval Augmented Generation (RAG) directly into the voice agent. This allows agents to access company databases in real time, reducing latency while protecting user privacy.
With a unified agent definition, companies can centrally manage dialog behavior, knowledge sources, and communication channels. This setup, combined with RAG, is meant to streamline customer service and make it easier to automate and scale voice-based workflows. Example use cases include medical assistants retrieving treatment guidelines or support agents accessing up-to-date product information during a call.
The update also introduces a "Batch Calling" feature, which lets companies automate mass calls for alerts, surveys, or personalized messages. The platform supports both inbound and outbound calls and integrates with SIP trunking systems.
The release brings new compliance options aimed at enterprise customers. Version 2.0 is fully HIPAA-compliant, making it suitable for healthcare applications. Elevenlabs also provides optional EU data residency to help organizations address data sovereignty requirements.
