The dialog-optimized AI Blenderbot 3 can research information on the Internet and integrate it into conversations. Meta publishes a web demo and models of the system to collaboratively improve its security.
Blenderbot 3 is Meta's version of Google's Lamda, the AI chat system that recently successfully faked consciousness to a Google engineer. Blenderbot has been in the works since around 2020, with the goal of developing a chatbot with personality, empathy, and extensive knowledge of the world.
The last major update dates back to 2021, when Meta added to Blenderbot 2 the ability to research information on the Internet from reliable sources to reduce its susceptibility to nonsensical statements. Meta also integrated a long-term memory that allows the bot to store relevant information about its answers or talking partners.
Blenderbot 3 is based on Meta's open-source language model
According to Meta, Blenderbot 3 scores 31 percent better dialog ratings than its predecessor in initial tests with human counterparts. The human testers rated Blenderbot 3 as twice as knowledgeable compared to the second version. In addition, the system is said to be factually incorrect in 47 percent less cases.
Blenderbot 3.0 in action. | Video: Meta
This progress is especially due to the upgrade of the language model: Blenderbot 3 is based on the 175 billion parameter model of Meta's open-source language transformer OPT and is, according to Meta, about 58 times larger than Blenderbot 2.
Compared to GPT-3, Blenderbot gives more up-to-date answers 82 percent of the time and more specific answers 76 percent of the time, according to Meta. However, Blenderbot's training data is more up-to-date than GPT-3's.
Open-source approach for greater security
With the release of the open source model and a web demo, Meta's primary goal is to drive data collection, discourse about, and advancement of secure conversational AI.
"Our research goal is to collect and release conversational feedback data that we and the broader AI research community can leverage over time to eventually find new ways for conversational AI systems to optimize both safety and engagingness for everyone who uses them," Meta writes.
Initial analysis of the web demo showed that 0.16 percent of Blenderbot 3 responses were flagged as unfriendly or inappropriate. The "ideal" 0 percent would likely require user-level personalization, Meta's research team suspects. It would be a difficult balance between security and willingness to engage in dialog - the bot might try to change the subject on sensitive topics, for example.
Long-term goal: Language as the main interface for computers and useful virtual assistants
Meta refers to Blenderbot 3 as a "significant advance" over chatbots currently available to the public. However, the system is "certainly not at a human level" and occasionally gives incorrect, inconsistent, off-topic, or otherwise unsatisfactory answers, the researchers said.
Meta's new chatbot has *opinions* about its CEO. https://t.co/hUkJxw3UTr pic.twitter.com/Hq4mZTY1DV
- Max Woolf (@minimaxir) August 5, 2022
Through interactions with the demo, Meta hopes to continue to improve the model and provide updated model snapshots to the AI community. The team's goal is to one day "AI-powered computers that everyone can chat with in genuinely helpful and interesting ways."
Head here for the Blenderbot 3 web demo, which is currently only available from the U.S. The code, datasets, and smaller models are freely available from Parlai.
The most powerful 175-billion model, as with OPT, is only available to select individuals and institutions from the research community upon request.
