Content
summary Summary

When ChatGPT & Co. have to check their answers themselves, they make fewer mistakes, according to a new study by Meta.

Ad

ChatGPT and other language models repeatedly reproduce incorrect information - even when they have learned the correct information. There are several approaches to reducing hallucination. Researchers at Meta AI now present Chain-of-Verification (CoVe), a prompt-based method that significantly reduces this problem.

New method relies on self-verification of the language model

With CoVe, the chatbot first responds to a prompt such as "Name some politicians who were born in New York." Based on this output, which often already contains errors, the language model then generates questions to verify the statements, such as "Where was Donald Trump born?"

CoVe relies on separately prompted verification questions. | Image: Meta AI

These "verification questions" are then executed as a new prompt, independent of the first input, to prevent the possible adoption of incorrect information from the first output. The language model then verifies the first input against the separately collected facts. All testing was done with Llama 65B.

Ad
Ad

Chain-of-verification significantly reduces hallucinations in language models

The team shows that answers to individual questions contain significantly fewer errors, allowing CoVe to significantly improve the final output to a prompt. For list-based questions, such as the politician example, CoVe can more than double accuracy, significantly reducing the error rate.

For more complex question-and-answer scenarios, the method still yields a 23 percent improvement, and even for long-form content, CoVe increases factual accuracy by 28 percent. However, with longer content, the team also needs to check the verification answers for inconsistencies.

In their tests, the Meta team can also show that instruction tuning and chain-of-thought prompting do not reduce hallucinations, so Llama 65B with CoVe beats the newer, instruction-tuned model Llama 2. In longer content, the model with CoVe also outperforms ChatGPT and PerplexityAI, which can even collect external facts for its generations. CoVe works entirely with knowledge stored in the model.

In the future, however, the method could be improved by external knowledge, e.g. by allowing the language model to answer verification questions by accessing an external database.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Meta AI has developed a new method called Chain-of-Verification (CoVe) that significantly reduces misinformation from language models such as ChatGPT.
  • CoVe works by having the chatbot generate verification questions based on its initial response, and then execute them independently of the original input to prevent the acquisition of false information. The language model then compares the original input with the separately collected facts.
  • The method has been shown to more than double accuracy for list-based questions and improves factual accuracy by 28 %, even for long content. In the future, CoVe could be improved by integrating external knowledge, such as accessing an external database to answer verification questions.
Sources
Max is managing editor at THE DECODER. As a trained philosopher, he deals with consciousness, AI, and the question of whether machines can really think or just pretend to.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.