Ad
Skip to content
Subscribe Now
AI in practice
Copy the url to clipboard Share this article Go to comment section

Gemini 2.0 Flash Thinking: Google's smallest model takes lead in Chatbot Arena

Maximilian Schreiner
Maximilian Schreiner View the LinkedIn Profile of Maximilian Schreiner
Jan 22, 2025
Image description
Google

Key Points

  • Google DeepMind has released a new version of its Gemini 2.0 Flash Thinking AI model, which has taken the top spot in the chatbot arena, according to ratings platform lmarena.ai, with a 17-point improvement over the last checkpoint in December 2024.
  • Technical improvements include code execution, an expanded context window of one million tokens, and improved consistency between thinking processes and responses. In particular, the significantly increased context window should open up new application possibilities.
  • Demis Hassabis, CEO of Google DeepMind, emphasizes that this development builds on more than a decade of experience in AI planning systems. The combination of these proven approaches with modern foundation models is proving to be particularly powerful, especially in math and science benchmarks.

Google's experimental AI model Gemini 2.0 Flash Thinking has jumped ahead of its competitors, scoring impressive results in math, science, and general performance tests.

According to testing platform lmarena.ai, the latest version of Gemini has made significant gains in the Chatbot Arena, improving its score by 17 points since December 2024. This puts it ahead of competitors like OpenAI's GPT-4o models and Anthropic's Claude 3.5 Sonnet.

The model shows improvements across nearly all categories, taking the lead in complex tasks, programming, and creative writing. The only area where it still needs work is style control - how it formats its outputs.

Under the hood, Google says they've added new features like code execution and expanded the model's context window to handle up to one million tokens. They've also improved how well the model's thinking process lines up with its final responses.

Ad
DEC_D_Incontent-1

Google relies on years of experience with planning systems

Google DeepMind's CEO Demis Hassabis says this progress builds on more than ten years of work with AI planning systems, going all the way back to AlphaGo. By combining these tried-and-true planning methods with modern foundation models, they've seen particularly strong results in math and science testing.

Ad
DEC_D_Incontent-2

This update follows the first version of Flash 2.0 Thinking, which Google launched in December 2024. That version introduced explicit thought processes that help the model improve its reasoning, and it also performed well in testing.

AI News Without the Hype – Curated by Humans

As a THE DECODER subscriber, you get ad-free reading, our weekly AI newsletter, the exclusive "AI Radar" Frontier Report 6× per year, access to comments, and our complete archive.