Anthropic CEO Dario Amodei has confirmed that the company is working on a new version of its flagship AI model, Claude Opus, but he's staying tight-lipped about when it will actually drop.
The update, known as Opus 3.5, was initially slated for this year. However, Amodei now only vaguely mentions that it will arrive "at some point," reflecting an industry-wide trend of delays in the development of large language models with advanced capabilities.
In an interview with podcaster Lex Fridman, Amodei noted that the recently updated and released Haiku 3.5 already matches the performance of the original Opus 3 while being faster and cheaper to run. The company's next goal is to further shift the performance curve.
The holdup with Opus 3.5 mirrors a wider trend in the AI industry. Since the debut of GPT-4, there hasn't been a major leap forward in what large language models can do. Instead, progress has been more subtle - refining the nuances and bolstering efficiency.
Leading companies such as OpenAI and Google are reportedly grappling with delays and disappointing results from their latest models, seeking new ways to develop more powerful AI systems.
Month-long development cycles with many uncertainties
In the podcast, Anthropic's CEO pulled back the curtain on just how complex it is to develop these AI models. Just the pre-training phase can drag on for months, Amodei says, and it takes a huge amount of computing power - tens of thousands of specialized chips like GPUs or TPUs.
After the initial training, the model goes through an elaborate fine-tuning process. One key part of this is "reinforcement learning from human feedback." That's where human experts painstakingly review the model's outputs and score them on different criteria to help the model learn and improve.
Then there's a battery of internal tests and external audits to check for safety issues, often working with AI safety groups in the US and UK.
While breakthroughs in AI might look like scientific leaps to people outside the industry, a lot of the progress actually comes down to boring technical details, Amodei says. The hardest parts are usually things like software development and making the models run faster, not big conceptual advances. The "intelligence" and "personality" of the models can change unpredictably with each new version. Getting it right is more of an art than a science, according to Amodei.
OpenAI has said similar things before - that developing AI models isn't some clean, automated industrial process, but rather an "artisanal, multi-person effort."
