OpenAI's new premium o1 model proves effective for complex scientific tasks but remains largely irrelevant for average users. Experts suggest most people should continue using standard AI models instead.
Early experiences with OpenAI's new premium o1 model and its Pro version indicate the offering makes sense only for a small group of users. "You really need to have particular hard problems to solve in order to get value out of it," explains Ethan Mollick, professor at the Wharton School of the University of Pennsylvania and co-director of its Generative AI Labs, in a post on X.
OpenAI offers o1 in two versions: The standard version comes with the Plus subscription for $20 per month, while the Pro version with additional computing power costs $200 monthly.
For most users, Mollick recommends sticking with ChatGPT, Claude, or Gemini. While the new o1 model shows impressive capabilities, he describes it as "a little weird" and "not for most people most of the time."
Other users report that o1 struggles with tasks requiring multiple conversation rounds. "It struggles very badly at multi turn problems, and problems requiring iterative back and forth in general"," one user posted on X.
Immunologist: "A truly innovative idea"
The system's strengths emerge primarily in scientific tasks at doctoral level, finance, and other highly specialized fields. "Discovering uses will require real R&D efforts," Mollick notes.
Interesting use cases come from science: Immunologist and biomedical researcher Professor Derya Unutmaz from Jackson Laboratory in Connecticut, who studies aging and cancer immunotherapy, reports "designing highly sophisticated biological experiments based on a truly innovative idea" using o1-pro.
An example inspired by the controversial "Sparks of AGI" paper also shows o1's superior ability to visualize concepts via SVG.
"Do your best to replicate this image with an svg"
Claude versus GPT-4o versus o1-pro versus Claude, first try. pic.twitter.com/pFrhPrz13P
- Ethan Mollick (@emollick) December 5, 2024
In the Microsoft paper, researchers reported how GPT-4 got better and better at "painting" a unicorn over the course of its training - and lost this ability again after post-training with RLHF. In this case, however, there was no picture template because the model was not multimodal.
ChatGPT Pro for 200 US dollars could be a bargain
OpenAI is granting ten ChatGPT Pro scholarships to medical researchers at leading US institutions. An API version will follow soon. OpenAI CEO Sam Altman calls o1 the "smartest model in the world."
However, practical experience shows this "most intelligent model" mainly serves highly specialized use cases. For these applications, the $200 price tag might be relatively modest - especially in finance, which Mollick sees as a key use case. When asked about specific financial applications, Mollick responded: "You won't see them shared."
OpenAI plans to announce additional products in the coming days - including a likely release of Sora. Since video generation requires significantly more computing power than text, ChatGPT Plus subscribers will probably receive preferred access to Sora.
