- Benchmark added
OpenAI has expanded its lineup with GPT-4o mini, a new AI model that offers improved performance at a lower cost.
GPT-4o mini offers higher performance while costing 60 percent less than OpenAI's most affordable model to date, according to the company. OpenAI claims it's the most powerful and cost-effective small model available today. Developers pay 15 cents per 1M input tokens and 60 cents per 1M output tokens (roughly the equivalent of 2500 pages in a standard book).
The new model is available immediately to users of ChatGPT's Free, Plus, and Team plans. Enterprise customers will have access within the next week.
ChatGPT users can now use GPT-4o mini instead of GPT-3.5 Turbo. Developers can still access GPT-3.5 via the API if they prefer not to switch to GPT-4o mini. OpenAI plans to remove GPT-3.5 from the API in the future, but hasn't set a specific date.
GPT-4o mini supports text and images via the API and is expected to soon support all multimodal input and output, including video and audio. OpenAI attributes the efficiency gains to improvements in model architecture and optimized training data and methods.
Fintech startup Ramp uses GPT-4o mini to extract data from uploaded receipts, according to OpenAI. Email client Superhuman tested the model for an auto-suggest feature for email responses.
In the MMLU (Measuring Massive Multitask Language Understanding) benchmark for general knowledge and language comprehension, GPT-4o mini scored 82 percent. This makes it one of the highest performing small models in this specific benchmark.
For comparison, GPT-3.5 scored 70 percent and GPT-4o scored 88.7 percent. Anthropic's smallest and cheapest model, Claude 3 Haiku, scored around 75 percent. As always, benchmark results don't fully reflect real-world performance.
This announcement comes as OpenAI faces increasing pressure on its model business. Competitors like Anthropic sometimes offer better or cheaper models via API, while other vendors and open-source models are also competing in the market for efficient, small, and low-cost models, including Microsoft.
