Benchmarks from German AI startup Aleph Alpha show that the startup's latest AI models can keep up with OpenAI's GPT-3. A success that should not lull Europe into a false sense of security.
ChatGPT has catapulted artificial intelligence into the public discussion like no other product before it. Behind the chatbot is the U.S. company OpenAI, which made headlines with the large-scale language model GPT-3 and later with the text-to-picture model DALL-E 2. The impact of systems like ChatGPT or Midjourney on education and work, which can be felt today, was foreseeable even then.
The underlying language models are often referred to in research as foundation models: a large AI model that, due to its generalist training with large datasets, can later take on many tasks for which it was not explicitly trained.
In Germany, for example, there were early calls from the German AI Association for a European initiative to build the necessary infrastructure for large European AI models. This was the only way to secure digital sovereignty, data protection, and possibly even Europe's economic future. Since then, the LEAM project has been driven forward by various companies in the industry.
Aleph Alpha: Foundation Models from Germany
The term foundation model is controversial because it can be interpreted as a precursor to general artificial intelligence, critics say. But beyond such more philosophical discussions, the big AI models from OpenAI and others form the basis for potentially transformative technologies like ChatGPT or Bing search with an unstable personality.
OpenAI unveiled GPT-3 in May 2020 and opened access via an API a month later. Just over a year later, Jurassic-1 Jumbo from Israeli AI startup AI21-Labs launched its first competing product. In China, there are several alternatives, among the first in 2021 were Wu Dao and PanGu-Alpha.
And in Germany? Here, it's the Heidelberg-based startup Aleph Alpha that wants to compete with OpenAI. Aleph Alpha released the first language models of the Luminous family in April 2022. The largest model to date, Luminous-Supreme, with 70 billion parameters, has been available since August 2022.
Now Aleph Alpha is releasing benchmark results comparing the largest Luminous model to OpenAI's GPT-3 ("davinci" - 175 billion parameters), BigScience's BLOOM (176 billion parameters) and Meta's OPT (175 billion parameters).
With Luminous, Aleph Alpha closes the gap to GPT-3
The models competed in a variety of tasks, including classification, common sense reasoning, and natural language inference. Aleph Alpha uses the Evaluation Harness from the open research collective Eleuther AI.
For almost all tasks, Luminous matches the performance of GPT-3, and in one category even exceeds it, with less than half the parameters. In practice, this means better scalability and lower resource consumption for Luminous-Supreme. Meta OPT and BigScience Bloom, released last year, are on average a few percentage points behind.
No time for complacency: Europe must continue funding AI
Aleph Alpha's success is good news for the startup and good news for Europe, as it is the first European AI model at GPT-3 level.
"What our customers and partners are telling us in their use cases is also reflected in the sum of the tests we have conducted: Luminous is a strong alternative in many environments and thus an important step towards Europe's technological sovereignty," Jonas Andrulis, founder and CEO of Aleph Alpha, classifies the results.
But the results also provide another perspective: It took more than two years for a European alternative to OpenAI's GPT-3 davinci model to become available. Jurassic-1 Jumbo also achieves comparable benchmark results and has been on the market for more than a year and a half.
In the meantime, OpenAI has released other iterations of the davinci model, most recently text-davinci-003, which was trained with human feedback and is the basis for ChatGPT.
In addition, in early April 2022, Google unveiled the 540-billion-parameter PaLM, which significantly outperformed GPT-3 davinci in almost all benchmarks.
This is not to diminish the success of Aleph Alpha: No other European company or public initiative has produced such a powerful model. But the example also shows that Europe and Germany need to take the development and funding of AI even more seriously.
Aleph Alpha is already testing its new model, and it could compete with GPT-4
Thanks to Aleph Alpha and Luminous-Supreme, Europe is now better positioned in the AI race. With its 70-billion-parameter model, the company has leaped to the industry's forefront and demonstrated that more can be achieved with fewer parameters.
Aleph Alpha also launched a new offering this month, the Luminous Supreme Control model, which is positioned as an alternative to OpenAI's newer models.
Another AI system is currently in testing: with 300 billion parameters, Luminous-World is Aleph Alpha's largest AI model to date and is said to offer new capabilities for "highly complex and critical applications". It is expected to be released later this year.
The 300-billion-parameter model will likely compete directly with OpenAI's GPT-4 and could position the German startup as a key player in international competition.
At the same time, the company wants to further develop its own technology in the area of transparency and explainability: "This is the only way we can ensure sustainable and ethical development in line with our values and standards - this is about more than privacy," says Andrulis.
Aleph Alphas Luminous models are available through their Luminous App and API.
