According to Scott Stein, CTO of Microsoft, 2022 was the most exciting AI year ever. But he expects even more highlights (and scaling) in 2023. Is this an indication of GPT-4's performance?
In a look back and forward at the state of AI, Scott Stein calls the coming year "the most exciting year that the AI community has ever had." That's notable because Microsoft is a major investor in OpenAI, the AI company that has set new standards for commercializing large AI models with GPT-3, DALL-E 2, and ChatGPT.
According to rumors, GPT-4 will be introduced in the coming months. In his role as Microsoft CTO, Stein should know exactly what the new language model will be capable of. His wholehearted promise for 2023 could be an indication that he expects another leap in performance.
In addition, Stein says that further scaling of AI models could lead to even more and better performance - another possible indication of the GPT-4 architecture. It could be larger.
AI as a science accelerator
In addition to advances in generative AI for code, text, and images, which Stein sees as having the potential to boost productivity and creativity as well as improve workflows, Stein highlights the benefits of AI for science.
As all these huge AI systems continue to grow and evolve, I think we can expect that these advances are going to fundamentally change the nature of work, in some places more than others, and in some cases create a whole spate of new jobs that didn’t exist before.
Scott Stein
AI models developed for science would have the same scaling properties as large language models. They could "dramatically change" the performance of an application, from fluid dynamics to molecular dynamics to drug discovery, Stein Says.
"There’s immense opportunity there. This means better medicines, it means maybe we can find the catalyst we don’t have yet to fix our carbon emission problem, it means across the board accelerating how scientists and other folks with big ideas can work to try to solve society’s biggest challenges," Stein says.
Other AI companies and researchers also see great benefits in AI for scientific progress outside of IT, for example in brain research, language learning, or medicine.
Stein believes in scaling
A fundamental debate in the AI community is the ongoing benefit of scaling large AI models. So far, Stein said, the formula that larger AI models also developed better and more abilities has proven successful. Combined with scaling, the so-called phase transition in particular is an unexplained phenomenon that could lead to the emergence of new abilities in even larger models.
"It turns out that models trained on more data with more compute power just have a much richer and more generalized set of capabilities," Stein says. "To be clear, right now, we don’t see any end to the benefits of increased scale." Further progress will require higher computing power, which must be used more efficiently, Stein adds.
For safer AI, Stein cites refined data sets, filters, blocking potentially harmful prompts, and deliberate gradual release of large models as important factors. There also needs to be a plan for dealing with unforeseen abuse after a model is launched, such as restricting access via the API, he says.
