summary Summary

According to Scott Stein, CTO of Microsoft, 2022 was the most exciting AI year ever. But he expects even more highlights (and scaling) in 2023. Is this an indication of GPT-4's performance?

In a look back and forward at the state of AI, Scott Stein calls the coming year "the most exciting year that the AI community has ever had." That's notable because Microsoft is a major investor in OpenAI, the AI company that has set new standards for commercializing large AI models with GPT-3, DALL-E 2, and ChatGPT.

According to rumors, GPT-4 will be introduced in the coming months. In his role as Microsoft CTO, Stein should know exactly what the new language model will be capable of. His wholehearted promise for 2023 could be an indication that he expects another leap in performance.

In addition, Stein says that further scaling of AI models could lead to even more and better performance - another possible indication of the GPT-4 architecture. It could be larger.


AI as a science accelerator

In addition to advances in generative AI for code, text, and images, which Stein sees as having the potential to boost productivity and creativity as well as improve workflows, Stein highlights the benefits of AI for science.

As all these huge AI systems continue to grow and evolve, I think we can expect that these advances are going to fundamentally change the nature of work, in some places more than others, and in some cases create a whole spate of new jobs that didn’t exist before.

Scott Stein

AI models developed for science would have the same scaling properties as large language models. They could "dramatically change" the performance of an application, from fluid dynamics to molecular dynamics to drug discovery, Stein Says.

"There’s immense opportunity there. This means better medicines, it means maybe we can find the catalyst we don’t have yet to fix our carbon emission problem, it means across the board accelerating how scientists and other folks with big ideas can work to try to solve society’s biggest challenges," Stein says.

Other AI companies and researchers also see great benefits in AI for scientific progress outside of IT, for example in brain research, language learning, or medicine.

Stein believes in scaling

A fundamental debate in the AI community is the ongoing benefit of scaling large AI models. So far, Stein said, the formula that larger AI models also developed better and more abilities has proven successful. Combined with scaling, the so-called phase transition in particular is an unexplained phenomenon that could lead to the emergence of new abilities in even larger models.


"It turns out that models trained on more data with more compute power just have a much richer and more generalized set of capabilities," Stein says. "To be clear, right now, we don’t see any end to the benefits of increased scale." Further progress will require higher computing power, which must be used more efficiently, Stein adds.

For safer AI, Stein cites refined data sets, filters, blocking potentially harmful prompts, and deliberate gradual release of large models as important factors. There also needs to be a plan for dealing with unforeseen abuse after a model is launched, such as restricting access via the API, he says.

Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
  • Microsoft CTO Scott Stein says 2022 was the most exciting AI year ever. But he expects 2023 to be even more eventful.
  • Scaling could further improve the performance of large AI models. Is this Stein's outlook for GPT-4? Microsoft is the largest investor in OpenAI.
  • In addition to advances in generative AI tools for work, Stein believes AI will benefit many scientific disciplines.
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.