Microsoft predicts three AI trends for 2024: small language models (SLMs), multimodal AI, and AI in science.
Compared to LLMs, SLMs are much smaller AI models that are streamlined for efficiency. They are more accessible and affordable and can be better adapted to produce valuable results for specific use cases, particularly through fine-tuning with high-quality data or targeted database-driven implementation.
According to Microsoft, SLMs are becoming increasingly powerful, challenging the notion that size equals performance when it comes to language models. Microsoft AI researcher Sebastien Bubeck says SLMs could become as powerful as LLMs.
No AI model, large or small, has yet surpassed the performance of GPT-4 on all fronts, but it is expensive to run. This is a particular problem for Microsoft, which wants to sell valuable AI services to millions of users. Here, the cost of running AI is of enormous financial importance.
That is why Microsoft is betting on more efficient AI models, which is reportedly a focus of its AI research. The Phi and Orca SLMs are two of the latest results of this work.
A second major AI trend, according to Microsoft, is multimodal AI. It can process text, images, audio, and video, making technologies such as search tools and creativity applications more accurate and seamless, Microsoft says.
ChatGPT already offers a foretaste with the integration of GPT-4V or the image generator DALL-E3. Microsoft uses DALL-E3 and an LLM in Microsoft Designer for text-based image generation. Future OpenAI models are said to be even more multimodal. Google's new Gemini Ultra model is also multimodal by design.
Microsoft believes another trend will be the use of AI in science. The company expects machine learning to accelerate scientific discoveries and solve global problems such as climate change, energy crises, and disease.
Microsoft researchers are experimenting with AI for sustainable agriculture, life sciences, and materials sciences. In life sciences, for example, AI could help with image recognition for cancer diagnosis or the search for new drug compounds and molecules. In materials science, AI could speed the search for less toxic battery materials.
