Jeff Dean leads AI development at Google. He sees three key points that are still missing for truly intelligent AI - but where a path to a solution may already be emerging.
Before we talk about the intelligence of artificial intelligence, two key aspects are important: First, intelligence is a fuzzily defined term in any case. Second, there are good reasons to believe that intelligence is a spectrum and that different types of intelligence exist.
Often the term intelligence is equated with the human mind and here in particular with the abilities to learn efficiently and to be able to transfer what has been learned to different tasks, i.e., to generalize. Today's AI systems are usually only good at individual tasks, which is why they are sometimes denied intelligence.
Google's AI chief Jeff Dean takes a similar view: "I think the big challenge in AI is how to generalize a set of tasks you've already mastered to new tasks as easily and effortlessly as possible."
Three major milestones in AI development
For Dean, current AI training, which creates many specialized models, is inefficient and thus the first problem: "If you train a neural network from scratch, it's like forgetting all your training every time you try something new. That's crazy, right?" said Dean at a talk. Instead, he said, AI systems need to be developed that can "master thousands or millions of tasks" by generalizing their knowledge.
The second problem to be solved on the path to more general AI, Dean says, is how to process many different types of data, as humans have mastered. AI systems are predominantly specialized in one type of data, such as text, images, or video. Multimodal AI systems, such as those demonstrated by Google and other companies in 2021, are one approach to solving this problem.
The third problem, for Dean, is the lack of efficiency of current AI models that activated the entire neural network for every task, whether the task was challenging or simple. "That's also different from how our own brains work. Different parts of our brain are good for different things, and we're constantly calling on the parts of them that are relevant to the task at hand," Dean says.
Pathways: Google's way to more general artificial intelligence?
If researchers succeed in fixing these three issues, Dean expects more powerful AI systems: a handful of general-purpose models could then handle thousands or millions of tasks, according to Dean, dealing with all modalities rather than single ones, and able to combine them. To do this efficiently, they activate only the areas in the artificial neural network that are needed for a specific task.
According to Dean, the Pathways AI architecture designed by Google is a step toward this AI future. Pathways is a kind of blueprint for an intelligent AI multipurpose system that Google unveiled last fall, but for which no technical details are yet available.
Dean expresses confidence that Pathways is a big step toward a new generation of AI: "We're pretty excited about it, we think it's going to be a step forward in how we build AI systems."
AI models designed using the Pathways principle could gain a more profound understanding of the world and help solve "some of humanity's biggest problems." Dean cites drug development, more efficient education systems, and climate change as examples.
Renowned AI researcher François Chollet is less sanguine: he sees no significant progress toward general AI in the last decade - which is interesting since Chollet also works for Google and should be familiar with Pathways.
