Physicist David Deutsch argues that you can't test for AGI the way you test a piece of software. Instead, what sets humans apart is that they have their own history, shaped by choosing what matters to them in the first place.
"How do we know that people are general intelligences? By telling their story. Human thinking is not about mechanically translating motivations into actions - prompts into output. It is primarily about choosing motivations. Just as science is not about extracting theories from data, but about seeing a problem, guessing at explanations, criticizing and testing them."
Deutsch responds to a test proposed by OpenAI CEO Sam Altman: if a future GPT-8 could solve quantum gravity and explain why it picked that problem and how it arrived at its solution, Deutsch says that would be convincing evidence of general intelligence.
