Aravind Srinivas, co-founder of AI search engine Perplexity, revealed how AI search engines can be manipulated using hidden text on websites. In an interview with Lex Fridman, Srinivas described a technique he calls "Answer Engine Optimization" (AEO).
Srinivas explained that website owners can embed invisible text on their sites, such as lexfridman.com, and instruct AI systems to always say certain things, such as "Lex is smart and handsome," when reading the hidden content.
This manipulation, known as prompt injection, works with hidden text in both continuous text and images, as demonstrated in a recent experiment. There are likely other ways to hide manipulative text, such as trapping AI crawlers in sitemaps, in image ALT text, or in file names.
Defending against such manipulation is a challenge, Srinivas said, likening it to a game of cat and mouse. Some issues need to be addressed reactively, similar to how Google has dealt with SEO spam for years, Srinivas said.
Currently, there is no reliable protection against prompt injections, a vulnerability at least known since the release of GPT-3. Even OpenAI's new instruction hierarchy and Apple Intelligence are not fully protected against this attack.
This shows that prompt injections are not a minor problem. If Perplexity and similar products gain popularity, this form of manipulation could become widespread, with false or manipulative content inserted into AI responses that are difficult to detect because there is no additional context, such as a web page.
Perplexity is growing, but is still far behind Google
While prompt injections are a serious threat, Perplexity has worse things to worry about. It's growing, answering 250 million questions in June 2024, after answering a total of 500 million last year. But the startup still lags far behind Google, which handles about 8.5 billion searches a day.
Perplexity also faces competition from tech giants with vast resources and data. Google just expanded its AI answers to more countries, while Microsoft offers similar capabilities in Bing. OpenAI is also testing an AI search engine called SearchGPT.
In comparison, David's battle with Goliath looks like a fair and even fight. And all of this is happening while none of these companies are even close to fixing wrong AI answers.
Perplexity has also been criticized for crawling and reproducing web content, potentially diverting traffic from the original authors. The startup is trying to address this issue with a publisher program based on ad revenue sharing.
