Prompt engineering in LLMs is finding the right vector program
LLMs (Large Language Models) like OpenAI's GPT-4 act as repositories for millions of vector programs mined from human-generated data learned as a by-product of language compression, says AI researcher François Chollet. Prompt engineering then involves searching for the right "program key" and "program argument(s)" to accomplish a given task more accurately. Chollet expects that as LLMs evolve, prompt engineering will remain critical, but can be automated for a seamless user experience. This is in line with recent ideas from labs such as Deepmind, which is exploring automated prompt engineering.
My interpretation of prompt engineering is this:
1. A LLM is a repository of many (millions) of vector programs mined from human-generated data, learned implicitly as a by-product of language compression. A "vector program" is just a very non-linear function that maps part of…
— François Chollet (@fchollet) October 3, 2023
AI News Without the Hype – Curated by Humans
As a THE DECODER subscriber, you get ad-free reading, our weekly AI newsletter, the exclusive "AI Radar" Frontier Report 6× per year, access to comments, and our complete archive.
Subscribe nowAI news without the hype
Curated by humans.
- Over 20 percent launch discount.
- Read without distractions – no Google ads.
- Access to comments and community discussions.
- Weekly AI newsletter.
- 6 times a year: “AI Radar” – deep dives on key AI topics.
- Up to 25 % off on KI Pro online events.
- Access to our full ten-year archive.
- Get the latest AI news from The Decoder.