Prompt engineering in LLMs is finding the right vector program

LLMs (Large Language Models) like OpenAI's GPT-4 act as repositories for millions of vector programs mined from human-generated data learned as a by-product of language compression, says AI researcher François Chollet. Prompt engineering then involves searching for the right "program key" and "program argument(s)" to accomplish a given task more accurately. Chollet expects that as LLMs evolve, prompt engineering will remain critical, but can be automated for a seamless user experience. This is in line with recent ideas from labs such as Deepmind, which is exploring automated prompt engineering.

My interpretation of prompt engineering is this:

1. A LLM is a repository of many (millions) of vector programs mined from human-generated data, learned implicitly as a by-product of language compression. A "vector program" is just a very non-linear function that maps part of…

— François Chollet (@fchollet) October 3, 2023