Microsoft researchers have developed a new way to improve the effectiveness of few-shot prompting, a useful technique for guiding the behavior of AI models. The approach, called dynamic few-shot prompting, aims to make the process more efficient and scalable.
Few-shot prompting typically involves providing AI models with a small set of examples directly in the prompt. This method has proven effective in getting large language models (LLMs) to perform tasks reliably. However, as the number of examples grows, it can become unwieldy.
How dynamic few-shot prompts work
To address this issue, Microsoft employee Franklin Lindemberg Guimarães proposes storing examples in a vector database instead of including them in the prompt itself. The system retrieves only the most relevant examples for each user input, keeping the prompt lean and more adaptable to various tasks.
The dynamic few-shot prompt system consists of three main components:
- Vector store: Holds example prompts as input-output pairs, indexed by input
- Embedding model: Converts user input into a vector for querying the store
- AI model: Generates responses based on the prompt and examples
This approach offers several advantages. By selecting only the top three most relevant examples for each query, the system avoids overwhelming the model with irrelevant information. It also improves response accuracy and relevance while reducing processing costs by using fewer tokens.
Guimarães describes a use case where a chatbot performs three tasks: displaying data as a table, classifying text, and summarizing text. The dynamic few-shot prompting method selects the most appropriate examples for each user request. This could make prompts more versatile for similar tasks and reduce the number of prompts in your prompt library.
The technique seems particularly promising for many-shot prompts, where managing a large number of examples can be challenging. By combining intelligent example selection with the proven effectiveness of many-shot prompts, it could lead to more reliable and capable AI systems for specific tasks.
