OpenAI releases a new language model for GPT-3 trained with human feedback. It brings numerous improvements, according to OpenAI.
The new GPT-3 model "text-davinci-003" is based on the InstructGPT models introduced by OpenAI earlier this year, which are optimized with human feedback. These models have already shown that AI models trained with RLHF (Reinforcement Learning from Human Feedback) can achieve better results with the same or even lower parameters.
According to OpenAI alignment researcher Jan Leike, "text-davinci-003" is largely equivalent to the InstructGPT models, but is not identical. The new model "scores higher on human preference ratings without being fundamentally more capable" than the underlying base model. For fine-tuning, OpenAI required "very little compute and data to align it compared to pretraining".
Leike points out that the new GPT model still has "important limitations" and, for example, sometimes simply makes up things. However, such missteps should now "hopefully" be less frequent and less serious.
The model definitely still has important limitations! Basically all of the ones from our previous blog post will still apply, but hopefully less severe / more rarely now.https://t.co/r8zWWYSIJ6
- Jan Leike (@janleike) November 28, 2022
OpenAI first unveiled the human feedback training process in September 2019 based on GPT-2. The AI company considers human feedback in the AI training process, and thus the alignment of AI with human needs, to be the foundation of human-friendly AI.
New GPT-3 model generates better text and can handle more complex instructions
"text-davinci-003" can generate "clearer, more engaging, and more compelling content" and handle more complex instructions, according to OpenAI.
For instructions, users not only simply tell GPT-3 what text to generate, but also provide text examples or describe steps to perform. Through these one-shot prompts, users discovered capabilities of the model beyond just text generation that were not envisioned by OpenAI.
For example, GPT-3 can control Excel, translate text, or reason in chains of thought. Such emerging capabilities are a phenomenon of large language models. According to OpenAI, the new language model offers more creative leeway in this area.
"text-davinci-003" can also write longer texts, according to OpenAI. As a result, the language AI can now take on tasks that were previously unfeasible. The following example compares the same prompt between the old (002) and the new model (003).
text-davinci-002:
I am trying to write a web application. What backend frameworks could I use, and what are their pros and cons?
There are many backend frameworks that could be used for a web application. Some popular ones include Django, Ruby on Rails, and Express.js. Each of these frameworks has its own pros and cons, so it is important to choose one that will fit the specific needs of your application.
`text-davinci-003:
I am trying to write a web application. What backend frameworks could I use, and what are their pros and cons?
1. Node.js: Node.js is a JavaScript-based runtime environment that is popular for writing backend applications. Pros: Node.js is fast, lightweight, and easy to learn, and it has a large community of developers. Cons: Node.js can be difficult to debug and there is a lack
of robust tools for building large-scale applications.2. Django: Django is a popular Python-based web framework. Pros: Django is powerful and well-documented, making it easy to get started. It also supports a wide range of databases and APIs, allowing you to easily integrate with other services. Cons: Django can be difficult to scale for large applications and can be slow for certain types of requests.
3. Ruby on Rails: Ruby on Rails is a popular web framework based on the Ruby programming language. Pros: Ruby on Rails is easy to learn and can be used to quickly build powerful web applications. It also has a wide range of built-in features, such as authentication and authorization. Cons: Ruby on Rails can be slow and resource-intensive, and it can be difficult to scale.
Pricing for the new model is the same as before. According to rumors on Twitter and Reddit, OpenAI could introduce the GPT-3 successor GPT-4 in the first quarter of 2023. OpenAI is currently running a funding program that will probably give the first participants access to GPT-4.
OpenAI co-founder Wojciech Zaremba described potential new capabilities of upcoming language models in September. He believes AI will have a similar or even greater impact on human communication than the Internet.
