OpenAI unveils Reinforcement Fine-Tuning to build specialized AI models for complex domains

OpenAI is expanding its custom AI training offerings with a new method called Reinforcement Fine-Tuning (RFT). The technique aims to create specialized o1 models that can perform complex technical tasks with minimal training examples.

The new approach works differently from traditional supervised fine-tuning. Instead of just learning to copy the style and tone of training data, models can develop new ways of "thinking" through problems, according to OpenAI. When given a problem, the model gets time to work out a solution o1 style. An evaluation system then rates the answer - strengthening successful reasoning patterns while weakening incorrect ones.

OpenAI says this approach works especially well for specialized fields like law, finance, engineering, and insurance that need deep technical knowledge. As an example, the company highlights its collaboration with Thomson Reuters, where they trained the compact o1 Mini model to work as a legal assistant.

Reinforcement learning for expert systems

Justin Ree, a bioinformatician at Berkeley Lab, used RFT to study rare genetic diseases. He trained the system using data extracted from hundreds of scientific papers that included symptoms and their associated genes.