Content
summary Summary

Prompt2Model automates the generation of special-purpose NLP models that, in some cases, can outperform GPT-3.5 Turbo while being up to 700 times smaller.

Ad

Researchers at Carnegie Mellon University and Tsinghua University have developed a new system called Prompt2Model that can generate custom language models from prompts. The system aims to make the development of specialist AI models accessible to non-experts. Prompt2Model is not meant to be a GPT-4 alternative, but rather an automated pipeline for special-purpose NLP models that perform a particular task very well, are much smaller than large models, and can therefore run locally on weaker hardware.

The system first decomposes the prompt into a structured statement. It then looks for data sets that might be useful for the task at hand and uses OpenAI's GPT-3.5 Turbo to generate additional synthetic training data tailored to the task. It then identifies an appropriate pre-trained model for fine-tuning the hugging face and trains it on the collected data.

After training, Prompt2Model can create a web interface to interact with the model. The modular design allows customization of each pipeline component.

Ad
Ad

Prompt2Model shows promising results

The team evaluated the results of Prompt2Model in three benchmarks. In two tasks (SQuAD, Temporal), the resulting Flan-T5 models outperformed even GPT-3.5 Turbo, even though the Google model has almost 700 times fewer parameters. In the third benchmark (MCoNaLa) Prompt2Model was clearly behind the OpenAI model.

Prompt2Model has difficulty supporting tasks that require languages other than English, according to the team. The team cited GPT-3.5-Turbo's limited language support as the reason.

The fact that the team uses the OpenAI model to generate data is also probably Prompt2Model's biggest limitation, as OpenAI prohibits the use of its own models to train models that might compete with it, making Prompt2Model unusable for commercial applications. However, the team is exploring the integration of large open-source language models to get around the reliance on proprietary APIs.

More information and the code is available on GitHub.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Researchers at Carnegie Mellon University and Tsinghua University have developed Prompt2Model, a system that creates special-purpose NLP models.
  • Prompt2Model uses OpenAI's GPT-3.5 Turbo to generate synthetic training data and then uses pre-trained Hugging Face models for fine-tuning.
  • Reliance on OpenAI's GPT-3.5 Turbo is a limitation for commercial applications - alternative open-source language models are being investigated.
Sources
Max is managing editor at THE DECODER. As a trained philosopher, he deals with consciousness, AI, and the question of whether machines can really think or just pretend to.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.