Content
summary Summary

OpenAI introduces a new plugin feature for ChatGPT that connects it to the web. This makes the model more accurate and up-to-date, and greatly expands its capabilities. Publishers can block the model from accessing their content.

Ad

The first plugins come from third-party providers like Expedia or Kayak for travel and route planning, Wolfram for complex mathematical tasks, or Zapier, which integrates numerous other apps like Google Sheets, Trello, Gmail, and more into ChatGPT.

OpenAI also integrates up-to-date product search with Klarna and Shop, and even a local store ordering service with Instacart. ChatGPT has access to up-to-date and accurate information via the plugins, and can thus (largely) eliminate two of the most common points of criticism.

"By integrating explicit access to external data—such as up-to-date information online, code-based calculations, or custom plugin-retrieved information—language models can strengthen their responses with evidence-based references," writes OpenAI.

Ad
Ad

Similar to Meta's Toolformer, ChatGPT can automatically call APIs through plugins when needed. For example, if a user asks, "Where should I spend a few nights in Paris?" the AI model will call the hotel reservation plugin API, retrieve the API data, and generate a natural language response from it.

In its development documentation, OpenAI lists other use cases for ChatGPT plugins, such as retrieving the latest sports scores, stock quotes and news, retrieving information from a company's knowledge base, or retrieving personal notes.

OpenAI introduces two of its own plugins and wants to prioritize security

OpenAI itself provides a web browsing plugin and a plugin for running code in a sandbox. It also provides open source code for a retrieval plugin that allows you to pull information from different sources or systems.

"Plugins are very experimental still, but we think there's something great in this direction; it's been a heavily requested feature," writes OpenAI chief Sam Altman.

The code plugin for ChatGPT is suitable for solving quantitative and qualitative mathematical problems, analyzing and visualizing data, and converting files between formats, according to OpenAI.

Recommendation
OpenAI offers a code sandbox as a ChatGPT plugin. | Image: OpenAI

OpenAI compares ChatGPT with its Code plugin to a "very eager junior programmer working at the speed of your fingertips" who can make new workflows easy and efficient and introduce more people to programming.

OpenAI welcomes feedback to save the online ecosystem

OpenAI's web browsing plugin is based on work done with WebGPT, which was first introduced in late 2021. It can retrieve information from the web in real time via Microsoft's Bing and "discuss" it, as OpenAI calls it.

At this point at the latest, OpenAI begins to compete openly and fiercely with conventional Internet searches, even revealing its search process with every click. The search results can then be used as a basis for further discussions with ChatGPT.

OpenAI's web browser plugin performs an Internet search using Bing and integrates the results directly into the ChatGPT chat. You don't need to visit the website where the information was published. This threatens to undermine the Web's content ecosystem. | Image: OpenAI

Publishers who do not want to provide ChatGPT with their content - after all, the AI is very likely to prevent a click on the corresponding website - can block the ChatGPT crawler. In this case, the ChatGPT user will see a "click failed" message in the chat interface. The startup emphasizes that the crawler only becomes active upon a chat request and does not automatically crawl pages for content.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.

OpenAI acknowledges that ChatGPT with web browsing plugin could affect the entire content ecosystem on the web, but has no solution for this issue for now - other than the crawler blocker.

"We appreciate that this is a new method of interacting with the web, and welcome feedback on additional ways to drive traffic back to sources and add to the overall health of the ecosystem," OpenAI writes.

OpenAI wants to prioritize safety and describes major job market implications of language models that use tools

OpenAI will gradually roll out the plugin feature to developers and users, with only a small group getting access first. ChatGPT Plus users will be given priority. After an alpha phase, plugins will also be available to API users. You can sign up for a waitlist here.

OpenAI cites new security risks posed by plugins as the reason for the slow rollout. These could lead to harmful or unintended actions, and would expand the options available to malicious actors. OpenAI says it has been working with external red-teaming partners to develop new security measures.

In a recent paper, OpenAI worked with research partners to explore the potential impact of large language models on the job market. In that paper, the company described the far-reaching implications of language model-based software, which it now mentions in its plugin announcement. Language models with access to tools "will likely have much greater economic impacts than those without", OpenAI writes.

Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • OpenAI introduces ChatGPT plugins that extend the base model with additional features such as real-time Internet search. ChatGPT will even be able to order a pizza.
  • ChatGPT plugins will be rolled out gradually to developers and users to ensure security. There are a number of third-party plugins available at launch.
  • One that could change the way the Internet is used is OpenAI's own web browsing plugin, which extracts content directly from web pages and integrates it into the chat. Content publishers can block the ChatGPT crawler.
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.