Recently, there have been user reports that GPT-4 performance has degraded. According to an OpenAI staff member, this is not the case, at least for the API model.
There have been several reports on Hacker News and Reddit where users of ChatGPT and GPT-4 are complaining about poorer performance. Visual indications of some changes are that the GPT-4 logo in ChatGPT is now purple instead of black. Moreover, text generation is sometimes faster, which some people take as an indication of less computation.
No downgrade for the GPT-4 API, but ChatGPT is constantly changing
Logan Kilpatrick, who works with the OpenAI developer community, confirmed on Twitter that the GPT-4 model behind the API has been static since mid-March. Possible changes in the output are due to the non-deterministic nature of the technology.
The situation is different for the GPT-4 integration for ChatGPT, which according to Kilpatrick is "always changing". He says that as a heavy ChatGPT user with several hours a day, he hasn't noticed any negative changes and that OpenAI is open to feedback. As for the reports of poorer performance, however, there is "nothing specific" to say at this time.
I’m intrigued if you have any specific examples, as someone quite literally in ChatGPT many hours a day, still very useful to me.
As a side comment, this is one of the things I am most expected about with shared links.
You can keep testing prompts week after week.
Also, I…
— Logan.GPT (@OfficialLoganK) June 2, 2023
So, there you have it: All external GPT-4 apps should have the same performance as of mid-March, while ChatGPT is constantly changing, according to Kilpatrick, so fluctuations in performance are to be expected simply because of the non-deterministic nature of the technology.
Is it just the end of wow?
Other possible changes to the GPT-4 API model or its integration with ChatGPT may be related to OpenAI's plan to make GPT-4 more efficient. According to OpenAI CEO Sam Altman, increasing efficiency and reducing costs are priorities for the coming months.
An alternative explanation for GPT-4's perceived poorer performance is a more human one: the initial "wow" effect has worn off, and people now notice the AI system's weaknesses in text or code more than before. Hype is followed by disillusionment, then realism.
We asked OpenAI for a statement on the worse ChatGPT performance on 05/31/2023, but have not yet received a response.