Is GPT-4 getting worse? Peter Welinder, VP Product at OpenAI, comments on the rumors: "No, we haven't made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one," says Welinder. His hypothesis: "When you use it more heavily, you start noticing issues you didn't see before."
Still, he asked on Twitter for examples of where users felt GPT-4's performance had regressed. OpenAI will look into this, Welinder promises. One example from the comments, where GPT-4 only gives the correct answer after a second attempt, he claims is a bug.
Field reports of GPT-4 performance degradation have been around for a few weeks now, often pointing to a performance difference between access via API and via ChatGPT, where the OpenAI language model integration is regularly adjusted. The opacity of this process may also contribute to uncertainty among users.
No, we haven't made GPT-4 dumber. Quite the opposite: we make each new version smarter than the previous one.
Current hypothesis: When you use it more heavily, you start noticing issues you didn't see before.
— Peter Welinder (@npew) July 13, 2023