OpenAI's ChatGPT has stunned people with compelling responses, reactions, and solutions - both positive and negative. The coding platform Stack Overflow, in particular, has experienced the negative side and temporarily banned ChatGPT.
We have reported on the positive aspects of ChatGPT, specifically its ability to handle user queries flexibly and contextually, even when they go beyond text generation.
But there is also the other side: ChatGPT has little or no connection to the real world and therefore sometimes generates ridiculous errors - which the system then presents in a confident manner. The following example illustrates this impressively.
- Angelo Zuffianò (@Agz5885) December 5, 2022
Stack Overflow responds to ChatGPT spam
Because of this error-proneness, Stack Overflow, a popular platform among programmers, has now banned ChatGPT until further notice. On Stack Overflow, programmers meet to discuss solutions to code problems. They often cite code examples.
The ban is temporary and applies to all areas of Stack Overflow. The exception is the "about me" texts in user accounts. Most ChatGPT-generated contents appeared in the comments, the mod team writes.
Users are no longer allowed to post ChatGPT-generated replies and lines of code in the comments. Since ChatGPT's launch on Dec. 1, "a large number of people are posting a lot of answers [...] (thousands)" of ChatGPT-generated replies, according to the mods.
These comments would have to be read in detail by a knowledgeable person to determine if they contained false information. This "swamped" the current quality curation infrastructure.
It is simply too easy to use ChatGPT to generate answers that are very likely to be wrong, but appear as if they are the solution, the moderator team writes in a statement.
"Because the average rate of getting correct answers from ChatGPT is too low, the posting of answers created by ChatGPT is substantially harmful to the site and to users who are asking or looking for correct answers."
Next steps are to be discussed with Stack Overflow employees and users.
Are large language models harmful or helpful? Discussions become more intense
A similar discussion about the content quality of large language models and especially about AI-generated fake information has been going on at least since the introduction of GPT-2. At that time, OpenAI announced that it would only release the language model incrementally to prevent the mass generation of fake content.
Releasing the full GPT-2 version was "too dangerous," the company said in February 2019. Today, the much more powerful GPT-3 is available to everyone, and it appears that the Fake News dystopia has not yet come true.
Recently, a Meta project was hit by LLM criticism: A demo of the Galactica scientific language model had to be taken offline a few hours after its presentation. There were vocal protests from the research community that Galactica published false information with a high degree of conviction and also linked it to partially made-up citations.
Proponents of large language models argue that these problems can be solved primarily through practical testing. Moreover, they argue, the models are powerful autocompletion that was not invented to replace humans, who would still have to perform verification and inference.
Stack Overflow's decision against ChatGPT content, of all things from a coding platform whose target audience should be open to such innovations, rather strengthens the positions of the system's critics. Considering the rapid development of language models, however, it is only a snapshot.
