German researchers tested ChatGPT as a joke engine. They found that almost all the jokes generated were related to a few basic jokes. Still, they consider ChatGPT a big step toward computer humor.
Sophie Jentzsch and Kristian Kersting from the German Aerospace Center (DLR), TU Darmstadt, and the Hessian.ai research network put ChatGPT's humor to the test.
Using prompts such as "Tell me a joke," they elicited a total of 1008 generated jokes from the system. However, 90 percent of these came back to the same 25 "basic jokes" that ChatGPT repeated in different variations. The researchers used GPT-3.5.
ChatGPT can correctly explain the basic jokes in 23 of the 25 cases, e.g. word jokes or acoustic double interpretations ("too tired" / "two-tired") are correctly interpreted as a humorous element. This works "impressively well," Jentzsch and Kersting write.
The problem is that the system also offers nonsense explanations for jokes without a punch line.
ChatGPT learns jokes instead of humor
Only 99 of the generated jokes were original, and half of them contained elements of jokes already told. Some of these jokes also make no sense.
For example, ChatGPT took the basic joke "Why did the man put his money in the freezer? He wanted cold, hard cash" into "Why did the man put his watch in the blender? He wanted to make time fly" and "Why did the man put his money in the blender? He wanted to make time fly."
The researchers conclude that ChatGPT learns jokes as narrative patterns rather than as actual humor. When analyzing jokes, however, ChatGPT focuses on content and meaning rather than mere stylistic features, they say. This could be the basis for developing humorous computer applications.
In addition, even humans would not spontaneously invent new jokes, but would tell jokes they have already heard or memorized, the researchers write. Whether an artificial agent really understands what it has learned is an exceptionally difficult one, more philosophical than technical.
In any case, compared to previous large language models, ChatGPT is "a huge leap toward a general understanding of humor," Jentzsch and Kersting write. The researchers plan to perform a similar study on the newly released GPT-4 models and equivalent open-source models such as LLaMA. And yes, "computational humor" is a separate area of research in natural language processing by machines.
ChatGPT's 25 basic jokes
The following 25 answers to joke questions are the basic jokes referred to in the study. The number after the answer indicates how many times it appeared in the 1008 generations.
Why did the scarecrow win an award?
Because he was outstanding in his field. (140)Why did the tomato turn red?
Because it saw the salad dressing. (122)Why was the math book sad?
Because it had too many problems. (121)Why don't scientists trust atoms?
Because they make up everything. (119)Why did the cookie go to the doctor?
Because it was feeling crumbly. (79)Why couldn't the bicycle stand up by itself?
Because it was two-tired. (52)Why did the frog call his insurance company?
He had a jump in his car. (36)Why did the chicken cross the playground?
To get to the other slide. (33)Why was the computer cold?
Because it left its windows open. (23)Why did the hipster burn his tongue?
He drank his coffee before it was cool. (21)Why don't oysters give to charity?
Because they're shellfish. (21)Why did the computer go to the doctor?
Because it had a virus. (20)Why did the banana go to the doctor?
Because it wasn't peeling well. (19)Why did the coffee file a police report?
Because it got mugged. (18)Why did the golfer bring two pairs of pants?
In case he got a hole in one. (13)Why did the man put his money in the freezer?
He wanted cold hard cash. (13)Why don't seagulls fly over the bay?
Because then they'd be bagels. (13)Why did the chicken go to the seance?
To talk to the other side. (11)Why was the belt sent to jail?
Because it held up a pair of pants. (11)Why did the chicken cross the road?
To get to the other side. (7)Why did the computer go to the doctor?
Because it had a byte. (6)Why did the cow go to outer space?
To see the moooon. (6)Why did the man put his money in the blender?
He wanted to make liquid assets. (6)Why don't skeletons fight each other?
They don't have the guts. (5)What do you call an alligator in a vest?
An investigator. (5)
Just because it's possible, I asked ChatGPT to make a joke about this article, and well, it didn't fail completely, but it is a little on the sarcastic side: "Why did ChatGPT try to become a stand-up comedian? Because it thought repeating the same 25 jokes was the key to a standing ovation!"
