According to a study by researchers at Harvard University and Graz University of Technology, GPT-4 performs better than humans in the cognitive reappraisal of negative situations.
In two studies, scientists at Harvard University investigated how well humans and GPT-4 can reframe negative situations. This "cognitive reappraisal" is an effective method for reducing negative feelings in different contexts.
The researchers had human subjects and the GPT-4 reinterpret a total of 18 negative situation descriptions that expressed either sadness, fear, or anger. Other subjects then rated the quality of the reinterpretations on the dimensions of effectiveness, empathy, novelty, and specificity.
In both studies, the researchers found that GPT-4 outperformed humans on effectiveness, empathy, and novelty. There was no significant difference between humans and GPT-4 in the consistency of the reappraisal with the original situation description (reappraisal specificity).
In the second study, participants were motivated to provide better reinterpretations with bonus payments. Although participants spent more time on their reinterpretations as a result, the difference in quality between humans and GPT-4 did not decrease. This suggests that the difference is due to ability rather than effort, according to the researchers.
An analysis of the semantic similarity between the original situation descriptions and the reappraisals showed that the human reappraisals tended to be more similar to the original descriptions. Humans stuck much more closely to the described situation rather than giving a general answer. The opposite was true for GPT-4.
The results suggest how both human and GPT-4 reappraisals can be improved: Humans can learn to go beyond the specific situation and provide a broader interpretation, while GPT-4 could benefit from focusing more on the details.
AI could assist humans with reappraisals
The researchers emphasize that the findings have implications for the use of AI in reappraisal and mental health support. Models such as GPT-4 could improve human reappraisals by providing suggestions for improving initial human reappraisals.
However, the studies have several limitations. They focused on reappraising very specific situations created by others and were limited. Humans may outperform GPT-4 in more complex situations, and participants were unaware that some reappraisals were made by an AI. Previous research has suggested that this may influence evaluations.
A summer 2023 study supports the Harvard researchers' findings, showing that GPT-4 can more accurately describe the possible range of emotions in a given scenario than humans. But that doesn't automatically make ChatGPT a better psychotherapist.
