Content
summary Summary

Harvard University student Maya Bodnick wanted to see if GPT-4 could pass Harvard's first-year home essays in the humanities and social sciences.

Ad

Bodnick had GPT-4 write seven essays on topics such as economic concepts, presidentialism in Latin America, and a literary analysis of a passage from Proust.

She then gave the essays to the professors to grade, indicating for bias purposes that the essay had been written either by her or by GPT-4. In fact, GPT-4 wrote all of the essays using the essay assignment as a prompt.

The result: A. A. A. A-. B. B-. Pass. GPT-4 earned a "respectable" 3.57 GPA, according to Bodnick.

Ad
Ad

The student said she submitted GPT-4's answers unedited, but with two caveats: She pieced together the essay from multiple answers to meet the word limit, since GPT-4 generates a maximum of 750 words at a time.

She also asked the reviewers to ignore the missing citations that GPT-4 could not provide. These would normally be part of the grade.

Great grades with special praise and little criticism

In addition to some excellent grades, reviewers gave special praise such as "beautifully written" or "well articulated paper." In one paper on conflict resolution, the reviewer criticized a flowery writing style with too many adjectives and metaphors.

In terms of content, the reviews were also positive, for example, praising the high level of detail or the structured approach to each question. The lowest grade of "B-" was given to the paper on Latin American presidentialism, where the reviewer criticized the paper for ignoring positive assessments of presidentialism as well as economic aspects.

"I think we can extrapolate from ChatGPT-4’s overall solid performance that AI-generated essays can probably get passing grades in liberal arts classes at most universities around the country," Bodnick writes. At Princeton or UC Berkeley, these grades might have been B's and C's instead of A's and B's. GPT-4 would still have passed. The seven essays can be viewed here.

Recommendation

A new learning and teaching paradigm

AI will entirely change the way the humanities and social sciences are taught, Bodnick believes. It has never been easier to cheat on a homework assignment, she said, and the technology is likely to get even better at things like citation.

Recently, major AI companies volunteered to develop a watermarking system. However, OpenAI withdrew its detector due to a lack of accuracy, and OpenAI CEO Sam Altman has consistently expressed doubt that an AI text recognition system will work in the medium to long term.

So testing will have to return, at least in part, to face-to-face interviews rather than essays, says Bodnick. She sees the fact that GPT-4 was able to write her essays as an indication of possible developments in her future profession.

That's why, she says, it's important for the education system to use her findings to think about how careers are changing and how to prepare for them - rather than trying to identify and punish AI essays.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.

"My gut reaction is that liberal arts majors — who spend most of their academic career writing essays — are going to face even greater difficulties in a post-AI world. AI isn’t just coming for the college essay; it’s coming for the cerebral class."

Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • Harvard student Maya Bodnick tested GPT-4 by having it write essays for introductory humanities and social science courses. The submitted essays earned a "respectable" 3.57 GPA.
  • Reviewers praised the clarity and structured style of the AI-generated essays, and were pleased with the overall quality of the content.
  • Bodnick emphasizes that the use of AI such as GPT-4 in essays is changing the way humanities and social sciences are taught. She recommends an emphasis on face-to-face conversations and preparation for future careers in an AI-driven world.
Sources
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.