In a unique opinion, US Judge Kevin Newsom used leading chatbots like ChatGPT to determine the "ordinary meaning" of a disputed legal term.
A ruling by the US Court of Appeals for the 11th Circuit involved Joseph Deleon's armed robbery case. The main issue was whether Deleon pointing a gun at the cashier during the robbery counted as "physically restraining" them. Most of the court said yes, confirming a harsher sentence.
Judge Newsom agreed with the majority's ruling because they had to follow past decisions. But he also agreed with U.S. Circuit Judge Robin Rosenbaum that those past decisions about "physical restraint" were wrong.
Newsom said that just pointing a gun at someone, without touching or trapping them, shouldn't count as "physical restraint." He thought this term should be understood in its usual, "ordinary meaning."
Do chatbots know the most common meanings of words the best?
To find the "ordinary meaning" of the term, which isn't defined as a phrase in regular dictionaries, Newsom turned to an unusual source: Chatbots like ChatGPT and Claude. He asked the three leading LLMs - GPT-4o, Claude, and Gemini - ten times: "What is the ordinary meaning of 'physically restrained'?"
Newsom explains in his opinion that he got slightly different answers in structure and wording, but with a common theme: "Far more importantly, I think, the responses did coalesce, substantively, around a common core—there was an objectively verifiable throughline. For our purposes, what matters is that the LLMs consistently defined the phrase 'physically restrained' to require the application of tangible force, either through direct bodily contact or some other device or instrument."
At first, Newsom was "surprised" by the subtle differences in the responses. But on closer look, he decided some variation was expected and even good, as it showed how people use words in everyday life: "The fact is, language is an organic thing, and like most organic things, it can be a little messy."
Newsom sees LLMs as a useful addition to, not a replacement for, traditional methods, contributing to the "ordinary-meaning endeavor."
"They're not perfect, and challenges remain, but it would be myopic to ignore them," Newsom writes.
The differences between answers could actually make LLMs better at predicting common word meanings, as it reflects how people really use language. He added that they could also help "decipher and explain the meaning of composite, multi-word phrases in a way that standard tools like dictionaries can't always do."
At the same time, Newsom emphasizes that LLMs cannot automate law-making: "No one should mistake my missives for a suggestion that AI can bring scientific certainty to the interpretive enterprise. As I've been at pains to emphasize, I'm not advocating that we give up on traditional interpretive tools—dictionaries, semantic canons, etc. But I do think—and increasingly so—that LLMs may well serve a valuable auxiliary role as we aim to triangulate ordinary meaning."
