Australian courts are seeing a rise in documents seemingly created by AI systems like ChatGPT, often containing major errors.
Judges in various courts have criticized individuals, companies, and lawyers for submitting applications with non-existent case citations or nonsensical content, reports the Australian news portal Crikey.
While judges can't always confirm AI use, telltale signs include fabricated legal material and distinctive language patterns. In one instance, a lawyer admitted to using "legal software" that produced a list of non-existent cases. Another case saw a legal representative submit 600 pages of "rambling, repetitive, nonsensical" material.
Private citizens also appear to be using AI in court. Judges have disregarded personal references they suspected were AI-generated. In a Tasmanian Supreme Court appeal, Justice Alan Michael Blow found the appellant cited a non-existent case.
Some documents even contained ChatGPT prompts. The Australian Competition and Consumer Commission (ACCC) noted that one company's resolution ended with "use British spelling please ChatGPT".
LLMs undermine the legal system
This issue extends beyond Australia. Lawyers worldwide have faced dismissals and penalties for using AI-generated misinformation in court. A Stanford University study found AI legal research tools make errors in one out of six requests, highlighting the need for careful review of AI-generated information. Legal regulators and courts have responded with guidelines or bans on AI use.
However, the large number of examples suggests that there may be many more unreported cases. This underscores the importance of educating people to understand the capabilities and limitations of AI systems. Currently, much of the limited education is driven by those with commercial interests.
