How to use AI but steer clear of hallucinated cases

If you ask lawyers for their best advice on using artificial intelligence in their practices, some will suggest it be avoided at all costs. But despite the many news stories about attorneys getting caught citing hallucinated cases in court filings, there are ways to use the technology cautiously and wisely, lawyers and technologists tell the ABA Journal. "If you give AI a prompt, and it gives you some stuff that sounds great but is suspiciously specific, ask it some follow-up questions. It could even be something general, like 'How confident are you in that?' Ask it to vet its own answers. Where are the holes? What are the problems? Flag the issues and put AI to work," —Bobby Williams Jr., director at iDiscovery Solutions, who trains attorneys on proper AI use "There has to be a method and strategy in place so that how the authorities are cited—the facts and quotations, the analysis, all of it—was independently verified." —Jean Cha, Cha Law Ethics "Just like when you’re using Westlaw, you don’t just take the case cite because it came up in your query. You go back and read the cases. You can’t just say, ‘Oh, AI gave me this without verifying that the case actually exists.” —Julie Bennett, litigator/investigator for the Indiana Supreme Court Disciplinary Commission and president of the National Organization of Bar Counsel “Although the instances of lawyers citing hallucinated cases are the ones that tend to make the news, more nefarious and less reported are the instances where AI mischaracterizes a real case. Ultimately, nothing substitutes for actually reading the cases you cite.” —David S. Kemp, AI innovation team, Ropes & Gray “The best way to use AI is in the middle of a project to see if I missed something along the way. But it should never supplant our judgment. It’s not infallible and may lack context.” —David Majchrzak, partner and general counsel at Rosing Pott & Strohbehn and president of the Association of Professional Responsibility Lawyers “Read every case your colleague on the other side has cited, and if she or he has cited an hallucinated case or an inaccurate quotation, call and ask them to correct the submission. If you don’t—or if you do and the citation isn’t corrected—then you are as responsible as that other lawyer for the inaccuracies.” —Dan D. Kohane, member, Hurwitz Fine “Push back on the AI. Challenge it about the information it provides. It may well admit it was wrong, though not always. Don't be afraid to change the prompt or use the same prompt more than once to see if the results change. Since AI is based on predictions, the same prompt may well surface additional information or errors. You can also ask the AI if it is certain about the data it provided you.” —Jennifer Ellis, lawyer and legal technology consultant “Don't use general-purpose chatbots for caselaw research. Tools like ChatGPT and Anthropic’s Claude generate text by probability, so they produce citations that look real without knowing whether the case exists. They mimic the shape of a citation, not its substance. For research, use tools grounded in actual legal databases. Now the frontier models like ChatGPT are getting better at avoiding fake cases. But regardless, they should never be relied upon.” —Josh Kubicki, visiting lecturer, University of Indiana Maurer School of Law For an overview on how state attorney regulation agencies are approaching lawyers' use of AI, click here.