Advice About AI for New Law Students

On this site last week, Professor Vikram Amar offered some tips for new law students. I agree with his sage advice and have given my own somewhat overlapping advice in the past. In Professor Amar’s latest column, he cautions students against over-use of Artificial Intelligence (AI). I agree with that advice as well, but the topic merits further detailed consideration. Accordingly, in this column, I offer some dos and don’ts when it comes to AI usage during your time as a law student. Don’t Cheat Others In the past several years, nearly every institution of higher education has developed and adopted policies regarding AI usage in classrooms, when studying, on exams, and for papers. In addition, individual instructors often have further policies. The rules regarding AI serve multiple purposes but one overriding goal is to prevent cheating. The concern is hardly hypothetical. Last spring, a Brown University economics professor discovered that most of his students likely cheated by using AI on a take-home midterm exam. Law schools and higher education more generally are responding with new approaches. Princeton University is abandoning the honor system it long employed in favor of proctored exams. Subject to disability accommodations and other limited exceptions, the University of Chicago Law School is banning “the use of electronic devices such as laptops, tablets, and phones in the classroom” across the first-year curriculum. Cornell Law School, where I teach, has adopted new procedures to replace the new procedures we adopted just a few years ago, when large language models first became available. It is tempting to say that students who cheat with AI (or some other way) are only cheating themselves, but that simply is not true. They are cheating themselves (about which more in a moment), but they are also cheating their classmates, employers, and their future clients. Many aspects of law school are competitive. Students compete for grades, which affect whether they are selected for law review, prestigious clerkships, and other employment opportunities. A student whose grades do not reflect their legal acumen takes a place from a more qualified student. Cheating at any educational level is unethical, but it is especially problematic in law school. Lawyers occupy positions of trust. That is why, to practice law, one must not only fulfill the academic requirements and pass the bar exam but also be certified by their law school for “character and fitness.” The AI-related rules that law schools have in place are undoubtedly flawed in various respects. But so long as those rules exist, you should follow them in letter and spirit. Don’t Cheat Yourself I recently learned that law students in some classes (perhaps including ones I have taught) would type an instructor’s query into a chatbot and recite the answer the chatbot spits out. If doing so fools the instructor, that is certainly impressive, but it completely misses the point of the Socratic method. Instructors pose questions for various reasons: to ensure that students have done the reading; to ensure they have understood it; and most essentially, to train students in critical thinking. Typing a question into a chatbot and vocalizing its response serves none of these purposes. From time to time, one of my students will come up to me after class and say something like this: I understood the material when I came to class, but now I’m confused. Unless I’m a much worse teacher than I’ve been led to believe, that kind of statement reveals that the student had at best a superficial understanding of the material when they came to class. Often the goal of law school instruction is to show students that there are deeper levels of complexity, conflict, and indeterminacy in the law than a casual reading might suggest. In all my classes, I aim to model and give students practice in identifying these deeper levels so that once they are lawyers they can do it for themselves. If you use an LLM to answer Socratic questions in class, you will miss a central element of your legal education. In so doing, you really are only cheating yourself. Don’t Trust but Do Verify Many lawyers now use AI in one way or another in their practice. Don’t students need to learn how to use it effectively and responsibly? Indeed, they do, and you will begin to learn how to do so in your legal research and writing class (which may go by a different name at your law school. For example, at Cornell, we call it Lawyering.) If the wisdom of reasonably informed crowds is to be believed, we could well have Artificial General Intelligence (AGI) by 2031. A capable AGI should be able to do whatever intellectual work a human can, including most tasks lawyers perform. That is undoubtedly scary news for new law students who will be entering the legal workforce in 2029. Perhaps it means you’ve made the wrong decision in choosing law school rather than, say, training to be an elevator mechanic or AI ethicist. But if you have already bet that there will be something for lawyers to do that AI cannot do equally well, you need to be able to learn how to do that something. You also need to be able to evaluate the output of AI. Thus far, I’ve pointed to the downsides of AI for legal education, but there is little doubt that, used judiciously, it can make lawyers more effective. In my own work, for example, I find both general-purpose and law-specific LLMs most useful for giving me a general sense of an area of law or a specific topic with which I’m not familiar. Research tasks that might have taken me an hour using pre-LLM computer tools now take about fifteen minutes. But some of that time savings is eaten up by the essential step of checking the LLM’s work. General-purpose chatbots hallucinate, making it absolutely inexcusable that hundreds of lawyers in the U.S. have been found to have included fabricated quotations and citations in official filings. And those are just the ones that were detected! Law-specific LLMs are somewhat less likely to hallucinate because of deliberate design decisions their developers made to mitigate these types of errors, but they can make mistakes too. In my experience, they are quite good at identifying relevant cases but not at all good at accurately explaining exactly what a case held. Someday, maybe even someday soon, LLMs will be more trustworthy, but a good lawyer checks (and double-checks) all of their citations before filing a legal document regardless of whether it was produced by a human or an AI. Ours is a profession that places great weight on legal authority. You will learn to pay attention to every word in a complicated statute or regulation. You certainly shouldn’t take at face value everything a stochastic parrot tells you, no matter how smart the parrot. Do Give Grace to Your Elders During the oral argument in a 2023 case involving Google’s liability for ISIS content on YouTube, Justice Elena Kagan at one point underscored the limits of the Supreme Court’s ability to fashion appropriate rules. Referring to herself and her colleagues, she said, “these are not like the nine greatest experts on the internet.” Justice Kagan’s particular point was that the Court should hesitate to fashion a new rule, leaving that task for Congress instead, but she might also have been making a broader point: people trained in law are not experts in computer technology but must nonetheless sometimes make decisions about how it is used and regulated. (In the particular case, even a decision to leave matters to Congress was a decision.) Accordingly, please have some patience with and sympathy for your instructors. We have considerable experience in law school pedagogy, but nearly all of that experience dates from a pre-AI world. I am hardly the oldest member of my faculty, but for perspective, when I started law school (1987), the World Wide Web had not yet been created (1989). LLMs have been widely available for less than four years. We are doing the best we can to adapt to rapidly changing technology that is already transforming the practice of law.