Should Law Schools Teach Responsible Use of AI?

No.

I could stop there and write my shortest-ever Substack, but I probably should say a little more.

To clarify, I’m not in favor of teaching the irresponsible use of AI, or teaching students to employ it without regard to safeguards and best practices to ensure lawyers’ compliance with their duties to clients and tribunals. But I think it is at best a misallocation of time and resources, and at worst a disservice to students, for law schools to attempt to train students on the use of AI tools as part of becoming a “practice ready” lawyer.

There is a movement within legal education to take on responsibility for educating law students about the responsible use of AI in the practice of law. First out of the gate, not surprisingly given its location, was the University of San Francisco, which announced a partnership with Accordance and Anthropic to integrate GenAI into the first-year curriculum. Suffolk Law School, which under Dean Andy Perlman has long emphasized the use of technology in the practice of law, integrated the Hotshot AI platform into its mandatory 1L legal skills course. This May, UNLV Law School announced a new course on the responsible use of AI in legal practice. The common refrain is that AI-savvy graduates will have a leg up in the legal employment market.

An article by a “content strategist” (God help us) from Thompson Reuters hits the tiresome theme of “AI is coming, whether we like it or not”:

The debate about AI consuming most of the work that teaches essential lawyering skills to junior attorneys is forcing a reckoning with the long-held assumption that law schools were never designed to produce practice-ready lawyers and that it was always the profession’s job.

Indeed, AI is forcing that uncomfortable truth into the open faster than anyone anticipated because essential lawyering work — the document review, contract markup, research memo creation — dictated how a junior lawyer learned to spot the issue buried on page 47, to sense when a clause was off, and to develop the instinct that no classroom can fully replicate. Now, as more law firms deploy AI to handle precisely those entry-level tasks, the organic training moments that used to define the first two to three years of legal practice are evaporating.

The debate about training “practice-ready” lawyers has indeed been going on for decades, and I mostly don’t want to relitigate those issues here. But I do want to make a brief argument that, specifically with regard to AI, legal educators should not assume that we are in the best position to provide the practical training that recent law school graduates need. The reason is threefold, with each branch related to professional competence: (1) the vast majority of us law school faculty, even those with relatively recent practice experience or those in clinical or skills-training positions, are not up to speed on what law firms and other legal practice organizations are doing right now with AI; (2) even if a faculty member is pretty sharp on today’s applications, the technology is changing so rapidly that we have no idea what the students will be doing with AI tools 2+ years down the road when they graduate; and (3) we need all the time we can get to train students in the fundamentals of legal reasoning, critical thinking, and the exercise of sound judgment, which continue to be essential in a world in which human lawyers work extensively with AI tools to deliver legal services to clients that satisfy standards of competence, ethics, and professionalism.

This post is not directly about the related issue of the use of AI by students as a shortcut to learning in traditional law school courses. In June, University of Texas Law School Dean Bobby Chesney recently sent a memo to all UT Law faculty (h/t Renee’s Roundup) stating his views on AI and Legal Education. And recently the University of Chicago Law School banned the use of laptops and smartphones in 1L classes, in part to curb the in-class use of AI, although there are other good pedagogical reasons for banning laptops for note-taking purposes. There is an indirect relationship between cracking down on AI use in law school classes and being skeptical about training students to use it in practice, and this reason is suggested by the Chicago Law statement about AI use in class:

We need to ensure that our students actually learn to think critically, strategically, and independently without relying on AI; but we also must face the reality that AI tools are already widely available to our students, and our graduates will be expected to be prepared to use them in legal practice.

The first part of that sentence is the focus of my argument here. We, as legal educators, have specific, hard-acquired competence in teaching students to “think critically, strategically, and independently.” We’ve been doing that for a long time, and the evidence (from the Carnegie Report and elsewhere) indicates that we’re actually pretty good at it. We know less about the AI tools in use in legal practice. The comparative advantage is with law schools on the fundamentals of legal reasoning, and with the profession on the deployment of AI tools to assist lawyers in performing the specific tasks required to represent the particular clients of the firm. We will always be several steps beyond practitioners in teaching AI-assisted lawyering, but we will always be better than them at teaching critical reasoning and analysis.

The case for teaching AI use in law schools seems like an easy one to make: Law firms are spending a gazillion dollars deploying AI (e.g. the widely reported news that Kirkland & Ellis is spending $500 million to build its own custom AI platform). Some of this is driven by perceived client pressure to leverage AI to reduce the cost of legal services or possibly to structure alternative billing arrangements (cue another iteration of the decades-long debate over hourly billing, how much clients hate it, yet how resilient it is). Unsurprisingly this spending by law firms has the structure of an arms race in a brutally competitive industry, where firms are reporting that they are under pressure from clients to use AI to cut the cost of providing legal services. If Kirkland is spending half a billion dollars, everyone else had better match their spending or be left in the dust.

On the subject of arms races, law firms are also competing in the labor market to attract top law students. (The effect of this competition on the first-year law student experience is the subject of a future Substack.) I’ve heard stories of law firm partners publicly proclaiming that they simply will not hire law students who are not already proficient in the use of AI. I don’t know whether that’s just public posturing – I suspect no firm would turn down a student with good grades at a T14 law school just because they can’t write a series of Harvey prompts – but what matters is the students are hearing this message. Since their anxiety levels are already through the roof due to the law firm recruiting schedule, they’re putting pressure on law schools to teach AI skills. And as long as we’re teaching AI skills, it’s important to show students how to exercise them responsibly, seeing AI as a partner in delivering legal services to clients alongside human lawyers.\

[One of thousands of dopey AI-generated images of robots and people working together, part of the ocean of AI slop overwhelming the internet.]

This is separate from a benefit of AI that I’m not talking about here, which is its potential to address the gap in access to legal services. I wrote recently about the downside of AI use by pro se litigants, and there was an amusing recent Law.com article about a pro se litigant who got past the motion to dismiss and summary judgment stages and actually had to try a case with the help of AI (dog, meet the car you were chasing), but there’s little doubt that people who could not, or at least are not willing to, pay for a lawyer to provide legal services are turning to AI tools. As a matter of social policy, on net, this may end up turning out to be a good thing – I have no idea. Along the way, we also have to figure out how unauthorized practice of law rules apply to AI. But the question here is whether law schools should be taking on the project of training students to work effectively with AI tools in their representation of clients. Here are three reasons to believe the answer is no.

I was at a conference a couple of months ago where someone involved in AI implementation at a law firm was talking about tuning up a firm’s version of Harvey so it could summarize deposition transcripts. It took five pages of prompts to get it right. I thought, “hmm, that’s interesting.” Summarizing dozens, if not hundreds, of depo transcripts in a complex case is arguably necessary but certainly soul-sucking work, and if it’s possible to use an AI tool to do the job, that’s all to the good. But it’s a task specific to a particular area of practice – namely, complex civil litigation – and one that different firms, or even different litigation teams within the same firm, may want to approach differently. For example, the task of reviewing and summarizing the transcripts would differ if the purpose were filing dispositive motions, building on fact discovery during the expert discovery stage, or cross-examining the witnesses at trial. AI tools might also be used to do things like do a first-pass read through a huge pile of documents, come up with questions for subsequent depositions based on the fact discovery to date, identify inconsistencies in the evidence, prepare detailed chronologies of events, and things like that.

I’m a former civil litigator, so I can imagine similar uses to which AI tools could be put, and maybe even design a simulation for classroom use that would allow the students to mess around with the prompts and refine the product’s capabilities as a discovery-management assistant. But I don’t have the granular experience with the day-to-day tasks that make up representation in criminal prosecution or defense, various forms of transactional practice (M&A, capital markets, corporate governance, etc.), patent prosecution, estate planning, regulatory work, and so on. Based on conversations with colleagues and faculty workshops I’ve attended over the years I can imagine, for example, that an M&A practice might scrape a bunch of SEC filings off EDGAR and use AI to determine what provisions are “market” in a given type of transaction. But there are other areas of transactional practice that I don’t have any feel for. And, if we’re being honest, all I really know is big-firm-type practice, not serving as a government lawyer, working as a small-town generalist (like the judge I clerked for), working in an in-house legal department, and the like.

My claim here, which is really more of a hypothesis, is that there are fewer AI-related skills that transcend practice areas, as compared with fundamental legal-analytical skills such as students have always learned in law school. Jamal Greene put it very well in a Bluesky post, nicely appropriating the lingo of the tech industry:

I see my job as assessing a set of reasoning and organizational skills that are portable across practice areas and technologies.

In this way AI differs from previous generation tools like Westlaw and Lexis, which have been integrated into the 1L legal research and writing curriculum since the time I was in law school (although we were also required to learn how to Shepardize with books and pocket parts – ugh). Straightforward legal research is a skill that is portable across practice areas. From what we know about the way law firms and other legal practice organizations are using AI, however, there is not some use-case for AI that is common to a diversity of practice settings. To the extent proponents of teaching AI in law school envision it as simply a more advanced version of Westlaw or Lexis, there’s no harm in adding it to legal research and writing classes. The claims made for AI fundamentally transforming the nature of law practice envision something much more comprehensive, however.

The other variation among law firms and other organizations is their internal AI policies, procedures, checklists, and other tools for managing the human-computer interface. (A couple of samples here.) In-house lawyers at law firms have spent a significant amount of time in the last couple of years selecting vendors, vetting the products, identifying the right tool for a particular task, training firm lawyers, and developing policies addressing issues such as (1) protecting confidential information and attorney-client privileged communications when documents are fed into an AI system or the system is queried by a firm lawyer; (2) monitoring the output if a system for hallucinated case citations and quotes, and otherwise ensuring compliance with duties of candor to tribunals; (3) ensuring that AI does not become a crutch that replaces informed judgment by firm lawyers; (4) disclosure to clients of AI use; (5) dealing with client requests/demands that AI be used as a cost-cutting measures, including crafting language for engagement letters respecting billing for AI use. Many firms have AI governance committees or similar structures for handling these implementation issues, with the involvement of in-house loss prevention counsel. I don’t see how law schools can duplicate all of this effort and create similar resources for guiding and constraining AI use, in which case, what’s the point of doing a half-baked job at something law firms are already doing fairly well?

Unlike previous-generation legal research tools like pre-AI Westlaw and Lexis, there is a very real risk that the products used by students in a 1L legal research and writing (or legal practice skills, or whatever it’s called) class will be badly out of date by the time the students graduate and start their first job. Seemingly every article you read about the use of AI in either legal education or law practice says something like, “ChatGPT version x.0 produced a lot of hallucinations, or wrote only a B-minus Contracts exam, but the new ChatGPT version y.0 produced a perfect brief with no fake citations, aced the NextGen bar exam, and got hired as a Supreme Court law clerk.” Okay, I’m exaggerating a little bit, but it’s undeniable that, with the oceans of money pouring into AI research, the products are changing, and improving, at a pace that is almost impossible for non-specialists to keep up with. I’ve seen references to a new Moore’s Law for AI, with evidence that large-language models are showing improvements of 2.5 points of IQ per month. If that’s true, then educators will inevitably be lagging the adoption of cutting-edge products by practitioners. (Old-school Westlaw and Lexis, by contrast, did not change at anything like this rate.) And this is all just with reference to generative AI using large-language models, such as would be used for research and drafting projects; we haven’t even started to talk about agentic AI and the uses to which it might be put by lawyers. (See, e.g., this article about the use of multimodal generative and agentic AI in litigation.)

I’m also starting to hear about how the real action, at least in AmLaw100-type firms, is not the application of Legora or Harvey to the needs of the firm, but the building of completely private, custom solutions. I’ve heard people say that no one is going to get ahead in this space unless they invest in building their own ecosystem. As noted, the AI investment frenzy in general has the quality of an arms race, where the object is not so much to acquire a good thingie, but make sure that your thingie is better than that of your competitors. If that’s true, then we in legal education will at some point have to admit that we’re training our students for a hypothetical deployment of AI resources that no firm is actually doing.

Along with changes in the technology comes changes in the risk/utility balance of AI tools. For example, right now the big issue in the legal media is hallucinated case citations. Almost every day there’s another report in Law360 on a show cause or sanctions order entered by an exasperated judge in response to fake citations submitted in a brief by a clueless lawyer who somehow never heard of the possibility of AI hallucinations. See the database of these cases maintained by French lawyer Damien Charlotin (who seems to have turned his monitoring of AI hallucinations into a business – good for him). At this point, I don’t think even a public flogging of lawyers in the courthouse square will suffice for general deterrence. Nevertheless CLEs on ethics and AI inevitably spend a lot of time on the hallucination problem.

Here’s the thing, though: At some point the technology will likely get better, using techniques like retrieval-augmented generation (RAG), and the hallucination problem will be yesterday’s news. We’re not there yet, as this article by several Stanford University researchers demonstrates, but given the rapid pace of development of AI technology, and the importance of solving the hallucination problem, it doesn’t seem farfetched to predict that in a couple of years this will no longer be the issue it is today. A law school class that over-emphasizes the hallucination problem will appear outmoded to the students, who may end up concluding that these out-of-touch law school faculty are not the best sources of information on using AI in legal practice.

I suppose this problem is solvable by hiring specially trained instructors who are able to stay up with all the latest developments in law firm tech. But this is not how law firms do things, ever. If you think law firms are going to add extra faculty with specialized qualifications, I’ve got a bridge to sell you. They’re going to make existing faculty, probably in the legal skills curriculum, take this on as an extra assignment, probably without relieving them of any other duties. I am not casting any shade on legal skills faculty, who are often among the best teachers at any law school. However, those faculty members have probably already been tasked by the dean or associate dean with the last couple of flavors of the month, many directed by ABA accreditation standards, like “opportunities for developing a professional identity” or “bias, cross-cultural competency, and racism.” In my experience, when one of these new requirements comes down, the instinct of many deans is to add it to the plates of already overworked legal skills teachers.

Alternatively – and this is something I see in connection with teaching the required Professional Responsibility course – deans might opt to bring in a bunch of adjunct from the local legal community. Here again I’m intending no shade on adjuncts, who generally do have the required practical experience, but as a former associate dean I can tell you that the teaching ability of adjuncts varies widely and is difficult to assess prior to bringing them on board as teachers. Also, given institutional norms of academic freedom and resource constraints, it is difficult for law school administrators to develop a coherent curriculum using adjuncts. There’s really no way to know what they’re teaching in their classes without dedicating a lot of time to reviewing syllabuses, sitting in on classes, and things like that. If the concern is specifically with the responsible use of AI in practice, legal educators do not want necessarily to defer to the profession in setting those standards.

One area of agreement among law firm AI policies and the guidelines published by bar associations and other professional organizations is that human lawyers must remain in the loop. For the oversight of AI by humans to be meaningful, however, it cannot be simply checking a brief produced by an AI system to ensure that it does not include fake case citations or quotes. The human has to be intimately involved in the research, analysis, and drafting process at all stages. An appellate judge who works extensively on legal technology writes:

There’s immense value in what I call the “blank page exercise” - the process where legal professionals must think through issues from scratch, wrestle with complex legal concepts, and develop their own analytical frameworks. When we allow AI to consistently provide the first draft to inexperienced practitioners, we risk creating a generation of lawyers and judges who become editors rather than authors of legal thought.

One of the major concerns with the use of AI by students in higher education is not cheating as such, in the sense of passing off text generated by a chatbot as their own work, but the use of LLMs to brainstorm or generate ideas. The problem is not just cognitive offloading, i.e. the induced laziness that results from reliance on AI, but lack of engagement at the early, formative stages of a project. I’ll just appeal here to the phenomenology of legal writing: Anyone who has sat down to draft a complaint, a dispositive motion, or an appellate brief is familiar with the effortful process of getting one’s mind around the factual record and the law, thinking about how that unruly mass could be organized in a clear, persuasive manner, figuring out where the weak spots are in the presentation and thinking about ways to shore them up, and so on. The problem with offloading that process to a machine is not only that the finished product would not be as good. Heck, let’s assume for the sake of argument that it would be just as good as that produced by the most skilled and experienced human lawyer. Even in that case the problem remains that the human lawyers involved in the representation do not have the same deep familiarity with the matter. This will have downstream effects on the lawyer’s ability to think strategically, exercise judgment, and provide effective representation. The human has to be in the loop at all stages of the representation, not involved only episodically, to check the outputs of an algorithmic drafting process.

Although AI technology is rapidly evolving, you know what is not rapidly evolving? The legal principles that inform the ethically responsible use of AI. As scholars have observed, in connection with AI and legal education, competent use of AI “requires not just prompting skills but a user who knows what questions to ask and how to spot flaws in the output.” There’s simply no conceivable use of AI in law practice that does not require solid training in legal analysis in order to use properly.

Like almost every law professor I know, I’ve spent time with AI to see the uses students can make of it. The tools are impressive. Recently I was working on developing better prompts in the course of having Claude answer some of the hard hypotheticals in my Torts casebook. It did a very good job, and 10 years ago, if a student had turned in an answer like that on an exam, they would probably be getting the CALI award in the class. In terms of the product, even the free versions of AI products I’ve messed around with are awfully good. In professional education, however, we should be more concerned about the process than the product. Here is an illustration from one of my favorite Substacks, by a philosophy professor whose commentary I admire a great deal:

Consider the difference between building a piece of furniture bespoke by hand and pushing buttons on a CNC machine in an IKEA factory. Set aside the difference in furniture quality; in the former case you are a craftsman and in the latter case you are not. The craftsman understands types of woods and how they mill and plane, he knows their Janka hardness and how they glue, join, and react to moisture. A true woodworker is a wood engineer who can reason out construction from first principles and build jigs on the fly. An IKEA factory button-pusher is none of those things.

The deep understanding that constitutes craftsmanship is one of the central qualities that constitutes lawyers as professionals and not merely providers of a useful service. We have three years, which isn’t much time, to teach students deep familiarity with the craft of legal analysis and argumentation. Let’s not get distracted with something we’re comparatively less capable than their employers of teaching, which in any event will look very different when they get out into practice anyway. Have some faith in what we’ve always done, and done well, in legal education.