Reading · AI + Education = Simplified

In the first part of this interview, we explored what are some of the new questions that are emerging from in legal education as a result of AI. We continue that conversation here along with discussing some of the new policies and practices emerging from law schools.

About the Interview series

As part of my work in this space, I seek to highlight the folks I’ve been in conversation with or learning from over the last few years as we navigate teaching and learning in the age of AI.

If you have experiences around AI and education in higher education classrooms that you would like to share, consider being interviewed for this Substack.

A polished, lightly whimsical cartoon-style portrait shows a smiling woman with shoulder-length brown hair, gold earrings, a bold gold chain necklace, a black blazer, and a green-and-blue patterned top. She faces forward against a warm cream-colored background filled with scholarly and creative details: stacked books, leafy branches, an open notebook, a pen, a mug, soft geometric sparkles, faint handwritten notes, a lightbulb drawing, and delicate network-like lines. The overall effect is friendly, thoughtful, and academic, suggesting creativity, teaching, and intellectual inquiry.

Lance: What are you concerned about, or challenged by, around AI?

Kirsten: Well, one worry is our own novice status as AI users: new users leading new users. I see this among my colleagues in the legal writing field; most of us don’t have practice experience using these tools, so we’re exploring them ourselves while trying to come up with best practices and standards instruction and study in our discipline.

I also worry about the training of junior lawyers. So many more senior lawyers cut their teeth on activities that may now be done by AI, and I don’t know that we know how one learns the skills we considered the path to good legal judgment without that “rote” experience. We did these basic lawyering tasks over and over, and over time became smart partners or senior lawyers with a certain amount of wisdom as a result of this time and repetition

What’s the connection between those two things–rote tasks and developing wisdom? Or will wisdom come from developing a different skill set? I think a lawyer’s ability to evaluate text is going to be a more important skill now, but do we know how to teach that at the level needed, as opposed to the skill of generating text? In other words, what does it take for a lawyer to effectively evaluate text generated by a machine? Do we know how to teach that? I don’t know that we’ve even named that skill or broken it down into its component parts.

Then there’s the availability of jobs. Everybody’s worried about entry-level lawyering jobs and how many there will be. And there’s worry about.whether who’s allowed to practice law will expand. There’s a lot of discussion about whether the companies producing these tools should be allowed to practice law with them even if the owners are not lawyers.

Let me talk about this a little more. Some say.AI use by nonlawyers is unauthorized practice of law and shouldn’t be allowed; others are thinking about whether some representation through a machine, independent of a human lawyer, would be good for access to justice. I believe the Legal Services Corporation says 92% of the “substantial” civil legal needs of low-income people are not met. Could we change that number with AI tools?

What if a person needing services could go to a kiosk in a big-box store, pay some amount of money, say, $199, and get legal services tailored to me through an app powered by an AI model? Maybe that would fill a gap. But, it’s possible that we are now talking about having no lawyer in the loop; the machine would be the advisor. That’s a big change to how we imagine the practice of law, and it’s worrisome from a lot of directions; what it means for legal jobs, the quality of the advice, and for the integrity of the system.

And then there is the worry of the hallucinations. We see case after case where lawyers, and pro se parties-people representing themselves–are being sanctioned in court for filing documents with AI-generated false information. These false statements aren’t limited to made-up cases that don’t exist; it includes nonexistent quotations from real cases, or statements that incorrectly attributes a proposition to case,, or made-up facts about the legal conflict

There’s a database that includes over a thousand cases documenting these errors. These errors impact the integrity of the judicial system: if legal precedent is cited but doesn’t exist, and no one’s catching it, then this raises questions about the validity of the law available in research resources. When you think about the need for accuracy in legal research–we want to know the law we are relying on is absolutely correct–citing things that don’t exist and stating principles no judge ever articulated undermines the stability of a system in which we have placed a lot of faith. That’s very worrisome.

Lance: And that collection is the ones that got caught: how many goals got past the goalie?

Kirsten: Right.

Lance: As you were talking, my mind went to “Claude, Esquire.” I’ll just go to my friend Claude, Esquire.

Kirsten: People already do.

Lance: Where my mind goes is that this is how the law profession gets the Uber treatment. You have the rise of these tools: it’s $200 for Claude, Esquire, but for an additional 50 bucks, you can get a one-hour review from an underemployed lawyer. You can see it happening.

Kirsten: That’s a useful observation!.Richard Susskind has talked about the future of lawyering for more than a decade, maybe two. He talks about commodification of legal services: there are many tasks that are simple enough to be sold as a commodity rather than as a bespoke service.

Are there tasks in the law amenable to successful delivery by artificial intelligence with less.or without human supervision? Susskind and others have talked about that for a long time, so you’re not saying anything wild.

But what might be slightly a little different here is the direct-to-consumer piece–that is, not lawyers using AI tools for commodified tasks, but me, the consumer, buying that legal task directly from an AI tool. Consumers have been able to buy books and software to do legal things themselves such as wills and trusts for a long time; purchasing AI legal services would be more individualized to the consumer than a book. That individualized connection to the potential client is what makes AI being used in this feel like an encroachment into human legal practice. But there are interesting papers advocating for that, at least experimentally.

Lance: It’s one of those things where it’s hard to put the toothpaste back. By the time there was enough evidence of inferior outcomes or harms, the process would already have worked through. You’d have all these “Uber lawyers,” and the system would already be in place.

Kirsten: Right. The supply-demand problem would already have challenged our ability to think carefully about the issue. I think that’s why some state supreme courts have experimented with alternative systems of delivering legal services, not necessarily in the generative AI context, but in other contexts. This is often referred to as a “sandbox.”

State regulators carve out a place where the ethics rules that would have limited a particular kind of legal practice don’t apply, and they approve providers individually, letting them perform some otherwise impermissible services under supervision. Utah is an example of using a sandbox for this purpose. The sandbox approach lets regulators evaluate the benefits and harms of an alternative way to provide legal services. We might see this with generative AI tools as well.

Lance: I want to move into supporting law faculty around AI. You’ve been running a program for law faculty, facilitating a national conversation amongst law faculty about AI. How did you end up in this role? Did you step forward, or did everybody else step back?

Kirsten: My dean says my leadership here was the product of a well-timed sabbatical! I have both a JD and a PhD. My PhD is in human communication and rhetoric. During my sabbatical in January or February of 2023, right after the fall 2022 release of ChatGPT, I took one look at it.and thought, this is a rhetorical tool, through and through. Since the practice of law is a rhetorical practice, this technology is going to transform everything.

So I began to study generative AI. I’d been teaching legal research and writing for over 20 years at that point, so I had a lot of knowledge to apply. I started thinking about generative AI not so much as a technology but as a rhetorical practice and a rhetorical artifact generator. And my goal was to figure out what AI would mean for what lawyers do every day? I saw AI as paradigm-shifting, requiring us to really revisit how we train lawyers and how they practice. As I studied, I figured out that I needed to talk to about this with others. Around this time, AAC&U had a webinar on AI; and I attended. Then, I reached out to a listerv of other law professors and asked, “Did anybody else go to this webinar, and does anybody want to talk about it?” A few people did, and that’s how I founded the

Legal Writing and Generative AI Convo group. Every month we get together and talk about a new topic, mostly around legal communication and education. I have a colleague at the University of North Dakota, and she and I work on this together. We come up with a semester’s worth of topics, invite people, introduce the topic (we may have a lot to say or very little),break people into groups to talk, and then debrief.

We’ve had all kinds of conversations. A couple of summers ago we read John Warner’s book More Than Words in the group, from a legal writing education and practice perspective. We explored how AI impacts citation versus acknowledgment practices. We considered the future of writing samples for legal hiring. We’ve talked about the role of faculty in policing AI use: is it ever appropriate to mislead your students to catch AI use, and should you ever use detection methods? And we’ve asked what a rigorous writing experience looks like with AI in the mix, since the American Bar Association requires a writing experience for students as part of its accreditation standard.

We started by inviting anybody we knew, and now there are 550 people on our Convo list. The point of the group is to have a community where we can talk about these issues and explore our thoughts. As a rhetorician, I believe knowledge is created through discourse, so we’re creating a body of knowledge.about AI by meeting and talking.

It’s been an absolute thrill, and we haven’t run out of topics for three years now. The motivation was that I wanted to talk with others about generative AI, which I think is the most compelling thing to happen in legal communication since I’ve been teaching.

Share AI + Education = Simplified

Lance: That’s such an important message: find your people and have the conversation. You’re doing it, and I love to hear it, because those disciplinary conversations are so important.

Kirsten: You were an inspiration, too. You were out there talking about AI and collecting syllabus policies very early on. I remember sharing your information in my sphere: “There’s this guy, he’s got great AI stuff, and he’s creating community around it.” I thought, there’s someone being really brave in this. Can I do the same thing? I even asked for you when we did the AAC&U AI Institute, because I knew you’d be helpful to us.

Lance: Thank you. That’s incredibly kind of you. It sounds like you’ve got a wonderful community of practice covering lots of topics. What do you think people are hungry for in all of this?

Kirsten: When I think about my colleagues engaged in this space, we’re all hungry for a way to do the right thing about AI in legal education.

I’ve spent a lot of time learning about AI and how it intersects with law and legal education, but not everybody has that time, so they ask: Can you help me get up to speed? Can you give me a set of steps? Can you tell me the right policy, the right assessment, what students will need?

Thoughtful legal educators are hungry for a set of best practices, and for ideas on how to avoid the problem of AI interfering with their students’ education. That’s really worrisome for folks; that AI use will get in the way of learning. Studies suggest our ability to learn requires effort, and AI can take away from that effort, so it can be dangerous to a legal education in that way. And my colleagues are hungry for conversation; somewhere to talk about this without feeling stigmatized, or that they’ll reveal they don’t know something or are doing the wrong thing.

The idea of AI “stigma” is real. I find people are still reluctant to talk about how they use AIthemselves, because.they think it’ll be perceived as the wrong thing to do. I just had a conversation with colleagues planning a presentation, and it was interesting to hear how varied our personal ethical boundaries were on AI use. One says, “I never use it for writing, only editing”; another, “I only use it for research”; another, “I use it for all the things.” That’s a lot of variation; how do we make consistent ethical boundaries in this context?? That’s what we’re all struggling with: to do the right thing by our students, by our scholarship, and by the law. That’s what faculty are hungry for guidance about.

Lance: I love that summation: what the right thing is, and how do we know it? So much of the work we’re doing right now is hypothetical; we’re all novices figuring this out.

Kirsten: And research in this area is challenging. There’s only one group I know of in legal writing doing anything close to well-designed empirical research, because they have that background. That’s how we’ll figure out what to do, to some degree, but there isn’t a lot of bandwidth yet for that kind of research.

Lance: There’s a way in which we’re suffering from our own riches. We live in an age with access to so much research, and we’ve figured out so many things, that we have inordinately high standards for what everyone will agree is legitimate. That’s important, but it also means that every study can be dismissed for not being completely covered in perfect 24-karat gold.

Kirsten: Agreed. I’m thinking about the quantitative studies that give us statistically significant results, but there’s also a lot of room for qualitative studies. And those require time, and there aren’t clear paths right now for the research. I’m working with a colleague to better understand law students’ experiences and attitudes related to AI, which involves survey design, distribution, and IRB approval: things we aren’t routinely involved in as law researchers. We’re trying to enter that space to get that information, and that’s a challenge when a lot of law faculty aren’t trained that way and the workload is already heavy.

Refer a friend

Lance: What are your thoughts about these emerging AI use policies like the policies that have been announced in the last month or so like the UC Berkeley AI Policy or the University of Chicago’s Law School’s new policy of no tech in classes for the first year?

Kirsten: The emerging policies reflect a well-founded desire to ensure that students learn what law school is designed to teach. The worry is 100% legitimate: if students can use AI tools to complete law school work without the intellectual struggle that educational experts know is necessary for durable learning, then the potential is there to graduate students who are not fully capable of the reasoning, analysis, and judgment skills we’ve always associated with being a lawyer. Law schools, whether they like it or not, are in the position of needing to address generative AI not just as a problem of ensuring that students are “practice ready,” having emerging technology skills, but also as a problem of whether AI use in law school will erode foundational learning. I don’t think I can overstate the importance of this problem. So schools must respond to these related problems that actually require different solutions. What you see in the policies like Chicago’s and Berkeley’s are responses on the “foundational learning” side of the equation.

I’ll say that no one knows what the optimal response is. We just don’t have experience with how these technologies impact legal education. What makes the Berkeley and Chicago policies interesting, however, is that they both take an “institution-level ban” approach. Chicago has a first-year “classroom technology ban” and Berkeley has opted to ban AI as a “writing tool” throughout law school (with a professor’s prerogative “opt out”). Other schools have ban-like policies and others have approaches that range from a greater embrace of AI usage to more classroom- or professor-guided options.

Lance: You mention that no one knows what an optimal response is, but what are your insights to the question of navigating this?

Kirsten: I don’t have answers about an ideal policy, but I’ll share some thoughts I’ve been working through.

First, I think changing assessment practices is our first line of defense against the “outsourcing of intellectual effort” to AI tools. Assessment changes might even be the most important thing we do. Today, assessments that are proctored and remove AI access are the gold standard, I think, for understanding whether students have learned many of the kinds of knowledge and skills we care about in law school. And it’s not just summative assessment in an exam at the end of the semester. In an AI-saturated world, I think we need to increase the opportunities for formative assessment where students are required to apply their mind to a task without AIs assistance. This is not because students, as lawyers, will necessarily function AI-free but because they’ll need to have this knowledge and those skills to effectively use AI tools. This shift in assessment might mean using valuable classroom time differently than perhaps we’ve done in the past.

Second, I think I worry less about the technology present in the classroom and more about how I use the classroom time to increase engagement with and without technology. Passively listening to a lecture is not the kind of engagement we need now. Writing, quizzing, peer-to-peer interactions, dialogue, surveys, working with an AI interlocutor, even, could increase engagement in the classroom, which I think is what Chicago is after here. Certainly Socratic dialogue has its place, but that dialogue can also be a passive experience for the listening students.

I share Chicago’s apparent view that classroom time is now perhaps more valuable than ever. I understand Chicago’s rationale for the classroom ban—to require students to look up from their computer and engage, to use their brains. But in the right environment, with the right instruction, there’s nothing mutually exclusive about using AI and using one’s brain—even when learning skills like good judgment and reasoning. But I worry if we remove AI altogether from the classroom learning experience, then we lose the opportunity to influence students’ understanding of what responsible use looks like. This is more complex than what I’m conveying here, and I’m still noodling this over.

Finally, you mentioned the Berkeley policy. This policy style tries to regulate how students write outside the classroom. I agree wholeheartedly with Berkeley’s writing goals—to ensure that the writing experience is rigorous and educationally meaningful. But I question whether an attempt to ban AI use is the right answer—particularly for upper-level law student writing. I think a better approach might be to redefine the goals of writing, ask what “rigorous” legal writing experiences look like today, and spend more time with students in human-to-human interactions about their writing. But what I’ve just described is a big ask of an already busy faculty and requires a lot of rethinking of what we’re doing.

So I think these are good faith, early efforts to regulate aspects of the educational experience that, frankly, might not be able to be “regulated” in a traditional sense. And I think we’ll see an evolution of these policies.

Lance: Final question: what’s next for you?

Kirsten: I’m going to keep leading these conversations. Every semester, my colleague and I ask, is this the last semester we do the Convo? Is there still a need? So far, the answer is yes. We’ll keep doing the Convo groups, and anyone can reach out to me to be added to the list.

I’ve been working on a few research projects: the survey, as I mentioned, and a piece about authenticity in legal writing. When I hear people’s complaints and concerns about AI, they often boil down to asking, does the writing feel authentic? I want to think about that more in the context of legal communication, bringing legal ethics into it.

And I’m continuing to figure out how to transform the 1L legal writing curriculum into something that makes sense and has rigor, uses valid assessment techniques, and prepares law students for practice, with and without AI assistance. Mostly, I’ll try to keep my head above water.

Lance: And breathe, of course.

“We’re all hungry for a way to do the right thing”

Comments