It's Time To Take The Risks Of AI Seriously

In George R. R. Martin’s A Song of Ice And Fire, televised as Game of Thrones, self-interested elites squabble over titles, territory and treasure, while a supernatural force creeps across the land threatening all human life.

Intended as a metaphor for the climate emergency, Martin’s fictional device evokes society’s inability effectively to address all existential risk. The most convincing audition for the role of real world Night King is currently being performed by generative artificial intelligence (AI). This, at least, is the eerie conclusion of a new report by The Brookings Institution.1

Why should we look past the spectacle of “Neo Royalism” playing out in geopolitics and focus instead on AI?2 In a self-described ‘pre-mortem’, Brookings researchers conclude that “the risks of utilizing AI in education overshadow its benefits”. This is because AI threatens to undermine children’s “foundational development” in two ways: by encouraging them to outsource their thinking and by eroding the relationships that make learning possible.

Brookings is no Cassandra. As its report recognises, there are obvious benefits to the use of AI in education. Children with speech and communications challenges able to talk to their peers using digital voices; AI-generated virtual reality environments that make history lessons come alive; learning materials that adapt for contextual relevance based on user feedback and AI prompts—these and many more applications promise to enhance learning for millions of children. Then there is the potential for AI to reduce teachers’ administrative workload, providing more time for student relationships.

But Brookings researchers conclude that these benefits “pale in comparison to the risks of AI,” which include undermining children’s cognitive and emotional development, eroding trust in learning relationships, increasing inequality, and much more (see Figure 1 below).

Figure 1: Potential benefits and risks of AI in education (Brookings)

There is good evidence that adults who use AI extensively experience atrophy in certain basic cognitive functions, but retain the critical acuity needed to check AI outputs. As Brookings points out, we cannot be anything like as sanguine about the impact of unrestrained AI use among children and adolescents who have yet to develop that critical acuity. We also have a responsibility to safeguard children and adolescents that is quite different from our responsibility towards our adult peers.

The risk is that our framing of the AI opportunity occludes our moral sense. MIT scholar Justin Reich has rightly challenged the assumption that the rapid adoption of all new technology is synonymous with progress. Reich argues that we should be particularly suspicious of ‘arrival’ technologies, those that enter our professional and personal lives without our conscious deliberation. AI is such a technology: introduced virtually overnight, freely available to students outside of schools and colleges, and designed for general use, so lacking the sort of guardrails needed to support students in their learning.

The rapid adoption of all new technology
is not synonymous with progress.

In fact, there is very thin evidence to suggest that adopting new technologies ever equates to progress in education. For example, researchers at Teachers College, Columbia University, reached a tellingly different conclusion when comparing middle schoolers’ engagement with continuous prose online and in print. If learning is the objective of reading, they found, print is a superior technology to digital media because it allows children to make more durable connections between new information and what they know already (what educational psychologists call semantic encoding). Educators should recall this example, among many others, when facing the deadening pressure from politicians to use this or that AI platform.

My own view is that the existential nature of the risks posed by AI makes the case for its inclusion in the school curriculum stronger, not weaker. Students with access to AI tools need to learn how and when to use them responsibly, how and when to question their outputs, and how and when they should not be used at all. They need to receive this guidance from their teachers and to explore these ideas in the context of the trusted relationships that make learning possible. This is necessary, not least so that AI can remain, in the words of Daron Acemoglu, ‘human centred’.

Brookings recommends a three pronged approach. First, schools should carefully titrate students’ access to AI, as part of an overall curriculum plan and using evidence-based pedgagogies that are age- and context-appropriate. Platforms that give teachers control over this process—such as the Aiden platform used by an increasing number of schools in the Gulf States—can be used productively.3 A parallel report, the OECD’s Digital Education Outlook, concludes that responsible use of AI is likely to involve successive iterations of learning tasks along with ample time for reflection.4 The most important thing is that educators gain control of the technology and not allow students to outsource brain work necessary to growth.

Second, educators a can develop their own and their students’ holistic AI literacy, to encourage responsible AI use in schools, colleges and beyond. This involves students being able to do more than just recognise when AI use is simply a modesty panel for naked plagiarism. Students need a moral framework for the use of AI in the home, in school, as a tool that plays into their personal relationships, in college and, eventually, in the workplace. The adults in their lives need to model all of these things and to do so informed by the best available evidence.

Third, educators must develop ethical procedures to protect students from the harmful effects of irresponsible AI use and inappropriate AI platforms. Most AI tools available to students were never intended for their use; others are based on foundational large language models that continue to evolve in ways that are, too often, unpoliced. Safeguarding students in this environment will require vigilance from educators but also a level of dirigisme from policymakers that is so far lacking. Something like Australia’s ban on social media may be too crude a policy instrument for AI, but embodies a commitment to a ‘no harm’ principle that should equally apply here.

I am a little more optimistic than Brookings’ analysis would suggest. My reasons for optimism lie in my day-to-day interaction with students. Among my own, there is already a groundswell of AI scepticism. I doubt this experience is unique.5 Some adolescents recognise the negative effects of outsourcing ‘real’ thinking to chatbots; many more are distrustful of their outputs. The fact that AI learning tools are of uneven quality across the curriculum—reasonable in English, for example, downright poor in Chemistry—has provided a healthy basis for comparison and stoked a critical response.

My second reason for optimism reflects the fact that schools and school systems are notoriously slow adopters of technology. Although the sector is alive globally with discussion about potential applications, I have attended enough conferences and network meetings and visited enough schools and colleges to know that what’s happening on the ground is often quite modest. Thankfully, the two exceptions to that global rule—the Gulf States and the District of Shanghai—are also the jurisdictions in which governance frameworks are most evolved.

Brookings researchers do not identify new risks from the use of AI in education. The value of their analysis lies in how clearly they draws the vectors of potential misuse. Left unchecked, the result of these processes could be generational, ecological, and potentially irreversible. It’s hard to imagine a world made up of people without cognitive agency, unable to form durable relationships with one another; but I doubt any of us would want to live there.