I keep reading that using AI damages students’ capacity to think and learn. The claim is often presented as though it were established fact. I find this odd because I also read a great deal of research suggesting that AI can support deeper conceptual understanding.
The studies cited to support the negative claim share the common feature of using AI within the existing curriculum using standard assessments. A good example is a randomized controlled trial by Bastani and colleagues involving nearly 1,000 secondary-school mathematics students (Bastani et al., 2025). Students given access to an unrestricted GPT-4-based assistant performed substantially better while using it. But when the AI was removed, they scored 17 per cent lower than students who had studied using conventional resources. The influential conclusion: generative AI can improve performance while undermining learning.
This was a good study and it also tested a more carefully constrained AI tutor, which performed as well as the no-AI control group. That third condition shows that how we design AI in education makes a difference. But the study illustrates a deeper problem with how we currently research educational technology.
“Learning” was defined as improved performance on an examination covering the existing mathematics curriculum, completed without AI. Given that definition, any new capability in the interaction between student and AI was not relevant and was not looked for.
Imagine a mathematics class where students learn to calculate using an abacus and use it in exams. An experimental group gets calculators during lessons, but must use an abacus in the final exam. If they perform worse, we might conclude that calculator use didn’t prepare them to operate an abacus. We could not conclude that calculators had damaged their capacity to learn mathematics. The Bastani study establishes that students given unrestricted GPT-4 became less able to solve problems with pen and paper when the new technology was withdrawn. It does not establish that they became less capable of mathematical activity in an environment where AI remained available, nor does it investigate what different mathematical capabilities education might require in such an environment.
Educational researchers cannot always simply take performance on existing assessments as a measure of ‘learning’. The research also has to ask what should we be learning. Many educationalists argue that it is not fair on students who will probably work and live in an AI-permeated world if schools do not teach them how to use AI.
Critics argue that educational research should follow the medical model: randomized controlled trials to discover whether interventions work. But the standardised tests needed for this model are normally those developed to measure learning within the existing system. Students are not patients in need of a cure for a clearly defined condition. Is ignorance a clearly defined condition? The aims of education are always contestable, and especially so now when we are introducing a radically new technology that many claim should change the nature of education itself.
Design-based research offers a different approach. Instead of asking whether a tool improves existing test scores, it asks: Does design X generate the anticipated process M? Did that process contribute to the intended outcome Y? Crucially: Did the experience of the design give us reason to reconsider the desirability of the outcome itself? This last question is important. Research should not merely refine the means while leaving the ends untouched. Our conception of what counts as learning may change when we encounter learners, teachers, and the realities of practice.
A recent Japanese study provides a concrete example. First-year university students worked in interdisciplinary groups to design a classroom for 2050 (Naganuma et al., 2026). They developed expertise in different areas, shared it through a jigsaw activity, and used generative AI to introduce additional possibilities. The researchers traced the development of ideas through classroom talk, interactions with AI, successive artefacts, and final designs.
The initial conjecture was that widening the range of ideas available would lead to more innovative designs. The intervention did produce “idea expansion”: students generated possibilities that went beyond the original materials. Yet relatively few groups turned this expanded range into coherent, genuinely innovative proposals. In a second iteration, more sophisticated prompting produced even more AI-generated ideas. However, this led to fewer contributions from students themselves. Generating more possibilities did not automatically produce better collective thinking. The next stage, not yet reported upon, will focus on teaching students how to select the most promising new ideas.
This result is valuable precisely because the original proposal did not simply “work”. It revealed a missing educational capacity: generating possibilities is different from judging which possibilities are promising, connecting them coherently, and deciding which deserve development. AI may be particularly good at widening possibilities while making human selection, judgment, and collective deliberation more visible and essential.
The study did more than test whether a tool improved performance. It helped produce a better account of what the educational aim should actually be.
This suggests a broader understanding of rigor. Good research requires careful observation, appropriate comparisons, and where useful, experimental testing. But research into educational futures also requires conceptual rigour: making explicit the desired ends, the processes expected to produce them, and the criteria used to recognize them, then exposing all three to criticism. It means actively searching for neglected evidence and rival explanations, asking what evidence would persuade us that our favoured idea was mistaken.
There is no conflict between this approach and randomized controlled trials. Once a design, its underlying theory, and its intended outcomes are sufficiently clear, an RCT may help establish whether it produces a difference and under what conditions. But moving too quickly to measurement can avoid the most intellectually demanding work: developing a worthwhile intervention and a plausible theory of how it actually operates in practice.
Research supports dialogue; it does not replace it
The fundamental questions of education differ from those of natural science. They are not “does this work?” but “what should we teach and how should we teach it?” Empirical research cannot answer that alone. It must contribute to genuine dialogue about what education should aim at. I do not mean here just a consultation where stakeholders comment on a predetermined design, but a real conversation where different perspectives encounter and challenge one another, with the possibility that everyone, including researchers, may change their minds.
Researchers contribute evidence about what is possible. What happens when AI enters a classroom? What do students actually do with it? What capacities do they develop, and what might they lose? Such evidence is valuable. But it cannot determine what should happen. That requires dialogue bringing together empirical evidence, practical wisdom, cultural values, concern for particular learners and communities, and competing visions of a better future.
To research the future, then, is to formulate possible futures clearly enough to build and examine them rigorously; to investigate what happens using appropriate methods; and to remain willing to revise both our designs and our understanding of educational purposes. We need research that tests ideas, but also research that helps us generate better ideas to test.
Naganuma, S., Shibukawa, S., Minematsu, T., Ohno, A., & Wakihama, Y. (2026). Beyond expert knowledge toward idea innovation: Potential and challenges of a generative AI-supported jigsaw method. International Journal of Educational Technology in Higher Education, 23(1), Article 24. https://doi.org/10.1186/s41239-026-00598-6
Bastani, H., Bastani, O., Sungu, A., Ge, H., Kabakcı, Ö., & Mariman, R. (2025). Generative AI without guardrails can harm learning: Evidence from high school mathematics. Proceedings of the National Academy of Sciences, 122(26), Article e2422633122. https://doi.org/10.1073/pnas.2422633122
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.