Hello friends,
tl;dr: I’ve been prototyping an AI tool for helping people develop critical thinking skills while using AI tools. The key insight is that blank chat prompt boxes are terrible for learning—instead, we need reasoning scaffolds and Socratic mirrors that support human meaningmaking rather than pretending to do it for users. I built seven quick tool prototypes to quickly evolve and test the technical stack, interaction logic, and content. You can read about and sign up to test the tool here.
I’ll demo the tool on October 15, alongside neat work from other fellows in the Future of Life Foundation’s programme on AI for human reasoning. If you’re in the Bay Area and would like to come, email me and I’ll try to sort an invitation.
Last week, I was in Tunis for 3 days working with a development organisation’s portfolio leadership team on their 2026 strategy. I used a mechanism for structuring idea elicitation and refinement that I’ve been developing over the last year, and it worked much better than I could have hoped for. While in Tunis—because of my poor scheduling—I delivered my module on a new way to teach public sector strategy for Protocol School (alongside a bunch of superb faculty fellows from all over), then delivered the same module again at the Lee Kuan Yew School of Public Policy the day I got back to Singapore. I’ll write more about that when the dust settles.
This week is about an adjacent project: An effort to create a prototype AI tool for helping people learn to think critically while using AI tools, funded by the Future of Life Foundation’s programme on AI for human reasoning. I was simultaneously prototyping three distinct layers: The delivery layer (code and webapp), the mechanism layer (human-machine interaction logic), and the content layer. The Tunis strategy work used the same mechanism, with a different delivery experience (slides and pen-and-paper vs. webapp) and different content.
The simple but powerful idea behind this tool is my thesis that the unparameterised blank chat prompt box—which has become the default mode for interacting with LLMs—is actually terrible for helping people learn to use these tools effectively. The blank prompt assumes users already know how to structure their thinking, frame problems, and evaluate responses. But if the goal is developing critical thinking skills, that assumption is often unjustified.
A much better approach is to scaffold the interaction. Put explicit context around the kind of input you’re hoping the user will provide, then be structured about how you process that input and what you reflect back.
If we build AI tools with this design philosophy, they can provide users with a Socratic mirror that supports meaningmaking. These AI tools will not pretend or appear to do meaningmaking for the user. Instead, they’ll provide reasoning scaffolds that explicitly support the user in developing their own capacity for meaningmaking. Reasoning scaffolds work by reflecting the user’s thought processes back at them for consideration and refinement, rather than generating meaning on the user’s behalf.
I vibecoded an extremely hacky series of prototypes over 3-4 days, iterating simultaneously across these three layers: Delivery instance (technical implementation), interaction logic (how content gets structured and delivered), and content (what actually gets delivered).
The first two prototypes were to validate whether the basic interaction logic worked. Could I take a pen-and-paper mechanism and translate it into something that didn’t require me physically present? I also wanted to see if automating the Socratic mirroring I normally provide manually would work. Having a human interlocutor function as the mirror takes a lot of time, and this kind of mechanical processing is something machines are, in theory, very good at.
Versions 3-4 implemented LLM responses as Socratic mirrors. This required figuring out how to prompt based on user input so that the LLM would return something like a mirroring response users could evaluate. In testing, I established that these stimulated users to actively consider whether the mirrored responses were consistent with their own understanding and intent, whether they wanted to change it for clarity.
Versions 5-7 focused on distribution, moving to web (vs local) delivery, making API integration more robust, and adding logging. Version 7, which I’m using for scaled-up testing, is where both the mechanism and user experience seem to work. Today, I tested it with a small group of college students in Singapore. The short summary is that even in this prototype phase with several UI glitches, the tool works much better—and much, much faster—than I’d expected. I’ll report on the first wave of testing soon.
The broader insight from this exercise is about building AI tools that enhance rather than replace human capacity for meaningmaking. Many of the AI tools I see today seem designed to either “do the thinking for users” (this includes many so-called “agentic” tools) or provide so little structure (by way of the empty, context-free text entry field) that users don’t learn effective patterns for interacting with machines while preserving human-ness.
The alternative I propose is to build tools that explicitly scaffold human meaningmaking processes. These tools should be designed to elicit meaningmaking by users, and to surface contradictions, highlight assumptions, and reflect thinking patterns back for user evaluation. This requires being very clear about what humans do that machines cannot and what machines can do better than humans, and designing the human-machine interaction accordingly.
I wrote up some background on this first project to develop an AI tool for scaffolding human meaningmaking in the context of AI tool use (meta, I know).
Students now have access to LLMs that can write essays, but seem to be losing the capacity to think critically. I solve this problem by reconsidering the interaction logic between human users and the AI tools they use. I’ve developed an AI tool that inverts the usual logic of the empty, unconstrained chat box — the goal is to help users learn to think critically and do the meaningmaking work that only humans can do. Initial tests of the mechanism shows users going from vague statements to sharp arguments in under two hours. This tool represents a scalable approach to critical thinking education and an alternative to current AI tools that make students passive consumers of machine-generated content.
If you’re interested in testing the tool and/or learning more about the course as it develops, please sign up here. The tool is fully functional, so testers can benefit by refining an actual argument they want to make (e.g., for a paper in a class, a policy paper for potential implementation, a startup business plan, or a strategy proposal for management).
Read about the substantive and the pedagogical innovations of this new public strategy course.
And I’ll demo the tool on October 15, alongside neat work from other fellows in the Future of Life Foundation’s programme on AI for human reasoning. If you’re in the Bay Area and would like to come, email me and I’ll try to sort an invitation.
This is the first publicly available tool to come out of nearly 3 years of work on meaningmaking and 17 years of work on uncertainty (as distinct from risk). I have several more in the pipeline…
How to use AI without becoming stupid: “The Vaughn Tan Rule goes like this: Do NOT outsource your subjective value judgments to an AI, unless you have a good reason to, in which case make sure the reason is explicitly stated.”
The right way to use AI tools: “… he started using ChatGPT to draft emails in French. It felt like a net positive — enabling better communication with his French friends — until he started to feel his brain ‘get a little rusty.’ He found himself grasping for the right words to text a friend.”
Connection innovations: AiryString achieves a ~26% reduction in weight and a ~23% increase in flexibility by removing the tape from the zipper (compared to a standard #5 VISLON YKK zipper.
See you next week,
VT
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.