In “The High Cost of AI Without Literacy,” I wrote about discovering that all forty-four of my scheduled tasks were quietly running on the most powerful, most expensive model available, and what it cost me to leave them on the default. That was a story about scale, and about governance.
But lately I’ve been watching the same lesson play out at a much smaller, more human scale: a colleague tries an AI tool for the first time, pastes in a document, asks a couple of questions, and within a few minutes hits a wall. “I barely did anything. How am I already out?”
It’s the most common first encounter people have with these tools, and it’s worth slowing down on, because the answer explains a lot about how they work.
When you work with an AI model, it doesn’t just read your latest message. It re-reads the entire conversation every time. That includes your first prompt, every reply it gave you, and especially anything you pasted in. Drop in a twenty-page policy PDF and that whole document gets counted again on every turn that follows. So “running out in five minutes” almost never means heavy use. It usually means a few expensive moves. One giant paste, one endless thread, the same background sent over and over.
This is the same root cause I hit with my automated pipeline, just wearing different clothes. At scale, it looked like a surprise bill. For an individual, it looks like a tool that gives up on you mid-thought. Either way the cost is invisible, and nothing in the interface tells you what’s happening underneath.
And the fixes are refreshingly unglamorous:
Paste the paragraph you actually need, not the whole document.
Start a fresh chat when you switch topics instead of dragging the old one along.
Say the background once; don’t repeat it in every message.
And, the through-line from the last piece, don’t default to the most powerful model for work that doesn’t need it.
Right behind the token wall is another moment of confusion. Someone generates a graphic, or a summary, or a table they like, runs the same prompt again to recreate it, and gets something different. Sometimes very different. The reaction is a kind of betrayal: “I asked for the exact same thing. Why did it change?”
Because that’s how generative AI works. It’s probabilistic rather than deterministic. It predicts a plausible response instead of looking up a fixed one, so the same prompt can yield different results each time. That’s helpful when you’re brainstorming and a real problem when you need consistency or precision. Knowing which situation you’re in is half the skill.
This has a direct, practical consequence. For anything that has to be correct, like a policy detail, a figure, or a quote, you don’t want the model improvising from memory. You want it working from an actual source. The friendly version of the idea is simple: point it at the document and ask it to answer only from there, with a citation back to the page. (The technical name for this is retrieval, or grounding, but you don’t need the jargon to use it.) For most people it’s the single biggest lever for cutting hallucinations, and almost nobody reaches for it, because no one told them they could.
That same probabilistic nature is exactly why you shouldn’t trust a chatbot to calculate for you. Ask it to total a column, reconcile a budget, or compute a variance, and it will hand back a clean, confident number that may simply be wrong, or quietly built from figures you never gave it. It isn’t a calculator. It’s predicting what an answer should look like. The fix is to flip the job. Have it write the formula or script, an Excel formula or a few lines of Python, that you run. The computation then happens the same way every time. Use the AI to build the tool, not to be the calculator.
So the right question often isn’t “which model is smartest?” It’s “what does this task actually need?” Brainstorming and routine drafting want a fast, general tool. A policy answer that has to hold up wants a grounded one. Exact numbers want a formula, not a guess. It’s the same instinct as choosing a model, just one level up. Match the tool to the job.
None of this requires being technical. It requires being intentional, which is really what literacy means here.
That’s the gap I keep running into. We hand people access and call it adoption. What we don’t hand them is the handful of habits that separate a tool that fights you from one that works with you. So I built those habits into something: a short, practical, plain-language web guide for faculty and staff. It covers the prompt formula, how tokens and cost actually relate, when to start a new chat, how to ground answers in your own documents so they hallucinate less, choosing the right model for the task, and a quick checklist you can run before you press Enter.
It’s the resource I wish I could hand to everyone the first time they ask, “Wait, why did it stop?” or “Why won’t it give me the same answer twice?”
If model selection was the literacy lesson for people running AI at scale, this is the one for everyone else. A few small habits, and the walls mostly go away.
Have thoughts or want to collaborate? Email me at joepsabado@gmail.com or connect with me on LinkedIn, or visit CampusAIExchange.com for more resources on responsible AI adoption in higher education.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.