"AI" is antithetical to calm tech principles.

The Calm Tech Institute defines calm technology clearly:

• Technology should require the smallest possible amount of attention

• Technology should inform and create calm

• Technology should make use of the periphery

• Technology should amplify the best of technology and the best of humanity

• Technology can communicate, but doesn’t need to speak

• Technology should work even when it fails

• The right amount of technology is the minimum needed to solve the problem

• Technology should respect social norms

It’s my belief that the current generation of LLM-powered “AI” tools, mostly chatbot-style interfaces, fail on many of these principles. Let’s take a look at each and see why.

Note: I do understand that LLM-powered “AI” tools are not intended to be calm tech. This post is a way for me to explore how I feel about this technology through the lens of “calmness.”

If you look at the attention required to use an LLM to perform real tasks, it’s not trivial.

Since LLMs cannot be implicitly trusted due to the nature of the tech and its potential for hallucinations, they require careful oversight and guidance.

If you’ve ever been in a position of training an employee to do a job, you understand the burden this places on you. It’s often much harder (or perceived to be much harder) to train someone else than to just do it yourself.

Now imagine that no matter how long the employee has been trained, there’s always a 10% chance they do something completely unexplainable and potentially damaging to the desired outcome of their work. This is the situation we face when using LLMs to do real work.

It could also be argued that because you can set an LLM to a task and go do other things, that it does meet this criterion. While technically true, I do not think any tech that presents the benefit of “do more things in the same amount of time” has the principles of calm tech, or even human wellbeing, in its DNA.

The marketing around LLM-powered tools has nothing to do with calm.

The selling point is “do more, faster, while paying less money to real humans.”

This means that the areas of concern you could traditionally offload to another working brain now have to live in your brain perpetually (see previous point.) You’ll never be able to truly outsource something to an expert LLM in the same way you could to a human. There’s no accountability, there’s no human desire for approval or reward, and there’s always that nagging chance of hallucination.

Until those problems are solved, and I think that’s a long way off with LLM-powered “AI”, this generation of tools does not create calm. Instead:

• It creates a sense that you need to be doing more with less, which is stressful

• It makes you feel like if you’re not 10x-ing your output you’re falling behind

• You’re under constant threat of professional extinction, especially if you’re in creative fields like music, art, or software engineering

To be fair, these are all issues with the marketing, but the messaging does inform the way people look at the tools and maybe even how they use them.

In my personal experience using LLMs to write code, I find it inspires a sort of frenzy as the LLM purports to have done the thing I’ve asked it to do, which means I can push it to do more, and more, and more… until the “AI” written code has reached a point that it’ll take me weeks to review properly. This is not a calm feeling. It’s one of drowning in work that I dislike while outsourcing the work I enjoy to an emotionless machine.

Some might argue that LLMs can do non-calm tasks for us and reduce our need to engage with them. I would challenge those people to come up with a non-calm task that’s non-critical enough to trust to a machine that has a 10% chance of completely fabricating the output: falsely confirming the task was done, falsely confirming how it was done, or sharing fabricated information in the doing of the task.

Imagine asking an LLM to book an important medical appointment for you, only to find out that it misreported the date of the appointment to you, or booked it for the wrong person at your doctor’s office.

In my mind, this mode of failures preclude LLM-based “AI” from doing anything that’s important enough to cause me stress. So, it’s relegated to the unimportant and therefore already calm tasks of my day, which defeats the purpose.

I do think the current generation of LLM-powered “AI” chatbots make use of the periphery enough to perhaps not fail this point, but only on the surface.

For instance, rather than actively searching Google and scanning through the results, I can set ChatGPT to a web search using a “research” mode and then go do something else.

The problem is that I still have to scan through cited sources (if any are provided) and confirm the information the LLM has given me, thanks to its propensity for confidently relaying hallucinatory information.

It doesn’t eliminate the effort, it just defers it to a later stage.

Plus, to engage with an LLM-powered “AI” chatbot, it generally must have focus. Whether it’s in your IDE (having the “AI” tab open) or in your browser, it requires quite a bit of attention to use properly.

The one exception to this which I’ve had personal experience is Meta’s LLM-enabled glasses. These are a passive tech that is worn and called upon when needed, which I think fits better into the idea of calm tech. The downside is that, being a part of Meta’s ecosystem, these will undoubtedly be used to flood you with notifications that distract you from your day, failing this point entirely.

LLMs are good at producing average results. This is part of their nature.

An “average best output” machine will never amplify the best of technology nor humanity.

In fact, it seems that as LLM usage spreads, it’s being used predominantly to produce empty-calorie brain rot, both in the form of consumed media and vibe-coded apps that break at the first real-world use.

LLM-powered “AI” tools are also like Dunning-Kruger amplifiers. Users ignorant of a specific field suddenly feel like experts because they “asked ChatGPT.” It’s becoming quite exhausting explaining to these people that LLMs can, and often will, be wrong.

We now have a whole new class of arguments where despite your own expertise in an area, you’re left sparring with ChatGPT via its human proxy who trusts the machine more than they trust you.

It’s clear to me that LLM-powered “AI” fails spectacularly on this principle.

LLMs communicate primarily through human language, if not speech. Some LLM-powered “AI” tools can actually talk, but I think worrying about that borders on taking this principle too literally.

I think the point of this principle is that technology should use optimal means of communication for the scope of what it needs to communicate. If you look at the clarification for this principle, it reads:

• Does your product need to rely on voice, or can it use a different communication method?

• Consider how your technology communicates status.

To me, it seems that this principle mostly applies to technology that demands your attention. In that sense, most LLM-powered “AI” tools are more on-demand than in-command, which means they’re not sending notifications to you unless you’re working with them specifically.

The most interesting question to me around whether LLM-powered “AI” tools satisfy this principle would be: is human language the best interface for the work being done by these tools?

This will vary wildly, but in general, I’m not sure human language is the optimal way for machines to communicate important information to us, as evidenced by the propensity of LLMs to be extremely verbose when they don’t need to be.

A light turning on is clear and concise.

A thousand-word essay starting with “You’re absolutely right, and here’s why you’re nailing it…” from a digital yes-man is a source of great confusion if you don’t blindly trust the output.

This principle provides a very interesting lens to look at LLMs through.

1. When an LLM fails, what happens?

2. What happens if that LLM is your only way to do the work you rely on it for?

For point 1, I think we’ve all seen LLMs try and fail to fulfill a request. Usually, when it fails, it provides wrong information rather than refusing to provide the information. The insidious part of this is that the failure is invisible. Unless you know enough about the task to know when it’s being done wrong, you’ll never know the AI failed until someone more knowledgeable tells you.

The best case scenario is if the server has an outage (in which case the LLM doesn’t work) or if the response is a garbled mess of tool-calls (like I’ve seen using Claude in Roo Code fairly frequently.) At least in these failure modes, you know something has gone wrong.

Now, what happens when one of those failure modes occurs and you can’t do the work without the LLM?

This might occur if you’re doing work you have no business doing, like an accounting manager generating flyers for the company picnic. Or the CEO designing a documentation website.

These are areas in which you have not invested the amount of time to personally know how to do the thing, so when your crutch (the LLM-powered “AI”) is unavailable, you have no intrinsic ability to complete the work in the time allotted.

I’ve seen anecdotes that this scenario is happening with some regularity in tech companies, especially with more junior employees who don’t have the foundational knowledge required to go back to their own skills when their tools fail.

The bottom line here is that almost every failure mode of an LLM-powered “AI” tool that I’ve seen is not a failure mode that allows the tech to continue working in any reasonable sense.

The problem with evaluating LLM-powered “AI” tools against this principle is that the current generation of tools are generalists, and the messaging is that they can solve every problem.

The minimum amount of technology to solve every problem is probably infinite, so the principle kind of short circuits here.

But if we look at a specific use-case, such as writing copy, do LLM-powered “AI” tools utilize the minimum amount of technology to solve the problem?

I’d argue that the minimum amount of technology required is a pencil, paper, and a human brain. By that heuristic these tools fail spectacularly.

But to be a bit more realistic, we must look at modern tools for writers in general:

• A keyboard

• A mouse

• A screen

• A computer

• A word processor

When you break it down like that, it’s quite a stack.

Now, what happens in an LLM when you ask it to write a paragraph?

This is an area that I do not understand thoroughly enough to evaluate against the known tech stack of a traditional writer, but I would be interested in learning what the full call chain of a simple copy request in ChatGPT looks like. If anyone knows of a resource like that or a way to visualize each step of the process when you send a command to an LLM, let me know!

Caveats aside, I do believe that the output of LLM-powered “AI” tools when it comes to specific domains like writing is so sub-par that it’s often not worth whatever resources it requires.

I think LLM tech itself does well here.

It gives you a familiar interface - human language - and can solve problems conversationally. This is somewhat revolutionary in terms of “respecting social norms.”

Usually, when a new technology emerges it, you must adapt your communication to its interface and syntax. Using human language, however sub-optimal it may be for human-machine interactions, is the easiest UI I can imagine teaching someone.

I think that my be the key to why we’re seeing such mass adoption, even among non-technical users.

The business side of LLM tech, however, has trampled over social norms like they never existed.

I think the presence of an LLM-powered “AI” wearable simply called “Friend” is evidence enough.

Not to mention the idea that you simply don’t need to hire anyone for anything now, because “AI” can do it all. This, while being patently false, has also either hurt the job market, or given C-suites an excuse to slash positions in certain fields. I’m not sure which, and I’ve seen it explained both ways.

Also, how much of our richness of experience is derived from doing a task slowly? I expect we’ll discover that most the experiential value comes from the process, rather than the destination. “AI” gives us a shortcut much akin to only reading the last chapter of the book to find out what happens, except it might all be made up and we’ll never know unless we go back and read the book anyways (which we won’t.)

We’re watching Jurassic Park unfold before our very eyes, as VC-bloated tech startups rush forward so fast to find out if they can, that they never stop to think whether they should.

As a society, we’re now seeing the ramifications of jamming computers in our toddlers' pockets and putting our lives on social media. You can see the response in some countries like Australia, where they’ve instituted a social media ban for kids under 16 years old.

I expect LLM tech will follow a similar trajectory, but we’re rushing forward so fast that the harm will be set in too deeply before we stop to reflect. And that, I think, is the biggest tragedy of all.

When we see what a generation of people raised on AI-generated Sora feeds and hallucinated information from ChatGPT looks like, we might wish we had done things differently.

Clearly, I don’t think so.

Can it be used in a calm way? Sure. But the current marketing machine isn’t interested in telling us about that. It wants us to 10x our output for half the pay or else.

To be clear, I don’t think the technology is evil or inherently bad. I think the way it’s been incorporated into tools and marketed to the masses will have negative ramifications. I think outsourcing our ability to reason and think is bad. But this mostly comes down to the fact that humans on average are lazy, and most of us will take the easy way out every time, even if we know it’s not the best choice for ourselves or those around us.

I also think LLM technology stands to be absolutely transformative for assistive tech, but that also hinges on eliminating hallucinations. You cannot tell a blind person the cross-walk sign says “go” if there’s any chance of failure at all. Though maybe the company that makes a digital seeing eye dog will rack up enough cash to afford the settlement when it happens. We do have self driving cars, after all.

Pessimism aside, I think we can use this tech mindfully and without much personal detriment if we’re careful. There are, however, potential harms to consider. Though it’s a very muddy topic to navigate, it’s worth paying attention and watching for tangible, real-world ramifications happening right now as a result of the tech. It’s up to us to evaluate whether those harms are a reasonable cost for what the technology brings to the table.

Whatever your determination is, I believe the current generation of LLM-powered “AI” tools are antithetical to most of the principles of calm tech.