The Rise of Consumer On-Prem

Once upon a time, everyone who had a website was running a server. This was a pain in the butt. You need to know how to run a server, and if you have lots of traffic, how to manage a fleet of servers and network switches. And also how to write software and deploy to these servers. Hard to do. This was the on-prem world.

Cloud got big 20 years ago because it’s easier than on-prem. Just put the software on AWS. Even better, buy a subscription to software running on AWS. Total abstraction.

I predict that on-prem will get big for a class of people who never had it: consumers.

I’ve been running OpenClaw on an old Mac mini for the last two months. It turns on the espresso machine for me every morning. Summarizes the news. I haven’t used web-based Anthropic, Gemini, or OpenAI since.

Perhaps the coolest part of my OpenClaw setup was when I started sending it voice messages. To transcribe them it downloaded Whisper and ran it locally. On the Mac mini. Wow. At the time I was paying $25 per day for Anthropic credits, so the idea of running a model locally for free was A+.

Oh, here is another observation. The people running the big AI companies are nuts. In a good way if you are an investor in them. They will probably generate lots of returns. But they are not thinking about me. I am priority 10^20 on Altman or Amodei’s list. I believe none of the privacy guarantees they offer.

So here’s what I think will happen in the next couple of years.

People like me — with something to lose and an understanding of technology — will run an OpenClaw-like agent on a Mac mini with the M* chip. We’ll use open-source models. It will be entirely private. This will be my personal and my family on-prem cluster. It’ll be Apple hardware. Apple under Cook is great at iterating on hardware spec and this is all they need to do. Increase the FLOP capability of the M chip until it can run a model as good as Opus is today. Apple sucks at software, so they won’t make the agent. Whoever makes the agent and grabs all my context will have a massive moat (unless they are silly enough to offer context export).

People who are mass market won’t understand the privacy implications of cloud vs on-prem. Their only hardware might be a phone. They will buy a cloud agent and pay $5/month for it, which will be cheaper than a Mac mini. In a sci-fi novel I could imagine their agent would be called “MTHR.” MTHR does everything for them. Files taxes, pays bills. Humans just live. MTHR does boring.

The above are two companies. Brand matters today in software. The former is: privacy, privacy, privacy. The second is: just fix it, I don’t care. This means, to the thesis of this article, affluent class’s Mac mini = box at home that runs your important code = consumer on-prem.