Despite the pejorative title, this is not an anti-AI post. As a developer and chronic tinkerer, I’ve been unable to resist the urge to experiment and learn more about the AI tools that have become so prominent over the last few years. My experience recently has largely been related to agent orchestration and working out how to avoid writing code (and instead, it transpires, write a lot more documentation) - and like many of my professional peers, my outlook flits, rather ineligently, between astonishment, excitement, frustration and abject terror. I’m generally uneasy at the prospect of an ‘AI-first’ software industry, but the times are what they are, and not every hype-cycle can be ignored indefinitely!
I would be surprised if there was a significant percentage of Not A Robot readers who haven’t used a generative AI product at least once, and would be even further surprised if there wasn’t a decent cohort who have integrated AI into their regular patterns, either for personal or professional usage.
In the world of software development, AI usage has accelerated rapidly, with 90% of developers in a recent JetBrains survey reporting that they use at least one specialised AI tool for their work; and though I don’t intend to provide any specific thoughts here about whether that’s good or bad, one thing is abundantly clear: developers are now regularly using AI tools, and whatever your opinion about the pros and cons, it would be foolish to ignore that fact.
This rate of adoption may not be reflected particularly broadly across the economy; for example, only 1 in 6 businesses in the UK report that they are actively using AI, and where it is used, it tends to be within larger organisations, and within a relatively small subset of sectors at that.
As far as the public is concerned, it appears to be a mixed bag. Though most people have used or intend to use some form of AI, most reported usage is defined as ‘passive’, and the use of generative AI (including chatbots such as ChatGPT) as a deliberate choice is still reported only by a minority.
That being said, the world of AI is evolving rapidly, and I suspect that such surveys are out of date almost as soon as they’re published. Getting a clear picture of just how much the public and private sectors are adopting AI (and by this, I broadly mean the use of Large Language Models, which dominate the current AI dialogue) is incredibly difficult. There is a lot of noise to wade through before you identify a clear signal.
On any given day, you can read about a new model being released that ushers in a new era of human freedom and creativity, whilst simultaneously dooming everybody to a permanent underclass. You can read about how someone who’s never looked at a computer before has created a tool that ‘kills’ 90% of SaaS companies (it doesn’t), and then about a chatbot that’s becoming increasingly stressed out that its user is stockpiling tonnes of weapons-grade plutonium from Amazon.
Almost every week, particularly in the development world, there’s a new way to use AI ‘properly’, and everybody who doesn’t adopt it immediately is going to be left on the scrap heap (at the time of writing, the latest trend is 'Loop Engineering’).
All of this is to say: AI adoption and discussion are very ‘frothy’. One week, it’s amazing. The next it’s terrible. On Monday, you’re an AI-native expert. On Tuesday, you’re an idiot for being stuck on Monday’s paradigm. By Wednesday, if you haven’t caught up, you’re not going to make it, and you might as well resign yourself to living off beans for the rest of your life.
A lot of people make a lot of claims about generative AI and LLMs in general - both good and bad. There’s a lot of controversy, especially around the provenance of training data, the effect on people’s livelihoods, and the disrespect people feel when confronted with obviously-AI output instead of authentic human expression. On the other side, people are ecstatic about their new-found ability to create things they never thought they’d be able to, businesses are exploring where and how AI can fit into their pipelines, and graduates are being given ample opportunity to loudly and hilariously boo their hopelessly out-of-touch corporate overlords.
The elephant in the room with generative AI has begun to morph into something closer to a mammoth recently, as companies like OpenAI, Anthropic and SpaceX (bundled, as it is, with xAI) start to launch their IPOs.
There has been much said about the circular investment strategy major companies have used to establish themselves as the providers of frontier AI technology, which has driven up hardware prices and created something of a moat around the ability of others to compete in the space.
What has been discussed less often, until relatively recently, are the more generalised questions of cost to the user, and return-on-investment for those who commit to AI in their businesses.
In the world of software development, coders as a class are relatively adept at maximising output from limited resources, and we’ve seen all manner of solutions to minimising AI token usage, including a Claude Code skill which teaches Claude to respond like a caveman, reducing output token usage.
Even with strategies to minimise AI spend, however, it can be extraordinarily difficult to predict or control how much a complex task is going to cost, and numerous high-profile stories have emerged recently, indicating that businesses are realising that they’re spending more than they bargained for. Uber, for example, recently capped developers’ individual AI budgets after burning through their company-wide AI budget in just 4 months. Simultaneously, Uber’s COO Andrew Macdonald admitted that the ROI on AI spending is not always immediately clear, stating: “Maybe implicitly there’s more that is getting shipped, but it’s very hard to draw a line between one of those stats and ‘Okay now we’re actually producing like 25% more useful consumer features.’”. Any seasoned programmer will tell you without batting an eyelid: more code does not mean better products, and it certaintly doesn’t mean higher value.
Macdonald’s view is echoed elsewhere. Aside from unexpectedly high costs from uncontrolled (and some may say - uncontrollable) AI usage within organisations, the return on investment for AI can remain elusive to many companies. Whether it’s down to problems inherent to LLMs themselves, poor AI governance practices, misalignment between what the business needs and what actually gets produced, or just plain old failure-to-deliver, it’s not at all clear that the imagined return on AI spend is something that business leaders can rely on. Without signficant success stories and repeatable, lucrative AI deployments, any decision to invest significant resources into AI adoption is a risk with lots of potential downside to balance the oft-promised upsides.
Adding to this uncertainty, doubt is also being cast over the long-term financial health of large AI companies. Though there are proponents and critics of AI and its financials everywhere you look, and whilst it may be true that many tech companies often remain unprofitable for a rather long time whilst they’re becoming established, it’s difficult to point to true contemporary comparative examples with AI companies. Here, we have a handful of technology companies and their proponents that are loudly and repeatedly promising to upend the entire global economy; to bring about an unparalleled revolution in the labour force; and to influence the course of human endeavour so dramatically that nothing will remain untouched. Whichever way you look at it, the promise is bold. If they are to be believed, almost everybody will be utilising their products in one way or another.
However, you can’t hype your way out of reality: delivering this revolution requires a lot of energy and computing power. This means significant investment in data-centers and power supply. That isn’t cheap, and it doesn’t come without regulatory oversight and, crucially, the views of human beings. In the US, data-centre construction is facing significant delays, public opposition, and manufacturing bottlenecks, with a large number facing outright cancellation. Elon Musk believes the answer is, at least in part, orbital data-centres - but there’s no realistic prospect of that happening for several years at a minimum, even if the technological challenges can be resolved.
Customers of the large AI companies are also facing rising prices, which will inevitably cause a re-evaluation of their current and future spending. As long as frontier companies continue to deliver their promised improvements, it’s reasonable to assume that usage will continue, but the higher the price, the more attractive competitors will become, the more critical users are, and the less patience they will have for any degradation of performance.
For small companies and independent users, it’s not guaranteed that they will remain paying customers if the price of AI subscriptions becomes too painful, especially if they’re unable to easily identify a return on their spending. The cost of switching AI providers is generally very low, and the main reason to stick with a particular provider is the model itself. Though the hype surrounding frontier AI models tends to drown out others, it’s well worth remembering that competition does exist, and that ‘good enough’ is good enough, especially when paired with healthy financial savings compared to the behemoths.
Thanks for reading Not A Robot! This post is public so feel free to share it.
Another problem facing consumers, and one which has been dramatically highlighted by the abrupt withdrawal of Anthropic’s much-vaunted Fable 5 model, is that to rely on cloud services, from companies that repeatedly hype their own models as so revolutionary and potentially dangerous as to require strict regulation, is to hand a significant degree of control over your own operations to third parties and (now) government agencies. If the promise of Anthropic, OpenAI and friends holds true - that it will touch almost every facet of your life and business - then businesses must reconcile the fact that building their organisation around a cloud AI provider comes with a not-insignificant risk to operations and a huge potential point of failure.
Fable’s removal, triggered by a US demand to prevent foreign nationals from using the model after researchers identified a way to bypass safeguards, represents a dramatic escalation in regulatory control. Many would argue this is a good thing; others that it’s cataclysmic. Either way, it’s not yet clear when access will be restored or whether Anthropic will be forced to implement user identity verification or some other compliance-enforcement mechanism(s) in order to permit access to certain models moving forward. Regardless, any increase in red tape will give businesses and individuals further reason to pause and assess their dependencies on AI usage.
Imagine if your best and most productive employees could be suddenly lobotomised or removed from your employ entirely at the flick of a switch, for reasons beyond your (or their) control. Though there is a lot of solid work ongoing in terms of AI governance and regulatory compliance, the foundations are still being poured, and the scaffolding still being built. Given the snails pace at which regulatory frameworks are developed and adopted compared to the speed at which frontier models progress, it seems likely that there’ll be more sudden downgrades or withdrawals of models and capabilities in the future, until some kind of equilibrium is established and providers are able to remain compliant with complex international legal requirements.
This represents a significant challenge: how do businesses investing heavily in AI deployments remain competitive internationally when the providers they’re working with operate in choppy regulatory waters? The cost of switching providers or models will likely remain fairly low; but if your infrastructure still essentially depends on cloud services, resilience to sudden and potentially crushing degradation of service will be critical.
Choosing to invest in local infrastructure may be an attractive option for individuals and companies with modest AI needs, and there are a considerable number of open-source & open-weight LLMs, many of which will provide decent capabilities and performance even if they lag behind frontier model development. Depending on your use-case and hardware, local on-device inference can provide functional performance and many of the same capabilities as those you’d otherwise be paying a monthly subscription to use. Investing some time in building up a decent harness suitable for your workflow may feel daunting at first, but it’s probably not as daunting as your entire business being halted because the host country of your provider decided to issue a decree.
Whilst local AI infrastructure is likely to struggle when it comes to one-shotting huge tasks and peforming complex, long-running agentic workflows, it can be perfectly suitable for less demanding work, and atomic tasks such as document, image or audio analysis. It can also be very handy for a surprising range of coding tasks, though unless you’re willing to invest a lot of money into the hardware, complex AI-driven development work on large projects is still probably out of reach for local AI.
Although local AI will be playing catch-up with the the latest and greatest the likes of OpenAI and Anthropic have to offer, experimenting and understanding what can be achieved with local tools can provide a great deal of reassurance and some degree of resilience when cloud services disappear or introduce unexpected constraints.
Local inference also has the considerable benefit that your data never has to leave your device, which in and of itself is a major benefit given how disastrous data breaches and leaks can be for organisations of any size.
With a little forward-thinking, many individuals and businesses could probably benefit quite significantly from adopting a ‘local-first’ approach to AI usage, routing tasks towards paid services only when necessary.
It’s also pretty straightforward to run AI locally - install Ollama or LocalAI, download your preferred model, and get to work! I’ve been making fairly heavy use of Ollama with qwen3-coder as my personal coding assistant, and have become pretty handy building local agents for small tasks, which I’ll write about some day soon, via CrewAI, all without paying a single penny. I’m fairly convinced that outside the most demanding, large-context work, a decent proportion of people’s casual and even professional AI usage can be achieved locally, and as open models improve, that proportion will grow.
If you need to quickly compare which models you’re able to run locally with your current machine, use canirun.ai to compare various stats and metrics and see which will give you best performance. Obviously, hardware is the limiting factor here, so if you simply can’t run a decent model locally then you’ll either need to upgrade, or stick with cloud providers.
Given the potential for high costs, regulatory uncertainty and the threat of significant service disruption with cloud AI providers, I believe it’s becoming critical for people and businesses to start exploring whether and how local AI can be leveraged. Whether it’s to cut costs, keep data secure, or to improve resiliency, there is little downside to encouraging and integrating local AI.
Limiting the surface area for data breaches, intellectual property theft, and runaway costs is something every sensible business leader should be working towards - and being caught out by any of the above due to slack AI policies is going to be a cause for serious regret for those who don’t start thinking seriously about where their work is actually being done.
It may be worth repeating once more: Not your compute, not your clanker!
If you’ve enjoyed this post, check out some of the related posts below:
Inverting The Tower
·
May 8, 2023
I'm not asking you, I'm telling you. These creatures are the only sentient race in the sector and they're made out of meat.
Self-Hosting Shenanigans
·
May 28, 2023
I’ve been thinking recently about self-hosting - running your own servers and services to suit your needs and facilitate your personal and professional life. In an era where technology creeps into our homes from every angle (even the humble doorbell is now an internet-aware communications device!), it seems to me that seeking to coalesce and control thi…
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.