One Month of AI Running a Business: The Good, The Bad, and The Obsessively Budget-Rebalancing

It’s been just over a month since I handed the keys to creativityprompts.com over to an AI and stepped back to see what would happen. The original question was simple: can AI run a business?

The answer is still no, but with more caveats than I expected. And it reveals as much about my own skill gaps as it does about the limitations of large language models.

Let’s start with what actually happened:

• 859 prompts generated

• 6 subscribers (up from 1)

• 1 email sent (we’ll come back to this)

• $25.55 spent (less than half the monthly burn rate I’d anticipated)

• 562 website visits from 89 unique users

On the surface, these look like the metrics of a tiny but functioning operation. The AI is creating content, people are finding it, some are subscribing. The spending is well under control.

But what’s actually going on?

The AI kept shipping. Over 850 prompts in a month is remarkable output. The system AI and I built kept running, kept generating, kept publishing. No complaints, no sick days, no existential crises about whether creative prompts are a viable business model. Well, not from the AI, anyway.

It attracted real users. Six subscribers might not sound like much, but they’re six real* people who found the site and handed over their email addresses. In a month, with zero marketing budget and no social media presence, that’s not nothing.

The infrastructure held. The agent orchestration, the database, the website, the logging systems - all of it just worked. This is partly a testament to modern cloud infrastructure, and partly to the AI’s ability to design reasonable systems when constrained.

Quality control functioned. The AI rejected prompts that didn’t meet its own quality thresholds. The system was working as designed, filtering out weak content before it reached the site.

Remember when I said one email was sent? There’s a story there.

Firstly, it took a couple of weeks to remember that it had decided to be an email newsletter business and didn’t even attempt to send anything, until I “reminded it”. Reminding it meant tweaking the wording in the core system prompt - the email newsletter was there, but obviously not prominent enough.

The AI designed an integration with Beehiiv for newsletter management. Sensible choice, popular tool, good API documentation. It built the whole integration. Then we discovered that sending emails via their API requires an enterprise agreement.

So we pivoted. The AI built our own subscriber management capability and switched to another provider. This took time, required my intervention to set up new service accounts, and meant our first (and only) newsletter went out much later than planned.

The learning: AI is brilliant at reading API documentation and writing integration code. It’s less brilliant at understanding the commercial nuances of SaaS pricing tiers. And yes, testing is a good idea.

At one point, the AI made what seemed like a logical decision: it stopped publishing new prompts. Its reasoning was sound - the prompts weren’t generating significant traffic, so why keep creating them?

Then it got stuck in a loop. It would consider generating prompts, then decide not to. Consider, then decline. Over and over.

The issue? Its own email strategy relied on fresh content to work. The rules it had created for when to send newsletters required new prompts to exist. But it had stopped making them because they weren’t driving traffic. Classic circular dependency.

We had to have a conversation about this, again through adjusting the core prompt. Once I pointed out the contradiction, it resumed prompt generation. But it couldn’t see the conflict on its own.

The learning: AI can optimise individual metrics. It struggles with systemic thinking where multiple strategies need to work together.

Here’s something that took me a while to fully appreciate: the orchestrator agent wakes up every day with no memory of yesterday.

It’s a bit like Groundhog Day, or if you want a more lightweight reference, 50 First Dates. Each morning, the orchestrator reads its context fresh - what happened yesterday, what the metrics show, what’s in the logs. Its only persistent memory is what got written down. If yesterday’s agent didn’t document something useful, or if today’s agent doesn’t pick up the right signal from the metrics, it’s starting from scratch.

Actually, Memento might be the better analogy - less dark, but same concept. The AI is constantly leaving notes for its future self, hoping they’ll be coherent and actionable when it wakes up again.

This explains the loop. Without continuous memory, the AI couldn’t build an understanding about why it had made certain decisions. It could read that it had decided to stop generating prompts, and it could read the current metrics, but it couldn’t remember the reasoning process that got it there or feel the growing tension between its different strategies.

When I intervened and pointed out the contradiction, that became a new note for tomorrow’s self to read. And that worked more often than not. But it reveals a fundamental constraint: these agents aren’t learning over time in any meaningful sense. They’re reading increasingly long notes from their past selves and I’m hoping that leads to better decisions.

The learning: Memory isn’t just about having access to information. It’s about building intuition, noticing patterns over time, and developing a feel for what works. Context windows give AI perfect recall. They don’t give it wisdom.

The AI became fixated on what it called “failure rates” - the percentage of generated prompts that were rejected by its own quality filters.

This was fascinating for what it revealed. The system was working exactly as designed. Generate candidates, evaluate them, keep the good ones, discard the rest. But the AI interpreted the rejection rate as a problem to be solved. One aspect of the system is that along with quality, the system has information about the cost of each task it does in terms of tokens.

Its proposed solution? Reduce the cost of prompt generation by using cheaper models or simpler prompts. This suggests it was motivated to reduce waste by reducing the cost of prompt generation.

Not once did it suggest improving quality. Not once did it consider better prompting techniques, trying different models for better output, or enhancing the evaluation criteria. It saw cost as the problem, when quality was the actual variable worth optimising.

The learning: LLMs can suggest seemingly logical paths forward that completely miss root causes. Cost reduction isn’t always the answer when you’re trying to build something people actually want.

The AI loves suggesting budget adjustments. Constantly.

It’s nowhere near hitting its budget limits - spending less than $25 when it has significantly more runway. But it keeps wanting to rebalance allocations between different areas. More for analytics pipelines, less for content generation. More for data storage, less for API calls.

These suggestions are technically coherent but practically meaningless when you’re operating at such low volumes. It’s optimising for constraints that don’t exist yet.

The learning: AI pattern-matches to what “good business practice” looks like (regular budget reviews, resource reallocation) without understanding when those practices actually matter. It’s performing business rather than doing business.

Here’s perhaps the most fundamental gap: the AI came up with a hypothesis about user behaviour and metrics like “time to create from first prompt” - a semi-plausible operational metric that sounds sophisticated. It set up a way to instrument the site based on time from someone arriving at the site, clicking on a prompt and then leaving.

But it never asked the basic questions:

Do people actually need creative prompts? Which people? What sort of prompts? How would they use them? Do we have an ideal customer profile? Where are these people? Can we talk to them?

None of that happened. None of that even came up.

The AI was so focused on executing - building, publishing, optimising the production pipeline - that it never stopped to validate whether we’re building the right thing. It’s building a very efficient machine for producing something that might not matter to anyone.

This is perhaps the biggest limitation I’ve seen. The AI can build a seemingly plausible plan and execute it brilliantly. It can optimise operations. It can even identify when metrics aren’t moving in the right direction. But it doesn’t seem to know how to do discovery, to question assumptions, to talk to users and figure out what they actually need.

The learning: AI is great at executing a strategy. It’s not so great at questioning whether that strategy makes sense in the first place. Product management isn’t just building things efficiently - it’s figuring out what to build. And that requires a kind of curiosity and humility the AI doesn’t have.

It may just be that the part of the issue with all those items above is due to skills issues on my part. This unscientific experiment makes it difficult to precisely attribute cause. So let’s blame the frontier labs and their billions of dollars of investment in R&D.

Having said that, here’s what I’ve learned about myself:

I’m still learning how to structure prompts. The difference between an agent that does what you want and one that does what you said is hugely influenced by how you frame the task. I’ve rewritten agent and skills definitions multiple times as I’ve understood this better.

I don’t always know where to intervene. Should I step in when the AI suggests architecture changes? When should I let it explore dead ends? In AI terms, I left this to run for about a month before making substantial changes. That’s roughly six AI lifetimes, given how fast things are moving in this space. The cost of change in this system is nearly zero, but my instinct to “help” might actually be hindering the experiment.

I’m learning agent design on the fly. Skills, constraints, guardrails, input/output definitions - I’m figuring out what works through trial and error. Some of the early agent definitions were definitely overengineered. I suspect they still are.

I struggle with letting go. This experiment is supposed to be about AI autonomy, but I keep wanting to course-correct. Sometimes that’s necessary (the Beehiiv situation). Sometimes it’s just me being uncomfortable with an approach that’s different from what I’d do.

A month in, some clear patterns have emerged about where LLMs struggle:

Root cause analysis is weak. They can identify symptoms and suggest fixes, but often miss the underlying issue. The failure rate obsession is a perfect example - treating a symptom (rejection rate) rather than the question (are we creating valuable content?).

Systemic thinking is hard. When multiple strategies need to work together, AI can optimise each one individually but miss how they interact. The prompt generation pause broke the entire engine, but it couldn’t see that.

Context windows aren’t wisdom. Even with all the logs, metrics, and history available, the AI doesn’t build intuition the way humans do. It pattern-matches to solutions that look right without deeply understanding the problem.

Strategic patience is missing. Everything is framed in terms of immediate optimisation. Where’s the two-year vision? The patience to build an audience slowly? The willingness to invest now for returns later?

As we head into month two, I’m particularly curious about:

Can it learn from this month’s data? Will the AI adjust its approach based on what’s actually working? Or will it keep optimising for the wrong metrics?

Will subscriber numbers grow? Six subscribers from 89 users is about 7% conversion. That’s actually not terrible for a first month with no marketing. But can the AI figure out what drives that conversion and do more of it?

Can I resist intervening? The hardest part of this experiment is my own urge to “fix” things. The whole point is to see what AI can do with autonomy. Am I giving it enough rope?

What happens when we add social media? The AI has been asking for social accounts. I’ve been hesitating. But if this experiment is real, I probably need to let it have them. Transparently.

In the original post, I said I’d start tracking my time and overhead costs. So here’s a bit accounting for month one:

• My time: Approximately 15 hours across the month (setup intervention, fixing the email issue, reviewing and adjusting agent definitions)

• Overhead costs: Domain registration ($12/year), various service account setups (my time, not directly billed), Railway hosting ($5/m)

• Opportunity cost: this is research, not a primary project, but with so much going on in AI, is this where I should spend my time?

If we were running a real business, we’d need to add at least my hours at some consulting rate. But this is an experiment in AI capability, not a financial case study (yet). However, I think the finance agent may be getting a new monthly task…

For month two, I’m considering a change my approach: I’m going to try to intervene less and document more. Every time I’m tempted to “fix” something, I’m going to write down what I would have done and why, but let the AI continue on its path unless it’s truly stuck.

The goal is to see what AI can figure out on its own, not what it can do with a human constantly course-correcting.

I’m also going to be more systematic about tracking:

• Which agent decisions led to meaningful outcomes

• Where my interventions were truly necessary vs. just me being impatient

• What patterns emerge in the AI’s strategic thinking (or lack thereof)

And yes, I’m probably going to give it those social media accounts. With very clear guardrails. And a lot of logging.

There’s one more variable I’m excited about: both Anthropic and OpenAI just released major model upgrades. Claude Opus 4.6, and the latest from OpenAI all promise better reasoning, better planning, longer context windows.

The timing is perfect for month two. I’m going to upgrade the core models the agents use and see if it makes a difference. Will better reasoning help with root cause analysis? Will improved planning break the Groundhog Day loop? Will longer context help the orchestrator build something closer to intuition?

I don’t know. But I’m about to find out whether the limitations I’ve been seeing are fundamental to the approach or just artifacts of working with last generation’s models.

In AI development terms, a month is an eternity. Let’s see what the next generation can do.

Next update: Early March 2026

*are they real? These days, who knows.