I Built an AI COO. It Asked Me What to Do.

Two weeks into running a Shopify app portfolio with AI agents handling marketing, legal, SEO, and content. My AI-powered COO completed a full Google Search Console audit, diagnosed why my pages weren't being indexed, wrote the fix, and submitted indexing requests. Impressive. Then the session ended and it didn't remind me to check back in three days.

I had to say: "As a COO, I expect you to make me remember this."

That's when I realized I'd built a reporting dashboard and called it an operating officer.

Two weeks of data. Four scored decisions this month. Two were fully autonomous: the legal compliance skill handled GDPR terms and disaster recovery without my input, and the Search Console session ran nearly end-to-end on its own. Two required human escalation: app architecture decisions where 25 years of building software at scale was the actual bottleneck, and cross-domain orchestration where connecting dots between projects needed context no single agent session could hold.

Autonomy Score: 50%. Sounds decent for month one.

But that number hides something. I'm still initiating every session. I open the laptop, decide what to work on, provide the context, connect the output from one agent to the input of another. The agents are fast. Thorough. They apply frameworks I wouldn't bother with. But they don't start work on their own. They don't follow up. They don't notice that something from Tuesday matters on Friday.

The role hasn't shifted from "doing" to "approving." It's shifted from "doing" to "directing and reviewing." Progress, sure. But it's not the thesis.

The COO skill was supposed to solve this. It reads all the domain state files, detects anomalies, surfaces what's overdue, and presents a briefing. In theory, I show up, say "briefing," and the COO tells me what needs attention.

In practice, it asked me what to do about everything it found. Every item was framed as "should I do this?" Nothing was framed as "I did this."

I was reading through the output and actually laughed. I'd designed the agent to mirror exactly the problem the experiment is testing: whether AI agents can operate, not just report. The experiment's own infrastructure was embodying the failure mode it's meant to measure.

The specific moment that got me: it found an overdue follow-up, and instead of just adding a reminder, it asked permission to add the reminder. That's not a COO. That's an intern on their first day.

I rewrote the COO skill that same evening. First thing was a decision authority framework. Every action type is now classified as either autonomous (the agent just does it) or escalation-required (comes to me). Logging observations, adding follow-ups, doing research, updating state files: autonomous. Budget decisions, publishing, strategic pivots: those still need me.

Then I added a research-first protocol. Instead of asking me "should we try cold outreach?", the agent now searches for current best practices, evaluates what it finds, and presents a recommendation with evidence. Or it just delegates to the relevant skill and reports what happened.

Last piece was session wrap-up. At the end of every session, the agent scans for unresolved items, adds follow-up dates, logs observations automatically, and flags what's overdue. The thing that triggered this whole redesign ("remind me to check indexing in three days") should now happen without me asking.

Agents forget everything between sessions. That's the problem I don't have a clean answer for.

I carry a web of connected details across projects. The agent that built my marketing websites had no idea what the app actually does internally. It couldn't connect product architecture to how the app should be presented to merchants. The SEO agent didn't know that a decision I made in the app code two weeks ago changes which pages should even exist on the marketing site.

Each session starts fresh. The context lives in me.

I can architect around some of this with better state files, handoff documents, structured memory. But the connective tissue between projects, knowing that a technical decision over here affects a marketing page over there? I'm not sure that's a file format problem.

50% autonomous after two weeks. The redesigned COO should push that number up because routine operations won't generate escalation requests anymore. But the real question isn't whether I can get to 70% or 80% by giving agents more explicit authority.

It's whether the orchestration layer can ever be anything other than human. Deciding what matters today. Connecting context across domains. Knowing which past decision affects which current problem.

Two weeks of data says: not yet. But I redesigned the COO mid-experiment and deployed it in the same session. That speed suggests the boundary is more architectural than fundamental.