AI Offices, Pokemon Speed Runs, and the Agent Harness Problem

Open Source Ready is a weekly newsletter for developers who care about the future of open source in an AI first world. Each issue recaps our Thursday livestream where we demo tools, debate ideas, and build in public.

The big demo. I ran Pokemon Red headless inside PyGameBoy on a stereOS VM. The goal: speed run from boot to Viridian City using agents.

The harness is Claude Code. Screenshots every 10 turns go into a frames folder. The agent reasons about what it sees and sends directional commands. First six logs are in the papercomputeco/pokemon repo so people can see the progression.

Discoveries along the way: doors have an 8-second cooldown so you don’t walk back through them. After getting a Pokemon you have to talk to the professor and walk around him, which is clunky for agents. At a certain point the game asks yes/no questions instead of just advancing text, and the agent defaults to “no.” All this nuance had to be discovered through trial and error. None of it is in any docs. The agent has to earn it.

The self-healing loop works through Factorial Learning Environments. Take two steps forward, hit a wall, take three steps back, reevaluate. Once a route is discovered it gets written to JSON in the Pokedex logs. Next run skips the discovery phase. That’s the AlphaEvolve pattern: run a generation, find the best paths, fork off those for the next generation.

The pokemon-kafka repo takes this further. Agent output goes through the Tapes proxy into a Kafka stream. A Flink consumer does anomaly detection: stuck at a door, failed a conversation check, missed a battle. It writes corrections back. Over four generations the fitness score went from cold-start noise to consistent improvement.

What I’m building is a runbook. A runbook to speed run Pokemon. And that’s actually the full circle of how this applies to real work. The game is arbitrary. The method isn't. Mechanical Orchard is doing the same thing with COBOL monoliths: map the capabilities, use the spec as the iteration loop, let agents refactor toward the target.

This week’s newsletter covers two streams worth of reads, demos, and debates. The theme running through both: agents are everywhere, but the hard part isn’t spinning them up. It’s knowing what to hand them.

A DeepMind paper dropped on discovering multi-agent learning algorithms with large language models. The AlphaEvolve algorithm is the interesting part. Spin up a thousand agents. Pick the best four. This pattern for scaling agents is not unlike the world is heading with developers running entire companies through agent fleets.

The “AI Office” concept popped up on X as an article. The idea is built around agent orchestrators, managers, and interns. Same vibes as Gastown which took off late last year. The pattern is now converging into a full stack — OpenClaw orchestrating agents over a Vercel frontend and Supabase for memory. It feels very AI-engineering-forward. But for deeper work like starting VMs, SSHing into boxes, or spinning up three services to iterate, agents still fall over. The harness is the bottleneck.

MCP’s new tasks feature is interesting here. A way for an agent to fire off a long-running job and check on it later, or get a webhook back. That’s a killer feature you’d have a hard time implementing in a CLI. But CLI tools have basically taken over anyway.

The real unlock might just be skills files that tell your agent which CLI commands to run, rather than expensive MCP servers pumping tokens into your context window. Playwright MCP is insanely expensive token-wise. Same with Chrome MCP. A CLI wrapper gets you the same result without the inference overhead.

John demoed two new data providers for Tapes: Kafka and Postgres. Both shipped in v0.2.0. The Postgres support is immediately useful for multiplayer scenarios where multiple users write to the database. The Kafka integration is still finding its demo story, but I‘ve started integrating it into my Pokémon jouney, now using conlfuent cloud. Probably the move is deploying it to a team that centralizes all their agent telemetry through a single Kafka pipe.

The end-to-end tests use Dagger, which deserves a shout. You can define services in Go code, spin them up dynamically, run your tests against them, and tear everything down. Like a programmable docker-compose. The test spins up Postgres, Ollama (pulling the small Gemma model), and the Tapes proxy, then validates the full pipeline.

We also dropped two new podcast episodes this week:

• Engine X: Sunsetting Open Source with James Strong and Marco Ebert on what happens when you sunset an open source project

• Rewriting SQLite in Rust with the Glauber team on the Turso rewrite

DeepMind paper on using AlphaEvolve to discover new multi-agent reinforcement learning algorithms. The system treats source code as a genome and uses LLMs as genetic operators. Spin up a thousand candidates, pick the best four, evolve. This is exactly the loop I'm running with the Pokemon agent. DeepMind just published the theory.

One CLI for all of Google Workspace. Dynamically built from Google’s Discovery Service. 40+ agent skills. Written in Rust. “Not an officially supported Google product.” Sure.

OpenAI’s blog on the Codex agent harness and the iteration loop between code, Chrome tools, and automated verification. The diagram toward the bottom is worth the scroll — it's basically what I built for Pokemon, except OpenAI has a budget for it.

Block laid off 40% of staff and blamed AI. Cal Newport explains why that’s mostly cover for pandemic-era overhiring. Even AI boosters didn’t buy it.

Create more value than you consume. Stop worrying about running 68 agents simultaneously.

Better GitHub notifications. Almost a thousand stars. Still figuring out what to do with it, but the notifications problem remains unsolved.

Thanks for reading Open Source Ready! This post is public so feel free to share it.

Stay ready,

bdougie