Brooklyn Hacker

Just before the American gridiron football season started, I hit up a bunch of old hacker buddies of mine and made a proposition:

Let’s make an LLM powered Fantasy Football league. No human decision allowed - every choice over the season must be made by a model. For fun, we set up some ground rules:

1. Each manager picks a model and uses it for the season.
2. Everything around the model is fair game. Prompt engineering, agents, RAG, fine tuning, LoRAs - only requirement is the inference source must be the model the manager picked.
3. The model makes every judgment - input and output tokens must be shared with the rest of the league for each week.
4. If your model is really bad, you can beg for mercy from the rest of the league. A grand jury of three managers selected at random will decide on the appropriate punishment for your model switch.

My friends Brent, Carter, Dominik, Ricky, Greg, Brian, Kevin and Swift were all enthusiastically in and we had an absolute blast. The first draft for the league was for model selection. For order of selection, we each took this quick machine learning quiz and used the scores to establish who could pick a model first. Highest scores get the pick of the best models with some highly surprising picks early on.

I suspect that was due to the way the draft order for the players was derived. Once the managers had their models, we used the scores on Artificial Analysis for Humanity’s Last Exam to assign the draft order in ascending order. This meant that the less capable models ended up having better talent preference.

Of the many things we all learned about applying AI to a complex and rapidly changing problem, these are prob my top three:

Speed was key.

We set up the league with full platform defaults, including the 2 minute time limit per draft pick. I was tempted to lower this because I thought the models would be way faster than that. I was so wrong here - it lead to several tragic autopicks and harried player search.

Tool use was more important than parameter count.

Greg was gracious enough to set up an MCP server for the league ahead of the event so that each manager could easily pull the data on the draft from the fantasy platform we used. How well each model did interfacing with that MCP server was *wildly* variant. Brian who chose last and ended up with gpt-oss ended up putting together an absolute wagon of a team because it was 1) fast and 2) always leaned on the tools it was given.

The agents got worked.

Me and a few others thought for sure that building a suite of agents all responsible for discrete decision making on the draft would be an unfair advantage. In truth, OpenAI Agent, Claude Code and just Cursor worked better than everything we built. This is a space we def need to explore as we develop our stacks over the course of the season.

All in - can AI play fantasy football? I think our answer is “we’ll find out.” We have had a promising start to the season but we’re already getting some comic errors happening with trades.

If you want to check out our league and roast the AI drafts, we have it up here.

We’re also throwing some of the banter up on YouTube over the season.

https://youtu.be/HNgLYKFN3bw