Measure First, Optimize Last: My Approach to AI Evals

If you can’t measure it, you’re guessing. Here’s how I think about evals, practical examples at ai-evals.io . Start with pain, not tooling My eval approach is pain-point driven: I can’t compare what I can’t measure. I can’t trust an AI system to run on its own if I can’t quantify failure. If those two are unresolved, I am not “doing evals.” I am guessing. Treat it like automation engineering I…