Know the Game You’re Playing
“It’s remarkable how much you don’t know about the game you’ve been playing all of your life.” Mickey Mantle
Mark Pettyjohn
Writing is thinking
Latest posts
AI Benchmarks: Are We Juking the Stats?
I think I’ve seen this movie before. I have experience in education (teaching prek, K, 4th, 6th, and college). I have got experience in tech (90’s AOL/ICQ script kiddie/webmaster turned founder). Today I’m your average generative AI user. I care about SWE-bench rankings about as much as I cared about standardized test scores as a teacher. Aside from quickly discerning if a model is at least GPT-4…