Fable 5 vs. GPT-5.6 Sol on an NP-Hard Problem: Does /goal Help?
I gave Claude Fable 5 and GPT-5.6 Sol the same unpublished NP-hard optimization problem, with and without their native /goal mode. Fable 5 is a beast; /goal is not a game changer.
Charles AZAM - Blog
Thoughts on AI systems, engineering culture, and building things that work.
Latest posts
GLM-5 with Coding Agents: Competitive Scores, Weak Reliability
Zhipu AI's GLM-5 leads SWE-bench and LiveCodeBench. I tested it on an unpublished NP-hard optimization problem and 89 coding tasks. The best-case is competitive. The typical case is not.
I Forked 4 cli coding agents to Run the Same Model. A scaffholding deepdive.
Deep dive into the architecture of Codex, Gemini CLI, Mistral Vibe, and OpenCode. Same model, 2x performance gap — the scaffolding is what matters. Updated with GLM-5 results: every agent improved 38-54%, but the ranking stayed the same.
I benchmarked 4 CLI coding agents on an NP-hard optimization problem I solved by hand 8 years ago.
Claude Code (Opus 4.6), Codex (GPT-5.3-Codex xhigh), Gemini CLI (Gemini-3-Pro-Preview), and Mistral (Devstral-2) tackle a fiber network optimization problem. Claude Code beat my 8-year-old C++ solution by 62 points. Updated with GLM-5 results across two agent frameworks and Terminal-Bench.
A Multi-Tenant Coding-Agent Runtime in 2,000 Lines of Python
The agent loop itself is ~50 lines — the real work is user isolation, session persistence, and execution backends. Lessons from building omniagents, and how it compares to OpenHands.
Evaluate Your Own RAG: Why Best Practices Failed Us
We benchmarked our production RAG system across embedding models, chunk sizes, chunking strategies, and retrieval modes. The results contradicted common wisdom.
PrediBench: A Live LLM Forecasting Benchmark That Cannot Be Contaminated
A benchmark whose test set lies in the future cannot be in any training set. Models research live Polymarket events, commit probability estimates, and are scored on returns and Brier calibration. Co-built with Aymeric Roucher.