Han Lee's blog on machine learning engineering, compound AI systems, search and information retrieval, and recsys — exploring machine learning, LLM agents, and data science insights from startups to enterprises.
Welcome to the Permanent Underclass. Here's Your Token Budget for the Month - Technology, software, and financial industries are all operating in headless-chicken mode. The pendulum swung from tokenmaxxing or PIP’ed in Q1 2026 to the opposite extreme of token budgets and throttling AI usage today. Everyone is getting whiplash from reactionary policies instead of pursuing a forward-looking AI…
Experimentation-centric designs, not benchmark running scripts - Most conversations about evals collapse into which SaaS tool to buy, which metrics to track, which LLM-as-a-judge prompt to slap on, or which single headline benchmark score to worship. SWE-bench percentage. Humanity’s Last Exam. ARC-AGI. Pokémon Red. Or most often, a directive from the top saying LGTM. These scores are useful, but…
AI agents compress the cost of collecting, cleaning, and operating public data. - For years, a lot of data businesses were not really selling unique or proprietary data. They were selling the operational burden of collecting, cleaning, storing, analyzing, and packaging data into a useful downstream application. That workflow looked roughly like this: Fig. 1. A typical data + AI workflow goes…
Whatever you bolt onto the model today is what the next model will eat. - If you have been building agentic products in the last twelve months, you have been writing harness code: system prompts, tool wrappers, planner-executor loops, retry policies, context compaction strategies, allowlists for which tools an agent can call from which surface, judges for when to stop, and fallbacks for when the…
Cognitive surrender is the blue pill of 2026. - In April, a partner at Sullivan & Cromwell — the firm that advises OpenAI on the safe and ethical deployment of artificial intelligence — sent an apology letter to Chief Bankruptcy Judge Martin Glenn of the Southern District of New York. An emergency motion the firm had filed in the Prince Global Holdings Chapter 15 case turned out to be riddled with…
Agents needs to live in a runtime. We need the runtime to research and develope agents. - Eleven years ago, Sculley et al. drew the diagram everyone in MLOps has seen: a tiny black box labeled “ML Code” surrounded by a sprawl of much larger boxes — data collection, feature extraction, configuration, monitoring, serving infrastructure. The point of the diagram was that the model code is the…
We’ve never had determinism in software. We just had the illusion of it. Here’s a fact that most people outside computer science don’t know: in 1936, Alan Turing proved that there is no way to build a program that can check whether another program will even finish running. This is the Halting Problem. A few years later, Rice’s theorem took this further — Henry Gordon Rice proved that it is…
Backyard furnaces, fake grain reports, dead sparrows, and poisoned flowers — your company's AI transformation is repeating history. - In 1958, Mao ordered every village in China to produce steel. Farmers melted down their cooking pots in backyard furnaces and reported spectacular numbers. The steel was useless. The crops rotted. Thirty million people starved. In 2026, every other company is having…
The infrastructure that determines what your agent can actually learn - Model architecture gets all the attention. Post-training recipes follow close behind. The reinforcement learning (RL) environment — what the model actually practices on, how its work gets judged, what tools it can use — barely enters the conversation. That’s the part that actually determines what the agent can learn to do. A…
Understanding agentic model development and agent frameworks through the lens of Super Mario - Agentic AI is a fairly recent development that combines reasoning (OpenAI, 2024) and tool use (Schick et al, 2023) in the same AI model. But an agentic AI system is not just the model, but also the harness, environments, tools, rewards, evaluations and benchmarks, and all of the infrastructure to support…
Apache Spark's #1 committer reveals how contrarian decisions and AI-first strategy drive Databricks' 60% YoY growth - Reynold Xin, Apache Spark’s #1 committer famous for “deleting more code than others wrote,” reveals how Databricks maintains 60% YoY growth while competitors struggle. In a candid interview at Hysta Rising, he shares the contrarian strategies, technical decisions, and AI-first…
Deconstructing prompt-based meta-tool architecture and context injection patterns for AI engineering - Claude’s Agent Skills system represents a sophisticated prompt-based meta-tool architecture that extends LLM capabilities through specialized instruction injection. Unlike traditional function calling or code execution, skills operate through prompt expansion and context modification to modify…
Product strategy for the shift from human interfaces to AI reasoning engines - If you’re building B2B software, your product strategy needs to account for a fundamental shift happening right now. The transition from IT 2.0 (SaaS) to IT 3.0 (AI-native systems) isn’t just changing how enterprises operate—it’s rewriting the rules of what makes products valuable and defensible. In my previous article…
From SaaS glue work to AI-native systems: mindset, toolset, skillset, data - If you’re leading enterprise transformation, you need to understand what’s happening to your workforce, your tech stack, and your competitors right now. Three waves of enterprise IT have systematically created and destroyed entire job categories. The third wave is accelerating, and it will reshape your organization…
Understanding common metrics in LLM benchmarks - Every time AI labs release new models, we see an evaluation metric called $\text{pass@}k$, where $k$ can be any integer number such as $\text{pass@}1$. It might sound like passing a test at the $k$th attempt, but this metric is far more sophisticated and plays a crucial role in how we build reliable AI applications in production. As an example,…
A RACI Matrix Guide to Navigating the Blurred Lines Between PM, Engineering, and Design Roles - The bottom line: Modern AI tools like ChatGPT, Claude, and Cursor are acting as powerful democratizers in software development. They are blurring traditional role boundaries, enabling product managers to draft code, engineers to mock up interfaces, and designers to prototype functionality. This newfound…
Save planet earth, stop using Pydantic everywhere - I love Pydantic. And I’ve witnessed some of the worst code written with Pydantic. Pure spaghetti, non-performant code. There are two major anti-patterns in abusing Pydantic for maximum spaghetti. First anti-pattern is serdes debt. Instead of using Pydantic only at service boundaries for validation, it’s being used everywhere, incurring heavy…
Skating to where the puck is going to be, not where it was - Agent frameworks and workflow builders exist because the LLM model itself is not yet strong enough to be autonomous in completing the tasks. It can be now. For those who are technical enough to do post training. And these post trained LLMs will render most agent frameworks and workflow tools obsolete. Workflow Builders We human users…
Failing agent computer interaction design by wrapping MCP on top of REST API - Model Context Protocol (MCP) is a prominent technology in 2025, generating buzz comparable to ChatGPT in 2023 and RAG in 2024. However, many common implementations simply create an MCP wrapper over existing API services. This is a suboptimal design choice. This blog post will outline the design principles of RESTful…
An third party MLOps post mortem on xAI Grok's 'white genocide' - Update 2025-05-15: xAI published a post on X detailing that “an unauthorized modification was made to the Grok response bot’s prompt on X.”. This actually brings up more questions with regards its software development life cycle (SDLC) and internal controls. xAI Grok’s “White Genocide” Incident On May 14, 2025, xAI’s chatbot Grok…