How to Do Evals in 2026
A tool-agnostic framework for evaluating AI agents
Alexey On Data
Practical writing on building, testing, and operating AI systems, workflows, and automation, based on my projects and experiments. Subscribe to get ideas you can reuse in your own work.
Latest episodes
Deploy a Full-Stack App with AI Coding Assistants
Part 3 of AI Dev Tools Zoomcamp
Getting an AI Engineering Job
The 5-step framework that will land you a job
Build and Ship a Full-Stack App with AI Coding Assistants
Part 2 of the AI Dev Tools Zoomcamp series
(Re)Building a FAQ System for DataTalks.Club
Serving 100k+ Learners with a Serverless FAQ Assistant
What AI Forward-Deployed Engineers Do
An Analysis of 113 AI FDE Job Postings
AI-Native Development: Specifications, Loop and Graph Engineering
Part 1 of the AI Dev Tools Zoomcamp series
How to Write a Good README
16 sections your project should have in the README
My PyPI Release Pipeline for Python Libraries
How I release Python packages with agents and without
Choosing a Portfolio Project. The Definite Guide
When preparing personal plans for AI Shipping Labs members, I often get the question, “What should I build for my portfolio?” There are too many options, and no clear way to pick one.
Six Projects That Didn’t Make It
But I learned a ton while building them
The System I Built for AWS Access Without Keys
I sometimes run offline workshops.
AI Design for Non-Designers
My workflow for AI-generated interfaces: how I use mockups and design guidelines to avoid AI slop
How to Set Up Your Coding Agent: A Step-by-Step Guide
Start with one real task, then add skills, subagents, and extra tools when you need them
Minsearch: The Small Search Library Behind My RAG Workshops and Courses
How a small in-process Python library replaced Elasticsearch in my RAG notebooks that I use for teaching.
The System I Built to Ship Code From a Phone
A remote server, tmux, voice-driven Claude Code, and a workflow designed around the in-between time
11 Workshops to Build Production AI Agents (RAG, MCP, Guardrails & Deployment) in One Place
I consolidated all the GenAI workshops I've ever taught into one place. Here's what's inside and how to take them.
Building an AI Book Generator with a Plan-Then-Execute Pipeline
How a niche bedtime-book project became a system for generating chapters, audio, EPUB files, and Amazon KDP print artifacts.
9 Real-Life AI Projects from AI Engineering Buildcamp Graduates
This is what the students of the second cohort built + AI Shipping Labs new
From Google Docs to an Automated FAQ System for DataTalks.Club Courses
At DataTalks.Club, we run free courses like ML Zoomcamp, Data Engineering Zoomcamp, MLOps Zoomcamp, and LLM Zoomcamp.