Making developers awesome at machine learning
Overdue Last read · last published · next check
Last read 17 hours ago, longer than this feed's 6 hours schedule.
In this article, you will learn seven concrete regression tests for catching the orchestration-layer failure modes that matter most before deploying an AI agent to...
In this article, you will learn what latent spaces are and how they serve three distinct roles - descriptive, generative, and predictive - across a...
In this article, you will learn the conceptual and practical differences between retrieval and memory in agentic AI systems, and how to combine both effectively....
In this article, you will learn seven async patterns for running AI agents concurrently in Python, what each pattern is suited for, and the production-level...
In this article, you will learn how prompt caching and fine-tuning differ as strategies for reducing cost and latency in agentic AI systems, and how...
But cutting your runtime token burn is just the first problem.
With the vocabulary and the failure modes in place, here's the build.
Day 100 in production isn't really about chunking strategies anymore.
This chapter is divided into eight parts; they are: • Metrics for LLM Inference • Measuring a Single Request • Warmup and Synchronization • Measuring GPU Work with CUDA Events • Measuring Memory Usage • Measuring Concurrent Requests • Multiple GPUs and Multiple Machines • Cost per Token The most common inference metrics are: • Latency: How long a request takes from start to finish.
In this article, you will learn how static, dynamic, and continuous batching work in LLM inference, and why the differences between them matter at production...
This chapter is divided into nine parts; they are: • Reading Logits from a Model • Greedy Decoding • Temperature Sampling • Top-$k$ Sampling • Nucleus Sampling • Repetition Penalties • Beam Search • Stop Conditions • Structured Output Constraints The model returns a vector of logits for every position in the input sequence.
