Agentic Evals Pyramid
Building AI agent evals that work: the 3-layer pyramid approach
rwilinski.ai
My musings about Applied AI, LLMs, RAGs and agents
Latest posts
Test Your AI Agent Before It Tests Your Users' Patience
What are evals? How to use LLM generated data to test your agent.
The Principles for Reliable, Safe and Fast AI Agents
Definitive guide to building better AI agents driving business value.
Generating structured data with LLMs - Beyond Basics
Advanced techniques for improving speed, accuracy and cost of JSON generation with LLMs.
Benchmarking OpenAI LLMs for JSON Generation
A quick experiment to benchmark OpenAI LLMs for JSON generation and analyze the results focusing on error rates, performance and costs.
Make Your RAG Suck Less
Common pitfalls in designing and implementing RAGs and how to fix them
GraphQL@Edge with global DynamoDB, AWS Lambda and Cloudfront
How to use GraphQL@Edge to serve a globally replicated DynamoDB table
Lessons learned using Single-table design with DynamoDB
Lessons learned using Single-table design with DynamoDB and GraphQL in production