HOT TAKE
The best verifier is an agent that did not write the code.
Who should check the work? Same context or Fresh context
LAST WEEK’S TAKE
Close, but more of you back writing better rules over upgrading your model.
VOICE AGENTS FORUM // SAN FRANCISCO
Latency, interruptions, handoff, evaluation, observability, reliability. Voice agents bring a set of engineering and operational problems that only become more important once people start using them.
Join us at Digital Jungle SF on September 16 for a day with engineers, founders, researchers, and operators building and deploying voice systems. Compare how teams are approaching the stack, where technical choices create tradeoffs, and what to consider as voice agents move from prototypes into production.
Early Bird tickets end September 1.
HIDDEN GEMS
AI code review orchestration guide
Seven specialized agents, coordinated through a CI-native plugin architecture, provide a practical design for scaling automated reviews while controlling noise, duplication, severity, cost, and failures.
Static retrieval training and benchmark
Training on 660 million pairs and aggressive quantization produce an 8 MB retrieval model that can embed English Wikipedia on a laptop in under eight minutes.
Coding agent harness training guide
Running each rollout inside the deployed harness lets TRL train on captured model calls and hidden-test rewards, preserving the agent’s genuine tool loop rather than recreating it.
Reproducible evals for SRE MCP servers
Deterministic telemetry, isolated agent runs, and blind scoring form an open evaluation framework for comparing MCP tools against direct SQL across realistic incident-investigation scenarios.
JOB OF THE WEEK
Senior Machine Learning Engineer // Thorn // Remote, US
Thorn is hiring a Senior ML Engineer to build, deploy, and maintain models that help identify victims, detect abusive content and behavior, and support the removal of child sexual abuse material. The role covers model development, evaluation, data labeling, and production infrastructure.
Responsibilities
Own end-to-end development of production ML systems and algorithms
Define data sourcing, labeling requirements, and evaluation processes
Deploy and maintain models with product engineering teams
Present technical work and collaborate with external partners
Requirements
5+ years of experience in ML or AI
Experience building production ML systems and pipelines
Production experience with computer vision or NLP
Clear communication and willingness to work in online safety
LUNCH & LEARN · SESSION 21
Tomorrow, Friday 14 August, 16:00-17:00 UTC. Free and virtual.
Om Shree is presenting TDQS (Tool Definition Quality Score), a framework for measuring MCP tool quality that came out of a study of 856 tools across 103 live MCP servers. He’ll cover what separates a tool definition an agent can use reliably from one that produces confusing behavior, and what to check in your own tools. Community discussion on the MCP ecosystem follows the talk.
The last session ran on a related theme. Leo Walker walked through Arcade’s ToolBench benchmark for grading MCP servers, and why a strong compliance grade still doesn’t guarantee a clean connection. John Dellenbaugh demoed the stateless protocol change that lets you scale MCP servers across instances without a sticky gateway or a session store, and Ben Kearns pushed back on what happens to authorization when a client carries its own identifier.
MLOPS COMMUNITY
Supervising a coding agent may be more cognitively demanding than watching a self-driving car because its intent and acceptable end state are far harder to observe.
Self-driving works within defined inputs, goals, and physical constraints; software development is open-ended, with much of its intent missing from the code.
Graph-based workflows can run independent agent tasks in parallel, then use fresh context to verify and merge results.
Growing skill libraries need evaluation, pruning, and reliable selection across models.
Better agents still demand better ways to supervise their reasoning.
The winning design cut CLI startup from 142.0 ms to 98.6 ms, yet it did not exist when the search began.
A genetic coding-agent loop generates diverse approaches, keeps strong performers, and mutates or combines them across generations.
Binary gates exclude broken variants, while a frozen numeric rubric prevents the scoring criteria shifting after results appear.
File-based results preserve audit trails, survive crashes, and keep parallel sub-agents isolated.
Measured evolution can find designs that intuition and one-shot sampling miss.
Successful MCP adoption could leave an enterprise running hundreds of servers, each adding compute, maintenance, security review, and on-call responsibilities.
Broad vendor tool catalogs increase token use, latency, and attack surface while exposing far more capabilities than individual agents need.
Internal servers fragment identity, deployment, and telemetry unless teams share a clear platform contract.
Gateways, generated tool definitions, and agent-specific backends each reduce some duplication but introduce their own tradeoffs.
MCP scale depends on governing the full estate before server sprawl sets in.
A deployment assistant combining reusable instructions and live tools can now travel across compatible agent clients without being repackaged for each one.
Agent Plugins 1.0 places native Agent Skills and MCP configuration inside one predictable directory structure and manifest.
Client-specific features remain in namespaced folders that unsupported clients can safely ignore.
Installation, permissions, trust, distribution, and runtime behavior stay under each client’s control rather than being forced into the shared format.
The result is a narrow portability layer that preserves room for clients to differ.
IN-PERSON EVENTS
New York - August 14
Silicon Valley - August 27
Melbourne - September 2
Shanghai - September 5
AGNTCon + MCPCon China - September 6
Toronto - September 10
AGNTCon + MCPCon Japan - September 10
Pune, India - September 12
New York - September 15
San Francisco, Voice Agents Forum - September 16
VIRTUAL EVENTS
Coding Agents Lunch & Learn Session 21 - August 14
Reading Group: Prompt Injection as Role Confusion - August 18
MEME OF THE WEEK
ML CONFESSIONS
I got 0.94 AUC on a churn model and presented it twice. The second time someone asked what account_status_updated_at was doing in the feature list. It was the timestamp on the cancellation record.
I said I’d double-check and moved to the next slide. Rebuilt it over the weekend without that column and got 0.71. Sent the new numbers round on Monday with no explanation. Nobody replied.
Share your confession here.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.