HOT TAKE
The scariest agent isn’t the one that hallucinates. It’s the one with the right answer and the wrong permissions.
What matters more in production: accuracy or access?
LAST WEEK’S TAKE
A not-irrelevant number of people still chose relevance when retrieval gets sensitive.
HIDDEN GEMS
Loobster gives Claude Code and Codex a repeatable plan-build-test-secure loop with approval gates between phases, so agent work is easier to review before merging.
Internal AI Engineering Stack Guide
Built around Cloudflare’s R&D workflow, the stack combines AI Gateway, Workers AI, MCP portals, AGENTS.md files, and automated code review for scaling agentic engineering work.
Model Serving Community Report
Across vLLM, KServe, llm-d, Kubernetes inference work, and OGX, recent releases point to harder production questions around routing, observability, hardware support, and agentic workloads.
Training and evaluation utilities cover data preparation, draft-model training, checkpoints, and benchmark testing for comparing speculative-decoding methods across Qwen, Gemma, Eagle3, DFlash, and DSpark.
JOB OF THE WEEK
Software Engineer – Backend // Moss // San Francisco, CA, US / Remote
Moss builds real-time semantic search infrastructure for conversational AI. This backend role focuses on production retrieval systems, including APIs, caching, indexing, observability, cloud and edge infrastructure, and reliability work for latency-sensitive customer deployments at scale.
Responsibilities
Build query path, APIs, caching, indexing, and data plane.
Improve latency, correctness, and reliability across live retrieval workloads.
Add observability, load testing, alerting, and incident response processes.
Debug customer deployments and convert failures into infrastructure improvements.
Requirements
5+ years operating latency-sensitive production backend systems.
Strong distributed systems fundamentals across APIs, databases, caches, queues.
Experience with monitoring, incident response, capacity planning, and SLAs.
Familiarity with regulated environments, ideally SOC 2 or HIPAA.
MLOPS COMMUNITY
The scary bit isn’t that agents can write code. It’s that code lets them build their own tools.
Code gives agents a tight feedback loop: compile, run tests, measure performance, pass failures back, and improve.
That makes coding agents a route into broader knowledge work, where sandboxes and scripts can reach business systems.
The harder problem becomes context: tasks, docs, chats, decisions, permissions, and where agents are allowed to act.
The agent stack starts to look less like a chatbot and more like an operating layer for work.
A patient gets 25 pages of discharge instructions. An agent can help, but in healthcare, the dangerous question is what it is allowed to do.
MCP sits above the data layer, enforcing role-specific, time-bound access for patients, providers, and caregivers.
Clinical and regulatory steps stay deterministic, with providers approving decisions before action.
Restricted tools, trace logs, and behavior metadata help catch drift and prove auditability.
In regulated AI, permissioning becomes the product.
The keyboard isn’t disappearing; the job is moving up the stack. This piece looks at the shift from writing every line to orchestrating agents, with trust, oversight, and shared memory as the real constraints.
Agentic coding changes the role from implementation to intent, architecture, review, and testing.
Speed still needs vigilance, especially around security, UX, documentation, and production readiness.
The missing layer is persistent context across agents, tools, and decisions.
The hard part is no longer only generating work, but keeping agents grounded, connected, and accountable.
Agents don’t just need more context. They need to stop rummaging through it like a junior analyst with infinite API credits.
Retrieval quality can be checked cheaply with score spreads and statistical signals before paying for reranking or another LLM call.
Search skills can teach agents when to use dense, sparse, lexical, or hybrid retrieval.
Long-term memory needs forgetting, deduplication, and graph/vector structure, not just bigger context windows.
Agentic retrieval is becoming less about finding chunks and more about controlling search behavior.
IN PERSON EVENTS
Munich - July 23
San Francisco, Voice Agents Forum - September 16
VIRTUAL EVENTS
Coding Agents Lunch & Learn Session - July 3
MEME OF THE WEEK
ML CONFESSIONS
v0.8.0 Added a model registry.
v0.9.0 Changed the scorer to pull the model tagged latest.
v1.0.0 Shipped a retrained ranker. Offline metrics looked good.
v1.0.1 Found out latest meant the model artifact, not the feature definitions.
v1.0.2 Production was serving the new model with the old transforms. Half the numeric features scaled wrong. One categorical was mostly unknowns.
v1.0.3 Rolled back. Rankings went up.
v1.0.4 Added feature-set versioning. Added a deploy check.
v1.0.5 Pinned the model. Pinned the feature set. Made releases slightly slower.
Share your confession here.
No posts
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.