Melchi
How attention architectures evolved to keep KV cache from eating your GPU, and what that means if you self-host. Already comfortable with KV cache and attention? Skip the theory and jump straight to the interactive KV Cache Calculator to size VRAM for your model, batch size, and target GPU. If you’re planning to self-host a large language model, you’ve probably sized VRAM based on…
TL;DR — Compute is usually the largest line item on your cloud bill. Bills tell you what you spent, not what you used. Measure utilisation with percentiles (P95/P99), not averages. Prefer always-on elastic infrastructure over scheduled shutdowns, and let Kubernetes bin-pack workloads to squeeze more value out of every node. Back in 2015, public cloud services were not well understood. Large…
I’ve been doing some interesting work with the team at MFlow writing HTTP clients that consume financial data, and it’s been eye-opening to see how different API platforms choose to protect their resources. Best practices for client-side rate limiting seem to be scarce when compared to server-side, so here are my thoughts on the subject and some code samples. TL;DR — wrap *http.Client…
TL;DR — A standard Linux container is an isolation boundary, not a security boundary. Every container on a host shares one kernel, so a single kernel exploit can compromise the whole node. gVisor inserts a user-space kernel ( runsc ) between your container and the host, dramatically shrinking the attack surface. It’s now production-grade — Google runs Cloud Run, App Engine and Cloud…