kiac Joins the CNCF Landscape, Local AI Got Crazy

I have been gone for a month, and I want to start by telling you where I was, because it was one of the better stretches I have had in years.

I was in Japan. First for work: KubeCon + CloudNativeCon Japan at PACIFICO Yokohama which sold out again. Saloni and I gave our talk on whether your Kubernetes disaster recovery is actually ready, and the honest answer from the room was mostly “we think so, we have never tested it.” At the community day I sat on a panel on sustainability and talked about what we are building in CNCF TAG Operational Resilience, which is the least glamorous and most important corner of this ecosystem: the boring guarantees that keep systems up when the interesting parts fail.

Then I stopped working. We stayed at a ryokan near Mt Fuji, woke up to the mountain out the window, and did nothing productive at all. Tokyo Disneyland with a character room, then DisneySea, then a Pokemon room, which my family has decided is the single best hotel decision I have ever made. A lot of food. A lot of walking. No laptop for days at a stretch. I came back genuinely recharged, which is not a thing I say often.

And while I was away, the industry went completely feral. Two trillion-parameter-class models shipped with the weights actually released. A desktop app that trains models on your Mac. A GitHub competitor launched on the day GitHub went down. A GitHub repo that hit 100,000 stars in under 48 hours. Meta going open source again. Also, one small thing that made my week.

This is the catch-up edition so read it till the end

kiac is now in the CNCF Landscape. My little side project, the one that runs real Kubernetes on Apple’s container framework where every node is its own lightweight VM, is now listed in the Cloud Native Landscape under Platform / Certified Kubernetes - Installer. It sits at 333 stars, MIT licensed, supporting Kubernetes 1.32 through 1.36 on pinned digests, with a built-in LoadBalancer controller, metrics-server, local-path storage, Gateway API through Traefik, a full Cilium eBPF datapath when you run with the full kernel, and IPv6 plus dual-stack since v0.4.0. You pick kubeadm or k3s and you get nodes that are actually isolated, because each one is a real VM with its own kernel, not a shared-kernel container pretending to be a node.

Why it matters: Not because a Landscape entry is a trophy. Because of what it says about local development. For a decade, “local Kubernetes” meant accepting that your nodes were a polite fiction, which is fine until you are testing anything that touches the kernel, the CNI, node failure, or isolation. kiac exists because I got tired of that gap between my laptop and the cluster I actually ship to. If you are on Apple silicon, install it, break it, and file the issue. That is the only thing that makes a project like this get better. kiac on GitHub | How it works

local.ai launched, and it is the benchmark this space has been missing. It is an independent local-AI benchmark: not tokens per second on a spec sheet, but complete end-to-end system combinations of model, quantization, hardware, inference engine, and runtime config, scored on intelligence, actual task completion time, and cost including electricity. Measurement data comes from Exo Labs, and the site has no commercial ties to hardware vendors or model makers. The findings are exactly the ones that break people’s assumptions: two quantizations that decode at nearly identical speeds but finish tasks three times apart, a device with better raw specs losing on measured work, a heavily compressed large model matching a lightly quantized small one. There is an early-access name claim open now, and a referral leaderboard where the top referrer takes home a DGX Spark from NVIDIA plus an M4 Pro Mac Mini, with the next nine getting Mac Minis. I am on the board and climbing, so if you were going to sign up anyway, use my link and help me get up that ladder. Sign up with my invite | What local.ai measures

Meta open-sourced Muse Glimmer, and Muse Spark 1.2 weights are next. On August 10, Meta Superintelligence Labs released Muse Glimmer, a 30B model distilled from Muse Spark, under Apache 2.0, built specifically for always-on local agent workflows. At roughly 4-bit it lands under 20GB, which puts it on a Mac or a single decent consumer GPU, and it decodes 3.1x faster with DFlash speculation. Zuckerberg’s own post said Meta will also release the weights for Muse Spark 1.2, their latest foundation model, and credited Alexandr Wang and the MSL team. Read that second part carefully: if it lands, that is a US lab putting an actual frontier-tier model into open circulation, which no American lab has done at that level. Meta going quiet on open source was one of last year’s running stories. This is the reversal. Meta’s announcement

Alibaba shipped a 2.4 trillion parameter model and then gave the weights away. Qwen3.8-Max was announced August 3 as a 2.4T-parameter MoE built for autonomous coding, with Alibaba claiming 10-plus-day self-evolving development runs from an empty folder to production, 500-plus turns of chip design optimization, and a full project trace published on GitHub. API pricing is $2 per million input, $6 output, $0.25 implicit caching. Then on August 12 the open weights actually landed on Hugging Face, a 2.4T-A95B checkpoint, text-only for now and without the 1M context, under a new revenue-share license. Two days later came the one most of us will run: Qwen3.8-27B, dense, Apache 2.0, vision-language, and small enough for a consumer GPU. Simon Willison’s verdict is worth internalizing before you deploy it: excellent, but it defaults to overthinking everything. Qwen3.8 announcement | Qwen3.8-27B on Hugging Face | Simon’s review

GLM-5.3 arrived as a cyber model, with the weights held back. Z.ai shipped GLM-5.3 on August 14 through its coding plan. Same base as 5.2, gains from extended post-training rather than a new pretraining run, and roughly 50% better coding in their internal evals. The headline number is Terminal-Bench 3.0 going from 4.6 to 28.3, and the headline claim is that the model helped security teams find 2,436 vulnerabilities across 269 open source projects, some of them decades old. Open weights are staged after safety evaluation, roughly two weeks out, which breaks the GLM-5.2 pattern of shipping MIT weights within days. Hold that thought against the Anthropic argument further down. z.ai/blog/glm-5.3

Kimi K3’s weights actually landed. We covered the announcement last time with the caveat that the weights were not out. They shipped to Hugging Face on July 27: 2.8 trillion parameters, the largest open model ever released, 1M context, Kimi Delta Attention and Attention Residuals, activating 16 of 896 experts. Kimi-K3 on Hugging Face

Unsloth shipped a desktop app that both runs and trains models locally. Announced August 11, open source, on Mac, Windows and Linux, supporting MLX, GGUF, diffusion image and video, and audio. It handles CPU and multi-GPU setups across NVIDIA, AMD, Intel and Apple silicon, claims 2x faster training with 70% less VRAM, self-healing tool calls with sandboxed code execution, private web search and RAG, MCP support, and exports to NVFP4 and GGUF. It also lets you point Claude Code and Codex at local models. Plenty of apps run models on your machine. This is the first mainstream one that trains them too. unsloth.ai | GitHub

And the small-hardware crowd kept quietly winning. An open source engine running Gemma 4 26B in 2GB of RAM on any M-series Mac. antirez writing native MiniMax-H3 inference for Apple Silicon in C. Cactus shipping Needle2, a 14MB agentic model aimed at phones, wearables, smart home devices and robots. None of these got a keynote. All of them move the floor of what “local” means. turbo-fieldfare | h3.c | Needle2

I was on vacation. The publishing was not.

How to run isolated tenant clusters on shared GPU infrastructure, on the NVIDIA developer blog. Ekin Karabulut and I published this on August 3, and it is the pattern I keep getting asked about: how do three teams share expensive GPUs without three sets of infrastructure and without stepping on each other. The answer combines KAI Scheduler for topology-aware, quota-based GPU scheduling with hierarchical queues and gang scheduling, and vCluster for tenant clusters that each get their own API server, CRDs, RBAC and cluster-admin, while sharing the underlying nodes. The walkthrough runs NLP, Vision and Recommender teams on a single L40S with GPU Operator and CDI, guaranteed minimum quotas plus burst, and each team seeing only their own workloads. It scales the same way from one GPU to hundreds of nodes. Read it on developer.nvidia.com

Putting a front door on local AI with agentgateway, on the AAIF blog. August 14. Running models locally solves ownership and solves nothing about governance. This post builds up from a bare Ollama instance on a DGX Spark to a gateway that does protocol-aware token counting, named models so clients ask for chat or coder instead of a version string, per-consumer virtual keys, shared token budgets, and health-based failover in 24 extra lines of YAML, with Prometheus metrics per model on port 15020. Favourite detail: a single greeting to Gemma burned 1,274 completion tokens on reasoning overhead, which you would never have seen without the gateway in front. The rule that matters: a gateway is only useful if clients cannot route around it, so bind the model server to loopback and mean it. Read it on aaif.io

Running Goose fully offline on a DGX Spark, also on the AAIF blog. The companion piece, from July 17, and the one I keep pointing people at when they ask whether offline agents are real yet. Goose driving qwen3.5:35b-a3b through Ollama on a Spark, 23GB on disk and 29GB resident, decoding at 76.23 tokens per second at about 91% peak GPU utilisation, writing files and running shell commands, finishing the primes task in 3 minutes 48 seconds. The number that matters is the last one: zero external network calls, verified by socket analysis, because the agent and the model never open a socket that leaves 127.0.0.1. It is honest about the limits too. Bounded tasks work today, long autonomous workflows still break. Read it on aaif.io

Building a mini AI platform: LLM routing, MCP tools and agents, on the vCluster blog. August 17. Three teams, three tenant clusters, one control plane cluster running on a laptop through vind, and agentgateway sitting in front of all three traffic types: LLM calls, MCP tool federation, and agent-to-agent task delegation. One auth mechanism, one rate-limiting policy, one observability story, roughly 150 lines of YAML, with OpenTelemetry-tagged cost attribution per team, model and operation. If you want to see what an internal AI platform actually looks like before you buy one, this is a working sketch you can run in Docker tonight. Read it

And a stack of posts on the KubeSimplify blog. Running Qwen3.8-27B on DGX Spark, published the day after the weights dropped. A hands-on with an AI SRE copilot on my own hardware, and what it actually does versus what the category claims. Nemotron 3.5 Lightning on the Spark. Devin Outposts on Kubernetes, on why your AI agent needs your cluster rather than someone else’s. And two from myself and Shubham Katara that I would read as a pair: sharing GPUs at scale with HAMi, and dynamic MIG on RTX PRO 6000 tested live on Kubernetes. Qwen3.8-27B on Spark | AI SRE copilot, hands-on | Nemotron 3.5 Lightning | Devin Outposts on Kubernetes | HAMi dynamic MIG

Thank you, Utho. All of that GPU work runs on a box Utho Cloud gave us for experimentation and research: eight RTX PRO 6000 Blackwell cards, 96GB each. It is the reason we can slice, share, partition and break things in public instead of theorising about them. Utho has also committed to deploying 10,000 GPUs over the next seven to eight quarters to build sovereign AI compute inside India, which is a bet worth watching. More blogs and videos from that box are queued up. Utho GPU cloud

Kubeflow graduated. CNCF announced it yesterday, August 17. Kubeflow standardises the full AI and ML lifecycle on Kubernetes, from data processing and notebooks through distributed training, fine-tuning, inference and serving, across public, private and hybrid environments. Graduation is CNCF saying the governance and the production adoption are both real. For a project that spent years being called sprawling and hard to adopt, arriving as the graduated standard for cloud native AI operations is a genuine turn. CNCF announcement

Cloud Native Buildpacks graduated too, and k8gb moved to incubating. Buildpacks graduated on August 11, with 535 contributors across 164 organisations, adopters including DigitalOcean, GitLab, Google and HashiCorp, a third-party security audit from Quarkslab and OSTIF, and an OpenSSF Best Practices badge behind it. k8gb, the Kubernetes global balancer that does multi-cluster DNS-based failover, became an incubating project on August 5. Both are unglamorous infrastructure that quietly removes a category of bespoke glue from your platform. Buildpacks | k8gb

Kubernetes v1.37 lands August 26, and there are things to check before you upgrade. From the official sneak peek: static pods can no longer reference Secrets or ConfigMaps, and the PreventStaticPodAPIReferences feature gate is gone. kubectl run --filename is deprecated. kube-proxy ipvs mode starts printing deprecation warnings, heading for disabled-by-default around v1.40 and removal around v1.43, so check your mode now with kubectl -n kube-system get configmap kube-proxy -o jsonpath='{.data.config\.conf}' | grep 'mode:'. And cgroup v1 keeps closing: since v1.35 failCgroupV1 defaults to true, so a kubelet on a cgroup v1 node will refuse to start unless you explicitly override it. Coverage of the cycle also points to DRA device taints and tolerations going GA, KYAML output for kubectl, and rootless kubelet moving to beta, which is the one I am most curious to test. v1.37 sneak peek

MCP got its biggest revision since launch, and it is now stateless. The 2026-07-28 spec removes the initialize/initialized handshake and the Mcp-Session-Id header entirely. That single change means an MCP session is no longer pinned to whichever server instance answered first, so you can stop building sticky sessions and shared state around it and let ordinary HTTP infrastructure route the traffic. Multi Round-Trip Requests keep interactive tools working without a long-lived session. Extensions graduate from convention to a governed system with reverse-DNS identifiers, capability negotiation, and independent versioning. And there is now a formal deprecation policy with a minimum twelve-month window, which is worth stealing for your own APIs. If you run MCP servers in production, this is your migration. The 2026-07-28 spec | Changelog

The Linux Foundation launched the Tokenomics Foundation. Announced with 30 founding members including IBM, Oracle, SAP, ServiceNow, JPMorganChase, Nebius, Lenovo and Broadcom, it exists to define vendor-neutral standards for the economics of AI: what tokenomics means, how to measure cost to serve, AI value frameworks for business impact, and token cost telemetry landing in the FOCUS specification. Jim Zemlin’s line is the whole pitch: tokens are the commercial expression of the AI economy, and there is no shared way to connect spending to value. Every platform team I talk to is being asked what the AI spend returned, and everyone is answering with a different spreadsheet. linuxfoundation.org

Cursor launched a GitHub competitor on the day GitHub fell over. Origin, Cursor’s code hosting service, started rolling out in beta to paid plans on August 17, pitched as git hosting built for agent scale rather than for humans, with a tab in the desktop client, a CLI, and sync from existing GitHub repos so you can try it without leaving. It is Cursor’s first major product move since the SpaceX acquisition. It shipped the same day GitHub had a global incident starting 13:40 UTC that hit API, Actions, Issues, Pull Requests, Webhooks and authentication, with roughly 20% error rates on web and API and archive downloads failing at 50%, and Copilot the last service to recover. The timing was luck. The pitch is not. SiliconANGLE on Origin

DeepSeek open sourced an agent harness and broke the GitHub star record. DeepSeek Harness went up on August 13, MIT licensed, Node.js, developer preview, and passed 100,000 stars in under 48 hours, faster than anything before it. The design is the reason it caught: everything is a plugin. The model adapter, the tool registry, the session log, the permission layer, storage, and the agent loop itself are all replaceable. npx @deepseek-ai/dsh web gets you a UI on port 3080. It is moving fast and breaking compatibility while it does, so pin your version, but if you have been thinking about what an agent harness should look like architecturally, read this one. github.com/deepseek-ai/deepseek-harness

KERNEL open sourced hypeman, the VM manager behind a million browsers a month. Agentic workloads are bursty and untrusted: one user can start 2,500 browsers in a minute, each running whatever code the agent decides to visit. hypeman puts Firecracker, QEMU, Cloud Hypervisor and Apple’s Virtualization.framework behind one uniform lifecycle API of create, boot, pause, snapshot, restore, fork and shutdown. VMs from the same image share a read-only disk and write to their own overlay, forks hardlink the source’s memory snapshot instead of copying it, and any normal container image boots as a VM through a custom initrd. They run about 128 chromium VMs per bare metal host at 8x CPU oversubscription. The multi-hypervisor part is not academic: NVIDIA’s vGPU drivers only work with QEMU, while Firecracker wins on host efficiency, so they stopped choosing. MIT, Go, and it runs locally on Apple silicon. github.com/kernel/hypeman

moli is a headless browser written for agents, not for humans. Rust, dual Apache-2.0 and MIT, a complete web runtime with DOM, JavaScript through V8, CSS through Servo’s Stylo, layout and networking, speaking CDP, WebDriver Classic and WebDriver BiDi from a single binary with no separate ChromeDriver. It renders on demand rather than always, and outputs HTML, Markdown, JSON or semantic trees. Peak memory on agent workloads is about 102MB against Chromium’s 349MB, and median memory on a public crawl was 73MiB against Chrome’s 773MiB, with CDP ready in 35ms. If you are running hundreds of concurrent agent browsers, that ratio is your infrastructure bill. github.com/lexmount/moli

Google published SAM, a sovereign agent mesh. Apache-2.0, Go, a peer-to-peer network where autonomous agents discover each other, authenticate under a zero-trust model with cryptographic identities that travel across environments, and invoke each other’s tools over MCP with no manual wiring. It splits into a control plane for identity, a router for connectivity, and local nodes, and there is a public testnet. Early, but it is the agent-to-agent networking problem taken seriously instead of solved with a shared API key. github.com/google/sam

Agent Baseline put out a v1.0 draft for public comment. Published July 30 and discussed at Black Hat, it is a controls framework for running agents in an organisation: an authoritative registry with a stable identifier for every in-scope agent definition and deployment, recorded business purpose, accountable owners and risk classification, operating and approval and exception status with decision history, and integrity requirements for agentic evidence. It reads like the thing your security team is going to hand you in six months, so it is better to read it now while it is still a draft you can comment on. agentbaseline.org

Claude Opus 5 shipped on July 24. Anthropic’s agent-focused model at $5 per million input and $25 output, the same price Opus 4.8 cost, with a Fast Mode at $10 and $50 that runs about 2.5x faster. It was built for long-horizon agentic work, the kind that needs hundreds of model calls to finish a task, and it currently sits at the top of SWE-bench Verified. anthropic.com

Jeff Dean left Google after 27 years, and Hassabis moved to Chairman. Announced August 5. Dean is founding Discovery Loop, a public benefit corporation aimed at automating discovery in machine learning, science and engineering, and taking Sanjay Ghemawat, Oriol Vinyals and Quoc Le with him. Google is investing. Demis Hassabis steps back from DeepMind CEO to Chairman and adds Alphabet chief scientist, with Koray Kavukcuoglu taking day-to-day operations. That is the deepest bench in the industry reorganising itself in a single morning. Google’s post

Anthropic clarified its position on open weights, and the industry closed ranks the other way. After more than 20 companies including NVIDIA, Microsoft, Meta, Google, OpenAI, Hugging Face and Mistral signed a letter urging Washington not to restrict open-weight models, Dario Amodei published a response on July 27: Anthropic has never advocated for a ban, non-dangerous open models are a public good, but it wants mandatory safety testing on sufficiently capable models, anti-distillation enforcement, and chip export controls. He explicitly rejected the claim that open weights make defenders structurally safer than attackers. Read it next to GLM-5.3 holding its weights back for a safety evaluation. The disagreement is no longer about whether to open models. It is about what has to happen first. Anthropic’s position

DeepSeek V4 Pro went GA and got a lot more expensive. The API pricing page updated on August 12 with no blog post or changelog. It sits second on SWE-bench Verified at 96.4% behind Opus 5, at a fraction of the cost per test, and roughly matches Fable 5 and Kimi K3 on Terminal Bench 2.1. Then on August 17 DeepSeek introduced peak and off-peak pricing, with V4 Pro rates rising by as much as 12x. The cheap-frontier-model era has a ceiling after all. DeepSeek API docs

Also worth knowing: Google shipped Gemini 3.7 Flash on August 13. AMD acquired Taalas to etch models directly into silicon for inference. Cloudflare launched Cloudflare OS, an open platform for agents and apps, and Kitesurf, an agent-first browser running in V8 isolates. And GitHub shipped stacked pull requests, which developers have been building homegrown tooling for since roughly forever.

Open-weight AI is having its Kubernetes moment. Tobi Knaup’s essay from July 25 is the best framing I have read of this entire month. The argument is that open weights are following the same arc containers and Kubernetes did: proprietary first, then an open standard that wins not on being better but on being ownable, portable and inspectable. If you have watched one of these cycles from inside, the pattern recognition is uncomfortable and useful. tobi.knaup.me

Cross-model KV cache transfer, from NVIDIA. A closed-form linear mapping that lets you reuse one model’s KV cache in a different, larger model from the same family instead of re-running prefill after a swap. On Qwen3 14B to 32B, a single source layer explains 56% of the variance in target keys and 32% in values, rising to 79% and 65% with multiple source layers, and the ridge regression mapper retains 73 to 98% of standalone-prefill accuracy while running 2.7 to 25x faster than re-prefilling. If you route between model sizes, this is a real serving optimisation, not a curiosity. arxiv.org

Smaller, faster, safer: running Kimi and GLM at scale. Cloudflare’s August 3 writeup on serving frontier open models in production. FP8 KV cache halves cache size, taking addressable context on Kimi K2.6 from about 686,000 tokens to 1.37 million and lifting concurrency from 32 to 64 requests, for 41% higher peak throughput at 30% lower per-token cost. INT4 weights take GLM 5.2 from 705GB to 421GB with decode throughput up 16 to 55%. A page-tagging system validates cache mappings to prevent cross-tenant corruption for under 1% overhead. Accuracy stays within 0.8 points across their evals. This is what the open-weights flood looks like once it hits real infrastructure. blog.cloudflare.com

Two uncomfortable ones. Zed’s “Why LLMs can’t really build software” argues the gap is not code generation but maintaining a coherent mental model of a system, and it is the sharpest version of that argument I have seen. And “Stealing reasoning traces from proprietary LLM APIs” is a reminder that your thinking tokens are an asset with an attack surface. zed.dev | stolen-thoughts.com

Two videos on agent harnesses and agent memory. The first walks through the Hermes agent harness in about 20 minutes, covering loop engineering, the memory system and the self-improving cycle where completed tasks get summarised into reusable skill documents. The second is a clear explanation of agent memory as an engineering problem: how to store, retrieve and maintain it, with SQLite and graph RAG. If you are building anything agentic, watch them in that order. Hermes harness in 20 min | AI agent memory explained

A free short course on vLLM. Fast and Efficient LLM Inference with vLLM, taught by Cedric Clyburn of Red Hat on DeepLearning.AI: nine lessons, about an hour and 38 minutes, covering PagedAttention and continuous batching, quantization with LLM Compressor, benchmarking with GuideLLM, and quality evaluation with lm-eval. If you have been running vLLM by copying flags from a blog post, this fills the gap. deeplearning.ai

AI Infra Summit, Santa Clara, September 15 to 17. Full-stack AI infrastructure across compute, data movement, AI data centers, physical AI, and data and models, with 400-plus speakers and around 8,000 attendees. Main stage includes Jeff Dean, Intel’s Lip-Bu Tan, NVIDIA’s Ian Buck, Amazon’s Peter DeSantis and Meta’s Santosh Janardhan. If you work on the layers under the model, this is the one event that covers all of them on one floor. Two ways in: use code KUBE15 for 15% off full-access and VIP tickets, or apply for a free expo pass. Register with KUBE15 | Apply for an expo ticket | Event site

KubeCon + CloudNativeCon North America, Salt Lake City, November 9 to 12. Far enough out to plan, close enough to book. LF Events

I landed, opened my reader, and went through a month of releases in two days. First reaction was straight FOMO. Everyone shipped while I was eating ramen and taking rides at DisneySea.

Then it settled down. Look at what actually landed. Meta open sourced a 30B agent model. Alibaba put out 2.4 trillion parameters, and two days later the 27B that people will actually run. Moonshot released 2.8 trillion that almost nobody can serve. DeepSeek dropped an agent harness and got 100,000 stars in two days. Unsloth put model training on your desktop.

Nobody is charging you for capability anymore. That part is basically free now.

What is not free is everything around it. Where does the model run. Who is allowed to call it. What did that call cost and which team pays for it. Can this agent reach my production cluster if it decides to. Look at the cloud native half of this edition again and you will see every item is somebody trying to answer one of those. KAI Scheduler and vCluster for the GPU question. agentgateway for the who-can-call-it question. hypeman for the sandbox question. Tokenomics Foundation for the cost question. Agent Baseline for the who-owns-this-agent question.

That is the part that pays you, and I want you to take it seriously. Downloading Qwen 27B takes ten minutes. Running it for three teams without them stepping on each other is a quarter of work.

It is also why the kiac thing made me happy this month. It is a small tool. I built it because I was tired of pretending the nodes on my laptop were real nodes. Now someone with the same problem can find it. Build what you needed, put it out, let people break it. That is the whole thing.

One question for you this week. If the model is the cheap part now, what is the expensive part in your stack? Hit reply and tell me, I want to know what people are actually stuck on.

And if you run models locally, go claim your name on local.ai. Use my link and let’s see how far up that leaderboard we can push it.

Good to be back.

For our Japan trip posts you can check this Instagram - https://www.instagram.com/saloni_1814/

Share