Qwen3.8 27B, Nemotron 3.5, Muse, DeepSeek V4 Pro: A Huge Week for Open-Weight AI

Hi everyone,

This week was unusually rich in open-weight releases.

Meta released Muse Glimmer. Qwen finally released the weights of Qwen3.8 2.4T. NVIDIA released Nemotron 3.5 Lightning. DeepSeek just pushed a major V4 Pro update.

And, just as I was finishing this article, Qwen3.8 27B was released. It was worth the wait. According to Qwen’s evaluations, it improves very significantly over Qwen3.6 27B essentially everywhere, with particularly spectacular gains on long-horizon agentic coding: DeepSWE jumps from 13.3 to 42.2, while QwenSWEBench goes from 49.3 to 79.0. The 27B model also beats Qwen3.7-Plus on many coding and agentic benchmarks, and even beats Opus4.6 Max on QwenSWEBench, CoWorkBench, and LiveCodeBench. At this point, Qwen seems so far ahead that catching up is becoming difficult for everyone else.

There are also a few interesting changes compared with Qwen3.6. preserve_thinking, which Qwen3.6 introduced as an option for keeping reasoning traces across turns, is now enabled by default. Qwen3.8 also introduces reasoning_effort, with low, medium, and xhigh levels to trade reasoning depth against cost. Qwen now recommends temperature=1.0 and top_p=0.95 for thinking mode generally, and, for very long agentic runs, recommends allowing up to 262K tokens for reasoning and 131K for the final answer. I’ll publish a full analysis of Qwen3.8 27B within the next few days, with a separate look at its quantized versions later.

Note: I think this “preserve_thinking” (which is a feature we can find in other models) should be carefully evaluated, especially its impact on inference cost. Qwen3.8 is a crazy thinker (they recommend allowing “up to 262K tokens for reasoning”). So if you preserve reasoning for each turn, the context may grow to millions of tokens. In practice though, for tool calls, reasoning is often much shorter and more/better reasoning may yield fewer turns, so I guess it can work. But I’m really interested to know how important it is, in terms of accuracy/efficiency, to preserve the reasoning traces. KV cache quantization will likely be very important.

Finding enough time to cover all of this properly is hard. I don’t want to publish articles that simply repeat benchmark tables and model cards. A proper analysis means testing accuracy, token efficiency, memory, speed, and then doing it again for several quantized versions.

For Glimmer, I already published an article covering the model and its architecture:

• Muse Glimmer: Meta’s 30B Model Built for Efficient Inference

I also already have speed measurements for all the non-GGUF quantized versions I’m testing.

At the moment, essentially all my compute capacity, several RTX Pro 6000 GPUs provided by Verda,1 is being used to evaluate the quantized versions that run through vLLM: NVFP4, INT4, and my own mixed-precision versions.

You can already find those here:

• My quantized Muse Glimmer collection (they all run, but consider all of them very bad until I publish some official evaluations)

I’ll probably publish a short article focused only on speed first, since those results are already ready.

Then, expect perhaps two deeper articles comparing Glimmer with Gemma 4 and Qwen3.6/3.8, and another article focused specifically on the quantized versions. I may include GGUF versions in that one as well, but I’m not sure yet. There is only so much GPU time, and writing time, in a week.

So, for this Weekly Kaitchup, I’ll focus on the three releases that I won’t cover in more detail:

• Qwen3.8 2.4T: we finally know what is inside it, so I can check how well my predictions held up.

• Nemotron 3.5 Lightning: essentially the new Nano-sized Nemotron.

• DeepSeek V4 Pro 0813: a very large update that makes the word “Preview” much more meaningful.

A few weeks ago, before Alibaba disclosed the architecture, I tried to estimate what hardware would be required to run Qwen3.8 2.4T:

• Qwen3.8: What Hardware Will You Need to Run Alibaba’s 2.4T Model?

At the time, we knew only one particularly important number: 2.4 trillion parameters.

Everything else had to be inferred.

Now we have the weights and architecture, so it’s a good opportunity to see what I got right and what should be corrected.

Qwen3.8-2.4T-A95B has 2.4T total parameters and 95B activated parameters. It has 92 layers and uses the hybrid architecture introduced with Qwen3.5, alternating Gated DeltaNet blocks with periodic full attention. Its MoE has 512 experts, with 10 routed experts plus one shared expert activated per token. Native context is 262K, extendable to just over one million tokens.

The downloadable 2.4T checkpoint is also more limited than the hosted Qwen3.8-Max: it is text-only and requires thinking mode, while the hosted Max version adds vision, non-thinking mode, built-in tools, and a 1M default context.

So, how did my predictions go?

This one was the most important and easiest assumption.

I wrote that a dense 2.4T model would make very little sense for inference and that Qwen3.8 was almost certainly going to be a highly sparse MoE.

That was correct.

Only 95B of 2.4T parameters are active for each token, or roughly 4% of the entire model.

I assumed that approximately 99% of Qwen3.8’s parameters would live inside routed experts, based largely on Kimi K2 and other very large MoEs.

This turned out to be close.

Using the released dimensions, 512 experts, 92 MoE layers, hidden size 8192, and expert intermediate size 2048, the routed expert matrices account for roughly 2.371 trillion parameters, or about 98.8% of the entire 2.4T model.

My estimate in the original article was 2.374T routed-expert parameters.

I used Kimi K2 as a proxy.

Kimi routes each token through 8 of 384 experts, or about 2.08% of its routed experts.

Qwen3.8 uses 10 of 512, or about 1.95%.

So, while the architectures are different, the routing sparsity I used for the estimate was close enough.

This is the main miss.

Using Kimi K2 as a proxy, I estimated that Qwen3.8 might require compute roughly comparable to a 75B dense model per token, before routing and communication overhead.

The actual number is 95B activated parameters.

That is around 27% higher than my estimate.

So, the general prediction, a 2.4T model with only a small fraction active, was correct, but Qwen3.8 is somewhat more computationally expensive per token than I expected.

Wrong but close.

I estimated 1.387 TB for a version where the routed experts are stored in NVFP4 while the rest of the network remains at higher precision.

A community NVFP4 checkpoint from RadixArk now does exactly that: only the routed-expert linear layers are quantized to NVFP4 while the other components retain their original precision.

Its repository is 1.48 TB.

So my estimate was around 6–7% too low, but the overall hardware conclusion was correct.

A single 8xB300 node is enough to deploy this model.

NVIDIA also released Nemotron 3.5 Lightning 30B-A3B this week.

If the naming feels slightly confusing, Nemotron 3.5 Lightning is effectively the updated model occupying Nemotron 3 Nano’s position in the family.

I previously tested the original Nano here:

• Nemotron 3 Nano: A Very Fast Model That Doesn’t Think Too Much

And I covered the much larger Super model here:

• Nemotron 3 Super: 1M Tokens, Small KV Cache

Lightning still uses the same general idea of interleaving Mamba-2 processing with selected attention layers and sparse experts, and it supports context lengths up to 1M tokens.

So this is not a new size class. It is much closer to a refreshed Nano.

But NVIDIA has brought several ideas developed across the rest of the Nemotron 3 family back into this smaller model.

The biggest additions are around inference and agents.

Multi-Token Prediction is now part of the model training, followed by an additional MTP-boosting phase. NVIDIA also releases DFlash and DSpark draft models for speculative decoding, so there are multiple ways to accelerate generation depending on concurrency and hardware.

This is interesting because MTP was already one of the important additions in Nemotron 3 Super.

NVIDIA also says that 3.5 Lightning received harness-optimized training for agent workloads. Large Nemotron models can do the expensive planning and reasoning, while Lightning is supposed to execute the many smaller calls produced by long-running agents.

There are BF16 and NVFP4 checkpoints, and NVIDIA says the model can reach up to 4x the output speed of similarly sized models. On its PinchBench test, NVIDIA reports roughly 86% accuracy while completing 10,000 tasks around 30% faster than Qwen3.6 35B at comparable accuracy. Those are NVIDIA’s numbers, so I would still like to reproduce the speed/accuracy trade-off independently.

Finally, DeepSeek (quietly) released DeepSeek-V4-Pro-0813.

When DeepSeek introduced V4 in April, both V4 Pro and V4 Flash were explicitly called Preview models. The architecture was extremely innovative.

V4 Pro has around 1.6T parameters with 49B active and combines Compressed Sparse Attention with Heavily Compressed Attention. DeepSeek also introduced Manifold-Constrained Hyper-Connections and trained with the Muon optimizer. At a one-million-token context, DeepSeek reported that V4 Pro required only 27% of the single-token inference FLOPs and 10% of the KV-cache memory of DeepSeek V3.2.

Accuracy was the problem.

The original V4 Pro was rather underwhelming for such a huge model, particularly on agentic tasks. Then DeepSeek released Flash-0731, and the Flash model was suddenly beating Pro Preview.

I wrote about that two weeks ago:

• DeepSeek-V4-Flash-0731 and Inkling Small: Smaller, but Better?

At the time, I wrote that Flash-0731 was “a very promising sign for the next V4 Pro update.” That update is here now. The model now also exposes three reasoning-effort settings: low, high, and max.

The accuracy jump is much more important.

A few examples from DeepSeek’s own evaluation table:

• DeepSWE: 12.8 (it was lower than Qwen3.8 27B!) → 62.7
  Current Flash-0731: 54.4

• Terminal Bench 2.1: 72.1 → 87.9
  Current Flash-0731: 82.7

• NL2Repo: 38.5 → 61.5
  Current Flash-0731: 54.2

• Cybergym: 52.7 → 83.3
  Current Flash-0731: 76.7

• Toolathlon Verified: 55.9 → 74.1
  Current Flash-0731: 70.3

• AutomationBench: 12.8 → 31.8
  Current Flash-0731: 25.1

Pro-0813 is now ahead of the current Flash-0731 model.

DeepSWE is the most spectacular improvement: 12.8 to 62.7.

As I discussed this week, agentic evaluations can move significantly depending on the harness, configuration, allowed steps, tools, and other details. Independent testing remains necessary.

A $50 Coupon to Try Verda’s GPUs

In collaboration with Verda, I’m sharing a $50 coupon that you can redeem in your Verda account, after provisioning your account with $5, to try their GPUs (B200, B300, RTX Pro 6000, …).

Coupon code: KAITCHUP-50

Follow these instructions to redeem it.

Note: I share this coupon because I really think it’s a good deal. I don’t receive any form of compensation from Verda, or any usage information, related to this coupon. Verda also provides compute sponsorship for some of my articles.

That’s all for this week.

If you like reading The Kaitchup, consider sharing it with friends and coworkers (there is a 20% discount for group subscriptions):

Share The Kaitchup – AI on a Budget

Have a nice weekend!