Qwen3.8: What Hardware Will You Need to Run Alibaba’s 2.4T Model?

Alibaba’s Qwen3.8 is a 2.4-trillion-parameter frontier model. The company has indicated that it plans to release the model’s weights, potentially making Qwen3.8 one of the largest openly downloadable AI models ever produced.

For comparison, Alibaba’s largest open-weight model to date, Qwen3-Coder-480B-A35B-Instruct, is roughly five times smaller. Qwen3.8 would still have about 400 billion fewer parameters than Kimi K3.

With both K3 and Qwen3.8 performing nearly at the frontier, was closing the accuracy gap with OpenAI and Anthropic largely a matter of scaling from hundreds of billions of parameters to more than two trillion?

An open-weight release for Qwen3.8 is excellent news. It could give researchers, companies, and the broader AI community unprecedented access to a model at this scale.

But what could most people realistically do with a 2.4-trillion-parameter model?

Very little locally. Even with aggressive quantization, compression, or distributed inference, a model of this size would be far too large to run on a typical personal computer. Its release could still be highly significant for research institutions, infrastructure providers, and well-resourced community projects. For casual users, however, direct local deployment would remain largely impractical.

Let’s check, with a few assumptions, what you would need to run NVFP4, Q4, and Q1 GGUF versions.

As I write this, Alibaba has not disclosed when it plans to release the model’s weights. That could change quickly. They may even be released by the time this article is published.

Alibaba has not yet disclosed how many parameters are activated for each token, how many experts the model contains, or whether every layer uses mixture-of-experts routing.

Nevertheless, a dense 2.4-trillion-parameter transformer would be extraordinarily expensive to serve. It is thus reasonable to assume that Qwen3.8 uses a highly sparse mixture-of-experts, or MoE, architecture.

Until Alibaba publishes the architecture, Moonshot AI’s Kimi K2 provides a useful proxy for estimating how a trillion-parameter MoE model might distribute its weights.

Kimi K2 contains:

• Total parameters: 1.04 trillion

• Activated parameters: 32.6 billion

• Routed experts: 384

• Experts selected per token: 8

• Routing sparsity: 8/384, or 1/48

• Shared experts: 1

Note: These figures come from the Kimi K2 technical report.

98.9% of Kimi K2’s parameters are located inside routed experts, while only about 1.1% are always-active or otherwise non-routed weights. The trend is the same for all the MoE with 500B+ parameters: Nearly 99% of the parameters are in the routed experts.

In other words, Qwen3.8 could store 2.4 trillion parameters while using only a small fraction of them for each token. If its routing sparsity resembled Kimi K2’s, the computation required per token might be comparable to that of a roughly 75-billion-parameter dense model, before accounting for routing and distributed-communication overhead.

BF16 stores each parameter using 16 bits, or two bytes.

This does not include the KV cache, inference-engine workspace, CUDA graphs, temporary activations, multimodal components, tokenizer files or checkpoint metadata.

A practical deployment would consequently need more than 4.8 TB of combined accelerator memory. I won’t speculate on the KV cache size. There are too many variables that influence it: number of linear layers, number of KV heads, etc.

I hope Alibaba releases an official NVFP4 version. Producing a high-quality NVFP4 quantization of a model this large would be prohibitively expensive for most community projects. NVIDIA has also been relatively slow to publish its own NVFP4 conversions of open-weight models; for example, its version of Qwen3.6 appeared only recently.

NVFP4 stores each parameter as a 4-bit value, with a single FP8 scaling factor shared across all 16-value blocks. This results in an effective storage cost of approximately 4.5 bits per parameter, plus a negligible FP32 scaling value for each tensor.

Fortunately, routed experts tend to be relatively robust to quantization. In MoE models, the expert weights can often be stored at lower precision while the comparatively small set of shared, attention, embedding, and other always-active parameters remains at higher precision. This mixed-precision approach can substantially reduce storage requirements while limiting the effect on model quality.

So, with 99% of parameters quantized to NVFP4:

• 2.374 trillion routed-expert parameters at 4.5 bits each

• 25.8 billion remaining parameters at 16 bits each

The expert weights consume:

The BF16 remainder consumes:

Total:

That’s an average of 4.62 bits per parameter.

Compared with a 4.8 TB BF16 checkpoint, that saves approximately:

or 71.1% of the original weight memory.

A common GGUF format such as Q4_K_M is approximately 4.8 bits per weight in representative models, while Q4_K_S is closer to 4.6 bits per weight. Actual ratios vary with architecture and which tensors remain at higher precision.

For a 2.4-trillion-parameter checkpoint:

• Q4_K_S-like: 4.58 bits; approximately 1.374 TB

• Q4_K_M-like: 4.84 bits; approximately 1.452 TB

• Higher-overhead Q4: 5.0 bits; approximately 1.50 TB

With runtime allocations and a useful KV cache, a deployment should budget at least 1.6–1.8 TB of usable memory, and more for long contexts or concurrent users.

I’m mentioning TQ1 since Unsloth has previously released very good TQ1 versions of Qwen3.5. We have no guarantee they can do/will do the same for Qwen3.8.

TQ1_0 uses a compact ternary representation at approximately 1.69 bits per weight.

Purely as storage arithmetic:

It will not be a normal desktop model. Even the Q4 version would be roughly twenty times larger than a 70-billion-parameter Q4 model.

A BF16 deployment would need at least 4.8 TB for weights and roughly 5.5 TB after adding a modest 15% operational allowance.

That points toward configurations such as:

• Approximately 24 B300 GPUs, assuming 288 GB each.

• Approximately 32 B200 GPUs, assuming 180 GB each.

NVIDIA’s eight-GPU DGX B200 provides 1.44 TB of HBM and can be configured with 2–4 TB of system RAM. NVIDIA’s Blackwell Ultra B300 provides 288 GB of HBM per GPU, or approximately 2.3 TB in an eight-GPU system.

BF16 Qwen3.8 is firmly a multi-server deployment.

A 1.4–1.5 TB quantized model is more approachable, but “approachable” still means rack-scale hardware.

An eight-GPU B300 server with approximately 2.3 TB of HBM should have enough room for the quantized weights, runtime allocations, and a reasonable KV cache.

An eight-GPU B200 system has only 1.44 TB of HBM. It might barely load the 1.387 TB mixed-NVFP4 estimate, but it would leave almost no memory for the inference engine or KV cache. A two-node, 16-B200 configuration would be much more practical.

A 507 GB TQ1 checkpoint would still need roughly 600 GB or more after runtime overhead.

It could fit in:

• A 768 GB or 1 TB RAM server.

• Four B200 GPUs with sufficient aggregate memory.

• Three B300 GPUs in a custom configuration.

• An eight-GPU node with considerable unused capacity.

A current maximum-memory Mac Studio offers up to 512 GB of unified memory, which would be too tight once runtime overhead and the KV cache are included.

So, running even the most compressed version of Qwen3.8 won’t be cheap.

Hopefully, Alibaba will also release smaller versions of Qwen3.8.