Meta's First Training Chip Makes Official Debut

In March of this year, META 0.00%↑ disclosed plans for a series of chips including the MTIA 300. Recently, they finally lifted the veil of mystery surrounding the first chip in the roadmap.

According to reports, the MTIA 300 is Meta’s first AI training chip optimized for Deep Learning Recommendation Models (DLRM). Unlike GenAI training, DLRM training requires relatively low floating-point computing power but demands large memory capacity, high network bandwidth, and frequent collective communications. This combination typically results in low accelerator utilization. The MTIA 300 addresses these challenges through three key innovations:

• A chipset with built-in network interface cards featuring 12 × 800 Gbps RDMA NICs, ensuring high network performance and eliminating PCIe overhead;

• A dedicated message engine for collective offload that, while occupying only one-third of the chip area, delivers throughput comparable to the compute engines;

• Near-memory computing that places the message engines close to HBM, caches, and I/O, equipped with dedicated hardware to accelerate reduction-based collective operations such as ReduceScatter.

Although the MTIA 300 was originally designed for DLRM, these broadly applicable design principles also apply to subsequent MTIA-series chips optimized for GenAI models (MTIA 400, 450, and 500).

To the best of our knowledge, the MTIA 300 is the first accelerator with built-in NIC chips and a general-purpose collective offload engine, enabling efficient scale-up and scale-out communication. In addition, the MTIA 300’s moderate floating-point performance combined with higher HBM bandwidth and capacity was initially optimized for training DLRM models, but these characteristics also make it suitable for GenAI inference. Performance evaluations show that the MTIA 300 is comparable to contemporary GPUs in performance while offering cost advantages.