MOIRAI-2 Technical Report: Discussing the Recent Advances in Foundation Time Series Models

MOIRAI is a notable foundation forecasting model released by Salesforce researchers.

Recently, MOIRAI-2 came out, and the authors also shared a technical report on the updated model. I previously wrote 2 tutorials showing how to use MOIRAI for electricity demand forecasting and for sparse data.

In this article, we take a closer look at MOIRAI-2’s architecture. The report highlights details such as:

• The differences between MOIRAI-1 and MOIRAI-2, and what issues inspired the authors to make these changes.

• The challenges the researchers faced when upgrading MOIRAI-1.

• New directions for future foundation models.

Let’s get started!

✅ Find the 2 projects for this MOIRAI-2 here: AI Projects Folder (Project 21 and Project 22) (on electricity-demand forecasting and sparse data)

For Black Friday, I’m offering a total 20% off the yearly AI Horizon Forecast subscription!

Get the discount 🎁

MOIRAI-2 differs substantially from its predecessor. In particular:

• Architecture: Shifts from an encoder-only design to a decoder-only, autoregressive model.

• Output: Drops the complex “mixture of distributions” head and uses a simpler quantile head that directly outputs 9 quantiles, trained with multi-quantile loss.

• Single patch length: MOIRAI-1 used different patch lengths per frequency, while MOIRAI-2 uses a single patch length = 16.

• One variant: MOIRAI-2 offers a single 11M-parameter model with a detailed size-vs-performance ablation study.

Next, let’s briefly outline the model’s overall architecture.