On June 30, 2026, Google Research published TabFM, a tabular foundation model released with a scikit-learn-compatible API, pretrained weights, and a GitHub repo.
To understand why this release is worth taking seriously, you have to understand what it is pushing against. Tabular data is the format that most practising data scientists actually work with every day. For roughly a decade, the algorithm that won on that format, consistently, was gradient-boosted decision trees. XGBoost, LightGBM, CatBoost. Deep learning also got its space, specially when dealing with lots of data and high cardinality interactions.
With the advent of the Transformers architecture, there a current of thought that tabular models could be built on top of a generalist foundational layer to re-use for any problem. But, the results were not encouraging.
Now, Google’s TabFM is not another “let us apply a transformer to CSV files” attempt. The framing has shifted and the approach is to pretrain a model across many tasks so it learns a reusable prediction procedure, but then, at inference time, treat the training rows as context rather than gradient signal. We will cover details later in the post, but this training paradigm is different to other tabular foundational models and it’s worth diving a bit deeper into it.
After having read about it, it personally feels that TabFM packages the field’s best ideas into something practitioners can finally try (although with some calibrated optimism, not hype yet).
The blog post “Introducing TabFM: A zero-shot foundation model for tabular data”, by Weihao Kong and Abhimanyu Das, Research Scientists, Google Research was released in Google research blog space.
It comes with a companion Git and Hugging face link.
Why did gradient-boosted trees dominate tabular ML for so long? The peer-reviewed evidence for GBDT dominance and the 3 specific reasons neural networks struggled.
Why did early deep learning models for tables mostly fail? TabNet, TabTransformer, SAINT, FT-Transformer — what went wrong and why.
What is a tabular foundation model, and why is the framing different? The conceptual shift from per-dataset training to in-context learning.
How did we get from TabPFN to Google TabFM? The 4-branch family tree and what each branch contributed.
How does Google TabFM actually work? The exact 4-stage architecture, what
.fit()does and does not do, and the training setup.What is TabFM-Ensemble, and why does it matter for reading the benchmark? Why the headline numbers and the zero-shot numbers are not the same thing.
What does the TabArena benchmark evidence actually show for TabFM? TabArena, independent evaluations, and where the gaps are.
When should you reach for TabFM, and when should you stick with XGBoost? A practical framework for the decision.
Where does tabular ML and foundation model research go from here? What the trajectory of this field suggests about the next few years.
Let’s get started!
Instead of trusting my thoughts and notes about boosted trees, I wanted to rely on Grinsztajn et al. (NeurIPS 2022) paper “Why do tree-based models still outperform deep learning on tabular data?” to help us provide some solid data on boosted tree performance. Grinsztajn ran a systematic benchmark across 45 datasets, applied extensive hyperparameter tuning to both tree-based models and neural networks, and found that tree-based methods remained state-of-the-art on medium-sized tabular data. Even setting aside the speed advantage trees have, trees were more accurate.
The Grinsztajn’s goes further than just reporting the results of performance differences. It also identifies 3 structural reasons neural networks struggled on this format:
Robustness to uninformative features. Real datasets contain columns that provide no signal for the target — noise variables, proxies, redundant encodings. Decision trees handle these gracefully because splits on uninformative features are not chosen. Neural networks, particularly early tabular architectures, tend to overfit to noise, especially with limited data. Adding regularisation helps, but the benefit is inconsistent.
Preserving feature orientation. Trees make axis-aligned splits: “is column A greater than threshold T?” This is a natural fit for tabular data, where many relationships between features and targets are genuinely piece-wise or threshold-like. Neural networks learn smooth, rotation-invariant functions. That is a strength in vision and language, where rotating a representation does not destroy its meaning. In tabular data, where each column has a fixed, domain-specific interpretation, rotation-invariance is a liability.
Learning irregular functions. Tabular targets often depend on the data through highly irregular, non-smooth mappings. A tree ensemble can represent those patterns directly through its split structure. Fitting them with gradient descent on a neural network requires the architecture to carve out the right function class, which takes more data, more regularisation, and more tuning to get right.
Now, bare in mind that this was a 2022 paper, and lots has happened in the past few years related with neural networks and the transformer architecture. But the underlying foundations of these 3 reasons still hold. Dont get me wrong, deep learning can and does beat gradient boost trees for tabular data, but only if conditions are relevant (massive datasets, high cardinality of categorical features requiring embeddings, etc).
The architectures published between roughly 2019 and 2022 represent serious research.
TabNet (2019) introduced sparse sequential attention to select features at each decision step.
TabTransformer (2020) applied self-attention to categorical embeddings.
SAINT (2021) used inter-sample attention in addition to feature-level attention — it literally attends across rows as well as columns.
FT-Transformer, introduced by Gorishniy et al. (NeurIPS 2021), built a clean, well-tuned Transformer adaptation for tabular data and also introduced a strong ResNet baseline.
The FT-Transformer paper clearly described how the new architecture was competitive with the best tree-based methods on some datasets, but no single deep learning architecture was universally superior.
The fundamental problem is that tabular columns are heterogeneous in ways that break the assumptions that make transformers work so well in NLP.
In language, a token embedding carries a relatively stable meaning across documents. The word “bank” can mean different things in different sentences, but the token itself is drawn from a consistent vocabulary and its representation can be trained to be general. If you want a deeper dive into the Transformers architecture focused on language (the tech behind LLMs), check my series about it.
In tabular data, a column called “value” in one dataset might represent dollars, in another a sensor reading, in a third a count of events. The number 1000 in a revenue column has a completely different statistical relationship to the target than the number 1000 in a latitude column. The same value in the same column across two different tables may mean entirely different things.
And herein live the problem: a model trained on one tabular dataset cannot straightforwardly transfer features learned there to another dataset. Each fit starts from scratch. That eliminates the key advantage pretraining provides in NLP: the ability to build shared representations that generalise across tasks.
TabNet, TabTransformer, SAINT, and FT-Transformer are genuinely useful papers. They clarified what was possible with deep learning on tables and set up the benchmarks that later work would beat. But they were still fundamentally per-dataset models — fit, tune, predict — and on most well-tested real datasets, a well-tuned XGBoost was at least competitive and usually faster.
A classical neural network starts almost from scratch on your dataset. You give it your rows, your columns, and your target. Then, through gradient descent, it slowly adjusts its weights until it learns the patterns in that specific table.
For example, suppose you give the model a small table like this:
Then you ask it to predict this row:
A classical neural network would start with random weights and train on this dataset. It would repeatedly adjust its internal parameters until it learned, from this table alone, that larger houses tend to be more expensive, central homes carry a premium, and distance from the centre matters.
A tabular foundation model works differently. Most of the learning has already happened before you use it.
During pretraining, the model is exposed to millions of tabular prediction tasks. Not one dataset. Not one schema. Not one business problem. Many different synthetic or real tables, with different feature types, target variables, noise patterns, missing values, correlations, and nonlinear relationships. Over time, the model does not just learn one task. It learns something closer to the shape of tabular prediction itself.
If we continue with our example above, the foundational model has already seen many house-price-like problems during pretraining. It has seen tables where size matters, where location matters, etc.
So when it sees this new table, it reads the labelled rows as examples of the local rules. It may infer something like:
“In this dataset, size increases price, central neighbourhoods are expensive, and being close to the centre adds value. A 3-bedroom, 85 m² central property 2 km from the centre should probably be priced near the upper-middle part of the table.”
If we encapsulate what each paradigm is doing in a question like format, these would be:
A classical neural network asks: “What pattern can I learn from this dataset if I train long enough?”
A tabular foundation model asks: “Given everything I learned from many previous tabular tasks, what pattern does this new dataset seem to follow?”
This is why these models are often described as doing in-context learning. The training rows are no longer only used to update weights. They become part of the prompt. The model looks at them, infers the relationship between features and target, and produces predictions without necessarily taking a gradient step on your data.
Think of it this way. The traditional approach is like hiring a specialist who trained for years on one domain: strong there, but they need to re-learn from scratch in every new domain. A tabular foundation model is more like a researcher who has worked across dozens of fields — they have not seen your specific problem before, but they have developed a pattern-recognition capacity that transfers. They read your data, identify the structure, and start predicting well without months of domain-specific training.
Google TabFM converges concept from 4 major advanced in foundational models, borrowing row compression from TabICL and in-context learning from TabPFN, and packaging the result into an officially supported scikit-learn-compatible release. Let’s briefly cover these earlier models.
The problem TabPFN set out to solve was whether in-context learning was even possible for tabular prediction. Could a model trained on synthetic data do tabular prediction by treating training rows as context?
The answer, published in Nature (Hollmann et al.), was yes.
TabPFN’s architecture processes data in 3 stages.
In the first stage, each feature column is embedded independently using a transformer with inducing-point attention — this captures column-level statistics without requiring quadratic attention over all rows.
In the second stage, each row is compressed into a small number of CLS-like vectors, reducing the variable-length feature sequence to a fixed-dimension row embedding.
In the third stage, an ICL transformer receives all row embeddings — training and test rows together — and test rows attend to training rows to produce predictions.
TabPFN “worked”. On small datasets (up to roughly 10,000 rows), it was competitive with AutoML pipelines in a single forward pass. It was the first clean proof that the in-context learning paradigm transfers to tables. But, the limit was scale. ~10k rows is too small for most production datasets.
The obvious next question was whether the approach could be pushed to larger datasets. TabPFN-2.5 and TabPFN-3 scaled the original architecture to handle up to 50k rows and 2k features. This required the Muon optimiser (replacing AdamW) and scalable softmax in attention to manage memory at larger context lengths.
On TabArena — the most rigorous independent tabular benchmark available — TabPFN-3 in default configuration (single forward pass, no tuning) achieves an Elo of 1673. AutoGluon running a 4-hour ensemble achieves 1695. That is a meaningful gap closed: a single-pass, zero-configuration model matching a heavily-resourced AutoML system to within 1%.
TabPFN’s approach has an architectural cost. Running full row/column attention over the entire table is O(T²) in the number of rows. At 10k rows that is manageable; at 100k rows it is not.
TabICL (Qu et al., ICML 2025) addressed this by changing the scaling strategy. Instead of running attention over the raw feature matrix, TabICL first encodes each row into a fixed-dimension embedding, then runs the in-context-learning (a.k.a. ICL) transformer over those row embeddings. This is a much cheaper operation — the cost of building row embeddings is roughly linear in the number of features, and the ICL step then operates on compact vectors rather than full feature sequences.
TabICLv2 (Qu et al., Feb 2026) extended this further: a new synthetic data generation engine, the Muon optimiser, and scalable softmax in attention. TabICLv2 without any tuning surpasses RealTabPFN-2.5 (which uses tuning, ensembling, and real-data fine-tuning) on both TabArena and TALENT. It generalises to million-scale datasets within 50GB of GPU memory. The weights are fully open.
This is the architectural idea Google TabFM directly borrows: row compression to fixed-dimension vectors, followed by an ICL transformer over those vectors.
A parallel research question ran alongside the scaling work: does pretraining on purely synthetic data generalise well enough?
TabDPT (Xu et al., NeurIPS 2025) explored the alternative. It combines ICL-based retrieval augmentation with self-supervised learning on real tabular corpora, rather than purely synthetic SCM data. The finding was that real tabular data does improve downstream generalisation beyond what synthetic-only priors achieve, and that performance scales predictably with both model size and the quantity of pretraining data.
TabDPT represents the “synthetic priors alone may not be enough” branch of the family tree. Google TabFM does not appear to use real tabular pretraining data — the model card states synthetic SCM pretraining, and the generator code is not public. Whether that is a limitation will become clearer as evaluation data accumulates.
Google TabFM is not a revolution. It is a synthesis. It takes the core in-context learning approach from TabPFN, the row compression + ICL scaling architecture from TabICL, and applies synthetic SCM pretraining — the approach proven across the whole family. Understanding that lineage is understanding why TabFM looks the way it does. It is not a fresh start; it is the field’s current best ideas packaged into an official, scikit-learn-compatible release.
Google TabFM processes a table in 4 sequential stages.
Let’s understand this with an example. Suppose we have a customer churn dataset like the one below.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.