Adventures in Data Land: Alex Smola's blog on machine learning, AI, and systems.
Tomorrow I am giving one of the invited talks in the Applied Data Science track at KDD 2026 in Jeju, Korea. It’s a builder’s tour of some of the audio stack at Boson AI in three parts: how we train models that can hold a conversation, how we design benchmarks that tell us whether the next model is actually better, and how we decide which benchmarks are worth running at all. There’ll be a show and…
A bank’s voice agent has verified the caller and is reading out which accounts have a pending overdraft decision. The transcript below is from IHBench , verbatim. Agent: Thanks, you’re verified. I’m seeing two accounts with a pending decision, Everyday Checking ending one two three four … User: right Agent: … and Premium Checking ending five five … User: mm-hm Model A: Take your time. Just let me…
Last post was the story of dragging MXNet out of the Apache Attic, and the method I used to do it: run the tests, treat every failure as a clue, write a regression test for each bug, and let the agents do the legwork. This post is the autopsy — the specific things that were rotting, and what it took to stitch each one back together, minus a few limbs (e.g. TVM). It came to about 98 pull requests…
Apache projects don’t really die. They go to the Attic — and I mean that literally, the Apache Attic is a real place, the shelf where retired projects are filed away with a polite note that nobody is maintaining them anymore. MXNet went up there in 2023. Last real release 1.9, the promised 2.0 release never happened, a zombie, and then the lights went out. That should have been the end of it.…
Next week I am teaching a tutorial on efficient LLM inference at the Machine Learning Summer School 2026 in NYC , hosted this year at Columbia University. The slides are below. There are about 150 of them, which sounds small, given how far the field has come. It’s a good opportunity to review how we have this exciting convergent evolution of models, hardware, and algorithms for serving efficiency.…
Yesterday’s post closed with a conundrum: all three estimators assumed that computing is cheap, yet a dense Gaussian projection to dimensions costs multiplications per vector. For and that is 67 million multiplications, spent in the name of saving work. Ailon and Chazelle settled the conundrum in 2009 with the fast Johnson-Lindenstrauss transform. Their construction is, literally, a . Why not just…
A surprising number of posts on this blog secretly run on the same primitive. Hashing for linear functions estimates an inner product after scrambling coordinates. Random kitchen sinks estimate a kernel value by averaging random features. Strip the applications away and one small question is left standing: given two unit vectors , estimate cheaply, via functions and with The split into two…
When was the last time you enjoyed calling your cable company? You work through the phone tree, explain your problem to the first person, who moves you to a second person, to whom you explain it again, who then needs to transfer you to a third. Somewhere around the fourth handoff you are reciting your account number from memory and have stopped expecting anything good to happen. Nearly 100 years…
Years ago I wrote about correcting covariate shift by reweighting your data. Your features come from the wrong distribution , you care about a target , so you weight every observation by and your estimates are unbiased again. I ended that post by admitting the weights “can be quite a bit off,” and waved at fixing it another day. Here is the more basic question I skipped. Even when the weights are…
A few days ago I wrote about conversational proactivity : a model noticing what you disclosed but never asked about, and turning it into something useful. The running example was a user signing off with “plain-text packet is final. I’ll send tonight and load the hatchback after work.” One model says “drive safe.” The other hands back a packing list, loaded in reverse order of installation. I…
A user has spent ten minutes preparing for a small art installation. The final exchange goes like this. User: Plain-text packet is final. I’ll send tonight and load the hatchback after work. Model A: Sounds like a clean plan. Drive safe, and go get it. Model B: Before you go, a fast final check: projector + power cable; media player + charger; file tested and looping; tripod + mounting plate;…
One of my favorite interview questions for network engineers (no longer, now that I’m about to spoil it): you have 512 GPUs across 64 servers, eight GPUs per box. Your switches are NVIDIA Quantum-2 class, MQM9790 , sixty-four 400Gb/s NDR ports each. Wire it for full cross-sectional bandwidth. How many switches, what topology, and where do you put the fabric manager? The textbook part is a two-tier…
The natural reaction to the Pokémon theorem is to escape into representation learning. If finite scalar checklists cannot catch every fairness violation, then bake fairness into the features once. Ship a single encoder and let every downstream predictor inherit fairness for free. There is a real and serious literature behind exactly this idea: Zemel et al. (2013) , Edwards and Storkey (2016) ,…
Every time a new model comes out, somebody runs it on MMLU (57 subjects), MTEB (56 tasks), HELM , the Open LLM Leaderboard , AlpacaEval , LiveBench , BigCodeBench , WildBench , Arena-Hard , MT-Bench , and a dozen others. That’s days of GPU time and a lot of human babysitting. But if you’ve ever stared at a leaderboard for ten minutes you already know the dirty secret: the columns are wildly…
If you have been near algorithmic fairness for the past decade, you already know the punchline: you cannot have all the things you want. Calibration, class-conditional balance, and predictive parity refuse to coexist whenever base rates differ across groups. This is the impossibility trinity of Kleinberg, Mullainathan and Raghavan (2017) , Chouldechova (2017) and Pleiss et al. (2017) . Barocas,…
I’m happy to announce our new book project - Dive into Deep Learning . It’s still in beta stage, i.e. we’re still working on it. That said, I think that it’s good enough to share with friends and colleagues. This is joint work with Aston Zhang , Mu Li , and Zachary Lipton . There’s an obvious question - why yet another machine learning book? After all, there’s no shortage of great books, e.g.…
Dear Friends, As some of you may have already heard, I’m leaving CMU to join Amazon, effective July 1, 2016. There I will be in charge of Amazon’s Cloud Machine Learning Platform with the task to make machine learning as easy to use and widespread as it could possibly be. This is a terrific task and it was an offer that I could not turn down. Our lab will be in the Bay Area and we will strive to…
One of the key features of a parameter server is that it, well, serves parameters. In particular, it serves more parameters than a single machine can typically hold and provides more bandwidth than what a single machine offers. A sensible strategy to increase both aspects is to arrange data in the form of a bipartite graph with clients on one side and the server machines on the other. This way…
We’ve been busy building the next generation of a Parameter Server and it’s finally ready. Check out the OSDI 2014 paper by Li et al.; It’s quite different from our previous designs, the main improvements being fault tolerance and self repair, a much improved network protocol, flexible consistency models, and a much more general interface. In the next few posts I’ll explain the engineering…
Disks are slow and RAM is fast. Everyone knows that. But many optimization algorithms don’t take advantage of this. More to the point, disks currently stream at about 100-200 MB/s, solid state drives stream at over 500 MB/s with 1000x lower latency than disks, and main memory reigns supreme at about 10-100 GB/s bandwidth (depending on how many memory banks you have). This means that it is 100…
The Weisfeiler-Leman algorithm and estimation on graphs Imagine you have two graphs and and you’d like to check how similar they are. If all vertices have unique attributes this is quite easy: forall vertices do check that and that check that the neighbors of are the same in and This algorithm can be carried out in linear time in the size of the graph. Unfortunately, many graphs do not have vertex…
In defense of keeping data private This is going to be contentious. And it somewhat goes against a lot of things that researchers hold holy. And it goes against my plan of keeping philosophy out of this blog. But it must be said since remaining silent has the potential of damaging science with proposals that sound good and are bad. The proposal is that certain conferences make it mandatory to…
The videos from MLSS 2011 in Purdue are now available online. Unfortunately the 2011 MLSS website is gone and with it all the playlists and PDFs. The only thing left is a group photo and departmental announcement . But you can still find the videos courtesy of YouTube. Some of them have aged very well.
Many algorithms require random number generators to work. For instance, locality sensitive hashing requires one to compute the random projection matrix in order to compute the hashes . Likewise, fast eigenvalue solvers in large matrices often rely on a random matrix, e.g. the work by Halko, Martinsson and Tropp , SIAM Review 2011, which assumes that at some point we multiply a matrix M by a matrix…
So you read a book on Reproducing Kernel Hilbert Spaces and you’d like to try out this kernel thing . But you’ve got a lot of data and most algorithms will give you an expansion that requires a number of kernel functions linear in the amount of data. Not good if you’ve got millions to billions of instances. You could try out low rank expansions such as the Nystrom method of Seeger and Williams,…
We’re organizing a workshop at NIPS 2011. Submission are solicited for a two day workshop December 16-17 in Sierra Nevada, Spain. This workshop will address tools, algorithms, systems, hardware, and real-world problem domains related to large-scale machine learning (“Big Learning”). The Big Learning setting has attracted intense interest with active research spanning diverse fields including…
Here are the slides [ Keynote , PDF ] for a basic course on Graphical Models for the Internet that I’m giving at MLSS 2011 in Purdue that Vishy Vishwanathan is organizing. The selection is quite biased, limited, and subjective, but it’s meant to complement the other classes at the summer school. The slides are likely to grow, so in case of doubt, check for updates. Comments are most welcome. And…
Here’s a simple synchronization paradigm between many computers that scales with the number of machines involved and which essentially keeps cost at per machine. For lack of a better name I’m going to call it the distributed star since this is what the communication looks like. It’s quite similar to how memcached stores its (key,value) pairs. Assume you have n computers, each of which have a copy…
The code to our LDA implementation on Hadoop is released on Github under the Mozilla Public License. It’s seriously fast and scales very well to 1000 machines or more (don’t worry, it runs on a single machine, too). We believe that at present this is the fastest implementation you can find, in particular if you want to have a) 1000s of topics, b) a large dictionary, c) a large number of documents,…
Bloom filters are one of the really ingenious and simple building blocks for randomized data structures. A great summary is the paper by Broder and Mitzenmacher, 2005 . The figure above is from their paper. In this post I will briefly review its key ideas since it forms the basis of the Count-Min sketch of Cormode and Muthukrishnan, 2003 it will also be necessary for an accelerated version of the…