Once a week, we write this email to share the links we thought were worth sharing in the Data Science, ML, AI, Data Visualization, and ML/Data Engineering worlds.
And now…let’s dive into some interesting links from this week.
.
.
.
Airbnb’s Eval-driven development: Lessons from evaluating GenAI at scale
At Airbnb, we build LLM-powered features across our product, with recent launches including review highlights, AI customer support, smart communication features for guests and hosts, and more. Behind the scenes, we also use AI to help us spot trends and understand what’s working, guiding where we improve the product next. Each product team may have its own evaluation criteria, process, workflows, etc. However, these are built on top of some common foundations and principles. An infrastructure team provides tooling and best practices, incorporating learnings across domains so that they are shared with everyone building products at Airbnb. In this article, we wanted to share some of these best practices and learnings with the broader engineering community…I was reading about compression recently when I stumbled upon something crazy: that compressors and LLMs are, at their core, trying to solve the exact same problem. In this post, I’m going to walk us through the basics of compression to understand its deep relationship with language modeling…
Why Fair Coins Tend to Land on the Side They Started — A Wobbly Coin Flip Simulator
A few years ago, my colleagues and I decided to test the Diaconis-Holmes-Montgomery (DHM) hypothesis that a fair coin, when flipped in the air and caught in the hand, tends to land on its starting side slightly more often than 50% (Diaconis, Holmes, & Montgomery, 2007). In fact, DHM suggested that the effect would be about 1%, and they indicated that in order to detect such a small effect, a diagnostic empirical test would have to feature about 250,000 tosses. As a group, we ended up flipping a series of coins for a total of 350,757 times. In the end, the data strongly supported the DHM hypothesis, albeit with two nuances: the size of the effect depends on the person flipping the coin, and the effect appears to wane with practice. A two-page summary is here, and the full paper is here…City2Graph: Geospatial Graphs for Network Analysis and GNNs
City2Graph is a Python library that turns buildings, streets, public transport feeds, origin–destination matrices, zones, and points of interest into spatial and heterogeneous graphs. It bridges GeoPandas, NetworkX, and PyTorch Geometric so the same geospatial data can support network analysis, urban research, and Graph Neural Networks (GNNs)…A growing body of work empirically tests whether AI models when acting as agents comply with or violate applicable law, including corporate law, tort law, labor law, property law, and contracts. In this paper, we explore a related question: examining why AI models might obey the law. To this end, we draw on and extend the methods for measuring legal compliance pioneered in Tom Tyler’s seminal work, Why People Obey the Law (1990, 2006). Across three studies, we adapt Tyler’s survey methodology—which was originally devised to study the factors explaining human subjects’ compliance with law—to nine AI models. We elicit the AI models’ reported legal compliance alongside the four factors that Tyler proposed to explain compliance: deterrence, morality, peer disapproval, and obligation to obey the law…
Quantum Computing for the Probabilistic Bayesian
In this post, I share what I’ve learned about quantum computing from the perspective of a probabilistic Bayesian. I strip away the physics to reveal a familiar picture: qubits as amplitude vectors whose squares are probabilities, gates that shape distributions through interference, and measurement as sampling. The parallel to Bayesian inference is striking. Our priors and likelihoods shape posteriors; quantum gates shape amplitudes. What if the quantum computer isn’t a magic box, but a sampler purpose-built for the combinatorial spaces we already struggle to navigate classically?…CUDA Shared Memory Swizzling
When we write CUDA kernels that use shared memory, we have to be careful about shared memory bank conflicts. Having severe shared memory bank conflicts can introduce a significant performance penalty. One simple way to deal with shared memory bank conflicts is to use padding. However, padding can waste shared memory and can have other drawbacks. In this blog post, I would like to discuss how to deal with shared memory bank conflicts using swizzling. Swizzling is a more complicated technique that can be used to avoid shared memory bank conflicts without wasting shared memory…Cross-Validation From Scratch and a Surprise at n=100
In K-Fold Cross-Validation, the training data is split into K chunks; the model is trained K times, each time holding out a different chunk. Performance is averaged across all K folds, giving a more stable estimate. A special case is Leave-One-Out CV (LOOCV), where each individual observation serves as its own validation set. It’s thorough but computationally expensive. I was told that, bias LOOCV < 10-fold < 5-fold; whereas variance LOOCV > 10-fold > 5-fold. Is that true? Also, what’s with the repeats, does that really reduce variance? Let’s check them out…Parametric Models vs. Empirical Distributions and Ordinal Regression
A primary goal of this article is to compare the accuracy of empirical cumulative distribution functions (ECDFs) and their logical extensions—cumulative probability ordinal regression models—with that of parametric distribution models when the population distribution may not be normal…
The 97% That Predicts the Past: How logistics ML’s favourite benchmark number is made
There is a number that keeps appearing in supply-chain machine learning…about 97% accuracy at predicting late deliveries on the DataCo SMART Supply Chain dataset, the field’s most popular public benchmark. A peer-reviewed 2026 paper reports 97.47% with a metaheuristically tuned XGBoost. A thesis reports 97.58% and adds, proudly, that there were no false negatives at all. Dozens of public notebooks land within a point of the same figure. Here is the thing about that number. The dataset’s target column, Late_delivery_risk, is not an observation. It is an arithmetic consequence of two other columns in the same table: it equals 1 exactly when Days for shipping (real) exceeds Days for shipment (scheduled). I measured this directly on all 180,519 rows: the comparison reproduces the label 97.55% of the time, and every single one of the 4,423 disagreements is a canceled shipment, where the label is forced to zero…If a model is allowed to see the “real shipping days” column — how long the delivery actually took — then predicting lateness is not prediction. It is subtraction…How colour drained from our furniture
I'm moving apartments. As I was packing up, I found myself looking at my grey IKEA couch: the L-shaped one that can be turned into a bed. Why did I buy such a drab sofa?…To find out what happened to our European living rooms, I tallied the colour of every couch in IKEA’s (now discontinued) catalogues from 1960 to 2021. 3,500 couches later, I have confirmed my suspicions….
System Design Case Study: Tinder
Design the core of a location-based dating app: a user sees a feed of nearby candidate profiles, swipes like or pass on each one, and when two people like each other they match and can start talking…Tinder looks like a nearby-search problem: find profiles near me, filter by preference, done. That is only half the problem, and the smaller half. Every swipe is a write, and a user issues one swipe for every card in their feed — the feed is read far less often than its contents are acted on. The system that has to survive load is the swipe path, not the candidate-search path. Two invariants sit underneath that write path and cannot bend. A match is not something a user directly creates — it is a derived consequence of two independent likes, so the system must notice a mutual like without scanning anything. And a profile a user has already swiped must never resurface in their feed; the system must track every exclusion cheaply enough to check it on every request…I wrote an AI textbook — how long until AI can do it better?
Models being stagnant in long-form, non-fiction writing should be alarming to those reliant on models autonomously solving grand, open science problems in the near future. The models today struggle to organize and compellingly present some of the most established science in their area. This seems like a natural prerequisite that we should expect the models to master before they can solve broad, open-ended problems on their own. Until this is solved, the progress of LLMs for science will look closer to solving low-hanging fruit and merging distant connections across fields, rather than any sort of revolutionary insight…
.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.