Machine Learning blog by Vishal Bakshi
I’m dusting off my ColBERT maintenance hat and getting back into the rhythm of things. The first thing I’m working on for the next release is exploring the most likely integration of flash-kmeans ( arxiv , github , pypi ) as a replacement for faiss-gpu. flash-kmeans (2026 Yang, et al) is a ridiculously fast “IO-aware batched K-Means clustering implemented with Triton GPU kernels.” I’ll be digging…
In his book “The Power of PULL,” Rob Snyder articulates a sort of separation-of-concerns between customer/demand and business/supply. Demand is what buyers are trying to accomplish in their lives. Supply is what we build. These are two separate things. There is no such thing as demand for supply. Demand exists as something independent out there in the world, whether or not our supply exists and…
ChatGPT’s illustration of the Cavern The Cavern represents the space of solutions. It’s vast. Not infinite, but large enough that no one person can map it. Even large groups can’t map it in full. The Cavern is an interconnected network of tunnels and open spaces. It’s mostly dark, with some light coming in from the outside. Everyone in your organization has heard that he objects inside the Cavern…
It takes translation labor to collaborate well on a cross-functional team. It’s the worthwhile cost of learning and building from other perspectives! What is translation labor? It’s a term I made up for this phenomenon. I like the sound of it. Comment below if there’s a better term! translation labor (noun) the work of mapping between two or more people’s understanding of a problem or solution.…
Snapshot of the book cover I recently came across a LinkedIn post sharing lessons from the book The Go-Giver (A Little Story about a Powerful Business Idea) by Bob Burg and John David Mann. I picked it up at a local bookstore and read it this week. The Go-Giver is a parable about a go-getter named Joe who, through his relationship with a man named Pindar, learns the Five Laws of Stratospheric…
Background I recently published a blog post on how I’m excited to start my professional ML career after 8 years of being a data analyst, educator, fast.ai student and community member, and now stanford-futuredata/ColBERT maintainer. In response, I have received messages from folks in the industry wanting to talk about career/project opportunities. As I was preparing for these conversations, I…
Where I Am Today In 285 days, my current role as a data analyst will come to an end. This is both a closing and an opportunity. I have worked as a data analyst for the last eight years, and it has been an incredibly rewarding experience. Working at all levels of the pipeline (including low-tech and no-tech environments), I’ve navigated constructive technical conversations in the context of complex…
Background I recently learned that it’s best practice to use different torch.allclose tolerances based on the precision of the floating point value. As a reminder, torch.allclose uses absolute and relative tolerances as follows: ∣input_i − other_i∣ ≤ atol + rtol × ∣other_i∣ bitsandbytes uses the following heuristic: if dtype == torch.float32: atol, rtol = 1e-6 , 1e-5 elif dtype == torch.bfloat16:…
Background I’ve been redoing my colbert-ai index comparisons between PyTorch versions using bitsandbytes’ torch.allclose tolerances . There are three PyTorch version changes that cause index artifact changes: 2.0.1 to 2.1.0 ( BertModel forward pass outputs diverge for all inputs ), 2.4.1 to 2.5.0 ( certain batch sizes cause BertModel output divergence ), and 2.7.1 to 2.8.0 (detailed in this blog…
Background I’ve been redoing my colbert-ai index comparisons between PyTorch versions using bitsandbytes’ torch.allclose tolerances . In this blog post I explore colbert-ai index artifact differences between PyTorch versions 2.0.1 and 2.1.0. Comparing Intermediate and Final Index Artifacts Final Index Artifacts Using the more lenient bitsandbytes tolerances, avg_residual.pt and bucket_weights.pt…
Background I’ve recently been documenting how PyTorch version changes impact stanford-futuredata/ColBERT ( colbert-ai on PyPI) intermediate and final index artifacts. The index artifact I’ll focus on in this blog post is the very important local_sample_embs tensor. This is the sample of token embeddings used to calculate the centroids, which are later on used during search. Instead of loading and…
Background In this notebook I’m going to explore how (and hopefully why) you can start with different codes.indices but end up with the same ivf and ivf_lengths when indexing a document collection using colbert-ai . I came across this behavior by accident. I was comparing final and intermediate colbert-ai index artifacts between installs using torch==2.0.1 and torch==2.1.0 and found that even…
I recently surpassed my goal of publishing 50 machine learning blog posts in 2025. I shared that on Twitter and got this interesting question from Skylar Payne: Do you have one that is your favorite or one you are most proud of? — Skylar Payne ( @skylar_b_payne ) September 3, 2025 I browsed through the blog posts I’ve published this year, and I found it really hard to pick just one because each…
Background In a previous blog post I showed how I traced index artifact differences between colbert-ai installs using torch==1.13.1 (the current pinned version) and torch==2.1.0 (the first version which produces different index artifacts) to a difference in floating point differences in the forward pass of the underlying BertModel . In a subsequent blog post I showed how the index artifact…
Background I’ll summarize the main points from the “Steering Semantic Data Processing with DocWrangler” paper by Shreya Shankar et al and share my commentary ( something I’ve been doing more of lately ). This work is inspiring and fascinating. Shreya previewed DocWrangler during the AI Evals course, but reading the paper—especially the user study section—helped me grasp its magnitude. While I lack…
I have recently found myself using the term “non-deterministic” to describe LLM behavior. However, something feels off about using that term and I’m nearly convinced that not only is it (sometimes) incorrect, it is imprecise, as it leaves unexplained a critical charactericistic of LLM behavior that makes LLMs different from deterministic functions. First, defining “deterministic algorithm”…
Background In a previous blog post I outlined two things: Which two subsequent PyTorch versions caused a divergence in stanford-futuredata/ColBERT index .pt artifacts (ConditionalQA document collection): Version A Version B All .pt Shapes Match? (Matches) All .pt Values Match? (Matches) 1.13.1 2.0.0 Yes (10/10) Yes (10/10) 2.0.0 2.0.1 Yes (10/10) Yes (10/10) 2.0.1 2.1.0 No (9/10) No (0/10) 2.1.0…
Background I recently released colbert-ai==0.2.22 which removed the deprecated transformers.AdamW import among other changes . I’m now turning my attention to upgrading the PyTorch dependency to 2.x, which will not only introduce compatibility with modern version installations of torch but will also allow the integration of the AnswerAI fastkmeans library as a replacement for the faiss-gpu and…
Yesterday I learned of the Modal docs example showing how to start a jupyter server via a Modal tunnel . I was elated to see this because it solved my problem of not being able to specify a custom image when using modal launch jupyter . I have a Dockerfile which installs colbert-ai from the main branch of the stanford-futuredata/ColBERT repo with a specific PyTorch and Transformers version: FROM…
Background This blog post is part journal, part reflection, and part planning around the topics of AI engineering, AI evals, applied AI, and my career path into machine learning. When I decided to take Hamel and Shreya’s AI evals course , I had recently watched a very short talk on LLM reliability by Featherless AI CEO Eugene Cheah. I had also learned about the realevals.xyz benchmark , which…