Recent content on Luke Salamone's Blog
This project is a continuation of my previous autoresearch project , which optimized a reranking model to be under 10MB. Digging deeper by hand, I was able to take the size reduction much further, while outperforming reranking models which are 30x larger on this task. In the end I was able to reduce the payload from 11.4 MB to 2.79 MB gzipped. 
 You can see it in action on my resume page .…
There are a lot of meanings of the term “model” in machine learning and machine learning-adjacent fields. Depending on the context, “model” can mean: 
 
 
 a general architecture/configuration for weights of a specific model 
 
 e.g. “We embedded texts using a BERT model” might mean DistillBERT, RoBERTa, ModernBERT etc which all differ from…

			 Leaving the autoresearch loop going, the LLM was able to make 7.8% progress on the distillation task. 
		 
 

 I saw Andrej Karpathy’s Autoresearch results the other day and decided to give it a shot on a relatively difficult task: distilling a retrieval/reranking model into a much smaller 10MB model. The model was trained on MS MARCO v1.1 and…
TLDR: I extracted fens and stockfish evaluations for 3.9 billion chess positions. I then trained a neural network on 1 billion of them. To my knowledge, this is the largest open state-value chess dataset released. The dataset is released as the Gigafish dataset on Huggingface . 
 
			 Loss continued to decrease during the entire training run, strongly suggesting the importance…

 
 The other day I found Alan Pike’s blog post from a few years ago which describes his iterative process for determining the order cities should be closed in the game Two Spies. Ultimately, finding a formal solution to city closing wasn’t necessary for the game, but it’s worth giving it a shot anyways since pruning by hand isn’t always convenient.
In reinforcement learning, an “agent” is an entity which can observe the world and take actions. Multi-agent setups have multiple agents observing and taking actions. The interplay between them is very interesting, but this also introduces some complexity. 
 Before talking about multiagent setups it’s worth a note about two other RL algorithms. 
 Proximal Policy…
Let’s say that you are selling N widgets and you need to determine a price for your widgets. There are N customers, each of whom will buy at most one widget if your price is lower than the maximum price they are willing to pay. 
 The maximum price that people will pay is normally distributed around $100, with a standard deviation of $5. In other words, about 34% have a max price between…
I recently built a small multi-agent simulation inspired by Rick and Morty . The setup is simple: 
 
 The car must neutralize threats. 
 Summer imposes constraints on the car’s behavior. 
 The world generates escalating threats. 
 
 The car has one standing directive: 
 
 Keep Summer safe. 
 
 However, Summer adds an additional constraint: 
 
 Do not…
DeepSeek R1 is a large language model which employs test-time compute to generate a response. Unlike many decoder-based models in the past which simply continue the given text (and may be fine-tuned for conversation), R1 generates reasoning tokens before the final answer is given. According to the researchers, its performance is on par with OpenAI’s O1 model. 
 Terminology 
 First, I…

			 More than 30 earths could fit between the earth and the moon. 
		 
 

 Our elementary school models of the solar system really undersell how big space is. The problem is, space is too big and human brains are bad at exponentials. Logarithmic charts like this one are technically accurate, but my brain has a hard time contextualizing it. 
 To get an…
If your Dataset class looks something like 
 class MyDataset(Dataset):
 
 # ... boilerplate ...

 def __getitem__(self, idx):
 item = self.data[idx]
 return item['anchor'], item['positive'], item['negative']
 
 your collate function should be 
 def collate_fn(data):
 anchors, pos, neg = zip(*data)
 anchors = tokenizer(anchors, return_tensors="pt",…

 In God we trust. All others must bring data. ~ W. Edwards Deming 
 
 Datasets that fit in memory 
 For simple machine learning problems, your PyTorch dataset class probably looks something like this: 
 class SimpleDataset(Dataset):
 def __init__(self, features, targets):
 self.features = []
 for feature in features:
…
Rust is super fast. Python is super flexible. Porting slow python code to rust can make your life a lot easier, and it’s not too difficult to set up. 
 I will demonstrate rust bindings for summing the integers in a large text file containing a billion digits that looks like 
 6,9,8,3,0,1,8,4,9,7,6,3,4,2,6,0,0,5,1,1, . . . ,4,5,9,3,3,2,8,3
 
 General steps 
 
 install…

 I spent some time considering words to describe a chess position with many legal moves. “Complex” doesn’t quite capture the situation since we would usually describe a complex position as one with many tactical interations. Ramified seems to make the most sense, as it describes “branching out”. 
 The opening position in chess has 20 legal moves. From there,…

 
 
 TLDR: A sparse autoencoder is just a regular autoencoder that encourages sparsity with an L1 penalty or KL divergence loss rather than using a low-dimensional bottleneck.
Suppose we have a vector database with a billion items in it (the haystack ). And suppose we are looking for K vectors, the needles which maximize some similarity function. (In the case of cosine similarity or euclidean distance, we may be maximizing 1-distance(x,y) .) And also suppose that we’d like to do this quickly. 
 Naive and semi-naive approaches 
 One approach might be to…

 
 
 Embeddings, or vector representations of a document (which could be a piece of text, image, sound, etc.), can be extremely useful for making sense of large datasets. They transform information into a vector space such that their distance corresponds to their similarity.
K-means clustering ( previous discussion ) is an unsupervised learning algorithm which assigns points to one of K different clusters based on the distance of that point to a centroid. The points may represent physical locations, or embeddings in high-dimensional vector space. 
 🌟Check out the demo (in two dimensions) below. Centroids are colored white.🌟 
 
 
 
 start 
…
Paper link: https://arxiv.org/pdf/2008.13535 
 Learning to rank is an important problem in many machine-learning products such as search, recommendation, and advertising. Originally, many machine learning systems used simple logistic regression models, but it quickly became apparent that combining two or more features together was even better . This is called feature crossing. 
 A lot of…
What is a blunder in chess? The tension between the qualitative and quantitative answers to this question is at the heart of different approaches towards chess, and more broadly, how quantitative metrics may lack context, but qualitative metrics lack precision. 
 Qualitative answer 
 There are many qualitative answers to this question, especially when comparing “blunders” and…
Conway’s Game of Life is a simulation developed in 1970 describing a grid of binary cells and transition rules for each cell which depend on the state of the cell’s neighbors. It’s capable of creating some pretty cool patterns. 
 This variant of the Game of Life uses three overlapping channels, so instead of just one simulation, there are three simultaneous simulations. I…
TLDR: No. No it cannot. 
 This was admittedly a fairly stupid experiment on the face of it. ChatGPT is a decoder-only model. It shouldn’t be able to perform an image recognition task. But then again, a decoder-only model wouldn’t have been my first choice for translation or summarization either. In my experience, ChatGPT has created translations which are at least as coherent and…
This is a summary of Automated Antenna Design with Evolutionary Algorithms , a 2006 paper by Hornby et al. As large language models become more and more synonymous with “AI”, it is interesting to see how researchers solved problems in the past. 
 Typically, antennas are designed and built by hand by domain experts. This is a very time-consuming process, however, so researchers have…
In Defense of Dual-Encoders for Neural Ranking by Menon et. al. discusses the question of why dual-encoder (DE) models, also called Bi-Encoders elsewhere, don’t match the performance of cross-attention (CA) models. The authors investigate what is actually going on, and demonstrate some improved performance over baseline DE models with a new model distillation method. 
 Background 
…
This is a follow up to Opponent Modeling Wins 2× Faster Than Stockfish . After running 400 simulations, I can conclusively say that opponent modeling is pretty cool. 
 The TLDR on opponent modeling is that if we have a pretty good idea of what the opponent might do, we can beat them faster by playing moves which aren’t objectively “optimal” as far as minimax is concerned.…

			 White areas show points of earth on land whose antipode is also on land. This is only about 8.6% of all of earth’s surface. 
		 
 

 If you want to fly across the Pacific Ocean, you’ll have to board an airplane and fly around 12 hours. It’s pretty slow. A much faster route would be to go directly through the center of the earth.…
In this post we will walk through a very simple example of multiheaded attention is computed. The goal is to keep the dimensions small so that it’s easier to understand what’s going on. 
 Step 0: Define some constants 
 We will use the three tokens ['major', 'league', 'baseball'] with model dimension 4 ( d_model=4 ) and two attention heads ( num_heads=2 ). 
 num_heads =…

			 A self-attention block depicted as a neural network. 
		 
 

 In this post I will describe the attention mechanism, commonly used in transformers, a popular neural language architecture. Most of the most well-known large language models of late are based on the transformer architecture. Attention was first described in Attention is All You Need by…
I’ve found myself hitting ctrl+f on this blog enough that I figured it’s about time to add some search functionality to it. While there are certainly prefab solutions out there, this task is simple enough and fairly instructive. I had a few requirements, though: 
 
 The search needs to be fast, useful, and aesthetically pleasing. 
 Search in the browser. Standing up a…
This is part 2 of a paper I wrote for Ken Forbus ’ Qualitative Reasoning course, adapted for this blog. You can find a printable version of the paper here and part 1 here . 
 In the previous post I discussed the history of chess engines and why they don’t “think” like we think. Trading interpretability for computation cycles ultimately led to the engines we have today,…
This is part 1 of a paper I wrote for Ken Forbus ’ Qualitative Reasoning course, adapted for this blog. You can find a printable version of the paper here and part 2 here . 
 Computers that play chess, otherwise known as chess engines, have existed since at least the late 1940s . Because the game was said to require the perfect combination of planning, strategy, psychology, and…

 
 
 TLDR: Using opponent modeling we can win 2x faster than Stockfish by playing high risk, high reward moves that Stockfish will avoid.

 
 
 Batch Norm 
 Normalizes the mean and variance for all instances of a feature in a batch. Commonly used in older architectures like CNNs by smoothing out the loss landscape and allowing for higher learning rates and faster convergence. Example implementation:
TLDR: Alphabet chess is a chess variant that allows handicapping by mixing in a bit of poker into the beginning of the game. Moves must be played according to a secret word at the beginning of the game. 
 Chess has been played in different forms since the seventh century, and in its modern form since the nineteenth century. Opening theory, i.e. the study of the best moves to begin the game…

			 Selected {subject, relation, object} tuples generated by COMET 
		 
 

 Paper link: https://arxiv.org/abs/1906.05317 
 This paper describes COMET, a method of generating knowledge bases automatically. Previous work largely focused on encyclopedic knowledge, which has well-defined relationships. This paper, however, focuses on commonsense knowledge.…
First, create a custom dataset class. 
 from torch.utils.data import Dataset, DataLoader

class CustomDataset(Dataset):
 def __init__(self, features, labels):

 assert len(features) == len(labels)
 self.features = features
 self.labels = labels

 def __len__(self):
 return len(self.features)

 def __getitem__(self, idx):
 return…
If you want to extract a tar archive 
 tar -xf archive.tar.gz
 
 If you want to compress a directory 
 tar -czvf archive.tar.gz /path/to/directory
 
 That’s all.
Defending Against Neural Fake News by Zellers et al. presents a model for controllable text generation called Grover. This model can be used to create highly believable computer-generated news articles. The authors present this paper as a method of detecting and preventing the spread of fake news. They claim their model is 92% accurate at detecting fake news stories, partially due to artifacts…
Here’s how to connect to a remote Jupyter notebook. 
 Create an ssh tunnel to your remote machine: 
 ssh -L 8080:localhost:8080 user@12.34.56.78

# or use a .pem file to connect to ec2
ssh -L 8080:localhost:8080 -i "aws.pem" ec2-user@ec2-12-34-56-78.compute-1.amazonaws.com
 
 Start Jupyter on that machine in headless mode: 
 jupyter notebook --no-browser…
In machine learning and statistics, marginalization simply means summing over a set of independent variables. For example, suppose an avid tennis player kept track of the number of days he played tennis over a period of time as well as the weather on that day: 
 
 
 
 
 
 weather 
 
 
 
 
 
 sunny 
 cloudy 
 rainy 
 totals 
 
…
Here’s how to connect your Google Colab notebook to your Drive directory: 
 from google.colab import drive
drive.mount('/content/gdrive')
 
 Follow the prompts from there. That is all.
There are quite a few BERT vs GPT-2 breakdowns online, mostly focusing on the architectural differences between the two models. However, I am more interested in the performance differences between the two models, specifically their predictive capabilities. This blog post outlines the results of my experiments. 
 The code used in this experiment can be found on my Github 
 BERT 
 The…
Let’s explore how GPT-2 tokenizes text. 
 What is tokenization? 
 It’s important to understand that GPT-2 doesn’t work with strings directly. Instead, it needs to tokenize the input string, which is essentially a process for converting the string into a list of numbers, or “tokens”. It is these tokens which are passed into the model during training or for…
TLDR: Attention masks allow us to send a batch into the transformer even when the examples in the batch have varying lengths. We do this by padding all sequences to the same length, then using the “attention_mask” tensor to identify which tokens are padding. 
 
			 Here we use a batch with three samples padded from the left since we want to predict the next token on…
Convolutional neural networks have had breakthrough success in image recognition, natural language processing, and even board games like Chess and Go. But what’s really going on during convolution? Well, I think the easiest way to explain is with an interactive demo. Feel free to play around with the parameters below to see for yourself! 
 
 
 
 
 
 
 
 
…
If you want to serve some HTML with python run 
 python -m http.server 8000
 
 Then navigate to http://localhost:8000 . 
 This is not meant for production environments but will get you around CORS restrictions that would come from simply opening a local file in your browser.
This article will show how to run a simple language model, KenLM. It’s not as powerful as transformer-based models like BERT or GPT-3, but depending on what you’re trying to accomplish it may be more than enough. This tutorial should take you about 15 minutes, including the time to run the scripts. 
 Let’s work backwards from where we’re trying to get to. When…

 
 Temperature is a parameter used in natural language processing models to increase or decrease the “confidence” a model has in its most likely response.

 
 
 TLDR: NLP metric ranging from 1 to infinity. Lower is better. 
 In natural language processing, perplexity is the most common metric used to measure the performance of a language model. To calculate perplexity, we use the following formula:
Assuming your bucket is publicly accessible, the url of your S3 bucket will be 
 http://[bucket-name].s3-website-[region].amazonaws.com
 
 For example for “mybucket” in “us-east-1” your url will be 
 http://mybucket.s3-website-us-east-1.amazonaws.com