Batch Size Causes BertModel Forward Pass Divergence Between torch==2.4.1 and torch==2.5.0 for colbert-ai.

Background I’ve recently been documenting how PyTorch version changes impact stanford-futuredata/ColBERT ( colbert-ai on PyPI) intermediate and final index artifacts. The index artifact I’ll focus on in this blog post is the very important local_sample_embs tensor. This is the sample of token embeddings used to calculate the centroids, which are later on used during search. Instead of loading and…