Vector Search benchmark comparing USearch HNSW across precisions (f32, f16, bf16, i8) vs Lucene in-memory HNSW (f32, i8) baselines, leveraging Apache Spark for distributed indexing and search.

Quick Start

gradle build --warning-mode all # ensure all dependencies are resolved
gradle run --args="-h" # print supported arguments & available options
gradle run --args="unum-wiki-1m" # index & search `f32` vectors across all cores
gradle run --args="yandex-deep-10m --max-vectors 100000" # limit vectors for testing
gradle run --args="msft-spacev-100m --precision f32,i8" # test specific precisions

When running on larger machines, consider overriding JVM settings based on your hardware. Auto-detect optimal settings for your machine:

# For 750+ GB machines with custom heap size
JAVA_OPTS="-Xmx512g -Xms64g -XX:ParallelGCThreads=$(nproc)" gradle run --args="msft-spacev-100m"
# For development with limited resources
JAVA_OPTS="-Xmx8g -Xms2g" gradle run --args="unum-wiki-1m --max-vectors 100000"

For small test runs comparing the impact of multi-threading you may run:

JAVA_OPTS="-Xms2g -Xmx8g" gradle run --args="unum-wiki-1m --max-vectors 10000 --queries 10000 --batch-size 100 --threads 1"
JAVA_OPTS="-Xms2g -Xmx8g" gradle run --args="unum-wiki-1m --max-vectors 10000 --queries 10000 --batch-size 100 --threads 8"

To test Spark distributed execution locally, you may run:

JAVA_OPTS="-Xmx8g -Xms2g" gradle run --args="unum-wiki-1m --shards 2 --max-vectors 100000"
JAVA_OPTS="-Xmx8g -Xms2g" gradle run --args="unum-wiki-1m --shards 32 --batch-size 32768 --engines lucene"
JAVA_OPTS="-Xmx512g -Xms64g -XX:ParallelGCThreads=$(nproc)" gradle run --args="msft-spacev-100m --shards 32 --batch-size 32768 --engines lucene"

Benchmarks USearch (f32, f16, bf16, i8) against Lucene (f32) on Wiki dataset locally, producing clean output like, the following results obtained for the 100M msft-spacev-100m subset of Microsoft SpaceV:

🚀 PERFORMANCE METRICS
┌──────────────┬──────────────┬──────────────┬──────────────┬─────────────┐
│ Engine       │ Precision    │ IPS          │ QPS          │ Memory      │
├──────────────┼──────────────┼──────────────┼──────────────┼─────────────┤
│ Lucene       │ F32          │ 20,665       │ 864          │ 49.0 GB     │
│ Lucene       │ I8           │ 26,408       │ 1,218        │ 20.8 GB     │
│ USearch      │ F32          │ 96,119       │ 126,582      │ 96.0 GB     │
│ USearch      │ BF16         │ 113,090      │ 129,870      │ 64.0 GB     │
│ USearch      │ F16          │ 124,297      │ 144,928      │ 64.0 GB     │
│ USearch      │ I8           │ 137,329      │ 166,667      │ 48.0 GB     │
└──────────────┴──────────────┴──────────────┴──────────────┴─────────────┘
🎯 RECALL & NDCG METRICS
┌─────────────┬─────────────┬─────────────┬─────────────┬─────────────┬─────────────┐
│ Engine      │ Precision   │ Recall@10   │ NDCG@10     │ Recall@100  │ NDCG@100    │
├─────────────┼─────────────┼─────────────┼─────────────┼─────────────┼─────────────┤
│ Lucene      │ F32         │ 90.00%      │ 88.17%      │ 94.70%      │ 93.34%      │
│ Lucene      │ I8          │ 90.00%      │ 87.75%      │ 94.97%      │ 93.30%      │
│ USearch     │ F32         │ 90.03%      │ 88.63%      │ 95.76%      │ 95.21%      │
│ USearch     │ BF16        │ 90.12%      │ 88.46%      │ 95.62%      │ 95.21%      │
│ USearch     │ F16         │ 90.27%      │ 88.59%      │ 95.78%      │ 95.31%      │
│ USearch     │ I8          │ 90.34%      │ 88.66%      │ 95.81%      │ 95.31%      │
└─────────────┴─────────────┴─────────────┴─────────────┴─────────────┴─────────────┘

Hardware: AWS m7i.metal-48xl instances (92 physical cores, 192 threads, 2 sockets). OS: Linux 6.8.0-1024-aws (Ubuntu 22.04.5 LTS). CPU: Intel Xeon Platinum 8488C @ 2.4GHz. Memory: 768 GB RAM total. Java: OpenJDK 21.0.5 with Java Vector API (--add-modules=jdk.incubator.vector). JVM: 128GB heap (-Xmx128g) with ZGC garbage collector (-XX:+UseZGC) for sub-10ms pauses. Library versions: USearch v2.20.8, Lucene v9.12.0.

IPS stands for Insertions Per Second, and QPS is Queries Per Second. Recall@K is computed as a fraction of search queries, where the known "ground-truth" Top-1 result appeared among the Top-K approximate results. NDCG@K stands for Normalized Discounted Cumulative Gain at K, which measures the effectiveness of the search results by considering the position of the relevant documents.

Datasets

The BigANN benchmark is a good starting point if you are searching for extensive collections of high-dimensional vectors. Still, it rarely considers datasets with more than tens of millions of entries. In the era of 100+ core CPUs and Petabyte-scale storage in 2U servers, larger datasets are required.

Dataset	Codename	DType	NDim	Metric	Size