Building blobd: single-machine object store with sub-millisecond reads and 15 GB/s uploads
Deep dive into blobd, an object store designed for constant low latency reads using async Rust, io_uring and atomic writes
Wilson Lin
Blog for Wilson Lin
Latest posts
Building a high recall vector database serving 1 billion embeddings from a single machine
Introduction to CoreNN, an open source vector database that scales to 1 billion vectors on a single machine with high recall and throughput.
From 3 TB RAM to 96 GB: superseding billion vector HNSW with 40x cheaper DiskANN
An analysis of DiskANN, a newer graph-based ANN index built for cheaper disk while still retaining high recall and throughput.
From Kevin Bacon to HNSW: the intuition behind semantic search and vector databases
An explainer to how modern fast and accurate vector searching works, with interactive demos.
Building a web search engine from scratch in two months with 3 billion neural embeddings
End-to-end deep dive of the project, spanning a large GPU cluster, distributed RocksDB, and terabytes of sharded HNSW.
Exploring Hacker News by mapping and analyzing 40 million posts and comments
Accessible end-to-end deep dive of crawling, embedding, projecting, and semantic searching all of Hacker News for my Hackerverse project.