TLDR (but stay for the animations!): Lance is a successor to Iceberg / Delta Lake, more optimized for random reads, and supports adding ad-hoc columns without needing to copy all the data. Some big things happened in the big data over object storage world in 2025: Iceberg V3 spec got released and added cool stuff like VARIANT . turbopuffer announced a vector search over object storages (similar to…
In his book Designing Data-Intensive Applications , Martin Kleppmann suggests that all data applications follow a similar pattern. Their goal is to read data, run some transformation on it, and store the result somewhere, all to have a faster way to read that data later. We see this pattern everywhere: A RDBMS (e.g. Postgres, MySQL) receives rows and computes a B-Tree. A log search engine (e.g.…
CPUs Speed Limit Runtime Deadline const loadScript=r=>new Promise((e,t)=>{var n=document.createElement("script");n.src=r,n.onload=e,n.onerror=t,document.head.appendChild(n)}),waitForDocumentLoad=()=>new Promise(e=>{document.addEventListener("DOMContentLoaded",e)});function getSize(){return window.innerWidth…
About a year ago, I tried thinking which database I should choose for my next project, and came to the realization that I don't really know the differences of databases enough. I went to different database websites and saw mostly marketing and words I don't understand. This is when I decided to read the excellent books Database Internals by Alex Petrov and Designing Data-Intensive Applications by…