Data infrastructure engineering, DuckDB extensions, and building tools that scale.
Nine months of telemetry from Query.Farm's DuckDB extensions — 110 million loads — show each release landing slower than the one before it. Fourteen days in, v1.5.0 reached 33.7% of the population I can see; v1.5.5 reached 12.1%. The decline survives excluding CI traffic, excluding the v1.4 LTS line, and restricting to patch releases only.
The companion to my post on DuckDB upgrade speed. Once you strip out the CI traffic that is 70% of my telemetry, the extension catalog isn't the monopoly the load counts imply or the flat field the raw config-days imply — and a third of real installations turn out to be on a desktop.
You have a large Parquet file and an API that has to return its contents, but responses have a size ceiling, so the caller pages through it. LIMIT/OFFSET is the obvious way to write that and it is the wrong one. I measured file_row_number against it: 2.53x faster on a file with 163 row groups. Speed is the boring half of the answer. OFFSET will also hand back the wrong rows without telling you.
When DuckDB ships a new version, every community extension has to be rebuilt against it before anyone can install one. I measured how much of the catalog is actually there when a release lands, how long extension updates take to merge, and why the same builds run 2.5x faster on different CI. The answer to all three is distribution work.
A DuckDB extension that compiles for WebAssembly has only proven that it compiles. Whether it loads, and whether it actually runs, are separate questions. I built a Node harness to ask them across 124 community extensions. Here's what it found and the fixes that came out of it.
Quack — the new client/server protocol extension for DuckDB — was announced today. It's been built since November 2025, almost exactly as long as I've been building VGI. Here's what's interesting, what's surprising, and where the two projects converge and diverge.
A working status page for the Arrow-related issues and PRs I've filed against duckdb/duckdb — the UNION saga, schema/data disagreements in nested appenders, type fidelity through round-trips, and the bigger question of how defensive the C Data Interface should be.
A detailed comparison of GeoSilo's delta-integer encoding against GeoArrow's columnar float64 layout with Parquet BYTE_STREAM_SPLIT, covering compression ratios, precision tradeoffs, and when each approach wins.
I needed a self-contained binary that downloads a program over HTTPS and exec's it — small enough to run in a FROM scratch container with nothing else. Go and Rust couldn't get small enough. Zig could.
A survey of the TIME data type across 14 databases, comparing supported ranges, maximum values, and whether the special value 24:00:00 is accepted.
I open-sourced the telemetry client I use across Query.Farm's DuckDB extensions. It's two files, one function call, and it only collects platform and version info.
I built an RPC framework for Python that uses Apache Arrow IPC as the wire format and Python Protocol classes as the interface definition. No .proto files, no codegen — just type annotations.
The Inflector extension for DuckDB now supports configurable acronyms, so case conversions preserve terms like HTML, API, and URL as fully uppercase — configured through a native DuckDB setting.
Fair Weather scores every hour of the next 48 hours on a 0–100 scale based on temperature, humidity, wind, UV, precipitation, air quality, and daylight —...
If you've ever wanted to extend DuckDB with custom functions, table functions, or new capabilities — this is your chance to learn how. In this free 2.5-hour...
We’re excited to introduce our newest DuckDB community extension: ADBC Scanner — a fast, flexible way to connect DuckDB to external databases using Arrow...
Since I live on the U.S. East Coast, I’d be happy to hold a workshop here in the USA. If you're interested, comment to help decide a location and date. If...
I'm excited to share today’s release of the crypto extension, bringing improvements and new capabilities: Updated crypto_hash API – This function now...
I’ll be teaching a free, 2-hour workshop about DuckDB extension development the morning of DuckDB Developer Day. 📅 Date: January 30, 2026 📍 Location:...
For everyone building on the Airport extension for DuckDB https://airport.query.farm/ - there is now support for Go from Vladimir Gribanov. airport-go — a Go...
🆕 Created a Kafka topic with 8 partitions on my M3 MacBook Air (one per core) ⚡ Inserted 50M messages from DuckDB (47 bytes each) in 11.62s (4.3 million...
Transform string cases and inflect column names directly in SQL—no more wrestling with naming conventions across your data pipeline. What does it do? ✅...
I’m trying to get our DuckDB extensions fully indexed in Google so users can more easily discover what we’re building — especially all the tools that make...
Transform and query JSON data with elegant expressions—all within SQL. No more fighting with nested JSON or writing complex extraction logic. What does it...
I’ve been working on integrating DuckDB with Apache Kafka, and now you can read from and write to Kafka topics using the SQL you already know. In the demo...
I am happy to introduce the MiniJinja Extension for DuckDB, brought to you by Query.Farm! Now you can bring the power of the industry-standard MiniJinja...
#DuckDB Developer Meeting 2026, hosted by DuckDB Labs will be on January 30, 2026, in #Amsterdam. This specialized event is designed for developers working...
We've just released a powerful new community extension that brings JSON schema validation and data enrichment directly to your SQL queries. No more context...
Now you can bring powerful template rendering directly into your SQL queries. Generate dynamic text, HTML, reports, and configuration files without leaving...
Here are some of the highlights with my commentary - 🧩 Extensions ◆ Updated C API + C extension API - this means extensions will have a stable API build...
I’ll be presenting: “Building on Flight: Real-World Lessons from the DuckDB Airport Extension” I'm excited to reconnect with the Arrow community. If you’re...
👉 Just set your dialect with the dialectic_flavor PRAGMA. This means you can attempt to run SQL built for other systems without rewriting it for DuckDB —...
The team at yeet.cx has built an observability platform that makes clever use of DuckDB Wasm together with our Textplot DuckDB extension. The result?...
We talked about why DuckDB’s in-process, C++-based architecture is changing how people think about data systems, and we dug into the ecosystem that’s...
But when I'm dealing with optimization, I'm always left with that feeling, did I find a local or global minima? The only way to know for sure is to gather...
I am excited to announce Query.Farm is expanding! We've integrated 12 high-quality DuckDB community extensions from the QuackScience project, strengthening...
If you’ve used DuckDB with Pandas, you know how easy it is to query DataFrames directly. But what if your data is too big to fit in memory, or you want to...
How to use Python generators and Apache Arrow's RecordBatchReader to stream large datasets into DuckDB without loading everything into memory at once.
Create stunning bar charts, density plots, and sparklines without leaving DuckDB. Perfect for quick data exploration, monitoring dashboards, CLI analytics,...
Excited to share the stochastic extension - bringing comprehensive statistical distributions directly to DuckDB SQL! ✨ What's included: 15+ probability...
The RapidFuzz extension is now available as a DuckDB Community Extension! 🦆✨ You can now perform high-performance fuzzy string matching directly in your...
Just launched a DuckDB Community Extension that brings high-performance probabilistic approximate membership filters to SQL! What it does: ✅ Fast set...
Whether you’re working on database indexing, data partitioning, caching, or implementing Bloom filters, hashfuncs offers fast and reliable hashing optimized...
🚀 New DuckDB Marisa Extension by Query.Farm Add fast, space-efficient 🧠 MARISA trie support to DuckDB for string lookups, autocomplete 🔍, prefix...
🔍 We tested scalar functions, remote table creation, and selection queries using both BIGINT and 32-byte strings — scaling up to 100 million rows. The...
I’ve been blown away by the enthusiasm and curiosity around Airport (https://airport.query.farm), our DuckDB extension that brings the power of Arrow Flight...
🛫 Airport lets DuckDB use Arrow Flight, unlocking new ways to query remote data seamlessly and quickly. ✨ Bonus: the demo includes built-in UTF-8 tests,...
The Tributary extension by Query.Farm — enables real-time querying and analysis of Kafka streams directly in DuckDB. With Tributary, you can consume...
Extension availability across versions and platforms isn’t always obvious—and the docs don’t always have the answers. So I built a DuckDB Extension...
I'm excited to launch Radio, a new DuckDB extension that enables real-time interaction with event-driven systems like WebSocket and Redis Pub/Sub. With...