Ramblings around Postgres, Tech and everything else in between...
Maintaining a home lab with a diverse fleet of servers is a constant balancing act, but the real challenge is finding a way to pool mismatched, leftover drives across separate machines without turning storage administration into a second full-time job. Over time, my server farm accumulated an assortment of mismatched disks—a fast NVMe SSD on one machine, a spinning mechanical HDD on another, and a…
Sometimes the most entertaining way to test a new orchestration tool isn't to run a standard benchmark, but to lock two state-of-the-art AI models in a virtual room and make them debate historical facts. This week, I got to do exactly that using an interesting new tool called polly within the omniagent ecosystem. The idea was simple: set up a relay where Claude and Gemini (specifically the…
This is an opening post (of a series) about my attempt at automatic B-tree index compaction integrated into autovacuum. Index bloat is one of the most common performance problems in production PostgreSQL deployments — and one of the most misunderstood. People assume that a fully vacuumed table has a healthy index - which isn't true. There is a significant gap between an index that has been cleaned…
In distributed database environments, balancing durability and performance is a constant tug-of-war. PostgreSQL’s synchronous_commit parameter sits at the heart of this, giving administrators a dial to choose exactly when a COMMIT returns success to the client. The idea of remote_receive was born from a simple question: does skipping the standby's disk write yield a measurable, real-world…
Cloud AI is incredibly convenient, but the costs scale rapidly once you start leaning on long-context workflows. I've spent the last year relying on Gemini Pro for everything from Postgres automation to complex scripting, but the honeymoon phase is officially over (for me). As my projects grew in scope, so did the token consumption, eventually hitting a brick wall that forced a rethink of my…
It's that time again. Postgres v18.4 release (along with 17.10 , 16.14 , 15.18 , 14.23 ) has dropped some serious hints in the git logs, and it's bringing a significant payload of CVE tagged patches. As a seasoned Postgres end-user and an erstwhile DBA, whenever I see a flurry of high-vulnerability security commits, I immediately start recommending that customers begin planning their patching…
Finally - an update to the AI workbench, and one hell of an update it is :) ! The home server farm has now been blessed with dedicated silicon, and it is an absolute beast: the NVIDIA RTX 5060 Ti 16GB . The old setup—which was painfully grinding through CPU inference at a mere 1 token/sec on 46GB of RAM—has officially been retired. After months of relying on remote APIs, lightning-fast local…
Understanding Google AI Credits: What You're Actually Paying For If you've subscribed to Google AI Pro [1] (or are eyeing an upgrade), you've probably noticed the word "credits" appearing everywhere — in your billing dashboard, in the Gemini app, and inside your code editor [5]. But what are they? How do they reset? And if your family is on the plan, who's burning through them? [3] I…
Technical Features Surfing niche streaming sites without inline film ratings is a recipe for endless tab-opening and "analysis paralysis." To scratch my own itch, I put together a small userscript called Masala Script to fix this exact problem. For context, Einthusan is a massive streaming directory for South Asian cinema. While it's an excellent digital archive, exploring its thousands of…
PostgreSQL 18 introduces native "Skip Scan" for multicolumn B-tree indexes, a major optimizer enhancement. However, a common misconception is that pre-v18 versions purely resort to sequential scans when the leading column isn't filtered. In reality, the cost-based planner has long been capable of leveraging such indexes when efficient. How it works under the hood: In pre-v18 versions, if an index…
Unless you've built a massive real-time notification system with thousands of distinct channels, it is easy to miss the quadratic performance bottleneck that Postgres used to have in its notification queue. A recent commit fixes that with a spectacular throughput improvement. The commit in question is 282b1cde9d , which landed recently and targets a future release (likely Postgres 19, though as…
If you've ever had to scrub significantly large tables—whether updating older records or deleting expired ones—you know the pain of trying to do it efficiently without locking the entire table. One common trick is to iterate over the ctid (the physical location of the tuple) to process the table in chunks. Until now, this was strictly a single-threaded affair. But with the latest commit to…
The hidden cost of knowing too much. That's one way to describe what happens when your data is skewed, Postgres statistics targets are set high, and the planner tries to estimate a join. For over 20 years, Postgres used a simple O(N^2) loop to compare (equi-join) Most Common Values (MCVs) during join estimation. It worked fine when statistics targets are small ( default_statistics_target defaults…
The Postgres planner just got a new superpower: X-ray vision for your black-box functions. For years, Postgres has been able to "inline" simple SQL functions. If you wrote CREATE FUNCTION ... LANGUAGE SQL AS 'SELECT ...' , the planner would often rip out the function call and paste the raw SQL directly into the main query. This was magic. It meant that WHERE clauses could be pushed down, indexes…
A recent commit to the PostgreSQL master branch brings a nice quality-of-life optimization for a very common SQL pattern - improving performance by up to 64% for SELECT COUNT(h) where h is a NOT NULL column. If you've ever wondered whether you should use COUNT(*) or COUNT(1) , or if you've been dutifully using COUNT(id) on a non-null column, this change is for you. Note: This feature is currently…
The PostgreSQL Buildfarm is a global network of machines that continuously test PostgreSQL across a wide range of operating systems, architectures, compilers, and branches. Over the past few years, I have created and maintained several buildfarm members, each with its own quirks and strengths. In this post, I’ll share a status-update working on the following animals: alligator, dodo, woodpecker,…
Curiosity often leads to the most interesting technical adventures. This time, I decided to explore something off the beaten path: running Debian GNU/Hurd inside a virtual machine on my Windows 11 host and compiling PostgreSQL from source. This post is part 1 of a multi-part series documenting the process, challenges, and discoveries along the way. Future parts will dive deeper into advanced…
After my positive first impressions with Pi-hole , I decided to take the next logical step: eliminating my dependence on external DNS resolvers entirely. While Quad9 served me well, there's something unsettling about routing all my DNS queries through a third party, no matter how trustworthy they appear. The solution? Unbound - a validating, recursive, caching DNS resolver that can operate…
(This is first of a 2 part series, where I explore pi-hole as a DNS caching solution-Here's Part 2 ) I've had a Raspberry Pi 4 sitting idle for months, and finally decided to put it to good use by setting up Pi-hole for network-wide DNS caching and ad blocking. After less than a day of operation, I'm genuinely impressed with both the installation process and the results. The Installation:…
Performance bottlenecks in Postgres logical replication or Change Data Capture (CDC) stream can be subtle, but one specific wait event, ReorderBufferWrite , often points directly at how your application interacts with the database during these processes. Let's unpack this wait-event and see how your application's workload patterns can trigger it. Core Concepts: Logical Decoding and WAL To…
FOR KEY SHARE optimization and the SLRU Trap Optimizing database concurrency is a constant balancing act. We often tweak locking strategies in PostgreSQL, aiming to allow more simultaneous operations without compromising data integrity. A common scenario involves shifting from stricter row-level locks to more lenient ones. But sometimes, what seems like a straightforward optimization can lead to…
How much storage does an empty table in Postgres take? This is a post about Postgres tables that store ... well basically ... Nothing . The idea for this post came from this tweet that hinted that an empty table on most databases today takes 16Kb of storage. Now admittedly Franck was probably reminiscing the good-old days so this is probably quite out of context, but it did get me thinking, and…
In continuation of the recent posts... Finally got a on-premise chat-bot running! Once downloaded, the linux box is able to spin up / down the interface in a second. (myvenv) ai@dell:~/proj/ollama$ time ollama run mistral >>> /bye real 0m1.019s user 0m0.017s sys 0m0.009s That, on a measly ~$70 Marketplace i5/8GB machine is appreciable (given what all I had read about the NVidia RTX 4090s etc.).…
Trying out Ollama - Your 10 year old box would do too. TLDR Yes, you CAN install an AI engine locally No, you DON'T need to spend thousands of dollars to get started! Agreed, that your ai engine wouldn't be snappy, it's still great to get started. Server You'd realise that any machine should get you going. I had recently bought a second-hand desktop box (Dell OptiPlex 3020) from FB Marketplace and…
I am just starting my AI journey, and trying to get Ollama to work on my linux box, was an interesting non-AI experience. I noticed, that everytime I was trying out something new, my linux box got reliably stuck every single time I pulled a new model. htop helped point out, that each time I did a ollama pull or ollama run , it spun up a ton of threads. Often things got so bad, that the system…