Half-baked thoughts on whatever I feel like writing about
Crash Course: Postgres data storage topology Every row in a table in Postgres is stored as a tuple. Tuples are stored in an 8 KB container referred to as a page. A page is a range of space within an actual file on disk, located within the file via an offset. These files exist within a directory for the specific database that they make up, and are up to 1 GB in size.
One thing I’ve noticed as I’ve been using AI more and more to build projects and features: there’s often just a lot of code. And not only source code, but tests, specs, docs, diagrams, etc. It can make reviewing PRs…intimidating. A couple examples.
TL;DR: I theorize that coding agent errors compound over time leading to increasingly worse outcomes as a session continues. I built Blackbird based on this theory, which restarts each task in a plan with a fresh context window, with a new task’s instructions as its starting point. This minimizes deviation from intention leading to better outcomes.
TL;DR – I built blackbird, a CLI tool to break your work down into manageable chunks that Claude Code or Codex can follow easily. Then I used that to build key-keeper, a browser extension to securely share API keys with websites.
…and I guess I got a little distracted, because I built a POC document ingestion and vector search app instead.
Text (or other content, like images, with the right model) can be embedded returning a vector Most LLM providers have embeddings models/endpoints that can take input and return a vector Vectors are represented as arrays of numbers (generally 32-bit floats) They have a magnitude This is the length of the vector. Length is NOT the number of components in the vector – that’s dimension, and is usually…
In case you didn’t know, there’s a fantastic Data Engineering community on Discord (invite here). I’ve been a member of this server since the very start, and I’ve seen it grow substantially over the years. If you read that last sentence again, you’ll notice that it’s really vague. When was the start? Was I really there at the start, or did I just join (waves hands) a while ago? How much has the…