Posts

ls ./2026/

2026-08-13~6 min

Explaining FoundationDB's Architecture Through Compartmentalization Compartmentalization explains FoundationDB's architecture: find the responsibilities that got coupled together, split them, and scale only the parts that can scale.

2026-07-30~9 min

Why Apache Avro Needs Two Schemas Avro payloads cannot be decoded without the writer schema, and that two-schema model is what makes rolling upgrades and schema evolution work.

2026-06-09~6 min

Designing Fakes That Prove Correctness You fake the handful of methods your code actually calls, prove the fake stays honest, and then make it fail the way production does.

2026-05-05~5 min

BugBash 2026, or how the correctness decade has started A retrospective on BugBash 2026, and why software engineering looks brighter than ever.

2026-03-26~8 min

Why Fakes Beat Mocks and Testcontainers Trait-based fakes test partial failures that mocks and Testcontainers fundamentally cannot simulate. They are also the on-ramp to simulation-driven development.

2026-03-23~13 min

Diving Into Coverage-Guided Fuzzing How a 600-line Rust fuzzer discovers a 3-byte crash sequence in 9,000 iterations instead of 16 million

2026-02-25~8 min

Building Index-Backed Query Plans in DataFusion What I learned about manually constructing physical query plans for secondary index queries, and the library that came out of it.

2026-02-05~8 min

Simulating Leader Election on top of FoundationDB Can LLM-generated code be trusted for distributed systems? Only if you have the right feedback loop.

2026-01-30~11 min

FoundationDB's Transaction Model for Layer Engineers How optimistic concurrency control works in FoundationDB, and how to design data structures that stop fighting it

2026-01-15~9 min

What I Tell Colleagues About Using LLMs for Engineering LLMs amplify expertise, they don't replace it. Here's what works: planning, context, feedback loops, and building systems that let AI discover bugs.

ls ./2025/

2025-12-26~11 min

2025: A Year in Review Reflections on returning to engineering, discovering simulation as a superpower, learning to work with LLMs, and the compounding value of years of distributed systems work.

2025-12-19~6 min

Specs Are Back, But We're Missing the Tools LLMs revived specifications. Model checkers like Fizzbee work for algorithms. But for APIs and business logic? The tooling doesn't exist yet.

2025-12-09~13 min

Designing Rust FDB Workloads That Actually Find Bugs Patterns and principles for writing Rust simulation workloads that catch bugs before production does.

2025-11-12~9 min

Diving into Kubernetes' Watch Cache Understanding how Kubernetes apiserver caches etcd, the 3-second timeout, and K8s 1.34 consistent read feature

2025-10-30~22 min

Diving into FoundationDB's Simulation Framework How FoundationDB achieves legendary reliability through deterministic simulation, interface swapping, and one trillion CPU-hours of testing

2025-10-02~3 min

From Arc to Box: One Deref Bound to Rule Them All How Rust's Deref trait solved a complex API design problem in FoundationDB-rs without traits or enums

2025-09-24~14 min

A Practical Guide to Application Metrics: Where to Put Your Instrumentation A comprehensive guide on where and how to instrument metrics in your applications, covering everything from API endpoints to background jobs

2025-09-08~6 min

Testing: prevention vs discovery Most testing prevents old bugs from returning. But what if we built systems where LLMs could actively discover new bugs instead?

2025-08-18~4 min

Shipped vs. Operated, or How Many Bash Scripts Does It Take?The difference between shipped and operated software is the difference between something you can run and forget, and something that demands ongoing, hands-on care. Choosing the former protects your team’s focus and sanity.

2025-08-11~3 min

Two Podcast Episodes on Topics Developers Rarely Talk About Two podcast episodes—one from Oxide and one from Antithesis—on debugging at the limits and building correctness into systems from day one.

2025-07-02~8 min

Three Years of Nix and NixOS: The Good, the Bad, and the Ugly A review of Nix/NixOS after using it on all my machines for three years. I'll cover what works, what doesn't, and why it's the first OS that's stuck with me.

2025-06-04~6 min

Thank You, DataFusion: Queries in Rust, Without the Pain Why Apache DataFusion, a composable Rust-native SQL engine rather than a rigid framework, finally let me plug real queries into our FoundationDB-based system after years of watching from the sidelines.

2025-06-03~7 min

Bypassing FoundationDB's Transaction Limits with Record Layer Continuations A technical deep-dive into FoundationDB Record Layer continuations, explaining how they enable long-running operations by segmenting work across multiple FDB transactions, effectively bypassing the inherent 5-second and 10MB limits.

2025-05-18~6 min

Unlocking Tokio's Hidden Gems: Determinism, Paused Time, and Local Execution Discover lesser-known Tokio features like current-thread runtimes for !Send futures, seeded runtimes for deterministic tests, and paused time for precise temporal control in your Rust applications.

2025-04-18~9 min

What if we embraced simulation-driven development?How deterministic simulation testing can help us build more reliable distributed systems and bridge the gap between development and production environments.

2025-04-11~6 min

So, You Want to Learn More About Deterministic Simulation Testing?A curated collection of resources about deterministic simulation testing for distributed systems.

2025-03-27~2 min

Key design tip: reverse number scanning in ordered key-value stores A key design trick for ordered key-value stores like HBase, FoundationDB, and RocksDB: reverse a number by subtracting it from its max value so a forward scan finds the highest one first.

2025-03-07~3 min

Debugging FoundationDB's Data Distributor A deep dive into the internals of FoundationDB's Data Distributor and how it manages shard placement and team priorities.

2025-02-11~4 min

Ensuring Safety in FoundationDB's Rust Crate Strategies and techniques for enhancing safety in the FoundationDB Rust crate

2025-01-15~2 min

Back in engineering!I'm moving back in core engineering

ls ./2024/

2024-04-22~3 min

Redwood’s memory tuning in FoundationDB Learn how to tune FoundationDB for Redwood storage Engine

2024-03-12~4 min

True idempotent transactions with FoundationDB 7.3 Learn how to avoid FDB's biggest caveats by using a new feature called automatic idempotency in FoundationDB

2024-01-22~4 min

The unseen treasures of Infrastructure Engineering: Academic Papers Are you not using academic papers in your favorite RSS app? You should!

ls ./2022/

2022-01-17~11 min

Best resources to learn about data and distributed systems Learning distributed systems is tough. You need to go through a lot of academic papers, concepts, code review, before being able to have a global pictures. Thankfully, there is a lot of resources out there that can help you. Here's the best resources I used to learn distributed systems

ls ./2021/

2021-02-21~5 min

Crafting row keys in FoundationDB An overview of all the helpers to generate row keys with FoundationDB

2021-01-11~9 min

Notes about ETCD List of ressources gleaned about ETCD

ls ./2020/

2020-09-30~9 min

10 years of programming and counting 🚀A retrospective of the last 10 years

2020-09-23~3 min

Announcing Record-Store, a new (experimental) place for your data Opensourcing Record-Store

2020-09-18~4 min

Diving into ETCD's linearizable reads A code-level dive into how etcd achieves linearizable reads with the ReadIndex protocol on top of Raft, including how it enables safe follower reads.

2020-07-30~10 min

Notes about Raft's paper List of ressources gleaned about Raft

2020-03-24~11 min

Announcing Kafka-on-Pulsar: bring native Kafka protocol support to Apache Pulsar We are excited to announce that StreamNative and OVHcloud are open-sourcing “Kafka on Pulsar” (KoP). KoP brings the native Apache Kafka protocol support to Apache Pulsar by introducing a Kafka protocol handler on Pulsar brokers

2020-02-14~7 min

Contributing to Apache HBase: custom data balancing In today’s blogpost, we’re going to take a look at our upstream contribution to Apache HBase’s stochastic load balancer, based on our experience of running HBase clusters to support OVHcloud’s monitoring.

2020-01-30~14 min

Notes about FoundationDB List of ressources gleaned about FoundationDB

ls ./2019/

2019-12-08~6 min

Diving into Kafka's Protocol A walkthrough of Kafka's wire protocol, from the API_VERSIONS handshake through producing and the consumer group coordination protocol, with a sequence diagram for each exchange.

2019-11-17~4 min

Diving into Hbase's MemStore A code-level dive into HBase's MemStore, from the ConcurrentSkipListMap backing HBase 1.x to the in-memory compactions Accordion introduced in HBase 2.0.

2019-08-04~8 min

What can be gleaned about GFS successor codenamed Colossus?Piecing together, from scattered public talks and papers, how Google evolved GFS into Colossus, and why a Kubernetes-native distributed file system is still missing.

2019-05-27~3 min

Playing with TTL in HBase A hands-on walkthrough of HBase's TTL feature, how cell timestamps and compaction interact to expire data, and how to control it by setting your own version timestamp.

2019-02-03~6 min

Handling OVH's alerts with Apache Flink How OVH built Beacon, an Apache Flink pipeline handling alert deduplication, escalation, and notification on top of its Metrics Data Platform, and what running Flink in production taught us.

2019-02-03~4 min

What are ACID transactions?A walkthrough of the ACID properties of database transactions, from the 1983 Haerder and Reuter paper that coined them to isolation levels and PostgreSQL's MVCC implementation.

2019-01-27~4 min

Hbase Data Model An overview of HBase's data model, row keys, column families and qualifiers, and why key design determines whether you get good performance or a hotspotted region.

ls ./2017/

2017-10-11~8 min

Introducing HelloExoWorld: The quest to discover exoplanets with Warp10 and Tensorflow A personal project importing NASA's Kepler and K2 telescope data into Warp10 and building a TensorFlow model to detect exoplanet transits from the dips in starlight.

2017-10-08~8 min

Engage maximum warp speed in time series analysis with WarpScript An introduction to WarpScript, the stack-oriented language behind the Warp10 time series platform we ran at OVH's Metrics Data Platform to store and analyze sensor data.

ls ./2016/

2016-05-13~7 min

Event-driven architecture 101 An introduction to event-driven architecture: queue messaging systems applying the Unix pipe philosophy to microservices, and the simplicity, modularity, and scalability it buys you.

2016-01-04~7 min

Let’s talk about containers A look at what containers actually are, from Docker's write-once-run-everywhere hype to why a statically compiled Unix process, not a small VM, is the right mental model.