Clément Sauvage

What's actually behind our IP addresses ?

Everything you never questioned about IP addresses # Most people know their device has an IP address. Few know what’s actually happening behind it.

Rolling Update / Canary Deployment / Blue-Green Deployment

Rolling Update: You replace instances incrementally, batch by batch, until all are running the new version. No extra infrastructure is needed and there’s no downtime. The downside is that if something goes wrong mid-way, rollback is slow and messy. Canary Deployment: You route a small percentage of real traffic to the new version, watch your metrics, and gradually increase that percentage…

How OR clauses were silently killing our query performance

Recently at work (on my last day), I dealt with a SQL query that got pretty complex because of new features making the business logic increasingly involved. The query ended up quite lengthy, but it was still the best way to respond to the new requirements. We were iterating over rows in a PostgreSQL database by batches across 20 parallel goroutines, until all rows had been processed.

TruthBell

Lock / Mutex / Semaphore

Lock : A general term for any synchronization mechanism that prevents concurrent access to a shared resource. When a thread acquires a lock, others must wait until it’s released. Mutex : A lock that only allows one thread in at a time, with strict ownership. Only the thread that locked it can unlock it. If another thread tries to acquire an already locked mutex, it blocks until the owner…

Where do the bits goes when the power die ?

I just read a reddit post on the r/datahoarder subreddit, from someone wondering what’s the best way to cold store 1–2TB of data for a long time? The top comment stated the following:

Module / Package / Library / SDK

Module : A single file containing code. In Python, it’s a .py file. In JavaScript, it’s typically a .js file. The smallest unit of code organization you can import. Package : A collection of modules organized in a directory structure. Usually has an init .py (Python) or package.json (JavaScript) file that marks it as a package. Think of it as a folder that groups related code together.…

About

👋 Hey! I am Clément Sauvage a Senior Software Engineer currently working on the Search & Infrastructure at Front . Previously, I worked for 5 years at Leboncoin , France’s biggest C2C Marketplace (+30M users). I wrote and scaled dozens of services that serve up to 20k+ RPS in Go , communicate through Kafka Queues, gRPC , or simple REST APIs. We used a lot of different storage systems, such…

Blogging is useless, so let's setup your own blog

Blogging probably won’t give you anything but scraping bot traffic. I still find it quite fun to have my own piece of the internet I can tweak on my own, make anything with, and also share information with myself in the future, but in a public way.

Search Playground

Scalar Quantization - Making Vector Search Lean and Fast

If you’ve worked with semantic search or dense retrieval systems, you know the problem: embeddings are expensive. Not expensive to compute (well, that too), but especially expensive to store and search through.

Write-Through / Write-Around / Write-Back

Common patterns to write to cache includes: Write-Through # Data flow : App → Cache → DB (synchronously) App writes to cache, then immediately writes to DB (both sync)

Real-time Communication Patterns: Websockets vs Server-Sent Events vs Long Polling vs Short Polling vs WebRTC

Real-time Communication Patterns # WebSockets How it works : Full-duplex TCP connection, both client/server can send data anytime Key differentiator : True bidirectional real-time communication Ideal for : Chat apps, live gaming, collaborative editing, live trading Trade-off : More complex, requires connection management Server-Sent Events (SSE)

Favorites

Here is a non-exhaustive list of my favorite games, movies, podcasts, etc… Have fun discovering something new! Video Games # A Way Out Bioshock Boomerang Fu Cuphead Dishonored Dusk Goldeneye 007 (n64) Gris It Takes Two Killing Floor Franchise Life is Strange series Neon White Rime Tetris Towerfall The Elder Scrolls V: Skyrim The Last of Us The Talos Principle Uncharted Board Games # Chess…

Kafe - A TUI tool to observe and manage Kafka clusters

Designing Data Intensive Applications - Partitioning

We previously discussed replicating our data, which has multiple benefits, one being allowing a higher read throughput. But to allow even higher read and write throughput, we need to partition our data, that is to split it into smaller parts and distribute these parts across multiple nodes.

Learning chess and the parallels with software engineering

I’ve recently became a bit obsessed with chess, and although I am very much an amateur, I am taking the learning pretty seriously, as I think progressing is a huge part of the fun that comes with this game.

Designing Data Intensive Applications - Replication (part 4) - Leaderless Replication

Leaderless Replication # All the previously discussed ways of replicating the data were based on the presence of a leader, which would be the node the clients sends data to, and which is then responsible for replicating the data to the other replicas.

Denormalization - When Breaking the Rules Makes Sense

Remember all those database normalization rules we learned in school? The ones about avoiding redundancy at all costs and keeping our data neat and tidy? Sometimes we need to throw those rules out the window. Enter denormalization: the art of strategically making our database “messier” for the sake of performance.

CPU Architecture basics every developer should know

Usually, as software engineer, we spend a lot of time working with high-level concepts: frameworks, design patterns, clean code… But it can allow us to miss or forget what’s happening under the hood of our machines. Most precisely, about the CPUs. Which is a shame, considering how it can sometimes help us write better and more effient code.

[Archived] Left or right wing

How is Spotify search so fast ?

I find search issues quite fascinating. We all know how fast computers can be, but we also know they have limits. So how can some service provide such precise and fast search features ?

The Information Retrieval & Search Glossary

Information retrieval is a field of its own in the world of IT. Especially when considering how complex it can get, and the depth of knowledge that can be required to operate search systems.

Traces / Spans / Events / Logs / Metrics

These terms are recurring when working on observability and monitoring, which are two related but distinct concepts, in the context of managing and maintaining software systems. They both aim to ensure that systems are running smoothly and to detect and diagnose issues when they arise, but they approach this goal from different perspectives.

Designing Data Intensive Applications - Replication (part 3) - Multi-Leader Replication

Multi-Leader replication # Single leader replication present the downside that if we cannot connect to the leader, we cannot write to the database.

Python by example

Designing Data Intensive Applications - Replication (part 2) - Problems with replication lag

Problems with replication lag # As a reminder, the main reasons to want replication are: To tolerate nodes failures (high availability) For better scalability (Processing more requests than a single node can handle) For lower latency (placing replicas geographically closer to users) In a leader-based replication setting, as all writes goes to a single node, and read-only queries can go to any…

Designing Data Intensive Applications - Replication (part 1) - Leaders and Followers

Chapter 5: Replication # Replication simply is the act of keeping a copy of the same data on multiple machines, usually connected by a netwok. This practice has many benefits, such as:

SLA / SLO / SLI

SLO: Service Level Objective # An SLO is a specific, measurable goal or target that a service aims to achieve. It is a component of an SLA and defines the acceptable level of service.

Designing Data Intensive Applications - Encoding and Evolution (part 2) - Modes of Dataflow

Modes of Dataflow # In this part, we will explore the different ways data can flow between systems, and how the choice of encoding format can affect the dataflow.

Designing Data Intensive Applications - Encoding and Evolution (part 1) - Formats for Encoding Data

Chapter 4: Encoding and Evolution # Software application changes overtime, as we justified in the chapter 1 by the need of evolvability . Data schema also needs to be open to change over time. The changes in the data schema usually requires change in the code, but unfortunately, this is often impossible to operate instant update of all the code:

encoding / decoding vs serialization / deserialization vs marshalling / unmarshalling vs parsing + transcoding & codec

Encoding/Decoding : Converting data between different representations (e.g., Base64, UTF-8) Transcoding : Converting content from one encoded format to another (e.g., MP4 → WebM) Codec (Coder-Decoder) : Software/hardware implementing encoding/decoding for specific formats (e.g., x264 for H.264) Serialization/Deserialization : Converting objects/data structures to/from a format suitable for storage…

Designing Data Intensive Applications - Storage and retrieval (part 2)

Transaction processing or analytics # A transaction represent a group of reads and writes that form a logical unit. The access pattern of looking up for typically a few records by their primary key, and writing some records, is called OLTP (Online Transaction Processing).

Designing Data Intensive Applications - Storage and retrieval (part 1)

Chapter 3: Storage and retrieval # There is a big difference in storage engines that are optimized for transaction processing and those that are optimized for analytics.

Partitioning theory made simple

Congratulation, your application is successful, you have many active users! That means that more and more data is accumulating in your systems, and your database is starting to get clogged up with the increasing amount of data it accumulates, each query having to scan over more and more rows.

Designing Data Intensive Applications - Data models and query languages

Chapter 2: Data models and query languages # Data models have effects on how the software is written, but also on how we think about the problem we are solving.

Designing Data Intensive Applications - Reliable, Scalable and Maintainable applications

General definitions # Reliability : Tolerating hardware & software faults. Human error Scalability : Measuring load & performances. Latency percentiles, throughput.

[Archived] HN jobs explorer

Shipping fast, then iterating

I have been a pretty avid reader of the Indie Hackers website for a long time, as far as 2019 as I remember. And recently my interest has peaked back.

Refactoring is an investment

Writing clean code is not something to take lightly. Of course, There are some cases where you need to push to production a quick (& dirty ?) solution. But most of the time, in an enterprise environment, you need code that is maintainable, modulate, easy to evolve, that is agreeable to work with.

IDE functionalities I cannot live without

The title is of course an overstatement, but in the day-to-day life of someone’s spending hours upon hours in their IDE, I always cannot thank myself enough when a simple functionality or shortcut is saving me minutes/hours of work.

Messages / Event

In the context of messaging systems, messages and events are related but not necessarily the same. They represent different types of data that can be exchanged between components or services within a distributed system. The distinction between the two are:

Stupid Simple System Design

I always see all the discussions about this tech and that one on Hacker News . About this new revolutionary data storage system, about how that company claims they saved X% on their bill by making a switch on their architecture.

User-level threads / Kernel-level threads

User-level threads are not recognized by the operating system as they are implemented by the user (e.g in Go, the goroutines are user-level threads that are managed by the runtime). The advantage of these are that they are simple to use and synchronize. We can start as many user-lever threads as we want, but they have a small overhead on start/stop, and they cost additional memory. Kernel-level…

Gophercon UK 2023 throwbacks

Thanks to Leboncoin, I could attend the 2023 edition of the Gophercon UK. It was a great and enriching experience, especially as it was my first developer conference. If you are able to attend to a Gophercon or similar conference, and you are definitely passionate about software, I highly advise you to take the leap. Attending to such an event, I could:

Concurrency in programming simply explained

I used ton be pretty confused by all the techniques used to achieve concurrency in programming, even the way concurrency should be executed seemed unclear to me. I’ll try to lay down all the things around concurrency in programming and clarify all the terms.

Database replication vs sharding

When using database with applications at scale, every part of tech stack end up facing scalability issue. Databases encounter a unique set of issue to solve, which requires unique techniques.

mock / stub / fake / dummy

Referring to Martin flower’s article : Dummy objects are passed around but never actually used. Usually they are just used to fill parameter lists. Fake objects actually have working implementations, but usually take some shortcut which makes them not suitable for production (an in memory database is a good example). Stubs provide canned answers to calls made during the test, usually not…

What do we mean by platform in tech ?

What exactly are we talking about when by the platform in a tech company ? What is a platform team, and what are they achieving ? I couldn’t even grasp these questions when joining my first platform team in my career, the Search platform at Leboncoin.

The Lindy effect in software

The longer a piece of clothing has been in trend, the safest it is to say it will last and stay in trend. In the same fashion, if a book has been read through the last decades or centuries, it is unlikely it will become irrelevant soon, especially compared to the more recent books.