Recent content on Max Inden
I gave three short talks at this year’s HTTP Workshop . Increase and evolve HTTP/3 & QUIC Slides: mxinden-bot.github.io/slides/04-quic-discussion A discussion deck. HTTP/3 sits around 30% and has plateaued. I cover the ways to measure that share. Then what holds adoption back: server and framework support; “security” software and enterprise middleboxes that break QUIC; and the…
I joined the Working Draft podcast to talk about HTTP/3 and WebTransport. We traced the evolution from HTTP/1 through HTTP/2 to HTTP/3, why head-of-line blocking pushed the web toward QUIC, and how building transport and security together enables 1-RTT and 0-RTT handshakes, connection migration, and encrypted DNS. We closed on WebTransport as a modern alternative to WebSocket, supporting both…
I gave a talk on the upcoming WebTransport protocol in the Browser Devroom at FOSDEM 2026. WebTransport is an upcoming protocol (standardized by the IETF) and Web API (standardized by the W3C) for bidirectional communication on the web. It provides multiplexed streams and unreliable datagrams on top of HTTP/3 and HTTP/2. This talk explains how WebTransport works at the protocol level, how it maps…
Andrew, a colleague, and I presented an outlook for Firefox and the web. The Web’s transport stack is changing rapidly, with QUIC, HTTP/3, and encrypted DNS seeing broad adoption. This talk gives an overview of the modern network protocols Firefox already deploys and invests in, including QUIC and HTTP/3’s growing share of Web traffic. It will highlight what Firefox actually sends on the wire…
I joined Glen on the netstack.fm podcast to talk about modern networking in Firefox. We discussed Mozilla ’s work on Firefox’s QUIC and HTTP/3 stack — improving UDP I/O, congestion control, and overall performance — and why QUIC matters as a fast, encrypted, and evolvable transport for HTTP/3, WebTransport, and beyond. Thanks for having me, Glen! https://netstack.fm/#episode-11
Motivation Around 20% of Firefox’s HTTP traffic today uses HTTP/3, which runs over QUIC, which in turn runs over UDP. This translates to substantial UDP I/O activity. Firefox uses NSPR for most of its network I/O. When it comes to UDP I/O, NSPR only offers a limited set of dated APIs, most relevant here PR_SendTo and PR_RecvFrom , wrappers around POSIX’s sendto and recvfrom . The N in…
I presented my recent work on Firefox’s HTTP3/QUIC stack in the Network Devroom at FOSDEM 2025. QUIC is a new transport protocol on top of UDP, transporting a large portion of the Internet traffic today. UDP I/O performance is crucial for QUIC implementations, where e.g. system call overhead can significantly impact throughput at high network speeds. To improve QUIC throughput, Firefox is…
I don’t see myself making major contributions to (rust-) libp2p in the near future and thus I am stepping down as a maintainer. As announced before, I have left Protocol Labs in December 2023 . After a 2 month re-orientation break, I have decided to move on entirely. My first commit was 5y ago, a small bug fix in our address handling . Since then lots happened. A couple of milestones I was…
How you can connect everything (browsers & non-browsers), everywhere (public or private), all at once (using libp2p). Slides
I presented at IPFS Camp 2022 on mitigating Denial-of-Service attacks in peer-to-peer networks. I discussed resource management strategies such as enforcing backpressure and provided examples of coding pitfalls to avoid in Rust and Go. You can find the recording and slides of my talk below. Slides DOS Denial-of-service attack Hard in peer-to-peer as identities are cheap Relevant for any scarce…
Dennis and I presented Hole punching in the wild , learnings from running libp2p hole punching in production, measured from vantage points across the globe in the network devroom at FOSDEM 2023. At FOSDEM 2022 I presented libp2p’s hole punching mechanism, overcoming NATs and firewalls with no dependencies on central infrastructure. One year has passed since. We rolled it out to live…
I presented an overview on Peer-to-peer Browser Connectivity options in the network devroom at FOSDEM 2023. Connecting from the browser to a public server with a valid TLS certificate is easy. But what if the server has a self-signed certificate? What if it isn’t public? What if it is another browser? This talk covers the intricacies of browser communication beyond the standard…
We’ll take a closer look at the recently added decentralized hole punching feature, allowing NAT and firewall traversal without the need for any central coordination servers (STUN and TURN). Recording
We will discuss the current state of the project, eyeball the various language implementations, take a look at the many live networks running on top of libp2p today and finally cover the project roadmap for the years to come. Recording
We have been working on hole punching support in libp2p for the last year. It finally landed in the Go and Rust libp2p implementation. Read all about it on the IPFS blog: https://blog.ipfs.io/2022-01-20-libp2p-hole-punching/
Gave a talk on hole punching in libp2p at FOSDEM 2022. Peer-to-peer hole punching without centralized infrastructure How libp2p can traverse NATs and firewalls without coordination through central STUN and TURN servers. libp2p is a universal, cross-platform, multi-language, modular peer-to-peer networking library powering multiple large-scale networks, for example IPFS. In the first part of the…
Gave a talk on the state of libp2p at FOSDEM 2022. State of libp2p Status quo and future roadmap of the peer-to-peer networking library libp2p. libp2p is a universal, cross-platform, multi-language, modular peer-to-peer networking library powering multiple large-scale networks, for example IPFS, Ethereum 2, Filecoin or Polkadot. We will discuss the current state of the project, eyeball the various…
On Tuesday 20th of July I gave a talk introducing libp2p , a modular network stack. Recording Slides
In our previous session we looked into Mixnets, more specifically Loopix. With this session we stayed in the space, talking about a network adopting many concepts from Loopix, namely Nym. As a preparation, we read section 1 through 4 of the Nym whitepaper. Introduction Network-level surveillance The Nym Network Design Mixnet for Network-Level Privacy Diaz, Claudia, Harry Halpin, and Aggelos…
34th session was on Mixnets as lots of people suggested in the past. As a preparation we read the Loopix paper. Piotrowska, Ania M., et al. “The loopix anonymity system.” 26th {USENIX} Security Symposium ({USENIX} Security 17). 2017. https://www.usenix.org/system/files/conference/usenixsecurity17/sec17-piotrowska.pdf The corresponding talk by Ania Piotrowska is very much worth watching…
In our third BGP session we focused on BGP security, more specifically we discussed how to perform hijack and interception attacks using BGP. The primary paper of the session was: Birge-Lee, Henry, et al. “Sico: Surgical interception attacks by manipulating bgp communities.” Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security. 2019. A good introduction…
After our first BGP session, introducing the protocol based on the convergence problem of BGP, our second session on BGP covered route propagation and tuning. The primary paper was: Gray, Caitlin, et al. “BGP Beacons, Network Tomography, and Bayesian Computation to Locate Route Flap Damping.” Proceedings of the ACM Internet Measurement Conference. 2020.
Below is a summary of our efforts to optimize flow control in the Rust Yamux implementation . While not a novel approach, I still find the end result worth sharing thus my forum post. https://discuss.libp2p.io/t/optimizing-yamux-flow-control-sending-window-update-frames-early/843
We decided to turn our interest to BGP which we will devote 3 sessions to. In today’s session - the first one - we introduced BGP, looked at the convergence problem, as well as the solution suggested in the paper below. Gao, Lixin, and Jennifer Rexford. “Stable Internet routing without global coordination.” IEEE/ACM Transactions on networking 9.6 (2001): 681-692. To play around…
I have created a new GPG key replacing my old GPG key. I did not loose access to the old one, nor was it, as far as I am able to tell, ever compromised. As I did with the old key, I will use the new key both for signing my open source work as well as securing my e-mail communication. You can find both the old and the new key on all major key servers. Along with the new key you find a signature of…
What better way to start a new year than with a paper discussing how to change time? In the 30th session we discussed a paper which I think has much up its sleeves - Attacking the Network Time Protocol . First off the paper gives us a good introduction to the inner working of the network time protocol. Next up it examines the broader ecosystem as well as why we need accurate time in the first…
With the 28th session we jumped into the space of byzantine fault tolerant consensus protocols. We covered fault tolerant consensus with various Paxos variants in the past, but this session was the first one looking into how to solve the byzantine generals problem . Instead of using PBFT [1] as a first paper we went with Hotstuff [2] instead. The reasoning behind this choice was (a) Hotstuff…
I find it helpful to know the orders of magnitude by which certain computer operations differ. Certainly it is not worth the effort to pay attention to every digit or learn these by heart, especially since they differ (slightly) across systems, but having a basic understanding of what a tiny fraction of time a CPU cycle occupies compared to sending a TCP packet is incredibly helpful whenever…
We have long been planning to cover the caching mechanisms in CPUs. As a shared knowledge base for the discussions in this session we chose the following two articles by Martin Thompson among other things known for his work on the LMAX Disruptor : CPU Cache Flushing Fallacy including a good overview over the different caches in modern Intel CPUs. Write Combining exemplifying the advanced…
In the session today we covered Madhavapeddi Shreedhar and George Varghese paper “ Efficient fair queuing using deficit round-robin ” [1]. While the session was not so much about the relatively simple algorithmic details of deficit-round-robin (still worth checking out) we talked about: Its benefits over basic FIFO queuing and thus its impact for congestion controlled traffic (tcp)…
After a bit of a break due to current pandemic we decided to carry on and continue our meetings as virtual calls. Ignoring the usual initial hiccups and the missing whiteboard the medium worked well for us. Topic and reading of this session was the ACM Queue article BBR: Congestion-Based Congestion Control [1], as well as the Dropbox article Evaluating BBRv2 on the Dropbox Edge Network [2]. We…
I recently stumbled upon the idea of an Elimination Back-off Stack promising to be a parallel , linearizable and lock-free stack. In case you are not familiar with it, I would suggest either reading my previous post or the corresponding paper [1] itself. Being quite intrigued by the ideas of the above stack I wrote my own implementation in Rust with a little help from crossbeam .
Reading The Art of Multiprocessor Programming [1] I came across the Elimination Back-off Stack [2] datastructure introduced in 2004 by Danny Hendler, Nir Shavit, and Lena Yerushalmi. It promises to be a parallel lock-free stack. How can a stack allow parallel operations without going through a single serialization point, e.g. a Mutex or an Atomic? Let’s dive into it. A lock-free stack A…
In January 2017 I joined the company CoreOS as a test-engineer helping the monitoring team and the rkt container engine team write reliable software. Eventually I joined the CoreOS’ monitoring team full-time as a software engineer and ultimately was invited to be part of the upstream Prometheus team due to my contributions to the Alertmanager sub-project. Over the next 2 years and 4 month I…
At the end of the previous session one of us suggested to dive into congestion control algorithms. This has found a greater echo, thus the 23rd session covered congestion control algorithms in general and TCP’s Reno as well as TCP’s Tahoe in particular. This weeks reading was: Chapter 13 “TCP Reno and Congestion Management” from the comprehensive online book “An…
In the 22nd session we took a look at io_uring - a new Kernel interface for asynchronous I/O. Tyler, who is currently implementing an io_uring library in Rust [4] for his database sled [7] guided us through the concepts as well as a bunch of source code. Tyler started off introducing the status quo of I/O interfaces within the Linux Kernel like read, pread and preadv, jumped over to asynchronous…
We started the new year with a session on epidemic / gossip protocols. To decide what to read I compiled the following list of papers that I either enjoyed reading in the past, or that were recommended to me. The Swim (Scalable failure detection and membership protocol) paper won the poll. Das, Abhinandan, Indranil Gupta, and Ashish Motivala. “Swim: Scalable weakly-consistent infection-style…
Last Tuesday we meet again to discuss different attacks and possible countermeasures for distributed hash tables. More in particular we looked at Kademlia and its security extension S/Kademlia [1], possible eclipse attacks on the Ethereum network [2], a novel approach of hiding its own connection buckets as well as using an existing social graph as a network topology in the Whanau paper[3],…
I have been organizing a distributed systems paper reading group in Berlin for the last year. We meet every other week discussing a paper in the distributed systems space. This could be anything from Chandy–Lamport’s algorithm for global distributed snapshots [1] to things like conflict free replicated datatypes [2]. The event is open for anyone interested. I only ask people to come…
Within my work at Red Hat and Kubernetes SIG instrumentation I have been working on kube-state-metrics , a Prometheus exporter exposing the state of a Kubernetes cluster to a Prometheus monitoring system. In particular I have focused on performance optimizing metric rendering for both latency as well as resource usage. Below I want to describe our approach of metric driven performance tuning,…
Kube-state-metrics exposes Prometheus metrics of the state of a given Kubernetes cluster. The project uses the standard Prometheus client Golang library, which is not optimized for the very specific use case of kube-state-metrics. This talk covers different optimizations like metric caching and improved text marshaling dividing CPU usage by a factor of 6 and memory and response time by a factor of…
Monitoring plays a crucial role in a microservice architecture. Restricting the management and configuration of the monitoring stack to the operations team results in workflow bottlenecks. Instead one could provide a self-service monitoring platform, enabling each team to easily setup monitoring for their applications and customize it to their needs. This gives each team the ability to deeply…
What If Component xxx Dies? Introducing Self-Healing Kubernetes Kubernetes promises healing your application on all kinds of failure scenarios, but why not self-heal Kubernetes itself? This talk introduces self-hosted Kubernetes (K8s inside itself) to autonomously recover from failure scenarios with the help of e.g. itself, systemd and checkpointing. We will ask and answer questions like “What…
Kubernetes is a powerful system to build and operate a modern cloud-native infrastructure. Monitoring with Prometheus ensures that Kubernetes stays healthy. Prometheus is a stateful application, so operating it in a cloud native environment can be a challenging task. The Prometheus Operator makes running highly available Prometheus clusters, and even an entire end to end monitoring pipeline,…
Distributing and deploying software inside (Docker-) containers for security, isolation and ease of use is the new big thing. But once you got all your services nicely wrapped - who takes care of all these containers? The open source project Kubernetes, originating from Google, helps you manage containerized applications, as the operating system of your datacenter, treating hundreds of machines as…
Alertmanager deduplicates, groups, and routes alerts from Prometheus to all kinds of paging services. With it comes a dated UI which does not live up to the expectations of the users, nor does it attract new contributors. From this talk, you will learn how we addressed these issues when building the new UI from scratch. We made it friendlier to users by removing unnecessary domain language noise.…
Finished my very first Triathlon today here in Berlin. I survived the swimming way better than I thougt, biking wasn’t a big problem, but the 10 k at the end were more of a stumbling then a running.
Sunday five mile hike around Angels Island. Great to get out of the city from time to time.
My first 16 miles run. Nearly blacked out at the end. Amazing route!
After two delayed flights, one canceled flight and two lost bags I made it back home to Germany.