Back in November 2025, when doing tests of C4 on a loopback address, I observed that C4 achieved lower data rates than Cubic or even BBR. Since “performance under loopback” was not a high priority scenario, I filed that in the long pile of issues to deal with later. Then, in early January 2026, I read a preprint of a paper by Kathrin Elmenhorst and Nils Aschenbruck titled “2BRobust – Overcoming…
Our new congestion control algorithm, C4 , is designed to work well in difficult network conditions, and in particular on Wi-Fi networks exhibiting high delay jitter. We noticed the “Wi-Fi jitter” issue two years ago (see the weird case of Wi-Fi latency spikes ). We would see sudden jumps in measured RTT, from 1 or 2 milliseconds to 50 tor even 200ms. Since that, we have been looking for…
For the past 6 months, I have been working on a new congestion control algorithm, called C4 for “Christian’s Congestion Control Code”, together with Suhas Nandakumar and Cullen Jennings at Cisco. Our goal is to design a congestion control algorithm that serves well real time communication applications, and is generally suitable for use with QUIC. This leads to the following priorities: Works well…
This morning, I was contacted by Victor Stewart , who has been contributing to picoquic for a long time. Victor is using the new Codex extension that OpenAI released last week. He was impressed with the results and the amazing gains in productivity. I started from a very skeptical point of view. I saw initial attempts at using various AI systems to generate code, and they required a rather large…
The Great Firewall Report , a “long-term censorship monitoring platform”, just published an analysis of how the “Great firewall” handles QUIC . I found that a very interesting read. The firewall work is distributed between an analysis center and cooperating routers, presumably at the “internet borders” of the country. The router participation appears simple: monitor incoming UDP flows, as defined…
The IETF draft defining multiparty extension for QUIC is almost ready, but there was an interesting debate during the IETF meeting in Bangkok about its status. One of the reasons is that we have only a few implementations, and not all that much deployment in production. The other reason is that congestion control for multiparty is still somewhat experimental. The draft is carefully worded to only…
A couple weeks ago, I got a message from researchers who had found a couple of pretty nasty bugs in Picoquic. These were quickly fixed, but they gave me a bad feeling. In theory, the core functions of Picoquic were well tested, and bugs of this kind were fixed a long time ago. But the code went through quite a bit of churn, as more functionality like multipath, ack frequency and web transport were…
This is a story about bugs, but also about testing. On Tuesday November 12, I received an email from a research group investigating how to test QUIC protocol implementations. During their tests they identified two bugs in Picoquic, which both involve parsing HTTP 3 protocol headers. One bug involved reading one byte out of bound, the other dereferencing a NULL parser. These bugs are now fixed in…
Marten Seemann and I just published an entry on his blog, A p2p Vision for QUIC . This is a technical argument, explaining how a series of small improvements to the QUIC standards would suffice to make QUIC a great peer-to-peer (P2P) protocol. I personally love it, because the Internet suffers greatly from excessive centralization, with big platforms acting as gatekeepers. Currently, pretty much…
The first investigation of loopback performance focused on the randomness of the RTT measurements on loopback, and the fix for BBR was to not make congestion control decisions based on delays if the RTT were small. After that, the results “looked good”, although there was quite a bit of variability in the results. The tests were made by sending 1GB of data. Looking at traces, it seemed that many…
I did a quick experiment last week, measuring performance of picoquic on a loopback interface. The goal was to look at software performance, but I wanted to tease out how much of the performance depends on CPU consumption and how much depends on control, in particular congestion control. I compared connections using Cubic and my implementation of BBRv3. I observed that BBRv3 was significantly…
I just fixed another bad bug in Picoquic. The bug, found by Asakura Mizu, relates to unbounded storage of TLS Stream . The issue dates to the decision back in 2017 to embed TLS in QUIC, and to send the TLS data in a series of “crypto streams”. Initially, the crypto stream was just a data stream with the number 0, but after discussion we modeled the TLS stream as a set of 3 “crypto streams”, one…
QUIC is an encrypted control protocol. That’s a big difference with TCP, and it prevents some of the known attacks against TCP, such as a third party sending a TCP “Reset” packet (RST) and closing a TCP connection. In QUIC, the equivalent of the RST packet would be a “Connection Close”. It will only be accepted if sent inside an encrypted packet, and once the connection is established only the two…
I was just checking my mail today when I found a new issue filed in picoquic: Stack-based buffer overflow in picoquic_verify_retry_token . Oh well, time to eat some humble pie. The issue happens in the function that decodes the “Retry Token” - see Section 8.1.2 of RFC 9000. This function tries to decrypt a retry token into uint8_t text[128]. This token is received from network, so it could be…
Back in December, Marten Seeman found an attack against QUIC . Malevolent clients could exploit the “path validation” mechanism to create large queues of “Path Response” messages, eventually saturating the server”s memory. Back then, it turned out that the picoquic implementation was not vulnerable, because it only responded to the last challenge received. Marten has since found another similar…
I started working on QUIC multipath in 2017, see this old draft . 7 years and 21 drafts later, we have still not converged on a solution. We thought we were close last November with version 6 of the IETF draft , but no, we had to go back to the drawing board. At this point, my implementation of QUIC in picoquic supports three ways to implement multiparty: the so called “simple” way , with minimal…
Marten Seemann made a great contribution to QUIC interoperability by setting the QUIC interop runner . The site runs series of interoperability tests between participating QUIC implementations (17 of them when I am writing this) and reports that in a large result matrix. It is a nice complement to the internal tests of the implementations, and it was flagging an interesting issue: the test L1 was…
I am following the discussions on the IETF mailing lists, and I got engaged in a review of a proposal by Fred templin to better implement packet fragmentation and reassembly at the IPv6 layer . I am not a great fan of that proposal, largely because of the many arguments against IP level fragmentation explained in RFC8900 and the security issues described in RFC7739 . I think it is a bad idea to…
I just spent some time revisiting how the picoquic code handles failing links. The QUIC specification in RFC 9002 makes reference to exponential backoff. If a node does not obtain any ACK for some time, the “Probe timeout” (PTO) fires. The node sends an extra packet to “probe” the state of the path, and solicit an ACK. If the ACK arrive, it should acknowledge all the pending packets that have been…
In a previous post I observed that the classic way to compute RTT statistics and retransmission timers does not work well. When acknowledgements are too frequent, the correlation between successive RTT measurements causes the smoothed RTT estimate to track closely the last values, and the RTT variations to be widely underestimated. This imprecision impacts both the effectiveness of loss recovery…
Last week, the QUIC Working Group was reviewing an errata for RFC 9002 , the description of loss recovery and congestion control for QUIC. There was an error in the description of the algorithm used to compute the variable rttvar , which describes the expected variation of the round trip time (RTT). The order of instructions was wrong, leading to underestimating the rttvar by 1/8th. That’s of…
About two weeks ago, I was told by developers of “Media over QUIC” that there was an issue when running over Wi-Fi. After a few seconds, there would be some kind of event, triggering congestion control implemented in Picoquic to reduce the bandwidth, and then resulting in pretty bad performance. It seems due to issues with the Wi-Fi driver on the Mac, as I wrote in a toot on Mastodon . Now that I…
I just implemented in Picoquic the new ACK processing algorithm (proposed for QUIC multipath)[https://github.com/quicwg/multipath/pull/217], which processes ACK independently of the path over which they arrive. It looked good, but there was an interesting regression. The tests that simulated transmission over satellite links were failing. The previous version showed a file transfer concluding in…
A friend, Marc Blanchet, asked me last December whether it would be possible to use QUIC in space. Sure, the delays would be longer, but in theory it should be possible to scale the various time-related constants in the protocol, and then everything else should work. I waited to have some free time, and then I took the challenge, running a couple of simulations to see how Picoquic would behave on…
I have been working on the Picoquic implementation of QUIC since 2017. Picoquic distinguished itself by performing very well on GEO satellite links. The main reason is that 40 years ago, I was studying protocols for transport of data over satellite links for my PhD. So, of course, I wanted to support the scenario well in my implementation of QUIC. Which explained why this morning someone was…
My blog was fist published on WordPress , but I am getting repeated feedback that not having advertisements would be better, and also that a blog on networking really should be accessible on IPv6. So, I am taking the plunge and migrating the blog to the server of my personal company, Private Octopus . The new blog is published as a static web site, developed using Jekyll . The upside of Jekyll is…
People who install it love the Pi-hole . The Pi-hole is a DNS software server, typically running on a Rasberry Pi , that can filter the DNS requests coming out of a local network and, for example, drop connections to advertisers. Once you install that, web browsing becomes notably snappier, not to mention much more private. But there is a catch. If you also configure your device or your browser to…
Back in May, there was a discussion on the QUIC implementers’ chat room. What if a client played games with acknowledgements during the initial exchange? Could it be used in a DOS amplification attack? Or maybe some other form of DOS attack? In the proposed attack the attacker sends a first packet, which typically contains the TLS 1.3 “Client Hello” and its extensions. The server will respond with…
Many network applications work better with low latency: not only video conference, but also video games and many more transaction-based applications. The usual answer is to implement some form of QoS control in the network, managing “real-time” applications as a class apart from bulk transfers. But QoS is a special service, often billed as an extra, requiring extra management, and relying on the…
It seems that many people have started using Mastodon recently. But the place feels a bit empty, especially if like me you join a small server. How do I find my friends on Mastodon? Today, there are two ways: they tell me their handle using some other service, or I stumble upon one of their posts promoted by a common friend. This might work eventually, but it is much less straightforward than just…
The QUIC specification was finalized by the IETF a year ago, but the interop tests often detect the occasional failure. The most challenging tests are series of 50 connection attempts in a lossy environment, with about 30% packet loss. The tests only pass if all attempts succeed. You might think that 30% packet loss is not a realistic environment, but that’s a very good way to test the…
Many Internet Service Providers are nice companies who love their customers, but quite a few have developed a sideline of collecting logs of traffic and selling that to advertisers, or in fact to whoever is willing to pay. Even the FTC is not happy about that . New technology like “DNS over HTTP”, or DoH , enables encrypted DNS resolution through the services of a third party, and thus hide whole…
Multipath transmission is interesting, and I have been working quite a bit on implementing multipath support in QUIC. There are multiple ways to do that some simple and some more systematic , but all designs have the issue of timers. In single-path QUIC, the situation is simple: data is always sent on the same path, acknowledgements are always received on the same path. In multipath QUIC, data is…
I authored two drafts proposing two different solutions for Multipath QUIC: QUIC Multipath Negotiation Option ; and, in collaboration with colleagues at Ali Baba, Multipath Extension for QUIC . Apart from some details that could easily be aligned, the main difference is that the “negotiation option” maintains the property of QUIC Transport to have a single packet number space for all application…
In the description of the multipath implementation in Picoquic , I mentioned that I struggled with the absence of multipath support in the “QLOG” format and the associated tools . Maybe the developers have been spoiled with too many good tools, but we came to rely on tools like “qvis” ( https://qvis.edm.uhasselt.be/ ) to visualize the logs of QUIC connections, showing for example nice graphical…
Yesterday, I was providing a broad brush description of the implementation of multipath QUIC in picoquic . Today, I will present a bit more details on a specific point, the measurement of transmission delays. Just like TCP, QUIC implementations use round trip time (RTT) estimates in many ways: For loss detection, RTT estimates are used to evaluate how long to wait for an acknowledgement before a…
I just finished implementing multipath support for QUIC per draft-liu-multipath-quic-02 in picoquic . That took me quite a bit more time than I initially thought. As I was modifying picoquic’s code to handle multipath, I realized I needed to add many tests in the test suite, because there are lots of corner cases. The initial test was simple. Start a QUIC connection on a single path, add a second…
Sometimes, one looks at a bug list and find one that seems easy enough. I just did that two days ago, looking at issue #1023 in the Picoquic project, “Simulate DDOS by repeated Client-Hello” . I entered that issue four weeks ago, when doing a review of possible improvements. I had some free time, it seemed easy, so I just started the work. Guess what, this turned out to be a vivid illustration of…
I have spent quite some time developing Picoquic , an implementation of the standard version of QUIC developed in the IETF. My initial goal was just to test whether the standard could be easily implemented, and provide informed feedback to the working group as the standard progressed. Then, I realized that many other developers were happy to use my implementation as test tool, especially in 2017…
The IETF is standardizing SNI encryption and DNS encryption in a bid to improve privacy. Both techniques rely on hiding private traffic in a crowd of benign traffic and are mostly useful if such crowds exist. I set up to estimate that, and the results so far are not optimistic. There is some concentration of Internet services on big Content Distribution Networks, Cloud Providers and Hosting…
A week ago, I wrote a blog about the value of gathering QUIC logs (qlog) and processing them to derive statistics and better understand how the protocol interacts in practice with networks. My example was an analysis of packet losses. In many logs packet losses appeared to not happen as isolated random incidents, but rather as “trains” of successive errors. I have gathered more logs since and see…
As the IETF nears completion of the QUIC version 1 specification , we see renewed interest in extensions such as “forward error correction” or “unreliable streams”. In both cases, the goal is to mitigate the effect of packet losses, either by using redundancy to correct them, or by delivering the data to the application immediately to reduce latency. But before we devise such schemes, I think we…
Since 1988, the “slow start” mechanism designed by Van Jacobson has been used in most TCP-IP stacks. Slow start doubles the TCP congestion window every round trip. That works well for most connections, but not for satellite links as already explained in 1999 in RFC 2488 . The time to reach link saturation varies as the base 2 logarithm of the ratio between initial window and saturation window…
I had an interesting spam letter in my mail yesterday. It started like many other spam propositions: Dear Manager, We are an agency engaging in registering brand name and domain names. Today, Our center received an application from Dianshi International Ltd and they apply to register huitema as their brand name and some top-level domain names. We found the main body of domain names is same as your…
Back in January, I was observing that having free for all innovation on congestion control might be problematic . I was invited to discuss that in front of the TCPM working during the IETF meeting in Montreal, this July. After presenting the issue in a few slides, I saw a number of TCP experts line at the microphone and explain me that I was wrong. Their main argument was that if this was true,…
"Senator, we run ads." That was Mark Zuckerberg's response to Orrin Hatch, senator from Utah, asking how Facebook could possibly sustain a business without having customer pays. The senator was then mocked, as an old guy who obviously did not have a clue about the Internet economy. Well, not so fast. Orrin Hatch is definitely not young, but you should not laugh at his question. Zuckerberg's…
In a previous post, I pointed out that the move to "application level" transports enables transport innovation and bypasses the traditional gatekeepers, the OS developers. The one big fear is that applications would seek greater performance through selfish innovation, and that the combined effect of many large applications doing that could destabilize the Internet. The proposed solution is to have…
Maybe you have heard of QUIC, a new transport protocol that combines functionalities of TCP and TLS, plus some new features like parallel handling of streams and latency minimization through 0-RTT. The work started at Google 5 years ago, as part of their effort to speed up the web. The IETF picked up the early work from Google , and is working through its standardization . There is a great deal of…
(Corrected March 4, 2018) I am participating in the standardization of the QUIC protocol . That's why I am writing a prototype implementation of the new transport protocol: Picoquic . And the development involves regular testing against other prototypes, the result of which are shown in this test result matrix . This is work in progress on a complex protocol development, so when we test we…
Back in May and June this year, I was reviewing the state of SNI encryption. I found in reviewing the archives of the TLS mailing list. I collected the list of attacks that demonstrated holes in previous proposals, and documented them in an Internet draft ( https://datatracker.ietf.org/doc/draft-huitema-tls-sni-encryption/ ) that has just been adopted as a work item by the TLS working group (…