Comparing Compression Algorithms for Moving Big Data

This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.

In a previous post , I wrote about the best way to transfer a directory across the network. In this post, I compare Brotli v Gzip v Zstd v LZ4 on blockchain dataset to determine which compression can give me the fastest transmission rates to move data around. I took a Badger DB directory that I needed to transmit over the wire, and ran various compression algorithms on it to see how fast can they…

Read on manishrjain.com ↗