Reading · Volution Notes · RSS Amplifier

Followup

This is the second part on this topic preceeded by:

Summary

The next phase in my RAID5 chunk size benchmark tries to take into account the file-system, namely Ext4.

Therefore in this part I quickly describe the methodology and the raw data. (An in-depth analysis and conclusions would follow in a separate part.)

The methodology...

mdadm --create a RAID5 md block device from 3x 4 TiB drives (Western Digital Gold (WD WD4002FYYZ)); the chunk size ranges from 4 KiB up to 64 MiB (and all the 2's powers in between);
use only the first 512 MiB from each drive (thus resulting in an 1 TiB md block device); (the bandwidth for this region, for each individual drive, is almost ~200 MiB/s;)
mke2fs an Ext4 file-system by using proper -E stride=<...>,strip_width=<...> values;
git clone (without checkout) the cdnjs.com repository; (the clone is done shallowly, with a depth of 128 commits, which at the time of the execution contained the commit 344797430da2d5...;) (this first clone is not benchmarked, as its performance depends mainly on the network;)
git repack this first clone; (this operation is mostly CPU intensive;)
git clone (also without checkout) the first repository to a second one (on the same file-system); (this second clone is benchmarked;)
git checkout (from the second repository) the commit 344797430da2d5...;
git status the checked-out work-tree;
du the checked-out work-tree; (around 158 GiB;)
find all the files in the checked-out work-tree; (around 4.9 M files;)
cat all the files (sorted by inode);
sample ~1/64 (~1.5%) of all the files; (around 80 K files;)
md5sum the sampled files (sorted by path, then by inode, and then randomly) with a level of parallelism from 1 up to 512 (and all the 2's powers in between);
before each of these steps are executed, an sync and sysctl vm.drop_caches=3 are issued;
in total ~4.5 TiB of IO (mostly reads) was generated in ~41 hours;

The raw data (metrics)...

The following metrics are reported:

elapsed -- the "real-world" ("wall-clock") time for the given operation;
CPU -- the time that the CPU was actually used (both user-space and kernel-space); (this shouldn't vary much;)
IO/s -- the real IO bandwidth; (as the ratio between IO and the real time;)
IO/s (CPU) -- the bandwidth as the ratio between the IO and the CPU used time; (this also shouldn't vary much;)
IO/s (OS) -- the bandwidth as the ratio between the IO and the OS (kernel-space) used time; (this would indicate any OS overhead;)
waits -- the context-switches as a consequence (mainly) of IO;
(there are in total 630 datapoints for about 7 metrics;)