I was investigating possible disk corruption when copying a disk image between servers, but needed a way to visualise what might be happening. The disk image is tens of gigabytes, so looking at it in hexdump wasn’t a lot of fun. A little Python to the rescue instead:
#!/usr/bin/python3
from PIL import Image, ImageColor
import nbd
h1 = nbd.NBD()
h2 = nbd.NBD()
original = "original.qcow2";
copied = "copied.qcow2";
blksize = 4096
zero_block = bytearray(blksize)
width = 4096
output = "output.png"
red = ImageColor.getrgb("red")
green = ImageColor.getrgb("green")
blue = ImageColor.getrgb("blue")
white = ImageColor.getrgb("white")
black = ImageColor.getrgb("black")
# Open original disk image.
h1.connect_systemd_socket_activation(["qemu-nbd", "-f", "qcow2", original])
# Open copied disk image.
h2.connect_systemd_socket_activation(["qemu-nbd", "-f", "qcow2", copied])
assert(h1.get_size() <= h2.get_size())
# Open the output visualisation.
height = int(h1.get_size() / blksize / width) + 1
image = Image.new("RGB", (width, height), white)
# Iterate over the blocks of each.
x = 0
y = 0
for i in range(0, h1.get_size(), blksize):
b1 = h1.pread(blksize, i)
b2 = h2.pread(blksize, i)
if b1 == b2:
# Same
image.putpixel((x, y), green)
elif b1 != zero_block and b2 == zero_block:
# Zeroed
image.putpixel((x, y), black)
else:
# Different but not zeroed
image.putpixel((x, y), red)
x = x+1
if x == width:
x = 0
y = y+1
print("%d/%d\r" % (y, height), end='')
print()
image.save(output)
print("Saved to %s" % output)
h1.close()
h2.close()
The resulting image showed that stretches of the disk were getting zeroed out during the copy (which was definitely not supposed to happen).


Nice, but I was looking for a screenshot of that image too 🙂
Yeah the original image was enormous. I attached a small subset of it now.
I should say the red blocks in the image are expected as we’re copying & making changes. The black dots are not expected. Something is zeroing or dropping blocks
Thats cool!. Sometime ago, I wanted some visuals not for the entire disk but for files (BTRFS). Idea is to visualize matching and unmatching data blocks between two files (not diff) and mark/color those regions. Later it can be integrated with deduplication. But unfortunately I didn’t proceed further.
Though it’s irrelevant for Linux, it would be fun to re-create something like windows disk-defragmentation tool for Linux file systems. I can watch blocks moving around all day long 🙂
You might like https://rwmj.wordpress.com/2018/11/04/nbd-graphical-viewer/ & the last part of this talk https://archive.fosdem.org/2019/schedule/event/nbdkit/