RAID10 to ZFS Mirror Tutorial

The Beginning

After hearing about ZFS for the past year, I decided to investigate it. I read the book FreeBSD Mastery: ZFS by Michael W Lucas. It is an excellent book and highlights the power and reliability of the ZFS filesystem. A must read before continuing!

I have a 3TB RAID10 running on my server at home. I decided I wanted to change this to a ZFS mirror drive. I might make a double mirror but I’ll keep it a regular mirror for now as that gives me 2 backup drives if I have to destroy the pool and start again.

Below are the steps I took to do this. I don’t take responsibility for any data loss that occurs if you follow these steps. These are to help you get an idea of the steps required.

Current RAID10 mdadm Setup

$ sudo mdadm -D /dev/md0
/dev/md0:
           Version : 1.2
     Creation Time : Sun Apr 22 13:01:05 2018
        Raid Level : raid10
        Array Size : 5860270976 (5.46 TiB 6.00 TB)
     Used Dev Size : 2930135488 (2.73 TiB 3.00 TB)
      Raid Devices : 4
     Total Devices : 4
       Persistence : Superblock is persistent

       Update Time : Tue Aug 13 15:53:26 2024
             State : clean 
    Active Devices : 4
   Working Devices : 4
    Failed Devices : 0
     Spare Devices : 0

            Layout : near=2
        Chunk Size : 64K

Consistency Policy : resync

              Name : hermes:0
              UUID : 9386caf9:9fc6b2cf:30b4fe6f:62e3a1bd
            Events : 290839

    Number   Major   Minor   RaidDevice State
       6       8       32        0      active sync set-A   /dev/sdc
       4       8        0        1      active sync set-B   /dev/sda
       5       8       48        2      active sync set-A   /dev/sdd
       3       8       16        3      active sync set-B   /dev/sdb

Backup RAID10 Files to a Backup Drive

Before starting this, make a backup to an extra hard drive. rsync -avh /raid10 /backupdrive

Shutdown Running Processes

The fewer things running during this whole process will ensure your data will be more consistent.

Stop any cron jobs you have running. Check /etc/cron/ or /etc/cron.d/
Stop all docker containers. See what’s running with docker ps

Remove 1 of 4 drives from RAID10

This drive will be used to start a new ZFS zpool. Removing one of RAID10 drives is OK. You won’t lose data. I also change the mount point of this mdadm device so that I can use the same directory for my zpool later.

mdadm –detail /dev/md0
umount /mnt/hermes
#remove the drive you want from the RAID array
mdadm /dev/md0 –fail /dev/sda –remove /dev/sda
mdadm –stop /dev/md0 
# remove the superblock so it can't accidently join this array again
mdadm –zero-superblock /dev/sda
# assemble the RAID, is it still ok?
mdadm –assemble –scan
# change mount point of RAID10 from /mnt/hermes to /mnt/hermes-old
edit /etc/fstab
mount /mnt/hermes-old

Create 1 drive ZFS at /mnt/hermes

# find disk id for /dev/sda
ls -la /dev/disk/by-id/
# reformat drive
fdisk /dev/sda
# create the pool using the disk-id, much safer than /dev/sda
zpool create -O compression=on -o ashift=12 -o altroot=/mnt hermes /dev/disk/by-id/ata-TOSHIBA_HDWD130_AAAAAAAA

Create ZFS datasets

You don’t need to do datasets, but a lot of the power of ZFS comes from what you can do at the dataset level. I am turning off access time (atime) because it will give me a speed increase. zfs create -o atime=off hermes/immich-photos

Move files to the ZFS Pool

rsync -avh /mnt/hermes-old/blah/ /mnt/hermes/blah/

Attach Another Drive to ZFS pool, Create Mirror

# remove the other sync set from /dev/md0
mdadm /dev/md0 –fail /dev/sdb –remove /dev/sdb
# prepare it for use as ZFS device
mdadm –zero-superblock /dev/sdb
# find disk id for /dev/sda
ls -la /dev/disk/by-id/
# attach it to the ZFS pool you already made
zpool attach hermes ata-TOSHIBA_HDWD130_82VGAX9AS ata-TOSHIBA_DT01ABA300V_80VZ0X5AS
# Let ZFS resilver (sync) the new drive
# should happen automatically

Next Steps

I will keep my RAID10 running as a degraded array. It still works but now it’s just RAID1. I will confirm I have all my data and start enabling services again.

After everything is OK, I will re-add my other RAID10 drives to the ZFS pool.

I hope this gives others who are running mdadm the steps they should follow to change to ZFS.