Automatic Backups Up To External Drive
Asking the internet for the “best” way to perform basic computer tasks, often isn’t a worthwhile time investment. It is possible to strike gold and get the perfect answer. It also seems equally likely the advice will be contradictory or unsuitable for the task.
If the goal can be achieved with the knowledge (and hardware) I already have, things usually go much smoother. Even if it involves referencing a man page or two.
The goal is to setup automatic data backups when an external drive is connected. Breaking it up into smaller tasks we get:
- Find suitable hardware
- Decide on the target backup directories
- Create the necessary configuration files
- Test if it works.
Hardware, Assemble!
I had a old 5.25 inch pc bay enclosure that was being used to house four 3.5 inch storage devices. It didn’t serve much practical purpose as the PC case had plenty of room to store the drives in the normal way. The slot doors were kinda busted so I decided to toss it. While removing the enclosure, I concluded that one of the two SSDs could instead be used for external storage. This seemed an improvement over fumbling with the dozen or so poorly labeled 16GB USB thumb drives that were lying around.
I also happened to have an unused 3.5 inch external SATA enclosure sitting around. This was most likely purchased around the same time as the rest of the hardware. Besides using a mini-USB connector, the enclosure seemed perfect for my objective 1. All that was left was some unscrewing and re-screwing.
The Files
Certainly, some other users will be of the mind, “I back up everything!… twice”. That’s a healthy attitude to have. I’m just looking to copy my dotfiles, documentation, source code for essential programs, and some personal projects. Perhaps a collection of some small binaries for convenience as well. These are files I would need in order to migrate my current setup to other hardware. Or files that might be useful to add to another existing system.
Copying “everything” would require a much larger drive, which for me means spending $currency_symbol.
It would also increase the backup process time.
I’m targeting times in the “two seconds” range, not “two minutes”.
Software Configuration
The fun part!
Some quick housekeeping first.
I formatted the drive using gparted, creating a single ext4 partition.
Yes, this could also be done from the command line.
Drive management is one of the few tasks where I prefer to use a graphical program.
The feedback is clearer.
It’s less likely that I’ll make some regrettable mistake 2.
After plugging the enclosure into my machine,
the Dolphin file manager shows the drive available for mounting.
The lsblk command in the terminal also displays the drive.
The mounting of drives at specific filesystem locations
typically involves creating an entry in /etc/fstab.
This works for normal usage, but we need to do more than just mount the drive.
We need to know the name of the drive, which can change across boots.
The software solution we will employ involves multiple steps.
We need to create a udev rule to detect this specific usb storage device.
When connected, we want to run a oneshot systemd service to
run a bash script that handles the business logic for the data backup.
Easy.
Udev
Optionally skip this small rant about udev rules:
Creating a udev rule for a very specific circumstance can be frustrating. Is there a “shellcheck-like” udev rule validator? There have been times where I have spent an hour tweaking a line in order to detect a specific piece of hardware only to find out it was a small syntax error that prevented my original configuration from working.
There are a bunch of ways to configure udev, systemd, and the shell script. But the combination of settings I typically use end up working for me so I tend to stick with them. Admittedly, more careful study of the udev manual might reduce the amount of trial and error. Yes, reduce, not eliminate.
The first step is to gather the appropriate information about the device we are using.
Using the udevadm monitor --property command is the “best” way I have found to
get what we need.
Devices can be matched a number of ways.
I have used ATTR{idVendor} and ATTR{idProduct} for other past projects.
Some attribute values are just a few digits,
so there can be false positives when a single attribute is used.
Here I ended up using ENV{ID_MODEL} as a way to match the drive.
For storage devices,
even regular end users may have multiple units of the same model.
In this case, a unique property such as ID_SERIAL may be required.
I only have one unit of the drive in question, so ID_MODEL is sufficient
|
Systemd
Not a lot to say here.
Systemd is complex, and I don’t have time or the desire to learn it all.
I usually copy a service file from another thing I know works.
Fields I don’t understand or are problematic, I’ll delete.
Example: the service I copied this from specified a User, but I removed that line.
This service needs to be run as root because of the disk mounting.
|
Bash Script
If the udev rule creates a symlink successfully,
we now have a path we can use to reference our backup drive.
If /dev/USBSSD and $mountpoint are present, we can perform writes to the drive.
If not present, something went wrong and we need not continue.
|
Now the reason why we are here, backing up the data.
I use rsync because of familiarity, and convenience.
It is already installed on my machine.
|
Testing If It Works
When 90-USBSSD.rule is working correctly, and the matching drive is connected,
a symlink dev/USBSDD will be created.
The USBSSD.service will also be triggered.
This runs the USBSSD.sh script.
Someone, somewhere will say there is a better way to do this.
I believe them.
One valuable thing I have learned, is to use the same naming
convention for every file in the chain.
In this case I used USBSSD for everything.
So when I come back in a year and need to know what this random systemd
service involves, it will be apparent.
|
There was a bit of iteration which required changing the file(s) involved and copying them to the appropriate directory in the filesystem. When plugging in the device to test, checking the systemd service status was helpful to find where and how any failure occurred. After each change the system involved needed to be updated.
|
|
|
I should have kept a capture of a failed service to use as an example.
The printf statements in USBSSD.sh will show up in a systemd status report.
It’s an excellent tool to troubleshoot when something goes wrong.
The internet used to tell me all the time that systemd is bad.
I don’t have an informed opinion on that.
But I will say that the status command is good.
|
Conclusion
I can’t claim to understand all of the nuances, but it works. I would like to suppress the rsync info in the systemd log, using the log file in /tmp/ instead, but that’s not a big deal. 3 I tried using output redirection in the script to no avail.
|
After completing the task, I wanted to check what an earnest web search would produce.
The question:
How to automatically backup to external hard drive when it is plugged in on linux?
The response:
To automatically back up data to an external hard drive when it’s plugged in on Linux, you can use udev rules combined with systemd services. This method triggers a backup script when the drive is detected.
The search engine then included an example script, systemd service file, and udev rule file. Unfortunately the systemd service and shell script were significantly more verbose than they needed to be. They included extraneous variables that weren’t relevant to the task. I suspect because the answer was mostly an unrefined amalgamation of several such scripts posted on the web.
One positive: the suggested udev rule file was pretty good.
For RUN it suggested /usr/bin/systemctl start rather than
alternative methods that don’t work well in my experience.
It also included a remove action.
I don’t use one in my configuration,
but it was a nice reminder that the user might want to do something upon
disconnection of the drive as well (beyond deleting the mount point).
I did find it interesting that the response assumed the linux system in question was using systemd. I have no idea how other init systems work. Perhaps I’ll have to revisit this if I switch to a linux distribution that doesn’t use systemd.
-
I have nothing against this type of connector except I no longer own many cables for this type. I have to be careful not to misplace the few that I have. ↩︎
-
I prefer using the terminal, but I don’t need to format drives often. I should probably find a formatter with a tui so I can retire
gparted. The project tparted exists. I don’t know pascal so I can’t give an informed opinion. ↩︎ -
I have since fixed this by not using rsync’s
--log-fileflag at all. Redirection then works as one would expect. ↩︎