to migrate mailinabox backups from aws to scaleway

There's been a neat resource making the rounds in my socials recently that lists a bunch of European alternatives to popular cloud-adjacent tooling. I was browsing it the other day, clicked the AWS category because it's at the top and I'm basic like that, and had a minor epiphany:

wait couldn't I use one of these s3-compatible object storage services for my @MailInABox@fosstodon.org backups?

My backup...uh, let's call it a strategy...is a little haphazard, but my Mailinabox has been on top of things since the beginning: incremental backups encrypted with duplicity push to an AWS S3 bucket that costs me maybe a dollar and a half every month. Entirely automated, entirely out of my hair. (Yes, I've restored from backup. Twice even.)

A dollar and a half isn't much, but I'd love to give it to somebody other than Jeff Bezos. I kept clicking around, looked at some pricing, read some reviews, and decided to try moving everything to Scaleway. They've got a nice GUI for when I'm lazy, and nice documentation for setting up aws-cli.

first things first

In addition to the Mailinabox backups, my S3 buckets are the final resting place of some old backups for other services I'm not using anymore. I could probably delete them and never notice, but I'm sort of a digital hoarder and I never learned how to delete anything without first making sure I wouldn't miss it, so they're coming to Scaleway too.

I created my buckets in the Scaleway web console, then swapped over to aws-cli⁰. I couldn't figure out how to get it to move objects between profiles directly, so I used my local /tmp for some downloading-and-uploading:

# downloading from aws, using the default aws-cli profile
$ aws s3 \
  cp s3://old-bucket /tmp/old-bucket \
  --recursive
$ aws s3 \
  cp s3://another-old-bucket /tmp/another-old-bucket \
  --recursive
# uploading to scaleway
$ aws --profile scaleway s3 \
  cp /tmp/old-bucket s3://old-bucket \
  --recursive
$ aws --profile scaleway s3 \
  cp /tmp/another-old-bucket s3://another-old-bucket \
  --recursive

I realized belatedly that these should all be Glacier-class objects, since I don't need to access or update them (and I should absolutely be punished if I do those things). There's a --storage-class flag that I probably could have used during the upload; lucky for me, it works to modify existing objects too:

$ aws --profile scaleway-nl s3 \
  # yeah, the semantics are odd
  cp s3://old-bucket/backups s3://old-bucket/backups \
  --recursive --storage-class GLACIER

(If you have version control turned on, that kind of operation might make weird stuff happen? But I did not, so it was all fine.)

slightly more fraught things second

Mailinabox does daily backups, so first I made sure I wasn't anywhere close to that automated process:

$ cat /etc/cron.d/mailinabox-nightly
# Mail-in-a-Box --- Do not edit / will be overwritten on update.
# Run nightly tasks: backup, status checks.
44 3 * * *      root    (cd /home/gemma/mailinabox && management/daily_tasks.sh)

Three in the morning. No problem.

I did one more backup, just for peace of mind, following the instructions in the docs:

$ cd mailinabox
$ sudo -E management/backup.py

and then copied the AWS bucket over to the Scaleway bucket (keeping the same directory structure):

$ aws s3 \
  cp s3://mailinabox/backups /tmp/mailinabox/backups \
  --recursive
$ aws --profile scaleway s3 \
  cp /tmp/mailinabox/backups s3://mailinabox/backups \
  --recursive

After that, the moment of truth: Updating the Mailinabox settings. I'd already done a little digging through the forums to see if anybody else was using Scaleway, and found this helpful neighbor:

[I]f your Scaleway endpoint [in the bucket settings] is provided as https://bucketname.s3.fr-par.scw.cloud, and you’re backing up to a folder called mail within that bucket, your MIAB settings should be:

• S3 Region: Other
• S3 Host / Endpoint: s3.fr-par.scw.cloud (notice it does not include the bucket name)
• S3 Path: bucketname/mail

I also set up a new IAM account and configured a policy for it¹. No need to splash my root creds around willy nilly. Mailinabox saved my new config, refreshed the page, and...

It's at least successfully reading from the bucket! Another manual backup to see if anything breaks:

I'll let the whole thing stew for a few days, and then probably axe the AWS side this weekend. Buh-bye, Bezos².

#mailinabox #selfhost #smallweb #eu

⁰ The Scaleway GUI told me to use the low-level s3api sub-command, but s3 worked just fine.

¹ This policy adventure got a little weird: I tried to set up a bucket-specific policy allowing the new account access to only my mailinabox bucket, but that policy didn't include the ListBuckets permission. I tried to add a ListBuckets permission using the regular policy editor, but the options aren't that granular, and ObjectStorageBucketsRead didn't cover it. I couldn't find a JSON editor at that scope, so I ended up scrapping the bucket policy entirely and just giving the account access to all the buckets. It's fine. Thrilled to discover that Scaleway has ported over IAM's inscrutability along with the S3 interface.

² i'm sorry. not really. maybe a little.