Email Migration: 5,000+ Emails, Proton to Migadu

1. The Setup

I recently migrated my emails from Proton Mail to Migadu after a failed attempt to get myself into the Proton ecosystem and wanted to detail my process, as it was far more painful than expected.

To give some context: I had nearly 5000 messages stored, accounting for around 2.5 GB of space.

Overall, this process would have taken all day had I done it in one sitting, but I decided to break it up and it lasted a couple days before I was able to say that all my messages were stored in my new account.

2. Exporting Messages

To start, I needed to export my messages from Proton Mail. As I am using macOS, I was able to use the Proton Mail Export Tool. However, the downside is that this dumps every single email on your account into a single folder in the .eml format. They also export a JSON file for each message, in the case you're importing back into Proton Mail.

This means that anything in my Inbox, Sent, Archive, Trash, and user-created folders were all dumped out into a single folder with incomprehensible names.

Without a clear path to easily figure out how to re-organize my emails into a new account, I was left a bit annoyed at Proton's export process.

3. Importing Messages

Left with a pile of messages and no way to discern what they were without opening each one, I decided to try and use Thunderbird to import messages into my new Migadu IMAP account.

This led to a dead end as my two methods failed:

1. ImportExportTools NG does not work with my version of Thunderbird (135).
2. Manually dragging the .eml files onto a folder in Thunderbird worked for small batches of files, but seemed to lock up if I tried to import more than a few hundred at a time. It also seemed a bit buggy, as I ended up with many duplicate, and sometimes triplicate, messages.

At this point, I decided to take a step back and use mu, a command-line utility that would index my files and sync back and forth with Migadu for me.

Using my blog post, Email in Doom Emacs with Mu4e on macOS, (and skipping the mu4e parts) I was able to set up a minimal directory connected to my Migadu IMAP account. Using my terminal, I simply moved all of my messages into the mu directory and synchronized the account, and voila, my messages synchronized successfully to the remote server and my other email clients.

However, the remaining issue was that I now had all 5000 messages in the Archive folder and needed to figure out how to organize them back into their proper directories.

4. Organizing Messages Into Folders

As with any problem, I used Python as my hammer to fix the problem. I started by creating the directories required in Thunderbird, fetching them with mbsync so that they appeared in my mu directory, and using Python to organize my messages into the newly-created sub-folders.

4.1. Sent Messages

I started by organizing my Sent messages. This required checking each file for the From header and moving them to the Sent folder.

cd ~/.maildir/migadu/Archive/cur
nano _sent.py

# _sent.py
import os
import glob
import shutil
# Loop through all files in the current folder
for file in glob.glob("*.eml"):
	# Create boolean to check if we should move the file
	move = False
	# Open the current file
		f = open(file, 'r')
	# For each line in file, find the From header
	for line in f:
		if line.startswith("From:"):
			# If we find ourself, mark the message for move
			if "user@example.com" in line:
				move = True
	# Close the file
	f.close()
	# Move the file, if marked for move
	if move == True:
		filepath = os.path.join("/Users/YOUR_USERNAME/.maildir/migadu/Archive/cur/", file)
		new_filepath = os.path.join("/Users/YOUR_USERNAME/.maildir/migadu/Sent/cur/", file)
		shutil.move(filepath, new_filepath)

python3 _sent.py

The only downside to my current approach is that it was the quick and dirty option, so I re-ran it while editing the user@example.com string for each email I wanted to move. If I had wanted to create a more well-defined solution, I would have created an array of addresses to check for and have the if statement check against that array.

Regardless, I was able to run this with the addresses I wanted to move to the Sent folder and was soon finished.

4.2. Archive Sub-Folders

Next, I needed to move the remaining ~3000 messages from the Archive folder into dated sub-folders, organized as such:

• Archive/2016
• …
• Archive/2025

To do this, I followed a similar approach as the method above but check for the Date header instead of the From header.

cd ~/.maildir/migadu/Archive/cur
nano _archive.py

This approach requires finding the X-Pm-Date header and splitting it by the spaces contained within. Once split into a list, we must select the fourth element, as that contains the year which will match the directory we should move it to.

For example, the header X-Pm-Date: Fri, 07 Feb 2025 16:12:08 +0000 will be split into a list as such:

[
    'X-Pm-Date:', # 0
    'Fri,',       # 1
    '07',         # 2
    'Feb',        # 3
    '2025',       # 4
    '16:12:08',   # 5
    '+0000'       # 6
]

From this list, we select the fourth element (2025) and use that to build the destination path.

# _archive.py
import os
import glob
import shutil
# Loop through all files in the sub-folders under Archive
for file in glob.glob("*.eml"):
	# Create boolean to check if we should move the file
	move = False
	# Open the current file
	f = open(file, 'r')
	# For each line in file, find the X-Pm-Date header
	for line in f:
		if line.startswith("X-Pm-Date"):
		# Split the line into a list by spaces;
		# Then select the item that contains the year
			year = line.split(" ")[4]
			move = True
	# Close the file
	f.close()
	# Move the file, if marked for move
	if move == True:
		filepath = os.path.join("/Users/YOUR_USERNAME/.maildir/migadu/Archive/cur/", file)
		new_filepath = os.path.join(f"/Users/YOUR_USERNAME/.maildir/migadu/Archive/{year}/cur/", file)
		shutil.move(filepath, new_filepath)

python3 _archive.py

At this point, we've now moved all Sent messages to the Sent box and organized all messages under the Archive folder into their correct sub-folders.

If you exported other files, such as files from your Inbox, Trash, etc., you could follow a similar approach and determine the best header or attribute to identify them for further organization.

4.3. Synchronize the Results

Before synchronizing the files in their new locations, I needed to remove the characters at the end of the file name since mu appends IDs to the end of file names.

cd ~/.maildir/migadu/Archive
nano _sync_prep.py

This script prepares the Archive sub-folders for synchronization, but the same concept applies to the Sent folder, except you'd replace */cur/* with * if this script were inside the Sent/cur directory.

import glob
import shutil
# Loop through all files in the sub-folders under Archive
for file in glob.glob("*/cur/*"):
	# Remove the characters at the end of the file name created by =mu=
	new_file = file.split(",U=",1)[0]
	# Move the file to the new file name
	shutil.move(file, new_file)

python3 _sync_prep.py

Finally, we can synchronize the results.

mbsync -aV

5. Removing Duplicates

My only remaining issue at the time of writing is identifying and removing duplicate messages. I have toyed with simple Python and command-line solutions to identify duplicate files, but could not get them to effectively define all the duplicates found in any specific directory.

I've even tried using the fclones utility, to no avail. It seems that something in the Proton export, my manual Thunderbird method attempt, or possible sync issues between Thunderbird -> Migadu <-> mu caused duplicates where content within the message has been modified.

Although I now seem to be wasting space and in need of a deduplication tool, I have all of my messages migrated to my new service.

...or, comment on this post on Bubbles!