Imapsync
To synchronize imap account “foo” on “imap.truc.org” to imap account “bar” on “imap.trac.org” with foo password “secret1” and bar password “secret2”:
imapsync \ --host1 imap.truc.org --user1 foo --password1 secret1 \ --host2 imap.trac.org --user2 bar --password2 secret2
Description
The command imapsync is a tool allowing incremental and recursive imap transfer from one mailbox to another.
By default all folders are transferred, recursively.
We sometimes need to transfer mailboxes from one imap server to another. This is called migration.
imapsync is a good tool because it reduces the amount of data transferred by not transferring a given message if it is already on both sides. Same headers and the transfer is done only once. All flags are preserved, unread will stay unread, read will stay read, deleted will stay deleted. You can stop the transfer at any time and restart it later, imapsync works well with bad connections.
You can decide to delete the messages from the source mailbox after a successful transfer, it can be a good feature when migrating live mailboxes since messages will be only one side. In that case, use the --delete option. Option –delete implies also option --expunge so all messages marked deleted on host1 will be really deleted. (you can use --noexpunge to avoid this but I don't see any good real world scenario for the combinaison --delete --noexpunge).
You can also just synchronize a mailbox B from another mailbox A in case you just want to keep a “live” copy of A in B. In that case --delete2 can be used, it deletes messages in host2 folder B that are not in host1 folder A.
imapsync is not adequate for maintaining two active imap accounts in synchronization where the user plays independently on both sides. Use offlineimap (written by John Goerzen) or mbsync (written by Michael R. Elkins) for 2 ways synchronizations.
Usage
Installation
Instructions from imapsync creator site https://imapsync.lamiral.info
First check if epel-release is enabled, if not, install it
yum install epel-release
Install imapsync and its dependencies
yum install imapsync
It should now work, to test it you can use a command that show some basic info
imapsync
or test a live server
imapsync --testslive
At the time of writing the epel repository doesn't contain the latest version available, it is suggested to use following process :
after installation of imapsync install more PERL modules required by latest version
yum install perl-App-cpanminus \ perl-Dist-CheckConflicts \ perl-HTML-Parser \ perl-libwww-perl \ perl-Module-Implementation \ perl-Module-ScanDeps \ perl-Package-Stash \ perl-Package-Stash-XS \ perl-PAR-Packer \ perl-Regexp-Common \ perl-Sys-MemInfo \ perl-Test-Fatal \ perl-Test-Mock-Guard \ perl-Test-Requires \ perl-Test-Deep \ perl-File-Tail \ perl-Unicode-String cpanm Encode::IMAPUTF7 # this module is not available as a rpm package
now download latest imapsync
wget https://imapsync.lamiral.info/imapsync
set it executable
chmod +x imapsync
and test it
./imapsync --testslive
Command Example
to migrate email with the password of your users, put the password of you users in two files, pass1 and pass2.
imapsync --noauthmd5 --host1 localhost --port1 993 --ssl1 --user1 yourUser --passfile1 /root/pass1 \ --host2 yourRemoteIP --port2 993 --ssl2 --delete2 --user2 yourUserRemote --passfile2 /root/pass2
if you don't know the password of you user, and you are migrating from two NS that are both under your administration, you can use the master user of dovecot where the admin is root in NS (bear in mind to activate the option 'Root can log in as another user' in the /email/Mailboxes panel of nethgui). Of course you still need to write the pass file the password of your root user.
imapsync --noauthmd5 --host1 localhost --port1 993 --ssl1 --user1 yourUser*root --passfile1 /root/pass1 \ --host2 yourRemoteIP --port2 993 --ssl2 --delete2 --user2 yourUserRemote*root --passfile2 /root/pass2
* folder exclusion
You might need to exclude the junk folder for example
imapsync --noauthmd5 --host1 localhost --port1 993 --ssl1 --user1 yourUser --passfile1 /root/pass1 --exclude "^Junk$" \ --host2 yourRemoteIP --port2 993 --ssl2 --delete2 --user2 yourUserRemote --passfile2 /root/pass2 --exclude "^Junk$"
you can use several time the `–exclude` command
For a massive migration could be easier and less prone to mistakes to create two files, one executable with command and switches (sync-lookup.sh) and the second only a text file (file.txt) with user, hosts and passwords ( never tried to use with it a password file as example above)
example of sync-lookup.sh
#!/bin/sh # # $Id: sync_loop_unix.sh,v 1.8 2018/02/12 21:53:40 gilles Exp gilles $ # Example for imapsync massive migration on Unix systems. # See also http://imapsync.lamiral.info/FAQ.d/FAQ.Massive.txt # # Data is supposed to be in file.txt in the following format: # host001_1;user001_1;password001_1;host001_2;user001_2;password001_2; # ... # Separator is character semi-colon ";" it can be changed by any character changing IFS=';' # in the while loop below. # # Each line contains 6 columns, columns are parameter values for # --host1 --user1 --password1 --host2 --user2 --password2 # and a trailing empty fake column to avoid CR LF part going # in the 6th parameter password2. Don't forget the last semicolon. # # You can add extra options after the variable "$@" # Use character backslash \ at the end of each supplementary line, except for the last one. # You can also pass extra options via the parameters of this script since # they will be in "$@" # The credentials filename "file.txt" used for the loop can be renamed # by changing "file.txt" below. echo Looping on account credentials found in file.txt echo { while IFS=';' read h1 u1 p1 h2 u2 p2 fake do { echo "$h1" | tr -d '\r' | egrep '^#|^ *$' ; } > /dev/null && continue # this skip commented lines in file.txt echo "==== Starting imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ====" imapsync --host1 "$h1" --user1 "$u1" --password1 "$p1" \ --host2 "$h2" --user2 "$u2" --password2 "$p2" \ "$@" echo "==== Ended imapsync from host1 $h1 user1 $u1 to host2 $h2 user2 $u2 ====" echo done } < file.txt example of file.txt <file> host1;user1;pass1;host2;user2;pass2; host1;userB;pass1B;host2;userB;pass2B;
Remember that with this method you MUST put switches on file sync-lookup-unix.sh and NOT on file.txt.
For the same reason you cannot use it for a migration from multiple mail server with different user access configuration
Security
You can use --passfile1 instead of --password1 to give the password since it is safer. With --password1 option any user on your host can see the password by using the 'ps auxwwww' command. Using a variable (like $PASSWORD1) is also dangerous because of the 'ps auxwwwwe' command. So, saving the password in a well protected file (600 or rw-------) is the best solution.
Sync 2 servers
--delete1 or --delete2 is used when you want to synchronise the deleted or moved emails
Options
imapsync [options]
To get a description of each option just run imapsync like this:
imapsync --help imapsync
The option list:
imapsync [--host1 server1] [--port1 <num>] [--user1 <string>] [--passfile1 <string>] [--host2 server2] [--port2 <num>] [--user2 <string>] [--passfile2 <string>] [--ssl1] [--ssl2] [--tls1] [--tls2] [--authmech1 <string>] [--authmech2 <string>] [--proxyauth1] [--proxyauth2] [--domain1] [--domain2] [--authmd51] [--authmd52] [--folder <string> --folder <string> ...] [--folderrec <string> --folderrec <string> ...] [--include <regex>] [--exclude <regex>] [--prefix2 <string>] [--prefix1 <string>] [--regextrans2 <regex> --regextrans2 <regex> ...] [--sep1 <char>] [--sep2 <char>] [--justfolders] [--justfoldersizes] [--justconnect] [--justbanner] [--syncinternaldates] [--idatefromheader] [--syncacls] [--regexmess <regex>] [--regexmess <regex>] [--maxsize <int>] [--minsize <int>] [--maxage <int>] [--minage <int>] [--skipheader <regex>] [--useheader <string>] [--useheader <string>] [--nouid1] [--nouid2] [--usecache] [--skipsize] [--allowsizemismatch] [--delete] [--delete2] [--expunge] [--expunge1] [--expunge2] [--uidexpunge2] [--delete2folders] [--delete2foldersonly] [--delete2foldersbutnot] [--subscribed] [--subscribe] [--subscribe_all] [--nofoldersizes] [--nofoldersizesatend] [--dry] [--debug] [--debugimap][--debugimap1][--debugimap2] [--timeout <int>] [--fast] [--split1] [--split2] [--reconnectretry1 <int>] [--reconnectretry2 <int>] [--noreleasecheck] [--pidfile <filepath>] [--tmpdir <dirpath>] [--version] [--help] [--tests] [--tests_debug]