Difference between revisions of "Cool Solution - Backup Cyrus mailserver"

From Univention Wiki

Jump to: navigation, search
 
(5 intermediate revisions by 2 users not shown)
Line 1: Line 1:
{{Version|UCS=3.0}}
+
{{Version|UCS=4.1}}
{{Cool Solutions Disclaimer}}
+
{{Cool Solutions Disclaimer|Repository=no}}
{{Review-Status}}
+
{{#seo:
 +
|title={{#replace:{{#replace:{{#replace:{{#replace:{{FULLPAGENAME}}|'|'}}|&|&}}|"|"}}|Cool Solution - |}} - {{SITENAME}}
 +
<!--|description=-->
 +
}}
  
 
The UCS mail system is based on the mailserver Cyrus. This article describes how to create a full backup of all relevant data concerning the Cyrus mailserver and how to copy the data to a second mail server, e.g. a failover server, or if the mailserver suffered from a hardware defect. For copying the files, the command line tool ''rsync'' is used.
 
The UCS mail system is based on the mailserver Cyrus. This article describes how to create a full backup of all relevant data concerning the Cyrus mailserver and how to copy the data to a second mail server, e.g. a failover server, or if the mailserver suffered from a hardware defect. For copying the files, the command line tool ''rsync'' is used.
  
Because the Cyrus mailserver saves its files only in irregular intervals a backup of all files is not possible while the mailserver daemon is running. The described procedure in this article aims for the shortest possible downtime.
+
Because the Cyrus mailserver only saves its files in irregular intervals, a backup of all files is not possible while the mailserver daemon is running. The described procedure in this article aims for the shortest possible downtime.
  
This backup will run twice. The first time all noncritical files are copied and in a second run, with a stopped Cyrus mailserver, to copy the latest written data. This article describes how to perfom these steps.
+
This backup will run twice. The first time, all noncritical files are copied and in a second run with a stopped Cyrus mailserver, the latest written data is copied. This article describes how to perfom these steps.
  
 
== Preparations ==
 
== Preparations ==
  
For creating the backup, a separate folder on a device with enough free storage space is required. In this article, the folder  ''/opt/cyrus-backup'' on the mailserver is used. After the backup is completed, the content can be copied to another server or a removable device.
+
For creating the backup, a separate folder on a device with enough free storage space is required. After the backup is completed, the content can be copied to another server or a removable device.
  
 
If a failover mailserver is to be created, a second UCS system must be installed with the mailserver component installed. To ensure the backed-up files are used correctly in the failover server, make sure the same package versions are installed before copying the files to the new server.
 
If a failover mailserver is to be created, a second UCS system must be installed with the mailserver component installed. To ensure the backed-up files are used correctly in the failover server, make sure the same package versions are installed before copying the files to the new server.
Line 19: Line 22:
 
== First rsync-call ==
 
== First rsync-call ==
  
In the first run all Cyrus files, including the mail spool directory, are saved. The folders, that must be copied, can be found in the ''/etc/imapd/imapd.conf'' file. The two directories are stored as the variables ''configdirectory'' and ''partition-default'':
+
In the first run, all Cyrus files, including the spool directory, are saved. The folders that must be copied can be found in the ''/etc/imapd/imapd.conf'' file. The directories are stored as the variables ''configdirectory'', ''partition-default'', ''partition-news'' and ''sievedir''. By default, the directories are the following:
 
 
 
<pre>
 
<pre>
 
configdirectory: /var/lib/cyrus
 
configdirectory: /var/lib/cyrus
 
partition-default: /var/spool/cyrus/mail
 
partition-default: /var/spool/cyrus/mail
 +
partition-news: /var/spool/cyrus/news
 +
sievedir: /var/spool/cyrus/sieve
 
</pre>
 
</pre>
 +
  
 
To save the backup to the local directory, run the following command:
 
To save the backup to the local directory, run the following command:
 
 
<pre>
 
<pre>
rsync -vaR --delete --delete-after /var/lib/cyrus /var/spool/cyrus/mail /opt/backup-cyrus
+
rsync -vaR --delete --delete-after --exclude="jobs" /var/lib/cyrus /var/spool/cyrus <backup path>
 
</pre>
 
</pre>
 +
''Note:'' this command will backup all folders inside ''/var/lib'' and ''/var/spool/cyrus'' excluding the ''/var/spool/cyrus/jobs'' folder for now.
  
 
''rsync'' is also used to save the backup to the failover server. During the copy process, rsync requests the password of the user who copies the files on the remote server. It is adviced to create a SSH key-pair if the backup should run automatically. To copy files to the failover server, rsync must be called twice:
 
''rsync'' is also used to save the backup to the failover server. During the copy process, rsync requests the password of the user who copies the files on the remote server. It is adviced to create a SSH key-pair if the backup should run automatically. To copy files to the failover server, rsync must be called twice:
 
 
<pre>
 
<pre>
rsync -va --delete --delete-after /var/lib/cyrus/ user@failoverserver:/var/lib/cyrus/
+
rsync -va --delete --delete-after /var/lib/cyrus/ <user>@<failover server>:/var/lib/cyrus/
rsync -va --delete --delete-after /var/spool/cyrus/ user@failoverserver:/var/spool/cyrus/
+
rsync -va --delete --delete-after /var/spool/cyrus/ <user>@<failover server>:/var/spool/cyrus/
 
</pre>
 
</pre>
  
Line 42: Line 46:
  
 
When creating the backup, the file ''mailboxes.db'' should be saved separately. This file's content can be recreated by Cyrus, but all information on read mails or marked mails would be lost. If this file is damaged, its content is irrevocably lost. The following command converts the file into a plaintext file:
 
When creating the backup, the file ''mailboxes.db'' should be saved separately. This file's content can be recreated by Cyrus, but all information on read mails or marked mails would be lost. If this file is damaged, its content is irrevocably lost. The following command converts the file into a plaintext file:
 
 
<pre>
 
<pre>
 
su - cyrus -c "/usr/sbin/ctl_mboxlist -d" > /var/lib/cyrus/mailboxlist.txt
 
su - cyrus -c "/usr/sbin/ctl_mboxlist -d" > /var/lib/cyrus/mailboxlist.txt
Line 50: Line 53:
  
 
To restore the file ''mailboxes.txt'' (if needed) at a later time, e.g. after restoring the backup, the following command is used:
 
To restore the file ''mailboxes.txt'' (if needed) at a later time, e.g. after restoring the backup, the following command is used:
 
 
<pre>
 
<pre>
 
su - cyrus -c "/usr/sbin/ctl_mboxlist -u < /var/lib/cyrus/mailboxlist.txt"
 
su - cyrus -c "/usr/sbin/ctl_mboxlist -u < /var/lib/cyrus/mailboxlist.txt"
Line 58: Line 60:
  
 
Because the just copied files can contain inconsistent data, a second run must be performed, but this time the cyrus mailserver must be stopped:
 
Because the just copied files can contain inconsistent data, a second run must be performed, but this time the cyrus mailserver must be stopped:
 
 
<pre>
 
<pre>
 
invoke-rc.d cyrus-imapd stop
 
invoke-rc.d cyrus-imapd stop
 
</pre>
 
</pre>
  
Now the files must be backed up again. rsync will only copy those files that have changed since the last time they were copied. To save the backup to the local directory, run the following command:
+
Now the files must be backed up again. rsync will only copy files that have changed since the last time they were copied. To save the backup to the local directory, run the following command:
 
 
 
<pre>
 
<pre>
rsync -vaR --delete --delete-after /var/lib/cyrus /var/spool/cyrus/mail /opt/backup-cyrus
+
rsync -vaR --delete --delete-after /var/lib/cyrus /var/spool/cyrus <backup path>
 
</pre>
 
</pre>
  
 
When the files are copied, the mailserver can be started again:
 
When the files are copied, the mailserver can be started again:
 
 
<pre>
 
<pre>
 
invoke-rc.d cyrus-imapd start
 
invoke-rc.d cyrus-imapd start
 
</pre>
 
</pre>
  
''Attention'': When copying the data to the failover server, the mailserver daemon should be stopped on both systems to prevent damaging the data.
+
'''Attention''': When copying the data to the failover server, the mailserver daemon should be stopped on both systems to prevent damaging the data.
  
 
== Restoring the backup ==
 
== Restoring the backup ==
Line 82: Line 81:
  
 
First, the mailserver daemon must be stopped:
 
First, the mailserver daemon must be stopped:
 
 
<pre>
 
<pre>
 
invoke-rc.d cyrus-imapd stop
 
invoke-rc.d cyrus-imapd stop
Line 88: Line 86:
  
 
To copy the backup back, the following commands must be executed:
 
To copy the backup back, the following commands must be executed:
 
 
<pre>
 
<pre>
rsync -va --delete --delete-after /opt/backup-cyrus/var/lib/cyrus/ /var/lib/cyrus/
+
rsync -va --delete --delete-after <backup path>/var/lib/cyrus/ /var/lib/cyrus/
rsync -va --delete --delete-after /opt/backup-cyrus/var/spool/cyrus/mail/ /var/spool/cyrus/mail
+
rsync -va --delete --delete-after <backup path>/var/spool/cyrus/ /var/spool/cyrus/
 
</pre>
 
</pre>
  
 
After the files are copied, the mailserver daemon must be started again:
 
After the files are copied, the mailserver daemon must be started again:
 
 
<pre>
 
<pre>
 
invoke-rc.d cyrus-imapd start
 
invoke-rc.d cyrus-imapd start
Line 103: Line 99:
  
 
Because the mail specific UCR variables were already copied to the failover server during its creation, the Cyrus daemon must only be started:
 
Because the mail specific UCR variables were already copied to the failover server during its creation, the Cyrus daemon must only be started:
 
 
<pre>
 
<pre>
 
invoke-rc.d cyrus-imapd start
 
invoke-rc.d cyrus-imapd start
 
</pre>
 
</pre>
 +
 +
== Further Links ==
 +
* UCS Cyrus mail server - https://docs.software-univention.de/cyrus-4.1.html
  
 
[[Category:EN]]
 
[[Category:EN]]
 
[[Category:Administration]]
 
[[Category:Administration]]

Latest revision as of 13:55, 8 September 2017

Produktlogo UCS Version 4.1

Note: Cool Solutions are articles documenting additional functionality based on Univention products. Not all of the shown steps in the article are covered by Univention Support. For questions about your support coverage contact your contact person at Univention before you want to implement one of the shown steps.

Also regard the legal notes at Terms of Service.

The UCS mail system is based on the mailserver Cyrus. This article describes how to create a full backup of all relevant data concerning the Cyrus mailserver and how to copy the data to a second mail server, e.g. a failover server, or if the mailserver suffered from a hardware defect. For copying the files, the command line tool rsync is used.

Because the Cyrus mailserver only saves its files in irregular intervals, a backup of all files is not possible while the mailserver daemon is running. The described procedure in this article aims for the shortest possible downtime.

This backup will run twice. The first time, all noncritical files are copied and in a second run with a stopped Cyrus mailserver, the latest written data is copied. This article describes how to perfom these steps.

Preparations

For creating the backup, a separate folder on a device with enough free storage space is required. After the backup is completed, the content can be copied to another server or a removable device.

If a failover mailserver is to be created, a second UCS system must be installed with the mailserver component installed. To ensure the backed-up files are used correctly in the failover server, make sure the same package versions are installed before copying the files to the new server.

All commands in this article are assumed to be run as user root.

First rsync-call

In the first run, all Cyrus files, including the spool directory, are saved. The folders that must be copied can be found in the /etc/imapd/imapd.conf file. The directories are stored as the variables configdirectory, partition-default, partition-news and sievedir. By default, the directories are the following:

configdirectory: /var/lib/cyrus
partition-default: /var/spool/cyrus/mail
partition-news: /var/spool/cyrus/news
sievedir: /var/spool/cyrus/sieve


To save the backup to the local directory, run the following command:

rsync -vaR --delete --delete-after --exclude="jobs" /var/lib/cyrus /var/spool/cyrus <backup path>

Note: this command will backup all folders inside /var/lib and /var/spool/cyrus excluding the /var/spool/cyrus/jobs folder for now.

rsync is also used to save the backup to the failover server. During the copy process, rsync requests the password of the user who copies the files on the remote server. It is adviced to create a SSH key-pair if the backup should run automatically. To copy files to the failover server, rsync must be called twice:

rsync -va --delete --delete-after /var/lib/cyrus/ <user>@<failover server>:/var/lib/cyrus/
rsync -va --delete --delete-after /var/spool/cyrus/ <user>@<failover server>:/var/spool/cyrus/

Backup of mailboxes.db

When creating the backup, the file mailboxes.db should be saved separately. This file's content can be recreated by Cyrus, but all information on read mails or marked mails would be lost. If this file is damaged, its content is irrevocably lost. The following command converts the file into a plaintext file:

su - cyrus -c "/usr/sbin/ctl_mboxlist -d" > /var/lib/cyrus/mailboxlist.txt

Reloading this file is only needed, when there is a difference in the Cyrus version between the primary mailserver and the restored server. The file is saved in /var/lib/cyrus and will be saved automatically during the second rsync-call.

To restore the file mailboxes.txt (if needed) at a later time, e.g. after restoring the backup, the following command is used:

su - cyrus -c "/usr/sbin/ctl_mboxlist -u < /var/lib/cyrus/mailboxlist.txt"

Second rsync-call

Because the just copied files can contain inconsistent data, a second run must be performed, but this time the cyrus mailserver must be stopped:

invoke-rc.d cyrus-imapd stop

Now the files must be backed up again. rsync will only copy files that have changed since the last time they were copied. To save the backup to the local directory, run the following command:

rsync -vaR --delete --delete-after /var/lib/cyrus /var/spool/cyrus <backup path>

When the files are copied, the mailserver can be started again:

invoke-rc.d cyrus-imapd start

Attention: When copying the data to the failover server, the mailserver daemon should be stopped on both systems to prevent damaging the data.

Restoring the backup

When restoring the data, the same steps are required as during the preparation of the failover server. It is assumed that the backup still resides in /opt/cyrus-backup.

First, the mailserver daemon must be stopped:

invoke-rc.d cyrus-imapd stop

To copy the backup back, the following commands must be executed:

rsync -va --delete --delete-after <backup path>/var/lib/cyrus/ /var/lib/cyrus/
rsync -va --delete --delete-after <backup path>/var/spool/cyrus/ /var/spool/cyrus/

After the files are copied, the mailserver daemon must be started again:

invoke-rc.d cyrus-imapd start

Starting the failover server

Because the mail specific UCR variables were already copied to the failover server during its creation, the Cyrus daemon must only be started:

invoke-rc.d cyrus-imapd start

Further Links

Personal tools