Using rsnapshot with systemd

rsnapshot nicely illustrates some of the features of systemd timers. As of Ubuntu 16.04, the rsnapshot package doesn't ship with systemd service and timer files, so you have to set them up yourself.

Before I show how to setup rsnapshot with systemd, I want to cover some of the benefits of using systemd timers instead of cron.

Cron syntax vs systemd timer syntax

systemd timer has a syntax for specifying repeating events than the old cron syntax.

Here's how to specify a "weekly" cron job at 3 AM with the old cron syntax:

 # cron syntax
 0  3 * * 1

Here's the same thing expressed in a systemd timer file:

 *-*-* 00:03:00

If you correctly guess that the right side in the systemd format represents Hour:Minute:Day, then you are well on your way to correctly guessing that the left side represents 'Year-Month-Day' format.

The old cron syntax looks nothing like a date and has to be memorized.

Logging and notification

A traditional well-behaved cron job produces no output unless there is an error, in which case the output will be mailed to you.

Because rsnapshot runs an important backup job, it's useful to have some detailed output about what it did, even on successful runs.

With systemd, output is sent to the systemd journal by default. You don't hav to set up any output redirection or log rotation just to have good logging. Once set up, you can get logs for any of your rsnapshot runs with syntax like this:

journalctl -u rsnapshot@daily

If your login use in the systemd-journald group, you also don't have to be root just to check out the logs. You can still get a nicely formatted email if rsnapshot fails. More on that below.

Easy status reporting

systemd makes it easy to check on the status of your cron job. When did it run last? Was the last run a success or a failure? What were the most recent logs?.  All those questions can answered with a quick check of the status command:

systemctl status rsnapshot@daily

You don't have to root to check that either.

Configuring systemd unit files for snapshot

Disable cron files

To avoid conflict, make sure you disable running rsnapshot via cron in /etc/cron.d/rsnapshot

One time Setup: email-on-failure

In Ubuntu 16.04, there's a not a built-in solution for getting email-on-mail when using systemd timers instead of cron. This is the one drawback I've found. However, once you set this up, you can re-use the solution for all your other systemd timers.

The following assumes you've already got outgoing email working on your box.

First, install a script which will mail diagnostic output for a failed systemd service ((more context](://bugs.launchpad.net/ubuntu/+source/systemd-cron/+bug/1583743)):

sudo wget -O /usr/local/bin/mail_on_failure https://raw.githubusercontent.com/systemd-cron/systemd-cron/master/src/bin/mail_on_failure.py 

Next, install a service file that will be used to call this scirpt when another service fails. Name it like: /etc/systemd/system/status-email-admin@.service

The contents would be:

[Unit]
Description=status email for %I to admin

[Service]
Type=oneshot
ExecStart=/usr/local/bin/mail_on_failure %i
User=nobody
Group=systemd-journal

Now for any service file that you want to send mail-on-failure, you'll include these bits in your systemd .service file:

[Unit]
# Like Cron would, send devops an email if this fails
OnFailure=status-email-admin@%n.service

[Service]
Environment="MAILTO=admin@example.com"

As you would for cron, you can customize the MAILTO environment variable to suit your needs for each service file.

rsnapshot systemd service file

Using a systemd template file, a single file can be used to configure rsnapshot runs for daily, weekly and monthly.

Create /etc/systemd/system/rsnapshot@.service with the following contents.

[Unit]
Description=rsnapshot (%I) backup
# Like Cron would, send an email if this fails
OnFailure=status-email-admin@%n.service

[Service]
Environment="MAILTO=admin@rideamigos.com"
Type=oneshot
Nice=19
IOSchedulingClass=3
ExecStart=/usr/bin/rsnapshot %I

Creating rsnapshot timer files

Now you can create timers for the daily, weekly, monthly rsnapshot runs. In the example files I've set the daily task to run at 5:30 AM, the weekly task at 4:30 AM and the monthly task at 3:30 AM. Adjust to suit.

/etc/systemd/systemm/rsnapshot-daily.timer

[Unit]
Description=rsnapshot daily backup

[Timer]
# Run every day at 5:30 AM local time
OnCalendar=05:30
Persistent=true
Unit=rsnapshot@daily.service

[Install]
WantedBy=timers.target

/etc/systemd/systemm/rsnapshot-monthly.timer

[Unit]
Description=rsnapshot monthly backup

[Timer]
# Run once per month at 3:30 local time, before daily and weekly runs
OnCalendar=*-*-1 03:30:00
Persistent=true
Unit=rsnapshot@monthly.service

[Install]
WantedBy=timers.target

/etc/systemd/systemm/rsnapshot-weekly.timer

[Unit]
Description=rsnapshot weekly backup

[Timer]
# Run once per week on Monday at 4:30 local time, before daily
OnCalendar=Monday  *-*-* 04:30:00
Persistent=true
Unit=rsnapshot@weekly.service

[Install]
WantedBy=timers.target

Enabling timers

Finally, you can enable the timer files you've created so that they'll start running the next time the schedule dictates.

 systemctl enable rsnapshot-daily.timer
 systemctl enable rsnapshot-weekly.timer
 systemctl enable rsnapshot-monthly.timer

The future

If this all seems more complex that it should be for scheduling rsnapshot backups, it is. Both rsnapshot and your OS distribution could offer better support for systemd and sending-email-on-failure with systemd. If they did, you could get benefits using systemd timers with rsnapshot by using setting your MAILTO environment and running systemd enable for the timers you wanted to turn on.

If this appeals to you, considering communicating with the rsnapshot project and possibly submitting a pull request to improve support here.