Backing up with Blazeback B2 and Restic

For a long while I had managed my backups with some combination of a remote server with ssh capabilities, and either duplicity or rsync (depending on where and type of data). These days there are so many better options. With the innovation of better storage (eg. cold storage), and competition coming out against s3, we have much better options to chose from.

History

For a long while my go-to backup solution was a small VM with a large drive on backupsy, and running duplicity against that. This cost me about $6/mo.

What really pushed me over the edge was their incredibly poor handling of a cluster failure. It happens, I expect my backups to have a certain amount of non-reliability. I don't even care if I lose all my data on a backup (though I'd prefer if it didn't happen). What put me over the edge was the lack of communication, lack of reaching out, and lack of transparency when I reached out to them when my VM suddenly stopped working.

So, it was time to move on..

Switching to Blazeback B2 and Restic

There are two parts of the backup puzzle:

• Somewhere to host the data
• The software to securely backup the data

Picking Restic

When I decided I was going to switch providers, I figured it would also be a good time to re-evaluate how I was doing the backup itself. I had been using duplicity for a while, but it was old, so I suspected there was a newer and better solution. My main goals were: 1) Do client-side encryption so that not even the backup provider would understand the data, 2) work with whatever provider I chose with minimal effort on my part, and 3) It needed to support snapshotting over time.

After a lot of searching, I landed on restic. It supports many different backends (s3, b2, sftp, local.. to name a few), and does client-side encryption. It can even mount a filesystem using fuse to be able to view a repo normally. Another big feature I was looking for was the ability to snapshot over time, but prune older copies in a degrading way. It had support for all of that.

The only downside is that it stores the files in an internal repository-style format, meaning that in the worst case scenario I wouldn't be able to decrypt a single directory on my own to get the files back. Given all of the pros, however, and the mounting capability, I decided the pros outweighed the cons.

Picking Blazeback B2

When I started looking for storage solutions, I really only looked at two: AWS S3, and Google cloud storage. I had done some googling to see what other folks had chosen for their backup needs, and blazeback b2 came up a lot, so I decided to check it out.

First off, after using it for several months, it's fantastic. They charge for storage and transactions (their cost calculator), but after running restic across 5 servers, daily, and storing 100 GB/mo, I'm only paying $0.50/mo. Their interface is pretty simple, but I don't really need anything fancy for what is ultimately an object-store. I just signed up, created a bucket, set up a bucket key, and away I went!

It's also worth noting that for Mac and PC users, they have their own dedicated backup client.

Putting it all together

Once I made the decisions, and signed up for an account, it was just a matter of dropping a script in the right place, and away we went!

Below is a backup script I drop on all my servers to back up the entire system. Tweak the SOURCE config if you only want to back up a certain path.

File: /etc/cron.daily/backup (Make sure to chmod +x /etc/cron.daily/backup)

#!/bin/bash
set -e

# Download and install restic:
# curl -L -o- https://github.com/restic/restic/releases/download/v0.9.3/restic_0.9.3_linux_amd64.bz2 | bunzip2 > /bin/restic && chmod gou+x /bin/restic

# Configuration
export B2_ACCOUNT_ID=xxx
export B2_ACCOUNT_KEY=yyy
export RESTIC_PASSWORD=zzz
export RESTIC_REPOSITORY=b2:bucket-name:subdir
export SOURCE=/


logger Beginning backup...

export GOGC=20 # Limit memory usage

# restic init

restic backup -v \
        --exclude /mnt --exclude /tmp --exclude /dev --exclude /sys --exclude /proc \
        --exclude-if-present .backup-ignore \
        --one-file-system \
        --limit-upload 512 \
        $SOURCE

# We should be the only ones using this, and sometimes backup above won't
# release in time, so this forces an unlock before next step
restic unlock

restic forget --prune --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --limit-upload 512

logger Backup finished