I’ve been using my Self-Hosted services for quite a while now, very happy with most of them and thinking about completely ditching the other, paid and not open source, applications. But, I ain’t Google or Apple, and I don’t have multiple data centers with several copies of my data! I need something that gives me confidence in running my services without worrying too much about the eventual device failure, or accidental data loss.
The solution is pretty straightforward: running backups! And everybody should do it.
Why Backup Matters
Data loss can occur due to various reasons such as hardware failures, software bugs (please report!), or accidental deletion. Without a backup, our loved pictures or important documents could be lost forever.
Similarly, data corruption can silently make our files unreadable or unusable. This can happen for many reasons and, again, without a backup, there’s usually no way to restore them.
Finally, storage devices have a limited life. Depending on the technology the expected lifetime varies a lot: for instance, an NVME storage device has an endurance that can be measured by the TBW (Terabytes Written). What happens next is that everything is lost, puff.
RAID
Hey, what about setting up RAID instead? For instance, with RAID 1 you have two storage devices with the same data, continuously mirrored.
If you think about it, RAID is not a form of backup: it just doesn’t help against data loss or corruption. It’s instead meant for continuity: if a storage device gets broken, the system can still run without problems until we replace it and restore the normal mirroring condition.
So yes, RAID is good, but not a backup.
Borg, a modern backup solution
Enter Borg Backup, a deduplicating archiver with compression and encryption. Borg works by creating incremental backups, storing only the changes made since the last one. It offers the possibility to compress and encrypt the backup on the host side which makes it also very good in terms of privacy.
It’s written in Python and it’s available for most Linux distros and MacOs.
Automating Backups with Borgmatic and Cron
Borg has its own CLI of course but the list of arguments can get quite long if you want to customize it properly. Some tools can help in this regard:
- Borgmatic (CLI)
- Vorta (GUI)
I’m personally running Borgmatic because in this context I tend to prefer having a simple configuration file and a CLI. I know many people are running Vorta with satisfaction.
Tons of things can be configured, like the target repositories, which files to include and which to exclude. If there are some commands you’d like to run before creating the actual backup and similarly other commands to run after. The encryption and compression methods. The number of daily, weekly, and monthly backups to keep on the repositories. And many other arguments.
I personally have the following simple configuration:
source_directories:
- /srv
- /etc/cron.d/borgmatic
- /etc/borgmatic/config.yaml
- /etc/systemd/resolved.conf.d/adguardhome.conf
repositories:
- path: <MY-REMOTE-REPOSITORY>
label: <MY-LABEL>
exclude_caches: true
exclude_patterns:
- /srv/jellyfin/media
- /srv/transmission/downloads
compression: auto,zstd
encryption_passphrase: <MY-PASSPHRASE>
retries: 5
retry_wait: 5
keep_daily: 7
keep_weekly: 4
keep_monthly: 12
before_actions:
- /srv/pre-backup-all.sh
after_actions:
- /srv/post-backup-all.sh
I’m backing up my services at /srv, some configuration files under /etc and I’m keeping 7 daily, 4 weekly, and 12 monthly backups.
I’m running this script daily, thanks to a simple Cron job.
Where to store such backups
The common rule would be the so-called “3-2-1”:
- 3x copies of your data
- 2x types of storage
- 1x copy offsite
and for now, I’m not following it properly :)
I’m just using an off-site backup server, which is compatible with Borg and it’s called BorgBase. It’s super easy to use, of course, compatible with Borg/Borgmatic and it also offers a 10GB free plan for trying it out.
Conclusion
As the rule says, I should add another copy. It could be either another off-site solution, like rsync.net, or adding an external USB HDD. The latter makes probably more sense in terms of differentiation and price. I’ll eventually update the post.
Finally, you can’t say you have a backup until you successfully restore your backup! Don’t skip this step or everything will be useless. Click here to learn how.
And again, happy hosting!