We[1] built borg into our environment[2] as soon as it was stable, release software. In the years since, it has (ironically) supplanted use of rsync as the de facto standard that our users back up to us with.
As one of our users have said[2], borg is "the holy grail" of backups as it does everything rsync always did, and produces remotely encrypted backups that the provider has zero insight into.
It also does not have the inefficiencies that the older, duplicity software has.
If you are willing to go without (borg specific) technical support and do your retention with borg instead of our zfs snapshots, there is a special, discounted rate available.[4]
Borg is definitely not the holy grail of backups, unless one has very loose requirements.
Desktop backup solutions have typically a set of functionalities to compare against (block-based, safe on untrusted repositories, with realtime backup, compression-efficient, and optionally, with a functional GUI), that show how open source solutions are strangely lacking, each in a way or another.
> produces remotely encrypted backups that the provider has zero insight into.
> in a multiple-client scenario a repository can trick a client into reusing counter values by ignoring counter reservations and replaying the manifest
Borg also has no efficient compression, as it doesn't use multithreading. There are multiple GitHub issues on this subject; I've just checked, and they're closed.
> Borg also has no efficient compression, as it doesn't use multithreading. There are multiple GitHub issues on this subject; I've just checked, and they're closed.
Depending on your use model, this may or may not affect you. Whether you use multiple threads to compress a single stream or not probably matters a lot less if you have 15-30 backups going on at the same time.
Nice to see someone offering public borg hosting! I have two security questions:
(1) Are you doing any hardening of the borg remote side? The server is pretty complex and it has lots of code paths which can be exercised by a potentially untrusted client.
(2) Is there a way to protect compromised clients (like a cryptolocker)? borg offers "append only" mode, but it does not allow pruning (obviously). And trying to run "prune" using separate trusted connection will commit untrusted changes as well.
inefficiencies that the older, duplicity software has.
As much as I love duplicity, it does sort of suck when you get into larger numbers of files and gigabytes of data. It's so darn slow sometimes it's almost unusable. Good to know borg is better, I've been meaning to check that out!
Duplicity and Borg use two very distinct technologies under the hood - duplicity does differential partial backup archives (which all require a full backup AND a full list of all previous differential archives) and Borg uses deduplication. So yes it is much more efficient in many ways, not simply because it is newer and somehow is more efficiently coded, it is so because it uses different technology and backup model.
I've been using borg to backup small things, like schoolwork, config files, mail, and code, to rsync.net for ~1 year now, and can vouch that it's pretty simple to setup. If you've got a simple use case like I do, the example config in the borg tutorial is pretty much all you need. I've got it set to automatically run as a cron job, and I haven't had to touch it except to occasjionally do sanity checks that everything is being back up correctly.
If your customers back up multiple servers, are they still under the same rsync.net account and could they see each other's data?
This is one of the issues I aimed to solve with BorgBase.com[1]: every single repo is its own backup user and can't see other repos. This separation allowed to add some Borg-specific features, like append-only mode, monitoring for stale backups or using a specific Borg version.
rsync.net has "subaccounts" which allow you to control side-by-side backup directories using standard UNIX permissions and ownerships. That is, if the default configuration is not exactly what you'd like.
Subaccounts even get their own .ssh/authorized_keys file.
It's all very unix centered and command-line focused. There's no web interface for features like this.
I see that you offer "append only mode" as a core feature. How do you deal with pruning of old backups in this case?
Regular borg only offers two options: either pruning does not decrease space used at all; or pruning permanently commits changes, negating any security advantages. Both of those seem bad enough to prevent offering "append-only" commerically?
In my experience pretty reliable. I've not seen major downtime and bandwidth will depend entirely on routing to you in particular (FWIW my ISP peers with Hetzner directly and congestion hasn't been an issue, I can typically saturate my 1Gbps connection at any time of day).
> borg is "the holy grail" of backups as it does everything rsync always did
I tried borg for some time for full backups from / and there were 3 things I didn't like about it:
1. Mounting and navigating a snapshot was extremely slow. Like wait a few seconds for any `ls` to finish.
2. It needed the encryption password for many things I think it shouldn't.
3. The resulting backups are very opaque.
I've since switched to making my backups by rsync'ing to btrfs subvolumes. I snapshot the previous backup and rsync the differences to the new one. I get to navigate each snapshot with the same snappiness as any other directory in the filesystem. btrfs can also instantly tell me which files differ between 2 snapshots, no matter how huge they are. I don't remember all the options that borg offers on synching, but I doubt it has the same breadth of options as rsync. So for me, rsync + COW filesystem > borg.
I am a little concerned with my use of btrfs, but I'm planning to do the same thing using ext4's reflinks for some disk and filesystem redundancy.
How do I get rotation schemas to work with borg? Say I want to keep the last 3 daily snapshots, one per week for the last 4 weeks, and one per month for the last 6 months...
I understand this is kinda hard with "zero knowledge" encryption, but it is possible and not a wild feature to request from the self-named holy grail of backups soft.
Yes, but which server will run the purge command? It cannot be the same server as the one you are backuping, since one of the goal of the backup is to protect the server if it is compromised.
You can run command from inside your backup script, from the client. It needs the encryption key anyway to add backups, this way it is just one extra command that does all the pruning (quite quickly as well). And you don't have to worry about full/incremental backups, nor really about any "rotation" or managing them manually, it's all deduplicated and Borg just keeps X number of daily/weekly/whatever snapshots (like duplicacy, like restic, like bup, etc. etc.).
Yes, I got this. But if your server is compromised, then the attacker can erase your backups, which ruins the point of doing backups in the first place. This is why there is an append-only mode.
What I do is to have two different SSH keys. The one that is used by the backup script is --append-only. The one that I keep locally on my notebook has full access, so I `borg prune` from there when I see the disk usage reaching critical levels.
Don't rely on append-only mode in Borg, it does not work really as advertised. Every transaction in that mode is recorded in a log, which if you are very careful you can go back into (but even reading or monitoring it is a PITA, the "interface" requires you to carefully track id's of files and check with dates etc).
And being careful is difficult, because in Borg, once you do any other backup command from a full access account (for example to do pruning) - it will automatically, no warning, go through the log and apply it. You should really really read up on that functionality first before relying on it, the way Borg has implemented it is close to anti-feature.
Regarding compromisation of strictly the server itself, I believe there are commands to check the state of the repository? Isn't that enough?
If we are still talking about rsync.net, and an attacker gains access to your account and deletes your borg archive, you can still restore your backup from the rsync.net ZFS snapshots.
The snapshots are not deletable even with your full credentials.
As one of our users have said[2], borg is "the holy grail" of backups as it does everything rsync always did, and produces remotely encrypted backups that the provider has zero insight into.
It also does not have the inefficiencies that the older, duplicity software has.
If you are willing to go without (borg specific) technical support and do your retention with borg instead of our zfs snapshots, there is a special, discounted rate available.[4]
[1] rsync.net
[2] https://news.ycombinator.com/item?id=17408624
[3] https://www.stavros.io/posts/holy-grail-backups/
[4] https://www.rsync.net/products/borg.html