←back to thread

What I Would Do If I Ran Tarsnap

(www.kalzumeus.com)
801 points tnorthcutt | 3 comments | 03 Apr 14 16:33 UTC | HN request time: 0.428s | source
Show context
Nanzikambe ◴[03 Apr 14 18:06 UTC] No.7524756[source]▶
Interesting article. I'd actually not heard of Tarsnap before, one question (to those who use it), why would a geek use it over:

  tar -cf - / --exclude='/proc/*' --exclude='/dev/*' [..] | \
      xz -z | \
      openssl enc -aes-256-cbc -e -salt | \
      > /mnt/your/networked/google/drive/backup.$(hostname -a).$(date "+%Y%m%d-%H%M%S").aes.tar.xz
I spent a while going through https://www.tarsnap.com/ and I didn't find any flexibility tarsnap offers over it. To make it work unattended, it's trivial to generate a unique key per backup for openssl (use a tmpfs) and then gpg encrypt the key and email it to sys admins or whatever mailing list before killing the tmpfs.

I could understand the appeal to less tech savvy users if there were a gui, or it featured cross platform support beyond those supported by tar, <insert compression tool>, openssl/aespipe/gpg/<insert encryption tool>, or the storage was super cheap.

So what's the value proposition here?

replies(5): >>7524774 #>>7524790 #>>7524804 #>>7524909 #>>7525099 #
tomp ◴[03 Apr 14 18:09 UTC] No.7524774[source]▶
Data deduplication, incremental backups.
replies(3): >>7524873 #>>7525132 #>>7525307 #
Nanzikambe ◴[03 Apr 14 18:20 UTC] No.7524873[source]▶
Heh apologies, my fault for trying to be clever, the mechanism I actually use is incremental and deduplicated. I substituted it for tar to simplify.

I actually use ZFS (filesystem), so my backup flow is closer to:

  TSTAMP="backup-$(date "+%Y%m%d-%H%M%S")"
  zfs snapshot -r $TSTAMP
  zfs send $TSTAMP | \
      xz -z | \
      openssl enc -aes-256-cbc -e -salt | \
      > /mnt/your/networked/google/drive/backup.$(hostname -a).$TSTAMP.aes.tar.xz
The underyling ZFS filesystem is deduplicated at filesystem level, and snapshots are incremental. THere're a few other minor differences (the dest is another ZFS host which syncs to Google drive, and I nuke the local snapshot after send because RAID 1+0 space is more expensive than RAID1 .. )
replies(2): >>7524963 #>>7525927 #
1. Nanzikambe ◴[03 Apr 14 18:29 UTC] No.7524963[source]▶
To answer my own question: deduplication :)

I had not considered multiple backup sources, mine is deduplicated per host, am I to understand tarsnap is deduplicated across all hosts sharing a set of keys?

replies(2): >>7526327 #>>7529217 #
2. beagle3 ◴[03 Apr 14 20:30 UTC] No.7526327[source]▶
I think that's the case.

Note: https://github.com/bup/bup does that too (though it does not encrypt), and http://liw.fi/obnam/ does too (and it does encrypt).

What tarsnap gives you that obnam doesn't is (a) managed cloud storage, (b) tarsnap's history and reputation, and (c) Colin's personal reputation. That's a lot, and it costs money above the S3 storage costs (which you could point obnam at).

3. vbit ◴[04 Apr 14 04:04 UTC] No.7529217[source]▶
Also, easier restore and snapshot deletion.

Consider how you would restore using incremental ZFS snapshots. You'd have to pull all the snaphots, unpack the base snapshot and then sequentially unpack each incremental snapshot.

In tarsnap, the server will compute the 'snapshot' you want for you, and will only send you the data blocks that belong to that snapshot.

In tarsnap, you can also delete any snapshot you want, and only blocks belonging exclusively to that snapshot will be deleted. In your system, deleting a snapshot means you lose all snapshots from that one until the next full snapshot.

Also, in ZFS you're limited to backing up complete datasets, but with tarsnap you can backup any set of files you want.