developing-human --help

Shares information on new learnings, in hopes that it will be useful to others. Perhaps even to you.

Topics revolve around programming, but not exclusively.

Usage: developing-human posts | projects | readings

developing-human posts btrfs-backups

Incremental backups with BTRFS

created: 2025-12-19

As part of installing a new distro on my laptop and moving my wife's computer over to linux, I've been exploring a more robust backup strategy. I landed on using BTRFS snapshots which are backed up to a basement server. Here I'll walk through how I setup automated backups using BTRFS, snapper, and snbk.

This post is the guide I wish I had been able to find when I started down this rabbit hole. It also serves as a guide for my future self.

Why BTRFS takes snapshots efficiently

B-TRee File System (BTRFS) is a Copy on Write file system, which means it doesn't overwrite data when it changes. Instead, it writes the changed data to a new block and the updated file includes the new block. If a previous snapshot references the same file, the file from the snapshot will keep using the old block.

This has two nice properties:

  1. It's extremely fast to take a snapshot, because no data has to be copied.
  2. Snapshots don't take up much storage space, because data can be shared. It takes up extra drive space based on how much data has changed since the snapshot was taken.

Subvolumes

Subvolumes are how BTRFS divides a partition. To most of the system they appear as directories. As an example, lets make one named jared and see that it looks like a directory:

$ btrfs subvolume create jared
Create subvolume './jared'

$ ls -l
total 0
drwxr-xr-x 1 me me 0 Dec 18 16:33 jared

But the filesystem knows jared is a subvolume:

$ sudo btrfs subvolume list .
ID 256 gen 25186 top level 5 path @
ID 257 gen 25186 top level 5 path @home
...snip (mostly snapshots)...
ID 854 gen 25183 top level 257 path me/example/jared

Subvolumes like jared may look like directories, but they:

  1. Can be mounted on boot via fstab, similar to a partition.
  2. Can be copied by taking a snapshot.
  3. Act as a barrier when taking other snapshots. For example, snapshotting / does not snapshot /home. In practice this means I could rollback / to last week without losing a week of data from /home.

Manually taking snapshots

To show that snapper and snbk aren't magic, lets walk through how to take snapshots and transfer them manually. Then we'll dig into automating it.

btrfs subvolume snapshot is used to take a snapshot. When making changes to jared, they won't be reflected by original-jared, but the data for hello.txt is shared on disk. These commands create a few files and take read only snapshots.

$ echo "world" > jared/hello.txt
$ btrfs subvolume snapshot -r jared original-jared
$ echo "earth" > jared/greetings.txt
$ btrfs subvolume snapshot -r jared other-jared
$ tree .
.
├── jared
│   ├── greetings.txt
│   └── hello.txt
├── original-jared
│   └── hello.txt
└── other-jared
    ├── greetings.txt
    └── hello.txt

Manually transferring to another BTRFS system

When copying a series of snapshots to another BTRFS system, it's important the shared data between subvolumes is preserved. This is the difference between copying every byte in the subvolume and copying only what has changed since the last snapshot. To make this easy, btrfs send is used to create the diff and btrfs receive can write it.

I formatted a drive with BTRFS on darius, my basement server, to setup backups. To copy the data over we'll unfortunately need to be root on both the client and the server, due to btrfs send and receive both requiring root permissions. SSH keys need to be setup, but as this post is getting plenty long I'll refer you to this thread for more directions on setting up SSH in this context.

From the client, we can send the snapshots over ssh. First this sends original-jared, then other-jared is sent with the -p (parent) flag specifying it's parent is original-jared. This lets the backup be incremental.

# btrfs send original-jared | ssh -i /root/.ssh/id_btrfs_backup darius "btrfs receive /data/backups/example"
At subvol original-jared
At subvol original-jared

# btrfs send -p original-jared other-jared | ssh -i /root/.ssh/id_btrfs_backup darius "btrfs receive /data/backups/example"
At subvol other-jared
At snapshot other-jared

Then from darius, we can see the data is copied and the snapshots exist.

me@darius $ tree /data/backups/example
/data/backups/example
├── original-jared
│   └── hello.txt
└── other-jared
    ├── greetings.txt
    └── hello.txt
	
me@darius $ sudo btrfs subvolume list /data/backups
ID 256 gen 10 top level 5 path data
ID 257 gen 2141 top level 256 path data/backups
... snip (other snapshots)
ID 856 gen 2130 top level 257 path example/original-jared
ID 857 gen 2133 top level 257 path example/other-jared

Automating snapshots with snapper

snapper is a tool which automates taking snapshots for BTRFS subvolumes. I use it to automatically take hourly snapshots of / and /home and rotate which are kept. I keep the last few hourly snapshots, then one per day for the past week.

This is useful for rolling back after a botched upgrade or similar, but it is not (yet!) a backup. If your drive dies, you've still lost data.

Snapper manages a "config" per subvolume to take snapshots of. Since the default config is root, we'll be specifying which config to use with -c jared in most commands. Let's create a config for jared:

$ sudo snapper -c jared create-config jared
$ ls /etc/snapper/configs
home  jared  root

This created a config file at /etc/snapper/configs/jared. I mostly leave defaults, but I reduce the number of snapshots it keeps via the TIMELINE_LIMIT_* options because the defaults seem high to me.

A snapshot can be created manually with:

$ sudo snapper -c jared create

We can see this snapshot with:

$ sudo snapper -c jared list
# │ Type   │ Pre # │ Date                            │ User │ Cleanup │ Description │ Userdata
──┼────────┼───────┼─────────────────────────────────┼──────┼─────────┼─────────────┼─────────
0 │ single │       │                                 │ root │         │ current     │
1 │ single │       │ Fri 19 Dec 2025 10:33:10 AM EST │ root │         │             │

But what this really did was create a jared/.snapshots subvolume, with number snapshots stored inside it:

$ sudo btrfs subvolume list . | grep /jared
ID 854 gen 25651 top level 257 path me/example/jared
ID 876 gen 25652 top level 854 path me/example/jared/.snapshots
ID 877 gen 25649 top level 876 path me/example/jared/.snapshots/1/snapshot

There's no magic here, but it's very convenient for automating the snapshots.

Automating backups with snbk

snbk is installed alongside snapper and is used to automate backing up snapshots to another location. There isn't a command to create a config automatically, but there are examples here. For jared, I created:

$ cat /etc/snapper/backup-configs/jared.json
{
    "config": "jared",
    "target-mode": "ssh-push",
    "automatic": true,
    "source-path": "/home/me/example/jared",
    "target-path": "/data/backups/example/jared",
    "ssh-host": "darius",
    "ssh-user": "root",
    "ssh-identity": "/root/.ssh/id_btrfs_backup"
}

We can confirm snbk picks it up with:

$ snbk list-configs
Name  │ Config │ Target Mode │ Automatic │ Source Path              │ Target Path                       │ SSH Host │ SSH User │ SSH Identity
──────┼────────┼─────────────┼───────────┼──────────────────────────┼───────────────────────────────────┼──────────┼──────────┼───────────────────────────
jared │ jared  │ ssh-push    │ yes       │ /home/me/example/jared │ /data/backups/example/jared       │ darius   │ root     │ /root/.ssh/id_btrfs_backup
...others...

And see that there are now two snapshots which are not yet pushed to darius:

$ sudo snbk -b jared list
# │ Date                            │ Source State │ Target State
──┼─────────────────────────────────┼──────────────┼─────────────
1 │ Fri 19 Dec 2025 10:33:10 AM EST │ read-only    │
2 │ Fri 19 Dec 2025 11:00:05 AM EST │ read-only    │

They can be pushed with transfer-and-delete which transfers any snapshots which are missing on the target, and deletes any snapshots on the target which no longer exist locally:

$ sudo snbk -b jared transfer-and-delete
Running transfer and delete for backup config 'jared'.
Transferring snapshot 1.
Transferring snapshot 2.

$ sudo snbk -b jared list
# │ Date                            │ Source State │ Target State
──┼─────────────────────────────────┼──────────────┼─────────────
1 │ Fri 19 Dec 2025 10:33:10 AM EST │ read-only    │ valid
2 │ Fri 19 Dec 2025 11:00:05 AM EST │ read-only    │ valid

If you find snapper/snbk aren't running automatically, use systemctl status snapper*timer to check that snapper-backup, snapper-cleanup, and snapper-timeline are all enabled and started. If they aren't, run systemctl enable --now snapper-backup and similar for the others.

Restoring data

Now let's suppose something terrible happens to jared (I deleted all related subvolumes) and we want to create jared from our backup. Since data in snapshots can be accessed as files, lets recreate the subvolume and copy the data back:

$ sudo btrfs subvolume create jared
Create subvolume './jared'

$ scp -r darius:/data/backups/example/jared/2/snapshot/* jared/
hello.txt                                                                              100%    6     1.2KB/s   00:00
greetings.txt                                                                          100%    6     3.1KB/s   00:00

$ tree jared
jared
├── greetings.txt
└── hello.txt

Then to play nice with snapper, let's recreate .snapshots which it assumes to exist.

$ sudo btrfs subvolume create jared/.snapshots
Create subvolume 'jared/.snapshots'

And finally take a snapshot to check snapper will continue to work:

$ sudo snapper -c jared create
$ sudo snapper -c jared list
# │ Type   │ Pre # │ Date                            │ User │ Cleanup  │ Description │ Userdata
──┼────────┼───────┼─────────────────────────────────┼──────┼──────────┼─────────────┼─────────
0 │ single │       │                                 │ root │          │ current     │
1 │ single │       │ Fri 19 Dec 2025 12:00:09 PM EST │ root │ timeline │ timeline    │
2 │ single │       │ Fri 19 Dec 2025 12:04:13 PM EST │ root

I should mention that restoring a root subvolume does have better support within snapper, and the snbk docs also have notes about performing the restore.

Learning more

If you want to dig in further, a few resources I found useful were:

  1. Fedora BTRFS series
  2. Arch Wiki's BTRFS page

Conclusion

Setting up BTRFS to take backups taught me a lot, and I really like it for taking backups. But I wish the tooling did more for restoring backups. If you're going to use it, I'd recommend understanding how to manually manage snapshots, because when you need it you'll need to understand these parts.

If you're looking for an alternative for managing BTRFS snapshots, btrbk is another good choice. Since btrbk had the same drawbacks around setting up root ssh keys and manual restoration, I decided to stick with snapper + snbk for the time being.