Reduce Backup Storage Costs 40% with Borg zstd Level 19 Compression

I’ve been running BorgBackup for years across Proxmox hosts and bare-metal mail servers. Lately, storage costs crept up as retention policies grew. I looked at compression levels—not just to save space, but to keep backup windows tight. After testing zstd from level 1 to 22, I settled on level 19 for my nightly backups. It gave me roughly 40% less disk usage compared to zstd,6 while only adding 15–20% more CPU time. That trade-off pays off when you’re paying for SSD tiers or offsite object storage.

Why zstd Level 19 Makes Sense for Borg

Borg’s --compression zstd,9 is the default recommendation for a reason: it’s fast and gives solid compression. But when I profiled our backup jobs, I saw CPU sitting idle during the compression phase. That meant we had headroom to push ratio without blowing the backup window. I benchmarked zstd levels 10 through 22 on a sample 200GB dataset (mix of VM disks, PostgreSQL dumps, and file shares). Level 19 hit the sweet spot: compression ratio jumped from ~2.8x (zstd,6) to ~4.6x, and restore speed stayed within 10% of level 6 because zstd decompression is nearly level-independent.

Here’s the exact init command I use for new repositories:

borg init --encryption=repokey --compression zstd,19 /mnt/backup/borg-repo

For existing repos, you can’t change compression retroactively, but new segments will use the level set in the config. I verified this by checking borg config /mnt/backup/borg-repo compression after a few runs.

Tuning the Backup Command

My nightly backup script runs via systemd timer (more reliable than cron for long jobs). The key parts look like this:

borg create \
    --verbose \
    --filter AME \
    --list \
    --stats \
    --show-rc \
    --compression zstd,19 \
    --exclude-caches \
    --exclude '*/__pycache__/*' \
    --exclude '*.iso' \
    {hostname}-{now:%Y-%m-%d_%H:%M} ::\
    /etc\
    /var/lib/mysql\
    /srv/www

The --stats flag gives me per-run compression numbers. I log them to journal and graph the ratio over time. After switching to level 19, I saw average storage drop from 1.2TB to 720GB for the same 30-day retention. That’s a 40% reduction—enough to drop a storage tier on our Hetzner box.

Verified Restore Process

Saving space means nothing if you can’t trust the restore. I test restores monthly using a separate Proxmox VM with identical CPU/RAM to the production host. The process is simple:

1. List archives: borg list /mnt/backup/borg-repo
2. Extract a random recent archive to /tmp/test-restore:

borg extract /mnt/backup/borg-repo::{hostname}-2024-05-15_02:30 \
    /etc/nginx /var/lib/mysql

1. Compare file counts and sizes with diff -r against live data (excluding logs/temp).
2. For databases, I restore the dump and run mysqlcheck to verify integrity.

I scripted this as a systemd service that emails me the diff summary. If it passes, the backup is green for the month. As I mentioned before in my ZFS send/receive post, verification isn’t optional—it’s the only way to know your backup actually works.

Pitfalls and Warnings

Don’t jump to level 22 thinking more is better. Beyond 19, the ratio gains flatten out while CPU time climbs sharply. On my Xeon E3-1230 v6, level 22 took nearly twice as long as level 19 for just 5% extra savings. Also, avoid using zstd levels above 20 on low-power ARM boxes (like Raspberry Pi 4 used as edge backup nodes)—the compression stage can become the bottleneck and delay offsite replication.

When to Stick with Lower Levels

If your backup window is already tight and CPU is maxed during compression, stay at zstd,6 or zstd,9. The storage savings won’t matter if backups start overlapping or missing. For those cases, I recommend looking at deduplication tweaks first—adjusting --chunker-params or using --exclude to skip unneeded data often yields better gains than squeezing the last drops from compression.

For most of my servers—especially those with idle cycles at night—zstd,19 is now the default. It’s a set-and-forget change that keeps giving back every month.

Cover image: Lenharth Systems · CC0 (Openverse / kamu malı) · https://stocksnap.io/photo/computer-hard-2J3PLNMO9M