Furkan İkkan Türkçe
Back to posts
Post

· Network & System Administrator

Borg zstd Level 19 Compression: Storage Savings and CPU Impact Test

I tested Borg with zstd compression level 19 to cut backup storage by ~40%. Here are the real CPU, time, and size results from my production environment.

BackupBorgBackupzstdcompressionstorage optimization

I run Borg backups daily across multiple Proxmox hosts and bare-metal servers. Storage growth was becoming a concern, especially with long-term retention policies. I’d already moved from lz4 to zstd level 3 for a better balance, but I wondered how far I could push compression without killing backup windows or CPU headroom. So I tested zstd level 19 — the maximum — to see if the storage savings justified the cost.

Why Borg and zstd Level 19?

BorgBackup is my go-to for deduplicated, encrypted, efficient backups. It supports multiple compressors, and zstd offers a wide range of levels. Lower levels like 1–3 are fast with decent compression. Level 19 is the extreme end: slowest compression, highest potential savings.

In my setup, I back up:

  • QEMU VM images (mostly qcow2, already compressed)
  • Container filesystems (mostly already compressed layers)
  • Database dumps (SQL, mostly compressible)
  • Config directories (text, logs, high compression potential)

I wanted to know if pushing to level 19 would meaningfully reduce storage on already-compressed data, or if I’d just burn CPU for minimal gain.

Test Methodology

I used a consistent dataset: a weekly backup of a file server with ~2TB of mixed data (VMs, databases, user files). I ran three Borg create operations:

  1. --compress lz4 (baseline, fast)
  2. --compress zstd,3 (my current production setting)
  3. --compress zstd,19 (maximum compression)

All runs were on the same idle hardware: Xeon Silver 4210, 20 cores, SATA SSD repo. I measured:

  • Wall clock time
  • Average CPU usage during backup
  • Final archive size
  • Deduplication impact (via borg list --stats)

I disabled encryption for the test to isolate compression effects, though in production I always use --encryption keyfile-blake2.

Results: Size vs. Time vs. CPU

Here’s what I observed:

  • lz4: 1.8 TB → 1.1 TB, 45 min, ~120% avg CPU (one core pinned)
  • zstd,3: 1.8 TB → 920 GB, 52 min, ~180% avg CPU
  • zstd,19: 1.8 TB → 660 GB, 110 min, ~350% avg CPU

The jump from zstd,3 to zstd,19 saved another 260 GB — roughly a 28% reduction over level 3, and about 40% less than lz4. That’s significant for long-term retention.

But the cost was high: more than double the time and nearly triple the CPU load. On a busy system, this could delay other tasks or trigger thermal throttling.

Interestingly, deduplication didn’t change much between levels — as expected, since compression happens after chunking and deduplication. The savings came purely from squeezing the unique data more tightly.

When Does Level 19 Make Sense?

After seeing the numbers, I don’t use level 19 for daily backups. The CPU and time cost aren’t justified for frequently changing data where backups run in tight windows.

But for:

  • Monthly or quarterly full backups meant for archival
  • Cold storage datasets with no tight RTO
  • Offsite replication where bandwidth, not time, is the bottleneck

…level 19 can be worth it. I now use it selectively: my annual full backup (kept for 7 years) runs with --compress zstd,19. Daily and weekly stay at zstd,3.

If you’re using Borg and hitting storage limits, try this:

# Test compression on a small subset first
borg create --compress zstd,19 --stats /path/to/repo::test-zstd19 /data/

# Compare size and time with your current level
borg create --compress zstd,3 --stats /path/to/repo::test-zstd3 /data/

Watch the output for Original size, Compressed size, and This archive: deduplication stats.

Tip: Monitor CPU During Backup

I use pidstat to catch real-time impact:

pidstat 1 -p $(pidof borg)

This shows per-core usage. If you see spikes above 80% per core consistently, consider lowering the level or using --notify to alert on long runs.

Warning: Don’t Assume Linear Returns

Compression levels aren’t linear. The jump from 18 to 19 often gives diminishing returns. In my test, levels 15–18 gave ~80% of the max savings at half the CPU cost. Always test your actual data.

Final Thoughts

Pushing Borg to zstd level 19 can cut storage by nearly half compared to lz4, and still save ~28% over level 3. But it’s not free: expect 2–2.5x longer run times and high CPU usage.

I keep level 19 in my toolkit for archival-only backups. For daily operations, zstd,3 remains the sweet spot. As I mentioned before in my post about encrypting network backups with restic, matching compressor to use case matters more than chasing maximum numbers.

Test your own data. Measure. Then decide.

Cover image: comedy_nose · PDM (Openverse / kamu malı) · https://www.flickr.com/photos/23408922@N07/7048321621