In my environment, I often need to back up remote systems without leaving temporary archives on disk. Using restic's --stdin and --stdout flags with tar and SSH lets me create encrypted backup streams directly over the network. This avoids local temp files, reduces I/O pressure, and limits exposure if a host is compromised.
Why pipe-based backups with restic make sense
Traditional backup workflows often involve creating a tarball locally, transferring it, then running restic backup. That means double the disk usage and a window where unencrypted data sits on disk. By piping tar output directly into restic via SSH, I eliminate the intermediate file. The data is encrypted in transit and at rest, and restic never sees the unencrypted stream on disk.
This approach works especially well for headless servers, containers, or VMs where local storage is limited or ephemeral. I use it for nightly backups of web servers and database dumps where I don’t want to rely on shared NFS or persistent volumes just for staging.
The core command: tar + SSH + restic
Here’s the one-liner I run from the backup host to pull and encrypt data from a remote host:
ssh user@remote-host "tar -czpf - /etc /var/www" | restic backup --stdin --stdin-filename backup.tar.gzThe remote tar command creates a gzipped tarball and sends it to stdout. SSH forwards that stream to the local restic process, which reads it via --stdin and treats it as a file named backup.tar.gz inside the repository. No temporary file is ever written locally or remotely.
To reverse the direction — pushing from local to remote — I use:
tar -czpf - /etc /var/www | ssh user@backup-host "restic backup --stdin --stdin-filename backup.tar.gz"This is useful when the backup host has limited inbound SSH privileges but can initiate outbound connections.
Encryption and integrity: what’s actually protected
Restic encrypts the backup stream client-side using AES-256 in GCM mode. The data is encrypted before it leaves the sending host, so even if SSH is compromised or intercepted, the backup remains unreadable without the restic password.
Importantly, restic verifies integrity on restore. If the stream is tampered with during transit, the backup will fail to load. This gives me end-to-end protection without relying on SSH alone for confidentiality.
I always test restores from these streams. A quick check:
restic restore latest --target /tmp/test-restoreIf the tarball extracts cleanly and files are intact, the pipe worked correctly.
Handling large datasets and performance
For large filesystems, I add buffering and compression tuning. On the tar side, I use --use-compress-program=pigz to parallelize gzip:
ssh user@remote-host "tar -cf - --use-compress-program=pigz /var/log" | restic backup --stdin --stdin-filename logs.tarOn the restic side, I limit upload speed with --limit-upload to avoid saturating the link during business hours:
restic backup --stdin --stdin-filename data.tar --limit-upload 5MI also monitor restic’s memory usage — streaming keeps it low, but very large tar streams can still cause spikes if the repository is locked or slow to respond. I’ve seen this when the restic server is under load; adding retries with a wrapper script helps.
Automation with cron and logging
I wrap the pipe in a script with logging and exit code checks:
#!/bin/bash
set -eo pipefail
REMOTE="user@backup-host"
SOURCE="/etc /var/www"
ssh "$REMOTE" "tar -czpf - $SOURCE" | \
restic backup --stdin --stdin-filename backup.tar.gz \
--limit-upload 10M \
--verbose
if [ $? -eq 0 ]; then
logger -t restic-pipe "Backup successful"
else
logger -t restic-pipe "Backup failed"
exit 1
fiI run this via cron at 2 AM. The pipefail setting ensures that if tar or SSH fails, the restic command doesn’t proceed with partial data.
Limitations and gotchas
This method isn’t ideal for incremental backups of changing files where you want block-level deduplication across runs. Because each tar stream is a new blob, restic treats it as a fresh file — deduplication only works if the tar output is identical, which rarely happens due to timestamps.
For true incremental backups, I prefer mounting the remote filesystem via SSHFS and letting restic scan it directly. But when that’s not possible — say, due to firewall restrictions or lack of FUSE support — the pipe method is a secure, efficient alternative.
Also, avoid using --stdin-filename with arbitrary names if you plan to import or manipulate snapshots later. Stick to consistent names like backup.tar.gz or data.tar so restore logic remains predictable.
When to choose this over other methods
I use this pipe-based approach when:
- The source host has no spare disk for staging
- Network bandwidth is limited but CPU is available for compression
- I need end-to-end encryption without relying on SSH alone
- The backup host initiates the connection (reverse SSH isn’t feasible)
For database dumps, I often combine it with pg_dump or mysqldump piped directly:
ssh db-host "pg_dump -Fc mydb" | restic backup --stdin --stdin-filename db.dumpSame principle: no temp files, encrypted stream, verifiable restore.
Final thoughts
Streaming backups via restic’s stdin/stdout is a simple but powerful pattern that aligns with Unix philosophy: do one thing well and compose with pipes. It’s saved me disk space, reduced attack surface, and made backup automation more reliable in constrained environments.
If you’re already using restic, try replacing your tar-and-copy workflow with a pipe. Test the restore path first — then let the cron run silently.
Cover image: f0976531950182 · PDM (Openverse / kamu malı) · https://www.flickr.com/photos/204310491@N07/55131284510