Linux Backup and Disaster Recovery: Planning and Restore Testing

A backup is useful only when the organization can restore the right data within the time the business can tolerate. A scheduled job reporting success is evidence that one command ran; it is not proof that the files are complete, readable, current, or independent of the failed server.

This guide is a planning and restore-testing workflow for Linux services. The commands are illustrative and read-only unless explicitly noted. Never run a restore against production data as a test. Choose a staging host or isolated database and confirm the dataset contains no unapproved personal information.

1. Define recovery objectives per service

Set a recovery point objective (RPO): how much recent data can be lost. Set a recovery time objective (RTO): how long the service may remain unavailable. An hourly backup may support a shorter RPO than a daily backup, but only if it actually completes, is replicated off-host, and can be restored. A database, uploaded media, object storage, and encryption keys may have different backup methods.

Create a dependency map. Restoring application files without the database, DNS, TLS certificate workflow, secrets, or queue state may not restore the customer workflow. Identify which components must be restored together to preserve consistency.

2. Inventory data before choosing tools

List database engines, volumes, uploaded files, configuration, secrets, and third-party services. Do not assume every directory under /var/www contains persistent data. The following commands help locate mounted filesystems and storage pressure; they do not create a backup.

findmnt -t ext4,xfs,btrfs,zfs
df -h
df -i

Some databases need transaction-consistent backups. Copying live database files with cp can produce an unusable result. PostgreSQL documents pg_dump for logical backups, but a large database with tight recovery objectives may require physical or continuous-archiving methods. MySQL/MariaDB have their own consistency options. Choose a method appropriate to the engine and size.

3. Keep copies off the failed host

A local snapshot protects against some accidental changes but does not protect against host loss, compromised credentials, or a storage failure affecting the whole server. Keep at least one copy in a different failure domain, with access controls that limit deletion. Encrypt backups at rest and during transfer, and define who can access restore keys.

Versioning and retention matter. If ransomware or an application bug corrupts data on Monday, a backup taken Monday evening may preserve the corruption. Retain several recovery points and test that older snapshots remain accessible.

4. Make backups observable

Record the start/end time, dataset, bytes, destination, exit status, and checksum or integrity result where applicable. Monitor missed schedules and unusually small or large backups. Alert a human who can act, not only a dashboard nobody checks. Avoid logging secret values or whole database rows.

systemctl list-timers --all --no-pager
journalctl -u example-backup.service --since '7 days ago' --no-pager

The unit name is an example; inspect your actual scheduler. A “success” status can still mean the wrong directory or empty database was backed up. Compare dataset counts and a documented manifest with expected ranges.

5. Restore in an isolated environment

A restore drill should use a new host or isolated container/network, not the live application. Record elapsed time and every manual step. Validate application login, a representative read, a safe write, media retrieval, background processing, and integration configuration. Do not trigger real payments, customer email, or provisioning in the test environment.

If the restore fails, preserve the evidence and fix the runbook. A disaster recovery plan that has never been rehearsed is only a hypothesis. Repeat after major architecture, database, encryption, or provider changes.

Troubleshooting and common errors

• Backup file exists but cannot be decrypted: verify key escrow and restore access before an incident.
• Database dump restores but application fails: check schema version, secrets, media, job queues, and external service dependencies.
• Restore time exceeds RTO: test a faster backup method, pre-provisioned infrastructure, or different service architecture.
• Backups all share one account: a compromised credential can erase every copy; separate privileges and deletion protections.
• Backup job “succeeds” with tiny output: compare expected size and record counts; investigate wrong paths or failed upstream commands.

Security considerations

Backups often contain the most complete copy of personal and financial data. Restrict access, encrypt, audit restores, and apply data retention rules. Keep test restores isolated from public DNS and outbound payment/email systems. Do not publish real backup locations or credentials in customer-facing guides.

FAQ

Is a provider snapshot the same as a backup?

It is one recovery mechanism, but its consistency, retention, and failure domain depend on the provider. Verify how it is created and whether it can be restored independently.

How often should restore tests run?

Choose a cadence based on change rate and business impact, and always test after major changes to storage, encryption, or deployment architecture.

Can I back up a running database by copying its data directory?

Not safely as a general rule. Use database-supported logical or physical backup procedures and test the resulting restore.

Continue reading

Learn how to deploy a Compose workload without losing persistent data and how to monitor Linux server health. Contact East4Serv to review backup and recovery objectives for your service.

References: PostgreSQL backup and restore, Docker volumes, and MySQL backup and recovery.