Skip to content

Backups and restore

View as Markdown

Everything here runs against a server on 127.0.0.1:58300 started as in the quick start, with --backup-root backups. First, some state worth saving — a run that finishes and compacts:

Terminal window
alias durable="durable-cli --server http://127.0.0.1:58300"
durable submit --op-key ops/keep-alive --step ping=step --input '{"keep":"me"}'
{"run_id":1}
Terminal window
for i in $(seq 1 60); do s=$(durable inspect --run 1); case "$s" in *'"done"'*|*'"failed"'*|*'"cancelled"'*) break;; esac; sleep 0.25; done; echo "$s"
{"finished_at":1774990000000,"id":1,"key":"ops/keep-alive","state":"done","steps":1}

A compaction first, so the backup carries a snapshot generation, not only records:

Terminal window
durable compact
{"generation":2}

POST /admin/backup (or durable backup) writes a snapshot-plus-records copy under --backup-root/<name> — while the server keeps serving. The copy is internally consistent and checksummed, so it’s safe to rsync elsewhere:

Terminal window
durable backup --name nightly
{"backup":"nightly"}

The name is a plain directory name — 1–64 of [A-Za-z0-9._-], never a path — so a backup can’t be steered outside the root. An existing name is 409 backup_exists; no --backup-root at startup is 403 backups_disabled.

durable restore is the offline twin — it never calls the server. It verifies the backup’s manifest and every file’s checksum, then writes a complete journal directory. It refuses the two mistakes that matter: an unrestorable source, and a target a live server holds — the journal’s lock is checked, so you can’t restore underneath a running process:

Terminal window
durable restore --from backups/nightly --to journal
Error: journal is in use: a server holds its journal lock; stop it, or restore into a new directory

The real restore path: stop the server (SIGTERM drains it), restore beside it, start on the restored journal:

Terminal window
kill $SRV; for i in $(seq 1 100); do kill -0 $SRV 2>/dev/null || break; sleep 0.1; done
durable restore --from backups/nightly --to journal-restored
restored backups/nightly -> journal-restored
Terminal window
./target/release/durable-server --journal journal-restored --listen 127.0.0.1:58300 \
--handler step=http://127.0.0.1:58320/step >server-restored.log 2>&1 &
SRV=$!
for i in $(seq 1 150); do curl -sf http://127.0.0.1:58300/health >/dev/null && break; sleep 0.1; done

And the point of the exercise — the run that existed at backup time is there, terminal state and finished_at intact:

Terminal window
durable inspect --run 1
{"finished_at":1774990000000,"id":1,"key":"ops/keep-alive","state":"done","steps":1}
  • Backups are cheap; take them often. --compact-every-secs plus a backup cron covers most disasters; a backup before every upgrade covers the rest.
  • Ship them off the host. A backup that dies with the disk it copied isn’t a backup.
  • Rehearse the restore. The flow above is the whole drill — if journal-restored starts and the runs read back, your backups are real.
  • Keep the old journal until the restored one proves itself — restored beside journal, not instead of it.

Was this page helpful?

Either answer opens a short issue on GitHub, so you can say what worked or what was missing.