Backups and restore
Everything here runs against a server on 127.0.0.1:58300 started as in the
quick start, with --backup-root backups. First, some state worth saving — a run
that finishes and compacts:
alias durable="durable-cli --server http://127.0.0.1:58300"durable submit --op-key ops/keep-alive --step ping=step --input '{"keep":"me"}'{"run_id":1}for i in $(seq 1 60); do s=$(durable inspect --run 1); case "$s" in *'"done"'*|*'"failed"'*|*'"cancelled"'*) break;; esac; sleep 0.25; done; echo "$s"{"finished_at":1774990000000,"id":1,"key":"ops/keep-alive","state":"done","steps":1}A compaction first, so the backup carries a snapshot generation, not only records:
durable compact{"generation":2}Backup: online, consistent, cheap
Section titled “Backup: online, consistent, cheap”POST /admin/backup (or durable backup) writes a snapshot-plus-records copy under
--backup-root/<name> — while the server keeps serving. The copy is internally consistent and
checksummed, so it’s safe to rsync elsewhere:
durable backup --name nightly{"backup":"nightly"}The name is a plain directory name — 1–64 of [A-Za-z0-9._-], never a path — so a backup can’t
be steered outside the root. An existing name is 409 backup_exists; no --backup-root at startup
is 403 backups_disabled.
Restore: offline, verified, lock-checked
Section titled “Restore: offline, verified, lock-checked”durable restore is the offline twin — it never calls the server. It verifies the backup’s
manifest and every file’s checksum, then writes a complete journal directory. It refuses the two
mistakes that matter: an unrestorable source, and a target a live server holds — the journal’s
lock is checked, so you can’t restore underneath a running process:
durable restore --from backups/nightly --to journalError: journal is in use: a server holds its journal lock; stop it, or restore into a new directoryThe real restore path: stop the server (SIGTERM drains it), restore beside it, start on the restored journal:
kill $SRV; for i in $(seq 1 100); do kill -0 $SRV 2>/dev/null || break; sleep 0.1; donedurable restore --from backups/nightly --to journal-restoredrestored backups/nightly -> journal-restored./target/release/durable-server --journal journal-restored --listen 127.0.0.1:58300 \ --handler step=http://127.0.0.1:58320/step >server-restored.log 2>&1 &SRV=$!for i in $(seq 1 150); do curl -sf http://127.0.0.1:58300/health >/dev/null && break; sleep 0.1; doneAnd the point of the exercise — the run that existed at backup time is there, terminal state and
finished_at intact:
durable inspect --run 1{"finished_at":1774990000000,"id":1,"key":"ops/keep-alive","state":"done","steps":1}Practice
Section titled “Practice”- Backups are cheap; take them often.
--compact-every-secsplus abackupcron covers most disasters; a backup before every upgrade covers the rest. - Ship them off the host. A backup that dies with the disk it copied isn’t a backup.
- Rehearse the restore. The flow above is the whole drill — if
journal-restoredstarts and the runs read back, your backups are real. - Keep the old journal until the restored one proves itself —
restoredbesidejournal, not instead of it.