Skip to content

Operations

Backup and restore runbook

Back up the go-sequent PostgreSQL event store and rehearse a verified restore: what to protect, continuous WAL archiving, a quarterly drill, point-in-time recovery and snapshots after restore.

Backup and restore runbook

The PostgreSQL event store is the part you cannot lose. Projections are disposable. Events and commands are not.

What to back up

Object Priority
commands, events, aggregates, *_types Critical
aggregate_unique_keys, idempotency_keys Critical for correctness
snapshot_records High. Rebuildable from events, but speeds up loads.
outbox High if asynchronous integrations lag
Projection and view tables Low. Rebuild through replay.
event_tombstones Medium. The GDPR audit trail.

Integrity comes from the hash chain, plus access control, backups and optionally WORM storage. There are no event signatures.

Continuous backup

  1. Enable continuous WAL archiving, or your cloud’s point-in-time recovery.
  2. Take base backups on a schedule with pgBackRest, Barman or your cloud’s automated backups.
  3. Encrypt backups at rest and restrict who can restore.
  4. Write down your retention policy next to your GDPR tombstone policy. Erasure is by tombstone only. See Event evolution and Locked decisions.

Verified restore drill

Run this every quarter.

  1. Restore into an isolated instance. Never overwrite production blindly.
  2. Run go run ./cmd/go-sequent migrate -dsn … only if schema objects are missing. Prefer restoring a consistent cluster snapshot that already includes the schema.
  3. Smoke test:
    • LoadStream for a known aggregate. This also checks the hash chain.
    • One CommandService.Execute with a SyncInTxn projector.
    • An idempotency key round trip for a tenant.
  4. Optionally rebuild projections with the replay pipeline. See Projection rebuild.
  5. Record the RTO and RPO you actually achieved, and fix the gaps.

For a stronger check, call the store’s VerifyChain for a sample of aggregates after the restore.

Point-in-time recovery

  • Prefer recovering to just before the incident.
  • After a restore, do not import Ruby Sequent dumps.
  • If dual writes happened during a bad failover, stop all writers and restore to a known-good LSN. Then let clients catch up through idempotent retries. See Failover and Idempotency.

Snapshots after restore

If snapshot_version or the aggregate schema changed, delete the incompatible rows in snapshot_records. The snapshotter refills them, or you can load from events only. See Snapshotting.

Docs track the go-sequent library. When APIs drift, the source wins.