Skip to content
go-sequentCQRS + Event Sourcing for Go financial systems

Operations

Projection rebuild runbook

Rebuild disposable read models with the go-sequent replay pipeline: prepare, replay, catchup, optimize and golive, with the library API, CLI schema stages and operational tips.

Projection rebuild runbook

The replay pipeline follows Sequent 8: prepare, replay, catchup, optimize, golive, with an optional abort.

Ledger and balance projectors must stay strongly consistent, so on the live path they remain SyncInTxn. Rebuild is for disposable read models, or for filling a new projection from history.

Library API

pipe := &replay.Pipeline{
    Name:          "balances",
    Source:        eventSource, // implements StreamAll
    Publisher:     syncInTxnWithProjectorsOnly,
    Schema:        &replay.PostgresSchema{Pool: pool}, // or replay.NoopSchema{}
    State:         replay.NewMemoryState(),            // or a DB-backed StateStore in your app
    ManagedTables: []string{"account_balances"},
}
if err := pipe.RunAll(ctx); err != nil {
    return err
}

Pipeline also has Prepare, Replay, Optimize, GoLive and Abort, so you can run the stages one at a time. Other fields are ReplaySchema, ViewSchema and BatchSize. When BatchSize is zero, Replay uses 1000.

The publisher replays through PublishProjectorsOnly. That excludes workflows, so a rebuild never re-triggers side effects such as emails or payments.

State is a StateStore. replay.NewMemoryState() forgets progress when the process exits. Use a durable store if a rebuild must survive a restart. The projector_replay_states table, created by migrations, is available for that.

Stages

Stage Action
prepare Create the replay_* schema, create managed tables with CREATE TABLE … (LIKE …), truncate
replay Stream historical events into the replay projectors and tables
catchup Apply events appended since the high-water mark while the live system keeps running
optimize Run ANALYZE and add indexes as needed
golive Briefly swap: move the live tables aside and promote the replay tables into the view schema
abort DROP SCHEMA replay_* CASCADE

CLI: schema stages only

The CLI drives the schema side of a rebuild:

go run ./cmd/go-sequent migrate -dsn "$GO_SEQUENT_PG_DSN"

go run ./cmd/go-sequent replay -dsn "$GO_SEQUENT_PG_DSN" \
  -projector balances -stage prepare -tables account_balances

# then, as needed: -stage optimize | golive | abort

The CLI accepts the stages prepare, optimize, golive and abort. It cannot stream events, because it does not know your projector registrations. The replay and catchup stages must run from your own application binary, with the pipeline above. See Migrations and CLI.

Operational tips

  • Size BatchSize for your event volume. Large batches mean fewer round trips and longer transactions.
  • During golive, pause writers or serialize them briefly if your swap is not lock-free.
  • After golive, keep SyncInTxn projectors as the source of truth for balances.
  • Track progress through projector_replay_states or your own StateStore.
  • If something looks wrong before golive, run abort. Live tables have not been touched yet.

Docs track the go-sequent library. When APIs drift, the source wins.