Skip to content

Migration Policy

MAID migrations are backward-compatible by default. Every migration ships with a working downgrade() unless it is explicitly labelled irreversible=True, and irreversible migrations require a fresh backup at runtime. A reviewer sign-off label in CI is planned but not yet active (see the irreversible=True section below).

This document is the source of truth for what a migration author and a reviewer must check before merging.

The default contract

Every migration has a @migration_policy(...) decorator. The defaults are:

from maid_engine.migrations.policy import migration_policy

@migration_policy(
    backward_compatible=True,   # default
    irreversible=False,         # default
    notes="",
)
class Add0042NewIndex(Migration):
    namespace = "engine"
    sequence = 42
    description = "Add covering index on entities(zone, kind)"

    async def upgrade(self, ctx): ...
    async def downgrade(self, ctx): ...

A migration is backward-compatible when an older engine binary can continue to run against the post-migration schema. In practice this means:

  • Additive columns: NULL-able, or with a server-default that the old code does not need to read.
  • Additive tables: old code does not query them.
  • Index changes: add CONCURRENTLY; never drop an index that an older binary's query plan relies on.

A migration is NOT backward-compatible when it removes or renames a column / table that older code reads, changes a column type in place, or adds a NOT NULL constraint without a server-default.

Expand-contract for renames

A column rename is two migrations:

  1. Phase 1 — expand. Add the new column. Dual-write from application code (both names). Backfill via a separate migration. This phase is backward-compatible.
  2. Phase 2 — contract. After at least one release where Phase 1 has shipped and been observed in production, drop the old column. This phase is NOT backward-compatible: rolling back past Phase 2 requires a restore from backup.

In-place type changes are forbidden. Use the same expand-contract pattern (new column with new type, dual-write, backfill, drop old).

irreversible=True

A migration is irreversible when there is no meaningful downgrade() — typically because it drops data that cannot be reconstructed. Setting irreversible=True is a serious commitment: the only rollback path is restore from a pre-migration backup.

Registration-time and runtime enforcement are implemented today; the CI gate is planned (see below). Together they are intended to enforce this:

  • At registration time, enforce_policy_at_registration() raises MigrationPolicyViolation if irreversible=True and the notes field is empty. The author must justify in prose why no downgrade() is possible.
  • At runtime, migrations/runner.py refuses to execute an irreversible migration unless ALL of:
  • MAID_MIGRATION_ALLOW_IRREVERSIBLE=1 is set in the environment, AND
  • a backup manifest exists with mtime ≤ 1 hour, AND
  • the migration advisory lock is held.
  • In CI (planned — not yet active): a pr-c-migration-policy-ci workflow is specified (per plan.md R4.12) to block a PR unless the label migration:irreversible-approved is applied by a code-owner reviewer. This gate is not yet wired up: the static validator exists at scripts/check_migration_policy.py, but it is not currently run by any GitHub Actions workflow (see .github/workflows/) and no CODEOWNERS file is configured. Until it is enabled, the runtime preconditions above are the only enforced control. The intended workflow also warns on any migration whose down() is empty (placeholder pass) but does not block on it — empty down() for a backward-compatible additive migration is legitimate (rolling back the code is enough).

Coverage: the registration-time and runtime gates apply only to Python migrations that carry a @migration_policy(...) policy. YAML migrations have no policy metadata — planner.py reads only namespace, sequence, description, operations, and rollback, so an irreversible: field in a YAML file is silently discarded and get_policy() falls back to the default (backward_compatible=True, irreversible=False). A migration that must be gated as irreversible therefore has to be authored in Python.

Pre-merge checklist for authors

  • [ ] The migration has a @migration_policy(...) decorator with explicit values for backward_compatible and irreversible.
  • [ ] The migration has a downgrade() that has been tested locally (uv run maid db migrate followed by uv run maid db rollback).
  • [ ] If irreversible=True, the notes field explains WHY no downgrade() is possible.
  • [ ] If irreversible=True, the PR has the migration:irreversible-approved label from a code-owner (planned CI gate — see note above; not yet enforced automatically).
  • [ ] If the migration is part of an expand-contract sequence, the PR description links to the prior phase's PR.
  • [ ] The migration is online-safe (no long-held exclusive lock) OR the PR description explains the maintenance window required.

Pre-merge checklist for reviewers

  • [ ] Read the upgrade() AND the downgrade(). Mentally simulate the downgrade against a database that contains data inserted post-upgrade.
  • [ ] If backward_compatible=True, verify that the prior release's binary can still serve traffic against the new schema. Adding a NOT NULL column without a default is the most common silent failure.
  • [ ] If irreversible=True, confirm the notes are honest (not "I was lazy"). Apply the migration:irreversible-approved label only after you would personally accept being paged at 03:00 to restore from backup.
  • [ ] If down() is empty, confirm that "rolling back the code is enough" is true. If it isn't, the migration is implicitly irreversible=True and must be re-labelled.

Why the policy is enforced in code, not just process

Process-only policies degrade. After three incident-free quarters someone adds an irreversible=True migration "just this once," reviewers nod, and six months later a routine rollback turns into a four-hour restore. The runtime preconditions in check_runtime_preconditions() (enforced by runner.py), together with the registration-time check in enforce_policy_at_registration() — and, once the CI gate is wired up, the static scripts/check_migration_policy.py check — make the slippage detectable on the PR that introduces it, not in the incident that exposes it.