Skip to content

Durable Persistence — Implementation Plan

Design Document: docs/designs/v3.1/01-durable-persistence.md Priority: P0 — Critical Path Estimated Duration: 10 weeks across 5 phases (Phase 0–4)


Summary

This plan implements the Durable Entity Persistence layer for MAID, solving the fundamental problem that all world state (NPCs, items, room modifications, bank balances, crafting progress, quest state, builder-created content) is lost on server restart. The implementation adds:

  • ComponentRegistry — type-safe string→class mapping for deserialization with pack ownership tracking
  • DirtyTracker — event-driven change detection with __setattr__ auto-dirty hooks on Component
  • EntityPersistenceManager — batched async saves with per-operation timeouts, tombstone queues
  • SaveSchedulerasyncio.Lock-based coordination with atomic drain-then-re-mark strategy
  • EntitySerializerEntityEntityDocument conversion with schema evolution and quarantining
  • Backup/restore — full snapshots, incremental backups, point-in-time recovery

Key prerequisite: ECS events (ComponentAddedEvent, etc.) are defined in core/events.py but never emitted by ECS operations. Phase 0 adds the missing emissions and several other infrastructure changes before persistence work can begin.

Existing infrastructure leveraged: - DocumentStore / DocumentCollection at packages/maid-engine/src/maid_engine/storage/document_store.py - Entity.to_dict() at core/ecs/entity.py:202-213 - Component base class at core/ecs/component.py (Pydantic BaseModel with component_type ClassVar) - EventBus at core/events.py with subscribe(), emit(), unsubscribe() - ContentPack protocol at plugins/protocol.py - World at core/world.py with EntityManager, RoomIndex - GameEngine at core/engine.py with content pack loading loop


Phase 0: Prerequisites (Week 1)

All Phase 0 tasks are blockers. Persistence implementation cannot begin until these complete.

0.1 Add ECS Event Emission

Package: maid-engine | Priority: P0 | Dependencies: none

Currently, EntityCreatedEvent, EntityDestroyedEvent, ComponentAddedEvent, and ComponentRemovedEvent are defined in core/events.py:103-129 but are never emitted by ECS operations. Without these events, DirtyTracker cannot function.

  • [ ] Modify Entity.add() in packages/maid-engine/src/maid_engine/core/ecs/entity.py:76-86 to emit ComponentAddedEvent via the entity's manager's event bus
  • Entity needs access to EventBus — thread through EntityManagerWorld._events
  • Guard: only emit if entity has a manager (standalone entities in tests skip emission)
  • Event payload: entity_id=self._id, component_type=component.get_type()
  • [ ] Modify Entity.remove() in entity.py:88-99 to emit ComponentRemovedEvent
  • Same guard and payload pattern as add()
  • Emit before removing from _components dict so handlers can still inspect the component
  • [ ] Modify EntityManager.create() in entity.py:245-248 to emit EntityCreatedEvent
  • Requires EntityManager to hold a reference to the EventBus
  • Use lazy binding: add set_event_bus(self, event_bus: EventBus) -> None method on EntityManager (because World.__init__ creates EntityManager() at line 102 before EventBus() at line 103 — constructor parameter won't work)
  • Emit after entity is added to _entities dict
  • [ ] Modify EntityManager.destroy() in entity.py:262-276 to emit EntityDestroyedEvent
  • Emit before removing from _entities dict and cleaning up indexes
  • [ ] Add TagAddedEvent and TagRemovedEvent event classes to core/events.py
  • [ ] Modify Entity.add_tag() to emit TagAddedEvent via the entity's manager's event bus (same guard pattern as component events)
  • [ ] Modify Entity.remove_tag() to emit TagRemovedEvent
  • [ ] Update World.__init__() in core/world.py:100-112 to call self._entities.set_event_bus(self._events) after both are constructed
  • [ ] Add set_event_bus() method and _event_bus: EventBus | None attribute to EntityManager
  • [ ] Ensure emit_sync() is used (not await emit()) since Entity.add() / remove() are synchronous methods
  • [ ] Write unit tests in packages/maid-engine/tests/core/test_ecs_events.py:
  • [ ] Test ComponentAddedEvent emitted on entity.add(component)
  • [ ] Test ComponentRemovedEvent emitted on entity.remove(component_type)
  • [ ] Test EntityCreatedEvent emitted on entity_manager.create()
  • [ ] Test EntityDestroyedEvent emitted on entity_manager.destroy(entity_id)
  • [ ] Test no events emitted for standalone entities (no manager)
  • [ ] Test event payloads contain correct entity_id and component_type
  • [ ] Test TagAddedEvent emitted on entity.add_tag(tag)
  • [ ] Test TagRemovedEvent emitted on entity.remove_tag(tag)

0.2 Add Upsert to DocumentStore

Package: maid-engine | Priority: P0 | Dependencies: 0.6

  • [ ] Add abstract upsert(self, doc_id: UUID, document: T) -> bool method to DocumentCollection base class in packages/maid-engine/src/maid_engine/storage/document_store.py:69
  • Docstring: "Insert or update document atomically. Returns True if successful."
  • [ ] Implement upsert() on InMemoryDocumentCollection in document_store.py:300
  • Set created_at only if doc is new; always update updated_at
  • Store document directly in _documents dict keyed by doc_id
  • [ ] Implement upsert() on PostgresDocumentCollection in document_store.py:499
  • SQL: INSERT INTO {table} (id, collection, data, created_at, updated_at) VALUES ($1, $2, $3::jsonb, NOW(), NOW()) ON CONFLICT (collection, id) DO UPDATE SET data = EXCLUDED.data, updated_at = NOW()
  • Requires composite PK from task 0.6
  • Add per-operation timeout via asyncio.wait_for(..., timeout=5.0)
  • Record query to profiling collector
  • [ ] Write tests in packages/maid-engine/tests/storage/test_document_store_upsert.py:
  • [ ] Test upsert creates new document when none exists
  • [ ] Test upsert updates existing document
  • [ ] Test upsert preserves created_at on update
  • [ ] Test upsert returns True on success
  • [ ] Test in-memory implementation
  • [ ] Test PostgreSQL implementation (integration test, can be skipped in CI without PG)

0.3 Add __setattr__ Hook to Component Base Class

Package: maid-engine | Priority: P0 | Dependencies: none

  • [ ] Add private attributes to Component class in packages/maid-engine/src/maid_engine/core/ecs/component.py:10:
  • _dirty_callback: Callable[[UUID], None] | None = PrivateAttr(default=None) — called on mutation
  • _owner_id: UUID | None = PrivateAttr(default=None) — entity UUID this component belongs to
  • _suppress_dirty: bool = PrivateAttr(default=False) — suppress during deserialization
  • [ ] Implement __setattr__ override on Component:
  • Call super().__setattr__(name, value) first
  • After super call: wrap dirty check in try/except AttributeError (Pydantic's PrivateAttr values are initialized in model_post_init, not __init__, so _suppress_dirty etc. may not exist when __setattr__ fires during construction)
  • Inside guard: if name does not start with _, and _suppress_dirty is False, and _dirty_callback is not None, and _owner_id is not None → call self._dirty_callback(self._owner_id)
  • [ ] Add notify_mutation(self) -> None convenience method to Component:
  • Docstring: "Explicitly mark the owning entity dirty after a container mutation (e.g., list.append, dict.__setitem__)."
  • If _dirty_callback and _owner_id are both set, call self._dirty_callback(self._owner_id)
  • [ ] Verify model_config = ConfigDict(validate_assignment=True, ...) is already set (it is — line 23)
  • [ ] Write tests in packages/maid-engine/tests/core/test_component_dirty.py:
  • [ ] Test __setattr__ fires callback on public field assignment
  • [ ] Test __setattr__ does NOT fire for private/underscore attributes
  • [ ] Test _suppress_dirty=True suppresses callback
  • [ ] Test notify_mutation() fires callback
  • [ ] Test no callback when _dirty_callback is None
  • [ ] Test no callback when _owner_id is None

0.4 Extend ContentPack Protocol with register_component_types()

Package: maid-engine | Priority: P0 | Dependencies: none

  • [ ] Add register_component_types(self, registry: ComponentRegistry) -> None to ContentPack protocol in packages/maid-engine/src/maid_engine/plugins/protocol.py:29
  • Add import: from maid_engine.persistence.registry import ComponentRegistry (TYPE_CHECKING)
  • Docstring: "Register component types for persistence serialization/deserialization."
  • [ ] Add default no-op implementation to BaseContentPack in protocol.py:213:
  • def register_component_types(self, registry: ComponentRegistry) -> None: pass
  • [ ] Modify GameEngine.start() in packages/maid-engine/src/maid_engine/core/engine.py:426-517:
  • In the content pack initialization loop (line 445), call pack.register_component_types(self._component_registry) before pack.register_commands() and pack.on_load()
  • The GameEngine must own a ComponentRegistry instance (added in Phase 1)
  • [ ] Update test helpers in packages/maid-engine/tests/helpers.py to include register_component_types in mock content packs
  • [ ] Write tests:
  • [ ] Test ContentPack protocol check passes with register_component_types
  • [ ] Test BaseContentPack default is no-op
  • [ ] Test GameEngine.start() calls register_component_types() before on_load()

0.5 Add Pagination Support to DocumentStore Queries

Package: maid-engine | Priority: P0 | Dependencies: none

The existing DocumentCollection.query() accepts QueryOptions which already has limit and offset fields (see document_store.py:50-67). Both InMemoryDocumentCollection.query() (line 395) and PostgresDocumentCollection.query() (line 658) already implement limit and offset. This prerequisite is already satisfied.

  • [ ] Verify that QueryOptions.limit and QueryOptions.offset work correctly by adding explicit pagination tests in packages/maid-engine/tests/storage/test_document_store_pagination.py:
  • [ ] Test querying with limit=10, offset=0 returns first 10 documents
  • [ ] Test querying with limit=10, offset=10 returns next 10 documents
  • [ ] Test querying past end of results returns empty list
  • [ ] Test both in-memory and PostgreSQL implementations

0.6 Composite Primary Key on Documents Table

Package: maid-engine | Priority: P0 | Dependencies: none

The current schema uses id UUID PRIMARY KEY (see document_store.py:837-839). Document IDs are only unique within a collection, not across collections. The upsert ON CONFLICT (collection, id) clause requires a composite unique constraint.

  • [ ] Modify PostgresDocumentStore.initialize() in document_store.py:819-854:
  • Change CREATE TABLE to use PRIMARY KEY (collection, id) instead of id UUID PRIMARY KEY
  • Full DDL: CREATE TABLE IF NOT EXISTS {table} (id UUID NOT NULL, collection VARCHAR(255) NOT NULL, data JSONB NOT NULL, created_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), updated_at TIMESTAMP WITH TIME ZONE DEFAULT NOW(), PRIMARY KEY (collection, id))
  • Keep existing indexes: idx_{table}_collection and idx_{table}_data GIN
  • [ ] Add B-tree indexes for persistence queries:
  • CREATE INDEX IF NOT EXISTS idx_{table}_world_id ON {table} ((data->>'world_id'))
  • CREATE INDEX IF NOT EXISTS idx_{table}_source_pack ON {table} ((data->>'source_pack'))
  • [ ] Create migration script at packages/maid-engine/src/maid_engine/storage/migrations/001_composite_pk.sql:
  • ALTER TABLE documents DROP CONSTRAINT IF EXISTS documents_pkey, ADD PRIMARY KEY (collection, id);
  • Include the three B-tree index creations
  • [ ] Document migration procedure for existing deployments
  • [ ] Write test verifying composite PK allows same UUID in different collections

0.7 Phase 0 Integration Tests

Package: maid-engine | Priority: P0 | Dependencies: 0.1–0.6

  • [ ] Write integration test in packages/maid-engine/tests/core/test_phase0_integration.py:
  • [ ] Test that creating entity + adding component emits both EntityCreatedEvent and ComponentAddedEvent
  • [ ] Test that __setattr__ on component fires dirty callback
  • [ ] Test that _suppress_dirty prevents false positives during Pydantic model_validate()
  • [ ] Test upsert works with composite PK on in-memory store

Phase 1: Core Infrastructure (Weeks 2–3)

1.1 Persistence Package Structure

Package: maid-engine | Priority: P0 | Dependencies: Phase 0

  • [ ] Create directory: packages/maid-engine/src/maid_engine/persistence/
  • [ ] Create __init__.py with public exports:
  • ComponentRegistry, EntitySerializer, DirtyTracker, EntityPersistenceManager, SaveScheduler
  • EntityDocument, ComponentData, QuarantinedComponent, QuarantineComponent
  • SaveResult, LoadResult, PersistenceSettings
  • ComponentConflictError, IntegrityError, MigrationGapError

1.2 Persistence Models

Package: maid-engine | Priority: P0 | Dependencies: 1.1

  • [ ] Create packages/maid-engine/src/maid_engine/persistence/models.py:
  • [ ] ComponentData(BaseModel) — fields: component_type: str, schema_version: int = 1, data: dict[str, Any]
  • [ ] QuarantinedComponent(BaseModel) — fields: component_type: str, schema_version: int, raw_data: dict[str, Any]
  • [ ] QuarantineComponent(Component) — ECS component holding quarantined data; component_type: ClassVar[str] = "persistence:quarantine", items: list[QuarantinedComponent] = Field(default_factory=list)
  • [ ] EntityDocument(BaseModel) — fields: entity_id: UUID, world_id: str = "default", tags: set[str], components: list[ComponentData], quarantined_components: list[QuarantinedComponent], source_pack: str, created_at: datetime, updated_at: datetime; add model_config with JSON encoders for UUID, datetime, set
    • Note: checksum and snapshot_id fields deferred to Phase 4 (backup/integrity). PostgreSQL provides storage-level integrity guarantees; checksums add CPU cost on every save/load without commensurate value until untrusted storage backends are supported.
    • Note: RoomStateDocument deferred until grid/room persistence integration is implemented. Define it when the consuming code exists.
  • [ ] Create packages/maid-engine/src/maid_engine/persistence/exceptions.py:
  • [ ] ComponentConflictError(Exception) — raised on duplicate registration
  • [ ] IntegrityError(Exception) — raised on checksum mismatch
  • [ ] MigrationGapError(Exception) — raised on incomplete migration chain
  • [ ] Write tests in packages/maid-engine/tests/persistence/test_models.py:
  • [ ] Test EntityDocument serialization/deserialization round-trip
  • [ ] Test ComponentData model_dump_json() produces deterministic output
  • [ ] Test QuarantineComponent.get_type() returns "persistence:quarantine"
  • [ ] Test set↔list JSON encoding for tags

1.3 Component Registry

Package: maid-engine | Priority: P0 | Dependencies: 1.1, 1.2

  • [ ] Create packages/maid-engine/src/maid_engine/persistence/registry.py:
  • [ ] ComponentRegistry.__init__() — internal dicts: _registry: dict[str, type[Component]], _schema_versions: dict[str, int], _aliases: dict[str, str], _pack_ownership: dict[str, str], _migrations: dict[str, list[tuple[int, int, Callable]]]
  • [ ] register(component_class, schema_version=1, aliases=None, pack_name="", allow_overwrite=False) — register type, raise ComponentConflictError on duplicate unless allow_overwrite=True, track pack ownership
  • [ ] register_migration(component_type, from_version, to_version, migrate_fn) — add migration function, keep sorted by from_version
  • [ ] migrate(component_type, data, from_version) -> dict — apply chained migrations v1→v2→v3; raise MigrationGapError if chain is incomplete
  • [ ] unregister_pack(pack_name) — remove all components and aliases owned by pack
  • [ ] resolve(component_type) -> type[Component] | None — resolve via aliases then registry
  • [ ] is_registered(component_type) -> bool
  • [ ] get_schema_version(component_type) -> int
  • [ ] get_registered_types() -> list[str]
  • [ ] Write tests in packages/maid-engine/tests/persistence/test_registry.py:
  • [ ] Test register and resolve a component type
  • [ ] Test ComponentConflictError on duplicate registration
  • [ ] Test allow_overwrite=True for hot-reload
  • [ ] Test alias resolution (renamed components)
  • [ ] Test unregister_pack() removes correct components and aliases
  • [ ] Test get_schema_version() returns registered version
  • [ ] Test register_migration() and migrate() chaining v1→v2→v3
  • [ ] Test MigrationGapError when chain is incomplete (v1→v2 registered, v2→v3 missing)
  • [ ] Test resolve() returns None for unregistered types

1.4 Entity Serializer

Package: maid-engine | Priority: P0 | Dependencies: 1.2, 1.3

  • [ ] Create packages/maid-engine/src/maid_engine/persistence/serializer.py:
  • [ ] EntitySerializer.__init__(registry: ComponentRegistry)
  • [ ] serialize(entity: Entity) -> EntityDocument:
    • Iterate entity.components, skip "persistence:quarantine" type
    • For each component: verify registered, create ComponentData with get_type(), get_schema_version(), model_dump()
    • Build EntityDocument with entity_id, tags, components, created_at
  • [ ] deserialize(doc: EntityDocument, world: World) -> Entity:
    • Create entity through World to maintain proper manager binding: entity = world.create_entity(entity_id=doc.entity_id) (requires create_entity to accept optional entity_id parameter)
    • Restore tags via add_tag(), set _created_at
    • Deserialize each component via _deserialize_component()
    • Quarantine unresolvable components in QuarantineComponent
    • Set _suppress_dirty = True during deserialization, clear after
    • After adding entity, ensure room index is updated: if entity has PositionComponent, call world._room_index.track(entity_id, position.room_id) or equivalent
  • [ ] _deserialize_component(comp_data: ComponentData) -> Component | None:
    • Resolve class via registry; return None if unregistered
    • Pre-filter: compare model_fields.keys() vs stored data keys, strip unknown fields with warning
    • Apply migrations if stored_version < current_version; return None on MigrationGapError
    • Call component_class.model_validate(data)
  • [ ] Write tests in packages/maid-engine/tests/persistence/test_serializer.py:
  • [ ] Test serialize → deserialize round-trip preserves all data
  • [ ] Test serialization includes all components
  • [ ] Test deserialization with missing component type quarantines data
  • [ ] Test deserialization with extra fields strips them with warning
  • [ ] Test deserialization with schema migration (v1→v2)
  • [ ] Test MigrationGapError causes quarantining
  • [ ] Test _suppress_dirty prevents false dirty marks during deserialization
  • [ ] Test QuarantineComponent survives round-trip serialization
  • [ ] Test deserialized entity with PositionComponent is tracked in room index

1.5 Persistence Settings

Package: maid-engine | Priority: P0 | Dependencies: 1.1

  • [ ] Add PersistenceSettings class to packages/maid-engine/src/maid_engine/config/settings.py:
  • Fields: save_interval: float = 300.0, batch_size: int = 50, upsert_timeout: float = 5.0, batch_timeout: float = 30.0, cycle_timeout: float = 60.0, max_tombstone_retries: int = 10, max_entity_doc_bytes: int = 1_048_576, enabled: bool = True
  • Environment prefix: MAID_PERSISTENCE__
  • [ ] Add persistence: PersistenceSettings field to the main Settings class
  • [ ] Write test verifying settings load from environment variables

1.6 Phase 1 Unit Tests

Package: maid-engine | Priority: P0 | Dependencies: 1.2–1.4

  • [ ] Create test directory: packages/maid-engine/tests/persistence/
  • [ ] Create packages/maid-engine/tests/persistence/__init__.py
  • [ ] Create packages/maid-engine/tests/persistence/conftest.py with shared fixtures:
  • [ ] component_registry fixture — pre-populated with test component types
  • [ ] entity_serializer fixture — initialized with test registry
  • [ ] sample_entity fixture — entity with PositionComponent and HealthComponent
  • [ ] in_memory_store fixture — initialized InMemoryDocumentStore
  • [ ] Ensure all tests from 1.2, 1.3, 1.4 pass
  • [ ] Run uv run pytest packages/maid-engine/tests/persistence/ -v
  • [ ] Run uv run mypy packages/maid-engine/src/maid_engine/persistence/

Phase 2: Change Tracking & Save Scheduling (Weeks 4–5)

2.1 Dirty Tracker

Package: maid-engine | Priority: P0 | Dependencies: 0.1 (ECS events), 1.1

  • [ ] Create packages/maid-engine/src/maid_engine/persistence/dirty_tracker.py:
  • [ ] DirtyTracker.__init__(event_bus: EventBus) — init _dirty_entities: set[UUID], _destroyed_entities: set[UUID], _subscribed: bool = False
  • [ ] start_tracking() — subscribe to EntityCreatedEvent, EntityDestroyedEvent, ComponentAddedEvent, ComponentRemovedEvent, TagAddedEvent, TagRemovedEvent
    • Store handler IDs returned by subscribe() for use in stop_tracking() (EventBus unsubscribes by handler ID, not by (event_type, handler) pair)
  • [ ] stop_tracking() — unsubscribe using stored handler IDs
  • [ ] mark_dirty(entity_id: UUID) — add to dirty set (skip if in destroyed set); warn if dirty set exceeds MAX_DIRTY_SIZE = 50_000
  • [ ] mark_clean(entity_id: UUID) — discard from dirty set
  • [ ] is_dirty(entity_id: UUID) -> bool
  • [ ] get_dirty_count() -> int
  • [ ] drain_dirty() -> tuple[set[UUID], set[UUID]] — atomically copy and clear both sets; return (dirty, destroyed)
  • [ ] peek_dirty() -> tuple[set[UUID], set[UUID]] — copy without clearing (for inspection)
  • [ ] Event handlers: _on_entity_created, _on_entity_destroyed, _on_component_added, _on_component_removed, _on_tag_added, _on_tag_removed — all synchronous (EventBus auto-wraps sync handlers)
  • Prerequisite: Entity.add_tag() and Entity.remove_tag() must emit TagAddedEvent/TagRemovedEvent (add to Phase 0.1 scope — tag mutations are persisted data in EntityDocument.tags and will be silently lost without tracking)
  • [ ] Wire DirtyTracker.mark_dirty to Component._dirty_callback:
  • When persistence layer binds a component to an entity, set component._dirty_callback = dirty_tracker.mark_dirty and component._owner_id = entity.id
  • This binding happens in EntityPersistenceManager when loading entities or when DirtyTracker starts tracking existing entities
  • [ ] Write tests in packages/maid-engine/tests/persistence/test_dirty_tracker.py:
  • [ ] Test entity created → marked dirty
  • [ ] Test entity destroyed → moved to destroyed set, removed from dirty set
  • [ ] Test component added → entity marked dirty
  • [ ] Test component removed → entity marked dirty
  • [ ] Test tag added → entity marked dirty
  • [ ] Test tag removed → entity marked dirty
  • [ ] Test mark_dirty() manual marking
  • [ ] Test mark_clean() removes from dirty set
  • [ ] Test drain_dirty() returns and clears both sets
  • [ ] Test drain_dirty() followed by mark_dirty() adds to fresh set
  • [ ] Test peek_dirty() does not clear sets
  • [ ] Test start_tracking() / stop_tracking() idempotency
  • [ ] Test __setattr__ hook integration: mutate component field → entity appears in dirty set
  • [ ] Test _suppress_dirty prevents dirty marking during model_validate

2.2 Entity Persistence Manager

Package: maid-engine | Priority: P0 | Dependencies: 1.2, 1.3, 1.4, 0.2

  • [ ] Create packages/maid-engine/src/maid_engine/persistence/persistence_manager.py:
  • [ ] Note on World/EntityManager API gap: The current World API has create_entity() and destroy_entity() but no add_entity(entity) for injecting pre-constructed entities, and no get_all_entities(). Before implementing this module:
    • Add add_entity(entity: Entity) -> None to EntityManager and World — must set entity._manager, add to _entities dict, rebuild component indexes (_by_component), rebuild tag indexes (_by_tag), and update _room_index if entity has PositionComponent
    • Add get_all_entities() -> Iterator[Entity] to EntityManager and World
    • Add remove_entity(entity_id: UUID) -> None (distinct from destroy() — only removes from tracking, doesn't clean up the entity itself)
  • [ ] Define EntitySource protocol: get_entity(UUID) -> Entity | None, get_all_entities() -> Iterator[Entity], add_entity(Entity) -> None, remove_entity(UUID) -> None — these must match the new World methods added above
  • [ ] Define SaveResult class: entities_saved: int, entities_failed: int, failed_entities: list[UUID], errors: list[str], success property, total_entities property
  • [ ] Define LoadResult class: entities_loaded: int, entities_failed: int, failed_entities: list[UUID], errors: list[str], success property
  • [ ] EntityPersistenceManager.__init__(entity_source, document_store, component_registry, settings=None):
    • Store references, create EntitySerializer
    • Get entity_collection = document_store.get_collection("entities", EntityDocument)
  • [ ] _delete_tombstones: dict[UUID, int] = {} — in-memory only; not persisted across restarts (on restart, orphaned entities are detected via reconciliation)
    • Apply PersistenceSettings defaults for timeouts and limits
  • [ ] save_entities(dirty_ids, destroyed_ids, world_id) -> SaveResult:
    • Merge destroyed_ids with _delete_tombstones for retry
    • Process deletes first with per-operation timeout
    • Track tombstone retry counts; drop after MAX_TOMBSTONE_RETRIES
    • Build save tasks for dirty entities, skip "transient" tagged entities
    • Execute via _execute_batch() with batch-level timeout
    • Aggregate independent per-entity result tuples into SaveResult
  • [ ] _save_single_entity(entity, world_id) -> tuple[UUID, bool, str | None]:
    • Serialize entity, set updated_at, world_id, source_pack
    • Preserve quarantined components from QuarantineComponent
    • Enforce MAX_ENTITY_DOC_BYTES size limit
    • Upsert with per-operation timeout
    • Return independent result tuple (not shared mutable object)
  • [ ] _execute_batch(tasks, entity_ids, batch_size=50) -> list[tuple]:
    • Process tasks in chunks of batch_size
    • Each chunk: asyncio.wait_for(asyncio.gather(..., return_exceptions=True), timeout=BATCH_TIMEOUT)
    • On batch timeout: report all entities in batch as failed
  • [ ] load_all_entities(world_id, page_size=1000) -> LoadResult:
    • Paginated loading via entity_collection.query(QueryOptions(filters={"world_id": world_id}, limit=page_size, offset=offset))
    • Overall load timeout of 120s
    • Idempotency guard: skip entities already in World
    • Deserialize each EntityDocument, add entity to World
    • await asyncio.sleep(0) between pages for event loop responsiveness
  • [ ] save_all_entities(world_id) -> SaveResult — force save all entities
  • [ ] get_entity_count(world_id) -> int — use entity_collection.count(filters={"world_id": world_id}) (NOT query(limit=0) which returns all results)
  • [ ] has_pack_entities(pack_name, world_id) -> bool
  • [ ] delete_pack_entities(pack_name, world_id) -> int
  • [ ] Write tests in packages/maid-engine/tests/persistence/test_persistence_manager.py:
  • [ ] Test save_entities() with dirty set saves correctly
  • [ ] Test save_entities() with destroyed set deletes from store
  • [ ] Test tombstone queue: failed delete → retried next cycle
  • [ ] Test tombstone dropped after MAX_TOMBSTONE_RETRIES
  • [ ] Test _save_single_entity() returns independent result tuple
  • [ ] Test entity size limit enforcement
  • [ ] Test "transient" tagged entities are skipped
  • [ ] Test load_all_entities() paginates correctly
  • [ ] Test load_all_entities() idempotency guard skips existing entities
  • [ ] Test load_all_entities() handles corrupted documents gracefully
  • [ ] Test has_pack_entities() returns correct boolean
  • [ ] Test delete_pack_entities() removes from store and World
  • [ ] Test save_all_entities() saves every entity

2.3 Save Scheduler

Package: maid-engine | Priority: P0 | Dependencies: 2.1, 2.2

  • [ ] Create packages/maid-engine/src/maid_engine/persistence/save_scheduler.py:
  • [ ] Define WorldSaveStartedEvent(Event) — fields: started_at: datetime
  • [ ] Define WorldSaveCompletedEvent(Event) — fields: started_at, completed_at, duration, entities_saved, entities_failed, success
  • [ ] SaveScheduler.__init__(persistence_manager, dirty_tracker, event_bus, save_interval=300.0):
    • Store references, init _save_lock = asyncio.Lock(), _last_save_time, _shutdown = False
  • [ ] start() — create asyncio.create_task(self._save_loop())
  • [ ] stop() — set _shutdown, cancel task, perform final save via trigger_save()
  • [ ] trigger_save() -> bool — call _perform_save()
  • [ ] _save_loop() — first save after 10s delay, then every save_interval; catch and log exceptions
  • [ ] _perform_save() -> bool:
    • Acquire _save_lock (TOCTOU-safe via async with)
    • Emit WorldSaveStartedEvent
    • drain_dirty() atomically before persisting
    • Return True if nothing to save
    • Call persistence_manager.save_entities() with CYCLE_TIMEOUT
    • Re-mark failed entities via dirty_tracker.mark_dirty()
    • On timeout: re-mark ALL entities (dirty + destroyed)
    • Emit WorldSaveCompletedEvent
  • [ ] get_next_save_time() -> datetime | None
  • [ ] get_stats() -> dict — for admin interface
  • [ ] Write tests in packages/maid-engine/tests/persistence/test_save_scheduler.py:
  • [ ] Test periodic save fires after interval
  • [ ] Test initial short delay (10s) on first save
  • [ ] Test trigger_save() performs immediate save
  • [ ] Test concurrent trigger_save() waits on lock (no TOCTOU)
  • [ ] Test save failure re-marks entities as dirty
  • [ ] Test timeout re-marks all entities including destroyed
  • [ ] Test WorldSaveStartedEvent emitted at start
  • [ ] Test WorldSaveCompletedEvent emitted at end with correct stats
  • [ ] Test stop() performs final save
  • [ ] Test get_stats() returns correct data

2.4 Observable Container Wrappers — DEFERRED

Deferred to Phase 4 or later. All three reviewers agreed that ObservableList/ObservableDict/ObservableSet add significant complexity without proportional value. The notify_mutation() method from Phase 0.3 is sufficient for correctness — content pack authors call it after mutating container fields in place. Observable containers can be added later as a convenience optimization if notify_mutation() proves to be a pain point in practice.

2.5 Phase 2 Integration Tests

Package: maid-engine | Priority: P0 | Dependencies: 2.1–2.3

  • [ ] Write integration tests in packages/maid-engine/tests/persistence/test_phase2_integration.py:
  • [ ] Test full save cycle: create entity → mutate component → save scheduler fires → entity persisted in store
  • [ ] Test only dirty entities are saved (clean entities not touched)
  • [ ] Test entity destruction → delete from store on next save cycle
  • [ ] Test save failure → entity re-marked dirty → retried next cycle
  • [ ] Test __setattr__ auto-dirty → save cycle persists changes
  • [ ] Test notify_mutation() for container mutations → save cycle persists
  • [ ] Test save events emitted with correct stats
  • [ ] Run uv run pytest packages/maid-engine/tests/persistence/ -v
  • [ ] Run uv run mypy packages/maid-engine/src/maid_engine/persistence/

Phase 3: Production Integration (Weeks 6–7)

3.1 GameEngine Persistence Integration

Package: maid-engine | Priority: P0 | Dependencies: Phase 2

  • [ ] Add persistence attributes to GameEngine.__init__() in packages/maid-engine/src/maid_engine/core/engine.py:94:
  • self._component_registry = ComponentRegistry()
  • self._dirty_tracker: DirtyTracker | None = None
  • self._persistence_manager: EntityPersistenceManager | None = None
  • self._save_scheduler: SaveScheduler | None = None
  • [ ] Add properties: component_registry, persistence_manager, save_scheduler
  • [ ] Modify GameEngine.start() in engine.py:426:
  • After await self._document_store.initialize() (line 438):
    1. Register persistence schemas: self._document_store.register_schema("entities", EntityDocument)
    2. Call pack.register_component_types(self._component_registry) in the pack loop
    3. Register QuarantineComponent in registry
    4. Create EntityPersistenceManager(world, document_store, component_registry, settings.persistence)
    5. Call await persistence_manager.load_all_entities(world_id="default")
    6. Log loaded entity count
  • Then run content pack on_load() calls (load-then-supplement: packs check has_pack_entities() to avoid re-creating defaults)
  • After await self._world.startup():
    1. Create DirtyTracker(self._world.events)
    2. Bind _dirty_callback on all loaded entities' components
    3. Call dirty_tracker.start_tracking()must be after entity loading to avoid false dirty marks from load-time entity creation events
    4. Create SaveScheduler(persistence_manager, dirty_tracker, world.events, settings.persistence.save_interval)
    5. Call save_scheduler.start()
  • Startup sequence invariant: initialize persistence → load entities → run pack on_load() → bind dirty callbacks → start dirty tracking
  • [ ] Modify GameEngine.stop() in engine.py:519:
  • Before await self._world.shutdown():
    1. Call await save_scheduler.stop() (performs final save)
    2. Call dirty_tracker.stop_tracking()
  • [ ] Modify GameEngine._tick_loop() in engine.py:590:
  • After await self._world.tick(delta) and before sleep calculation:
    • This is the save window — systems have finished, state is quiescent
    • If save_scheduler has a pending save, invoke snapshot phase here
  • [ ] Write tests in packages/maid-engine/tests/persistence/test_engine_integration.py:
  • [ ] Test GameEngine.start() loads persisted entities
  • [ ] Test GameEngine.stop() saves all dirty entities
  • [ ] Test register_component_types() called on each content pack
  • [ ] Test entities survive engine restart (start → create entity → stop → start → verify entity exists)

3.2 ContentPack Load/Supplement Pattern

Package: maid-engine | Priority: P0 | Dependencies: 3.1

The design calls for a "load-then-supplement" startup pattern: persisted entities are loaded first, then content packs check for existing entities before creating defaults.

  • [ ] Add has_pack_entities() convenience method on GameEngine or expose via persistence_manager
  • [ ] Update content pack authoring guide with idempotency pattern:
    async def on_load(self, engine: GameEngine) -> None:
        if await engine.persistence_manager.has_pack_entities("my-pack"):
            return  # Entities already exist from persistence
        # First run: create default entities
        self._create_default_world(engine.world)
    
  • [ ] Update StdlibContentPack.on_load() in packages/maid-stdlib/src/maid_stdlib/pack.py to check for persisted entities
  • [ ] Update ClassicRPGContentPack.on_load() in packages/maid-classic-rpg/src/maid_classic_rpg/pack.py to check for persisted entities
  • [ ] Update TutorialWorldContentPack.on_load() in packages/maid-tutorial-world/src/maid_tutorial_world/pack.py to check for persisted entities
  • [ ] Write tests:
  • [ ] Test pack on_load() creates entities on first run
  • [ ] Test pack on_load() skips creation when entities already persisted

3.3 Content Pack Component Registration

Package: maid-stdlib, maid-classic-rpg, maid-tutorial-world | Priority: P0 | Dependencies: 3.1

  • [ ] Implement register_component_types() on StdlibContentPack in packages/maid-stdlib/src/maid_stdlib/pack.py:
  • Register: PositionComponent, HealthComponent, ManaComponent, InventoryComponent, DialogueComponent, ExtendedRoomComponent, and all other stdlib components from packages/maid-stdlib/src/maid_stdlib/components/
  • Use pack_name="stdlib"
  • [ ] Implement register_component_types() on ClassicRPGContentPack in packages/maid-classic-rpg/src/maid_classic_rpg/pack.py:
  • Register all classic RPG components from packages/maid-classic-rpg/src/maid_classic_rpg/components/
  • Use pack_name="classic-rpg", set appropriate schema_version
  • [ ] Implement register_component_types() on TutorialWorldContentPack (if it has custom components)
  • [ ] Add startup migration chain validation: after all packs call register_component_types(), iterate all registered types and verify migration chains are complete (no gaps from v1 to current version). Log warnings or fail fast on gaps. (Design doc §6.2 requires this.)
  • [ ] Add a real migration verification test: deliberately version-bump a test component in maid-classic-rpg and register a dummy migration function to prove schema evolution works end-to-end
  • [ ] Write tests:
  • [ ] Test all stdlib components are registered after register_component_types() call
  • [ ] Test all classic-rpg components are registered
  • [ ] Test component types can be resolved by registry after registration
  • [ ] Test migration chain validation detects gaps
  • [ ] Test migration chain validation passes for complete chains

3.4 Builder Command Persistence Tagging

Package: maid-stdlib | Priority: P1 | Dependencies: 3.1

Builder-created entities (via @create, @dig) must be tagged with source_pack so they persist correctly.

source_pack population strategy: Entities must have their source_pack set at creation time: - Content packs set source_pack during on_load() entity creation (e.g., "stdlib", "classic-rpg") - Builder-created entities get source_pack = "builder" - The source_pack is stored as entity metadata (a tag like "source:stdlib" or a dedicated field on Entity) - EntitySerializer.serialize() reads source_pack from the entity's metadata when building EntityDocument

  • [ ] Modify @create command in packages/maid-stdlib/src/maid_stdlib/commands/building/create.py:
  • After creating entity, set source_pack metadata (e.g., add tag "builder_created" or store pack info)
  • Entities created by builders should persist by default (no "transient" tag)
  • [ ] Modify @dig command in packages/maid-stdlib/src/maid_stdlib/commands/building/dig.py:
  • Same persistence tagging for newly created rooms and exits
  • [ ] Modify @destroy command in packages/maid-stdlib/src/maid_stdlib/commands/building/destroy.py:
  • Destroying an entity should trigger EntityDestroyedEvent (already handled by ECS events from Phase 0)
  • Verify the entity is removed from persistence on next save cycle
  • [ ] Write tests:
  • [ ] Test @create entity survives save/load cycle
  • [ ] Test @dig room survives save/load cycle
  • [ ] Test @destroy entity is deleted from persistence

3.5 Admin Commands for Persistence

Package: maid-engine | Priority: P1 | Dependencies: 3.1

  • [ ] Create @save admin command (or add to existing admin commands):
  • Triggers save_scheduler.trigger_save() manually
  • Outputs save result: entities saved, failed, duration
  • Requires ADMIN access level
  • [ ] Create @persist status command:
  • Shows: dirty count, last save time, next save time, save interval, tombstone count
  • Shows: total persisted entities, entities per content pack
  • [ ] Create @persist flush command:
  • Forces save_all_entities() regardless of dirty state
  • Requires SUPERADMIN access level
  • [ ] Add persistence stats to admin dashboard API:
  • Extend GET /admin/dashboard/ to include persistence metrics
  • dirty_count, last_save_time, save_interval, persisted_entity_count
  • [ ] Write tests for admin commands

3.6 Dirty Callback Binding for Loaded Entities

Package: maid-engine | Priority: P0 | Dependencies: 2.1, 3.1

When entities are loaded from persistence or created at runtime, their components need _dirty_callback and _owner_id bound.

  • [ ] Create helper function bind_entity_dirty_tracking(entity: Entity, dirty_tracker: DirtyTracker) -> None:
  • For each component on the entity:
    • Set component._dirty_callback = dirty_tracker.mark_dirty
    • Set component._owner_id = entity.id
  • [ ] Hook into EntityCreatedEvent or ComponentAddedEvent to auto-bind new components:
  • When a new component is added to a tracked entity, bind its callback
  • This ensures components added after initial load are also tracked
  • [ ] Write tests:
  • [ ] Test loaded entity components have _dirty_callback bound
  • [ ] Test newly added component gets _dirty_callback bound
  • [ ] Test component mutation after binding marks entity dirty

3.7 Player Entity Persistence Strategy

Package: maid-engine | Priority: P1 | Dependencies: 3.1

The existing CharacterManager (legacy, in maid-classic-rpg) handles player account data. EntityPersistenceManager handles ECS entity state. Without explicit coordination, player entities risk being double-loaded or having state overwritten.

  • [ ] Define the boundary: CharacterManager owns account-level data (credentials, character list); EntityPersistenceManager owns runtime ECS state (position, health, effects)
  • [ ] Tag player entities as "transient" so EntityPersistenceManager skips them during save_entities(), OR migrate CharacterManager to delegate entity hydration to EntityPersistenceManager entirely
  • [ ] Document the chosen approach and update both subsystems accordingly
  • [ ] Write test: verify no duplicate entity creation when both systems interact during login

3.8 Phase 3 Integration Tests

Package: all | Priority: P0 | Dependencies: 3.1–3.7

  • [ ] Write integration tests in packages/maid-engine/tests/persistence/test_phase3_integration.py:
  • [ ] Full restart test: Start engine → create entities via builder commands → stop → start → verify entities exist with correct components
  • [ ] Content pack idempotency: Start with persistence → verify pack on_load() doesn't duplicate entities
  • [ ] Cross-pack persistence: Entities from stdlib and classic-rpg both persist and load correctly
  • [ ] Manual save: Trigger @save → verify all dirty entities persisted
  • [ ] Admin status: Verify @persist status shows correct stats
  • [ ] Hot reload: Unload pack → entities from that pack quarantine components → reload pack → components restored

Phase 4: Advanced Features (Weeks 8–10)

4.1 Backup/Restore System

Package: maid-engine | Priority: P2 | Dependencies: Phase 3

  • [ ] Create packages/maid-engine/src/maid_engine/persistence/backup.py:
  • [ ] BackupManager class:
    • create_snapshot(world_id, label) -> SnapshotManifest — full snapshot of all entities
    • list_snapshots(world_id) -> list[SnapshotManifest]
    • restore_snapshot(snapshot_id, world_id) — restore entities from snapshot
    • delete_snapshot(snapshot_id)
  • [ ] SnapshotManifest(BaseModel):
    • snapshot_id: str, world_id: str, created_at: datetime, entity_count: int, label: str
  • [ ] Store snapshots in persistence_meta collection with type: "snapshot"
  • [ ] Create @backup admin command:
  • @backup create [label] — create full snapshot
  • @backup list — list available snapshots
  • @backup restore <snapshot_id> — restore from snapshot (requires SUPERADMIN)
  • @backup delete <snapshot_id>
  • [ ] Write tests:
  • [ ] Test create snapshot captures all entities
  • [ ] Test restore snapshot replaces current entities
  • [ ] Test list/delete operations

4.2 Corruption Detection and Repair

Package: maid-engine | Priority: P2 | Dependencies: 4.1

Entity checksums and snapshot_id are deferred to this phase. Add checksum: str | None and snapshot_id: str | None optional fields to EntityDocument here.

  • [ ] Add _compute_checksum() to EntitySerializer: sort components by type, use model_dump_json(), SHA-256 hex digest
  • [ ] Add optional checksum computation to save path (off by default, enabled via PersistenceSettings.enable_checksums)
  • [ ] Create packages/maid-engine/src/maid_engine/persistence/integrity.py:
  • [ ] IntegrityChecker class:
    • check_all_entities(world_id) -> IntegrityReport — verify checksums on all persisted entities
    • repair_entity(entity_id) — recompute and save checksum
    • quarantine_entity(entity_id) — move corrupted entity to quarantine collection
  • [ ] IntegrityReport(BaseModel):
    • total_checked: int, valid: int, corrupted: list[UUID], missing_checksums: list[UUID]
  • [ ] Create @persist check admin command:
  • Runs integrity check and reports results
  • [ ] Write tests:
  • [ ] Test integrity checker detects checksum mismatch
  • [ ] Test repair recomputes correct checksum
  • [ ] Test report format

4.3 Performance Optimization: Bulk Upserts

Package: maid-engine | Priority: P2 | Dependencies: Phase 3

  • [ ] Add upsert_many(docs: list[tuple[UUID, T]]) -> list[bool] to DocumentCollection:
  • PostgreSQL: Multi-row INSERT ... VALUES (...), (...), ... ON CONFLICT DO UPDATE
  • Batch size: 50–100 rows per statement
  • In-memory: iterate and upsert individually
  • [ ] Update EntityPersistenceManager._execute_batch() to use upsert_many when available
  • [ ] Add executor offloading for large batches (>100 entities):
  • Main thread: snapshot via model_dump() for each dirty entity
  • run_in_executor(): JSON encoding, checksum computation, EntityDocument construction
  • Gate on EXECUTOR_THRESHOLD = 100
  • [ ] Write benchmarks:
  • [ ] Benchmark: 500 entity incremental save < 500ms
  • [ ] Benchmark: 10,000 entity full save completes in reasonable time
  • [ ] Benchmark: memory overhead < 100 bytes per entity for dirty tracking

4.4 Monitoring and Metrics

Package: maid-engine | Priority: P2 | Dependencies: Phase 3

  • [ ] Add persistence metrics to profiling system:
  • Save cycle duration histogram
  • Entity save success/failure counters
  • Dirty entity gauge
  • Tombstone queue size
  • Document store operation latencies
  • [ ] Add persistence stats to GameEngine.get_stats() in engine.py:918
  • [ ] Add WebSocket streaming for persistence events to admin dashboard
  • [ ] Write tests for metrics collection

4.5 Documentation

Package: all | Priority: P1 | Dependencies: Phase 3

  • [ ] Create docs/persistence.md — user-facing guide:
  • How persistence works (save cycles, dirty tracking)
  • Configuration (MAID_PERSISTENCE__* env vars)
  • Content pack integration (register_component_types, idempotency pattern)
  • Admin commands (@save, @persist, @backup)
  • Troubleshooting (quarantined components, save failures)
  • [ ] Create docs/content-pack-persistence.md — content pack authoring guide:
  • How to register component types
  • Schema evolution: adding/removing fields, migration functions
  • Using notify_mutation() for in-place container mutations (e.g., list.append, dict update)
  • The "transient" tag for non-persistent entities
  • Testing persistence in content packs
  • [ ] Update CLAUDE.md with persistence commands and configuration
  • [ ] Update docs/designs/v3.1/01-durable-persistence.md with implementation status notes
  • [ ] Add Google-style docstrings to all public APIs in persistence/ package

4.6 Phase 4 Integration Tests

Package: maid-engine | Priority: P2 | Dependencies: 4.1–4.4

  • [ ] Write end-to-end tests in packages/maid-engine/tests/persistence/test_phase4_e2e.py:
  • [ ] Test full backup/restore cycle
  • [ ] Test corruption detection catches tampered entity
  • [ ] Test bulk upsert performance is better than individual upserts
  • [ ] Test monitoring metrics are populated after save cycles
  • [ ] Test large world (1000+ entities) save and load

Dependencies Summary

Phase 0 (mostly parallel):
  0.1 ECS Events (incl. tag events)
  0.2 Upsert (depends on 0.6)
  0.3 Component __setattr__
  0.4 ContentPack Protocol
  0.5 Pagination (verify existing)
  0.6 Composite PK

Phase 1 (depends on Phase 0):
  1.1 Package Structure
  1.2 Models → 1.3 Registry → 1.4 Serializer
  1.5 Settings

Phase 2 (depends on Phase 1):
  2.1 DirtyTracker → 2.3 SaveScheduler
  2.2 PersistenceManager → 2.3 SaveScheduler
  (Observable containers deferred)

Phase 3 (depends on Phase 2):
  3.1 Engine Integration → 3.2 Load/Supplement → 3.3 Pack Registration
  3.4 Builder Persistence (depends on 3.1)
  3.5 Admin Commands (depends on 3.1)
  3.6 Dirty Callback Binding (depends on 2.1, 3.1)
  3.7 Player Entity Strategy (depends on 3.1)

Phase 4 (depends on Phase 3):
  4.1 Backup/Restore (incl. snapshot_id, checksums)
  4.2 Corruption Detection (depends on 4.1)
  4.3 Bulk Upserts (parallel with 4.1)
  4.4 Monitoring (parallel with 4.1)
  4.5 Documentation

Resource Allocation

Role FTE Primary Focus
Systems Developer 1.0 ECS events, Component hooks, Registry, Serializer, PersistenceManager
Infrastructure Developer 0.5 DocumentStore extensions, composite PK migration, bulk upserts
Integration Developer 0.5 Engine integration, content pack updates, admin commands
QA Engineer 0.5 Test authoring, performance benchmarks, crash recovery tests

Success Criteria

  • [ ] All world state (NPCs, items, room modifications, builder content) survives server restart
  • [ ] Only dirty entities are saved per cycle (not full-world O(N) writes)
  • [ ] Incremental save of 500 entities completes in <500ms
  • [ ] Full load of 50,000 entities completes in <5 seconds
  • [ ] Content packs can register component types and detect first-run vs restart
  • [ ] Builder-created content (@create, @dig) persists across restarts
  • [ ] Schema evolution handles added/removed component fields gracefully
  • [ ] Uninstalled content pack component data is quarantined (not lost)
  • [ ] Save failures are retried automatically on next cycle
  • [ ] Delete failures are tracked via tombstone queue and retried
  • [ ] No false-positive dirty marks during deserialization (_suppress_dirty)
  • [ ] Test coverage >80% for all persistence code
  • [ ] MyPy strict mode passes for all persistence modules

Prerequisites / Blockers from Other Design Docs

  • No external blockers identified. This design is self-contained within the existing maid-engine infrastructure.
  • The existing WorldPersistenceHelper (packages/maid-engine/src/maid_engine/world/persistence.py) handles grid/wilderness state separately and is not replaced by this work. Both systems coexist — world system state (grid layout, wilderness config) uses WorldPersistenceHelper, entity state uses EntityPersistenceManager.
  • The existing CharacterManager in maid-classic-rpg handles player account data separately and is not replaced. Player ECS entities will either be tagged "transient" (persisted only by CharacterManager) or migrated to use EntityPersistenceManager — see Phase 3.7 for the resolution.
  • Explicitly deferred: Observable containers (Phase 0.3 notify_mutation() is sufficient), entity checksums (PostgreSQL provides storage-level integrity), snapshot_id per entity (deferred to Phase 4 backup), RoomStateDocument (no consuming code yet), cross-world entity deep-copy (§9.1.1 in design doc — substantial standalone feature), BlobStore for large binary data (§9.3 in design doc).