leistungsbilanz-ts/docs/project-history-and-external-model-architecture.md

# Project History and External Model Architecture

## Status

This document records architectural direction for future work. It is not a final database schema and does not authorize implementing Revit exchange or migrating to PostgreSQL during unrelated phases.

## Goals

- persist undo/redo across page reloads and application restarts
- provide a project-wide version history with named restore points
- preserve an auditable sequence of user, import, migration and restore operations
- support future Revit/CSV round-trips using stable IFCGUID-based links
- keep the persistence boundary replaceable so PostgreSQL can be adopted later

## Separate Concerns

The following mechanisms solve different problems and must not be conflated:

- database backups protect against database loss or corruption
- project snapshots provide user-visible restore points
- change history records logical application operations
- undo/redo navigates or inverses eligible logical operations
- external import batches describe data exchanged with another model

Database backups remain necessary even after project version history exists.

## Revision and Change-Set Direction

History is project-wide. Every accepted mutating command should eventually execute through one server-side transaction that:

1. validates the expected current project revision
2. applies all domain writes
3. records one immutable change set with sufficient before/after data
4. increments the project revision

The conceptual entities are:

- `ProjectRevision`
  - monotonically increasing revision number within one project
  - timestamp, actor and source (`user`, `undo`, `redo`, `import`, `restore`, `migration`)
  - optional label or description
- `ProjectChangeSet`
  - one logical user or system command
  - contains one or more domain operations committed atomically
  - stores enough before/after state to validate and apply an inverse
- `ProjectSnapshot`
  - a logical, database-independent representation of one project revision
  - may be created periodically or explicitly named by a user
  - accelerates restore without replaying the entire history

Full event sourcing is not required. Current normalized tables remain the primary read model. Immutable change sets plus periodic snapshots provide the required history without forcing every query to rebuild state from events.

## Undo, Redo and Restore Semantics

- Undo applies an inverse as a new recorded revision; it does not delete history.
- Redo reapplies an eligible undone operation as a new recorded revision.
- A new edit after undo may hide the prior redo branch in the normal UI, while the historical revisions remain auditable.
- Restoring a snapshot creates a new revision whose state matches the selected historical revision.
- The server, not React component state, is the source of truth for available undo/redo operations.
- After a page reload, the client requests the current revision and eligible history actions.
- Commands include an expected project revision. A stale command is rejected instead of silently overwriting a newer project state.

The circuit-list editor uses the server state directly for operation
eligibility. React callbacks only initiate forward commands and provide
best-effort selection hints after the tree reload.

## Snapshot Storage

Project history must use logical project data, not copies of the SQLite database file. This keeps snapshots:

- project-scoped
- portable across SQLite and PostgreSQL
- independent from unrelated projects
- suitable for validation, comparison and selective restore tooling

Large snapshots may later be compressed or stored in object storage, with metadata and checksums retained in the database.

Implemented named-snapshot foundation:

- `project_snapshots` stores project ownership, source revision, schema
  version, name, optional description, logical payload, SHA-256 and creation
  metadata
- snapshot creation requires the expected current revision and reads plus
  stores the complete project state in one SQLite transaction
- the versioned payload contains project settings, distribution boards,
  circuit lists, sections, circuits with device rows, project devices, floors
  and rooms while excluding global and upgrade-only data
- snapshot creation/listing does not create a project revision or affect
  undo/redo eligibility
- public Create/List endpoints expose metadata but not the logical payload
- restore verifies the stored checksum and a hash of the complete current
  state before replacing supported project data in one transaction
- restore records a new `restore` revision containing both the selected target
  state and the complete pre-restore inverse; undo/redo therefore remains
  available after application restarts
- compatible upgrade-only Consumer links, mappings and reports are preserved
  separately and are never imported into the logical snapshot payload
- arbitrary restore payloads are rejected by the generic user-command
  endpoint; only server-stored snapshots can initiate a forward restore
- the project page provides an initially collapsed snapshot/timeline panel,
  explicit inline restore confirmation and cursor-based loading of older
  revisions; a successful restore reloads all project read models

Implemented automatic-snapshot policy:

- `project_snapshots.kind` distinguishes `named` from `automatic`
- the central revision boundary captures the complete post-command state after
  every 25 additional project revisions
- the newest 12 automatic snapshots per project are retained
- retention never removes named snapshots, immutable revisions or physical
  database backups
- automatic capture and retention run inside the same transaction as the
  triggering project revision

## Revit / CSV / IFCGUID Round-Trip Direction

External model exchange should use explicit staging and link entities instead of writing imported rows directly into circuits.

Conceptual entities:

- `ExternalModelSource`
  - identifies a Revit/IFC model or another external source within a project
- `ExternalImportBatch`
  - file name, checksum, import time, format/version and processing status
- `ExternalModelObject`
  - stable IFCGUID and optional Revit element identifiers
  - source attributes and last-seen import batch
- `ExternalObjectLink`
  - explicit relationship to a project device, circuit device row or another future domain entity
- `ExternalParameterMapping`
  - maps imported/exported columns to application fields
- `ExternalExportBatch`
  - records which project revision and import source produced an exported file

Required behavior:

- IFCGUID uniqueness is scoped to project and external model source.
- Parsing writes to staging first.
- Users preview additions, changes, missing objects and conflicts before applying them.
- Applying an import creates one project revision and one auditable change set.
- Imported source values and locally assigned planning values remain distinguishable.
- Export retains IFCGUID and source identifiers so Revit parameters can be matched deterministically.
- Missing objects and duplicate/conflicting identifiers are reported, never silently reassigned.

## SQLite and PostgreSQL Direction

SQLite remains appropriate for the current local, single-process development workflow and can handle initial external-object volumes. Data volume alone is not a reason to migrate immediately.

PostgreSQL becomes appropriate when one or more of these conditions apply:

- multiple users edit projects concurrently
- the API runs as a shared network service
- background import/export jobs need independent workers
- revision/history queries or external-object datasets need server-side concurrency and operational tooling
- centralized backup, access control and monitoring become required

To preserve that option:

- domain services must not depend directly on `better-sqlite3`
- new multi-write workflows use an injectable transaction/unit-of-work boundary
- database-generated behavior is kept behind repositories
- IDs remain stable UUIDs and revisions use explicit project-scoped sequence numbers
- history snapshots remain logical and database-independent
- SQLite-specific synchronous transaction callbacks are treated as an adapter detail
- real persistence tests are separated so PostgreSQL parity tests can be added later

Drizzle schemas and migrations are dialect-specific and will require a deliberate PostgreSQL schema/migration path. The goal is portable domain behavior, not pretending the current database schema files can be switched without work.

## Near-Term Refactoring Constraints

Completed foundation:

- real SQLite commit/rollback tests cover critical multi-write commands
- transaction ownership for row moves, synchronization, renumbering and migration
  is isolated in injected persistence adapters
- database backup and independent restore verification are automated
- the Consumer application path is removed and stable domain IDs are preserved
- editor grid projection and safety rules are separated from React rendering
- projects carry a monotonic `currentRevision` counter starting at zero
- project revision metadata and versioned forward/inverse change-set payloads
  have separate persistence entities
- forward and inverse payloads store complete serialized command envelopes;
  command type and schema version are derived from those envelopes rather than
  duplicated caller metadata
- command serialization rejects non-finite numbers, `undefined`, non-plain
  objects and cyclic payloads before a revision transaction starts
- the SQLite revision repository advances the project counter and stores both
  history records atomically
- revision appends use an expected-revision compare-and-set and reject stale
  callers without writing partial history
- a first internal `circuit.update` command validates project/list/section
  ownership and equipment-identifier uniqueness inside its transaction
- circuit updates derive their inverse from the persisted pre-command row and
  commit the domain update, inverse command, change set and revision together
- an internal `circuit-device-row.update` command applies the same boundary to
  device fields, validates project-device and room ownership and records
  automatically derived local override metadata in its normalized forward
  command
- integration tests apply the generated inverse as a new `undo` revision and
  verify rollback for stale revisions and late history failures on both update
  command types
- project-scoped relational undo/redo stacks persist eligible original change
  sets independently from immutable audit revisions
- stack transitions enforce LIFO order, keep undo/redo revisions out of the
  eligibility stacks and clear the redo branch on a new user command
- domain writes, audit revisions and stack transitions share one transaction;
  a forced stack failure rolls all of them back
- `GET /api/projects/:projectId/history` exposes current revision, stack depths
  and top change-set ids
- `GET /api/projects/:projectId/history/revisions` exposes a descending,
  cursor-paginated metadata timeline without leaking persisted forward or
  inverse command payloads
- a central application dispatcher executes the supported `circuit.update`
  and `circuit-device-row.update` envelopes without coupling the domain service
  to SQLite
- public command, undo and redo endpoints require an expected revision; undo
  and redo reconstruct the eligible persisted inverse or forward command
- the circuit tree returns the project's current revision; existing Circuit
  and CircuitDeviceRow cell edits use the public command endpoint, while their
  toolbar undo/redo actions use project-wide persistent history
- obsolete direct Circuit and CircuitDeviceRow field-update PATCH routes are
  removed so these editor writes cannot bypass revision history
- `circuit-device-row.insert` and `circuit-device-row.delete` preserve stable
  row ids, complete deleted-row snapshots and circuit reserve state through
  atomic insert/delete, revision and stack transitions
- `circuit.insert` and `circuit.delete` persist a complete circuit block with
  zero, one or multiple device rows; undo restores all ids and fields without
  renumbering
- standalone Circuit/CircuitDeviceRow insert/delete UI paths use these commands
  with client-generated stable UUIDs; obsolete direct structure POST and
  CircuitDeviceRow DELETE routes are removed
- `circuit-device-row.move` stores exact expected and target circuit/sort
  positions for one or multiple rows, recalculates all affected reserve states
  and reconstructs a deterministic inverse across multiple source circuits
- `circuit-device-row.move-with-new-circuit` additionally records the complete
  empty target-circuit snapshot; forward creation and all row moves share one
  transaction, while undo restores the source positions and deletes only an
  unchanged generated target
- the circuit-list editor executes existing-target and generated-target row
  moves through these commands; obsolete direct move and Circuit DELETE routes
  plus their separate transaction adapter are removed
- `circuit.reorder-section` requires the complete section circuit set and
  stores exact expected/target sort positions; its inverse changes no
  equipment identifier and never splits a circuit block
- `circuit.reorder-sections` applies the same invariants to every affected
  section from a sorted view in one transaction and one history entry
- circuit drag-and-drop and sorted-order application use these commands; the
  obsolete direct reorder route and transaction method are removed
- `circuit.renumber-section` is the separate explicit renumber operation; it
  stores every expected/target equipment identifier, uses collision-safe
  temporary values and restores the exact prior identifiers on undo
- the editor's explicit renumber action uses this command; obsolete direct
  renumber and identifier-restore routes plus their transaction adapter are
  removed
- the circuit-list editor loads project-wide undo/redo eligibility together
  with the tree, retries revision mismatches and keeps Undo/Redo available after
  page reloads without a session-local command stack
- `project-device.sync-rows` records synchronization, disconnect and reconnect
  as one atomic multi-row operation; complete expected/target sync snapshots
  protect local values, links and override metadata against silent stale writes
- `project-device.update` persists canonical source-device field changes
  independently and never synchronizes linked rows implicitly
- `project-device.insert` and `project-device.delete` preserve stable device
  ids; delete undo restores links only from complete unchanged disconnected-row
  snapshots in the same transaction
- `project.update-settings` persists both project voltage defaults, its exact
  inverse, revision and history transition atomically; the project page and
  existing PUT route require the current expected revision

Remaining constraints before completing project version history:

- extend project-scoped commands only when further project mutations are
  deliberately added to history; the generic versioned envelope is complete
- do not model IFCGUID as an overloaded circuit equipment identifier
- keep database backup/restore checks separate from project history tests

## Deferred Decisions

- whether large snapshot payloads remain in PostgreSQL or move to object storage
- branch visualization after undo followed by new edits
- user/role model and actor attribution
- supported Revit CSV dialects and parameter mapping UI
- PostgreSQL deployment topology and migration date