leistungsbilanz-ts/docs/project-history-and-external-model-architecture.md

Project History and External Model Architecture

Status

This document records architectural direction for future work. It is not a final database schema and does not authorize implementing Revit exchange or migrating to PostgreSQL during unrelated phases.

Goals

• persist undo/redo across page reloads and application restarts
• provide a project-wide version history with named restore points
• preserve an auditable sequence of user, import, migration and restore operations
• support future Revit/CSV round-trips using stable IFCGUID-based links
• keep the persistence boundary replaceable so PostgreSQL can be adopted later

Separate Concerns

The following mechanisms solve different problems and must not be conflated:

• database backups protect against database loss or corruption
• project snapshots provide user-visible restore points
• change history records logical application operations
• undo/redo navigates or inverses eligible logical operations
• external import batches describe data exchanged with another model

Database backups remain necessary even after project version history exists.

Revision and Change-Set Direction

History is project-wide. Every accepted mutating command should eventually execute through one server-side transaction that:

1. validates the expected current project revision
2. applies all domain writes
3. records one immutable change set with sufficient before/after data
4. increments the project revision

The conceptual entities are:

• ProjectRevision
  • monotonically increasing revision number within one project
  • timestamp, actor and source (user, undo, redo, import, restore, migration)
  • optional label or description
• ProjectChangeSet
  • one logical user or system command
  • contains one or more domain operations committed atomically
  • stores enough before/after state to validate and apply an inverse
• ProjectSnapshot
  • a logical, database-independent representation of one project revision
  • may be created periodically or explicitly named by a user
  • accelerates restore without replaying the entire history

Full event sourcing is not required. Current normalized tables remain the primary read model. Immutable change sets plus periodic snapshots provide the required history without forcing every query to rebuild state from events.

Undo, Redo and Restore Semantics

• Undo applies an inverse as a new recorded revision; it does not delete history.
• Redo reapplies an eligible undone operation as a new recorded revision.
• A new edit after undo may hide the prior redo branch in the normal UI, while the historical revisions remain auditable.
• Restoring a snapshot creates a new revision whose state matches the selected historical revision.
• The server, not React component state, is the source of truth for available undo/redo operations.
• After a page reload, the client requests the current revision and eligible history actions.
• Commands include an expected project revision. A stale command is rejected instead of silently overwriting a newer project state.

The circuit-list editor uses the server state directly for operation eligibility. React callbacks only initiate forward commands and provide best-effort selection hints after the tree reload.

Snapshot Storage

Project history must use logical project data, not copies of the SQLite database file. This keeps snapshots:

• project-scoped
• portable across SQLite and PostgreSQL
• independent from unrelated projects
• suitable for validation, comparison and selective restore tooling

Large snapshots may later be compressed or stored in object storage, with metadata and checksums retained in the database.

Implemented named-snapshot foundation:

• project_snapshots stores project ownership, source revision, schema version, name, optional description, logical payload, SHA-256 and creation metadata
• snapshot creation requires the expected current revision and reads plus stores the complete project state in one SQLite transaction
• the versioned payload contains project settings, distribution boards, circuit lists, sections, circuits with device rows, project devices, floors and rooms while excluding global and upgrade-only data
• snapshot creation/listing does not create a project revision or affect undo/redo eligibility
• public Create/List endpoints expose metadata but not the logical payload
• restore verifies the stored checksum and a hash of the complete current state before replacing supported project data in one transaction
• restore records a new restore revision containing both the selected target state and the complete pre-restore inverse; undo/redo therefore remains available after application restarts
• compatible upgrade-only Consumer links, mappings and reports are preserved separately and are never imported into the logical snapshot payload
• arbitrary restore payloads are rejected by the generic user-command endpoint; only server-stored snapshots can initiate a forward restore
• the project page provides an initially collapsed snapshot/timeline panel, explicit inline restore confirmation and cursor-based loading of older revisions; a successful restore reloads all project read models

Implemented automatic-snapshot policy:

• project_snapshots.kind distinguishes named from automatic
• the central revision boundary captures the complete post-command state after every 25 additional project revisions
• the newest 12 automatic snapshots per project are retained
• retention never removes named snapshots, immutable revisions or physical database backups
• automatic capture and retention run inside the same transaction as the triggering project revision

Revit / CSV / IFCGUID Round-Trip Direction

External model exchange should use explicit staging and link entities instead of writing imported rows directly into circuits.

Conceptual entities:

• ExternalModelSource
  • identifies a Revit/IFC model or another external source within a project
• ExternalImportBatch
  • file name, checksum, import time, format/version and processing status
• ExternalModelObject
  • stable IFCGUID and optional Revit element identifiers
  • source attributes and last-seen import batch
• ExternalObjectLink
  • explicit relationship to a project device, circuit device row or another future domain entity
• ExternalParameterMapping
  • maps imported/exported columns to application fields
• ExternalExportBatch
  • records which project revision and import source produced an exported file

Required behavior:

• IFCGUID uniqueness is scoped to project and external model source.
• Parsing writes to staging first.
• Users preview additions, changes, missing objects and conflicts before applying them.
• Applying an import creates one project revision and one auditable change set.
• Imported source values and locally assigned planning values remain distinguishable.
• Export retains IFCGUID and source identifiers so Revit parameters can be matched deterministically.
• Missing objects and duplicate/conflicting identifiers are reported, never silently reassigned.

SQLite and PostgreSQL Direction

SQLite remains appropriate for the current local, single-process development workflow and can handle initial external-object volumes. Data volume alone is not a reason to migrate immediately.

PostgreSQL becomes appropriate when one or more of these conditions apply:

• multiple users edit projects concurrently
• the API runs as a shared network service
• background import/export jobs need independent workers
• revision/history queries or external-object datasets need server-side concurrency and operational tooling
• centralized backup, access control and monitoring become required

To preserve that option:

• domain services must not depend directly on better-sqlite3
• new multi-write workflows use an injectable transaction/unit-of-work boundary
• database-generated behavior is kept behind repositories
• IDs remain stable UUIDs and revisions use explicit project-scoped sequence numbers
• history snapshots remain logical and database-independent
• SQLite-specific synchronous transaction callbacks are treated as an adapter detail
• real persistence tests are separated so PostgreSQL parity tests can be added later

Drizzle schemas and migrations are dialect-specific and will require a deliberate PostgreSQL schema/migration path. The goal is portable domain behavior, not pretending the current database schema files can be switched without work.

Near-Term Refactoring Constraints

Completed foundation:

• real SQLite commit/rollback tests cover critical multi-write commands
• transaction ownership for row moves, synchronization, renumbering and migration is isolated in injected persistence adapters
• database backup and independent restore verification are automated
• the Consumer application path is removed and stable domain IDs are preserved
• editor grid projection and safety rules are separated from React rendering
• projects carry a monotonic currentRevision counter starting at zero
• project revision metadata and versioned forward/inverse change-set payloads have separate persistence entities
• forward and inverse payloads store complete serialized command envelopes; command type and schema version are derived from those envelopes rather than duplicated caller metadata
• command serialization rejects non-finite numbers, undefined, non-plain objects and cyclic payloads before a revision transaction starts
• the SQLite revision repository advances the project counter and stores both history records atomically
• revision appends use an expected-revision compare-and-set and reject stale callers without writing partial history
• a first internal circuit.update command validates project/list/section ownership and equipment-identifier uniqueness inside its transaction
• circuit updates derive their inverse from the persisted pre-command row and commit the domain update, inverse command, change set and revision together
• an internal circuit-device-row.update command applies the same boundary to device fields, validates project-device and room ownership and records automatically derived local override metadata in its normalized forward command
• integration tests apply the generated inverse as a new undo revision and verify rollback for stale revisions and late history failures on both update command types
• project-scoped relational undo/redo stacks persist eligible original change sets independently from immutable audit revisions
• stack transitions enforce LIFO order, keep undo/redo revisions out of the eligibility stacks and clear the redo branch on a new user command
• domain writes, audit revisions and stack transitions share one transaction; a forced stack failure rolls all of them back
• GET /api/projects/:projectId/history exposes current revision, stack depths and top change-set ids
• GET /api/projects/:projectId/history/revisions exposes a descending, cursor-paginated metadata timeline without leaking persisted forward or inverse command payloads
• a central application dispatcher executes the supported circuit.update and circuit-device-row.update envelopes without coupling the domain service to SQLite
• public command, undo and redo endpoints require an expected revision; undo and redo reconstruct the eligible persisted inverse or forward command
• the circuit tree returns the project's current revision; existing Circuit and CircuitDeviceRow cell edits use the public command endpoint, while their toolbar undo/redo actions use project-wide persistent history
• obsolete direct Circuit and CircuitDeviceRow field-update PATCH routes are removed so these editor writes cannot bypass revision history
• circuit-device-row.insert and circuit-device-row.delete preserve stable row ids, complete deleted-row snapshots and circuit reserve state through atomic insert/delete, revision and stack transitions
• circuit.insert and circuit.delete persist a complete circuit block with zero, one or multiple device rows; undo restores all ids and fields without renumbering
• standalone Circuit/CircuitDeviceRow insert/delete UI paths use these commands with client-generated stable UUIDs; obsolete direct structure POST and CircuitDeviceRow DELETE routes are removed
• circuit-device-row.move stores exact expected and target circuit/sort positions for one or multiple rows, recalculates all affected reserve states and reconstructs a deterministic inverse across multiple source circuits
• circuit-device-row.move-with-new-circuit additionally records the complete empty target-circuit snapshot; forward creation and all row moves share one transaction, while undo restores the source positions and deletes only an unchanged generated target
• the circuit-list editor executes existing-target and generated-target row moves through these commands; obsolete direct move and Circuit DELETE routes plus their separate transaction adapter are removed
• circuit.reorder-section requires the complete section circuit set and stores exact expected/target sort positions; its inverse changes no equipment identifier and never splits a circuit block
• circuit.reorder-sections applies the same invariants to every affected section from a sorted view in one transaction and one history entry
• circuit drag-and-drop and sorted-order application use these commands; the obsolete direct reorder route and transaction method are removed
• circuit.renumber-section is the separate explicit renumber operation; it stores every expected/target equipment identifier, uses collision-safe temporary values and restores the exact prior identifiers on undo
• the editor's explicit renumber action uses this command; obsolete direct renumber and identifier-restore routes plus their transaction adapter are removed
• the circuit-list editor loads project-wide undo/redo eligibility together with the tree, retries revision mismatches and keeps Undo/Redo available after page reloads without a session-local command stack
• project-device.sync-rows records synchronization, disconnect and reconnect as one atomic multi-row operation; complete expected/target sync snapshots protect local values, links and override metadata against silent stale writes
• project-device.update persists canonical source-device field changes independently and never synchronizes linked rows implicitly
• project-device.insert and project-device.delete preserve stable device ids; delete undo restores links only from complete unchanged disconnected-row snapshots in the same transaction
• project.update-settings persists both project voltage defaults, its exact inverse, revision and history transition atomically; the project page and existing PUT route require the current expected revision

Remaining constraints before completing project version history:

• extend project-scoped commands only when further project mutations are deliberately added to history; the generic versioned envelope is complete
• do not model IFCGUID as an overloaded circuit equipment identifier
• keep database backup/restore checks separate from project history tests

Deferred Decisions

• whether large snapshot payloads remain in PostgreSQL or move to object storage
• branch visualization after undo followed by new edits
• user/role model and actor attribution
• supported Revit CSV dialects and parameter mapping UI
• PostgreSQL deployment topology and migration date