From 01fa527b9c8d93d6765b8fb701ab298cd9c7223a Mon Sep 17 00:00:00 2001 From: Grovy311 Date: Sat, 15 Aug 2026 00:26:56 +0200 Subject: [PATCH 1/5] Add production compose stack and stop idle load in containers The development stack was running permanently on a server: polling file watchers, a healthcheck that rendered a full page every five seconds and no memory limit grew next dev to 10 GB and pushed the host into swap. - add compose.prod.yaml running compiled output in separate api/web services - make the Dockerfile multi-stage with dev and prod targets, prune devDependencies and run the runtime image as node instead of root - bake API_INTERNAL_URL at build time; next start ignores it at runtime because rewrite destinations are resolved into routes-manifest.json - drop CHOKIDAR_USEPOLLING and WATCHPACK_POLLING - probe /health instead of /, which redirects to /projects and made every healthcheck render the project list - give every service a memory limit and forbid swap in production - rename the development compose project to leistungsbilanz-dev so its down command cannot target the production stack - bind development ports to localhost - close the http server and the SQLite handle on SIGTERM/SIGINT - match probe user agents in the navigation log filter; Node's fetch sends one, so the previous check never matched - exit docker-start.sh when either supervised process dies - remove drizzle.config.js, a compiled copy drizzle-kit never reads, and the pre-Next index.html/styles.css leftovers Co-Authored-By: Claude Opus 5 --- .gitignore | 2 + AGENTS.md | 33 +++++++++- Dockerfile | 51 ++++++++++++--- README.md | 23 +++++-- compose.prod.yaml | 102 ++++++++++++++++++++++++++++++ compose.yaml | 70 ++++++++++++--------- docs/deployment.md | 135 +++++++++++++++++++++++++++++++++------- drizzle.config.js | 11 ---- index.html | 17 ----- scripts/docker-start.sh | 30 ++++++++- src/db/client.ts | 8 +++ src/proxy.ts | 11 ++-- src/server/index.ts | 34 +++++++++- styles.css | 15 ----- 14 files changed, 424 insertions(+), 118 deletions(-) mode change 100644 => 100755 .gitignore mode change 100644 => 100755 AGENTS.md mode change 100644 => 100755 Dockerfile mode change 100644 => 100755 README.md create mode 100755 compose.prod.yaml mode change 100644 => 100755 compose.yaml mode change 100644 => 100755 docs/deployment.md delete mode 100644 drizzle.config.js delete mode 100644 index.html mode change 100644 => 100755 scripts/docker-start.sh mode change 100644 => 100755 src/db/client.ts mode change 100644 => 100755 src/proxy.ts mode change 100644 => 100755 src/server/index.ts delete mode 100644 styles.css diff --git a/.gitignore b/.gitignore old mode 100644 new mode 100755 index 8d44fb4..19da578 --- a/.gitignore +++ b/.gitignore @@ -1,6 +1,8 @@ node_modules/ dist/ .next/ +# Stray tsc output; drizzle-kit reads drizzle.config.ts. +/drizzle.config.js data/*.db data/backups/*.db .codex/*.log diff --git a/AGENTS.md b/AGENTS.md old mode 100644 new mode 100755 index f1a509d..d781e10 --- a/AGENTS.md +++ b/AGENTS.md @@ -531,13 +531,44 @@ Users must be able to override sizing suggestions. - Preserve stable UUIDs and explicit transaction boundaries for a later PostgreSQL adapter. +## Deployment Rules + +There are two compose files and they must stay separate. + +- `compose.yaml` is development only. Project name `leistungsbilanz-dev`, ports + bound to `127.0.0.1`, watch-mode servers, source bind mounts. +- `compose.prod.yaml` is the deployment. Project name `leistungsbilanz`, image + target `prod`, compiled output, no bind mounts, no file watchers. + +Rules that a running deployment depends on: + +- Never run the development stack permanently. `next dev` and `tsx watch` + produce load and grow in memory with no user activity. +- Never reintroduce `CHOKIDAR_USEPOLLING` or `WATCHPACK_POLLING`. They make the + watcher poll the source tree continuously. +- Container healthchecks target `/health` only. `/` is a redirect to + `/projects`, and `fetch` follows redirects, so probing `/` renders a full + page on every interval. +- Every service keeps a `mem_limit`; production services also keep a + `memswap_limit` of the same size so a leak kills the container instead of the + host. +- The production image runs as `node`. Data volumes written by an earlier root + container need a one-time `chown` (see `docs/deployment.md`). +- The Express API is not published on the host. It has no authentication and is + reached through the Next.js rewrite. +- Development migrates with `drizzle-kit`; the production image has no + devDependencies and migrates with `node scripts/run-migrations.js`. +- `SIGTERM`/`SIGINT` must close the HTTP server and the SQLite handle and then + exit. Do not reduce these handlers to logging. + ## Current Deferred Work - Revit/CSV/IFCGUID round-trip, except when Phase 14 is explicitly requested and `docs/spec/revit-csv-integration-requirements.md` is being followed - full electrical sizing - multi-user/PostgreSQL operation -- supported production deployment +- authentication, authorization and TLS termination; until they exist the + deployment belongs in a trusted network or behind an authenticating proxy Do not implement these while working on an unrelated phase. diff --git a/Dockerfile b/Dockerfile old mode 100644 new mode 100755 index 41a08fd..4b4ed55 --- a/Dockerfile +++ b/Dockerfile @@ -1,14 +1,51 @@ -FROM node:22 -WORKDIR /app +# syntax=docker/dockerfile:1 +# --------------------------------------------------------------------------- +# deps: complete dependency tree, shared by the build and the dev target. +# Stays on the full node image because better-sqlite3 needs a toolchain. +# --------------------------------------------------------------------------- +FROM node:22 AS deps +WORKDIR /app COPY package*.json ./ RUN npm ci +# --------------------------------------------------------------------------- +# dev: watch-mode target used by compose.yaml. Keeps devDependencies so +# tsx, drizzle-kit and next dev are available. Sources come from bind mounts. +# --------------------------------------------------------------------------- +FROM deps AS dev +ENV NODE_ENV=development +COPY . . +RUN mkdir -p data +EXPOSE 3000 3001 +CMD ["npm", "run", "dev:api"] + +# --------------------------------------------------------------------------- +# build: compile the API to dist/ and the frontend to .next/, then drop +# devDependencies so the runtime stage carries only what it executes. +# --------------------------------------------------------------------------- +FROM deps AS build +ENV NEXT_TELEMETRY_DISABLED=1 +# Next.js resolves next.config.mjs rewrites at build time and writes the literal +# destination into .next/routes-manifest.json. Setting API_INTERNAL_URL only at +# runtime has no effect on `next start`, so the API address of the target +# topology has to be known here. Server components read the variable at runtime. +ARG API_INTERNAL_URL=http://localhost:3000 +ENV API_INTERNAL_URL=$API_INTERNAL_URL COPY . . RUN npm run build:api && npm run build:web +RUN npm prune --omit=dev -RUN mkdir -p data && chmod +x scripts/docker-start.sh - -EXPOSE 3001 - -CMD ["sh", "scripts/docker-start.sh"] +# --------------------------------------------------------------------------- +# prod: runtime image. node:22-slim shares the Debian release of the build +# stage, so the compiled better-sqlite3 binding stays loadable. +# --------------------------------------------------------------------------- +FROM node:22-slim AS prod +ENV NODE_ENV=production +ENV NEXT_TELEMETRY_DISABLED=1 +WORKDIR /app +COPY --from=build --chown=node:node /app ./ +RUN mkdir -p data && chown node:node data +USER node +EXPOSE 3000 3001 +CMD ["node", "dist/server/index.js"] diff --git a/README.md b/README.md old mode 100644 new mode 100755 index c333917..db895e6 --- a/README.md +++ b/README.md @@ -4,8 +4,9 @@ Leistungsbilanz ist eine Webanwendung für die elektrische Ausführungsplanung. Mittelpunkt steht ein tabellenähnlicher Stromkreislisten-Editor, der Stromkreise, Gerätezeilen und wiederverwendbare Projektgeräte fachlich getrennt behandelt. -Das Projekt befindet sich in aktiver Entwicklung. Der lokale Entwicklungsbetrieb -mit SQLite und Docker Compose ist unterstützt. Ein Produktionsdeployment, +Das Projekt befindet sich in aktiver Entwicklung. Unterstützt sind der lokale +Entwicklungsbetrieb (`compose.yaml`) und ein Einzelbenutzer-Deployment im +vertrauenswürdigen Netz (`compose.prod.yaml`). Authentifizierung, Mehrbenutzerbetrieb und der Revit-/IFCGUID-Datenaustausch sind noch nicht implementiert. @@ -62,9 +63,21 @@ docker compose logs --follow docker compose down ``` -Der Compose-Stack startet Entwicklungsserver mit Quellcode-Mounts. Er ist kein -Produktionsdeployment. Details stehen in -[Deployment und Betrieb](docs/deployment.md). +`compose.yaml` startet Entwicklungsserver mit Quellcode-Mounts, veröffentlicht +seine Ports nur auf `127.0.0.1` und ist **kein Produktionsdeployment**. Es +gehört nicht in den Dauerbetrieb: `next dev` und `tsx watch` erzeugen auch ohne +Benutzer dauerhaft Last und wachsen im Speicher. + +## Deployment + +```bash +docker compose -f compose.prod.yaml up --build --detach +``` + +Startet die kompilierte API und `next start` in getrennten Containern, mit +Speicherlimits und ohne Dateibeobachter. Das Frontend hört auf Port 3090, die +API ist nur intern erreichbar. Vorbereitung eines bestehenden Datenvolumes und +weitere Details stehen in [Deployment und Betrieb](docs/deployment.md). ## Direkte lokale Entwicklung diff --git a/compose.prod.yaml b/compose.prod.yaml new file mode 100755 index 0000000..a7a56f3 --- /dev/null +++ b/compose.prod.yaml @@ -0,0 +1,102 @@ +# Production deployment. +# +# Differences to compose.yaml that matter operationally: +# - runs compiled output (node dist/, next start) instead of watch mode +# - no source bind mounts and no file watchers, so an idle stack costs ~0% CPU +# - healthchecks hit the cheap /health JSON endpoint, not a rendered page +# - every service has a hard memory limit and may not spill into host swap +# - the Express API is reachable only inside the compose network +# +# Usage: +# docker compose -f compose.prod.yaml up --build --detach +name: leistungsbilanz + +x-service-defaults: &service-defaults + init: true + restart: unless-stopped + stop_grace_period: 20s + logging: + driver: json-file + options: + max-size: "20m" + max-file: "10" + +services: + api: + <<: *service-defaults + build: + context: . + target: prod + args: + # Baked into the Next.js rewrite manifest; see the build stage in the + # Dockerfile. Must match the API service address in this network. + API_INTERNAL_URL: http://api:3000 + command: + - sh + - -c + - node scripts/run-migrations.js && exec node dist/server/index.js + environment: + NODE_ENV: production + PORT: "3000" + LOG_LEVEL: "${LOG_LEVEL:-info}" + # Not published on the host: the API has no authentication and is reached + # through the Next.js rewrite in the web service. + expose: + - "3000" + mem_limit: 512m + memswap_limit: 512m + volumes: + - leistungsbilanz-data:/app/data + healthcheck: + test: + - CMD + - node + - -e + - fetch('http://127.0.0.1:3000/health').then(response=>process.exit(response.ok?0:1)).catch(()=>process.exit(1)) + interval: 30s + timeout: 5s + retries: 3 + start_period: 30s + + web: + <<: *service-defaults + build: + context: . + target: prod + args: + # Baked into the Next.js rewrite manifest; see the build stage in the + # Dockerfile. Must match the API service address in this network. + API_INTERNAL_URL: http://api:3000 + command: + - node_modules/.bin/next + - start + - -p + - "3001" + - --hostname + - 0.0.0.0 + environment: + NODE_ENV: production + API_INTERNAL_URL: http://api:3000 + LOG_LEVEL: "${LOG_LEVEL:-info}" + depends_on: + api: + condition: service_healthy + ports: + - "3090:3001" + mem_limit: 1g + memswap_limit: 1g + healthcheck: + # /health is rewritten to the API, so this verifies the web process and + # its API connectivity for the cost of one JSON response. + test: + - CMD + - node + - -e + - fetch('http://127.0.0.1:3001/health').then(response=>process.exit(response.ok?0:1)).catch(()=>process.exit(1)) + interval: 30s + timeout: 5s + retries: 3 + start_period: 30s + +volumes: + leistungsbilanz-data: diff --git a/compose.yaml b/compose.yaml old mode 100644 new mode 100755 index 6a5ec7c..5d25db7 --- a/compose.yaml +++ b/compose.yaml @@ -1,26 +1,39 @@ -name: leistungsbilanz +# Local development only. Runs watch-mode servers against bind-mounted sources. +# For a deployment use compose.prod.yaml. +# +# The project name must stay distinct from the production project so that +# `docker compose down` in this directory can never tear down a running +# production stack. +name: leistungsbilanz-dev + +x-service-defaults: &service-defaults + init: true + restart: unless-stopped + stop_grace_period: 20s + logging: + driver: json-file + options: + max-size: "20m" + max-file: "10" services: api: + <<: *service-defaults build: context: . + target: dev command: - sh - -c - npm run db:migrate && npm run db:verify:circuit-schema && npm run dev:api environment: PORT: "3000" - CHOKIDAR_USEPOLLING: "true" LOG_LEVEL: "${LOG_LEVEL:-info}" - init: true - restart: unless-stopped - logging: - driver: json-file - options: - max-size: "20m" - max-file: "10" + # Development ports stay on the loopback interface; nothing here is + # authenticated and the API must not be reachable from the LAN. ports: - - "3000:3000" + - "127.0.0.1:3000:3000" + mem_limit: 1g volumes: - ./src:/app/src - ./scripts:/app/scripts @@ -32,15 +45,17 @@ services: - CMD - node - -e - - fetch('http://localhost:3000/health').then(response=>{if(!response.ok)process.exit(1)}).catch(()=>process.exit(1)) - interval: 5s - timeout: 3s - retries: 12 - start_period: 20s + - fetch('http://127.0.0.1:3000/health').then(response=>process.exit(response.ok?0:1)).catch(()=>process.exit(1)) + interval: 30s + timeout: 5s + retries: 5 + start_period: 30s web: + <<: *service-defaults build: context: . + target: dev command: - npm - run @@ -50,33 +65,30 @@ services: - 0.0.0.0 environment: API_INTERNAL_URL: http://api:3000 - WATCHPACK_POLLING: "true" NEXT_TELEMETRY_DISABLED: "1" LOG_LEVEL: "${LOG_LEVEL:-info}" - init: true - restart: unless-stopped - logging: - driver: json-file - options: - max-size: "20m" - max-file: "10" depends_on: api: condition: service_healthy ports: - - "3001:3001" + - "127.0.0.1:3001:3001" + # next dev grows steadily under long-running use; the limit turns that into + # a container restart instead of host-wide swapping. + mem_limit: 2g volumes: - ./src:/app/src - ./next.config.mjs:/app/next.config.mjs:ro - ./tsconfig.json:/app/tsconfig.json:ro - ./tsconfig.next.json:/app/tsconfig.next.json:ro healthcheck: + # Must stay on /health: "/" redirects to /projects and Node's fetch + # follows redirects, which turned every probe into a full page render. test: - CMD - node - -e - - fetch('http://localhost:3001/').then(response=>{if(!response.ok)process.exit(1)}).catch(()=>process.exit(1)) - interval: 5s - timeout: 3s - retries: 12 - start_period: 20s + - fetch('http://127.0.0.1:3001/health').then(response=>process.exit(response.ok?0:1)).catch(()=>process.exit(1)) + interval: 30s + timeout: 5s + retries: 5 + start_period: 30s diff --git a/docs/deployment.md b/docs/deployment.md old mode 100644 new mode 100755 index dfbb3f1..355cf40 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -2,21 +2,40 @@ ## Aktueller Status -Es gibt derzeit kein unterstütztes Produktionsdeployment. +Es gibt zwei getrennte Compose-Dateien. Sie dürfen nicht verwechselt werden. -`compose.yaml` ist ausschließlich für lokale Entwicklung vorgesehen. Es startet -`tsx watch` und `next dev`, bindet Quellcode vom Host ein und enthält weder TLS, -Authentifizierung, Reverse Proxy, Prozesshärtung noch ein zentral betriebenes -Datenbanksystem. Der Stack darf deshalb nicht als produktionsreif bezeichnet oder -öffentlich erreichbar gemacht werden. +| Datei | Projektname | Zweck | +| --- | --- | --- | +| `compose.yaml` | `leistungsbilanz-dev` | ausschließlich lokale Entwicklung | +| `compose.prod.yaml` | `leistungsbilanz` | Dauerbetrieb im LAN | -## Entwicklungs-Topologie +`compose.yaml` startet `tsx watch` und `next dev` und bindet Quellcode vom Host +ein. Dieser Stack ist **nicht** für Dauerbetrieb geeignet (siehe +[Warum kein Dev-Stack im Dauerbetrieb](#warum-kein-dev-stack-im-dauerbetrieb)). +Seine Ports sind deshalb an `127.0.0.1` gebunden. -| Komponente | Port | Healthcheck | Persistenz | -| --- | ---: | --- | --- | -| Next.js Web | 3001 | `GET /` | keine | -| Express API | 3000 | `GET /health` | `./data:/app/data` | -| SQLite | Datei | Integritäts-/FK-Prüfung via Backup und Skript | `data/leistungsbilanz.db` | +`compose.prod.yaml` startet kompilierten Code ohne Dateibeobachter. Es enthält +weiterhin weder TLS, Authentifizierung noch ein Benutzer-/Rollenmodell; die +Express-API wird deshalb nicht auf dem Host veröffentlicht, sondern nur über den +Next.js-Rewrite erreicht. Der Stack gehört hinter einen Reverse Proxy mit +Authentifizierung und darf nicht öffentlich erreichbar gemacht werden. + +## Topologie + +Entwicklung (`compose.yaml`): + +| Komponente | Port | Healthcheck | Speicherlimit | Persistenz | +| --- | ---: | --- | ---: | --- | +| Next.js Web | `127.0.0.1:3001` | `GET /health` alle 30 s | 2 GB | keine | +| Express API | `127.0.0.1:3000` | `GET /health` alle 30 s | 1 GB | `./data:/app/data` | + +Produktion (`compose.prod.yaml`): + +| Komponente | Port | Healthcheck | Speicherlimit | Persistenz | +| --- | ---: | --- | ---: | --- | +| Next.js Web | `3090` | `GET /health` alle 30 s | 1 GB, kein Swap | keine | +| Express API | nur intern | `GET /health` alle 30 s | 512 MB, kein Swap | Volume `leistungsbilanz-data` | +| SQLite | Datei | Integritäts-/FK-Prüfung via Backup und Skript | – | `data/leistungsbilanz.db` | Verwendete Umgebungsvariablen: @@ -24,15 +43,66 @@ Verwendete Umgebungsvariablen: - `API_INTERNAL_URL` – internes API-Ziel des Next.js-Rewrites, im Compose-Netz `http://api:3000` - `NEXT_TELEMETRY_DISABLED=1` -- `CHOKIDAR_USEPOLLING=true` und `WATCHPACK_POLLING=true` für lokale - Dateibeobachtung in Docker +- `NODE_ENV=production` – nur in `compose.prod.yaml` - `LOG_LEVEL` – steuert für beide Dienste die Ausgabestufe des strukturierten JSON-Loggers (`error`, `warn`, `info`, `verbose`, `debug`), Standard `info`. Setzbar über eine `.env`-Datei neben `compose.yaml` oder `LOG_LEVEL=verbose docker compose up`. -Beim API-Start laufen zuerst `npm run db:migrate` und -`npm run db:verify:circuit-schema`. +Beim API-Start laufen in der Entwicklung zuerst `npm run db:migrate` und +`npm run db:verify:circuit-schema` (drizzle-kit, eine devDependency). Das +Produktionsimage enthält keine devDependencies und migriert stattdessen über +`node scripts/run-migrations.js`, das denselben Migrationsordner mit +`drizzle-orm` anwendet. + +## Warum kein Dev-Stack im Dauerbetrieb + +`compose.yaml` lief einmal fünf Tage durchgehend auf einem Server. Ergebnis: +10,3 GB belegter Arbeitsspeicher im Web-Container, 12,2 % Dauer-CPU im +API-Container und ein Host, der 13 GB Swap belegt hatte. Vier Ursachen wirkten +zusammen; alle vier sind inzwischen behoben: + +1. **Polling-Dateibeobachter.** `CHOKIDAR_USEPOLLING` und `WATCHPACK_POLLING` + ließen `tsx watch` permanent den Quellbaum abklappern. Beide Variablen sind + entfernt; unter Linux funktioniert `inotify` auf Bind-Mounts. +2. **Healthcheck als Seitenrendering.** Der Web-Healthcheck rief `/` alle fünf + Sekunden auf. `/` ist ein `redirect("/projects")`, und Node's `fetch` folgt + Redirects – jede Prüfung rendert also die vollständige Projektliste, rund + 16.000-mal pro Tag. Der Healthcheck zeigt jetzt auf `/health` und läuft alle + 30 Sekunden. +3. **Kein Speicherlimit.** `next dev` hält Kompilierungs-State und wächst unter + Dauerlast unbegrenzt. Beide Compose-Dateien setzen jetzt `mem_limit`; in der + Produktion zusätzlich `memswap_limit` in gleicher Höhe, damit ein Leck den + Container beendet statt den Host in den Swap zu ziehen. +4. **Kollidierender Projektname.** `compose.yaml` hieß `leistungsbilanz` und + damit genauso wie das Produktionsprojekt; ein `docker compose down` im + Entwicklungsverzeichnis zielte auf den Produktionsstack. Der Entwicklungsname + ist jetzt `leistungsbilanz-dev`. + +## Produktionsdeployment + +```bash +docker compose -f compose.prod.yaml up --build --detach +docker compose -f compose.prod.yaml ps +docker compose -f compose.prod.yaml logs --follow +``` + +Das Frontend hört danach auf Port 3090. Die API ist nur innerhalb des +Compose-Netzes erreichbar. + +Das Produktionsimage läuft als Benutzer `node` statt als `root`. Wenn ein +bestehendes Datenvolume von einem früheren root-Container beschrieben wurde, +müssen dessen Dateien einmalig übereignet werden, sonst schlagen Schreibzugriffe +mit `SQLITE_READONLY` fehl: + +```bash +docker run --rm -v leistungsbilanz_leistungsbilanz-data:/data alpine \ + chown -R 1000:1000 /data +``` + +Ein bestehendes Ein-Container-Deployment über `scripts/docker-start.sh` wird +abgelöst, indem dessen Stack gestoppt und `compose.prod.yaml` mit demselben +Projektnamen `leistungsbilanz` gestartet wird; das Volume bleibt dabei erhalten. ## Logging @@ -68,23 +138,40 @@ einem unbekannten Zustand weiterzulaufen. Beide Dienste laufen deshalb mit `restart: unless-stopped`, damit Docker sie danach automatisch neu startet; ohne diese Policy würde ein Crash den Dienst dauerhaft unerreichbar lassen. -## Voraussetzungen für ein späteres Produktionssetup +Anfragen von Healthcheck- und Monitoring-Clients (`node`, `curl`, `wget`, …) +werden anhand des User-Agents aus dem Navigationslog gefiltert und `/health` +läuft gar nicht erst durch `src/proxy.ts`. -Vor einer produktiven Installation werden mindestens benötigt: +## Herunterfahren + +`SIGTERM` und `SIGINT` schließen den HTTP-Server, danach das SQLite-Handle, und +beenden den Prozess mit Code 0. Kommt der Server binnen 15 Sekunden nicht +herunter, beendet sich der Prozess trotzdem. Vorher wurde das Signal nur +geloggt: jeder `docker stop` lief in die Grace Period und endete mit `SIGKILL`, +möglicherweise mitten in einem Schreibvorgang. `stop_grace_period` steht in +beiden Compose-Dateien auf 20 s und liegt damit über dem internen Timeout. + +## Offene Punkte für einen vollwertigen Produktionsbetrieb + +Umgesetzt: + +- reproduzierbares Produktionsimage und Next.js-Produktionsstart +- kontrollierter Migrationsschritt vor dem API-Start +- strukturierte Logs mit Rotation, Healthchecks, Speicherlimits +- definiertes Herunterfahren + +Weiterhin offen: -- reproduzierbare Produktionsimages und ein Next.js-Produktionsstartskript - TLS-Termination und Reverse Proxy - Authentifizierung, Autorisierung und Benutzer-/Rollenmodell - definierte Secrets- und Konfigurationsverwaltung -- persistenter, gesicherter Datenbankbetrieb -- kontrollierter einmaliger Migrationsschritt vor dem API-Rollout -- Monitoring, strukturierte Logs und Alarmierung +- Alarmierung auf Basis der Healthchecks - getestete Backup-, Restore- und Rollback-Prozeduren - Entscheidung, ob SQLite für einen einzelnen Prozess genügt oder PostgreSQL für Mehrbenutzerbetrieb erforderlich ist -Bis diese Punkte umgesetzt und getestet sind, besteht die „Installation“ aus dem -lokalen Entwicklungsstart in der README. +Solange Authentifizierung fehlt, gehört der Stack ausschließlich in ein +vertrauenswürdiges Netz oder hinter einen authentifizierenden Reverse Proxy. ## Backup und Wiederherstellung diff --git a/drizzle.config.js b/drizzle.config.js deleted file mode 100644 index 004045a..0000000 --- a/drizzle.config.js +++ /dev/null @@ -1,11 +0,0 @@ -"use strict"; -Object.defineProperty(exports, "__esModule", { value: true }); -const drizzle_kit_1 = require("drizzle-kit"); -exports.default = (0, drizzle_kit_1.defineConfig)({ - dialect: "sqlite", - schema: "./src/db/schema/*.ts", - out: "./src/db/migrations", - dbCredentials: { - url: "./data/leistungsbilanz.db", - }, -}); diff --git a/index.html b/index.html deleted file mode 100644 index dcb632f..0000000 --- a/index.html +++ /dev/null @@ -1,17 +0,0 @@ - - - - - - Leistungsbilanz API - - - -
-

Leistungsbilanz Backend

-

Der aktuelle Stand laeuft als TypeScript Node/SQLite API.

-

Serverstart: npm run dev

-

Healthcheck: GET /health

-
- - diff --git a/scripts/docker-start.sh b/scripts/docker-start.sh old mode 100644 new mode 100755 index bf0d856..85a27a6 --- a/scripts/docker-start.sh +++ b/scripts/docker-start.sh @@ -1,4 +1,7 @@ #!/bin/sh +# Legacy single-container entrypoint. Kept for the existing deployment that +# still runs both processes in one container; new deployments use +# compose.prod.yaml, which runs the API and the web server separately. set -e echo "Running migrations..." @@ -6,11 +9,34 @@ node scripts/run-migrations.js echo "Starting API server on :3000..." node dist/server/index.js & +api_pid=$! echo "Waiting for API..." -until node -e "require('http').get('http://localhost:3000/health', r => process.exit(r.statusCode===200?0:1)).on('error',()=>process.exit(1))" 2>/dev/null; do +until node -e "fetch('http://127.0.0.1:3000/health').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))" 2>/dev/null; do + if ! kill -0 "$api_pid" 2>/dev/null; then + echo "API process exited during startup." + exit 1 + fi sleep 1 done echo "Starting Next.js on :3001..." -exec node_modules/.bin/next start -p 3001 +node_modules/.bin/next start -p 3001 & +web_pid=$! + +terminate() { + kill -TERM "$api_pid" "$web_pid" 2>/dev/null || true +} +trap terminate TERM INT + +# Previously the API ran unsupervised in the background: when it died the +# container stayed "up" and served a frontend whose every request failed. +# Exiting here lets the restart policy replace the container instead. +while kill -0 "$api_pid" 2>/dev/null && kill -0 "$web_pid" 2>/dev/null; do + sleep 5 +done + +echo "A server process exited; stopping the container." +terminate +wait "$api_pid" "$web_pid" 2>/dev/null || true +exit 1 diff --git a/src/db/client.ts b/src/db/client.ts old mode 100644 new mode 100755 index 02cfc7e..39f4125 --- a/src/db/client.ts +++ b/src/db/client.ts @@ -7,3 +7,11 @@ const defaultDatabaseContext = createDatabaseContext( export const db = defaultDatabaseContext.db; +/** + * Closes the process-wide SQLite handle. Only the composition root calls this, + * during shutdown, so pending writes are flushed before the process exits. + */ +export function closeDefaultDatabase(): void { + defaultDatabaseContext.close(); +} + diff --git a/src/proxy.ts b/src/proxy.ts old mode 100644 new mode 100755 index 4607ec0..a2cb084 --- a/src/proxy.ts +++ b/src/proxy.ts @@ -4,10 +4,13 @@ import { createLogger } from "./shared/logging/logger"; const logger = createLogger("web:navigation"); +// Node's fetch() sends "User-Agent: node", so the previous "no User-Agent" +// check never matched and every healthcheck was logged as page navigation. +const PROBE_USER_AGENT = /^(node|curl|wget|go-http-client|kube-probe)\b/i; + export function proxy(request: NextRequest) { - // The Docker healthcheck hits "/" every few seconds with no User-Agent - // header; skip it so real navigation isn't drowned out in the logs. - if (request.headers.get("user-agent")) { + const userAgent = request.headers.get("user-agent") ?? ""; + if (userAgent && !PROBE_USER_AGENT.test(userAgent)) { logger.info("page request", { method: request.method, path: request.nextUrl.pathname, @@ -17,5 +20,5 @@ export function proxy(request: NextRequest) { } export const config = { - matcher: ["/((?!_next/static|_next/image|favicon.ico|api).*)"], + matcher: ["/((?!_next/static|_next/image|favicon.ico|api|health).*)"], }; diff --git a/src/server/index.ts b/src/server/index.ts old mode 100644 new mode 100755 index 2f6ac5e..3ee21f1 --- a/src/server/index.ts +++ b/src/server/index.ts @@ -4,6 +4,7 @@ import { projectDeviceRouter } from "./routes/project-device.routes.js"; import { projectRouter } from "./routes/project.routes.js"; import { errorMiddleware } from "./middleware/error.middleware.js"; import { createLogger, toErrorMeta } from "../shared/logging/logger.js"; +import { closeDefaultDatabase } from "../db/client.js"; const logger = createLogger("api"); const app = express(); @@ -62,8 +63,35 @@ process.on("unhandledRejection", (reason) => { process.exit(1); }); -process.on("SIGTERM", () => logger.info("received SIGTERM")); -process.on("SIGINT", () => logger.info("received SIGINT")); +let shuttingDown = false; + +function shutdown(signal: NodeJS.Signals): void { + if (shuttingDown) return; + shuttingDown = true; + logger.info("shutting down", { signal }); + + // Without this the process only logged the signal and kept running, so every + // stop waited for Docker's grace period and ended in SIGKILL mid-write. + const forceExit = setTimeout(() => { + logger.warn("shutdown timed out, exiting anyway"); + process.exit(1); + }, 15_000); + forceExit.unref(); + + server.close((error) => { + if (error) logger.error("http server close failed", toErrorMeta(error)); + try { + closeDefaultDatabase(); + } catch (closeError) { + logger.error("database close failed", toErrorMeta(closeError)); + } + logger.info("shutdown complete"); + process.exit(error ? 1 : 0); + }); +} + +process.on("SIGTERM", shutdown); +process.on("SIGINT", shutdown); setInterval(() => { const memory = process.memoryUsage(); @@ -74,6 +102,6 @@ setInterval(() => { }); }, heartbeatIntervalMs).unref(); -app.listen(port, () => { +const server = app.listen(port, () => { logger.info("server started", { port }); }); diff --git a/styles.css b/styles.css deleted file mode 100644 index 58c8f79..0000000 --- a/styles.css +++ /dev/null @@ -1,15 +0,0 @@ -body { - margin: 0; - font-family: Arial, sans-serif; - background: #f5f6f8; - color: #1f2937; -} - -.info { - max-width: 760px; - margin: 40px auto; - background: #ffffff; - border: 1px solid #d1d5db; - border-radius: 8px; - padding: 24px; -} From 9504905c8f1d7b34dce2e70fc19a90955f502e4f Mon Sep 17 00:00:00 2001 From: Julian Appel Date: Sun, 23 Aug 2026 18:05:50 +0200 Subject: [PATCH 2/5] Upgrade project to Node 24 Node 22 leaves active LTS at the end of October. Move the Docker base image and the documented local requirement to Node 24. Pull @types/node from ^25 down to ^24 so the type definitions no longer describe a newer runtime than the one in use. Types ahead of the runtime let TypeScript accept APIs that are not there at execution time; the opposite direction is harmless. Add an engines field to record the supported version. No .nvmrc, since the project is not used with a Node version manager. Co-Authored-By: Claude Opus 5 --- Dockerfile | 2 +- README.md | 2 +- package-lock.json | 23 ++++++++++++++--------- package.json | 5 ++++- 4 files changed, 20 insertions(+), 12 deletions(-) diff --git a/Dockerfile b/Dockerfile index 41a08fd..5484443 100644 --- a/Dockerfile +++ b/Dockerfile @@ -1,4 +1,4 @@ -FROM node:22 +FROM node:24 WORKDIR /app COPY package*.json ./ diff --git a/README.md b/README.md index c333917..d7a7b09 100644 --- a/README.md +++ b/README.md @@ -70,7 +70,7 @@ Produktionsdeployment. Details stehen in Voraussetzungen: -- Node.js 22 +- Node.js 24 - npm ```powershell diff --git a/package-lock.json b/package-lock.json index b0abb41..417421b 100644 --- a/package-lock.json +++ b/package-lock.json @@ -22,12 +22,15 @@ "devDependencies": { "@types/better-sqlite3": "^7.6.13", "@types/express": "^5.0.6", - "@types/node": "^25.6.0", + "@types/node": "^24.10.1", "@types/react": "^19.2.14", "@types/react-dom": "^19.2.3", "drizzle-kit": "^0.31.10", "tsx": "^4.21.0", "typescript": "^6.0.3" + }, + "engines": { + "node": "24.x" } }, "node_modules/@drizzle-team/brocli": { @@ -1514,12 +1517,13 @@ "dev": true }, "node_modules/@types/node": { - "version": "25.6.0", - "resolved": "https://registry.npmjs.org/@types/node/-/node-25.6.0.tgz", - "integrity": "sha512-+qIYRKdNYJwY3vRCZMdJbPLJAtGjQBudzZzdzwQYkEPQd+PJGixUL5QfvCLDaULoLv+RhT3LDkwEfKaAkgSmNQ==", + "version": "24.13.3", + "resolved": "https://registry.npmjs.org/@types/node/-/node-24.13.3.tgz", + "integrity": "sha512-Dh8vAsV36ig5wa9OX4pXvMc9D3Veibfw2wix0CUwYODLD8nkj9UsLjASr49nPg+2eKzxhBV+v7L8pXvT4e639Q==", "devOptional": true, + "license": "MIT", "dependencies": { - "undici-types": "~7.19.0" + "undici-types": "~7.18.0" } }, "node_modules/@types/qs": { @@ -3688,10 +3692,11 @@ } }, "node_modules/undici-types": { - "version": "7.19.2", - "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.19.2.tgz", - "integrity": "sha512-qYVnV5OEm2AW8cJMCpdV20CDyaN3g0AjDlOGf1OW4iaDEx8MwdtChUp4zu4H0VP3nDRF/8RKWH+IPp9uW0YGZg==", - "devOptional": true + "version": "7.18.2", + "resolved": "https://registry.npmjs.org/undici-types/-/undici-types-7.18.2.tgz", + "integrity": "sha512-AsuCzffGHJybSaRrmr5eHr81mwJU3kjw6M+uprWvCXiNeN9SOGwQ3Jn8jb8m3Z6izVgknn1R0FTCEAP2QrLY/w==", + "devOptional": true, + "license": "MIT" }, "node_modules/unpipe": { "version": "1.0.0", diff --git a/package.json b/package.json index 9bc5ae3..d5ebe82 100644 --- a/package.json +++ b/package.json @@ -3,6 +3,9 @@ "version": "1.0.0", "description": "Spreadsheet-style circuit list editor for electrical distribution planning", "main": "dist/server/index.js", + "engines": { + "node": "24.x" + }, "scripts": { "dev": "npm run dev:api", "dev:api": "tsx watch src/server/index.ts", @@ -40,7 +43,7 @@ "devDependencies": { "@types/better-sqlite3": "^7.6.13", "@types/express": "^5.0.6", - "@types/node": "^25.6.0", + "@types/node": "^24.10.1", "@types/react": "^19.2.14", "@types/react-dom": "^19.2.3", "drizzle-kit": "^0.31.10", From c6cdfc42d5c347a529dff46ab173baa3e07daa07 Mon Sep 17 00:00:00 2001 From: Julian Appel Date: Sun, 23 Aug 2026 18:05:50 +0200 Subject: [PATCH 3/5] Keep manual quantity consistent when synchronizing device rows Synchronizing a project device pushes its quantity onto every linked device row, but manualQuantity is not part of the sync snapshot and was left untouched. A device whose quantity is lower than the row's left the row at manualQuantity > quantity, violating the snapshot invariant. The violation surfaced far from its cause: the invariant is only checked when a full state snapshot is read, and the automatic snapshot runs every 25 revisions. A project could therefore accumulate the broken row silently and then reject every subsequent command, because the failing validation rolls back the whole transaction including the revision bump. Recompute manualQuantity as the synchronized quantity minus the total of the linked external objects, matching the invariant that assertCircuitDeviceRowQuantity enforces elsewhere, and reject a synchronized quantity that falls below that total. Co-Authored-By: Claude Opus 5 --- ...ice-row-sync-project-command.repository.ts | 26 ++- ...ow-sync-project-command.repository.test.ts | 183 ++++++++++++++++++ 2 files changed, 208 insertions(+), 1 deletion(-) diff --git a/src/db/repositories/project-device-row-sync-project-command.repository.ts b/src/db/repositories/project-device-row-sync-project-command.repository.ts index da17458..eb722d7 100644 --- a/src/db/repositories/project-device-row-sync-project-command.repository.ts +++ b/src/db/repositories/project-device-row-sync-project-command.repository.ts @@ -14,6 +14,7 @@ import type { AppDatabase } from "../database-context.js"; import { circuitDeviceRows } from "../schema/circuit-device-rows.js"; import { circuitLists } from "../schema/circuit-lists.js"; import { circuits } from "../schema/circuits.js"; +import { externalModelObjects } from "../schema/external-model-objects.js"; import { projectDevices } from "../schema/project-devices.js"; import { executeProjectCommandTransaction } from "./project-command-transaction.persistence.js"; import { updateDerivedCircuitVoltage } from "./project-voltage.persistence.js"; @@ -119,9 +120,32 @@ export class ProjectDeviceRowSyncProjectCommandRepository ); for (const assignment of input.command.payload.rows) { + const values: typeof assignment.target & { + manualQuantity?: number; + } = { ...assignment.target }; + if (assignment.target.quantity !== assignment.expected.quantity) { + const externalTotal = tx + .select({ planningValues: externalModelObjects.planningValues }) + .from(externalModelObjects) + .where( + eq(externalModelObjects.circuitDeviceRowId, assignment.rowId) + ) + .all() + .reduce( + (sum, object) => sum + object.planningValues.effectiveQuantity, + 0 + ); + const manualQuantity = assignment.target.quantity - externalTotal; + if (manualQuantity < 0) { + throw new Error( + "Synchronized quantity is below the total quantity of linked external objects." + ); + } + values.manualQuantity = manualQuantity; + } const updated = tx .update(circuitDeviceRows) - .set(assignment.target) + .set(values) .where(eq(circuitDeviceRows.id, assignment.rowId)) .run(); if (updated.changes !== 1) { diff --git a/tests/project-device-row-sync-project-command.repository.test.ts b/tests/project-device-row-sync-project-command.repository.test.ts index 75d56a8..593a7f6 100644 --- a/tests/project-device-row-sync-project-command.repository.test.ts +++ b/tests/project-device-row-sync-project-command.repository.test.ts @@ -13,9 +13,13 @@ import { ProjectHistoryRepository } from "../src/db/repositories/project-history import { circuitDeviceRows } from "../src/db/schema/circuit-device-rows.js"; import { circuitSections } from "../src/db/schema/circuit-sections.js"; import { circuits } from "../src/db/schema/circuits.js"; +import { externalImportBatches } from "../src/db/schema/external-import-batches.js"; +import { externalModelObjects } from "../src/db/schema/external-model-objects.js"; +import { externalModelSources } from "../src/db/schema/external-model-sources.js"; import { projectDevices } from "../src/db/schema/project-devices.js"; import { projectRevisions } from "../src/db/schema/project-revisions.js"; import { projects } from "../src/db/schema/projects.js"; +import { externalCsvTestConfiguration } from "./fixtures/revit-csv-fixtures.js"; import { createProjectDeviceRowSyncProjectCommand, type ProjectDeviceSyncRowSnapshot, @@ -163,6 +167,79 @@ function getRow(context: DatabaseContext, rowId: string) { return row; } +function linkExternalObject( + context: DatabaseContext, + rowId: string, + effectiveQuantity: number +) { + context.db + .insert(externalModelSources) + .values({ + id: "source-1", + projectId: "project-1", + name: "Revit", + sourceType: "revit_csv", + }) + .run(); + context.db + .insert(externalImportBatches) + .values({ + id: "batch-1", + projectId: "project-1", + sourceId: "source-1", + importKind: "initial", + importedAtIso: "2026-08-02T16:00:00.000Z", + fileName: "revit.csv", + sha256: "a".repeat(64), + appliedProjectRevision: 0, + configurationVersion: 1, + configurationSnapshot: externalCsvTestConfiguration, + originalBytes: Buffer.from("test"), + document: { delimiter: ";", encoding: "utf-8", headers: [], rows: [] }, + }) + .run(); + context.db + .insert(externalModelObjects) + .values({ + id: "object-1", + projectId: "project-1", + sourceId: "source-1", + ifcGuid: "ifc-1", + lastSeenImportBatchId: "batch-1", + lastAcceptedImportBatchId: "batch-1", + acceptedSourceValues: { + rowNumber: 2, + roomNumber: "101", + roomName: "Büro", + familyAndType: "Leuchte: Standard", + selectionMarker: "Leuchte", + circuitIdentifier: "-1F1", + power: "30", + quantity: String(effectiveQuantity), + additionalSourceValues: {}, + }, + planningValues: { + displayName: "Leuchte", + internalDeviceType: "luminaire", + category: "single_phase", + connectionKind: "fixed", + effectiveQuantity, + powerPerUnitW: 30, + simultaneityFactor: 1, + cosPhi: null, + costGroup: null, + remark: null, + }, + overriddenFields: [], + externalRoomMappingId: null, + distributionBoardId: null, + linkedProjectDeviceId: null, + circuitDeviceRowId: rowId, + presenceStatus: "present", + }) + .run(); +} + function snapshot( context: DatabaseContext, rowId: string @@ -474,6 +551,112 @@ describe("project-device row sync project-command repository", () => { } }); + it("keeps manualQuantity from exceeding quantity when a synced quantity shrinks", () => { + const fixture = createTestDatabase(); + try { + fixture.context.db + .update(circuitDeviceRows) + .set({ quantity: 5, manualQuantity: 5 }) + .where(eq(circuitDeviceRows.id, "row-1")) + .run(); + const store = new ProjectDeviceRowSyncProjectCommandRepository( + fixture.context.db + ); + const expected = snapshot(fixture.context, "row-1"); + store.execute({ + projectId: "project-1", + expectedRevision: 0, + source: "user", + command: createProjectDeviceRowSyncProjectCommand( + "project-device-1", + "synchronize", + [ + { + rowId: "row-1", + expected, + target: { ...expected, quantity: 2 }, + }, + ] + ), + }); + const row = getRow(fixture.context, "row-1"); + assert.equal(row.quantity, 2); + assert.equal(row.manualQuantity, 2); + } finally { + fixture.context.close(); + } + }); + + it("subtracts linked external objects when a synced quantity shrinks", () => { + const fixture = createTestDatabase(); + try { + fixture.context.db + .update(circuitDeviceRows) + .set({ quantity: 5, manualQuantity: 2 }) + .where(eq(circuitDeviceRows.id, "row-1")) + .run(); + linkExternalObject(fixture.context, "row-1", 3); + const store = new ProjectDeviceRowSyncProjectCommandRepository( + fixture.context.db + ); + const expected = snapshot(fixture.context, "row-1"); + store.execute({ + projectId: "project-1", + expectedRevision: 0, + source: "user", + command: createProjectDeviceRowSyncProjectCommand( + "project-device-1", + "synchronize", + [{ rowId: "row-1", expected, target: { ...expected, quantity: 4 } }] + ), + }); + const row = getRow(fixture.context, "row-1"); + assert.equal(row.quantity, 4); + assert.equal(row.manualQuantity, 1); + } finally { + fixture.context.close(); + } + }); + + it("rejects a synced quantity below the linked external total", () => { + const fixture = createTestDatabase(); + try { + fixture.context.db + .update(circuitDeviceRows) + .set({ quantity: 5, manualQuantity: 2 }) + .where(eq(circuitDeviceRows.id, "row-1")) + .run(); + linkExternalObject(fixture.context, "row-1", 3); + const store = new ProjectDeviceRowSyncProjectCommandRepository( + fixture.context.db + ); + const expected = snapshot(fixture.context, "row-1"); + assert.throws( + () => + store.execute({ + projectId: "project-1", + expectedRevision: 0, + source: "user", + command: createProjectDeviceRowSyncProjectCommand( + "project-device-1", + "synchronize", + [{ rowId: "row-1", expected, target: { ...expected, quantity: 2 } }] + ), + }), + /below the total quantity of linked external objects/ + ); + const row = getRow(fixture.context, "row-1"); + assert.equal(row.quantity, 5); + assert.equal(row.manualQuantity, 2); + assert.equal( + fixture.context.db.select().from(projectRevisions).all().length, + 0 + ); + } finally { + fixture.context.close(); + } + }); + it("rolls back synchronized rows for a stale project revision", () => { const fixture = createTestDatabase(); try { From 906aa751c716a93b317a374456586079fd5ae9b0 Mon Sep 17 00:00:00 2001 From: Julian Appel Date: Sun, 23 Aug 2026 18:25:06 +0200 Subject: [PATCH 4/5] Run the Docker stack on built output instead of dev servers compose.yaml overrode the image's own entrypoint to start "tsx watch" and "next dev" with CHOKIDAR_USEPOLLING and WATCHPACK_POLLING enabled. The built dist/ and .next from the image were never used, and the polling watchers kept both containers busy around the clock whether or not anyone used the application. Split the two use cases: compose.yaml now runs node dist/server/index.js and next start with no source mounts and no watchers, and the watching setup moves to compose.dev.yaml. next build writes the rewrite destinations from next.config.mjs into .next/routes-manifest.json, so next start cannot pick up API_INTERNAL_URL at runtime the way next dev does. Pass it as a build argument. Give the web container its own probe. The healthcheck hit "/", which redirects to "/projects", and fetch follows redirects, so every probe rendered two pages every five seconds. "/health" could not serve as the probe because next.config.mjs rewrites it to the API. Add "/web-health" as a route handler, exclude it from the proxy matcher, and relax both intervals to 30s. Co-Authored-By: Claude Opus 5 --- Dockerfile | 7 +++ README.md | 18 +++++++- compose.dev.yaml | 87 +++++++++++++++++++++++++++++++++++++ compose.yaml | 65 ++++++++++++--------------- docs/deployment.md | 39 ++++++++++++----- package.json | 3 ++ src/app/web-health/route.ts | 10 +++++ src/proxy.ts | 9 ++-- 8 files changed, 184 insertions(+), 54 deletions(-) create mode 100644 compose.dev.yaml create mode 100644 src/app/web-health/route.ts diff --git a/Dockerfile b/Dockerfile index 5484443..31d28c4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -5,6 +5,13 @@ COPY package*.json ./ RUN npm ci COPY . . + +# next build writes the rewrite destinations from next.config.mjs into +# .next/routes-manifest.json, so "next start" cannot pick up a different +# API URL later. The value has to be known here, not just at runtime. +ARG API_INTERNAL_URL=http://localhost:3000 +ENV API_INTERNAL_URL=$API_INTERNAL_URL + RUN npm run build:api && npm run build:web RUN mkdir -p data && chmod +x scripts/docker-start.sh diff --git a/README.md b/README.md index d7a7b09..fc82fca 100644 --- a/README.md +++ b/README.md @@ -62,10 +62,24 @@ docker compose logs --follow docker compose down ``` -Der Compose-Stack startet Entwicklungsserver mit Quellcode-Mounts. Er ist kein -Produktionsdeployment. Details stehen in +`compose.yaml` startet den Produktionsstand: gebautes `dist/` und `next start`, +ohne Quellcode-Mounts und ohne Datei-Watcher. Details stehen in [Deployment und Betrieb](docs/deployment.md). +Für die Entwicklung mit Hot Reload gibt es einen eigenen Stack mit +Quellcode-Mounts und Watchern: + +```powershell +docker compose -f compose.dev.yaml up --build --detach +docker compose -f compose.dev.yaml logs --follow +docker compose -f compose.dev.yaml down +``` + +Die Watcher darin laufen im Polling-Modus, weil Bind-Mounts unter Windows und +macOS keine inotify-Events durchreichen. Das kostet dauerhaft CPU, auch wenn +niemand die Anwendung benutzt — deshalb gehört dieser Stack nicht auf einen +Server. + ## Direkte lokale Entwicklung Voraussetzungen: diff --git a/compose.dev.yaml b/compose.dev.yaml new file mode 100644 index 0000000..86e42e2 --- /dev/null +++ b/compose.dev.yaml @@ -0,0 +1,87 @@ +# Development stack: source mounts, watching dev servers, hot reload. +# docker compose -f compose.dev.yaml up --build +# +# The polling watchers below are needed for bind mounts on Windows and +# macOS, where inotify events do not cross the VM boundary. They cost +# continuous CPU, which is why the production stack in compose.yaml does +# not run watchers at all. +name: leistungsbilanz-dev + +x-logging: &logging + driver: json-file + options: + max-size: "20m" + max-file: "10" + +services: + api: + build: + context: . + command: + - sh + - -c + - npm run db:migrate && npm run db:verify:circuit-schema && npm run dev:api + environment: + PORT: "3000" + CHOKIDAR_USEPOLLING: "true" + LOG_LEVEL: "${LOG_LEVEL:-debug}" + init: true + restart: unless-stopped + logging: *logging + ports: + - "3000:3000" + volumes: + - ./src:/app/src + - ./scripts:/app/scripts + - ./data:/app/data + - ./drizzle.config.ts:/app/drizzle.config.ts:ro + - ./tsconfig.json:/app/tsconfig.json:ro + healthcheck: + test: + - CMD + - node + - -e + - fetch('http://localhost:3000/health').then(response=>{if(!response.ok)process.exit(1)}).catch(()=>process.exit(1)) + interval: 30s + timeout: 3s + retries: 5 + start_period: 20s + + web: + build: + context: . + command: + - npm + - run + - dev:web + - -- + - --hostname + - 0.0.0.0 + environment: + API_INTERNAL_URL: http://api:3000 + WATCHPACK_POLLING: "true" + NEXT_TELEMETRY_DISABLED: "1" + LOG_LEVEL: "${LOG_LEVEL:-debug}" + init: true + restart: unless-stopped + logging: *logging + depends_on: + api: + condition: service_healthy + ports: + - "3001:3001" + volumes: + - ./src:/app/src + - ./next.config.mjs:/app/next.config.mjs:ro + - ./tsconfig.json:/app/tsconfig.json:ro + - ./tsconfig.next.json:/app/tsconfig.next.json:ro + healthcheck: + test: + - CMD + - node + - -e + - fetch('http://localhost:3001/web-health').then(response=>{if(!response.ok)process.exit(1)}).catch(()=>process.exit(1)) + interval: 30s + timeout: 3s + retries: 5 + start_period: 20s diff --git a/compose.yaml b/compose.yaml index 6a5ec7c..503a4b2 100644 --- a/compose.yaml +++ b/compose.yaml @@ -1,82 +1,73 @@ name: leistungsbilanz +x-build: &build + context: . + args: + # Baked into .next/routes-manifest.json by next build; see Dockerfile. + API_INTERNAL_URL: http://api:3000 + +x-logging: &logging + driver: json-file + options: + max-size: "20m" + max-file: "10" + services: api: - build: - context: . + build: *build command: - sh - -c - - npm run db:migrate && npm run db:verify:circuit-schema && npm run dev:api + - node scripts/run-migrations.js && node scripts/db-verify-circuit-schema.js && node dist/server/index.js environment: + NODE_ENV: production PORT: "3000" - CHOKIDAR_USEPOLLING: "true" LOG_LEVEL: "${LOG_LEVEL:-info}" init: true restart: unless-stopped - logging: - driver: json-file - options: - max-size: "20m" - max-file: "10" + logging: *logging ports: - "3000:3000" volumes: - - ./src:/app/src - - ./scripts:/app/scripts - ./data:/app/data - - ./drizzle.config.ts:/app/drizzle.config.ts:ro - - ./tsconfig.json:/app/tsconfig.json:ro healthcheck: test: - CMD - node - -e - fetch('http://localhost:3000/health').then(response=>{if(!response.ok)process.exit(1)}).catch(()=>process.exit(1)) - interval: 5s + interval: 30s timeout: 3s - retries: 12 + retries: 5 start_period: 20s web: - build: - context: . + build: *build command: - - npm - - run - - dev:web - - -- - - --hostname - - 0.0.0.0 + - node_modules/.bin/next + - start + - -p + - "3001" environment: + NODE_ENV: production API_INTERNAL_URL: http://api:3000 - WATCHPACK_POLLING: "true" NEXT_TELEMETRY_DISABLED: "1" LOG_LEVEL: "${LOG_LEVEL:-info}" init: true restart: unless-stopped - logging: - driver: json-file - options: - max-size: "20m" - max-file: "10" + logging: *logging depends_on: api: condition: service_healthy ports: - "3001:3001" - volumes: - - ./src:/app/src - - ./next.config.mjs:/app/next.config.mjs:ro - - ./tsconfig.json:/app/tsconfig.json:ro - - ./tsconfig.next.json:/app/tsconfig.next.json:ro healthcheck: test: - CMD - node - -e - - fetch('http://localhost:3001/').then(response=>{if(!response.ok)process.exit(1)}).catch(()=>process.exit(1)) - interval: 5s + - fetch('http://localhost:3001/web-health').then(response=>{if(!response.ok)process.exit(1)}).catch(()=>process.exit(1)) + interval: 30s timeout: 3s - retries: 12 + retries: 5 start_period: 20s diff --git a/docs/deployment.md b/docs/deployment.md index dfbb3f1..06289a7 100644 --- a/docs/deployment.md +++ b/docs/deployment.md @@ -2,19 +2,27 @@ ## Aktueller Status -Es gibt derzeit kein unterstütztes Produktionsdeployment. +Es gibt zwei Compose-Stacks. -`compose.yaml` ist ausschließlich für lokale Entwicklung vorgesehen. Es startet -`tsx watch` und `next dev`, bindet Quellcode vom Host ein und enthält weder TLS, -Authentifizierung, Reverse Proxy, Prozesshärtung noch ein zentral betriebenes -Datenbanksystem. Der Stack darf deshalb nicht als produktionsreif bezeichnet oder -öffentlich erreichbar gemacht werden. +`compose.yaml` startet den gebauten Stand: `node dist/server/index.js` und +`next start`, ohne Quellcode-Mounts und ohne Datei-Watcher. Das ist der Stack +für einen Server. -## Entwicklungs-Topologie +`compose.dev.yaml` startet `tsx watch` und `next dev` und bindet Quellcode vom +Host ein. Die Watcher laufen im Polling-Modus, weil Bind-Mounts unter Windows +und macOS keine inotify-Events durchreichen; das kostet dauerhaft CPU, auch +ohne Benutzeraktivität. Dieser Stack gehört deshalb nur auf einen +Entwicklungsrechner. + +Beides enthält weder TLS, Authentifizierung, Reverse Proxy, Prozesshärtung noch +ein zentral betriebenes Datenbanksystem. Der Stack darf deshalb nicht öffentlich +erreichbar gemacht werden. + +## Topologie | Komponente | Port | Healthcheck | Persistenz | | --- | ---: | --- | --- | -| Next.js Web | 3001 | `GET /` | keine | +| Next.js Web | 3001 | `GET /web-health` | keine | | Express API | 3000 | `GET /health` | `./data:/app/data` | | SQLite | Datei | Integritäts-/FK-Prüfung via Backup und Skript | `data/leistungsbilanz.db` | @@ -24,15 +32,22 @@ Verwendete Umgebungsvariablen: - `API_INTERNAL_URL` – internes API-Ziel des Next.js-Rewrites, im Compose-Netz `http://api:3000` - `NEXT_TELEMETRY_DISABLED=1` -- `CHOKIDAR_USEPOLLING=true` und `WATCHPACK_POLLING=true` für lokale - Dateibeobachtung in Docker +- `CHOKIDAR_USEPOLLING=true` und `WATCHPACK_POLLING=true` – nur in + `compose.dev.yaml`, für Dateibeobachtung über Bind-Mounts hinweg - `LOG_LEVEL` – steuert für beide Dienste die Ausgabestufe des strukturierten JSON-Loggers (`error`, `warn`, `info`, `verbose`, `debug`), Standard `info`. Setzbar über eine `.env`-Datei neben `compose.yaml` oder `LOG_LEVEL=verbose docker compose up`. -Beim API-Start laufen zuerst `npm run db:migrate` und -`npm run db:verify:circuit-schema`. +Beim API-Start laufen zuerst die Migrationen und die Schemaprüfung +(`scripts/run-migrations.js` und `scripts/db-verify-circuit-schema.js`, im +Entwicklungsstack über `npm run db:migrate` und +`npm run db:verify:circuit-schema`). + +`API_INTERNAL_URL` wirkt für `next start` zur **Build-Zeit**: `next build` +schreibt die Rewrite-Ziele aus `next.config.mjs` fest in +`.next/routes-manifest.json`. `compose.yaml` reicht den Wert deshalb als +Build-Argument an das Image durch, nicht nur als Laufzeit-Variable. ## Logging diff --git a/package.json b/package.json index d5ebe82..414eb53 100644 --- a/package.json +++ b/package.json @@ -13,6 +13,9 @@ "docker:up": "docker compose up --build --detach", "docker:down": "docker compose down", "docker:logs": "docker compose logs --follow", + "docker:dev:up": "docker compose -f compose.dev.yaml up --build --detach", + "docker:dev:down": "docker compose -f compose.dev.yaml down", + "docker:dev:logs": "docker compose -f compose.dev.yaml logs --follow", "build": "npm run build:api", "build:api": "tsc -p tsconfig.json", "build:web": "next build", diff --git a/src/app/web-health/route.ts b/src/app/web-health/route.ts new file mode 100644 index 0000000..ce6f2c4 --- /dev/null +++ b/src/app/web-health/route.ts @@ -0,0 +1,10 @@ +import { NextResponse } from "next/server"; + +// Liveness probe for the web container itself. The "/health" path is +// rewritten to the API in next.config.mjs, so it cannot answer for this +// process. Kept as a route handler so a probe does not render a page. +export const dynamic = "force-dynamic"; + +export function GET() { + return NextResponse.json({ ok: true }); +} diff --git a/src/proxy.ts b/src/proxy.ts index 4607ec0..3f4b83a 100644 --- a/src/proxy.ts +++ b/src/proxy.ts @@ -5,8 +5,9 @@ import { createLogger } from "./shared/logging/logger"; const logger = createLogger("web:navigation"); export function proxy(request: NextRequest) { - // The Docker healthcheck hits "/" every few seconds with no User-Agent - // header; skip it so real navigation isn't drowned out in the logs. + // Probes hit "/web-health" and are excluded by the matcher below. The + // User-Agent guard stays as a fallback for anything else that polls + // without one, so real navigation isn't drowned out in the logs. if (request.headers.get("user-agent")) { logger.info("page request", { method: request.method, @@ -17,5 +18,7 @@ export function proxy(request: NextRequest) { } export const config = { - matcher: ["/((?!_next/static|_next/image|favicon.ico|api).*)"], + matcher: [ + "/((?!_next/static|_next/image|favicon.ico|api|web-health).*)", + ], }; From 204499d7e4010533b4edae304863a738e1d4e953 Mon Sep 17 00:00:00 2001 From: Julian Appel Date: Sun, 23 Aug 2026 23:13:35 +0200 Subject: [PATCH 5/5] Add Revit Dynamo diagnostic scripts --- .gitignore | 1 + dynamo/01_check_model_identity.py | 385 +++++++++++++ ..._electrical_fixture_parameter_inventory.py | 517 ++++++++++++++++++ dynamo/README.md | 69 +++ 4 files changed, 972 insertions(+) create mode 100644 dynamo/01_check_model_identity.py create mode 100644 dynamo/02_export_electrical_fixture_parameter_inventory.py create mode 100644 dynamo/README.md diff --git a/.gitignore b/.gitignore index 8d44fb4..ed5ffe6 100644 --- a/.gitignore +++ b/.gitignore @@ -4,3 +4,4 @@ dist/ data/*.db data/backups/*.db .codex/*.log +dynamo/output/ diff --git a/dynamo/01_check_model_identity.py b/dynamo/01_check_model_identity.py new file mode 100644 index 0000000..a3fe009 --- /dev/null +++ b/dynamo/01_check_model_identity.py @@ -0,0 +1,385 @@ +"""Read-only Revit 2026 model-identity diagnostics for a Dynamo Python node. + +Optional Dynamo input: + IN[0]: output directory or complete .json file path + +The script intentionally performs no Revit transaction and changes no model +data. OUT contains a compact summary plus the complete report. +""" + +import datetime +import json +import os +import platform +import sys +import tempfile +import traceback + +import clr + +clr.AddReference("RevitAPI") +clr.AddReference("RevitServices") + +from Autodesk.Revit.DB import ModelPathUtils, StorageType # noqa: E402 +from RevitServices.Persistence import DocumentManager # noqa: E402 + + +CHECKED_PARAMETER_NAMES = ("LB_ModelId", "LB_ProjectId") + + +def safe_text(value): + if value is None: + return None + try: + return str(value) + except Exception: + return None + + +def element_id_text(element_id): + if element_id is None: + return None + try: + return str(element_id.Value) + except Exception: + try: + return str(element_id.IntegerValue) + except Exception: + return safe_text(element_id) + + +def forge_type_id_text(value): + if value is None: + return None + try: + return value.TypeId + except Exception: + return safe_text(value) + + +def parameter_value(parameter): + result = { + "hasValue": False, + "raw": None, + "display": None, + } + try: + result["hasValue"] = bool(parameter.HasValue) + except Exception: + pass + + try: + storage_type = parameter.StorageType + if storage_type == StorageType.String: + result["raw"] = parameter.AsString() + elif storage_type == StorageType.Integer: + result["raw"] = int(parameter.AsInteger()) + elif storage_type == StorageType.Double: + result["raw"] = float(parameter.AsDouble()) + elif storage_type == StorageType.ElementId: + result["raw"] = element_id_text(parameter.AsElementId()) + except Exception as error: + result["readError"] = safe_text(error) + + try: + result["display"] = parameter.AsValueString() + except Exception: + pass + return result + + +def describe_parameter(parameter): + definition = None + try: + definition = parameter.Definition + except Exception: + pass + + name = None + if definition is not None: + try: + name = definition.Name + except Exception: + pass + + is_shared = False + try: + is_shared = bool(parameter.IsShared) + except Exception: + pass + + shared_guid = None + if is_shared: + try: + shared_guid = str(parameter.GUID) + except Exception: + pass + + data_type = None + group_type = None + if definition is not None: + try: + data_type = forge_type_id_text(definition.GetDataType()) + except Exception: + pass + try: + group_type = forge_type_id_text(definition.GetGroupTypeId()) + except Exception: + pass + + unit_type = None + try: + unit_type = forge_type_id_text(parameter.GetUnitTypeId()) + except Exception: + pass + + try: + storage_type = str(parameter.StorageType) + except Exception: + storage_type = None + + try: + is_read_only = bool(parameter.IsReadOnly) + except Exception: + is_read_only = None + + try: + user_modifiable = bool(parameter.UserModifiable) + except Exception: + user_modifiable = None + + return { + "name": name, + "parameterId": element_id_text(getattr(parameter, "Id", None)), + "isShared": is_shared, + "sharedGuid": shared_guid, + "storageType": storage_type, + "dataTypeId": data_type, + "groupTypeId": group_type, + "unitTypeId": unit_type, + "isReadOnly": is_read_only, + "userModifiable": user_modifiable, + "value": parameter_value(parameter), + } + + +def sorted_parameters(element): + parameters = [] + try: + parameters = [describe_parameter(parameter) for parameter in element.Parameters] + except Exception: + return [] + return sorted( + parameters, + key=lambda parameter: ( + (parameter.get("name") or "").casefold(), + parameter.get("parameterId") or "", + ), + ) + + +def named_parameter_occurrences(element, parameter_name): + result = [] + try: + parameters = element.GetParameters(parameter_name) + if parameters is not None: + result = [describe_parameter(parameter) for parameter in parameters] + except Exception: + parameter = None + try: + parameter = element.LookupParameter(parameter_name) + except Exception: + pass + if parameter is not None: + result = [describe_parameter(parameter)] + return result + + +def loaded_assembly_versions(): + result = {} + try: + from System import AppDomain + + for assembly in AppDomain.CurrentDomain.GetAssemblies(): + try: + name = assembly.GetName() + simple_name = str(name.Name) + if simple_name in ( + "DynamoCore", + "DynamoCoreWpf", + "DynamoRevitDS", + "RevitAPI", + "RevitServices", + ): + result[simple_name] = str(name.Version) + except Exception: + continue + except Exception: + pass + return dict(sorted(result.items())) + + +def get_cloud_identity(document): + result = {"isModelInCloud": False} + try: + result["isModelInCloud"] = bool(document.IsModelInCloud) + except Exception: + return result + if not result["isModelInCloud"]: + return result + + try: + model_path = document.GetCloudModelPath() + result["userVisiblePath"] = ModelPathUtils.ConvertModelPathToUserVisiblePath( + model_path + ) + for property_name, output_name in ( + ("GetProjectGUID", "projectGuid"), + ("GetModelGUID", "modelGuid"), + ): + try: + result[output_name] = str(getattr(model_path, property_name)()) + except Exception: + pass + except Exception as error: + result["readError"] = safe_text(error) + return result + + +def get_worksharing_identity(document): + result = {"isWorkshared": False} + try: + result["isWorkshared"] = bool(document.IsWorkshared) + except Exception: + return result + if not result["isWorkshared"]: + return result + + try: + model_path = document.GetWorksharingCentralModelPath() + result["centralModelPath"] = ModelPathUtils.ConvertModelPathToUserVisiblePath( + model_path + ) + except Exception as error: + result["readError"] = safe_text(error) + return result + + +def resolve_output_path(configured_path, report_name): + timestamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + default_directory = os.path.join(tempfile.gettempdir(), "leistungsbilanz-dynamo") + raw_path = safe_text(configured_path) + if raw_path is None or not raw_path.strip(): + directory = default_directory + file_path = os.path.join(directory, report_name + "-" + timestamp + ".json") + else: + expanded = os.path.abspath(os.path.expandvars(os.path.expanduser(raw_path.strip()))) + if expanded.lower().endswith(".json"): + file_path = expanded + directory = os.path.dirname(file_path) + else: + directory = expanded + file_path = os.path.join(directory, report_name + "-" + timestamp + ".json") + if not directory: + directory = os.getcwd() + if not os.path.isdir(directory): + os.makedirs(directory) + return file_path + + +def write_json(file_path, payload): + temporary_path = file_path + ".tmp" + with open(temporary_path, "w", encoding="utf-8", newline="\n") as output: + json.dump(payload, output, ensure_ascii=False, indent=2, sort_keys=True) + output.write("\n") + os.replace(temporary_path, file_path) + + +def get_input(index, default=None): + values = globals().get("IN", []) + try: + value = values[index] + return default if value is None else value + except Exception: + return default + + +def build_report(): + document = DocumentManager.Instance.CurrentDBDocument + if document is None: + raise RuntimeError("No active Revit document is available.") + + project_information = document.ProjectInformation + if project_information is None: + raise RuntimeError("The active document has no Project Information element.") + + application = document.Application + checked_parameters = { + name: named_parameter_occurrences(project_information, name) + for name in CHECKED_PARAMETER_NAMES + } + warnings = [] + for name in CHECKED_PARAMETER_NAMES: + occurrences = checked_parameters[name] + populated = [ + parameter + for parameter in occurrences + if parameter.get("value", {}).get("raw") not in (None, "") + ] + if not occurrences: + warnings.append(name + " is not bound to Project Information.") + elif not populated: + warnings.append(name + " exists but has no value on Project Information.") + elif len(occurrences) > 1: + warnings.append(name + " occurs more than once; use a shared-parameter GUID later.") + + return { + "reportSchemaVersion": 1, + "generatedAt": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "readOnly": True, + "environment": { + "revitVersionNumber": safe_text(getattr(application, "VersionNumber", None)), + "revitVersionName": safe_text(getattr(application, "VersionName", None)), + "revitSubVersionNumber": safe_text( + getattr(application, "SubVersionNumber", None) + ), + "pythonImplementation": safe_text(getattr(sys.implementation, "name", None)), + "pythonVersion": platform.python_version(), + "assemblies": loaded_assembly_versions(), + }, + "document": { + "title": safe_text(document.Title), + "pathName": safe_text(document.PathName), + "isFamilyDocument": bool(document.IsFamilyDocument), + "cloud": get_cloud_identity(document), + "worksharing": get_worksharing_identity(document), + }, + "modelIdentityCandidates": { + "projectInformationUniqueId": safe_text(project_information.UniqueId), + "projectInformationElementId": element_id_text(project_information.Id), + "checkedProjectParameters": checked_parameters, + }, + "projectInformationParameters": sorted_parameters(project_information), + "warnings": warnings, + } + + +try: + report = build_report() + output_path = resolve_output_path(get_input(0), "model-identity") + write_json(output_path, report) + OUT = { + "ok": True, + "filePath": output_path, + "projectInformationUniqueId": report["modelIdentityCandidates"][ + "projectInformationUniqueId" + ], + "warnings": report["warnings"], + "report": report, + } +except Exception as error: + OUT = { + "ok": False, + "error": safe_text(error), + "traceback": traceback.format_exc(), + } diff --git a/dynamo/02_export_electrical_fixture_parameter_inventory.py b/dynamo/02_export_electrical_fixture_parameter_inventory.py new file mode 100644 index 0000000..440a222 --- /dev/null +++ b/dynamo/02_export_electrical_fixture_parameter_inventory.py @@ -0,0 +1,517 @@ +"""Export all Electrical Fixtures instance/type parameters from Revit 2026. + +Optional Dynamo inputs: + IN[0]: output directory or complete .json file path + IN[1]: include empty parameters (default True) + IN[2]: maximum aggregated sample values (default 5) + +The script is read-only and performs no Revit transaction. +""" + +import datetime +import json +import os +import platform +import sys +import tempfile +import traceback + +import clr + +clr.AddReference("RevitAPI") +clr.AddReference("RevitServices") + +from Autodesk.Revit.DB import ( # noqa: E402 + BuiltInCategory, + FilteredElementCollector, + ModelPathUtils, + StorageType, +) +from RevitServices.Persistence import DocumentManager # noqa: E402 + + +def safe_text(value): + if value is None: + return None + try: + return str(value) + except Exception: + return None + + +def element_id_text(element_id): + if element_id is None: + return None + try: + return str(element_id.Value) + except Exception: + try: + return str(element_id.IntegerValue) + except Exception: + return safe_text(element_id) + + +def forge_type_id_text(value): + if value is None: + return None + try: + return value.TypeId + except Exception: + return safe_text(value) + + +def parameter_value(parameter): + result = {"hasValue": False, "raw": None, "display": None} + try: + result["hasValue"] = bool(parameter.HasValue) + except Exception: + pass + + try: + storage_type = parameter.StorageType + if storage_type == StorageType.String: + result["raw"] = parameter.AsString() + elif storage_type == StorageType.Integer: + result["raw"] = int(parameter.AsInteger()) + elif storage_type == StorageType.Double: + result["raw"] = float(parameter.AsDouble()) + elif storage_type == StorageType.ElementId: + result["raw"] = element_id_text(parameter.AsElementId()) + except Exception as error: + result["readError"] = safe_text(error) + + try: + result["display"] = parameter.AsValueString() + except Exception: + pass + return result + + +def describe_parameter(parameter, scope): + definition = None + try: + definition = parameter.Definition + except Exception: + pass + + name = None + data_type = None + group_type = None + if definition is not None: + try: + name = definition.Name + except Exception: + pass + try: + data_type = forge_type_id_text(definition.GetDataType()) + except Exception: + pass + try: + group_type = forge_type_id_text(definition.GetGroupTypeId()) + except Exception: + pass + + is_shared = False + try: + is_shared = bool(parameter.IsShared) + except Exception: + pass + + shared_guid = None + if is_shared: + try: + shared_guid = str(parameter.GUID) + except Exception: + pass + + unit_type = None + try: + unit_type = forge_type_id_text(parameter.GetUnitTypeId()) + except Exception: + pass + + try: + storage_type = str(parameter.StorageType) + except Exception: + storage_type = None + + try: + is_read_only = bool(parameter.IsReadOnly) + except Exception: + is_read_only = None + + try: + user_modifiable = bool(parameter.UserModifiable) + except Exception: + user_modifiable = None + + return { + "scope": scope, + "name": name, + "parameterId": element_id_text(getattr(parameter, "Id", None)), + "isShared": is_shared, + "sharedGuid": shared_guid, + "storageType": storage_type, + "dataTypeId": data_type, + "groupTypeId": group_type, + "unitTypeId": unit_type, + "isReadOnly": is_read_only, + "userModifiable": user_modifiable, + "value": parameter_value(parameter), + } + + +def has_meaningful_value(parameter_description): + value = parameter_description.get("value", {}) + return bool(value.get("hasValue")) or value.get("raw") not in (None, "") or value.get( + "display" + ) not in (None, "") + + +def read_parameters(element, scope, include_empty): + result = [] + try: + for parameter in element.Parameters: + description = describe_parameter(parameter, scope) + if include_empty or has_meaningful_value(description): + result.append(description) + except Exception as error: + return [], [safe_text(error)] + result.sort( + key=lambda parameter: ( + (parameter.get("name") or "").casefold(), + parameter.get("parameterId") or "", + ) + ) + return result, [] + + +def read_space(element, document): + try: + space = element.Space + except Exception as error: + return None, safe_text(error) + if space is None: + return None, None + + level_name = None + try: + level = document.GetElement(space.LevelId) + level_name = None if level is None else safe_text(level.Name) + except Exception: + pass + + return { + "uniqueId": safe_text(space.UniqueId), + "elementId": element_id_text(space.Id), + "number": safe_text(getattr(space, "Number", None)), + "name": safe_text(getattr(space, "Name", None)), + "levelName": level_name, + }, None + + +def read_family_identity(element, document): + symbol = None + try: + symbol = element.Symbol + except Exception: + try: + symbol = document.GetElement(element.GetTypeId()) + except Exception: + pass + + family_name = None + type_name = None + type_unique_id = None + type_element_id = None + if symbol is not None: + try: + family_name = safe_text(symbol.Family.Name) + except Exception: + family_name = safe_text(getattr(symbol, "FamilyName", None)) + type_name = safe_text(getattr(symbol, "Name", None)) + type_unique_id = safe_text(getattr(symbol, "UniqueId", None)) + type_element_id = element_id_text(getattr(symbol, "Id", None)) + + return { + "familyName": family_name, + "typeName": type_name, + "typeUniqueId": type_unique_id, + "typeElementId": type_element_id, + }, symbol + + +def parameter_inventory_key(parameter): + stable_id = parameter.get("sharedGuid") or parameter.get("parameterId") or "" + return "|".join( + ( + parameter.get("scope") or "", + stable_id, + parameter.get("name") or "", + parameter.get("dataTypeId") or "", + ) + ) + + +def sample_value_key(value): + return json.dumps(value, ensure_ascii=False, sort_keys=True, separators=(",", ":")) + + +def add_to_inventory(inventory, parameter, max_samples): + key = parameter_inventory_key(parameter) + entry = inventory.get(key) + if entry is None: + entry = { + "scope": parameter.get("scope"), + "name": parameter.get("name"), + "parameterId": parameter.get("parameterId"), + "isShared": parameter.get("isShared"), + "sharedGuid": parameter.get("sharedGuid"), + "storageType": parameter.get("storageType"), + "dataTypeId": parameter.get("dataTypeId"), + "groupTypeId": parameter.get("groupTypeId"), + "unitTypeId": parameter.get("unitTypeId"), + "occurrenceCount": 0, + "populatedCount": 0, + "sampleValues": [], + "_sampleKeys": set(), + } + inventory[key] = entry + entry["occurrenceCount"] += 1 + if has_meaningful_value(parameter): + entry["populatedCount"] += 1 + value = parameter.get("value") + value_key = sample_value_key(value) + if len(entry["sampleValues"]) < max_samples and value_key not in entry["_sampleKeys"]: + entry["_sampleKeys"].add(value_key) + entry["sampleValues"].append(value) + + +def finalize_inventory(inventory): + result = [] + for entry in inventory.values(): + clean_entry = dict(entry) + clean_entry.pop("_sampleKeys", None) + result.append(clean_entry) + return sorted( + result, + key=lambda entry: ( + entry.get("scope") or "", + (entry.get("name") or "").casefold(), + entry.get("sharedGuid") or entry.get("parameterId") or "", + ), + ) + + +def loaded_assembly_versions(): + result = {} + try: + from System import AppDomain + + for assembly in AppDomain.CurrentDomain.GetAssemblies(): + try: + name = assembly.GetName() + simple_name = str(name.Name) + if simple_name in ( + "DynamoCore", + "DynamoCoreWpf", + "DynamoRevitDS", + "RevitAPI", + "RevitServices", + ): + result[simple_name] = str(name.Version) + except Exception: + continue + except Exception: + pass + return dict(sorted(result.items())) + + +def resolve_output_path(configured_path): + timestamp = datetime.datetime.now().strftime("%Y%m%d-%H%M%S") + default_directory = os.path.join(tempfile.gettempdir(), "leistungsbilanz-dynamo") + raw_path = safe_text(configured_path) + if raw_path is None or not raw_path.strip(): + directory = default_directory + file_path = os.path.join( + directory, "electrical-fixture-parameters-" + timestamp + ".json" + ) + else: + expanded = os.path.abspath(os.path.expandvars(os.path.expanduser(raw_path.strip()))) + if expanded.lower().endswith(".json"): + file_path = expanded + directory = os.path.dirname(file_path) + else: + directory = expanded + file_path = os.path.join( + directory, "electrical-fixture-parameters-" + timestamp + ".json" + ) + if not directory: + directory = os.getcwd() + if not os.path.isdir(directory): + os.makedirs(directory) + return file_path + + +def write_json(file_path, payload): + temporary_path = file_path + ".tmp" + with open(temporary_path, "w", encoding="utf-8", newline="\n") as output: + json.dump(payload, output, ensure_ascii=False, indent=2, sort_keys=True) + output.write("\n") + os.replace(temporary_path, file_path) + + +def get_input(index, default=None): + values = globals().get("IN", []) + try: + value = values[index] + return default if value is None else value + except Exception: + return default + + +def build_report(include_empty, max_samples): + document = DocumentManager.Instance.CurrentDBDocument + if document is None: + raise RuntimeError("No active Revit document is available.") + if document.IsFamilyDocument: + raise RuntimeError("Open a Revit project document, not a family document.") + + application = document.Application + collector = ( + FilteredElementCollector(document) + .OfCategory(BuiltInCategory.OST_ElectricalFixtures) + .WhereElementIsNotElementType() + ) + source_elements = list(collector) + source_elements.sort(key=lambda element: safe_text(element.UniqueId) or "") + + elements = [] + types_by_unique_id = {} + inventory = {} + errors = [] + elements_without_space = 0 + + for element in source_elements: + element_errors = [] + try: + family_identity, symbol = read_family_identity(element, document) + instance_parameters, parameter_errors = read_parameters( + element, "instance", include_empty + ) + element_errors.extend(parameter_errors) + for parameter in instance_parameters: + add_to_inventory(inventory, parameter, max_samples) + + type_unique_id = family_identity.get("typeUniqueId") + if symbol is not None and type_unique_id and type_unique_id not in types_by_unique_id: + type_parameters, type_errors = read_parameters(symbol, "type", include_empty) + element_errors.extend(type_errors) + for parameter in type_parameters: + add_to_inventory(inventory, parameter, max_samples) + types_by_unique_id[type_unique_id] = { + **family_identity, + "parameters": type_parameters, + } + + space, space_error = read_space(element, document) + if space_error: + element_errors.append("MEP Space: " + space_error) + if space is None: + elements_without_space += 1 + + elements.append( + { + "uniqueId": safe_text(element.UniqueId), + "elementId": element_id_text(element.Id), + "categoryName": safe_text( + None if element.Category is None else element.Category.Name + ), + "family": family_identity, + "space": space, + "instanceParameters": instance_parameters, + "warnings": element_errors, + } + ) + except Exception as error: + errors.append( + { + "uniqueId": safe_text(getattr(element, "UniqueId", None)), + "elementId": element_id_text(getattr(element, "Id", None)), + "error": safe_text(error), + } + ) + + return { + "reportSchemaVersion": 1, + "generatedAt": datetime.datetime.now(datetime.timezone.utc).isoformat(), + "readOnly": True, + "complete": len(errors) == 0, + "scope": { + "builtInCategory": "OST_ElectricalFixtures", + "wholeDocument": True, + "elementTypesExcluded": True, + "includeEmptyParameters": include_empty, + }, + "environment": { + "revitVersionNumber": safe_text(getattr(application, "VersionNumber", None)), + "revitVersionName": safe_text(getattr(application, "VersionName", None)), + "revitSubVersionNumber": safe_text( + getattr(application, "SubVersionNumber", None) + ), + "pythonImplementation": safe_text(getattr(sys.implementation, "name", None)), + "pythonVersion": platform.python_version(), + "assemblies": loaded_assembly_versions(), + }, + "document": { + "title": safe_text(document.Title), + "pathName": safe_text(document.PathName), + "projectInformationUniqueId": safe_text(document.ProjectInformation.UniqueId), + }, + "summary": { + "elementCount": len(source_elements), + "exportedElementCount": len(elements), + "typeCount": len(types_by_unique_id), + "parameterDefinitionCount": len(inventory), + "elementsWithoutMepSpace": elements_without_space, + "elementErrorCount": len(errors), + }, + "parameterInventory": finalize_inventory(inventory), + "types": sorted( + types_by_unique_id.values(), + key=lambda entry: ( + (entry.get("familyName") or "").casefold(), + (entry.get("typeName") or "").casefold(), + entry.get("typeUniqueId") or "", + ), + ), + "elements": elements, + "errors": errors, + } + + +try: + include_empty_input = get_input(1, True) + include_empty = bool(include_empty_input) + try: + max_samples = int(get_input(2, 5)) + except Exception: + max_samples = 5 + max_samples = max(0, min(max_samples, 50)) + + report = build_report(include_empty, max_samples) + output_path = resolve_output_path(get_input(0)) + write_json(output_path, report) + OUT = { + "ok": True, + "filePath": output_path, + "complete": report["complete"], + "summary": report["summary"], + "errors": report["errors"], + } +except Exception as error: + OUT = { + "ok": False, + "error": safe_text(error), + "traceback": traceback.format_exc(), + } diff --git a/dynamo/README.md b/dynamo/README.md new file mode 100644 index 0000000..800eded --- /dev/null +++ b/dynamo/README.md @@ -0,0 +1,69 @@ +# Revit 2026 / Dynamo diagnostics + +This directory contains self-contained Python scripts for a Dynamo **Python +Script** node. They use only Dynamo's built-in Revit integration, the Revit API +and the Python standard library. No Dynamo package is required. + +The scripts are read-only. They do not start a Revit transaction and do not +change the open model. + +## Python engine + +Use the built-in `CPython3` engine in Revit 2026. Autodesk ships Dynamo with +Revit; optional PythonNet3 packages are not required by these diagnostics. + +## 01 - Check model identity + +File: `01_check_model_identity.py` + +The script reports: + +- Revit, Dynamo and Python versions; +- `ProjectInformation.UniqueId` as a native model-identity candidate; +- all occurrences and values of `LB_ModelId` and `LB_ProjectId` on Project + Information; +- all Project Information parameters; +- optional cloud/worksharing identity information when the API exposes it. + +Input `IN[0]` is optional. It may be either an output directory or a complete +`.json` file path. With no input, the report is written below the current +Windows temporary directory in `leistungsbilanz-dynamo`. + +## 02 - Inventory Electrical Fixtures parameters + +File: `02_export_electrical_fixture_parameter_inventory.py` + +The script reads every instance of +`BuiltInCategory.OST_ElectricalFixtures` in the complete current document. It +exports: + +- element, family, type and MEP Space identities; +- every instance parameter and value; +- every unique family-type parameter and value; +- an aggregated parameter inventory with occurrence counts and sample values; +- per-element warnings instead of aborting at the first unreadable element. + +Inputs: + +- `IN[0]` (optional): output directory or complete `.json` path; +- `IN[1]` (optional): include empty parameters, default `true`; +- `IN[2]` (optional): maximum sample values per aggregated parameter, default + `5`. + +The default output location is again the Windows temporary directory. The +generated report can contain model paths and project-specific parameter values; +review it before sharing or committing it. + +## Running a script + +1. Open the target model in Revit 2026. +2. Open Dynamo from **Manage > Visual Programming > Dynamo**. +3. Create a graph and add a **Python Script** node. +4. Select the `CPython3` engine for the node. +5. Copy the complete content of the desired `.py` file into the node. +6. Optionally connect a String node containing the output path to `IN[0]`. +7. Run the graph and inspect `OUT` for status, counts and the generated path. + +For the first test, run `01_check_model_identity.py` in the Revit main model. +Then run the parameter inventory. Keep both generated JSON files so their +structure can be checked before the production snapshot DTO is finalized.