Commit Graph

12 Commits

Author SHA1 Message Date
schmidt.florian ecadbe1477 Flow: count generation meters as sources (grid + solar → house)
ci / build-test (push) Successful in 1m14s
A meter's flow value is now its throughput — consumption OR generation output —
so a generation meter (solar) acts as a source that can feed downstream meters.
Setting a load meter's upstream to {grid, solar} now splits its consumption
across both proportionally, and the remainder under the sources (grid + solar −
load) surfaces as "Other" = export + battery/inverter losses. Negative values
(savings/balance virtuals) are clamped to 0 (a ribbon can't be negative). The
per-type KPI is relabelled "Top-level throughput" since it now spans generation.

Test: Generation_meter_counts_as_source (grid 75 + solar-gen 30 → house 40 →
28.57/11.43 split, 65 remainder). 69 Core + 48 Integration = 117 green.
Live-verified: electricity flow now shows Solar 1/2 as source nodes.

Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
2026-07-14 14:33:29 +02:00
schmidt.florian 7e34aeccc8 Meter chain topology + per-energy-type flow (Sankey) pages
ci / build-test (push) Successful in 1m24s
Adds a meter hierarchy and a flow view: a downstream meter is a *subsection*
of an upstream one (not an addition), so you can see where a main meter's flow
divides — e.g. official water → garden, pool, other; grid/battery → all → car.

- MeterLink (schema + migration AddMeterLinks): a directed from→to flow edge.
  Multi-parent allowed (a merge, e.g. grid + solar → house); multi-child is a
  split. Cascade-deletes with either endpoint; unique + distinct-endpoint checks.
- FlowService: per energy type + period, builds a Sankey graph — nodes = meters
  sized by consumption; link value = downstream meter's consumption, split
  proportionally across multiple upstreams; unaccounted remainder under a meter
  becomes a synthetic "Other" node; depth via topological longest-path.
- SankeyChart.razor: hand-rolled inline-SVG Sankey (ApexCharts has no Sankey
  type) — columns by depth, nodes stacked by value, bezier ribbons sized by flow,
  left→right, theme-aware, HTML-encoded labels, tooltips. Built as a MarkupString
  to sidestep Razor's <text> element clash.
- /energy/{id} page (one per energy type): KPIs (consumption + cost), the flow
  Sankey, and the meter list. NavMenu now lists a link per energy type
  (Electricity, Water, Gas, …) loaded from the DB.
- Meters admin: cycle-safe "Sub-meter of (upstream meters)" multi-select
  (descendants excluded to prevent cycles); reconciles meter_link rows on save.
- Reference data seeds a demo chain (Haus → Auto) so electricity flow shows
  Haus dividing into Auto + Other.

Tests: FlowServiceTests (single-parent remainder; two-parent proportional
split); render test now asserts the flow chain + covers /energy/{id}. 69 Core +
47 Integration = 116 green. Live-verified: Haus 95,450 kWh → Auto 51,909 +
Other 43,541 (flow conserved), all 5 energy-type pages render.

Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
2026-07-14 14:02:16 +02:00
schmidt.florian 09cd435c2b Admin write-CRUD, Home Assistant connector config, wiring audit
ci / build-test (push) Successful in 1m19s
Three requested phases.

1) Admin section (SDD §8.7) — MudBlazor inline-dialog CRUD, consistent pattern,
   delete guards, snackbar feedback, shared Confirm helper:
   - Energy types: create/edit/delete (blocks delete when meters reference it).
   - Meters: create/edit/delete; recomputes consumption when mode/baseline
     changes (NormalizationService over a fresh factory context, in a tx);
     delete cascades data (consumption+readings are Restrict → removed first).
   - A meter's ingest sources: manage on the meter-detail Sources tab
     (add/edit/delete MQTT/Tasmota/HA sources with typed config).
   - Tariffs: full CRUD (scope/component/value/validity).
   - Cost categories: CRUD + member management (meter or energy-type members).
   - Connectors: ingestion_endpoint CRUD (MQTT broker + Home Assistant);
     secrets referenced by env-var name only, never stored.
   - Settings: read-only effective-config view (settings are env-driven and
     reproducible, so an editable form would change nothing — kept honest).
   PV role is now editable on meters (MeterMeta.SetRole can clear a role).

2) Read Home Assistant — extracted a shared public HaEndpointConfig (was a
   private record in the worker), added HaConnectionTester (powers the connector
   "Test connection": checks base URL + env-resolved token, optionally reads one
   entity). Configuring an HA connector + an HA source on a meter drives the
   existing REST-poll worker end to end. (WebSocket push stays a future
   optimization; REST poll already reads HA.)

3) Wiring/placeholder audit — swept every OnClick/Href: all handlers are real,
   all internal links resolve to real routes, no TODO/stub/placeholder code.
   Fixed one genuine gap: MainLayout had no drawer toggle, so the nav was
   unreachable on narrow screens — added a hamburger button.

Tests: +6 (MeterMeta.SetRole role-removal; HaConnectionTester fail-closed
guard branches with a throwing HttpClientFactory proving no network on bad
config); render test now covers all admin routes. 69 Core + 45 Integration =
114 green. Live-verified in Docker: all admin pages 200, drawer toggle present,
Settings shows real effective config.

Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
2026-07-14 11:22:17 +02:00
schmidt.florian 1282acf82c SDD §8 panels (PV/oil/meter-detail) + fix reference-data-in-Docker
Complete the SDD §8 dashboard views that were deferred at the M5 boundary,
and fix a shipping bug that left the Docker demo empty.

Bug: "Load reference data" created meters/tank/tariffs but imported zero
readings in Docker. Root cause: sampledata/ was excluded by .dockerignore and
never copied into the build stage, so the App csproj's linked Content glob
resolved to nothing at publish time; ReferenceDataImporter then silently
skipped the missing CSVs after already writing its marker meter, leaving the
DB permanently "loaded" but empty.
  - .dockerignore: stop excluding sampledata/
  - Dockerfile: COPY sampledata/ into the build stage
  - ReferenceDataImporter: fail-fast (validate CSVs exist before the marker
    meter) and throw instead of silently skipping a missing file
  - Program.cs + MeterVaultOptions: opt-in MeterVault__SeedReferenceData
    (compose METERVAULT_SEED=true) for a one-command populated demo

New SDD §8 panels (read models in Infrastructure/Dashboard, Blazor pages):
  - §8.4 Solar/PV (/solar): generation from GenerationCounter meters;
    self-consumption / autarky % / self-consumption % / savings derived from
    meters tagged total_load & grid_import via Meter.Meta role config
    (MeterRoles/MeterMeta) — nothing hardcoded by name.
  - §8.5 Oil/consumable (/consumables): tank level (cm→L calibrated), fill
    gauge, deliveries log, burner runtime, effective L/h (fixed/empirical),
    forecast-to-empty, tariff cost, monthly series.
  - §8.6 Meter detail (/meters/{id}): raw readings, normalized consumption,
    source status, tariff timeline, events, measured-vs-estimated markers.
  - Reusable SeriesChart component; nav links; Meters list rows link to detail.

Tests: MeterMetaTests (Core, +10); DashboardRenderTests extended to assert the
three panel services compute real figures and the new routes render (108 total,
all green). Live-verified in Docker: seed imports 302 readings / 347 consumption
rows; panels render (generation 16,481 kWh, oil 3,967 L) cross-checking the DB.

Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
2026-07-14 09:52:11 +02:00
schmidt.florian a6edec2b12 Polish/audit: fix bugs found by 3 subsystem audits
ci / build-test (push) Successful in 2m45s
Correctness/data:
- Fix demo cost double-count: reference importer no longer imports the Kosten Strom/Wasser
  columns for categories that are metered (only Heizung), so Wasser rollup is 70€ not 140€.
- Spurious-decrease guard: only a reset/swap in the window (prevReading, thisReading] explains
  a decrease — an old historical reset no longer permanently disables the guard.
- Gate swap auto-detection on MappingProfile.DetectCumulativeSwaps (flag was ignored).
- Prorate basePrice by bucket length (day/month/year); guard virtual expressions against NaN/Inf.

Concurrency/infra:
- Blazor: register a DbContextFactory; CostService/DashboardService and the read pages now use
  short-lived per-operation contexts (no shared circuit DbContext); guard Trends re-entrancy.
- /events: wrap event insert + consumption recompute in one transaction (atomic); 404 (not 500)
  on unknown meter.
- MQTT worker: subscribe to newly-added topics on each tick; move client cleanup into finally.
- Migrations: CREATE MATERIALIZED VIEW IF NOT EXISTS + if_not_exists on CAgg/compression/
  hypertable calls (re-run-safe after a mid-migration crash).
- HA worker: prune stale poll-schedule entries; export: null dangling ImportBatchIds on restore.

API/security:
- API fail-closed by default: with no keys and AllowAnonymousApi off, /api/v1 returns 401
  (protects /export and /import). New MeterVault:AllowAnonymousApi opt-in.
- Cap /readings batch at 5000; report ignored (unknown-meter) count; enums as strings in JSON.

+4 regression tests (guard window, API closed, /events 404, no demo double-count). 98 tests
green; Docker deploy re-verified healthy with the API fail-closed.

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 12:56:51 +02:00
schmidt.florian e223278771 M7: release polish — export/import, docs, easy docker push
- Easy docker push (MQTTower ergonomics + the image push it lacks): VERSION file →
  version-tag.yml (semver-guard auto-tag) → docker-publish.yml (buildx multi-arch → GHCR,
  registry centralized for one-line retarget to git.finalfactory.de) + ci.yml + build-and-push.ps1.
- Verified end to end: deploy/Dockerfile builds; docker compose stack (app + timescaledb) comes
  up healthy; /healthz and the dashboard respond in-container.
- JSON config export/import (ExportService) with id remapping on restore + GET /export, POST
  /import endpoints; round-trip test preserves meter→type, meter-scoped tariff, category links.
- README, HA/Tasmota/MQTT wiring guide (docs/wiring.md), Unraid template.
- CLAUDE.md updated to reflect the built codebase.
- i18n: locale-aware number/currency formatting (de-DE); full de UI string localization deferred.

95 tests green (56 Core + 39 integration).

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 12:25:03 +02:00
schmidt.florian 9abc2937c2 M6: REST API + API-key auth + OpenAPI
- Minimal API under /api/v1 (SDD §9): POST /readings (idempotent HA push), GET /meters,
  /energy-types, /consumption, /cost, /dashboard/summary, POST /events (records + recomputes),
  GET+POST /tariffs, GET /sources/status.
- IngestionService.IngestByMeterAsync for direct REST push (batch-safe upsert via Local cache).
- ApiKeyFilter: X-Api-Key enforced against configured keys (open only when none set).
- ReverseProxyTrust middleware: adopt X-Forwarded-User/Remote-User behind Authelia/Traefik.
- Swagger/OpenAPI (Swashbuckle) at /swagger.
- Tests: push rejected without key (401), accepted + persisted with key; meters + swagger live.

94 tests green (56 Core + 38 integration).

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 12:16:49 +02:00
schmidt.florian d5419729e5 M5: Blazor dashboard (MudBlazor + ApexCharts)
- MudBlazor theme (dark default) + responsive drawer/appbar layout + nav.
- DashboardService read model: KPIs with period-over-period deltas, category breakdown,
  "what cost more/less" difference view, monthly trends.
- Pages: Overview (KPI cards + DeltaChip + donut + difference table), Trends (range-select
  bar chart), Meters (list + source status), Import (load reference dataset + CSV dry-run
  preview), Admin (energy types, tariffs). Charts isolated into components to avoid the
  ApexCharts/MudBlazor Color/Format name clashes.
- ReferenceDataImporter: one-click load of all four sheets as a starter dataset (meters,
  tank, tariff history, category memberships) — bundled sample CSVs copied to app output.
- End-to-end render test: import creates meters + consumption; overview/meters/trends/
  import/admin pages all return 200 with KPI cards rendered.

92 tests green (56 Core + 36 integration).

Deferred to polish: dedicated PV & oil/consumable panels, meter-detail page, full admin
CRUD, prev-year trend overlay.

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 12:11:12 +02:00
schmidt.florian 6fc710d55a M4: aggregation + tariff-aware cost engine
- ContinuousAggregates migration: consumption_daily/monthly/yearly in Europe/Berlin
  buckets via migrationBuilder.Sql(..., suppressTransaction: true), with refresh policies
  (end_offset >= 1 bucket; current bucket covered by real-time aggregation).
- TariffResolver (Core): time-ranged price resolution, scope precedence meter > type > global.
- CostService: Dapper-aggregated monthly consumption × resolved unit price (+base, -feed-in),
  month-dominant pricing; category rollups over member meters + meterless manual costs.
- Tests: water cost reconciles to the sheet's Kosten column, Wasser category rollup (Dez=70€),
  monthly CAgg refresh matches base, TariffResolver unit tests.

91 tests green (56 Core + 35 integration).

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 11:52:53 +02:00
schmidt.florian 4b0cad67df M3: live ingestion (MQTT/Tasmota + Home Assistant)
- PayloadExtractor: dot-path value/time extraction (Tasmota ENERGY.Total, bare scalars).
- MqttTopicMatcher: standard +/# wildcard matching.
- IngestionService: scale/offset, idempotent upsert on (meter_id, time), and a spurious-
  decrease guard for monotonic registers (allowed only with a reset/swap event) + source
  last-seen status.
- MqttMessageRouter + MqttIngestionWorker (MQTTnet 5): per-endpoint persistent connections,
  topic subscription, graceful degradation; secrets resolved by env-var reference.
- Home Assistant: HaStateClient (REST /api/states parse) + HomeAssistantWorker polling on
  each source's interval. HA-via-MQTT also works through the MQTT path.
- Ingestion workers gated by MeterVault:EnableLiveIngestion (off in tests).

85 tests green (53 Core + 32 integration): Tasmota payload → reading verified end to end.

Follow-up (polish): HA WebSocket push (state_changed) as an alternative to REST poll;
source-topic index caching in the router.

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 11:45:20 +02:00
schmidt.florian 5977c81002 M2: German CSV importer + reconciliation of all 4 fixtures
- German-dialect scalar parsers in Core (GermanNumber/Money/Date, ValueCell):
  decimal comma, thousands dot, unit suffixes, € currency, both date shapes.
- Declarative MappingProfile + RowClassifier (skip summary/blank/all-zero rows) +
  CsvImporter (CsvHelper) staging readings/events/manual-costs, with auto swap
  detection on register decreases and month-end anchoring for interleaved oil dates.
- Four built-in ReferenceProfiles (Strom/Wasser/Heizöl/Kosten).
- ImportService: commit as revertible import_batch + wholesale consumption recompute
  per affected meter (NormalizationService/MeterConfigFactory), revert by batch.
- Reconciliation tests: all 4 CSVs match the sheet's own columns within tolerance
  (electricity 5 meters + Netz Einsparung, water swap→12, oil tank incl. deliveries +
  burner hours, cost category totals). Commit/revert round-trip verified on Timescale.

69 tests green (53 Core + 16 integration).

Known follow-up (polish): historical imports can contend with the 30-day compression
policy's background job; tests pause it. Consider retry-on-deadlock or deferred
compression for large historical imports in production.

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 11:35:54 +02:00
schmidt.florian 48a7f5a825 M0: scaffold solution, EF+Timescale schema, /healthz
Five-project Clean Architecture solution (Core/Infrastructure/App + Core.Tests/
Integration.Tests) on .NET 10 with central package management, snake_case EF mapping,
and shared build/style config.

- Full domain entity set + EF DbContext for the SDD §5.3 schema (singular table names).
- InitialSchema migration (relational) + TimescaleHypertables migration (raw SQL:
  create_hypertable + compression on reading, hypertable on consumption).
- App wiring: Serilog (actually wired, unlike MQTTower), DbContext, migrate-on-startup,
  /healthz. Serves plain HTTP behind a reverse proxy (no HTTPS redirect).
- deploy/Dockerfile (2-stage, ICU-capable) + docker-compose (app + timescaledb).
- Integration.Tests: shared TimescaleFixture (Testcontainers) — migrations, hypertables,
  compression policy, and /healthz all verified green (4/4).

Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
2026-07-13 11:02:21 +02:00