schmidt.florian
7e34aeccc8
ci / build-test (push) Successful in 1m24s
Adds a meter hierarchy and a flow view: a downstream meter is a *subsection*
of an upstream one (not an addition), so you can see where a main meter's flow
divides — e.g. official water → garden, pool, other; grid/battery → all → car.
- MeterLink (schema + migration AddMeterLinks): a directed from→to flow edge.
Multi-parent allowed (a merge, e.g. grid + solar → house); multi-child is a
split. Cascade-deletes with either endpoint; unique + distinct-endpoint checks.
- FlowService: per energy type + period, builds a Sankey graph — nodes = meters
sized by consumption; link value = downstream meter's consumption, split
proportionally across multiple upstreams; unaccounted remainder under a meter
becomes a synthetic "Other" node; depth via topological longest-path.
- SankeyChart.razor: hand-rolled inline-SVG Sankey (ApexCharts has no Sankey
type) — columns by depth, nodes stacked by value, bezier ribbons sized by flow,
left→right, theme-aware, HTML-encoded labels, tooltips. Built as a MarkupString
to sidestep Razor's <text> element clash.
- /energy/{id} page (one per energy type): KPIs (consumption + cost), the flow
Sankey, and the meter list. NavMenu now lists a link per energy type
(Electricity, Water, Gas, …) loaded from the DB.
- Meters admin: cycle-safe "Sub-meter of (upstream meters)" multi-select
(descendants excluded to prevent cycles); reconciles meter_link rows on save.
- Reference data seeds a demo chain (Haus → Auto) so electricity flow shows
Haus dividing into Auto + Other.
Tests: FlowServiceTests (single-parent remainder; two-parent proportional
split); render test now asserts the flow chain + covers /energy/{id}. 69 Core +
47 Integration = 116 green. Live-verified: Haus 95,450 kWh → Auto 51,909 +
Other 43,541 (flow conserved), all 5 energy-type pages render.
Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
100 lines
11 KiB
Markdown
100 lines
11 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
## What this repo is
|
||
|
||
MeterVault is a self-hosted, local-first energy & utility metering platform: it ingests meter data from Home Assistant, Tasmota and MQTT on a schedule, stores every reading timestamped and immutable, normalizes it into consumption, and turns it into cost dashboards. Energy types (electricity, water, heating oil, gas, …) and meters are **user-defined, never hardcoded**.
|
||
|
||
**Status: implemented (M0–M7) + SDD §8 panels.** The full solution is built and green — five projects, ~108 tests, working Docker deploy. `docs/SDD.md` remains the design reference; the milestone map (§12) matches the git history (M0…M7 commits). The dedicated **PV/Solar** (`/solar`), **Oil/consumable** (`/consumables`) and **meter-detail** (`/meters/{id}`) views (SDD §8.4–§8.6) are implemented as read models in `Infrastructure/Dashboard` (`SolarService`, `ConsumableService`, `MeterDetailService`) — PV meters are found by `Mode == GenerationCounter` and grid/load meters by a `role` tag in `Meter.Meta` (`MeterRoles`/`MeterMeta`), so nothing is hardcoded by name. **Admin write-CRUD** (SDD §8.7) is implemented as MudBlazor inline-dialog pages: energy types, meters (+ recompute on mode/baseline change), a meter's ingest sources (meter-detail Sources tab), tariffs, cost categories + members, and connectors (`ingestion_endpoint`, secrets by env-var reference only). `/admin/settings` is a read-only effective-config view (settings are env-driven and reproducible, not DB-stored). **Home Assistant reading** is configured here: an HA connector (`BaseUrl` + `TokenEnv`) + an HA source (entity id) drives `HomeAssistantWorker`'s REST poll; `HaConnectionTester` powers the connector "Test connection" button. **Meter topology & flow**: `MeterLink` (a directed `from→to` edge; a downstream meter is a *subsection* of an upstream one, multi-parent allowed) drives a per-energy-type page `/energy/{id}` with a hand-rolled SVG **Sankey** (`SankeyChart.razor`, since ApexCharts has no Sankey type) computed by `FlowService` (link value = downstream consumption, split proportionally across multiple parents; unaccounted remainder → an "Other" node). Upstream meters are wired cycle-safely in the meter editor; the nav lists a link per energy type. Remaining refinements (HA WebSocket *push* — REST poll works today; commit-arbitrary-CSV-from-UI needs a meter-mapping wizard; full de-DE UI-string localization) are noted at their commits. Set `MeterVault__SeedReferenceData=true` (compose: `METERVAULT_SEED=true`) for a one-command populated demo.
|
||
|
||
## Source of truth
|
||
|
||
`docs/SDD.md` is the authoritative spec and build brief — read it before implementing anything. Key protocol from §0 that governs all work here:
|
||
|
||
- **Build strictly in milestone order (§12, M0→M7).** Each milestone is independently runnable and testable; do not start Mn+1 until Mn's tests pass.
|
||
- **The four CSVs in `sampledata/` are golden fixtures.** Every parsing / consumption / cost rule must reconcile against them (§13). **If a computed number disagrees with the spreadsheet, the spreadsheet wins** unless the discrepancy is a deliberately documented correctness fix.
|
||
- When a design decision is ambiguous, check §14 (open questions): if listed, take the stated default and flag it; if not listed, ask before guessing.
|
||
- Keep the **domain layer free of infrastructure concerns** (the domain model and DB schema are UI-agnostic by design).
|
||
|
||
## Committed tech stack (do not re-litigate; see SDD §4.1)
|
||
|
||
.NET (current LTS — .NET 10, .NET 8 acceptable), C# · ASP.NET Core + **Blazor Server** · **MudBlazor** components · **ApexCharts** (Blazor-ApexCharts) · **MQTTnet** · **PostgreSQL + TimescaleDB** · **EF Core (Npgsql)** for schema/CRUD + **Dapper** for hot-path time-series reads · `BackgroundService` hosted services for ingestion/aggregation · **xUnit + Testcontainers** (Timescale image) · Docker Compose + GHCR.
|
||
|
||
## Project layout
|
||
|
||
```
|
||
/src/Core domain entities + enums; pure Normalization engine (mode strategies,
|
||
expression evaluator); Parsing (German dialect); Costing (TariffResolver)
|
||
/src/Infrastructure MeterVaultDbContext + migrations (relational + raw-SQL Timescale);
|
||
Import (CsvImporter, profiles, ImportService), Ingestion (MQTT/HA workers,
|
||
IngestionService), Normalization service, Costing/Dashboard/Backup services
|
||
/src/App ASP.NET Core host: Blazor Server UI (Components/), REST API (Api/), hosted
|
||
workers, Program.cs (Serilog, migrate+seed on startup, /healthz)
|
||
/tests/Core.Tests unit (no Docker): parsers, normalizers, swap→12, tariff resolver
|
||
/tests/Integration.Tests Testcontainers (Timescale): reconciliation vs the 4 fixtures,
|
||
import commit/revert, ingestion, cost, CAgg refresh, API, export, render
|
||
/deploy Dockerfile, docker-compose.yml (app + timescaledb), build-and-push.ps1, unraid-template.xml
|
||
```
|
||
|
||
Central package versions live in `Directory.Packages.props`; shared build/style in
|
||
`Directory.Build.props` + `.editorconfig`. Snake_case table/column mapping via
|
||
`UseSnakeCaseNamingConvention`. EF migrations are exempt from code-style enforcement (see `.editorconfig`).
|
||
|
||
## Commands
|
||
|
||
```powershell
|
||
dotnet build # build the solution
|
||
dotnet test # all tests (Integration.Tests needs Docker for Testcontainers)
|
||
dotnet test tests/Core.Tests # unit tests only (no Docker needed)
|
||
dotnet test tests/Integration.Tests --filter "FullyQualifiedName~Reconciliation" # one class/area
|
||
dotnet ef migrations add <Name> -p src/Infrastructure -s src/App -o Persistence/Migrations
|
||
dotnet run --project src/App # run app + workers locally (needs a Timescale DB)
|
||
docker compose -f deploy/docker-compose.yml up # app + TimescaleDB together
|
||
```
|
||
|
||
**Timescale-in-EF gotchas** (already handled — follow the pattern): hypertable/CAgg DDL lives in
|
||
raw-SQL migrations; continuous-aggregate creation + policies use `migrationBuilder.Sql(..., suppressTransaction: true)`, one statement each; CAgg policy `end_offset` must be ≥ one bucket. Tests
|
||
pause the compression job (historical fixture data would otherwise deadlock imports).
|
||
|
||
## Core architecture (the part that spans multiple files)
|
||
|
||
**Data pipeline — one direction, layered (SDD §4.2, §5, §7):**
|
||
|
||
```
|
||
sources (Tasmota/HA/MQTT/manual/CSV)
|
||
→ Ingestion workers write raw `reading` rows (immutable audit truth)
|
||
→ Normalization derives append-only `consumption` (deltas in base unit)
|
||
→ TimescaleDB continuous aggregates roll consumption to hourly/daily/monthly/yearly
|
||
→ Cost engine joins aggregates with time-ranged `tariff`
|
||
→ Blazor dashboard + REST API read aggregates + cost views
|
||
```
|
||
|
||
**Invariants that shape everything:**
|
||
|
||
- **Raw `reading` is immutable audit truth.** Everything derived (consumption, cost, balances, forecasts) is computed on top and must be reproducible. Never mutate readings to fix a derived number.
|
||
- **Dashboards and charts read aggregates only — never scan `reading`.** This is what makes 1000 meters × 50 years feasible (§5.5). Raw is kept for a bounded window (default 3y); `consumption` + aggregates are the long-term source of truth.
|
||
- **`meter.mode` (measurement mode) is the central abstraction** for how raw readings become consumption (SDD §5.2): `cumulative_counter`, `generation_counter`, `runtime_counter` (Δhours × rate), `consumable_balance` (tank: deliveries − usage + forecast), `direct_delta`, `instant_rate`, `virtual` (expression over other meters). New ingestion/normalization logic dispatches on mode.
|
||
- **Nothing domain-specific is hardcoded.** Energy types are data. Cost **categories are decoupled from energy types** (Heizung may be oil today, heat-pump tomorrow). PV self-consumption/savings/net are **virtual meters** with user-defined expressions, not special-cased code. Tariffs are time-ranged (price history), scoped global / per-type / per-meter.
|
||
|
||
**Timescale vs EF split (SDD §5.3):** EF Core migrations own the relational tables. **Timescale-specific DDL — `create_hypertable`, compression policies, continuous aggregates, retention — is not expressible via EF's model builder and must live in raw-SQL migrations.** `reading` and `consumption` are hypertables.
|
||
|
||
**Time & DST (SDD §10):** store UTC everywhere; bucket and display in the instance timezone (default `Europe/Berlin`). "Daily cost" boundaries are local-midnight — use `time_bucket(..., 'Europe/Berlin')`.
|
||
|
||
**Secrets (SDD §6.4):** broker/HA tokens are **never** stored in DB plaintext. `ingestion_endpoint.config` holds a *reference* (env var name / Docker secret path) resolved at runtime.
|
||
|
||
## Reference-data behaviours the code must reproduce (from `sampledata/`)
|
||
|
||
These CSVs are the German-dialect *Energiebilanz* spreadsheet export and define the minimum feature bar (SDD §2, Appendix A). When writing the importer or normalization, honour:
|
||
|
||
- **German number dialect:** decimal comma (`180,8244706`), thousands dot (`2.940,19`), trailing-`€` currency (`120,00 €`), **unit suffixes on values** (`411kWh`, `49` cm, `2287` L) — strip and validate.
|
||
- **Two date formats:** `Monat YYYY` (German month names, monthly tables) and `DD.MM.YYYY` (event rows).
|
||
- **Skip inline summary rows** (`Total`, `Heute`, `Seitbeginn Tage`, `Seit YYYY`) and **all-zero future placeholder rows** (e.g. Dec 2026) — do not ingest them. **Negatives are valid** (savings, grid balance).
|
||
- **Water register swaps mid-series** (…861 → 2 → 15): consumption must stay continuous across the boundary via a `meter_swap` event.
|
||
- **Electricity has 5 meters** (Haus, Netz, Auto, Solar 1, Solar 2) plus derived columns. Verified relations to reproduce: `Netz Einsparung = Haus − Netz`, `Ersparnis = Netz Einsparung × €/kWh`, `Kosten = Verbrauchskosten − Ersparnis` — but implement these as **user-definable virtual-meter expressions**, not hardcoded formulas.
|
||
- **Heating oil is the versatility stress test:** a consumable/tank model where consumption is derivable two ways — tank-level Δ, or burner runtime × rate (rate `fixed` from nozzle spec, or `empirical` = Δlevel ÷ Δhours). Early rows (1997–2004) carry deliveries only (no burner hours yet). Includes cm→litre dipstick calibration and forecast-to-empty.
|
||
|
||
## Git
|
||
|
||
Remote `origin` is `https://git.finalfactory.de/FinalFactory/MeterVault.git` (Gitea; default branch `master`). CI/release is **Gitea Actions** under `.gitea/workflows/` — edit `VERSION` on `master` to tag + publish the image to the Gitea container registry.
|