MeterVault/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

What this repo is

MeterVault is a self-hosted, local-first energy & utility metering platform: it ingests meter data from Home Assistant, Tasmota and MQTT on a schedule, stores every reading timestamped and immutable, normalizes it into consumption, and turns it into cost dashboards. Energy types (electricity, water, heating oil, gas, …) and meters are user-defined, never hardcoded.

Status: implemented (M0–M7) + SDD §8 panels. The full solution is built and green — five projects, ~108 tests, working Docker deploy. docs/SDD.md remains the design reference; the milestone map (§12) matches the git history (M0…M7 commits). The dedicated PV/Solar (/solar), Oil/consumable (/consumables) and meter-detail (/meters/{id}) views (SDD §8.4–§8.6) are implemented as read models in Infrastructure/Dashboard (SolarService, ConsumableService, MeterDetailService) — PV meters are found by Mode == GenerationCounter and grid/load meters by a role tag in Meter.Meta (MeterRoles/MeterMeta), so nothing is hardcoded by name. Admin write-CRUD (SDD §8.7) is implemented as MudBlazor inline-dialog pages: energy types, meters (+ recompute on mode/baseline change), a meter's ingest sources (meter-detail Sources tab), tariffs, cost categories + members, and connectors (ingestion_endpoint, secrets by env-var reference only). Manual readings are entered from the meter-detail Readings tab ("Add reading"): a touch-first dialog prefilled with the meter's last register value and the current local time, with an on-screen keypad for phone entry at the meter, a live parsed-value + delta-since-last readout, and the decrease guard surfaced before saving. It goes through IngestionService.IngestByMeterAsync(quality: Manual), so it is stamped ReadingQuality.Manual and renormalizes inline like any other ingest — the layout of that dialog deliberately reserves fixed space for its verdict line, because anything that reflows moves the keys out from under the user's thumb mid-entry. /admin/settings is a read-only effective-config view (settings are env-driven and reproducible, not DB-stored). Home Assistant reading is configured here: an HA connector (BaseUrl + TokenEnv) + an HA source (entity id) drives HomeAssistantWorker's REST poll, or — with the connector's WebSocket push toggle (HaEndpointConfig.UseWebSocket) — HomeAssistantWebSocketWorker holds a persistent state_changed subscription and ingests in real time (the poll worker skips WS endpoints, so each is served once; HaWebSocketProtocol is the pure, unit-tested handshake/parse logic). HaConnectionTester powers the connector "Test connection" button. Meter topology & flow: MeterLink (a directed from→to edge; a downstream meter is a subsection of an upstream one, multi-parent allowed) drives a per-energy-type page /energy/{id} with a hand-rolled SVG Sankey (SankeyChart.razor, since ApexCharts has no Sankey type) computed by FlowService (link value = downstream consumption, split proportionally across multiple parents; unaccounted remainder → an "Other" node). Upstream meters are wired cycle-safely in the meter editor; the nav lists a link per energy type. CSV mapping wizard (/import/wizard): upload an arbitrary CSV, map columns → meters/roles, dry-run preview, then commit as a revertible import_batch (the /import page lists batches with one-click revert). The instant_rate mode is normalized (InstantRateNormalizer: rate integrated over time, trapezoidal). Remaining refinement: full de-DE UI-string localization (data parsing is already de-DE) — noted at its commit. Set MeterVault__SeedReferenceData=true (compose: METERVAULT_SEED=true) for a one-command populated demo.

Source of truth

docs/SDD.md is the authoritative spec and build brief — read it before implementing anything. Key protocol from §0 that governs all work here:

• Build strictly in milestone order (§12, M0→M7). Each milestone is independently runnable and testable; do not start Mn+1 until Mn's tests pass.
• The four CSVs in sampledata/ are golden fixtures. Every parsing / consumption / cost rule must reconcile against them (§13). If a computed number disagrees with the spreadsheet, the spreadsheet wins unless the discrepancy is a deliberately documented correctness fix.
• When a design decision is ambiguous, check §14 (open questions): if listed, take the stated default and flag it; if not listed, ask before guessing.
• Keep the domain layer free of infrastructure concerns (the domain model and DB schema are UI-agnostic by design).

Committed tech stack (do not re-litigate; see SDD §4.1)

.NET (current LTS — .NET 10, .NET 8 acceptable), C# · ASP.NET Core + Blazor Server · MudBlazor components · ApexCharts (Blazor-ApexCharts) · MQTTnet · PostgreSQL + TimescaleDB · EF Core (Npgsql) for schema/CRUD + Dapper for hot-path time-series reads · BackgroundService hosted services for ingestion/aggregation · xUnit + Testcontainers (Timescale image) · Docker Compose + GHCR.

Project layout

/src/Core            domain entities + enums; pure Normalization engine (mode strategies,
                     expression evaluator); Parsing (German dialect); Costing (TariffResolver)
/src/Infrastructure  MeterVaultDbContext + migrations (relational + raw-SQL Timescale);
                     Import (CsvImporter, profiles, ImportService), Ingestion (MQTT/HA workers,
                     IngestionService), Normalization service, Costing/Dashboard/Backup services
/src/App             ASP.NET Core host: Blazor Server UI (Components/), REST API (Api/), hosted
                     workers, Program.cs (Serilog, migrate+seed on startup, /healthz)
/tests/Core.Tests            unit (no Docker): parsers, normalizers, swap→12, tariff resolver
/tests/Integration.Tests     Testcontainers (Timescale): reconciliation vs the 4 fixtures,
                             import commit/revert, ingestion, cost, CAgg refresh, API, export, render
/deploy              Dockerfile, docker-compose.yml (app + timescaledb), build-and-push.ps1, unraid-template.xml

Central package versions live in Directory.Packages.props; shared build/style in Directory.Build.props + .editorconfig. Snake_case table/column mapping via UseSnakeCaseNamingConvention. EF migrations are exempt from code-style enforcement (see .editorconfig).

Commands

dotnet build                                   # build the solution
dotnet test                                    # all tests (Integration.Tests needs Docker for Testcontainers)
dotnet test tests/Core.Tests                   # unit tests only (no Docker needed)
dotnet test tests/Integration.Tests --filter "FullyQualifiedName~Reconciliation"  # one class/area
dotnet ef migrations add <Name> -p src/Infrastructure -s src/App -o Persistence/Migrations
dotnet run --project src/App                   # run app + workers locally (needs a Timescale DB)
docker compose -f deploy/docker-compose.yml up # app + TimescaleDB together

Timescale-in-EF gotchas (already handled — follow the pattern): hypertable/CAgg DDL lives in raw-SQL migrations; continuous-aggregate creation + policies use migrationBuilder.Sql(..., suppressTransaction: true), one statement each; CAgg policy end_offset must be ≥ one bucket. Tests pause the compression job (historical fixture data would otherwise deadlock imports).

Core architecture (the part that spans multiple files)

Data pipeline — one direction, layered (SDD §4.2, §5, §7):

sources (Tasmota/HA/MQTT/manual/CSV)
  → Ingestion workers write raw `reading` rows (immutable audit truth)
  → Normalization derives append-only `consumption` (deltas in base unit)
  → TimescaleDB continuous aggregates roll consumption to hourly/daily/monthly/yearly
  → Cost engine joins aggregates with time-ranged `tariff`
  → Blazor dashboard + REST API read aggregates + cost views

Invariants that shape everything:

• Raw reading is immutable audit truth. Everything derived (consumption, cost, balances, forecasts) is computed on top and must be reproducible. Never mutate readings to fix a derived number. Live ingestion recomputes the meter inline (IngestionService.RenormalizeAsync) — without it, polled readings never become consumption.
• Long gaps are apportioned, short ones are not (GapAttribution, SDD §7.1). An interval containing ≥2 whole calendar months is split across those months, proportional to elapsed time, marked Estimated. A monthly series contains exactly one and is untouched — that's what keeps the golden fixtures reconciling. GapSplittingIsInertOnFixturesTests asserts the rule declines to fire on the reference data, so this can't silently drift.
• Dashboards and charts read aggregates only — never scan reading. This is what makes 1000 meters × 50 years feasible (§5.5). Raw is kept for a bounded window (default 3y); consumption + aggregates are the long-term source of truth.
• meter.mode (measurement mode) is the central abstraction for how raw readings become consumption (SDD §5.2): cumulative_counter, generation_counter, runtime_counter (Δhours × rate), consumable_balance (tank: deliveries − usage + forecast), direct_delta, instant_rate, virtual (expression over other meters). New ingestion/normalization logic dispatches on mode.
• Nothing domain-specific is hardcoded. Energy types are data. Cost categories are decoupled from energy types (Heizung may be oil today, heat-pump tomorrow). PV self-consumption/savings/net are virtual meters with user-defined expressions, not special-cased code. Tariffs are time-ranged (price history), scoped global / per-type / per-meter.

Timescale vs EF split (SDD §5.3): EF Core migrations own the relational tables. Timescale-specific DDL — create_hypertable, compression policies, continuous aggregates, retention — is not expressible via EF's model builder and must live in raw-SQL migrations. reading and consumption are hypertables.

Time & DST (SDD §10): store UTC everywhere; bucket and display in the instance timezone (default Europe/Berlin). "Daily cost" boundaries are local-midnight — use time_bucket(..., 'Europe/Berlin').

In-app update (UpdateRunner): the dashboard shows a banner when a newer tag exists (UpdateCheckService, cached, never blocks a render). Triggering an update is off by default; MeterVault__AllowInAppUpdate is the only gate — no API key, by explicit owner decision. With it on, anything that can reach the app can trigger a rebuild+restart as root (realistically a DoS, since the build comes from the owner's own repo; RCE if that repo is compromised). The REST endpoint additionally requires an X-MeterVault-Update header — a CSRF guard, not auth, so a foreign page cannot drive it via a LAN browser. Launches detached via systemd-run because the update restarts the service. Treat any change here as security-critical; UpdateRunnerTests pins that the flag defaults off and that API keys alone don't enable it.

Secrets (SDD §6.4): broker/HA tokens are never stored in DB plaintext. Two forms, chosen per connector in the admin UI: a reference (token_env/password_env naming an env var or Docker secret path) resolved at runtime, or encrypted at rest (token_enc/password_enc) via SecretProtector over the ASP.NET Core data-protection key ring. Exactly one survives a save; EndpointSecret.Resolve is the single resolution path (encrypted wins). The key ring lives outside the app directory (MeterVault__DataProtectionKeyPath, default /var/lib/metervault/keys) because the LXC updater republishes /opt/metervault. ExportService drops *_enc values — they are bound to the originating key ring.

Reference-data behaviours the code must reproduce (from sampledata/)

These CSVs are the German-dialect Energiebilanz spreadsheet export and define the minimum feature bar (SDD §2, Appendix A). When writing the importer or normalization, honour:

• German number dialect: decimal comma (180,8244706), thousands dot (2.940,19), trailing-€ currency (120,00 €), unit suffixes on values (411kWh, 49 cm, 2287 L) — strip and validate.
• Two date formats: Monat YYYY (German month names, monthly tables) and DD.MM.YYYY (event rows).
• Skip inline summary rows (Total, Heute, Seitbeginn Tage, Seit YYYY) and all-zero future placeholder rows (e.g. Dec 2026) — do not ingest them. Negatives are valid (savings, grid balance).
• Water register swaps mid-series (…861 → 2 → 15): consumption must stay continuous across the boundary via a meter_swap event.
• Electricity has 5 meters (Haus, Netz, Auto, Solar 1, Solar 2) plus derived columns. Verified relations to reproduce: Netz Einsparung = Haus − Netz, Ersparnis = Netz Einsparung × €/kWh, Kosten = Verbrauchskosten − Ersparnis — but implement these as user-definable virtual-meter expressions, not hardcoded formulas.
• Heating oil is the versatility stress test: a consumable/tank model where consumption is derivable two ways — tank-level Δ, or burner runtime × rate (rate fixed from nozzle spec, or empirical = Δlevel ÷ Δhours). Early rows (1997–2004) carry deliveries only (no burner hours yet). Includes cm→litre dipstick calibration and forecast-to-empty.

Git

Remote origin is https://git.finalfactory.de/FinalFactory/MeterVault.git (Gitea; default branch master). CI/release is Gitea Actions under .gitea/workflows/ — edit VERSION on master to tag + publish the image to the Gitea container registry.