Reference-only secrets (SDD §6.4) meant adding a connector required editing a
file on the server and restarting the service. In practice that leads to the
token being pasted into the env-var *name* field, which fails with "environment
variable '<token>' is not set" and gives no hint what went wrong.
Add a second storage form, chosen per connector: type the secret in and it is
encrypted via ASP.NET Core data protection before it is stored. The env-var
reference stays as an equal alternative — this widens the choice rather than
replacing it. Exactly one form survives a save, so a stale secret cannot linger
and silently win; EndpointSecret.Resolve is the single resolution path.
The guarantee that matters is preserved: no plaintext in the database, so
pg_dump and JSON exports carry nothing usable. The trust boundary is stated
plainly in §6.4 — the key ring is on disk, so this protects against leaked
database content, not an attacker who already has the host, which is the same
boundary an env var has.
Details worth noting:
- Key ring defaults to /var/lib/metervault/keys, outside the app directory,
because the LXC updater republishes /opt/metervault on every update. Docker
gets a named volume. Overridable via MeterVault__DataProtectionKeyPath.
- Undecryptable ciphertext (key ring lost) falls back rather than throwing: an
ingestion worker on a timer should degrade, not crash.
- The stored secret is never sent to the browser; a blank field means
"unchanged", not "cleared".
- MQTT usernames are stored as-is — §6.4 covers tokens and passwords, and
encrypting a username would only blank the field on every edit.
- ExportService drops *_enc values: bound to the originating key ring, so
useless where an export would be restored. Expect to re-enter after a restore.
- HaConnectionTester now takes a resolved token, so the admin UI can test a
token that has been typed but not yet saved.
SDD §6.4 and §9 updated to describe both forms rather than contradict the code.
Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
MqttMessageRouter matched purely on topic with no endpoint predicate, and
RouteAsync was not even passed an endpoint id. Topic filters routinely overlap
between brokers — every Tasmota install publishes tele/+/SENSOR — so with two
brokers a message on A was ingested by a source bound to B. HA enforced the
binding on both workers; MQTT enforced it only at subscribe time.
Pass the endpoint id through: MQTTnet's event args carry the topic but not the
delivering connection, so CreateClient captures the id in the handler closure.
ResolveTopicsAsync drops its `|| EndpointId == null` clause to match, since an
unbound source is no longer routed and subscribing its topic everywhere would
only invite traffic nothing consumes.
That last part would silently kill unbound sources that work today, so a data
migration binds them to the single broker when exactly one exists — the case
where old and new behaviour coincide. Two or more brokers is left alone: the
old behaviour was already ambiguous and a guess could route a meter's data to
the wrong broker. HA sources are excluded; they have always required an
endpoint, so binding them would activate ingestion never previously running.
Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
HomeAssistantWebSocketWorker holds a persistent state_changed subscription per HA endpoint that opts in via the connector's WebSocket toggle (HaEndpointConfig.UseWebSocket): auth handshake, subscribe, ingest in real time, capped-backoff reconnect. The REST poll worker skips WS endpoints so each is served once. HaWebSocketProtocol holds the pure handshake/parse logic. Verified by 11 protocol unit tests + a live integration test against an in-process fake HA server. CLAUDE.md updated.
Claude-Session: https://claude.ai/code/session_01Kib2MniVFbD95fkgLgBBnB
The instant_rate mode existed in the enum but had no normalizer, so normalizing a power/flow sensor threw NotSupportedException. InstantRateNormalizer integrates the rate over time (trapezoidal, attributed to each interval's end reading); a per-hour rate in the meter's unit yields the consumption unit (kW->kWh, L/h->L). Registered in the engine; meter editor shows a mode hint. +4 Core tests.
Claude-Session: https://claude.ai/code/session_01Kib2MniVFbD95fkgLgBBnB
Adds the "sum meter" the user asked for: a meter that doesn't physically exist
but represents the sum of other meters in the flow view.
- FlowService: a Virtual-mode meter has no readings of its own; its flow value is
the sum of its upstream meters, resolved in topological order (so "Summe Solar"
= Solar 1 + Solar 2, and Grid + Summe Solar → House with the remainder as
"Other" = export / battery / inverter losses).
- Meters editor: a hint when Mode = Virtual explaining the sum-meter behaviour.
- Reference data seeds the full demo chain: Solar 1 + Solar 2 → Summe Solar;
Netz + Summe Solar → Haus → Auto + Other — so /energy/1 shows a multi-level
Sankey out of the box.
- Fix: SankeyChart Unit was passed as the literal string "_graph.Unit" (missing
@) so node labels read "_graph.Unit" instead of "kWh". Caught by screenshotting
the live page.
Test: Virtual_sum_meter_aggregates_its_upstreams. 69 Core + 50 Integration =
119 green. Live-verified with a browser screenshot of the multi-level flow.
Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
A meter's flow value is now its throughput — consumption OR generation output —
so a generation meter (solar) acts as a source that can feed downstream meters.
Setting a load meter's upstream to {grid, solar} now splits its consumption
across both proportionally, and the remainder under the sources (grid + solar −
load) surfaces as "Other" = export + battery/inverter losses. Negative values
(savings/balance virtuals) are clamped to 0 (a ribbon can't be negative). The
per-type KPI is relabelled "Top-level throughput" since it now spans generation.
Test: Generation_meter_counts_as_source (grid 75 + solar-gen 30 → house 40 →
28.57/11.43 split, 65 remainder). 69 Core + 48 Integration = 117 green.
Live-verified: electricity flow now shows Solar 1/2 as source nodes.
Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
Adds a meter hierarchy and a flow view: a downstream meter is a *subsection*
of an upstream one (not an addition), so you can see where a main meter's flow
divides — e.g. official water → garden, pool, other; grid/battery → all → car.
- MeterLink (schema + migration AddMeterLinks): a directed from→to flow edge.
Multi-parent allowed (a merge, e.g. grid + solar → house); multi-child is a
split. Cascade-deletes with either endpoint; unique + distinct-endpoint checks.
- FlowService: per energy type + period, builds a Sankey graph — nodes = meters
sized by consumption; link value = downstream meter's consumption, split
proportionally across multiple upstreams; unaccounted remainder under a meter
becomes a synthetic "Other" node; depth via topological longest-path.
- SankeyChart.razor: hand-rolled inline-SVG Sankey (ApexCharts has no Sankey
type) — columns by depth, nodes stacked by value, bezier ribbons sized by flow,
left→right, theme-aware, HTML-encoded labels, tooltips. Built as a MarkupString
to sidestep Razor's <text> element clash.
- /energy/{id} page (one per energy type): KPIs (consumption + cost), the flow
Sankey, and the meter list. NavMenu now lists a link per energy type
(Electricity, Water, Gas, …) loaded from the DB.
- Meters admin: cycle-safe "Sub-meter of (upstream meters)" multi-select
(descendants excluded to prevent cycles); reconciles meter_link rows on save.
- Reference data seeds a demo chain (Haus → Auto) so electricity flow shows
Haus dividing into Auto + Other.
Tests: FlowServiceTests (single-parent remainder; two-parent proportional
split); render test now asserts the flow chain + covers /energy/{id}. 69 Core +
47 Integration = 116 green. Live-verified: Haus 95,450 kWh → Auto 51,909 +
Other 43,541 (flow conserved), all 5 energy-type pages render.
Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
Three requested phases.
1) Admin section (SDD §8.7) — MudBlazor inline-dialog CRUD, consistent pattern,
delete guards, snackbar feedback, shared Confirm helper:
- Energy types: create/edit/delete (blocks delete when meters reference it).
- Meters: create/edit/delete; recomputes consumption when mode/baseline
changes (NormalizationService over a fresh factory context, in a tx);
delete cascades data (consumption+readings are Restrict → removed first).
- A meter's ingest sources: manage on the meter-detail Sources tab
(add/edit/delete MQTT/Tasmota/HA sources with typed config).
- Tariffs: full CRUD (scope/component/value/validity).
- Cost categories: CRUD + member management (meter or energy-type members).
- Connectors: ingestion_endpoint CRUD (MQTT broker + Home Assistant);
secrets referenced by env-var name only, never stored.
- Settings: read-only effective-config view (settings are env-driven and
reproducible, so an editable form would change nothing — kept honest).
PV role is now editable on meters (MeterMeta.SetRole can clear a role).
2) Read Home Assistant — extracted a shared public HaEndpointConfig (was a
private record in the worker), added HaConnectionTester (powers the connector
"Test connection": checks base URL + env-resolved token, optionally reads one
entity). Configuring an HA connector + an HA source on a meter drives the
existing REST-poll worker end to end. (WebSocket push stays a future
optimization; REST poll already reads HA.)
3) Wiring/placeholder audit — swept every OnClick/Href: all handlers are real,
all internal links resolve to real routes, no TODO/stub/placeholder code.
Fixed one genuine gap: MainLayout had no drawer toggle, so the nav was
unreachable on narrow screens — added a hamburger button.
Tests: +6 (MeterMeta.SetRole role-removal; HaConnectionTester fail-closed
guard branches with a throwing HttpClientFactory proving no network on bad
config); render test now covers all admin routes. 69 Core + 45 Integration =
114 green. Live-verified in Docker: all admin pages 200, drawer toggle present,
Settings shows real effective config.
Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
Complete the SDD §8 dashboard views that were deferred at the M5 boundary,
and fix a shipping bug that left the Docker demo empty.
Bug: "Load reference data" created meters/tank/tariffs but imported zero
readings in Docker. Root cause: sampledata/ was excluded by .dockerignore and
never copied into the build stage, so the App csproj's linked Content glob
resolved to nothing at publish time; ReferenceDataImporter then silently
skipped the missing CSVs after already writing its marker meter, leaving the
DB permanently "loaded" but empty.
- .dockerignore: stop excluding sampledata/
- Dockerfile: COPY sampledata/ into the build stage
- ReferenceDataImporter: fail-fast (validate CSVs exist before the marker
meter) and throw instead of silently skipping a missing file
- Program.cs + MeterVaultOptions: opt-in MeterVault__SeedReferenceData
(compose METERVAULT_SEED=true) for a one-command populated demo
New SDD §8 panels (read models in Infrastructure/Dashboard, Blazor pages):
- §8.4 Solar/PV (/solar): generation from GenerationCounter meters;
self-consumption / autarky % / self-consumption % / savings derived from
meters tagged total_load & grid_import via Meter.Meta role config
(MeterRoles/MeterMeta) — nothing hardcoded by name.
- §8.5 Oil/consumable (/consumables): tank level (cm→L calibrated), fill
gauge, deliveries log, burner runtime, effective L/h (fixed/empirical),
forecast-to-empty, tariff cost, monthly series.
- §8.6 Meter detail (/meters/{id}): raw readings, normalized consumption,
source status, tariff timeline, events, measured-vs-estimated markers.
- Reusable SeriesChart component; nav links; Meters list rows link to detail.
Tests: MeterMetaTests (Core, +10); DashboardRenderTests extended to assert the
three panel services compute real figures and the new routes render (108 total,
all green). Live-verified in Docker: seed imports 302 readings / 347 consumption
rows; panels render (generation 16,481 kWh, oil 3,967 L) cross-checking the DB.
Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
Correctness/data:
- Fix demo cost double-count: reference importer no longer imports the Kosten Strom/Wasser
columns for categories that are metered (only Heizung), so Wasser rollup is 70€ not 140€.
- Spurious-decrease guard: only a reset/swap in the window (prevReading, thisReading] explains
a decrease — an old historical reset no longer permanently disables the guard.
- Gate swap auto-detection on MappingProfile.DetectCumulativeSwaps (flag was ignored).
- Prorate basePrice by bucket length (day/month/year); guard virtual expressions against NaN/Inf.
Concurrency/infra:
- Blazor: register a DbContextFactory; CostService/DashboardService and the read pages now use
short-lived per-operation contexts (no shared circuit DbContext); guard Trends re-entrancy.
- /events: wrap event insert + consumption recompute in one transaction (atomic); 404 (not 500)
on unknown meter.
- MQTT worker: subscribe to newly-added topics on each tick; move client cleanup into finally.
- Migrations: CREATE MATERIALIZED VIEW IF NOT EXISTS + if_not_exists on CAgg/compression/
hypertable calls (re-run-safe after a mid-migration crash).
- HA worker: prune stale poll-schedule entries; export: null dangling ImportBatchIds on restore.
API/security:
- API fail-closed by default: with no keys and AllowAnonymousApi off, /api/v1 returns 401
(protects /export and /import). New MeterVault:AllowAnonymousApi opt-in.
- Cap /readings batch at 5000; report ignored (unknown-meter) count; enums as strings in JSON.
+4 regression tests (guard window, API closed, /events 404, no demo double-count). 98 tests
green; Docker deploy re-verified healthy with the API fail-closed.
Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
- Minimal API under /api/v1 (SDD §9): POST /readings (idempotent HA push), GET /meters,
/energy-types, /consumption, /cost, /dashboard/summary, POST /events (records + recomputes),
GET+POST /tariffs, GET /sources/status.
- IngestionService.IngestByMeterAsync for direct REST push (batch-safe upsert via Local cache).
- ApiKeyFilter: X-Api-Key enforced against configured keys (open only when none set).
- ReverseProxyTrust middleware: adopt X-Forwarded-User/Remote-User behind Authelia/Traefik.
- Swagger/OpenAPI (Swashbuckle) at /swagger.
- Tests: push rejected without key (401), accepted + persisted with key; meters + swagger live.
94 tests green (56 Core + 38 integration).
Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
- PayloadExtractor: dot-path value/time extraction (Tasmota ENERGY.Total, bare scalars).
- MqttTopicMatcher: standard +/# wildcard matching.
- IngestionService: scale/offset, idempotent upsert on (meter_id, time), and a spurious-
decrease guard for monotonic registers (allowed only with a reset/swap event) + source
last-seen status.
- MqttMessageRouter + MqttIngestionWorker (MQTTnet 5): per-endpoint persistent connections,
topic subscription, graceful degradation; secrets resolved by env-var reference.
- Home Assistant: HaStateClient (REST /api/states parse) + HomeAssistantWorker polling on
each source's interval. HA-via-MQTT also works through the MQTT path.
- Ingestion workers gated by MeterVault:EnableLiveIngestion (off in tests).
85 tests green (53 Core + 32 integration): Tasmota payload → reading verified end to end.
Follow-up (polish): HA WebSocket push (state_changed) as an alternative to REST poll;
source-topic index caching in the router.
Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
- German-dialect scalar parsers in Core (GermanNumber/Money/Date, ValueCell):
decimal comma, thousands dot, unit suffixes, € currency, both date shapes.
- Declarative MappingProfile + RowClassifier (skip summary/blank/all-zero rows) +
CsvImporter (CsvHelper) staging readings/events/manual-costs, with auto swap
detection on register decreases and month-end anchoring for interleaved oil dates.
- Four built-in ReferenceProfiles (Strom/Wasser/Heizöl/Kosten).
- ImportService: commit as revertible import_batch + wholesale consumption recompute
per affected meter (NormalizationService/MeterConfigFactory), revert by batch.
- Reconciliation tests: all 4 CSVs match the sheet's own columns within tolerance
(electricity 5 meters + Netz Einsparung, water swap→12, oil tank incl. deliveries +
burner hours, cost category totals). Commit/revert round-trip verified on Timescale.
69 tests green (53 Core + 16 integration).
Known follow-up (polish): historical imports can contend with the 30-day compression
policy's background job; tests pause it. Consider retry-on-deadlock or deferred
compression for large historical imports in production.
Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
schmidt.florian