An audit of this session's commits found several real problems, three of which
lose or expose data. Ordered by severity.
Live recompute was not atomic. RecomputeMeterAsync clears a meter's series with
ExecuteDelete, which commits by itself when no transaction is ambient, and only
then adds the rebuilt rows. Between the two the meter had *no* consumption:
a dashboard read reported zero, and a crash or cancelled request made the loss
permanent, for data the SDD treats as the long-term source of truth (§5.5).
Import and the events API already wrapped their recomputes; live ingestion,
which I added this session, did not. Now shares one transaction, joining an
ambient one rather than nesting.
The MQTT backfill migration counted brokers without regard to is_enabled. One
live broker plus a disabled leftover counted two, declined to backfill, and left
those sources unbound — which under endpoint-scoped routing means silently and
permanently dead. The "two or more is ambiguous" reasoning did not hold there:
the worker only ever connected to enabled endpoints. Corrected by a follow-up
migration rather than an edit, since the original may already have run; it
touches only rows still NULL, so hand-made bindings are safe.
A mapping edited after a dry run committed the *old* staged rows under the
*new* mapping. Readings went to the previous meter while the batch recorded the
current mapping — wrong data, provenance contradicting it, no exception. The
earlier fix re-validated but did not detect staleness. Commit now compares the
mapping against the one the preview was staged under and refuses.
"Test connection" sent a stored token to whatever Base URL was in the dialog.
Encrypting secrets at rest means the UI can decrypt what the operator can no
longer read, so this turned the button into an exfiltration primitive: point it
at any host, the token arrives as a Bearer header. A stored token now only goes
to the origin it was saved for; testing elsewhere requires typing it again.
A source that cannot ingest looked identical to a healthy one. Endpoint-scoped
routing made unbound and mis-bound sources silently dead, while the Sources tab
showed no connector at all and the delete dialog still promised sources would be
"unlinked". Added a Connector column that names the fault, stopped offering
disabled connectors (both workers filter on IsEnabled), and made the delete
warning say ingestion stops.
Virtual meters rendered four zero tiles: they evaluate on read and only
materialize when a cost category references them (§14.1), so summing
consumption is a confident lie about a working meter. They now report nothing
and the page explains why.
Re-importing an overlapping file failed at the database with EF's "An error
occurred while saving the entity changes", naming neither meter nor date — the
diagnosis problem a3af483 set out to fix, via the path its guard could not see.
Checked up front now, bounded by each meter's staged range.
The LXC updater left the service stopped on any failure. set -e plus an
explicit stop means Restart=always does not apply, so an OOM-killed publish or
a brief Gitea outage took MeterVault down until someone noticed. An EXIT trap
restarts the previous build and says so.
Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
Three defects found reviewing the last few commits.
Deriving consumption on ingest made the batch reading endpoint quadratic. A
recompute rewrites a meter's entire consumption series, and POST
/api/v1/readings ran one per reading -- 500 readings for one meter meant 500
full rewrites. IngestByMeterAsync takes renormalize:false and the endpoint
normalizes each touched meter once after the batch.
Percentage change divided by a possibly negative baseline. A net-export meter
going from -100 to -150 exported half again as much and would have been
reported as "+50%", reading as more consumption. A non-positive baseline now
reports no basis rather than a confident lie.
The data-protection key ring had no persistent home outside Docker Compose. The
LXC installer now creates /var/lib/metervault/keys at 0700 -- the app would
otherwise create it under the default umask, leaving a key ring world-readable
-- and the Unraid template maps it, since without that every UI-entered secret
was lost whenever the container was recreated. README documents the variable
and the trust boundary: keys on disk protect against leaked database content,
not against an attacker who already has the host.
Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
Reference-only secrets (SDD §6.4) meant adding a connector required editing a
file on the server and restarting the service. In practice that leads to the
token being pasted into the env-var *name* field, which fails with "environment
variable '<token>' is not set" and gives no hint what went wrong.
Add a second storage form, chosen per connector: type the secret in and it is
encrypted via ASP.NET Core data protection before it is stored. The env-var
reference stays as an equal alternative — this widens the choice rather than
replacing it. Exactly one form survives a save, so a stale secret cannot linger
and silently win; EndpointSecret.Resolve is the single resolution path.
The guarantee that matters is preserved: no plaintext in the database, so
pg_dump and JSON exports carry nothing usable. The trust boundary is stated
plainly in §6.4 — the key ring is on disk, so this protects against leaked
database content, not an attacker who already has the host, which is the same
boundary an env var has.
Details worth noting:
- Key ring defaults to /var/lib/metervault/keys, outside the app directory,
because the LXC updater republishes /opt/metervault on every update. Docker
gets a named volume. Overridable via MeterVault__DataProtectionKeyPath.
- Undecryptable ciphertext (key ring lost) falls back rather than throwing: an
ingestion worker on a timer should degrade, not crash.
- The stored secret is never sent to the browser; a blank field means
"unchanged", not "cleared".
- MQTT usernames are stored as-is — §6.4 covers tokens and passwords, and
encrypting a username would only blank the field on every edit.
- ExportService drops *_enc values: bound to the originating key ring, so
useless where an export would be restored. Expect to re-enter after a restore.
- HaConnectionTester now takes a resolved token, so the admin UI can test a
token that has been typed but not yet saved.
SDD §6.4 and §9 updated to describe both forms rather than contradict the code.
Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
The updater was emitted from a heredoc inside the installer, so a container
provisioned before it existed had no way to obtain /usr/bin/update — 'update'
just reported command not found, with no path forward short of reinstalling.
Ship it as deploy/install/metervault-update.sh and have the installer
install(1) it from the checkout it just built. An already-provisioned
container can now bootstrap from its own source tree after a git pull, and the
updater refreshes itself (atomic rename, since bash reads a running script
lazily) so the same stranding does not recur.
Uses echo rather than msg_warn for the missing-file case: msg_warn is not
otherwise relied on in this script, and an undefined function under the
framework ERR trap would abort an otherwise-successful install.
Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
Safety fix after a host-vs-container mishap: add a systemd-detect-virt guard at the top of the install script (before install.func is sourced) that refuses to run outside a container, so a mistaken invocation on the Proxmox host exits harmlessly instead of running 'apt upgrade' + installing PostgreSQL/.NET on the hypervisor (override METERVAULT_ALLOW_HOST=1). Also rewrite /usr/bin/update to a self-contained in-container rebuild (git pull + dotnet publish + restart) instead of re-running the ct script through the framework.
Claude-Session: https://claude.ai/code/session_01Kib2MniVFbD95fkgLgBBnB
The framework's motd_ssh runs 'chmod -x /etc/update-motd.d/*', which returns 'Operation not permitted' in some unprivileged LXCs. Under the community-scripts ERR trap that aborted an already-successful install right before /usr/bin/update was written. Write the update command first, then run motd_ssh/customize/cleanup best-effort (motd_ssh now guarded like customize already was).
Claude-Session: https://claude.ai/code/session_01Kib2MniVFbD95fkgLgBBnB
Adds deploy/ct/metervault.sh (host) + deploy/install/metervault-install.sh (in-LXC), modeled on MQTTower's community-scripts installer but adapted for MeterVault: it lives on public Gitea (not GitHub) and is a single Blazor app + TimescaleDB (not multi-mode). The host script sources the community-scripts framework, creates a Debian 12 LXC and runs the install script via pct exec. The install script sets up PostgreSQL 16 + TimescaleDB, installs the .NET SDK, builds the app from the public Gitea repo (no prebuilt tarball exists — releases ship as a container image), writes /etc/metervault/environment + a systemd unit on port 8760, and installs an 'update' command (git pull + rebuild). scripts/run-metervault-ct-install.sh is the maintainer helper. All three pass shellcheck (warning-level clean) and bash -n. README documents the one-liner.
Claude-Session: https://claude.ai/code/session_01Kib2MniVFbD95fkgLgBBnB
8080 is a very common, collision-prone port. 8760 (hours in a year) is an uncommon default that also avoids Home Assistant's 8123. Applied across Dockerfile (ASPNETCORE_URLS/EXPOSE), compose (mapping + healthcheck), the Unraid template, README and a code comment. The METERVAULT_PORT host override still works.
Claude-Session: https://claude.ai/code/session_01Kib2MniVFbD95fkgLgBBnB
Complete the SDD §8 dashboard views that were deferred at the M5 boundary,
and fix a shipping bug that left the Docker demo empty.
Bug: "Load reference data" created meters/tank/tariffs but imported zero
readings in Docker. Root cause: sampledata/ was excluded by .dockerignore and
never copied into the build stage, so the App csproj's linked Content glob
resolved to nothing at publish time; ReferenceDataImporter then silently
skipped the missing CSVs after already writing its marker meter, leaving the
DB permanently "loaded" but empty.
- .dockerignore: stop excluding sampledata/
- Dockerfile: COPY sampledata/ into the build stage
- ReferenceDataImporter: fail-fast (validate CSVs exist before the marker
meter) and throw instead of silently skipping a missing file
- Program.cs + MeterVaultOptions: opt-in MeterVault__SeedReferenceData
(compose METERVAULT_SEED=true) for a one-command populated demo
New SDD §8 panels (read models in Infrastructure/Dashboard, Blazor pages):
- §8.4 Solar/PV (/solar): generation from GenerationCounter meters;
self-consumption / autarky % / self-consumption % / savings derived from
meters tagged total_load & grid_import via Meter.Meta role config
(MeterRoles/MeterMeta) — nothing hardcoded by name.
- §8.5 Oil/consumable (/consumables): tank level (cm→L calibrated), fill
gauge, deliveries log, burner runtime, effective L/h (fixed/empirical),
forecast-to-empty, tariff cost, monthly series.
- §8.6 Meter detail (/meters/{id}): raw readings, normalized consumption,
source status, tariff timeline, events, measured-vs-estimated markers.
- Reusable SeriesChart component; nav links; Meters list rows link to detail.
Tests: MeterMetaTests (Core, +10); DashboardRenderTests extended to assert the
three panel services compute real figures and the new routes render (108 total,
all green). Live-verified in Docker: seed imports 302 readings / 347 consumption
rows; panels render (generation 16,481 kWh, oil 3,967 L) cross-checking the DB.
Claude-Session: https://claude.ai/code/session_01Lz2RqAsnQhetqWNoCDfexK
- Move CI/release workflows from .github/workflows to .gitea/workflows (Gitea Actions),
targeting the master branch.
- docker-publish: push to the Gitea container registry (git.finalfactory.de) with a
lowercased image name; login via github.token or a PACKAGES_TOKEN secret.
- version-tag: gitea-actions bot identity; optional RELEASE_TOKEN to re-trigger the image build.
- Update README/CLAUDE.md/Unraid template/build-and-push.ps1 to the Gitea registry + master.
Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
Correctness/data:
- Fix demo cost double-count: reference importer no longer imports the Kosten Strom/Wasser
columns for categories that are metered (only Heizung), so Wasser rollup is 70€ not 140€.
- Spurious-decrease guard: only a reset/swap in the window (prevReading, thisReading] explains
a decrease — an old historical reset no longer permanently disables the guard.
- Gate swap auto-detection on MappingProfile.DetectCumulativeSwaps (flag was ignored).
- Prorate basePrice by bucket length (day/month/year); guard virtual expressions against NaN/Inf.
Concurrency/infra:
- Blazor: register a DbContextFactory; CostService/DashboardService and the read pages now use
short-lived per-operation contexts (no shared circuit DbContext); guard Trends re-entrancy.
- /events: wrap event insert + consumption recompute in one transaction (atomic); 404 (not 500)
on unknown meter.
- MQTT worker: subscribe to newly-added topics on each tick; move client cleanup into finally.
- Migrations: CREATE MATERIALIZED VIEW IF NOT EXISTS + if_not_exists on CAgg/compression/
hypertable calls (re-run-safe after a mid-migration crash).
- HA worker: prune stale poll-schedule entries; export: null dangling ImportBatchIds on restore.
API/security:
- API fail-closed by default: with no keys and AllowAnonymousApi off, /api/v1 returns 401
(protects /export and /import). New MeterVault:AllowAnonymousApi opt-in.
- Cap /readings batch at 5000; report ignored (unknown-meter) count; enums as strings in JSON.
+4 regression tests (guard window, API closed, /events 404, no demo double-count). 98 tests
green; Docker deploy re-verified healthy with the API fail-closed.
Claude-Session: https://claude.ai/code/session_01WujdMtMJPbxDpDnMeK22rr
schmidt.florian