schmidt.florian
8fe5f4411b
An audit of this session's commits found several real problems, three of which
lose or expose data. Ordered by severity.
Live recompute was not atomic. RecomputeMeterAsync clears a meter's series with
ExecuteDelete, which commits by itself when no transaction is ambient, and only
then adds the rebuilt rows. Between the two the meter had *no* consumption:
a dashboard read reported zero, and a crash or cancelled request made the loss
permanent, for data the SDD treats as the long-term source of truth (§5.5).
Import and the events API already wrapped their recomputes; live ingestion,
which I added this session, did not. Now shares one transaction, joining an
ambient one rather than nesting.
The MQTT backfill migration counted brokers without regard to is_enabled. One
live broker plus a disabled leftover counted two, declined to backfill, and left
those sources unbound — which under endpoint-scoped routing means silently and
permanently dead. The "two or more is ambiguous" reasoning did not hold there:
the worker only ever connected to enabled endpoints. Corrected by a follow-up
migration rather than an edit, since the original may already have run; it
touches only rows still NULL, so hand-made bindings are safe.
A mapping edited after a dry run committed the *old* staged rows under the
*new* mapping. Readings went to the previous meter while the batch recorded the
current mapping — wrong data, provenance contradicting it, no exception. The
earlier fix re-validated but did not detect staleness. Commit now compares the
mapping against the one the preview was staged under and refuses.
"Test connection" sent a stored token to whatever Base URL was in the dialog.
Encrypting secrets at rest means the UI can decrypt what the operator can no
longer read, so this turned the button into an exfiltration primitive: point it
at any host, the token arrives as a Bearer header. A stored token now only goes
to the origin it was saved for; testing elsewhere requires typing it again.
A source that cannot ingest looked identical to a healthy one. Endpoint-scoped
routing made unbound and mis-bound sources silently dead, while the Sources tab
showed no connector at all and the delete dialog still promised sources would be
"unlinked". Added a Connector column that names the fault, stopped offering
disabled connectors (both workers filter on IsEnabled), and made the delete
warning say ingestion stops.
Virtual meters rendered four zero tiles: they evaluate on read and only
materialize when a cost category references them (§14.1), so summing
consumption is a confident lie about a working meter. They now report nothing
and the page explains why.
Re-importing an overlapping file failed at the database with EF's "An error
occurred while saving the entity changes", naming neither meter nor date — the
diagnosis problem a3af483 set out to fix, via the path its guard could not see.
Checked up front now, bounded by each meter's staged range.
The LXC updater left the service stopped on any failure. set -e plus an
explicit stop means Restart=always does not apply, so an OOM-killed publish or
a brief Gitea outage took MeterVault down until someone noticed. An EXIT trap
restarts the previous build and says so.
Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
MeterVault
A self-hosted, local-first energy & utility metering platform. MeterVault pulls meter data from Home Assistant, Tasmota and raw MQTT on a schedule, stores every reading timestamped and immutable, normalizes it into consumption, and turns it into cost dashboards. Energy types (electricity, water, heating oil, gas, district heat, …) and meters are user-defined — nothing is hardcoded.
Successor to a hand-maintained Energiebilanz spreadsheet. See docs/SDD.md for the
full design.
Features
- Automatic ingestion from MQTT/Tasmota (persistent subscriptions) and Home Assistant (REST poll or push), plus manual entry, a REST push API, and CSV import.
- Immutable raw readings on a TimescaleDB hypertable; a normalized, append-only consumption layer on top — reproducible, auditable.
- Seven measurement modes (cumulative/generation registers, burner runtime, tank/consumable, direct delta, instant rate, virtual). Handles meter swaps, counter resets, tank dip-sticks with calibration, and virtual meters defined by an expression (PV self-consumption, savings, net).
- Tariff engine with time-ranged price history (unit/base/feed-in), scoped global / per type / per meter; cost categories decoupled from energy types; meterless manual costs.
- Continuous aggregates (daily/monthly/yearly, local timezone) so dashboards never scan raw.
- Dashboard: cost KPIs with period-over-period deltas, "what costs most", a "what cost more/ less" difference view, trends, a PV/Solar panel (generation, self-consumption, autarky %, savings), an oil/consumable panel (tank gauge, deliveries, burner runtime, effective L/h, forecast-to-empty) and a per-meter detail view (raw readings, consumption, sources, tariff timeline, events), one-click reference-data load, CSV dry-run.
- Per-energy-type flow pages (Electricity, Water, …): a Sankey diagram of the meter chain — a downstream meter is a subsection of an upstream one (main → car, pool, garden, …), arrow thickness ∝ amount, with an auto-computed "Other/unmetered" remainder. Meters can have several upstreams (a merge, e.g. grid + solar → house).
- Admin UI: full create/edit/delete for energy types, meters (with consumption recompute on mode/baseline change, and cycle-safe upstream-meter wiring), ingest sources, tariffs, cost categories, and MQTT/Home-Assistant connectors; a "Test connection" for Home Assistant; effective-settings view.
- REST API + OpenAPI/Swagger, API-key auth, reverse-proxy trust (Authelia/Traefik).
- JSON config export/import for portability; Docker Compose + multi-arch image.
Quick start (Docker)
docker compose -f deploy/docker-compose.yml up -d
# open http://localhost:8760 → Import → "Load reference data" for a populated demo
# ...or start pre-populated: METERVAULT_SEED=true docker compose -f deploy/docker-compose.yml up -d
# API docs at http://localhost:8760/swagger
Quick start (Proxmox VE LXC)
A community-scripts–style installer builds a self-contained LXC (Debian + PostgreSQL/TimescaleDB + the app as a systemd service). Run on the Proxmox host:
bash -c "$(curl -fsSL https://git.finalfactory.de/FinalFactory/MeterVault/raw/branch/master/deploy/ct/metervault.sh)"
It asks the standard container questions, optionally loads the demo dataset, and prints the URL
(http://<ct-ip>:8760) plus the generated DB password. Re-run update inside the container to pull
the latest source and rebuild. It builds from this public Gitea repo (there is no prebuilt tarball —
releases ship as a container image). See deploy/ct/metervault.sh and
deploy/install/metervault-install.sh.
Configuration is via environment variables (Section__Key double-underscore mapping), e.g.:
| Variable | Purpose |
|---|---|
ConnectionStrings__Default |
PostgreSQL/Timescale connection string |
MeterVault__TimeZone |
Local timezone for buckets/display (default Europe/Berlin) |
MeterVault__ApiKeys__0 |
An API key accepted on the X-Api-Key header |
MeterVault__AllowAnonymousApi |
true to open the REST API without a key (trusted LAN only) |
MeterVault__ReverseProxyTrust |
true to honour X-Forwarded-User behind an auth proxy |
MeterVault__EnableLiveIngestion |
false to disable the MQTT/HA workers |
MeterVault__SeedReferenceData |
true to load the bundled demo dataset on first start (idempotent) |
MeterVault__DataProtectionKeyPath |
Where the key ring for UI-entered connector secrets lives (default /var/lib/metervault/keys) |
The REST API is closed by default: with no ApiKeys configured and AllowAnonymousApi off, it
returns 401. Set at least one API key (or open it explicitly for a trusted network).
Secrets (broker/HA tokens) are never stored in the database as plaintext. Each connector picks
one of two forms: the name of an environment variable, resolved at runtime, or the secret typed
into the admin UI and encrypted at rest under the data-protection key ring. Either way a pg_dump
or JSON export carries nothing usable.
Keep the key ring on persistent storage outside the app directory — the default
/var/lib/metervault/keys survives an LXC update, and the Compose file mounts a named volume for it.
Lose it and every UI-entered secret must be re-entered. The key ring is on disk, so this protects
against leaked database content, not against an attacker who already has the host; that is the same
trust boundary an environment variable has.
Pushing readings (Home Assistant)
curl -X POST http://localhost:8760/api/v1/readings \
-H "X-Api-Key: $METERVAULT_API_KEY" -H "Content-Type: application/json" \
-d '[{"meterId": 1, "time": "2026-01-01T12:00:00Z", "value": 47200}]'
See docs/wiring.md for wiring up Tasmota, MQTT and Home Assistant.
Development
dotnet build
dotnet test # integration tests spin a TimescaleDB via Testcontainers (needs Docker)
dotnet test tests/Core.Tests # fast unit tests, no Docker
dotnet run --project src/App
Architecture, project layout and conventions live in CLAUDE.md.
Releasing
Edit the VERSION file on master; Gitea Actions tags vX.Y.Z and builds/pushes a
multi-arch image to the Gitea container registry (.gitea/workflows/). Locally:
pwsh deploy/build-and-push.ps1 -Registry git.finalfactory.de -Image finalfactory/metervault -Push.
Requires a Docker-capable act_runner; the image build itself is self-contained.
License
Not yet chosen (see SDD §14). Add a LICENSE before the first public tag.