schmidt.florian
cedd60ab45
ci / build-test (push) Successful in 1m17s
Three defects found reviewing the last few commits. Deriving consumption on ingest made the batch reading endpoint quadratic. A recompute rewrites a meter's entire consumption series, and POST /api/v1/readings ran one per reading -- 500 readings for one meter meant 500 full rewrites. IngestByMeterAsync takes renormalize:false and the endpoint normalizes each touched meter once after the batch. Percentage change divided by a possibly negative baseline. A net-export meter going from -100 to -150 exported half again as much and would have been reported as "+50%", reading as more consumption. A non-positive baseline now reports no basis rather than a confident lie. The data-protection key ring had no persistent home outside Docker Compose. The LXC installer now creates /var/lib/metervault/keys at 0700 -- the app would otherwise create it under the default umask, leaving a key ring world-readable -- and the Unraid template maps it, since without that every UI-entered secret was lost whenever the container was recreated. README documents the variable and the trust boundary: keys on disk protect against leaked database content, not against an attacker who already has the host. Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
122 lines
6.6 KiB
Markdown
122 lines
6.6 KiB
Markdown
# MeterVault
|
||
|
||
A self-hosted, local-first **energy & utility metering platform**. MeterVault pulls meter data
|
||
from Home Assistant, Tasmota and raw MQTT on a schedule, stores every reading timestamped and
|
||
immutable, normalizes it into consumption, and turns it into cost dashboards. Energy types
|
||
(electricity, water, heating oil, gas, district heat, …) and meters are **user-defined — nothing
|
||
is hardcoded**.
|
||
|
||
Successor to a hand-maintained *Energiebilanz* spreadsheet. See [`docs/SDD.md`](docs/SDD.md) for the
|
||
full design.
|
||
|
||
## Features
|
||
|
||
- **Automatic ingestion** from MQTT/Tasmota (persistent subscriptions) and Home Assistant (REST
|
||
poll or push), plus manual entry, a REST push API, and CSV import.
|
||
- **Immutable raw readings** on a TimescaleDB hypertable; a normalized, append-only **consumption**
|
||
layer on top — reproducible, auditable.
|
||
- **Seven measurement modes** (cumulative/generation registers, burner runtime, tank/consumable,
|
||
direct delta, instant rate, virtual). Handles meter swaps, counter resets, tank dip-sticks with
|
||
calibration, and **virtual meters** defined by an expression (PV self-consumption, savings, net).
|
||
- **Tariff engine** with time-ranged price history (unit/base/feed-in), scoped global / per type /
|
||
per meter; **cost categories** decoupled from energy types; meterless manual costs.
|
||
- **Continuous aggregates** (daily/monthly/yearly, local timezone) so dashboards never scan raw.
|
||
- **Dashboard**: cost KPIs with period-over-period deltas, "what costs most", a "what cost more/
|
||
less" difference view, trends, a **PV/Solar panel** (generation, self-consumption, autarky %,
|
||
savings), an **oil/consumable panel** (tank gauge, deliveries, burner runtime, effective L/h,
|
||
forecast-to-empty) and a **per-meter detail view** (raw readings, consumption, sources, tariff
|
||
timeline, events), one-click reference-data load, CSV dry-run.
|
||
- **Per-energy-type flow pages** (Electricity, Water, …): a **Sankey diagram** of the meter chain —
|
||
a downstream meter is a *subsection* of an upstream one (main → car, pool, garden, …), arrow
|
||
thickness ∝ amount, with an auto-computed "Other/unmetered" remainder. Meters can have several
|
||
upstreams (a merge, e.g. grid + solar → house).
|
||
- **Admin UI**: full create/edit/delete for energy types, meters (with consumption recompute on
|
||
mode/baseline change, and cycle-safe upstream-meter wiring), ingest sources, tariffs, cost
|
||
categories, and MQTT/Home-Assistant connectors; a "Test connection" for Home Assistant;
|
||
effective-settings view.
|
||
- **REST API + OpenAPI/Swagger**, API-key auth, reverse-proxy trust (Authelia/Traefik).
|
||
- **JSON config export/import** for portability; Docker Compose + multi-arch image.
|
||
|
||
## Quick start (Docker)
|
||
|
||
```bash
|
||
docker compose -f deploy/docker-compose.yml up -d
|
||
# open http://localhost:8760 → Import → "Load reference data" for a populated demo
|
||
# ...or start pre-populated: METERVAULT_SEED=true docker compose -f deploy/docker-compose.yml up -d
|
||
# API docs at http://localhost:8760/swagger
|
||
```
|
||
|
||
## Quick start (Proxmox VE LXC)
|
||
|
||
A community-scripts–style installer builds a self-contained LXC (Debian + PostgreSQL/TimescaleDB +
|
||
the app as a systemd service). Run **on the Proxmox host**:
|
||
|
||
```bash
|
||
bash -c "$(curl -fsSL https://git.finalfactory.de/FinalFactory/MeterVault/raw/branch/master/deploy/ct/metervault.sh)"
|
||
```
|
||
|
||
It asks the standard container questions, optionally loads the demo dataset, and prints the URL
|
||
(`http://<ct-ip>:8760`) plus the generated DB password. Re-run `update` inside the container to pull
|
||
the latest source and rebuild. It builds from this public Gitea repo (there is no prebuilt tarball —
|
||
releases ship as a container image). See [`deploy/ct/metervault.sh`](deploy/ct/metervault.sh) and
|
||
[`deploy/install/metervault-install.sh`](deploy/install/metervault-install.sh).
|
||
|
||
Configuration is via environment variables (`Section__Key` double-underscore mapping), e.g.:
|
||
|
||
| Variable | Purpose |
|
||
|----------|---------|
|
||
| `ConnectionStrings__Default` | PostgreSQL/Timescale connection string |
|
||
| `MeterVault__TimeZone` | Local timezone for buckets/display (default `Europe/Berlin`) |
|
||
| `MeterVault__ApiKeys__0` | An API key accepted on the `X-Api-Key` header |
|
||
| `MeterVault__AllowAnonymousApi` | `true` to open the REST API without a key (trusted LAN only) |
|
||
| `MeterVault__ReverseProxyTrust` | `true` to honour `X-Forwarded-User` behind an auth proxy |
|
||
| `MeterVault__EnableLiveIngestion` | `false` to disable the MQTT/HA workers |
|
||
| `MeterVault__SeedReferenceData` | `true` to load the bundled demo dataset on first start (idempotent) |
|
||
| `MeterVault__DataProtectionKeyPath` | Where the key ring for UI-entered connector secrets lives (default `/var/lib/metervault/keys`) |
|
||
|
||
The REST API is **closed by default**: with no `ApiKeys` configured and `AllowAnonymousApi` off, it
|
||
returns 401. Set at least one API key (or open it explicitly for a trusted network).
|
||
|
||
Secrets (broker/HA tokens) are **never** stored in the database as plaintext. Each connector picks
|
||
one of two forms: the *name* of an environment variable, resolved at runtime, or the secret typed
|
||
into the admin UI and encrypted at rest under the data-protection key ring. Either way a `pg_dump`
|
||
or JSON export carries nothing usable.
|
||
|
||
Keep the key ring on persistent storage outside the app directory — the default
|
||
`/var/lib/metervault/keys` survives an LXC update, and the Compose file mounts a named volume for it.
|
||
Lose it and every UI-entered secret must be re-entered. The key ring is on disk, so this protects
|
||
against leaked database content, not against an attacker who already has the host; that is the same
|
||
trust boundary an environment variable has.
|
||
|
||
## Pushing readings (Home Assistant)
|
||
|
||
```bash
|
||
curl -X POST http://localhost:8760/api/v1/readings \
|
||
-H "X-Api-Key: $METERVAULT_API_KEY" -H "Content-Type: application/json" \
|
||
-d '[{"meterId": 1, "time": "2026-01-01T12:00:00Z", "value": 47200}]'
|
||
```
|
||
|
||
See [`docs/wiring.md`](docs/wiring.md) for wiring up Tasmota, MQTT and Home Assistant.
|
||
|
||
## Development
|
||
|
||
```bash
|
||
dotnet build
|
||
dotnet test # integration tests spin a TimescaleDB via Testcontainers (needs Docker)
|
||
dotnet test tests/Core.Tests # fast unit tests, no Docker
|
||
dotnet run --project src/App
|
||
```
|
||
|
||
Architecture, project layout and conventions live in [`CLAUDE.md`](CLAUDE.md).
|
||
|
||
## Releasing
|
||
|
||
Edit the [`VERSION`](VERSION) file on `master`; Gitea Actions tags `vX.Y.Z` and builds/pushes a
|
||
multi-arch image to the Gitea container registry (`.gitea/workflows/`). Locally:
|
||
`pwsh deploy/build-and-push.ps1 -Registry git.finalfactory.de -Image finalfactory/metervault -Push`.
|
||
Requires a Docker-capable `act_runner`; the image build itself is self-contained.
|
||
|
||
## License
|
||
|
||
Not yet chosen (see SDD §14). Add a `LICENSE` before the first public tag.
|