Files
MeterVault/src/App/Components/Pages/Admin/Connectors.razor
T
schmidt.florian 8fe5f4411b
ci / build-test (push) Successful in 1m16s
Fix defects found auditing the ingestion, import and connector changes
An audit of this session's commits found several real problems, three of which
lose or expose data. Ordered by severity.

Live recompute was not atomic. RecomputeMeterAsync clears a meter's series with
ExecuteDelete, which commits by itself when no transaction is ambient, and only
then adds the rebuilt rows. Between the two the meter had *no* consumption:
a dashboard read reported zero, and a crash or cancelled request made the loss
permanent, for data the SDD treats as the long-term source of truth (§5.5).
Import and the events API already wrapped their recomputes; live ingestion,
which I added this session, did not. Now shares one transaction, joining an
ambient one rather than nesting.

The MQTT backfill migration counted brokers without regard to is_enabled. One
live broker plus a disabled leftover counted two, declined to backfill, and left
those sources unbound — which under endpoint-scoped routing means silently and
permanently dead. The "two or more is ambiguous" reasoning did not hold there:
the worker only ever connected to enabled endpoints. Corrected by a follow-up
migration rather than an edit, since the original may already have run; it
touches only rows still NULL, so hand-made bindings are safe.

A mapping edited after a dry run committed the *old* staged rows under the
*new* mapping. Readings went to the previous meter while the batch recorded the
current mapping — wrong data, provenance contradicting it, no exception. The
earlier fix re-validated but did not detect staleness. Commit now compares the
mapping against the one the preview was staged under and refuses.

"Test connection" sent a stored token to whatever Base URL was in the dialog.
Encrypting secrets at rest means the UI can decrypt what the operator can no
longer read, so this turned the button into an exfiltration primitive: point it
at any host, the token arrives as a Bearer header. A stored token now only goes
to the origin it was saved for; testing elsewhere requires typing it again.

A source that cannot ingest looked identical to a healthy one. Endpoint-scoped
routing made unbound and mis-bound sources silently dead, while the Sources tab
showed no connector at all and the delete dialog still promised sources would be
"unlinked". Added a Connector column that names the fault, stopped offering
disabled connectors (both workers filter on IsEnabled), and made the delete
warning say ingestion stops.

Virtual meters rendered four zero tiles: they evaluate on read and only
materialize when a cost category references them (§14.1), so summing
consumption is a confident lie about a working meter. They now report nothing
and the page explains why.

Re-importing an overlapping file failed at the database with EF's "An error
occurred while saving the entity changes", naming neither meter nor date — the
diagnosis problem a3af483 set out to fix, via the path its guard could not see.
Checked up front now, bounded by each meter's staged range.

The LXC updater left the service stopped on any failure. set -e plus an
explicit stop means Restart=always does not apply, so an OOM-killed publish or
a brief Gitea outage took MeterVault down until someone noticed. An EXIT trap
restarts the previous build and says so.

Claude-Session: https://claude.ai/code/session_01V6joyergfvVLFEizH1hJLd
2026-07-18 19:52:48 +02:00

402 lines
19 KiB
Plaintext

@page "/admin/connectors"
@inject Microsoft.EntityFrameworkCore.IDbContextFactory<MeterVault.Infrastructure.Persistence.MeterVaultDbContext> DbFactory
@inject MeterVault.Infrastructure.Ingestion.HaConnectionTester HaTester
@inject MeterVault.Infrastructure.Security.SecretProtector Secrets
@inject ISnackbar Snackbar
@inject IDialogService DialogService
@using Microsoft.EntityFrameworkCore
@using MeterVault.Infrastructure.Ingestion
@using MudBlazor
<PageTitle>MeterVault — Connectors</PageTitle>
<div class="d-flex align-center justify-space-between mb-4 flex-wrap" style="gap:1rem">
<MudText Typo="Typo.h4">Connectors</MudText>
<MudButton Variant="Variant.Filled" Color="Color.Primary" StartIcon="@Icons.Material.Filled.Add" OnClick="@(() => OpenEdit(null))">
Add connector
</MudButton>
</div>
<MudAlert Severity="Severity.Info" Class="mb-4" Dense="true">
Secrets are never stored here. Credentials/tokens are referenced by the <b>name of an environment variable</b>
(or Docker secret) resolved at runtime.
</MudAlert>
@if (_endpoints is null)
{
<MudProgressLinear Indeterminate="true" Color="Color.Primary" />
}
else
{
<MudTable Items="_endpoints" Dense="true" Hover="true" Elevation="2">
<HeaderContent>
<MudTh>Name</MudTh>
<MudTh>Type</MudTh>
<MudTh>Enabled</MudTh>
<MudTh>Last status</MudTh>
<MudTh>Last seen</MudTh>
<MudTh Style="text-align:right">Actions</MudTh>
</HeaderContent>
<RowTemplate>
<MudTd DataLabel="Name">@context.Name</MudTd>
<MudTd DataLabel="Type">@context.Type</MudTd>
<MudTd DataLabel="Enabled">@(context.IsEnabled ? "yes" : "no")</MudTd>
<MudTd DataLabel="Last status">@(context.LastStatus ?? "—")</MudTd>
<MudTd DataLabel="Last seen">@(context.LastSeenAt?.ToString("yyyy-MM-dd HH:mm") ?? "—")</MudTd>
<MudTd DataLabel="Actions" Style="text-align:right">
<MudIconButton Icon="@Icons.Material.Filled.Edit" Size="Size.Small" OnClick="@(() => OpenEdit(context))" />
<MudIconButton Icon="@Icons.Material.Filled.Delete" Size="Size.Small" Color="Color.Error" OnClick="@(() => DeleteAsync(context))" />
</MudTd>
</RowTemplate>
</MudTable>
@if (_endpoints.Count == 0)
{
<MudAlert Severity="Severity.Normal" Class="mt-4">No connectors yet. Add an MQTT broker or a Home Assistant connection.</MudAlert>
}
}
<MudDialog @bind-Visible="_editOpen" Options="_dialogOptions">
<TitleContent>
<MudText Typo="Typo.h6">@(_working.Id == 0 ? "New connector" : $"Edit {_working.Name}")</MudText>
</TitleContent>
<DialogContent>
<MudSelect T="EndpointType" @bind-Value="_working.Type" Label="Type" Class="mb-2">
@foreach (var type in Enum.GetValues<EndpointType>())
{
<MudSelectItem T="EndpointType" Value="type">@type</MudSelectItem>
}
</MudSelect>
<MudTextField @bind-Value="_working.Name" Label="Name" Required="true" Class="mb-2" />
@if (_working.Type == EndpointType.HomeAssistant)
{
<MudTextField @bind-Value="_working.BaseUrl" Label="Base URL (e.g. http://homeassistant.local:8123)" Class="mb-2" />
<MudSwitch T="bool" @bind-Value="_working.UseDirectToken" Label="Enter the token here" Color="Color.Primary" Class="mb-1" />
@if (_working.UseDirectToken)
{
<MudTextField @bind-Value="_working.Token" InputType="InputType.Password" Class="mb-1"
Label="@(_working.HasStoredToken ? "Long-lived access token (stored — type to replace)" : "Long-lived access token")" />
<MudText Typo="Typo.caption" Color="Color.Secondary" Class="mb-2 d-block">
Encrypted before it is stored; database dumps and JSON exports carry nothing usable.
</MudText>
}
else
{
<MudTextField @bind-Value="_working.TokenEnv" Label="Token env-var name (e.g. HA_TOKEN)" Class="mb-1" />
<MudText Typo="Typo.caption" Color="Color.Secondary" Class="mb-2 d-block">
The variable's <em>name</em>, not the token. Set it on the server and restart the app.
</MudText>
}
<MudSwitch T="bool" @bind-Value="_working.UseWebSocket" Label="Real-time WebSocket push" Color="Color.Primary" Class="mb-1" />
<MudText Typo="Typo.caption" Color="Color.Secondary" Class="mb-2 d-block">
On: subscribe to state changes and ingest in real time. Off: REST poll on each source's interval.
</MudText>
<MudTextField @bind-Value="_working.TestEntityId" Label="Test entity id (optional, e.g. sensor.house_power)" Class="mb-2" />
<MudButton Variant="Variant.Outlined" StartIcon="@Icons.Material.Filled.NetworkCheck" OnClick="TestHaAsync" Disabled="_testing" Class="mb-2">
Test connection
</MudButton>
@if (_testing)
{
<MudProgressLinear Indeterminate="true" Color="Color.Primary" Class="mb-2" />
}
@if (_testResult is not null)
{
<MudAlert Severity="@(_testResult.Ok ? Severity.Success : Severity.Error)" Dense="true" Class="mb-2">@_testResult.Message</MudAlert>
}
}
else
{
<MudTextField @bind-Value="_working.Host" Label="Host" Class="mb-2" />
<MudNumericField T="int" @bind-Value="_working.Port" Label="Port" Class="mb-2" />
<MudSwitch T="bool" @bind-Value="_working.Tls" Label="TLS" Color="Color.Primary" Class="mb-2" />
<MudSwitch T="bool" @bind-Value="_working.UseDirectCredentials" Label="Enter credentials here" Color="Color.Primary" Class="mb-1" />
@if (_working.UseDirectCredentials)
{
<MudTextField @bind-Value="_working.Username" Label="Username (optional)" Class="mb-1" />
<MudTextField @bind-Value="_working.Password" InputType="InputType.Password" Class="mb-1"
Label="@(_working.HasStoredPassword ? "Password (stored — type to replace)" : "Password (optional)")" />
<MudText Typo="Typo.caption" Color="Color.Secondary" Class="mb-2 d-block">
Encrypted before it is stored; database dumps and JSON exports carry nothing usable.
</MudText>
}
else
{
<MudTextField @bind-Value="_working.UsernameEnv" Label="Username env-var name (optional)" Class="mb-2" />
<MudTextField @bind-Value="_working.PasswordEnv" Label="Password env-var name (optional)" Class="mb-2" />
}
<MudTextField @bind-Value="_working.ExtraTopics" Label="Extra topics (comma-separated, optional)" Class="mb-2" />
}
<MudSwitch T="bool" @bind-Value="_working.IsEnabled" Label="Enabled" Color="Color.Primary" />
</DialogContent>
<DialogActions>
<MudButton OnClick="@(() => _editOpen = false)">Cancel</MudButton>
<MudButton Color="Color.Primary" Variant="Variant.Filled" OnClick="SaveAsync">Save</MudButton>
</DialogActions>
</MudDialog>
@code {
private List<IngestionEndpoint>? _endpoints;
private bool _editOpen;
private bool _testing;
private HaTestResult? _testResult;
private EditModel _working = new();
private readonly DialogOptions _dialogOptions = new() { MaxWidth = MaxWidth.Small, FullWidth = true };
protected override Task OnInitializedAsync() => LoadAsync();
private async Task LoadAsync()
{
await using var db = await DbFactory.CreateDbContextAsync();
_endpoints = await db.IngestionEndpoints.AsNoTracking().OrderBy(e => e.Name).ToListAsync();
}
private void OpenEdit(IngestionEndpoint? endpoint)
{
_testResult = null;
if (endpoint is null)
{
_working = new EditModel();
}
else if (endpoint.Type == EndpointType.HomeAssistant)
{
var ha = HaEndpointConfig.Parse(endpoint.Config);
_working = new EditModel
{
Id = endpoint.Id, Type = endpoint.Type, Name = endpoint.Name, IsEnabled = endpoint.IsEnabled,
BaseUrl = ha.BaseUrl, TokenEnv = ha.TokenEnv, UseWebSocket = ha.UseWebSocket,
// Carry the ciphertext through untouched and never send the secret to the browser:
// the field stays blank and only a typed value replaces what is stored.
TokenEnc = ha.TokenEnc,
UseDirectToken = !string.IsNullOrWhiteSpace(ha.TokenEnc),
// The host the stored token was saved against; a stored token is never sent anywhere else.
SavedBaseUrl = ha.BaseUrl,
};
}
else
{
var mqtt = EndpointConfig.Parse(endpoint.Config);
_working = new EditModel
{
Id = endpoint.Id, Type = endpoint.Type, Name = endpoint.Name, IsEnabled = endpoint.IsEnabled,
Host = mqtt.Host, Port = mqtt.Port, Tls = mqtt.Tls,
UsernameEnv = mqtt.UsernameEnv, PasswordEnv = mqtt.PasswordEnv,
Username = mqtt.Username, PasswordEnc = mqtt.PasswordEnc,
UseDirectCredentials =
!string.IsNullOrWhiteSpace(mqtt.Username) || !string.IsNullOrWhiteSpace(mqtt.PasswordEnc),
ExtraTopics = string.Join(", ", mqtt.ExtraTopics),
};
}
_editOpen = true;
}
private async Task TestHaAsync()
{
_testing = true;
_testResult = null;
try
{
// Test what the connector would actually use — including a token typed but not yet
// saved, so a bad token is caught before it is stored.
if (_working.UseDirectToken)
{
if (!string.IsNullOrWhiteSpace(_working.Token))
{
_testResult = await HaTester.TestAsync(_working.BaseUrl, _working.Token, _working.TestEntityId);
}
else if (!SameOrigin(_working.BaseUrl, _working.SavedBaseUrl))
{
// Storing the token encrypted means the UI can decrypt something the operator
// can no longer read. Sending it to a Base URL edited in this dialog would turn
// "Test connection" into an exfiltration primitive — point it at any host and the
// token arrives as a Bearer header. A stored secret only ever goes to the origin
// it was saved for; testing elsewhere means typing the token again.
_testResult = new HaTestResult(false,
"Base URL differs from the saved one. Re-enter the token to test against a different host — "
+ "a stored token is only sent to the host it was saved for.");
}
else if (Secrets.TryUnprotect(_working.TokenEnc, out var stored) && stored is { Length: > 0 })
{
_testResult = await HaTester.TestAsync(_working.BaseUrl, stored, _working.TestEntityId);
}
else
{
_testResult = new HaTestResult(false, "Enter a token first.");
}
}
else if (string.IsNullOrWhiteSpace(_working.TokenEnv))
{
_testResult = new HaTestResult(false,
"Name the environment variable holding the token, or switch on \"Enter the token here\".");
}
else if (Environment.GetEnvironmentVariable(_working.TokenEnv) is not { Length: > 0 } envToken)
{
_testResult = new HaTestResult(false,
$"Environment variable '{_working.TokenEnv}' is not set on the server. Set it and restart the app, "
+ "or switch on \"Enter the token here\" to store the token directly.");
}
else
{
_testResult = await HaTester.TestAsync(_working.BaseUrl, envToken, _working.TestEntityId);
}
}
finally
{
_testing = false;
}
}
private async Task SaveAsync()
{
if (string.IsNullOrWhiteSpace(_working.Name))
{
Snackbar.Add("Name is required.", Severity.Warning);
return;
}
if (_working.Type == EndpointType.HomeAssistant
&& _working.UseDirectToken
&& string.IsNullOrWhiteSpace(_working.Token)
&& !_working.HasStoredToken)
{
Snackbar.Add("Enter the token, or switch off \"Enter the token here\" and name an env var.", Severity.Warning);
return;
}
var config = _working.Type == EndpointType.HomeAssistant
? new HaEndpointConfig
{
BaseUrl = Trim(_working.BaseUrl),
UseWebSocket = _working.UseWebSocket,
// Exactly one storage form survives a save: switching modes clears the other, so a
// stale token cannot linger and silently win at resolution time.
TokenEnv = _working.UseDirectToken ? null : Trim(_working.TokenEnv),
TokenEnc = _working.UseDirectToken ? ProtectOrKeep(_working.Token, _working.TokenEnc) : null,
}.ToJson()
: new EndpointConfig
{
Host = string.IsNullOrWhiteSpace(_working.Host) ? "localhost" : _working.Host.Trim(),
Port = _working.Port,
Tls = _working.Tls,
UsernameEnv = _working.UseDirectCredentials ? null : Trim(_working.UsernameEnv),
PasswordEnv = _working.UseDirectCredentials ? null : Trim(_working.PasswordEnv),
Username = _working.UseDirectCredentials ? Trim(_working.Username) : null,
PasswordEnc = _working.UseDirectCredentials
? ProtectOrKeep(_working.Password, _working.PasswordEnc)
: null,
ExtraTopics = SplitTopics(_working.ExtraTopics),
}.ToJson();
await using var db = await DbFactory.CreateDbContextAsync();
if (_working.Id == 0)
{
db.IngestionEndpoints.Add(new IngestionEndpoint
{
Type = _working.Type, Name = _working.Name.Trim(), Config = config, IsEnabled = _working.IsEnabled,
});
}
else
{
var existing = await db.IngestionEndpoints.FirstAsync(e => e.Id == _working.Id);
existing.Type = _working.Type;
existing.Name = _working.Name.Trim();
existing.Config = config;
existing.IsEnabled = _working.IsEnabled;
}
await db.SaveChangesAsync();
_editOpen = false;
Snackbar.Add("Saved.", Severity.Success);
await LoadAsync();
}
private async Task DeleteAsync(IngestionEndpoint endpoint)
{
await using var db = await DbFactory.CreateDbContextAsync();
var sourceCount = await db.MeterSources.CountAsync(s => s.EndpointId == endpoint.Id);
// Routing is endpoint-scoped, so "unlinked" now means those sources stop ingesting entirely
// rather than falling back to any broker. Say so plainly.
var note = sourceCount > 0
? $" {sourceCount} source(s) use it and will stop ingesting until reassigned to another connector."
: "";
if (!await Confirm.DeleteAsync(DialogService, "Delete connector", $"Delete '{endpoint.Name}'?{note}"))
{
return;
}
await db.IngestionEndpoints.Where(e => e.Id == endpoint.Id).ExecuteDeleteAsync();
Snackbar.Add("Deleted.", Severity.Success);
await LoadAsync();
}
private static string? Trim(string? value) => string.IsNullOrWhiteSpace(value) ? null : value.Trim();
/// <summary>
/// Whether two URLs address the same host. Compares scheme, host and port rather than the raw
/// string, so a trailing slash or a path tweak does not force the token to be re-typed. Fails
/// closed: anything unparsable counts as a different origin.
/// </summary>
private static bool SameOrigin(string? a, string? b) =>
Uri.TryCreate(a, UriKind.Absolute, out var left)
&& Uri.TryCreate(b, UriKind.Absolute, out var right)
&& string.Equals(left.Scheme, right.Scheme, StringComparison.OrdinalIgnoreCase)
&& string.Equals(left.Host, right.Host, StringComparison.OrdinalIgnoreCase)
&& left.Port == right.Port;
/// <summary>
/// Encrypts a newly typed secret, or keeps the stored ciphertext when the field was left blank.
/// The plaintext is never sent to the browser, so blank means "unchanged", not "cleared".
/// </summary>
private string? ProtectOrKeep(string? typed, string? existingCiphertext) =>
string.IsNullOrWhiteSpace(typed) ? existingCiphertext : Secrets.Protect(typed);
private static IReadOnlyList<string> SplitTopics(string? csv) =>
string.IsNullOrWhiteSpace(csv) ? [] : [.. csv.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries)];
private sealed class EditModel
{
public int Id { get; set; }
public EndpointType Type { get; set; } = EndpointType.HomeAssistant;
public string Name { get; set; } = "";
public bool IsEnabled { get; set; } = true;
// Home Assistant
public string? BaseUrl { get; set; }
public string? TokenEnv { get; set; }
public bool UseWebSocket { get; set; }
public string? TestEntityId { get; set; }
/// <summary>True to store the token here (encrypted); false to name an env var.</summary>
public bool UseDirectToken { get; set; }
/// <summary>Typed token. Always blank on load — a stored secret is never sent to the browser.</summary>
public string? Token { get; set; }
/// <summary>Stored ciphertext, round-tripped so leaving <see cref="Token"/> blank keeps it.</summary>
public string? TokenEnc { get; set; }
/// <summary>Base URL as saved, so an edited one can be told from the token's own host.</summary>
public string? SavedBaseUrl { get; set; }
public bool HasStoredToken => !string.IsNullOrWhiteSpace(TokenEnc);
// MQTT broker
public string? Host { get; set; } = "localhost";
public int Port { get; set; } = 1883;
public bool Tls { get; set; }
public string? UsernameEnv { get; set; }
public string? PasswordEnv { get; set; }
public string? ExtraTopics { get; set; }
public bool UseDirectCredentials { get; set; }
/// <summary>Username entered directly — not a secret, so shown when editing.</summary>
public string? Username { get; set; }
public string? Password { get; set; }
public string? PasswordEnc { get; set; }
public bool HasStoredPassword => !string.IsNullOrWhiteSpace(PasswordEnc);
}
}