feat: harden firewall policies telemetry and operations

Implement reliable agent apply, telemetry v2, analytics retention, guarded rollouts, enrollment, labels and webhooks. Improve ReUI frontend data safety and bounded list loading, and add Docker smoke, recovery and resource checks. Document local verification and pending native acceptance.

docs/frontend-validation.md

@@ -0,0 +1,96 @@
+# Frontend validation
+
+## Baseline (2026-10-24)
+
+- Existing portal appearance, ReUI Frame surfaces, density and theme are preserved.
+- Parent-task baseline: initial JavaScript approximately 214 KB gzip; CSS approximately 27 KB gzip.
+- Existing web suite: 4 files, 10 tests passing. Existing TypeScript check passes.
+
+## Safety checkpoint
+
+- Settings refreshes only hydrate a clean form. Successful saves update cache and accepted defaults together; edits are disabled while saving. A failed background refresh shows an error while keeping existing form values mounted.
+- Policy assignments cannot be changed until both assignments and catalog have loaded successfully. Failed reads no longer seed an empty assignment list. In-flight assignment reads are cancelled before writes; successful writes populate their query cache.
+- Dashboard, agent preview, fleet chart and observed host firewall show explicit error/retry states instead of reporting empty results.
+- Dashboard agent rows route directly to agent detail.
+- Command palette no longer claims to parse natural language. Combobox/listbox relationships and active options are explicit.
+- Sortable attributes now belong to the focusable drag handle. Table rows support Enter/Space and ignore interactive descendants.
+
+## Verification
+
+- Direct local TypeScript invocation: `node node_modules/typescript/bin/tsc --noEmit` from `apps/web`.
+- Direct local Vitest invocation: `node node_modules/vitest/vitest.mjs run` from `apps/web`.
+- Web suite now has 8 files and 20 passing tests. Coverage includes initial/stale query failures, strict ACL ports/CIDRs/ranges, bounded API retries and invalidation of derived agent policies after list mutations.
+- The development-only `/test/interaction.html` harness is outside the production entry graph. The parent task verified 15/15 mounted browser assertions with `#results[data-status=passed]` on isolated Vite `127.0.0.1:15178` and fixture-only fetches. It verifies QueryState DOM identity, focus, local edit state, mount count and recovery; failed assignment GET cannot issue PUT; the real settings page preserves dirty fields through data refresh/error, retains focus and saves once without reverting to stale cache.
+- `pnpm` self-bootstrap is restricted by sandbox symlink permissions; direct local binaries avoid dependency or lockfile changes. Registry CLI docs fetch encountered npm DNS failure; installed component APIs and ReUI MCP inline API were inspected instead.
+- No registry components were overwritten; no dependencies or lockfiles changed. No blanket style or layout rewrite.
+
+## Second checkpoint
+
+- Shell now uses the agreed `GET /api/v1/ui/summary` contract. Full agents/lists/policies catalogs are enabled only while the command palette is open and the matching permission is present, with 60-second catalog freshness. **Backend summary endpoint integration remains pending at this checkpoint.** A missing/failed endpoint displays an error and unknown/stale state; it never becomes healthy by falling back to empty catalogs.
+- Monitor derives counts from the server summary, uses the actual response timestamp and hides omitted unauthorized sections. Object search uses the shared five-minute online predicate. Healthy/warning trigger copy follows actual monitor data.
+- All central GET query functions pass TanStack cancellation signals into fetch. Optimistic agent/status and policy-set writes cancel pending reads first. Settings saves cancel pending reads before accepting their new cache value. Central mutation dependency invalidation covers shell counts, policy counts, assignments and derived agent previews.
+- HTTP errors preserve status. Authorization, validation and missing-resource errors do not retry; transient failures retry at most once; cancellations never retry.
+- Port ACL uses shared canonical validation with inline field errors and submit handling. Imports reject partial/reversed ranges, exponent notation, out-of-range ports and more than 50 entries. ACL/host/catalog failures show retry states. Read-only ACL actions are disabled.
+- Host firewall, ACL, blocked-IP and blocked-port detail components load through independent lazy boundaries. Agent-list clicks navigate directly to detail.
+- High contrast now changes theme-aware border/input/ring/muted-text tokens only when the existing user setting explicitly selects high contrast. Standard theme tokens are unchanged.
+- Astra review corrections preserve QueryState child identity across success/error; keyboard palette selection scrolls into view; disabled sortable handles retain `tabIndex=-1` after DnD attributes.
+
+### Build evidence
+
+Production Vite build passes (existing registry `use no memo`/source-map warnings remain). Compared with the first safety checkpoint, the agent detail route chunk fell from 54.10 KB (14.50 KB gzip) to 29.34 KB (8.82 KB gzip), with lazy ACL 19.91 KB / 6.35 KB gzip, host firewall 5.91 KB / 2.06 KB gzip, blocked IP 3.12 KB / 1.54 KB gzip and blocked ports 2.64 KB / 1.23 KB gzip. Shared main chunk is 334.94 KB / 104.04 KB gzip (prior checkpoint 332.67 KB / 103.31 KB gzip); CSS is 176.38 KB / 27.12 KB gzip (prior 175.57 KB / 27.01 KB gzip). These are chunk measurements, not a claim that the total initial network transfer decreased.
+
+Summary integration was subsequently verified by the parent task against the isolated API: the shell loads the endpoint without 404 and reports zero problems for the empty fixture database. No production deployment has been performed.
+
+## Analytics and diagnostics checkpoint
+
+- The shell now imports the exported shared `UiSummary` type; the duplicate frontend DTO has been removed.
+- Fleet and statistics charts read the shared, runtime-validated `TimeseriesResponse` contract from `/api/v1/stats/timeseries`. Requests use completed UTC half-open buckets and cancellation signals. Missing packet metrics remain null and create gaps; unsupported presence/hit snapshots do not become packet counts. The UI labels accepted traffic as permissions granted by EvoFirewall rules, with a note that other host rules may still block it.
+- Statistics keeps the KPI → tabs → chart/table structure. Period presets are 1 hour at 1 minute, 24 hours at 1 hour, 7 days at 1 hour and 30 days at 1 day. Optional selections are capped at 100 agents. Packet KPIs sum known buckets only and say so. Coverage, estimated/partial/truncated buckets, lost reports and unsupported metrics are visible. Legacy raw samples remain a separately labeled table with no client-generated hourly peaks.
+- CSV uses the same bounded request, the existing bearer-header/401/error flow and a 2 MiB client response cap. Tokens never enter URLs or filenames. Agent-name selections apply immediately; downloads abort on page unmount.
+- The new Frame diagnostics section uses `agent.diagnostics` directly: authoritative state, separate contact/telemetry/application timestamps including seconds, desired/applied revision and hash, structured apply error, capabilities, quality and detail counts. Missing diagnostics or missing acknowledgments remain explicitly unknown. Agent detail refreshes every 30 seconds.
+- External list health shows the last **successful** `refreshed_at`, retained entry count and current failure. It no longer substitutes the general `updated_at` timestamp for a successful source refresh. Manual-list modification time remains separate.
+- The UI contract now records the user's approved current-layout/style exceptions, installed ReUI version and opt-in accessibility behavior. No registry components, dependencies or lockfiles were overwritten.
+
+### Verification
+
+- TypeScript passes. Web suite: **28 tests in 11 files**, including UTC boundaries/limits, null metrics and coverage, diagnostics truthfulness and authenticated CSV validation.
+- Parent browser run: **23/23 mounted harness checks**, `#results[data-status=passed]`. New assertions cover drift versus contact, absent telemetry/apply acknowledgment, partial/truncated/lost-report quality, null packet gap display and unchanged null values. A real PortalShell fixture with an agents-read-only token checks one summary request, no closed-palette full catalog requests, hidden unauthorized navigation and omitted summary metrics.
+- ReUI audit checklist retrieved. Targeted C1–C4 searches for new UI code returned no animation/raw palette/native table/select/overlay/Date.now/Radix-asChild issues. No new animation was added. No lint script is configured in `apps/web`; lint was not introduced. Mounted checks are behavioral verification; full responsive visual screenshots and interactive CSV download checks remain separate.
+- Production build passes with existing registry directive/source-map warnings. This checkpoint: stats route 9.46 KB / 4.14 KB gzip; shared stats-window/chart chunk 10.91 KB / 4.37 KB gzip; agent detail route 34.64 KB / 10.08 KB gzip; shared main chunk 336.94 KB / 104.68 KB gzip; CSS 176.42 KB / 27.13 KB gzip. These are chunk sizes, not total initial transfer measurements.
+- Live integration was subsequently verified by the parent task: minute view showed 600 dropped, 1800 allowed, 100% coverage and 60/60 measured intervals, with no unsupported/partial badges after the traffic-only quality fix. A direct agent URL showed distinct desired/applied/contact/telemetry fields. The CSV endpoint returned HTTP 200 `text/csv` with 60 data rows and the browser button was clicked without error; the downloaded file was not captured. The automated harness itself still uses controlled fixtures. No HAR, responsive baseline or performance improvement claim is implied.
+
+## F1 Explain, temporary overrides and forms checkpoint
+
+- Explain uses the shared request/response schemas in a FormSheet, with strict source IP and paired port/protocol validation, input/forward, DNAT and connection state controls. The result keeps desired policy, last acknowledged revision/hash and incomplete external-firewall evidence distinct. Null/external/unsupported/transport-unknown decisions remain unknown. Logical witnesses and bounded match counts are displayed, with localized capability/scope limits. Editing inputs leaves the previous result labeled as stale until explicitly checked again. It is a read operation.
+- Overrides now support an optional reason and either a bounded duration or an explicit future date within 30 days. Validation emits only the selected expiry field, uses shared canonical CIDR validation and rejects invalid TTLs. The server owns author identity. Author, reason, expiry and expired history are shown. Copy explicitly states that expiration changes desired policy and takes effect on the next policy sync, including delayed enforcement for offline agents. Failed writes retain dirty fields; successful writes reset them; closing the sheet keeps its in-memory draft in the current agent card.
+- Statistics replaces the raw UUID textbox with the installed Base UI name/hostname combobox and chips, keyboard behavior, visible selection count, a 100-agent limit, permission/error states and a reset-to-fleet action. IDs remain internal query values.
+- Added mean packets/second, peak **bucket-average** packets/second and managed-rule drop ratio. These require complete traffic coverage, non-null metrics and no partial-traffic flag. A zero denominator gives unknown drop ratio. Partial tooltip pps is unknown. Estimated rates are labeled. The previous period has the exact same duration, resolution and selected agents; percentage comparisons are withheld for incomplete coverage, changed response cohorts, unknown measurements or a zero baseline.
+- Port ACL edit and import forms now use real React Hook Form, zodResolver, Controller/register, FieldError and the existing FormSheet. The previous live manual field state/submit handling was removed. Schema validation still canonicalizes the request at mutation time. Pending writes lock form dismissal; dirty dismissal requires discarding the draft; catalog refreshes do not reset edits. Imports validate source selection and strict ranges before submitting.
+- Explain and override/clone forms load lazily, preloaded on button focus/pointer intent. A loading status is accessible. After first opening, components stay mounted when closed so draft state is preserved; forms are keyed to the current agent.
+
+### F1 verification
+
+- TypeScript and production build pass; existing registry `use no memo` and source-map warnings remain. **34 Vitest tests in 12 files** pass. New cases cover Explain transport pairing/unknown semantics, expiry bounds/field exclusivity, import validation, complete-coverage rate gating and zero/unknown comparison baselines.
+- Parent browser verified `/test/f1.html`: **16/16 PASS**, alongside the earlier 23-check harness. It exercises invalid/reversed Port ACL input, a valid submit, dirty catalog refresh, invalid/valid import, keyboard combobox opening and name selection, Explain invalid input/unknown result/dirty-result labeling, expiry metadata, invalid TTL and failed/successful override writes with draft preservation.
+- The development F1 harness exposes React Profiler and fixture request counts in `#metrics`. The parent-observed run reported 156 commits and 436.2000000476837 ms total `actualDuration`; fixture counts were GET agents 1, GET explain 1, GET overrides 2, POST overrides 2 (one rejected, one successful). These describe that current development fixture run only, with no before/after performance conclusion.
+- The parent also exercised Explain against the isolated real QA API, including after a 60-second TTL expired: the desired result and older applied acknowledgment remained separate and external uncertainty was visible. The form issued no writes.
+- Targeted new-code ReUI C1–C4 checks pass without new animation, native table/select, raw state colors or overlay markup. No dependencies, lockfiles, registry internals or backend/shared files were edited in this batch. Lint remains unavailable because no web lint script is configured. Responsive screenshots, HAR and browser-downloaded CSV file inspection remain unclaimed. Independent Astra review follows this bounded checkpoint.
+- Final F1 build measurement: agent detail route 31.04 KB / 9.20 KB gzip; lazy Explain 11.81 KB / 4.47 KB gzip; lazy override/clone settings 9.70 KB / 3.72 KB gzip; ACL chunk 21.94 KB / 7.10 KB gzip; stats route 21.92 KB / 8.50 KB gzip; CSS 176.47 KB / 27.13 KB gzip. Within this F1 batch, moving the rare forms out of the eager agent route reduced that route chunk from 49.39 KB / 14.18 KB gzip before the boundary change. This is a chunk-level observation, not an initial-page network result.
+- `node scripts/bench/bundle.mjs` on Node **v25.5.0** measured all built JS/CSS assets across routes as **1,855,385 bytes**, **569,103 bytes gzip**. It is not initial transfer, a production baseline, a HAR trace or a performance-improvement claim. Docker's Node 22 runtime remains separately unverified in this frontend checkpoint.
+
+## F2/F3 snapshots, webhooks, scoped credentials and materialized groups
+
+- Existing rules, settings and agent layouts remain. Snapshots/canary, webhook administration, credential lifecycle and labels/group-assignment forms load lazily and preload on focus/pointer intent. Table-heavy forms use the FormSheet wrapper's explicit wide size with min-width containment; the registry Sheet/DataGrid/Frame implementations are preserved.
+- Snapshot creation selects a named staging agent. Target and canary selectors show names, cap selections at 100 and restrict canaries to selected targets. A conflict-free diff for the exact current snapshot and complete target set is required before publication. Creation sends the exact preview generations; confirm/rollback use the displayed rollout version, and release uses the desired revision. A successful publication remains pending until fresh exact server ACK evidence appears. Confirmation is always manual. Rollback describes a newly published prior desired policy, without asserting kernel success. Pin release warns that current live policy is validated before resuming it. Snapshot/history selectors explicitly show the latest 50 records.
+- Webhooks require settings-admin permission, display endpoint state and pending/dead/dropped queue health, and provide create/edit/delete plus endpoint deliveries. New endpoints default disabled. Secrets use a password field, require 32–256 characters on creation and are omitted when unchanged on edit. Editing maps only editable fields into the strict RHF schema; the mounted harness caught and verified the fix for metadata accidentally blocking PATCH. Missing encryption configuration disables writes with an explanation. No real webhook was sent during QA.
+- Credential status uses the shared runtime schema. A rotation request is separate from agent exchange/completion, requires supported approved agents and explicit confirmation, and displays request expiry. Install-link lifecycle metadata shows expiry, consumption and revocation; the panel omits commands/URLs from its query cache and displays no tokens or credentials. New-agent copy explains single use and a 24-hour expiry. Settings render the signing master as a write-only password, preserve the mask and explain that replacement invalidates pending invitation credentials; the settings cache is masked after saving.
+- Label metadata supports create/edit/delete; agent label drafts survive refetch and rejected writes. Group policy assignment selects either explicit named agents or an AND label selection, with explicit add/replace modes and ordered set names. Preview resolves the materialized cohort; apply uses its exact agent IDs/generations and requires a separate confirmation. Changes to labels do not dynamically change set assignments. A rejected CAS invalidates the preview, requiring a new one.
+- Large list detail and its breadcrumb loader use the shared bounded page contract, query keys include page/search/filters and requests forward AbortSignal. Catalog reads request metadata-only summary view. ResourcePage manual pagination uses server totals and disables page-local filtering/sorting. Search/filtering covers the full SQL collection; a same-list previous page stays visible while updating to preserve typing focus. POST/DELETE entry mutations request and runtime-validate the bounded summary response. A mixed valid/invalid import is blocked inline before any request, preserving the draft; valid duplicates can be normalized. Truncated resolved-CIDR previews keep the full count and an explicit shortened-preview label.
+
+### F2/F3 verification checkpoint
+
+- **42 Vitest tests in 14 files** pass, including snapshot/diff identity and canary-ACK action gates, write-only secret omission, link lifecycle, all-or-nothing import validation, bounded query parameters/cancellation and a 100,000-row server-pagination rendering regression. TypeScript and the production build pass, with the existing registry directive/source-map warnings. No web lint script is configured.
+- Parent browser confirmed **21/21 mounted checks** at `/test/f2.html`, alongside the earlier 23-check and F1 16-check harnesses. F2 covers missing webhook key/admin permission, inline secret validation, failed/successful writes, blank-secret edit, dirty label refetch/rejected save, group preview/explicit CAS, diff-before-canary, pending vs fresh ACK, manual confirm/version, release warning and rotation request status. Every request is fixture-intercepted; no external webhook is delivered. Failed harness runs keep the mounted fixture and fake fetch for inspection.
+- Parent also exercised the real isolated QA API through the UI: captured a snapshot, previewed two targets, published only the canary, supplied a controlled telemetry ACK, manually confirmed the remaining target and rolled back. Applied revisions/unknown values remained separate from desired revisions and pending ACK. Desktop Sheet width was corrected from the inherited 384px constraint to the wrapper's wide size; subsequent mobile/responsive evidence is recorded separately by the parent.
+- ReUI audit checklist and focused C1–C4 checks were reviewed. New code uses installed Frame/ResourcePage/FormSheet/Combobox/Alert/Badge controls, semantic variants, no new animation, raw state palette, native table/select or custom overlay. Wall-clock reads are limited to credential/link expiry semantics and documented in that file. No dependency or lockfile changes. Node 22 Docker, production fleet apply behavior, HAR traces and performance improvements are not inferred from these local checks.
+- Final F2/F3 production measurement on Node **v25.5.0**: all built JS/CSS assets **1,908,807 bytes / 590,263 bytes gzip**. Compared with the parent's pristine `aa475848` measurement **1,785,684 / 539,471**, this is **+6.90% / +9.42%**; compared with F1 **1,855,385 / 569,103**, **+2.88% / +3.72%**. The document-referenced static bootstrap assets (entry script, modulepreloads, stylesheet; not a HAR or route-complete transfer) total **871,972 / 243,973**, versus pristine **858,774 / 240,019** (**+1.54% / +1.65%**) and F1 **867,475 / 242,654** (**+0.52% / +0.54%**). Agent detail is **32,300 / 9,519**; lazy new forms: labels **12,034 / 4,587**, rollouts **11,461 / 4,493**, webhooks **9,246 / 3,694**, credentials **5,550 / 2,546**. These are bundle-size observations; added features increased total assets.