Compare commits

...
33 Commits
Author SHA1 Message Date
Denozordec 480756d832 feat(settings): enhance revision retention minutes validation
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 29s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / release (push) Successful in 3m14s
Introduced a preprocessing function to normalize input for the revision retention minutes field, ensuring it handles various input types correctly. Updated the schema to utilize this new validation method, improving data integrity and user experience.
2026-06-01 14:35:51 +07:00
Denozordec 135fb34e00 fix(pgmonitor): standardize field alignment in QueriesResponse struct
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 54s
CI / bird2 (push) Successful in 15s
CI / release (push) Successful in 3m46s
2026-06-01 14:18:03 +07:00
Denozordec 9efa3bbc8a feat(db): enhance PostgreSQL statistics monitoring and error handling
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 30s
CI / go (push) Failing after 24s
CI / bird2 (push) Has been skipped
CI / release (push) Has been skipped
Updated the PostgreSQL monitoring service to improve handling of `pg_stat_statements` availability. Introduced a new method to check if the extension is queryable and updated the response structure to include availability status and hints. Enhanced the documentation to clarify the requirements for enabling `pg_stat_statements`. Adjusted related components to reflect these changes, ensuring better user feedback in the monitoring interface.
2026-06-01 14:15:38 +07:00
Denozordec fad2bd3353 feat(db): implement PostgreSQL monitoring and maintenance features
CI / changes (push) Successful in 9s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 26s
CI / web (push) Successful in 33s
CI / go (push) Successful in 2m11s
CI / bird2 (push) Successful in 16s
CI / release (push) Successful in 3m27s
Added PostgreSQL monitoring and maintenance capabilities to the API, including new endpoints for instance-level metrics, maintenance operations, and job scheduling. Updated the HTTP API to support PostgreSQL monitoring routes and integrated a background scheduler for metrics collection. Enhanced the CLI with database commands for maintenance tasks. Updated documentation to reflect these changes.
2026-06-01 13:43:33 +07:00
Denozordec 930e42b0b0 fix(db): update prefix handling in module_prefix_snapshot_row
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 55s
CI / bird2 (push) Successful in 14s
CI / release (push) Successful in 3m21s
Modified the prefix column type in the module_prefix_snapshot_row table to TEXT, allowing for more flexible input. Adjusted related SQL queries and Go struct tags to ensure compatibility with JSON serialization. Cleaned up migration logic to handle prefix and community_id fields more robustly.
2026-05-25 11:16:03 +07:00
Denozordec 16b4923bd7 fix(db): change prefix column type to TEXT in prefix_snapshot_row
CI / changes (push) Successful in 8s
CI / openapi (push) Has been skipped
CI / commitlint (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 54s
CI / bird2 (push) Successful in 14s
CI / release (push) Successful in 3m44s
Updated the prefix column in the prefix_snapshot_row table from CIDR to TEXT to accommodate broader input formats. Adjusted related SQL insert statements accordingly.
2026-05-25 11:08:06 +07:00
DenozordecandCursor 1cfd062835 chore(db): drop unused schema and redundant indexes
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 54s
CI / bird2 (push) Successful in 14s
CI / release (push) Successful in 3m44s
Удалены мёртвые колонки, fetch_log и избыточные индексы.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:58:57 +07:00
DenozordecandCursor 21233bd578 chore(db): add job_audit brin index
BRIN-индекс по created_at для retention-удалений.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:58:57 +07:00
DenozordecandCursor 990cc739df refactor(db): normalize asn prefix cache and ttl cleanup
Строки asn_prefix_cache_row; периодический prune через evobgp-ingest.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:58:57 +07:00
DenozordecandCursor 3500bd4624 refactor(db): normalize module prefix snapshot rows
Строки префиксов в module_prefix_snapshot_row вместо JSONB blobs.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:58:56 +07:00
DenozordecandCursor 374575ec01 refactor(db): mark prefix snapshot contract migration
Маркер миграции contract-фазы H2.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:58:01 +07:00
DenozordecandCursor f57b430052 feat(db): backfill prefix snapshots and switch read path
Backfill миграция; чтение префиксов через snapshot; запись без дублирования rows.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:58:01 +07:00
DenozordecandCursor 8a9d60a5a7 feat(db): add prefix snapshot tables (expand)
Таблицы prefix_snapshot и prefix_snapshot_row; колонка prefix_snapshot_id.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:58:01 +07:00
DenozordecandCursor b963311b43 perf(db): keyset pagination for revision prefixes
Keyset по id/ord вместо OFFSET; anti-join для RevisionDiff.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:56:46 +07:00
DenozordecandCursor 50bdb8232b refactor(db): split revision preview from meta_json
Preview BIRD-фрагменты в config_revision_preview; meta_json только счётчик префиксов.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:55:52 +07:00
DenozordecandCursor ee8e24ffc6 perf(db): add revision and module query indexes
Индексы для diff/list префиксов и сортировки модулей; covering index для pagination.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:55:10 +07:00
DenozordecandCursor 44b94caacf docs(db): add postgres schema diagnostics runbook
Runbook для оценки размеров таблиц, seq scan и EXPLAIN перед/после миграций схемы.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:55:02 +07:00
DenozordecandCursor cbb4b467ad fix(agentserver): hide upstream errors from clients
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 58s
CI / bird2 (push) Successful in 14s
CI / release (push) Successful in 3m23s
Generic 502 detail как в httpapi; добавлен docs/production-checklist.md.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:34:03 +07:00
DenozordecandCursor e65cf0d958 feat(jobs): persist job lifecycle to PostgreSQL job_audit
UpsertQueued/Running/MarkTerminal через SetPersistHooks; исправлен deadlock
fireEnqueued под Registry mutex.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:33:57 +07:00
DenozordecandCursor 4a57c91e29 perf(pipeline): parallel prefetch and lean revision GET
GetRevisionSummary без preview_fragments; boundedRepoCtx для GetRevision;
parallel CDN prefetch с ctx из ingest.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:33:53 +07:00
DenozordecandCursor 2289107911 feat(httpclient): add circuit breaker for CDN and RIPEstat
Per-host circuit breaker с retry для CDN fetch и RIPEstat; порог 5 ошибок,
cooldown 30s.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:15:32 +07:00
DenozordecandCursor 782097420d fix(httpclient): replace DefaultClient with timed clients and retry
Пакет httpclient: timeout 45s, idle pool, DoWithRetry. Scheduler и nodecli
используют retry; pipeline/asnresolve/jobs — httpclient.New вместо DefaultClient.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:14:53 +07:00
DenozordecandCursor 82382d90f2 fix(pipeline): validate CDN URLs against SSRF
HTTPS-only CDN URLs; блокировка private/loopback/metadata IP и DNS-resolve
на fetch; проверка в httpapi при create/preview/patch CDN sources.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:14:10 +07:00
DenozordecandCursor 5a16a45922 fix(httpapi): redact agent_secret from speaker list responses
Секрет agent больше не отдаётся viewer/editor в GET/list speakers;
остаётся только при создании спикера.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:13:28 +07:00
DenozordecandCursor 6a6f6cedbc feat(pipeline): use stale snapshot when upstream fetch fails
При ошибке CDN/ASN/DoH ingest использует последний снимок префиксов
(или просроченный ASN-кэш), если EVOBGP_STALE_ON_UPSTREAM_ERROR не равен 0.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-25 10:13:07 +07:00
Denozordec 9639a03bfe feat(store): add test for clearing dispatch error on successful merge
CI / changes (push) Successful in 9s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 39s
CI / go (push) Successful in 58s
CI / bird2 (push) Successful in 16s
CI / release (push) Successful in 4m18s
- Introduced a new test case in speaker_meta_test.go to verify that the LastDispatchError is cleared when a successful dispatch status is merged.
- Updated MergeSpeakerMetaJSON function in speaker_meta.go to clear LastDispatchError if LastDispatchStatus is "ok".
- Enhanced speakerDispatchError function in network-metrics.ts to handle stale dispatch errors after successful agent sync.
2026-05-21 18:10:51 +07:00
DenozordecandCursor 48c10b7436 refactor(web): trigger release for network dashboard layout fixes
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 40s
CI / bird2 (push) Successful in 16s
CI / release (push) Successful in 3m37s
Follow-up для semantic-release: правки overview/sheet и dispatch-ошибок уже в fb108ec, заголовок с запятой в scope не парсился. Уточнена формулировка verify в releasing.md.

Semver: patch.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 17:56:03 +07:00
Denozordec 4db6438245 remove commitlint configuration and update release documentation to enforce single scope in commit messages
CI / openapi (push) Has been cancelled
CI / web (push) Has been cancelled
CI / changes (push) Has been cancelled
CI / go (push) Has been cancelled
CI / bird2 (push) Has been cancelled
CI / commitlint (push) Has been cancelled
CI / release (push) Has been cancelled
2026-05-21 17:54:41 +07:00
Denozordec fb108ec5ab refactor(NetworkOverviewTab, NetworkSpeakerDetailSheet, NetworkSpeakerStatusCard): improve layout and error handling
CI / changes (push) Successful in 9s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 39s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / release (push) Successful in 19s
- Refactored the layout of NetworkOverviewTab and NetworkSpeakerDetailSheet for better structure and readability.
- Enhanced error handling in NetworkSpeakerDetailSheet by introducing new error types for dispatch and agent errors.
- Updated NetworkSpeakerStatusCard to improve the display of speaker information and status.
- Adjusted styles in various components to ensure consistent spacing and alignment.
- Modified network-metrics.ts to include new functions for formatting speaker errors, improving user feedback on dispatch issues.
2026-05-21 17:50:27 +07:00
Denozordec a1ada06a76 feat(api): add live status tracking for speakers and BGP sessions
CI / changes (push) Successful in 9s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 24s
CI / web (push) Successful in 29s
CI / go (push) Successful in 43s
CI / bird2 (push) Successful in 15s
CI / release (push) Successful in 3m29s
- Introduced new schemas for `SpeakerLiveStatus`, `BgpSessionLive`, and `LiveSpeakerPoll` in OpenAPI documentation to support live status queries.
- Enhanced the `/v1/speakers` endpoint to include a `live` query parameter, allowing retrieval of real-time speaker and BGP status.
- Updated the HTTP API to collect and return live status data for speakers, improving monitoring capabilities.
- Modified frontend components to display live status information, enhancing user visibility into speaker health and BGP session states.
- Added a new endpoint `/v1/bird/status` for retrieving the local BIRD status, further enriching the network monitoring features.
2026-05-21 17:36:50 +07:00
Denozordec b5ed47902c feat(api): enhance peer session tracking and error handling
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 35s
CI / go (push) Successful in 52s
CI / bird2 (push) Successful in 16s
CI / release (push) Successful in 3m58s
- Added `PollError` field to `peerSessionOnSpeaker` and `liveSpeakerPoll` types to capture polling errors for speakers.
- Updated `matchPeerOnSpeakers` function to handle polling errors and adjust session state reporting.
- Modified frontend components to display polling error messages alongside session states, improving user visibility into peer connection statuses.
- Enhanced API response structure to include live speaker polling information, facilitating better monitoring of speaker health.
2026-05-21 15:40:44 +07:00
Denozordec 8a19c2a3f4 refactor(NetworkPeersCard): improve sessionBadge function formatting
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 36s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / release (push) Successful in 3m21s
- Reformatted the sessionBadge function for better readability by adjusting the parameter layout.
- No functional changes were made; this is purely a code style improvement.
2026-05-21 15:22:10 +07:00
Denozordec 1c0d78b552 feat(api): enhance peer session management and documentation
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 27s
CI / web (push) Failing after 35s
CI / go (push) Successful in 55s
CI / bird2 (push) Successful in 15s
CI / release (push) Has been skipped
- Added new fields to the API for tracking connected speakers and session states across multiple nodes, including `connected_speaker_id`, `connected_speaker_label`, `session_on_speakers`, `established_on_speakers`, and `session_mismatch`.
- Implemented a new endpoint for retrieving bird protocol sessions, enhancing the agent server functionality.
- Updated the OpenAPI documentation to reflect the new fields and query parameters, improving clarity for API consumers.
- Modified the frontend to display connected speaker information and session states, providing better visibility into peer connections.
- Updated deployment documentation to clarify the configuration requirements for enabling IP forwarding on VPS.
2026-05-21 15:18:21 +07:00
143 changed files with 8417 additions and 741 deletions
-3
View File
@@ -1,3 +0,0 @@
{
"extends": ["@commitlint/config-conventional"]
}
@@ -0,0 +1,408 @@
---
name: Технический аудит EvoBGP
overview: "Полный технический аудит EvoBGP для production-сценария (10+ клиентов, нестабильная сеть). Архитектура — hybrid control plane; сильные стороны: stale fallback, CDN/RIPEstat resilience, Ed25519 bundles. Критичные риски: in-process jobs, DoH без retry, misconfiguration demo-seed, отсутствие HA API."
todos:
- id: quick-ops-checklist
content: Применить production-checklist (SEED_DEMO=0, BUNDLE_SEED_HEX, DB/JOB/CONCURRENCY tuning, TLS)
status: pending
- id: fix-doh-retry
content: Добавить DoWithRetry для DoH в internal/pipeline/refresh.go
status: pending
- id: fix-job-meta-race
content: Исправить чтение j.Meta в worker.go через Snapshot() или locked accessor
status: pending
- id: cdn-preview-resilience
content: Перевести CDN preview на upstreamHTTPDo в routes_crud.go
status: pending
- id: partial-cdn-failure
content: "Partial CDN source failure: skip/degrade вместо fail всего модуля"
status: pending
- id: graceful-shutdown
content: Cancel/drain jobs при SIGTERM в cmd/evobgp-api и evobgp-all
status: pending
- id: ha-job-queue
content: "Roadmap: распределённая очередь jobs (PG claim или NATS) для HA API"
status: pending
isProject: false
---
# Технический аудит EvoBGP
## Executive summary
EvoBGP — **hybrid control plane**: один процесс [`evobgp-all`](cmd/evobgp-all/main.go) (monolith) или **reference Compose** с разделёнными воркерами ([`docs/architecture.md`](docs/architecture.md)). Data plane (BIRD + agent) отделён от control plane (API + PostgreSQL + jobs).
**Сильные стороны для нестабильной сети:**
- Stale snapshot fallback по умолчанию (`EVOBGP_STALE_ON_UPSTREAM_ERROR=1`) — [`internal/pipeline/collect_stale.go`](internal/pipeline/collect_stale.go)
- CDN/RIPEstat: retry (3×) + per-host circuit breaker — [`internal/httpclient/httpclient.go`](internal/httpclient/httpclient.go), [`circuit.go`](internal/httpclient/circuit.go)
- ETag conditional GET, ASN TTL-кэш, parallel collect с cap
- Подписанные бандлы Ed25519, verify перед apply
**Главные риски для 10+ клиентов:**
1. `jobs.Registry`**in-memory, только в процессе API** (ARCH-04)
2. DoH — **без retry/breaker** (критично при блокировках провайдеров)
3. Один failed CDN source **без stale cache валит весь модуль**
4. Production misconfiguration: `Bearer dev`, HTTP API, ephemeral bundle key
5. Data race на `Job.Meta` и alias pointers в `store.Memory`
---
## 1. Архитектура
### Стиль
```mermaid
flowchart TB
subgraph hybrid [Hybrid deployment]
All[evobgp_all monolith]
Split[evobgp_api + workers]
end
subgraph cp [Control plane]
API[HTTP API]
Jobs[jobs.Registry in-process]
PG[(PostgreSQL)]
end
subgraph dp [Data plane per speaker]
Agent[evobgp_agent]
BIRD[BIRD2]
NodeCLI[evobgp_node]
end
All --> API
Split --> API
API --> Jobs
API --> PG
NodeCLI --> API
Agent --> API
Agent --> BIRD
```
| Профиль | Стиль | Когда |
|---------|-------|-------|
| `microvps` / `evobgp-all` | Monolith | 1 VPS, shared Registry |
| reference Compose | Microservices-lite | API + scheduler/ingest/render/deploy |
| Remote speakers | Edge agents | Panel→Node dispatch |
### Узкие места (bottlenecks)
| # | Bottleneck | Где | Impact |
|---|------------|-----|--------|
| B1 | **In-process job queue** | [`internal/jobs/job.go:175-177`](internal/jobs/job.go) | HA API невозможен без потери/дублирования jobs; scheduler без `EVOBGP_CONTROL_PLANE_URL` создаёт **отдельный Registry** — [`cmd/evobgp-scheduler/main.go:58-60`](cmd/evobgp-scheduler/main.go) |
| B2 | **Module refresh = sync upstream fan-out** | [`internal/pipeline/collect_parallel.go`](internal/pipeline/collect_parallel.go) | До `EVOBGP_COLLECT_CONCURRENCY` (8 default, max 32) параллельных HTTP; worst case ~45s × retries на источник |
| B3 | **Default job concurrency = 8** | [`internal/jobs/job.go:264-268`](internal/jobs/job.go) | При burst refresh 10+ tenants — очередь растёт, goroutine блокируются на sem |
| B4 | **PostgreSQL pool default** | [`internal/db/open.go:28-38`](internal/db/open.go) | pgx default ~4 conns; при `JOB_MAX=16` + HTTP — contention без `EVOBGP_DB_MAX_CONNS=25` |
| B5 | **Live endpoints fan-out** | [`internal/httpapi/peers_live.go`](internal/httpapi/peers_live.go) | N goroutines × N speakers, 12s timeout каждый |
| B6 | **Broker — заглушка** | [`internal/broker`](internal/broker) | NATS URL логируется, очередь не распределена |
### Масштабируемость
- **Вертикальная:** хорошо до ~10–20 tenants при `evobgp-all` + tuning ([`docs/production-checklist.md`](docs/production-checklist.md))
- **Горизонтальная API:** **не поддерживается** — два `evobgp-api` = два независимых Registry; `job_audit` в PG — audit only, не очередь исполнения
- **Workers (ingest/render/deploy):** координируются через **общую БД**, не через jobs — OK для prefetch/drift
### Отказоустойчивость
| Сценарий | Поведение | Оценка |
|----------|-----------|--------|
| CDN/RIPEstat недоступен | Stale snapshot + circuit breaker | **Хорошо** (если был prior snapshot) |
| DoH недоступен | Fail модуля или stale domain snapshot | **Средне** (нет HTTP retry) |
| API restart mid-job | Job теряется из Registry; audit может быть inconsistent | **Плохо** |
| PG недоступен | API `/v1/ready` → 503 | **OK** |
| Agent unreachable | Deploy job succeed, drift в `evobgp-deploy` | **Частичный fail** (by design) |
**Рекомендация:** для 10+ клиентов — **`evobgp-all` на каждом CP** или один CP + tuning; HA API требует **распределённой очереди** (NATS/Redis + worker pool) — задокументировано как future work.
---
## 2. Анализ кода
### Антипаттерны
| ID | Проблема | Файл | Критичность |
|----|----------|------|-------------|
| A1 | **Concurrent map read/write** — worker читает `j.Meta` без lock, handler пишет через `mergeMeta`/`Snapshot` | [`worker.go:108,263,379`](internal/jobs/worker.go), [`job.go:103-111`](internal/jobs/job.go) | **high** |
| A2 | **Escape internal pointers** из Memory store | [`store/memory.go:416-475`](internal/store/memory.go) | **high** (tests/dev); **low** (prod PG) |
| A3 | **Fire-and-forget goroutine** на каждый auth | [`auth.go:79-81`](internal/httpapi/auth.go) | **medium** |
| A4 | **Silent error swallow** в prefetch | [`internal/ingest/run.go`](internal/ingest/run.go), `prefetch.go` | **medium** |
| A5 | **Bypass resilience layer** — CDN preview прямой `Do` | [`routes_crud.go:267`](internal/httpapi/routes_crud.go) | **medium** |
| A6 | **`EVOBGP_DEV_INSECURE` — dead code** | compose + [`server.go`](internal/httpapi/server.go) | **low** (misleading ops) |
| A7 | **Unused Registry** в ingest/render/deploy binaries | [`cmd/evobgp-ingest/main.go`](cmd/evobgp-ingest/main.go) | **low** (resource waste) |
### Maintainability
**Плюсы:** чёткое разделение слоёв (ARCH-01..10), `store.Backend`, OpenAPI как контракт, engineering rules, table-driven tests в birdfmt/pipeline.
**Минусы:**
- Дублирование retry-логики (httpclient vs nodedispatch inline loop)
- Env-tuning разбросан (`EVOBGP_*` в 15+ местах без central config struct для pipeline)
- `Job` comment «персистенция в БД пока не подключена» устарел — hooks есть в [`bootstrap.go:67-92`](internal/httpapi/bootstrap.go)
### Потенциальные баги и race conditions
1. **`j.Meta` data race** — `-race` на `TestParallelModuleRefresh_*` + concurrent `GET /v1/jobs/{id}` polling
2. **Memory store alias**`deploy.Run` читает `LastAppliedRevisionID` пока worker пишет
3. **peerLiveCache** возвращает slice без копии — [`peers_live.go:82-84`](internal/httpapi/peers_live.go)
4. **TOCTOU idempotency** — terminal job удаляется из `byIdempo`, повторный POST создаст новый job (by design, но клиент должен знать)
### Error handling
**Хорошо:**
- Префиксы ошибок (`httpclient:`, `birdfmt:`)
- HTTP 5xx через `writeProblem`, без raw `err.Error()` (ERR-01)
- `context.Context` в pipeline workers
**Пробелы:**
- `runRollback` без `workContext` — не отменяется — [`worker.go:500+`](internal/jobs/worker.go)
- Prefetch/ingest: ошибки не логируются
- `mergeBirdPostApplyMeta``context.Background()` 8s, игнорирует job cancel
---
## 3. Производительность
### Блокирующие операции
| Участок | Блокировка | Риск |
|---------|------------|------|
| `POST .../cdn-sources/preview` | Sync CDN fetch до 45s в HTTP handler | UI timeout, worker starvation |
| `GET /v1/peers/live` | N × agent HTTP, wg.Wait | Slow при многих speakers |
| Module refresh job | Sequential: ingest → render revision → optional deploy | Long job chain |
| `bird -p` / `birdc configure` | Subprocess в deploy | Disk I/O на ноде |
### Неэффективные алгоритмы / лишние запросы
- **Tenant refresh:** `aggregateTenantPrefixRowsAll` — parallel по модулям, но каждый модуль может refetch все CDN/ASN/DoH — [`aggregate.go:28+`](internal/pipeline/aggregate.go). Snapshot skip есть через `module_hash` — проверять hit rate в meta.
- **ASN resolve:** `PolitePause()` 150ms между AS — [`asnresolve/ripestat.go`](internal/asnresolve/ripestat.go) — при 50 AS = +7.5s minimum.
- **GetModulePrefixSnapshot** вызывается многократно в одном refresh (cdn_snapshot, collect_parallel) — potential duplicate DB reads.
- **Auth TouchAPIKeyLastUsed:** UPDATE на каждый request (async) — load на PG при high RPS.
### Кэширование
| Кэш | TTL | Gap |
|-----|-----|-----|
| ASN prefix cache | 1800s (`EVOBGP_ASN_CACHE_TTL_SEC`) | OK |
| CDN ETag in DB | Until 304/change | OK |
| Module prefix snapshot | Content-hash based skip | OK |
| peerLiveCache | In-memory, per-process | Не shared между API replicas; нет defensive copy |
| Circuit breaker state | Per-process | Не shared |
### Конкретные улучшения
```go
// 1. CDN preview — использовать upstreamHTTPDo вместо прямого Do
resp, err := pipeline.UpstreamHTTPDo(r.Context(), s.cdnHTTP, req) // extract upstreamHTTPDo
// 2. Job.Meta — читать под lock или через Snapshot()
st := j.Snapshot()
mid, _ := st["meta"].(map[string]any)["module_id"].(string)
// 3. Memory store — возвращать копии (как Postgres)
modCopy := *mod
return &modCopy, nil
```
---
## 4. Сетевое взаимодействие (критично)
### Текущее состояние
```mermaid
flowchart LR
subgraph resilient [Resilient path]
CDN[CDN fetch]
RIPE[RIPEstat]
CDN --> Breaker[Circuit breaker]
RIPE --> Breaker
Breaker --> Retry[DoWithRetry 3x linear 2s]
end
subgraph fragile [Fragile path]
DoH[DoH resolve]
Preview[CDN preview API]
AgentHealth[Agent health/bird]
DoH --> SingleDo[Single hc.Do]
Preview --> SingleDo
AgentHealth --> SingleDo
end
subgraph fallback [App-level fallback]
Stale[Stale snapshot]
SysDNS[System DNS]
DoH --> SysDNS
CDN --> Stale
RIPE --> Stale
end
```
| Upstream | Timeout | Retry | Breaker | Stale fallback |
|----------|---------|-------|---------|----------------|
| CDN ingest | 45s | 3× linear | per-host | yes |
| RIPEstat | 45s | 3× | per-host | yes + cache |
| DoH | 10s/profile | **no** | **no** | domain snapshot |
| CDN preview | 45s | **no** | **no** | N/A |
| Scheduler→API | 45s | 3× | no | N/A |
| Node dispatch | 30s | inline 3× | no | N/A |
### Пробелы для блокировок провайдеров
1. **DoH без retry** — transient timeout = fail; failover между profiles есть, но каждый profile — single shot
2. **429/408 не ретраятся** — только `>= 500`
3. **Нет jitter** — thundering herd при mass tenant refresh
4. **DNS rebinding TOCTOU** — SSRF check до fetch, HTTP dial без pinned IP — [`cdn_url.go:75-115`](internal/pipeline/cdn_url.go)
5. **Circuit breaker без half-open** — после 30s cooldown сразу full traffic — [`circuit.go:29-33`](internal/httpclient/circuit.go)
6. **Breaker per-process** — ingest container ≠ API container
### Рекомендации для нестабильной сети
| # | Изменение | Effort | Effect |
|---|-----------|--------|--------|
| N1 | DoH через `DoWithRetry` + optional breaker | Low | **High** для DOMAINS modules |
| N2 | Retry 429/503 с `Retry-After` + exponential backoff + jitter | Medium | **High** при rate limits |
| N3 | **Partial CDN failure** — continue с stale per-source, не fail whole module | Medium | **High** |
| N4 | Multiple DoH profiles + `failover` policy (already exists) — **документировать ops playbook** | Low | **High** (config, not code) |
| N5 | Pinned dialer / custom `Transport.DialContext` после SSRF resolve | Medium | **Medium** (SSRF hardening) |
| N6 | Proxy support (`HTTP_PROXY` / `EVOBGP_HTTP_PROXY`) для CDN/DoH | Medium | **High** в censored networks |
| N7 | Unify CDN preview на `upstreamHTTPDo` | Low | **Medium** |
---
## 5. Устойчивость и надёжность
### Graceful degradation
**Работает:**
- `EVOBGP_STALE_ON_UPSTREAM_ERROR=1` — ASN/CDN/domain stale — [`collect_stale.go`](internal/pipeline/collect_stale.go)
- CDN 304 без local cache → forced full GET — [`cdn_snapshot.go:141-159`](internal/pipeline/cdn_snapshot.go)
- DoH → system DNS fallback — [`doh_resolve.go:75-93`](internal/pipeline/doh_resolve.go)
- Deploy: job succeed even if agent wake fails (drift detection)
**Не работает / частично:**
- Один CDN source fail без cache → **весь module_refresh failed** — [`collect_parallel.go:221-223`](internal/pipeline/collect_parallel.go)
- Circuit open → immediate error, stale only if prior data exists
- API shutdown: HTTP drain 15s, **jobs не cancel/drain** — [`cmd/evobgp-api/main.go:67-72`](cmd/evobgp-api/main.go)
### Сценарии отказов
| Событие | Что произойдёт |
|---------|----------------|
| **Потеря CP↔PG** | Ready=false; running jobs fail; no new jobs persist audit reliably |
| **Потеря CP↔CDN** | Stale prefixes если были; иначе job fail; breaker opens 30s |
| **Потеря CP↔agent** | Deploy meta `dispatch_failed`; BIRD на старой ревизии; drift logs |
| **RIPEstat rate limit** | 429 → no retry → stale or fail |
| **Рост нагрузки** | Job queue; goroutine pile-up; PG pool exhaustion; `/metrics` shows queue depth |
| **API restart** | In-flight jobs lost; clients poll 404 or stale terminal state |
---
## 6. Безопасность
| ID | Finding | Severity | Fix |
|----|---------|----------|-----|
| S1 | `Bearer dev` → operator при demo-seed | **high** (misconfig) | `EVOBGP_SEED_DEMO=0` — [`auth.go:66-92`](internal/httpapi/auth.go) |
| S2 | API plain HTTP | **high** (ops) | TLS на edge (Traefik/nginx) |
| S3 | Ephemeral bundle key без `EVOBGP_BUNDLE_SEED_HEX` | **high** (ops) | Stable seed + pubkey на нодах |
| S4 | Compose defaults: weak PG password, `sslmode=disable` | **high** (ops) | Secrets manager, `sslmode=require` |
| S5 | `/metrics` без auth | **medium** | Network policy / mTLS |
| S6 | No rate limiting on auth | **medium** | Middleware limiter (e.g. per-IP) |
| S7 | CDN SSRF DNS rebinding | **medium** | Pinned dialer after resolve |
| S8 | `EVOBGP_CDN_ALLOW_PRIVATE=1` | **medium** | Never in prod |
| S9 | `EVOBGP_NODE_DISPATCH_INSECURE_TLS=1` | **medium** | Valid TLS to agent |
| S10 | Plaintext `EVOBGP_API_KEYS` in env | **medium** | DB keys via API |
| S11 | `editor` can cancel jobs | **low** | Restrict to operator |
| S12 | agent_secret `==` compare | **low** | `subtle.ConstantTimeCompare` |
**SQL injection:** не обнаружено — параметризованные запросы в [`repository/`](internal/repository/).
**Bundle crypto:** Ed25519 корректно; path traversal blocked в tar extract.
---
## 7. Конкретные рекомендации (prioritized backlog)
### High
| # | Описание | Как исправить |
|---|----------|---------------|
| H1 | DoH без retry | Обернуть `hc.Do` в `DoWithRetry(ctx, hc, req, 3)` в [`refresh.go:288,360`](internal/pipeline/refresh.go) |
| H2 | Data race `Job.Meta` | Читать через `Snapshot()` или добавить `MetaLocked()` accessor |
| H3 | CDN source partial failure | В `collectCDNPrefixRows`: при err без stale — log warning + skip source вместо `return nil, r.err` (config flag `EVOBGP_CDN_PARTIAL_OK=1`) |
| H4 | Production checklist enforcement | CI/deploy validation: reject `SEED_DEMO=1`, require `BUNDLE_SEED_HEX` |
| H5 | Job queue HA roadmap | Persist queued jobs in PG + worker claim (`SELECT FOR UPDATE SKIP LOCKED`) или NATS — ARCH-04 |
### Medium
| # | Описание | Как исправить |
|---|----------|---------------|
| M1 | CDN preview bypass | [`routes_crud.go:267`](internal/httpapi/routes_crud.go) → `upstreamHTTPDo` |
| M2 | Retry 429/503 | Extend `DoWithRetry` status check + parse `Retry-After` |
| M3 | Graceful shutdown | On SIGTERM: `Registry.RequestCancelAll()` + wait workers with timeout |
| M4 | Auth goroutine storm | Worker pool или sync touch with debounce |
| M5 | HTTP proxy support | Custom Transport reading `EVOBGP_HTTP_PROXY` |
| M6 | Memory store copies | Defensive copy in Get/List (dev/test safety) |
| M7 | Rate limiting | `golang.org/x/time/rate` on auth middleware |
### Low
| # | Описание | Как исправить |
|---|----------|---------------|
| L1 | Jitter in backoff | `wait + rand.Intn(wait/2)` in DoWithRetry |
| L2 | Half-open breaker | Single probe request after cooldown |
| L3 | Remove dead `EVOBGP_DEV_INSECURE` from compose | Docs + compose cleanup |
| L4 | Prefetch error logging | `log.Printf` or structured log in prefetch |
| L5 | peerLiveCache defensive copy | `append([]T(nil), views...)` on store |
---
## 8. Quick wins (максимальный эффект / минимум усилий)
1. **Ops (0 code):** [`docs/production-checklist.md`](docs/production-checklist.md) — `SEED_DEMO=0`, `BUNDLE_SEED_HEX`, `DB_MAX_CONNS=25`, `JOB_MAX=16`, `COLLECT_CONCURRENCY=16`, TLS edge, restrict metrics
2. **DoH retry** — 510 строк в `refresh.go`, reuse existing `DoWithRetry`
3. **CDN preview → upstreamHTTPDo** — 1 line change in handler
4. **Job.Meta read fix** — replace 4 reads in `worker.go` with `Snapshot()` parsing
5. **Log prefetch failures** — visibility без изменения behavior
6. **Document DoH failover playbook** — multiple profiles (Cloudflare, Google, Quad9) + `failover` policy for censored regions
7. **Run `go test -race ./internal/jobs/...`** in CI — catch Meta race
8. **Prefer `evobgp-all`** over split reference for &lt;20 tenants — eliminates Registry split bug
---
## Диаграмма: refresh под сетевым stress
```mermaid
sequenceDiagram
participant Op as Operator
participant API as evobgp_api
participant Job as module_refresh
participant CDN as CDN_upstream
participant PG as PostgreSQL
Op->>API: POST /modules/id/refresh
API->>Job: Enqueue
Job->>CDN: GET with ETag
alt CDN timeout or 5xx
CDN-->>Job: error after 3 retries
Job->>PG: load prior snapshot
alt stale exists
Job->>PG: CreateRenderRevision stale
Job-->>API: succeeded degraded
else no stale
Job-->>API: failed
end
else CDN 200
CDN-->>Job: new prefixes
Job->>PG: CreateRenderRevision
end
```
---
## Итоговая оценка зрелости
| Область | Оценка | Комментарий |
|---------|--------|-------------|
| Архитектура | 7/10 | Чистые слои; HA/API scaling — слабое место |
| Сеть/resilience | 6/10 | CDN/ASN хорошо; DoH/preview — пробелы |
| Concurrency | 6/10 | Registry продуман; Meta race, shutdown |
| Performance | 7/10 | Parallel collect, caching; tuning needed at scale |
| Security | 6/10 | Crypto OK; ops/config risks dominate |
| Maintainability | 8/10 | Docs, rules, OpenAPI, tests |
**Вердикт:** проект **готов для 10+ клиентов в single-CP deployment** (`evobgp-all` + PostgreSQL + production checklist) при условии ops discipline. Для **multi-CP HA** и **агрессивных сетевых блокировок** — приоритет: DoH retry, partial CDN failure, distributed job queue, HTTP proxy.
+2
View File
@@ -142,6 +142,8 @@ feat(web): add module create dialog on /modules
| `.cursor/` | `chore` |
| прочее в корне | `chore` |
**Запрещено:** несколько scope через запятую (`refactor(web, httpapi): …`) — semantic-release не распознает `type`, релиз не будет (см. [docs/releasing.md](../../docs/releasing.md)).
`type` определять по **содержимому diff**, не только по пути.
## Multi-change
+2
View File
@@ -303,6 +303,8 @@ jobs:
cache-dependency-path: package-lock.json
- name: Install release tooling
run: npm ci
- name: Verify releasable commit messages
run: node scripts/commit/verify-release-commits.mjs
- name: Semantic release
run: npx semantic-release
env:
+5
View File
@@ -12,6 +12,7 @@ import (
"evobgp/internal/birdfmt"
"evobgp/internal/config"
"evobgp/internal/dbcli"
"evobgp/internal/deploy"
"evobgp/internal/httpapi"
"evobgp/internal/ingest"
@@ -24,6 +25,9 @@ import (
// microVPS entrypoint: один процесс — HTTP API и фоновые воркеры scheduler, ingest, render, deploy (общий store и jobs.Registry).
func main() {
if len(os.Args) > 1 && os.Args[1] == "db" {
os.Exit(dbcli.Run(os.Args[2:]))
}
cfg := config.Load()
opts := httpapi.Options{
APIKeys: os.Getenv("EVOBGP_API_KEYS"),
@@ -52,6 +56,7 @@ func main() {
go render.Run(ctx, renderDeps)
go deploy.Run(ctx, deployDeps)
srv.StartBackground(ctx)
startBirdMetricsPoller(ctx)
httpSrv := &http.Server{
+5
View File
@@ -12,6 +12,7 @@ import (
"evobgp/internal/birdfmt"
"evobgp/internal/config"
"evobgp/internal/dbcli"
"evobgp/internal/httpapi"
"evobgp/internal/observability"
"evobgp/internal/platform"
@@ -19,6 +20,9 @@ import (
)
func main() {
if len(os.Args) > 1 && os.Args[1] == "db" {
os.Exit(dbcli.Run(os.Args[2:]))
}
cfg := config.Load()
seedDemo := os.Getenv("EVOBGP_SEED_DEMO") != "0"
opts := httpapi.Options{
@@ -39,6 +43,7 @@ func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
srv.StartBackground(ctx)
startBirdMetricsPoller(ctx)
httpSrv := &http.Server{
+22
View File
@@ -0,0 +1,22 @@
/** @type {import('@commitlint/types').UserConfig} */
module.exports = {
extends: ['@commitlint/config-conventional'],
plugins: [
{
rules: {
'scope-no-commas': ({ scope }) => {
if (scope && scope.includes(',')) {
return [
false,
'scope must not contain commas (semantic-release will not parse the commit type)'
];
}
return [true];
}
}
}
],
rules: {
'scope-no-commas': [2, 'always']
}
};
@@ -27,9 +27,7 @@ services:
network_mode: host
cap_add:
- NET_ADMIN
sysctls:
net.ipv4.ip_forward: "1"
net.ipv6.conf.all.forwarding: "1"
# sysctls нельзя с network_mode: host — включите ip_forward на VPS (см. docs/remote-speakers.md)
volumes:
- bird_etc:/etc/bird
- bird_run:/run/bird
+12
View File
@@ -52,6 +52,18 @@ opkey|01ARZ3NDEKTSV4RRFFQ69G5FAV|operator,nodekey|01ARZ3NDEKTSV4RRFFQ69G5FAV|nod
**Запрещено** в продакшене: не оставляйте demo-seed с известным токеном `dev` на боевых данных. Переменная `EVOBGP_DEV_INSECURE` в текущей версии **не влияет** на аутентификацию (оставлена в compose для совместимости; не включайте в production — см. SEC-02 в инженерных правилах).
### PostgreSQL monitoring и maintenance (control plane)
При `EVOBGP_DATABASE_URL` (не memory backend):
| Операция | Минимальная роль |
|----------|------------------|
| `GET /v1/monitoring/postgres/*`, `GET /v1/monitoring/correlation` | viewer |
| `POST /v1/postgres/vacuum`, `vacuum-analyze`, `analyze`, `reindex`, `cleanup` | **operator** (async job, rate limit 60s на kind) |
| `GET /v1/postgres/maintenance/logs` | viewer |
Метрики **instance-level** (не per-tenant). CLI: `evobgp-api db …` / `evobgp-all db …`.
### Синхронные «тяжёлые» GET (control plane)
- `POST /v1/modules/{module_id}/cdn-sources/preview` — загрузка CDN в том же HTTP-запросе (лимит тела ~8 MiB, см. OpenAPI).
+112
View File
@@ -0,0 +1,112 @@
# Диагностика схемы PostgreSQL (EvoBGP)
Runbook для оценки объёма БД и узких мест **перед** и **после** миграций оптимизации схемы. Выполнять на staging или production read-only сессией.
## Подключение
```bash
psql "$EVOBGP_DATABASE_URL"
```
## HTTP API (панель / мониторинг)
При подключённом PostgreSQL control plane отдаёт instance-level метрики (роль **viewer+**):
- `GET /v1/monitoring/postgres/overview` — подключения, TPS, cache hit, размер БД
- `GET /v1/monitoring/postgres/queries` — top queries (`pg_stat_statements`, если extension включён)
- `GET /v1/monitoring/postgres/locks`, `/tables`, `/recommendations`
- `GET /v1/monitoring/correlation?window=60` — корреляция refresh jobs и cache hit
Обслуживание (**operator**, async `202` + `job_id`): `POST /v1/postgres/vacuum`, `vacuum-analyze`, `analyze`, `reindex`, `cleanup`; журнал `GET /v1/postgres/maintenance/logs`.
CLI на CP: `evobgp-api db report|vacuum|analyze|cleanup` (см. `internal/dbcli`).
Миграция `000023` создаёт `pg_stat_statements`; для сбора статистики **обязательно** preload и перезапуск Postgres:
```text
# postgresql.conf или command в compose
shared_preload_libraries = 'pg_stat_statements'
```
После изменения — restart контейнера/сервиса Postgres. Без этого API `/v1/monitoring/postgres/queries` вернёт пустой список (`statements_available: false`), без 5xx.
## 1. Размеры таблиц и индексов
```sql
SELECT relname,
pg_size_pretty(pg_total_relation_size(relid)) AS total,
pg_size_pretty(pg_relation_size(relid)) AS heap,
pg_size_pretty(pg_indexes_size(relid)) AS indexes
FROM pg_catalog.pg_statio_user_tables
ORDER BY pg_total_relation_size(relid) DESC;
```
**Ожидание:** лидеры — `revision_materialized_prefix`, `config_revision` (TOAST от preview), JSONB-кэши.
## 2. Seq scan (горячие таблицы)
```sql
SELECT schemaname, relname, seq_scan, seq_tup_read, idx_scan
FROM pg_stat_user_tables
WHERE schemaname = 'public'
ORDER BY seq_tup_read DESC;
```
Сброс статистики после деплоя: `SELECT pg_stat_reset();` (только осознанно, теряется baseline).
## 3. Неиспользуемые индексы
```sql
SELECT indexrelname, idx_scan, pg_size_pretty(pg_relation_size(indexrelid)) AS size
FROM pg_stat_user_indexes
WHERE schemaname = 'public' AND idx_scan = 0
ORDER BY pg_relation_size(indexrelid) DESC;
```
## 4. Дубликаты в materialized prefixes
Перед UNIQUE `(revision_id, prefix, community_id, source)`:
```sql
SELECT revision_id, prefix, community_id, source, COUNT(*) AS n
FROM revision_materialized_prefix
GROUP BY 1, 2, 3, 4
HAVING COUNT(*) > 1
LIMIT 20;
```
## 5. Шаблон отчёта staging
| Метрика | До | После | Дата |
|---------|-----|-------|------|
| `revision_materialized_prefix` total | | | |
| `config_revision` total | | | |
| `module_prefix_snapshot` total | | | |
| `asn_prefix_cache` total | | | |
| Top seq_scan table | | | |
| Unused indexes (count) | | | |
## 6. EXPLAIN для типовых запросов
```sql
-- Список префиксов ревизии (keyset)
EXPLAIN (ANALYZE, BUFFERS)
SELECT prefix::text, community_id::text, source
FROM revision_materialized_prefix
WHERE revision_id = '<revision-uuid>'::uuid
ORDER BY id
LIMIT 51;
-- Diff added (anti-join)
EXPLAIN (ANALYZE, BUFFERS)
SELECT b.prefix::text
FROM revision_materialized_prefix b
LEFT JOIN revision_materialized_prefix a
ON a.revision_id = '<rev-a>'::uuid AND a.prefix = b.prefix
WHERE b.revision_id = '<rev-b>'::uuid
AND a.prefix IS NULL
ORDER BY b.prefix
LIMIT 5001;
```
Цель: Index Scan / Bitmap Index Scan по `(revision_id, …)`, без Seq Scan на больших таблицах.
+492
View File
@@ -49,6 +49,8 @@ tags:
description: Управление API-ключами tenant (operator). Секрет возвращается только при создании и ротации.
- name: Auth
description: Сессия текущего API-ключа (tenant и роль).
- name: Monitoring
description: Наблюдаемость PostgreSQL и корреляция (instance-level, viewer+). Maintenance — operator.
security:
- bearerAuth: []
@@ -755,6 +757,42 @@ components:
bgp_speaker_id:
type: ["string", "null"]
description: "`null` - политика для всех спикеров."
connected_speaker_id:
type: ["string", "null"]
description: >
Live (GET /v1/peers?live=1): спикер, на котором сессия Established; опрос CP birdc + agent /v1/agent/bird/protocols.
connected_speaker_label:
type: string
description: Человекочитаемая метка ноды из live-опроса.
session_on_speakers:
type: array
description: Состояние протокола пира на каждой опрошенной ноде.
items:
type: object
properties:
speaker_id:
type: string
label:
type: string
state:
type: string
established_on_speakers:
type: array
description: Ноды, где сессия в состоянии Established (один пир может быть на нескольких).
items:
type: object
properties:
speaker_id:
type: string
label:
type: string
state:
type: string
session_mismatch:
type: boolean
description: >
true если bgp_speaker_id задан, но на этой ноде нет Established
(сессия может быть на других нодах — это не ошибка для tenant-wide пиров).
policies_json:
type: string
description: >
@@ -809,6 +847,147 @@ components:
description: >
Расширяемый объект. Ключи agent_domain, agent_secret (только при создании),
agent_port, node_ipv4, bird_bgp_source_ipv4, bird_bgp_source_ipv6.
live:
$ref: "#/components/schemas/SpeakerLiveStatus"
description: >
При GET /v1/speakers?live=1 — runtime-статус agent и BGP-опроса на ноде.
additionalProperties: true
SpeakerLiveStatus:
type: object
description: Live runtime snapshot for one speaker (GET /v1/speakers?live=1).
properties:
label:
type: string
description: Человекочитаемая метка ноды (agent domain или CP master).
agent_ok:
type: boolean
description: true если agent /v1/agent/health успешен (master — local birdc poll).
agent_error:
type: string
agent_last_sync_at:
type: string
format: date-time
agent_last_applied_revision_id:
type: string
bgp_poll_ok:
type: boolean
description: true если birdc (CP) или GET /v1/agent/bird/protocols (replica) успешен.
bgp_poll_error:
type: string
bgp_sessions_total:
type: integer
bgp_established:
type: integer
sessions:
type: array
items:
$ref: "#/components/schemas/BgpSessionLive"
additionalProperties: true
BgpSessionLive:
type: object
properties:
name:
type: string
neighbor:
type: string
state:
type: string
additionalProperties: true
LiveSpeakerPoll:
type: object
description: Метаданные опроса одной ноды в GET /v1/peers?live=1.
properties:
speaker_id:
type: string
label:
type: string
ok:
type: boolean
session_count:
type: integer
poll_error:
type: string
additionalProperties: true
PostgresOverview:
type: object
description: Instance-level PostgreSQL snapshot (GET /v1/monitoring/postgres/overview).
additionalProperties: true
PostgresQueriesResponse:
type: object
properties:
collected_at:
type: string
format: date-time
source:
type: string
enum: [live, snapshot]
items:
type: array
items:
type: object
additionalProperties: true
PostgresRecommendations:
type: object
properties:
collected_at:
type: string
format: date-time
items:
type: array
items:
type: object
properties:
severity:
type: string
code:
type: string
title:
type: string
detail:
type: string
refs:
type: array
items:
type: string
PostgresMaintenanceBody:
type: object
properties:
table:
type: string
dry_run:
type: boolean
default: false
policy:
type: string
description: job_audit_retention | asn_cache_retention
limit:
type: integer
BirdLocalStatus:
type: object
description: Статус локального BIRD на хосте API (GET /v1/bird/status).
properties:
birdc_configured:
type: boolean
message:
type: string
error:
type: string
protocols_excerpt:
type: string
bgp_sessions_total:
type: integer
bgp_established:
type: integer
healthy:
type: ["boolean", "null"]
additionalProperties: true
BundleSigningPublicKey:
@@ -2156,6 +2335,13 @@ paths:
- $ref: "#/components/parameters/Cursor"
- $ref: "#/components/parameters/Limit"
- $ref: "#/components/parameters/SpeakerFilter"
- name: live
in: query
schema:
type: string
enum: ["1"]
description: >
Опрос birdc на CP и GET /v1/agent/bird/protocols на репликах; обогащает session_state и connected_speaker_*.
responses:
"200":
description: Успешно.
@@ -2173,6 +2359,12 @@ paths:
type: ["string", "null"]
has_more:
type: boolean
live_speaker_poll:
type: array
description: >
При live=1 — результат опроса каждой ноды (CP birdc + agent protocols).
items:
$ref: "#/components/schemas/LiveSpeakerPoll"
default:
$ref: "#/components/responses/DefaultProblem"
post:
@@ -2293,6 +2485,14 @@ paths:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Cursor"
- $ref: "#/components/parameters/Limit"
- name: live
in: query
schema:
type: string
enum: ["1"]
description: >
Live-опрос agent /v1/agent/health и BGP protocols на репликах; CP — local birdc.
Обогащает каждый item полем `live`.
responses:
"200":
description: Успешно.
@@ -2627,6 +2827,26 @@ paths:
default:
$ref: "#/components/responses/DefaultProblem"
/v1/bird/status:
get:
tags: [Deploy]
summary: Статус локального BIRD на хосте API
description: >
Опрос birdc через EVOBGP_BIRDC_SOCKET на процессе API (обычно CP master).
На репликах без birdc на CP — birdc_configured=false.
operationId: getBirdStatus
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/BirdLocalStatus"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/bird/reload:
post:
tags: [Deploy]
@@ -2975,6 +3195,278 @@ paths:
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/overview:
get:
tags: [Monitoring]
summary: PostgreSQL overview (instance-level)
operationId: getPostgresOverview
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresOverview"
"503":
description: PostgreSQL backend не подключён.
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/queries:
get:
tags: [Monitoring]
summary: Top queries (pg_stat_statements or snapshot)
operationId: getPostgresQueries
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresQueriesResponse"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/locks:
get:
tags: [Monitoring]
summary: Active locks
operationId: getPostgresLocks
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
properties:
items:
type: array
items:
type: object
additionalProperties: true
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/tables:
get:
tags: [Monitoring]
summary: Table sizes and scan stats
operationId: getPostgresTables
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
properties:
items:
type: array
items:
type: object
additionalProperties: true
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/recommendations:
get:
tags: [Monitoring]
summary: Heuristic optimization recommendations
operationId: getPostgresRecommendations
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresRecommendations"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/correlation:
get:
tags: [Monitoring]
summary: Timeline correlation (jobs vs cache hit)
operationId: getMonitoringCorrelation
parameters:
- $ref: "#/components/parameters/TenantId"
- name: window
in: query
schema:
type: integer
default: 60
description: Window in minutes (max 1440).
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
additionalProperties: true
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/vacuum:
post:
tags: [Monitoring]
summary: VACUUM (async job, operator)
operationId: postPostgresVacuum
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
"403":
$ref: "#/components/responses/Forbidden"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/vacuum-analyze:
post:
tags: [Monitoring]
summary: VACUUM ANALYZE (async job, operator)
operationId: postPostgresVacuumAnalyze
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/analyze:
post:
tags: [Monitoring]
summary: ANALYZE (async job, operator)
operationId: postPostgresAnalyze
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/reindex:
post:
tags: [Monitoring]
summary: REINDEX TABLE (async job, operator)
operationId: postPostgresReindex
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/cleanup:
post:
tags: [Monitoring]
summary: Retention cleanup (async job, operator)
operationId: postPostgresCleanup
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/maintenance/logs:
get:
tags: [Monitoring]
summary: Maintenance audit log
operationId: listPostgresMaintenanceLogs
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Cursor"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
properties:
items:
type: array
items:
type: object
additionalProperties: true
next_cursor:
type: string
has_more:
type: boolean
default:
$ref: "#/components/responses/DefaultProblem"
/v1/settings:
get:
tags: [Settings]
+26
View File
@@ -0,0 +1,26 @@
# Production checklist
Краткий чеклист перед выводом EvoBGP в production (10+ клиентов, нестабильная сеть).
## Обязательно
- `EVOBGP_SEED_DEMO=0` — отключить demo-tenant и токен `Bearer dev`.
- `EVOBGP_DEV_INSECURE` не задавать или `0` — не использовать lab-флаги в prod.
- `EVOBGP_BUNDLE_SEED_HEX` — задать стабильный hex-ключ подписи бандлов; сохранить pubkey для нод.
- PostgreSQL с TLS (`sslmode` не `disable`) при доступе вне private network.
- `EVOBGP_CORS_ORIGINS` — явный whitelist origin веб-панели.
- `EVOBGP_STALE_ON_UPSTREAM_ERROR=1` (по умолчанию) — stale snapshot при сбоях CDN/ASN/DoH.
## Рекомендуется
- `EVOBGP_JOB_MAX_CONCURRENT=16`, `EVOBGP_DB_MAX_CONNS=25`, `EVOBGP_COLLECT_CONCURRENCY=16` при росте tenants.
- `EVOBGP_NODE_DISPATCH_INSECURE_TLS=0` — только валидный TLS к agent.
- Ограничить `/metrics` сетевой политикой или reverse proxy.
- Профиль `evobgp-all` или HA API + персистентная `job_audit` (PostgreSQL).
- Мониторинг drift: `evobgp-deploy`, `last_applied_revision_id` vs published.
## Не использовать в prod
- `EVOBGP_CDN_ALLOW_PRIVATE=1` — только тесты/lab.
- Plaintext `EVOBGP_API_KEYS` без ротации (break-glass — временно).
- Ручное редактирование `evobgp_*.conf` на нодах без ревизии.
+17
View File
@@ -11,6 +11,8 @@ EvoBGP использует [Conventional Commits](https://www.conventionalcommi
| `feat!`, `fix!` или `BREAKING CHANGE:` в теле | major (1.0.0 → 2.0.0) |
| `docs`, `chore`, `test` | без релиза |
**Scope:** один идентификатор **без запятых** (`web`, `httpapi`, `api`). Заголовок `refactor(a, b): …` **не парсится** semantic-release → релиз не создаётся (commitlint на PR это тоже отклонит). Подробнее — раздел «Scope и semantic-release» ниже.
`refactor` — patch без новых функций: перестройка кода/UI при том же поведении для пользователя. По semver на одном уровне с `fix`, но семантически «мельче» `feat` (не minor).
Отдельного суффикса `1.x.y.fix` в semver нет: «fix» в Conventional Commits означает **patch** (третья цифра). Для починки пайплайна без смены продукта — `fix(ci):` или `ci:` (оба дают patch после настройки `.releaserc.json`).
@@ -62,6 +64,21 @@ API: `GET /version`, `GET /v1/version` — поля `version`, `git_sha`, `build
Web UI показывает версию из API (footer sidebar, страница «Мониторинг»).
## Scope и semantic-release
Парсер [conventional-commits-parser](https://github.com/conventional-changelog/conventional-changelog/tree/master/packages/conventional-commits-parser) (его использует semantic-release) **не понимает запятые в scope**:
| Заголовок | Парсится | Релиз |
|-----------|----------|-------|
| `refactor(web): fix layout` | да, `refactor` | patch |
| `refactor(NetworkOverviewTab, NetworkSpeakersCard): fix layout` | **нет**, `type: null` | **нет** |
Правило: **один scope** из таблицы в [.cursor/rules/conventional-commits.mdc](../.cursor/rules/conventional-commits.mdc) (`web`, `httpapi`, `api`, …).
На push в `main` job **release** запускает `scripts/commit/verify-release-commits.mjs` — в логе будут предупреждения о непарсящихся коммитах.
Если релиз «не создался», а CI зелёный: смотрите лог release — часто `No releasable commits`. Исправление: новый коммит с корректным заголовком (например `refactor(web): …`).
## CHANGELOG
Release notes — в Gitea Release; файл `CHANGELOG.md` генерируется в CI и прикрепляется как asset, **не** попадает в git history.
+15 -1
View File
@@ -47,10 +47,22 @@ Runbook для реплик **bird2 + evobgp-agent** на отдельных VPS
| Порт | Кто | Зачем |
|------|-----|-------|
| **443** | IP CP (`PANEL_IP_WHITELIST`) | HTTPS dispatch + health |
| **443** | IP CP (`PANEL_IP_WHITELIST`) | HTTPS dispatch, health, **`GET /v1/agent/bird/protocols`** (live peer sessions) |
| **179** | BGP peers | Data plane |
| **80** | ACME | Traefik → 443 |
## Подготовка VPS (перед `docker compose up`)
`bird2` — **`network_mode: host`**. Docker **не может** задать `net.ipv4.ip_forward` в таком контейнере; включите на **хосте**:
```bash
sysctl -w net.ipv4.ip_forward=1
sysctl -w net.ipv6.conf.all.forwarding=1
echo 'net.ipv4.ip_forward=1' | tee /etc/sysctl.d/99-evobgp-bird.conf
echo 'net.ipv6.conf.all.forwarding=1' >> /etc/sysctl.d/99-evobgp-bird.conf
sysctl --system
```
## Безопасность (три участка)
1. **CP → реплика:** HTTPS (LE) + Traefik ipallowlist + `agent_secret`.
@@ -82,6 +94,8 @@ Tenant `/v1/settings` (`bird_bgp_source_ipv4`) — fallback для master / ес
| Симптом | Проверка |
|---------|----------|
| `sysctl net.ipv4.ip_forward not allowed in host network` | Уберите sysctls из compose (уже так в main); включите ip_forward на VPS (см. выше) |
| `no service selected` | `--profile production` или `COMPOSE_PROFILES=production` |
| Offline в UI | `GET https://AGENT_DOMAIN/v1/agent/health` с CP; LE cert; whitelist |
| dispatch error | CP logs job meta; firewall 443; `agent_secret` |
| verify-bundle fail | pubkey совпадает с CP seed; пересоберите pubkey после смены seed |
+29 -1
View File
@@ -10,9 +10,12 @@ import (
"strings"
"time"
"evobgp/internal/birdfmt"
"evobgp/internal/nodecli"
)
const upstreamErrorDetail = "upstream request failed"
// Config holds evobgp-agent serve settings.
type Config struct {
Listen string
@@ -41,6 +44,7 @@ type Server struct {
func New(cfg Config) *Server {
s := &Server{cfg: cfg, mux: http.NewServeMux()}
s.mux.HandleFunc("GET /v1/agent/health", s.handleHealth)
s.mux.HandleFunc("GET /v1/agent/bird/protocols", s.handleBirdProtocols)
s.mux.HandleFunc("POST /v1/agent/sync", s.handleSync)
return s
}
@@ -68,6 +72,30 @@ func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, body)
}
func (s *Server) handleBirdProtocols(w http.ResponseWriter, r *http.Request) {
if !s.authorize(r) {
writeProblem(w, http.StatusUnauthorized, "missing or invalid Authorization")
return
}
sock := strings.TrimSpace(s.cfg.Socket)
if sock == "" {
writeProblem(w, http.StatusServiceUnavailable, "EVOBGP_BIRDC_SOCKET not configured")
return
}
ctx, cancel := context.WithTimeout(r.Context(), 10*time.Second)
defer cancel()
out, err := birdfmt.ShowProtocols(ctx, sock, strings.TrimSpace(s.cfg.BirdcBin))
if err != nil {
log.Printf("agentserver: bird protocols: %v", err)
writeProblem(w, http.StatusBadGateway, upstreamErrorDetail)
return
}
writeJSON(w, http.StatusOK, map[string]any{
"ok": true,
"sessions": birdfmt.ParseBGPSessions(out),
})
}
func (s *Server) handleSync(w http.ResponseWriter, r *http.Request) {
if !s.authorize(r) {
writeProblem(w, http.StatusUnauthorized, "missing or invalid Authorization")
@@ -100,7 +128,7 @@ func (s *Server) handleSync(w http.ResponseWriter, r *http.Request) {
})
if err != nil {
log.Printf("agentserver: sync: %v", err)
writeProblem(w, http.StatusBadGateway, err.Error())
writeProblem(w, http.StatusBadGateway, upstreamErrorDetail)
return
}
if s.cfg.OnSyncSuccess != nil {
+6 -4
View File
@@ -13,6 +13,8 @@ import (
"strconv"
"strings"
"time"
"evobgp/internal/httpclient"
)
// DefaultRIPEStatURL is the RIPEstat announced-prefixes data call (no API key).
@@ -24,7 +26,7 @@ const DefaultASOverviewURL = "https://stat.ripe.net/data/as-overview/data.json"
// AnnouncedPrefixes returns currently announced IPv4/IPv6 prefixes for the ASN (best-effort via RIPEstat).
func AnnouncedPrefixes(ctx context.Context, hc *http.Client, asn int64) ([]netip.Prefix, error) {
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
base := strings.TrimSpace(os.Getenv("EVOBGP_RIPESTAT_ANNOUNCED_PREFIXES_URL"))
if base == "" {
@@ -38,7 +40,7 @@ func AnnouncedPrefixes(ctx context.Context, hc *http.Client, asn int64) ([]netip
req.Header.Set("Accept", "application/json")
req.Header.Set("User-Agent", "evobgp-asnresolve/1.0")
resp, err := hc.Do(req)
resp, err := httpclient.DoWithBreaker(ctx, hc, req, 3)
if err != nil {
return nil, fmt.Errorf("ripestat fetch AS%d: %w", asn, err)
}
@@ -86,7 +88,7 @@ func AnnouncedPrefixes(ctx context.Context, hc *http.Client, asn int64) ([]netip
// ASHolderName returns the holder / organization label for the ASN from RIPEstat as-overview (best-effort).
func ASHolderName(ctx context.Context, hc *http.Client, asn int64) (string, error) {
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
base := strings.TrimSpace(os.Getenv("EVOBGP_RIPESTAT_AS_OVERVIEW_URL"))
if base == "" {
@@ -100,7 +102,7 @@ func ASHolderName(ctx context.Context, hc *http.Client, asn int64) (string, erro
req.Header.Set("Accept", "application/json")
req.Header.Set("User-Agent", "evobgp-asnresolve/1.0")
resp, err := hc.Do(req)
resp, err := httpclient.DoWithBreaker(ctx, hc, req, 3)
if err != nil {
return "", fmt.Errorf("ripestat as-overview AS%d: %w", asn, err)
}
+54
View File
@@ -0,0 +1,54 @@
package birdfmt
import (
"strings"
)
// BGPSession is one BGP protocol block from `birdc show protocols all`.
type BGPSession struct {
Name string `json:"name"`
Neighbor string `json:"neighbor,omitempty"`
State string `json:"state"`
}
// ParseBGPSessions extracts BGP protocol name, state, and neighbor (if present) from birdc output.
func ParseBGPSessions(output string) []BGPSession {
var out []BGPSession
var cur *BGPSession
for _, raw := range strings.Split(output, "\n") {
line := strings.TrimRight(raw, "\r")
trim := strings.TrimSpace(line)
if trim == "" {
cur = nil
continue
}
low := strings.ToLower(trim)
if strings.HasPrefix(low, "bird ") || strings.HasPrefix(low, "name ") || strings.HasPrefix(low, "table ") {
continue
}
if !strings.HasPrefix(line, " ") && !strings.HasPrefix(line, "\t") {
if isBGPProtocolSummaryRow(trim) {
fields := strings.Fields(trim)
state := extractBGPSessionStateLine(trim)
if state == "" && len(fields) >= 4 {
state = fields[3]
}
out = append(out, BGPSession{Name: fields[0], State: state})
cur = &out[len(out)-1]
} else {
cur = nil
}
continue
}
if cur == nil {
continue
}
for _, prefix := range []string{"Neighbor address:", "Neighbor Address:", "Neighbor:"} {
if idx := strings.Index(trim, prefix); idx >= 0 {
cur.Neighbor = strings.TrimSpace(trim[idx+len(prefix):])
break
}
}
}
return out
}
+26
View File
@@ -0,0 +1,26 @@
package birdfmt
import "testing"
func TestParseBGPSessions_neighborAndState(t *testing.T) {
sample := `
BIRD 2.14 ready.
Name Proto Table State Since Info
device1 Device --- up 10:00:00
evobgp_p_abc123 BGP master4 up 10:00:05 Established
Neighbor address: 198.51.100.2
Neighbor AS: 65001
evobgp_p_def456 BGP master4 up 10:00:06 Active
Neighbor address: 2001:db8::2
`
sessions := ParseBGPSessions(sample)
if len(sessions) != 2 {
t.Fatalf("got %d sessions want 2", len(sessions))
}
if sessions[0].Name != "evobgp_p_abc123" || sessions[0].State != "Established" || sessions[0].Neighbor != "198.51.100.2" {
t.Fatalf("session0: %+v", sessions[0])
}
if sessions[1].Neighbor != "2001:db8::2" || sessions[1].State != "Active" {
t.Fatalf("session1: %+v", sessions[1])
}
}
+16
View File
@@ -0,0 +1,16 @@
package birdfmt
import "strings"
// PeerProtocolName returns the BIRD protocol name for a control-plane peer UUID.
// Must stay in sync with pipeline peer rendering.
func PeerProtocolName(peerID string) string {
s := strings.ReplaceAll(strings.TrimSpace(peerID), "-", "")
if len(s) > 16 {
s = s[:16]
}
if s == "" {
s = "x"
}
return "evobgp_p_" + s
}
+209
View File
@@ -0,0 +1,209 @@
// Package dbcli implements control-plane PostgreSQL maintenance CLI (HTTP or local DSN).
package dbcli
import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
"evobgp/internal/db"
"evobgp/internal/httpclient"
"evobgp/internal/pgmonitor"
)
// Run executes db subcommands; args exclude program name and "db".
func Run(args []string) int {
if len(args) == 0 {
printUsage()
return 2
}
switch args[0] {
case "report":
return cmdReport(args[1:])
case "vacuum":
return cmdMaint(args[1:], "vacuum", "/v1/postgres/vacuum")
case "analyze":
return cmdMaint(args[1:], "analyze", "/v1/postgres/analyze")
case "cleanup":
return cmdCleanup(args[1:])
default:
fmt.Fprintf(os.Stderr, "dbcli: unknown command %q\n", args[0])
printUsage()
return 2
}
}
func printUsage() {
fmt.Fprintln(os.Stderr, `usage:
evobgp-api db report [--api-url URL] [--token TOKEN] [--format json]
evobgp-api db vacuum [--table NAME] [--dry-run] [--api-url URL] [--token TOKEN]
evobgp-api db analyze [--table NAME] [--dry-run] [--api-url URL] [--token TOKEN]
evobgp-api db cleanup --policy NAME [--dry-run] [--limit N] [--api-url URL] [--token TOKEN]
Local break-glass: set EVOBGP_DATABASE_URL (report only uses direct SQL).`)
}
func cmdReport(args []string) int {
fs := flag.NewFlagSet("report", flag.ExitOnError)
apiURL := fs.String("api-url", "", "control plane base URL")
token := fs.String("token", "", "Bearer token (operator)")
format := fs.String("format", "json", "output format (json)")
_ = fs.Parse(args)
if dsn := strings.TrimSpace(os.Getenv("EVOBGP_DATABASE_URL")); dsn != "" && *apiURL == "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
pool, err := db.OpenPostgresPool(ctx, dsn)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
defer pool.Close()
svc := pgmonitor.NewService(pool)
ov, err := svc.Overview(ctx)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeJSONStdout(ov, *format)
}
if *apiURL == "" || *token == "" {
fmt.Fprintln(os.Stderr, "report: --api-url and --token required without EVOBGP_DATABASE_URL")
return 2
}
body, err := apiGET(*apiURL, *token, "/v1/monitoring/postgres/overview")
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
var pretty any
if err := json.Unmarshal(body, &pretty); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeJSONStdout(pretty, *format)
}
func cmdMaint(args []string, _ string, path string) int {
fs := flag.NewFlagSet("maint", flag.ExitOnError)
table := fs.String("table", "", "table name")
dryRun := fs.Bool("dry-run", false, "dry run only")
apiURL := fs.String("api-url", "", "control plane base URL")
token := fs.String("token", "", "Bearer token (operator)")
_ = fs.Parse(args)
if *apiURL == "" || *token == "" {
fmt.Fprintln(os.Stderr, "maintenance: --api-url and --token are required")
return 2
}
payload := map[string]any{"dry_run": *dryRun}
if *table != "" {
payload["table"] = *table
}
body, err := apiPOST(*apiURL, *token, path, payload)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeRawJSON(body)
}
func cmdCleanup(args []string) int {
fs := flag.NewFlagSet("cleanup", flag.ExitOnError)
policy := fs.String("policy", "", "cleanup policy name")
dryRun := fs.Bool("dry-run", true, "dry run")
limit := fs.Int("limit", 10000, "max rows")
apiURL := fs.String("api-url", "", "control plane base URL")
token := fs.String("token", "", "Bearer token (operator)")
_ = fs.Parse(args)
if *policy == "" {
fmt.Fprintln(os.Stderr, "cleanup: --policy is required")
return 2
}
if *apiURL == "" || *token == "" {
fmt.Fprintln(os.Stderr, "cleanup: --api-url and --token are required")
return 2
}
payload := map[string]any{"policy": *policy, "dry_run": *dryRun, "limit": *limit}
body, err := apiPOST(*apiURL, *token, "/v1/postgres/cleanup", payload)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeRawJSON(body)
}
func apiGET(base, token, path string) ([]byte, error) {
u := strings.TrimRight(base, "/") + path
req, err := http.NewRequest(http.MethodGet, u, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
resp, err := httpclient.DoWithRetry(ctx, httpclient.New(60*time.Second), req, 3)
if err != nil {
return nil, err
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
if resp.StatusCode >= 300 {
return nil, fmt.Errorf("dbcli: GET %s: %s: %s", path, resp.Status, strings.TrimSpace(string(b)))
}
return b, nil
}
func apiPOST(base, token, path string, payload map[string]any) ([]byte, error) {
b, err := json.Marshal(payload)
if err != nil {
return nil, err
}
u := strings.TrimRight(base, "/") + path
req, err := http.NewRequest(http.MethodPost, u, bytes.NewReader(b))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
resp, err := httpclient.DoWithRetry(ctx, httpclient.New(60*time.Second), req, 3)
if err != nil {
return nil, err
}
defer func() { _ = resp.Body.Close() }()
out, _ := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusAccepted && resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("dbcli: POST %s: %s: %s", path, resp.Status, strings.TrimSpace(string(out)))
}
return out, nil
}
func writeJSONStdout(v any, format string) int {
if format != "json" {
fmt.Fprintln(os.Stderr, "only json format supported")
return 2
}
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
if err := enc.Encode(v); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return 0
}
func writeRawJSON(b []byte) int {
var v any
if err := json.Unmarshal(b, &v); err != nil {
_, _ = os.Stdout.Write(b)
return 0
}
return writeJSONStdout(v, "json")
}
+34 -11
View File
@@ -7,6 +7,7 @@ import (
"time"
"evobgp/internal/db"
"evobgp/internal/httpclient"
"evobgp/internal/jobs"
"evobgp/internal/observability"
"evobgp/internal/repository"
@@ -17,7 +18,7 @@ import (
// NewCDNHTTPClient returns the shared HTTP client for CDN and preview fetches (PERF-02 / ERR-03).
func NewCDNHTTPClient() *http.Client {
return &http.Client{Timeout: 45 * time.Second}
return httpclient.New(httpclient.DefaultTimeout)
}
// BootstrapWorkers opens the same store.Backend and jobs.Registry as New (without HTTP or bundle keys).
@@ -50,23 +51,45 @@ func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.R
}
cdnHTTP := NewCDNHTTPClient()
wk := &jobs.Worker{Store: backend, HTTPClient: cdnHTTP}
wk := &jobs.Worker{Store: backend, PgPool: pool, HTTPClient: cdnHTTP}
reg := jobs.NewRegistry(wk.Process)
wk.Registry = reg
if pool != nil {
audit := repository.NewJobAuditWriter(pool)
reg.SetTerminalHook(func(j *jobs.Job) {
jobMeta := func(j *jobs.Job) map[string]any {
if j == nil {
return
return nil
}
st := j.Snapshot()
status, _ := st["status"].(string)
var errMsg *string
if e, ok := st["error"].(string); ok && e != "" {
errMsg = &e
}
audit.MarkTerminal(context.Background(), j.TenantID, j.ID, status, errMsg, time.Now().UTC())
})
meta, _ := st["meta"].(map[string]any)
return meta
}
reg.SetPersistHooks(
func(j *jobs.Job) {
if j == nil {
return
}
audit.UpsertQueued(context.Background(), j.TenantID, j.ID, j.Kind, j.IdempotencyKey, j.ModuleID, jobMeta(j))
},
func(j *jobs.Job) {
if j == nil {
return
}
audit.UpsertRunning(context.Background(), j.TenantID, j.ID, j.Kind, j.IdempotencyKey, jobMeta(j))
},
func(j *jobs.Job) {
if j == nil {
return
}
st := j.Snapshot()
status, _ := st["status"].(string)
var errMsg *string
if e, ok := st["error"].(string); ok && e != "" {
errMsg = &e
}
audit.MarkTerminal(context.Background(), j.TenantID, j.ID, status, errMsg, time.Now().UTC())
},
)
}
observability.RegisterStoreBackend(backend)
return backend, reg, pool, nil
+295
View File
@@ -0,0 +1,295 @@
package httpapi
import (
"context"
"net/netip"
"os"
"strings"
"sync"
"time"
"evobgp/internal/birdfmt"
"evobgp/internal/nodedispatch"
"evobgp/internal/store"
)
const peerLiveCacheTTL = 15 * time.Second
type speakerBGPLive struct {
SpeakerID string
Label string
Sessions []birdfmt.BGPSession
Error string
}
type peerLiveCacheEntry struct {
at time.Time
views []speakerBGPLive
}
var peerLiveCache sync.Map // tenantID -> peerLiveCacheEntry
type peerSessionOnSpeaker struct {
SpeakerID string `json:"speaker_id"`
Label string `json:"label"`
State string `json:"state"`
PollError string `json:"poll_error,omitempty"`
}
type liveSpeakerPoll struct {
SpeakerID string `json:"speaker_id"`
Label string `json:"label"`
OK bool `json:"ok"`
SessionCount int `json:"session_count"`
PollError string `json:"poll_error,omitempty"`
}
func speakerDisplayLabel(sp *store.Speaker) string {
if sp == nil {
return ""
}
meta := store.ParseSpeakerMeta(sp.MetaJSON)
host := strings.TrimSpace(meta.AgentDomain)
if host == "" {
host = strings.TrimSpace(sp.Endpoint)
}
if strings.EqualFold(strings.TrimSpace(sp.Role), "master") {
if host != "" {
return "CP · " + host
}
return "CP (master)"
}
if host != "" {
return host
}
return sp.ID
}
func masterSpeakerID(speakers []*store.Speaker) string {
for _, sp := range speakers {
if sp != nil && strings.EqualFold(strings.TrimSpace(sp.Role), "master") {
return sp.ID
}
}
return ""
}
func (s *Server) collectSpeakerBGPLive(ctx context.Context, tenantID string, fresh bool) []speakerBGPLive {
if !fresh {
if v, ok := peerLiveCache.Load(tenantID); ok {
ent := v.(peerLiveCacheEntry)
if time.Since(ent.at) < peerLiveCacheTTL {
return ent.views
}
}
}
speakers := s.store.ListSpeakersForTenant(tenantID)
views := make([]speakerBGPLive, 0, len(speakers)+1)
if sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET")); sock != "" {
v := speakerBGPLive{Label: "CP (local BIRD)"}
if mid := masterSpeakerID(speakers); mid != "" {
v.SpeakerID = mid
for _, sp := range speakers {
if sp != nil && sp.ID == mid {
v.Label = speakerDisplayLabel(sp)
break
}
}
}
out, err := birdfmt.ShowProtocols(ctx, sock, strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN")))
if err != nil {
v.Error = err.Error()
} else {
v.Sessions = birdfmt.ParseBGPSessions(out)
}
views = append(views, v)
}
opts := nodedispatch.Options{Timeout: 8 * time.Second}
type resWrap struct {
sp *store.Speaker
res nodedispatch.BirdProtocolsResult
}
ch := make(chan resWrap, len(speakers))
var wg sync.WaitGroup
for _, sp := range speakers {
if sp == nil {
continue
}
meta := store.ParseSpeakerMeta(sp.MetaJSON)
if !store.SpeakerNeedsRemoteDispatch(sp.Role, meta) {
continue
}
wg.Add(1)
go func(speaker *store.Speaker) {
defer wg.Done()
ch <- resWrap{
sp: speaker,
res: nodedispatch.FetchBirdProtocols(ctx, speaker, opts),
}
}(sp)
}
wg.Wait()
close(ch)
for rw := range ch {
views = append(views, speakerBGPLive{
SpeakerID: rw.sp.ID,
Label: speakerDisplayLabel(rw.sp),
Sessions: rw.res.Sessions,
Error: rw.res.Error,
})
}
peerLiveCache.Store(tenantID, peerLiveCacheEntry{at: time.Now(), views: views})
return views
}
func liveSpeakerPollJSON(views []speakerBGPLive) []liveSpeakerPoll {
out := make([]liveSpeakerPoll, 0, len(views))
for _, v := range views {
out = append(out, liveSpeakerPoll{
SpeakerID: v.SpeakerID,
Label: v.Label,
OK: v.Error == "",
SessionCount: len(v.Sessions),
PollError: v.Error,
})
}
return out
}
func findPeerSession(sessions []birdfmt.BGPSession, protoName string, neighbor netip.Addr, hasNeighbor bool) *birdfmt.BGPSession {
for i := range sessions {
if peerSessionMatches(sessions[i], protoName, neighbor, hasNeighbor) {
return &sessions[i]
}
}
return nil
}
func matchPeerOnSpeakers(peer *store.BGPPeer, views []speakerBGPLive) (
bestState string,
connectedID string,
connectedLabel string,
establishedOn []peerSessionOnSpeaker,
on []peerSessionOnSpeaker,
mismatch bool,
) {
if peer == nil {
return "", "", "", nil, nil, false
}
neighbor, hasNeighbor := store.ParsePeerNeighbor(peer.Neighbor)
protoName := birdfmt.PeerProtocolName(peer.ID)
for _, v := range views {
if v.Error != "" && len(v.Sessions) == 0 {
on = append(on, peerSessionOnSpeaker{
SpeakerID: v.SpeakerID,
Label: v.Label,
PollError: v.Error,
})
continue
}
sess := findPeerSession(v.Sessions, protoName, neighbor, hasNeighbor)
if sess == nil {
on = append(on, peerSessionOnSpeaker{
SpeakerID: v.SpeakerID,
Label: v.Label,
State: "absent",
})
continue
}
hit := peerSessionOnSpeaker{
SpeakerID: v.SpeakerID,
Label: v.Label,
State: sess.State,
}
on = append(on, hit)
if strings.EqualFold(strings.TrimSpace(sess.State), "Established") {
establishedOn = append(establishedOn, hit)
}
if bestState == "" || sessionStateRank(sess.State) > sessionStateRank(bestState) {
bestState = sess.State
}
}
if len(establishedOn) > 0 {
bestState = "Established"
labels := make([]string, 0, len(establishedOn))
for _, e := range establishedOn {
labels = append(labels, e.Label)
}
connectedLabel = strings.Join(labels, ", ")
if len(establishedOn) == 1 {
connectedID = establishedOn[0].SpeakerID
}
} else if len(on) == 1 && on[0].PollError == "" && on[0].State != "" {
connectedID = on[0].SpeakerID
connectedLabel = on[0].Label
}
if peer.SpeakerID != nil && strings.TrimSpace(*peer.SpeakerID) != "" && len(establishedOn) > 0 {
want := strings.TrimSpace(*peer.SpeakerID)
found := false
for _, e := range establishedOn {
if strings.EqualFold(strings.TrimSpace(e.SpeakerID), want) {
found = true
break
}
}
mismatch = !found
}
return bestState, connectedID, connectedLabel, establishedOn, on, mismatch
}
func peerSessionMatches(sess birdfmt.BGPSession, protoName string, neighbor netip.Addr, hasNeighbor bool) bool {
if strings.EqualFold(strings.TrimSpace(sess.Name), protoName) {
return true
}
if !hasNeighbor || strings.TrimSpace(sess.Neighbor) == "" {
return false
}
peerAddr, ok := store.ParsePeerNeighbor(sess.Neighbor)
if !ok {
return false
}
return peerAddr == neighbor
}
func sessionStateRank(state string) int {
switch strings.ToLower(strings.TrimSpace(state)) {
case "established":
return 100
case "openconfirm", "opensent":
return 80
case "active", "connect":
return 60
case "idle":
return 20
default:
return 10
}
}
func applyPeerLiveFields(row map[string]any, peer *store.BGPPeer, views []speakerBGPLive) {
state, connID, connLabel, establishedOn, on, mismatch := matchPeerOnSpeakers(peer, views)
row["session_on_speakers"] = on
row["established_on_speakers"] = establishedOn
row["session_conflict"] = false
row["session_mismatch"] = mismatch
if state != "" {
row["session_state"] = state
}
if connLabel != "" {
row["connected_speaker_label"] = connLabel
}
row["connected_speaker_id"] = peerLiveSpeakerIDOrNull(connID)
}
func peerLiveSpeakerIDOrNull(id string) any {
if strings.TrimSpace(id) == "" {
return nil
}
return id
}
+79
View File
@@ -0,0 +1,79 @@
package httpapi
import (
"testing"
"evobgp/internal/birdfmt"
"evobgp/internal/store"
)
func TestMatchPeerOnSpeakers_establishedOnReplica(t *testing.T) {
peer := &store.BGPPeer{
ID: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
Neighbor: "198.51.100.2",
}
views := []speakerBGPLive{
{
SpeakerID: "master-id",
Label: "CP · bgp.shz.su",
Sessions: []birdfmt.BGPSession{
{Name: birdfmt.PeerProtocolName(peer.ID), Neighbor: "198.51.100.2", State: "Established"},
},
},
{
SpeakerID: "replica-id",
Label: "bgp2.shz.su",
Sessions: []birdfmt.BGPSession{
{Name: birdfmt.PeerProtocolName(peer.ID), Neighbor: "198.51.100.2", State: "Established"},
},
},
}
state, connID, connLabel, established, on, mismatch := matchPeerOnSpeakers(peer, views)
if state != "Established" || connID != "" || connLabel != "CP · bgp.shz.su, bgp2.shz.su" {
t.Fatalf("got state=%q conn=%q label=%q", state, connID, connLabel)
}
if mismatch || len(on) != 2 || len(established) != 2 {
t.Fatalf("on=%+v established=%+v mismatch=%v", on, established, mismatch)
}
}
func TestMatchPeerOnSpeakers_multipleEstablished(t *testing.T) {
peer := &store.BGPPeer{ID: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", Neighbor: "198.51.100.2/32"}
views := []speakerBGPLive{
{SpeakerID: "a", Label: "n1", Sessions: []birdfmt.BGPSession{{Name: birdfmt.PeerProtocolName(peer.ID), State: "Established"}}},
{SpeakerID: "b", Label: "n2", Sessions: []birdfmt.BGPSession{{Name: birdfmt.PeerProtocolName(peer.ID), State: "Established"}}},
}
_, connID, label, established, on, mismatch := matchPeerOnSpeakers(peer, views)
if mismatch || connID != "" || label != "n1, n2" || len(established) != 2 || len(on) != 2 {
t.Fatalf("connID=%q label=%q established=%+v on=%+v mismatch=%v", connID, label, established, on, mismatch)
}
}
func TestMatchPeerOnSpeakers_mismatchConfiguredSpeaker(t *testing.T) {
replica := "replica-id"
peer := &store.BGPPeer{
ID: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee",
Neighbor: "198.51.100.2",
SpeakerID: &replica,
}
views := []speakerBGPLive{
{SpeakerID: "master-id", Label: "CP", Sessions: []birdfmt.BGPSession{{Name: birdfmt.PeerProtocolName(peer.ID), State: "Established"}}},
{SpeakerID: replica, Label: "bgp2", Sessions: []birdfmt.BGPSession{{Name: birdfmt.PeerProtocolName(peer.ID), State: "Idle"}}},
}
_, _, _, _, _, mismatch := matchPeerOnSpeakers(peer, views)
if !mismatch {
t.Fatal("expected mismatch when configured replica has no Established")
}
}
func TestMatchPeerOnSpeakers_pollError(t *testing.T) {
peer := &store.BGPPeer{ID: "aaaaaaaa-bbbb-cccc-dddd-eeeeeeeeeeee", Neighbor: "198.51.100.2"}
views := []speakerBGPLive{
{Label: "CP (local BIRD)", Sessions: []birdfmt.BGPSession{{Name: birdfmt.PeerProtocolName(peer.ID), State: "Established"}}},
{SpeakerID: "replica-id", Label: "bgp2.shz.su", Error: "HTTP 404: Not Found"},
}
_, _, _, established, on, _ := matchPeerOnSpeakers(peer, views)
if len(established) != 1 || len(on) != 2 || on[1].PollError == "" {
t.Fatalf("on=%+v established=%+v", on, established)
}
}
+28 -16
View File
@@ -76,6 +76,8 @@ func (s *Server) registerV1(m *http.ServeMux) {
m.HandleFunc("GET /speakers/{speaker_id}/bundle/{revision_id}", s.handleNodeBundle)
m.HandleFunc("POST /nodes/enroll", s.handleNodeEnroll)
s.registerCRUDRoutes(m)
s.registerPostgresMonitoringRoutes(m)
s.registerPostgresMaintenanceRoutes(m)
}
func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
@@ -283,17 +285,21 @@ func (s *Server) handleListPeers(w http.ResponseWriter, r *http.Request) {
}
allPeers := s.store.ListPeers(a.TenantID)
page, next, more := store.PaginateOffset(allPeers, r.URL.Query().Get("cursor"), parseListLimit(r))
liveStates := s.liveBGPProtocolStates(r)
fresh := r != nil && strings.EqualFold(strings.TrimSpace(r.URL.Query().Get("live")), "1")
ctx, cancel := context.WithTimeout(r.Context(), 12*time.Second)
defer cancel()
liveViews := s.collectSpeakerBGPLive(ctx, a.TenantID, fresh)
items := make([]map[string]any, 0, len(page))
for _, p := range page {
row := peerJSON(p)
if st, ok := liveStates[peerProtocolNameForID(p.ID)]; ok && strings.TrimSpace(st) != "" {
row["session_state"] = strings.TrimSpace(st)
}
applyPeerLiveFields(row, p, liveViews)
items = append(items, row)
}
writeJSON(w, http.StatusOK, map[string]any{
"items": items, "next_cursor": strPtrOrNull(next), "has_more": more,
"items": items,
"next_cursor": strPtrOrNull(next),
"has_more": more,
"live_speaker_poll": liveSpeakerPollJSON(liveViews),
})
}
@@ -366,16 +372,9 @@ func extractBGPSessionState(line string) string {
return ""
}
// peerProtocolNameForID must stay in sync with pipeline peer protocol naming.
// peerProtocolNameForID forwards to birdfmt for tests and legacy callers.
func peerProtocolNameForID(peerID string) string {
s := strings.ReplaceAll(strings.TrimSpace(peerID), "-", "")
if len(s) > 16 {
s = s[:16]
}
if s == "" {
s = "x"
}
return "evobgp_p_" + s
return birdfmt.PeerProtocolName(peerID)
}
func (s *Server) handleListSpeakers(w http.ResponseWriter, r *http.Request) {
@@ -388,9 +387,22 @@ func (s *Server) handleListSpeakers(w http.ResponseWriter, r *http.Request) {
return
}
speakers := s.store.ListSpeakersForTenant(a.TenantID)
fresh := r != nil && strings.EqualFold(strings.TrimSpace(r.URL.Query().Get("live")), "1")
var liveByID map[string]map[string]any
if fresh {
ctx, cancel := context.WithTimeout(r.Context(), 12*time.Second)
defer cancel()
liveByID = s.collectSpeakerLiveStatus(ctx, a.TenantID, true, speakers)
}
items := make([]map[string]any, 0, len(speakers))
for _, sp := range speakers {
items = append(items, speakerJSONFromStore(s.store, sp))
row := speakerJSONFromStore(s.store, sp)
if liveByID != nil {
if live, ok := liveByID[sp.ID]; ok {
row["live"] = live
}
}
items = append(items, row)
}
writeJSON(w, http.StatusOK, map[string]any{
"items": items, "next_cursor": nil, "has_more": false,
@@ -569,7 +581,7 @@ func (s *Server) handleGetRevision(w http.ResponseWriter, r *http.Request) {
if !s.requireAtLeast(w, a, "viewer") {
return
}
rev, err := s.store.GetRevision(a.TenantID, r.PathValue("revision_id"))
rev, err := s.store.GetRevisionSummary(a.TenantID, r.PathValue("revision_id"))
if err != nil {
writeProblem(w, http.StatusNotFound, "Not Found", "revision not found")
return
+28
View File
@@ -242,6 +242,14 @@ func (s *Server) handlePreviewCDNSource(w http.ResponseWriter, r *http.Request)
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "url is required")
return
}
if _, err := pipeline.ValidateCDNURL(u); err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", invalidInputDetail)
return
}
if err := pipeline.ResolveCDNURLHost(r.Context(), u); err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", invalidInputDetail)
return
}
mod, err := s.store.GetModule(a.TenantID, r.PathValue("module_id"))
if err != nil {
writeStoreErr(w, err)
@@ -304,6 +312,16 @@ func (s *Server) handlePostCDNSource(w http.ResponseWriter, r *http.Request) {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
return
}
if body.URL != "" {
if _, err := pipeline.ValidateCDNURL(body.URL); err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", invalidInputDetail)
return
}
if err := pipeline.ResolveCDNURLHost(r.Context(), body.URL); err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", invalidInputDetail)
return
}
}
mid := r.PathValue("module_id")
x, err := s.store.CreateCDNSource(a.TenantID, mid, &body)
if err != nil {
@@ -324,6 +342,16 @@ func (s *Server) handlePatchCDNSource(w http.ResponseWriter, r *http.Request) {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
return
}
if body.URL != nil && strings.TrimSpace(*body.URL) != "" {
if _, err := pipeline.ValidateCDNURL(*body.URL); err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", invalidInputDetail)
return
}
if err := pipeline.ResolveCDNURLHost(r.Context(), *body.URL); err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", invalidInputDetail)
return
}
}
mid := r.PathValue("module_id")
x, err := s.store.UpdateCDNSource(a.TenantID, mid, r.PathValue("source_id"), &body)
if err != nil {
@@ -0,0 +1,203 @@
package httpapi
import (
"context"
"encoding/json"
"io"
"net/http"
"strings"
"sync"
"time"
"evobgp/internal/jobs"
"evobgp/internal/pgmonitor"
)
var (
pgMaintRateMu sync.Mutex
pgMaintLastByTK = map[string]time.Time{}
)
func (s *Server) registerPostgresMaintenanceRoutes(m *http.ServeMux) {
m.HandleFunc("POST /postgres/vacuum", s.handlePostgresVacuum)
m.HandleFunc("POST /postgres/vacuum-analyze", s.handlePostgresVacuumAnalyze)
m.HandleFunc("POST /postgres/analyze", s.handlePostgresAnalyze)
m.HandleFunc("POST /postgres/reindex", s.handlePostgresReindex)
m.HandleFunc("POST /postgres/cleanup", s.handlePostgresCleanup)
m.HandleFunc("GET /postgres/maintenance/logs", s.handlePostgresMaintenanceLogs)
}
func (s *Server) requireOperatorStrict(w http.ResponseWriter, a Auth) bool {
if strings.ToLower(a.Role) != "operator" {
writeProblem(w, http.StatusForbidden, "Forbidden", "operator role required")
return false
}
return true
}
func (s *Server) checkPgMaintRateLimit(tenantID, kind string) bool {
key := tenantID + ":" + kind
pgMaintRateMu.Lock()
defer pgMaintRateMu.Unlock()
if t, ok := pgMaintLastByTK[key]; ok && time.Since(t) < 60*time.Second {
return false
}
pgMaintLastByTK[key] = time.Now().UTC()
return true
}
type pgMaintBody struct {
Table string `json:"table"`
DryRun bool `json:"dry_run"`
Index string `json:"index"`
Policy string `json:"policy"`
Limit int `json:"limit"`
}
func (s *Server) decodePgMaintBody(r *http.Request) (pgMaintBody, bool) {
var body pgMaintBody
if r.Body == nil || r.ContentLength == 0 {
return body, true
}
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&body); err != nil && err != io.EOF {
return body, false
}
return body, true
}
func (s *Server) enqueuePostgresMaint(w http.ResponseWriter, r *http.Request, a Auth, kind string, meta map[string]any) {
if !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
return
}
if !s.checkPgMaintRateLimit(a.TenantID, kind) {
writeProblem(w, http.StatusTooManyRequests, "Too Many Requests", "wait before repeating this maintenance operation")
return
}
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
var idemPtr *string
if idem != "" {
idemPtr = &idem
}
meta["actor_prefix"] = actorPrefix(a)
j, _, err := s.jobs.Enqueue(a.TenantID, kind, idemPtr, nil, meta)
if err != nil {
writeInternalError(w, "postgres_maint_enqueue", err)
return
}
w.Header().Set("Location", "/v1/jobs/"+j.ID)
snap := j.Snapshot()
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
}
func (s *Server) handlePostgresVacuum(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresVacuum, map[string]any{
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL VACUUM",
})
}
func (s *Server) handlePostgresVacuumAnalyze(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresVacuumAnalyze, map[string]any{
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL VACUUM ANALYZE",
})
}
func (s *Server) handlePostgresAnalyze(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresAnalyze, map[string]any{
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL ANALYZE",
})
}
func (s *Server) handlePostgresReindex(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
table := body.Table
if table == "" {
table = body.Index
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresReindex, map[string]any{
"table": table, "dry_run": body.DryRun, "job_title": "PostgreSQL REINDEX",
})
}
func (s *Server) handlePostgresCleanup(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
if strings.TrimSpace(body.Policy) == "" {
writeProblem(w, http.StatusBadRequest, "Bad Request", "policy is required")
return
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresCleanup, map[string]any{
"policy": body.Policy, "dry_run": body.DryRun, "limit": body.Limit,
"job_title": "PostgreSQL cleanup",
})
}
func (s *Server) handlePostgresMaintenanceLogs(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
cursor := r.URL.Query().Get("cursor")
limit := parseLimitQuery(r, 20, 100)
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
items, next, hasMore, err := pgmonitor.ListMaintenanceLogs(ctx, s.pgMonitor.Pool(), cursor, limit)
if err != nil {
writeInternalError(w, "postgres_maint_logs", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "next_cursor": next, "has_more": hasMore})
}
func actorPrefix(a Auth) string {
if len(a.Token) >= 8 {
return a.Token[:8]
}
return a.Role
}
@@ -0,0 +1,130 @@
package httpapi
import (
"context"
"net/http"
"strconv"
"time"
)
func (s *Server) registerPostgresMonitoringRoutes(m *http.ServeMux) {
m.HandleFunc("GET /monitoring/postgres/overview", s.handlePostgresOverview)
m.HandleFunc("GET /monitoring/postgres/queries", s.handlePostgresQueries)
m.HandleFunc("GET /monitoring/postgres/locks", s.handlePostgresLocks)
m.HandleFunc("GET /monitoring/postgres/tables", s.handlePostgresTables)
m.HandleFunc("GET /monitoring/postgres/recommendations", s.handlePostgresRecommendations)
m.HandleFunc("GET /monitoring/correlation", s.handleMonitoringCorrelation)
}
func (s *Server) requirePostgres(w http.ResponseWriter) bool {
if s.pgMonitor == nil {
writeProblem(w, http.StatusServiceUnavailable, "Unavailable", "postgresql backend required")
return false
}
return true
}
func parseLimitQuery(r *http.Request, def, max int) int {
if v := r.URL.Query().Get("limit"); v != "" {
if n, err := strconv.Atoi(v); err == nil {
return n
}
}
return def
}
func (s *Server) handlePostgresOverview(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.Overview(ctx)
if err != nil {
writeInternalError(w, "postgres_overview", err)
return
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handlePostgresQueries(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.TopQueries(ctx, parseLimitQuery(r, 20, 100))
if err != nil {
writeInternalError(w, "postgres_queries", err)
return
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handlePostgresLocks(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.Locks(ctx)
if err != nil {
writeInternalError(w, "postgres_locks", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": out})
}
func (s *Server) handlePostgresTables(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.Tables(ctx, parseLimitQuery(r, 20, 100))
if err != nil {
writeInternalError(w, "postgres_tables", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": out})
}
func (s *Server) handlePostgresRecommendations(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
out, err := s.pgMonitor.Recommendations(ctx)
if err != nil {
writeInternalError(w, "postgres_recommendations", err)
return
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handleMonitoringCorrelation(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
window := 60
if v := r.URL.Query().Get("window"); v != "" {
if n, err := strconv.Atoi(v); err == nil {
window = n
}
}
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
out, err := s.pgMonitor.Correlation(ctx, window)
if err != nil {
writeInternalError(w, "monitoring_correlation", err)
return
}
writeJSON(w, http.StatusOK, out)
}
+21
View File
@@ -0,0 +1,21 @@
package httpapi
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestPostgresOverviewMemoryBackend503(t *testing.T) {
srv, err := New(Options{SeedDemo: true})
if err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodGet, "/v1/monitoring/postgres/overview", nil)
req.Header.Set("Authorization", "Bearer dev")
rec := httptest.NewRecorder()
srv.Handler().ServeHTTP(rec, req)
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
}
+14
View File
@@ -10,6 +10,7 @@ import (
"strings"
"evobgp/internal/jobs"
"evobgp/internal/pgmonitor"
"evobgp/internal/store"
"github.com/jackc/pgx/v5/pgxpool"
@@ -19,6 +20,7 @@ import (
type Server struct {
store store.Backend
pgPool *pgxpool.Pool
pgMonitor *pgmonitor.Service
jobs *jobs.Registry
bundlePriv ed25519.PrivateKey
keyResolver *apiKeyResolver
@@ -63,9 +65,14 @@ func New(opts Options) (*Server, error) {
if err != nil {
return nil, err
}
var pgMon *pgmonitor.Service
if pool != nil {
pgMon = pgmonitor.NewService(pool)
}
s := &Server{
store: backend,
pgPool: pool,
pgMonitor: pgMon,
jobs: reg,
bundlePriv: priv,
keyResolver: resolver,
@@ -89,3 +96,10 @@ func (s *Server) Store() store.Backend { return s.store }
// Jobs exposes the in-process async job registry (for scheduler / evobgp-all).
func (s *Server) Jobs() *jobs.Registry { return s.jobs }
// StartBackground starts PostgreSQL monitoring scheduler until ctx is cancelled.
func (s *Server) StartBackground(ctx context.Context) {
if s != nil && s.pgPool != nil {
pgmonitor.StartScheduler(ctx, s.pgPool)
}
}
+4 -1
View File
@@ -39,7 +39,10 @@ func speakerJSONFromStore(st store.Backend, sp *store.Speaker) map[string]any {
if strings.TrimSpace(sp.MetaJSON) != "" && sp.MetaJSON != "{}" {
var raw map[string]any
if json.Unmarshal([]byte(sp.MetaJSON), &raw) == nil {
m["meta_json"] = raw
delete(raw, "agent_secret")
if len(raw) > 0 {
m["meta_json"] = raw
}
}
}
if meta.AgentDomain != "" {
+129
View File
@@ -0,0 +1,129 @@
package httpapi
import (
"context"
"strings"
"sync"
"time"
"evobgp/internal/birdfmt"
"evobgp/internal/nodedispatch"
"evobgp/internal/store"
)
func countBGPSessions(sessions []birdfmt.BGPSession) (total, established int) {
total = len(sessions)
for _, s := range sessions {
if strings.EqualFold(strings.TrimSpace(s.State), "Established") {
established++
}
}
return total, established
}
func speakerLiveStatusJSON(sp *store.Speaker, view speakerBGPLive, health *nodedispatch.AgentHealthResult) map[string]any {
total, established := countBGPSessions(view.Sessions)
m := map[string]any{
"label": view.Label,
"bgp_poll_ok": view.Error == "",
"bgp_sessions_total": total,
"bgp_established": established,
}
if view.Error != "" {
m["bgp_poll_error"] = view.Error
}
if health != nil {
m["agent_ok"] = health.OK
if health.Error != "" {
m["agent_error"] = health.Error
}
if health.LastSyncAt != "" {
m["agent_last_sync_at"] = health.LastSyncAt
}
if health.LastAppliedRevisionID != "" {
m["agent_last_applied_revision_id"] = health.LastAppliedRevisionID
}
} else if sp != nil && strings.EqualFold(strings.TrimSpace(sp.Role), "master") {
m["agent_ok"] = view.Error == ""
if view.Error != "" {
m["agent_error"] = view.Error
}
} else if sp != nil && store.SpeakerNeedsRemoteDispatch(sp.Role, store.ParseSpeakerMeta(sp.MetaJSON)) {
m["agent_ok"] = false
m["agent_error"] = "agent health not polled"
}
if len(view.Sessions) > 0 {
sess := make([]map[string]any, 0, len(view.Sessions))
for _, s := range view.Sessions {
row := map[string]any{
"name": s.Name,
"state": s.State,
}
if strings.TrimSpace(s.Neighbor) != "" {
row["neighbor"] = s.Neighbor
}
sess = append(sess, row)
}
m["sessions"] = sess
}
return m
}
func (s *Server) collectSpeakerLiveStatus(ctx context.Context, tenantID string, fresh bool, speakers []*store.Speaker) map[string]map[string]any {
views := s.collectSpeakerBGPLive(ctx, tenantID, fresh)
viewByID := make(map[string]speakerBGPLive, len(views))
for _, v := range views {
if v.SpeakerID != "" {
viewByID[v.SpeakerID] = v
}
}
opts := nodedispatch.Options{Timeout: 8 * time.Second}
type healthWrap struct {
id string
h nodedispatch.AgentHealthResult
}
healthCh := make(chan healthWrap, len(speakers))
var wg sync.WaitGroup
for _, sp := range speakers {
if sp == nil {
continue
}
meta := store.ParseSpeakerMeta(sp.MetaJSON)
if !store.SpeakerNeedsRemoteDispatch(sp.Role, meta) {
continue
}
wg.Add(1)
go func(speaker *store.Speaker) {
defer wg.Done()
healthCh <- healthWrap{
id: speaker.ID,
h: nodedispatch.FetchAgentHealth(ctx, speaker, opts),
}
}(sp)
}
wg.Wait()
close(healthCh)
healthByID := make(map[string]nodedispatch.AgentHealthResult, len(speakers))
for hw := range healthCh {
healthByID[hw.id] = hw.h
}
out := make(map[string]map[string]any, len(speakers))
for _, sp := range speakers {
if sp == nil {
continue
}
view, ok := viewByID[sp.ID]
if !ok {
view = speakerBGPLive{SpeakerID: sp.ID, Label: speakerDisplayLabel(sp)}
}
var hp *nodedispatch.AgentHealthResult
if h, ok := healthByID[sp.ID]; ok {
hCopy := h
hp = &hCopy
}
out[sp.ID] = speakerLiveStatusJSON(sp, view, hp)
}
return out
}
+74
View File
@@ -0,0 +1,74 @@
package httpapi
import (
"testing"
"evobgp/internal/birdfmt"
"evobgp/internal/nodedispatch"
"evobgp/internal/store"
)
func TestCountBGPSessions(t *testing.T) {
total, est := countBGPSessions([]birdfmt.BGPSession{
{Name: "p1", State: "Established"},
{Name: "p2", State: "Idle"},
{Name: "p3", State: "established"},
})
if total != 3 || est != 2 {
t.Fatalf("total=%d established=%d", total, est)
}
}
func TestSpeakerLiveStatusJSON_masterUsesBirdPoll(t *testing.T) {
sp := &store.Speaker{ID: "m1", Role: "master", Endpoint: "https://cp.example"}
view := speakerBGPLive{
SpeakerID: "m1",
Label: "CP · cp.example",
Sessions: []birdfmt.BGPSession{
{Name: "evobgp_peer_x", State: "Established"},
},
}
m := speakerLiveStatusJSON(sp, view, nil)
if m["agent_ok"] != true || m["bgp_established"] != 1 || m["bgp_sessions_total"] != 1 {
t.Fatalf("got %#v", m)
}
}
func TestSpeakerLiveStatusJSON_replicaWithHealth(t *testing.T) {
sp := &store.Speaker{
ID: "r1",
Role: "replica",
Endpoint: "https://node.example",
MetaJSON: `{"agent_domain":"node.example","agent_secret":"s"}`,
}
view := speakerBGPLive{
SpeakerID: "r1",
Label: "node.example",
Sessions: []birdfmt.BGPSession{{Name: "p", State: "Idle"}},
}
health := &nodedispatch.AgentHealthResult{
OK: true,
LastSyncAt: "2026-05-21T12:00:00Z",
LastAppliedRevisionID: "rev-1",
}
m := speakerLiveStatusJSON(sp, view, health)
if m["agent_ok"] != true || m["agent_last_sync_at"] != "2026-05-21T12:00:00Z" {
t.Fatalf("got %#v", m)
}
if m["bgp_established"] != 0 || m["bgp_poll_ok"] != true {
t.Fatalf("bgp fields: %#v", m)
}
}
func TestSpeakerLiveStatusJSON_pollError(t *testing.T) {
sp := &store.Speaker{ID: "r1", Role: "replica", MetaJSON: `{"agent_domain":"x.example"}`}
view := speakerBGPLive{SpeakerID: "r1", Label: "x.example", Error: "HTTP 503"}
health := &nodedispatch.AgentHealthResult{OK: false, Error: "timeout"}
m := speakerLiveStatusJSON(sp, view, health)
if m["bgp_poll_ok"] != false || m["bgp_poll_error"] != "HTTP 503" {
t.Fatalf("got %#v", m)
}
if m["agent_ok"] != false {
t.Fatalf("agent_ok: %#v", m)
}
}
+62
View File
@@ -0,0 +1,62 @@
package httpapi
import (
"encoding/json"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestGetSpeaker_redactsAgentSecret(t *testing.T) {
srv, err := New(Options{InsecureDev: true, SeedDemo: true, BundleSeedHex: testBundleSeed})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
tenant, _, _, _, demoSpk := srv.Store().DemoIDs()
mustSetTestAPIKeys(t, srv, "vwkey|"+tenant+"|viewer")
req := httptest.NewRequest(http.MethodGet, "/v1/speakers/"+demoSpk, nil)
req.Header.Set("Authorization", "Bearer vwkey")
rec := httptest.NewRecorder()
srv.Handler().ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status %d body %s", rec.Code, rec.Body.String())
}
var out map[string]any
if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
t.Fatal(err)
}
if out["agent_secret"] != nil {
t.Fatalf("agent_secret must not appear at top level: %#v", out["agent_secret"])
}
meta, _ := out["meta_json"].(map[string]any)
if meta != nil {
if v, ok := meta["agent_secret"]; ok && v != nil && v != "" {
t.Fatalf("agent_secret must be redacted from meta_json: %#v", v)
}
}
}
func TestListSpeakers_redactsAgentSecret(t *testing.T) {
srv, err := New(Options{InsecureDev: true, SeedDemo: true, BundleSeedHex: testBundleSeed})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
tenant, _, _, _, _ := srv.Store().DemoIDs()
mustSetTestAPIKeys(t, srv, "vwkey|"+tenant+"|viewer")
req := httptest.NewRequest(http.MethodGet, "/v1/speakers", nil)
req.Header.Set("Authorization", "Bearer vwkey")
rec := httptest.NewRecorder()
srv.Handler().ServeHTTP(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status %d body %s", rec.Code, rec.Body.String())
}
body := rec.Body.String()
if strings.Contains(body, "agent_secret") {
t.Fatalf("list response must not contain agent_secret: %s", body)
}
}
+55
View File
@@ -0,0 +1,55 @@
package httpclient
import (
"sync"
"time"
)
const (
defaultBreakerThreshold = 5
defaultBreakerCooldown = 30 * time.Second
)
type hostBreaker struct {
mu sync.Mutex
failures int
openUntil time.Time
}
var hostBreakers sync.Map // string -> *hostBreaker
func breakerForHost(host string) *hostBreaker {
if host == "" {
host = "_"
}
v, _ := hostBreakers.LoadOrStore(host, &hostBreaker{})
return v.(*hostBreaker)
}
func (b *hostBreaker) allow() bool {
b.mu.Lock()
defer b.mu.Unlock()
return time.Now().After(b.openUntil)
}
func (b *hostBreaker) recordSuccess() {
b.mu.Lock()
defer b.mu.Unlock()
b.failures = 0
b.openUntil = time.Time{}
}
func (b *hostBreaker) recordFailure() {
b.mu.Lock()
defer b.mu.Unlock()
b.failures++
if b.failures >= defaultBreakerThreshold {
b.openUntil = time.Now().Add(defaultBreakerCooldown)
b.failures = 0
}
}
// ResetHostBreakers clears all circuit breakers (tests only).
func ResetHostBreakers() {
hostBreakers = sync.Map{}
}
+34
View File
@@ -0,0 +1,34 @@
package httpclient
import (
"context"
"net/http"
"net/http/httptest"
"sync/atomic"
"testing"
"time"
)
func TestDoWithBreaker_opensAfterFailures(t *testing.T) {
ResetHostBreakers()
var calls atomic.Int32
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls.Add(1)
http.Error(w, "fail", http.StatusBadGateway)
}))
defer srv.Close()
hc := New(5 * time.Second)
for i := 0; i < defaultBreakerThreshold*3; i++ {
req, _ := http.NewRequest(http.MethodGet, srv.URL, nil)
_, _ = DoWithBreaker(context.Background(), hc, req, 1)
}
req, _ := http.NewRequest(http.MethodGet, srv.URL, nil)
_, err := DoWithBreaker(context.Background(), hc, req, 1)
if err == nil || err.Error() == "" {
t.Fatal("expected circuit open error")
}
if got := calls.Load(); got == 0 {
t.Fatal("expected at least one upstream call")
}
}
+87
View File
@@ -0,0 +1,87 @@
// Package httpclient provides shared HTTP clients and retry helpers for outbound calls.
package httpclient
import (
"context"
"fmt"
"io"
"net/http"
"time"
)
const DefaultTimeout = 45 * time.Second
// New returns an HTTP client with timeout and tuned idle connection pooling.
func New(timeout time.Duration) *http.Client {
if timeout <= 0 {
timeout = DefaultTimeout
}
tr := http.DefaultTransport.(*http.Transport).Clone()
tr.MaxIdleConns = 100
tr.MaxIdleConnsPerHost = 10
return &http.Client{Timeout: timeout, Transport: tr}
}
// DoWithRetry executes hc.Do(req) up to maxAttempts times with linear backoff.
func DoWithRetry(ctx context.Context, hc *http.Client, req *http.Request, maxAttempts int) (*http.Response, error) {
if maxAttempts <= 0 {
maxAttempts = 3
}
var lastErr error
for attempt := 0; attempt < maxAttempts; attempt++ {
if attempt > 0 {
wait := time.Duration(attempt) * 2 * time.Second
select {
case <-ctx.Done():
return nil, ctx.Err()
case <-time.After(wait):
}
if req.GetBody != nil {
body, err := req.GetBody()
if err != nil {
return nil, err
}
req.Body = body
}
}
reqClone := req.Clone(ctx)
resp, err := hc.Do(reqClone)
if err != nil {
lastErr = err
continue
}
if resp.StatusCode >= 500 {
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
lastErr = fmt.Errorf("httpclient: upstream %s", resp.Status)
continue
}
return resp, nil
}
if lastErr != nil {
return nil, lastErr
}
return nil, fmt.Errorf("httpclient: request failed after %d attempts", maxAttempts)
}
// DoWithBreaker applies per-host circuit breaking then retries transient failures.
func DoWithBreaker(ctx context.Context, hc *http.Client, req *http.Request, maxAttempts int) (*http.Response, error) {
if req == nil || req.URL == nil {
return nil, fmt.Errorf("httpclient: nil request")
}
br := breakerForHost(req.URL.Hostname())
if !br.allow() {
return nil, fmt.Errorf("httpclient: circuit open for %s", req.URL.Hostname())
}
resp, err := DoWithRetry(ctx, hc, req, maxAttempts)
if err != nil {
br.recordFailure()
return nil, err
}
if resp.StatusCode >= 500 {
br.recordFailure()
return resp, nil
}
br.recordSuccess()
return resp, nil
}
+39
View File
@@ -0,0 +1,39 @@
package httpclient
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
)
func TestDoWithRetry_retriesOn500(t *testing.T) {
var calls int
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
if calls < 3 {
http.Error(w, "fail", http.StatusBadGateway)
return
}
w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte("ok"))
}))
defer srv.Close()
req, err := http.NewRequest(http.MethodGet, srv.URL, nil)
if err != nil {
t.Fatal(err)
}
resp, err := DoWithRetry(context.Background(), New(5*time.Second), req, 3)
if err != nil {
t.Fatal(err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
t.Fatalf("status %d", resp.StatusCode)
}
if calls != 3 {
t.Fatalf("want 3 calls, got %d", calls)
}
}
+12 -3
View File
@@ -3,11 +3,11 @@ package ingest
import (
"context"
"log"
"net/http"
"time"
"evobgp/internal/broker"
"evobgp/internal/config"
"evobgp/internal/httpclient"
"evobgp/internal/pipeline"
"evobgp/internal/store"
)
@@ -17,6 +17,8 @@ type Deps struct {
Store store.Backend
}
var lastMaintenance time.Time
// Run blocks until ctx is cancelled.
func Run(ctx context.Context, deps *Deps) {
cfg := config.Load()
@@ -24,7 +26,7 @@ func Run(ctx context.Context, deps *Deps) {
if deps == nil || deps.Store == nil {
log.Fatalf("evobgp-ingest: missing store (pass ingest.Deps from BootstrapWorkers or evobgp-all)")
}
hc := &http.Client{Timeout: 45 * time.Second}
hc := httpclient.New(httpclient.DefaultTimeout)
t := time.NewTicker(60 * time.Second)
defer t.Stop()
log.Printf("evobgp-ingest: active (CDN conditional GET / ETag prefetch)")
@@ -34,7 +36,14 @@ func Run(ctx context.Context, deps *Deps) {
log.Printf("evobgp-ingest: stopped")
return
case <-t.C:
if err := pipeline.PrefetchCDNSourceETags(context.Background(), deps.Store, hc); err != nil {
if deps.Store != nil && time.Since(lastMaintenance) > time.Hour {
deps.Store.RunPeriodicMaintenance(ctx)
lastMaintenance = time.Now()
}
prefetchCtx, cancel := context.WithTimeout(ctx, 50*time.Second)
err := pipeline.PrefetchCDNSourceETags(prefetchCtx, deps.Store, hc)
cancel()
if err != nil {
log.Printf("evobgp-ingest: prefetch: %v", err)
}
}
+49 -4
View File
@@ -182,6 +182,8 @@ type Registry struct {
workerStart func(j *Job)
workerSem chan struct{}
onTerminal func(j *Job)
onEnqueued func(j *Job)
onRunning func(j *Job)
}
type idempoKey struct {
@@ -209,6 +211,44 @@ func (r *Registry) SetTerminalHook(fn func(j *Job)) {
r.onTerminal = fn
}
// SetPersistHooks registers best-effort callbacks for job lifecycle persistence.
func (r *Registry) SetPersistHooks(onEnqueued, onRunning, onTerminal func(j *Job)) {
if r == nil {
return
}
r.mu.Lock()
defer r.mu.Unlock()
r.onEnqueued = onEnqueued
r.onRunning = onRunning
if onTerminal != nil {
r.onTerminal = onTerminal
}
}
func (r *Registry) fireEnqueued(j *Job) {
if r == nil || j == nil {
return
}
r.mu.RLock()
fn := r.onEnqueued
r.mu.RUnlock()
if fn != nil {
fn(j)
}
}
func (r *Registry) fireRunning(j *Job) {
if r == nil || j == nil {
return
}
r.mu.RLock()
fn := r.onRunning
r.mu.RUnlock()
if fn != nil {
fn(j)
}
}
func (r *Registry) fireTerminal(j *Job) {
if r == nil || j == nil {
return
@@ -271,8 +311,6 @@ func (r *Registry) pruneTerminalIfOver(maxJobs int) {
// Enqueue creates a job or returns an existing one for the same idempotency key.
func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, moduleID *string, meta map[string]any) (*Job, bool, error) {
r.mu.Lock()
defer r.mu.Unlock()
maxJobs := registryMaxJobsFromEnv()
r.pruneTerminalIfOver(maxJobs)
@@ -281,6 +319,7 @@ func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, module
if existing, ok := r.byIdempo[k]; ok {
st := existing.statusLocked()
if st == StatusQueued || st == StatusRunning {
r.mu.Unlock()
return existing, false, nil
}
delete(r.byIdempo, k)
@@ -302,8 +341,14 @@ func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, module
}
r.byID[j.ID] = j
r.pruneTerminalIfOver(maxJobs)
enqueuedHook := r.onEnqueued
workerStart := r.workerStart
r.mu.Unlock()
if r.workerStart != nil {
if enqueuedHook != nil {
enqueuedHook(j)
}
if workerStart != nil {
go func() {
r.workerSem <- struct{}{}
active := len(r.workerSem)
@@ -313,7 +358,7 @@ func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, module
<-r.workerSem
observability.RecordJobQueueDepth(len(r.workerSem), capacity)
}()
r.workerStart(j)
workerStart(j)
}()
}
return j, true, nil
+169
View File
@@ -0,0 +1,169 @@
package jobs
import (
"fmt"
"strings"
"evobgp/internal/pgmonitor"
)
func (w *Worker) pgService() *pgmonitor.Service {
if w == nil || w.PgPool == nil {
return nil
}
return pgmonitor.NewService(w.PgPool)
}
func (w *Worker) runPostgresMetricsRefresh(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.RefreshMetricsSnapshot(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresSlowQueryAgg(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.AggregateSlowQueries(ctx, 30); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresTableBloat(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.EstimateTableBloat(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresIndexUsage(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.AnalyzeIndexUsage(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresAutovacuumLag(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.DetectAutovacuumLag(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresMaint(j *Job, kind string) {
if w == nil || w.PgPool == nil {
j.Fail("postgresql not configured")
return
}
table, _ := j.Meta["table"].(string)
dryRun, _ := j.Meta["dry_run"].(bool)
actor, _ := j.Meta["actor_prefix"].(string)
ctx, cancel := j.workContext()
defer cancel()
auditID, _ := pgmonitor.InsertMaintenanceAudit(ctx, w.PgPool, j.TenantID, actor, kind, table, dryRun)
detail, err := pgmonitor.ExecMaintenance(ctx, w.PgPool, kind, table, dryRun)
var errMsg *string
status := StatusSucceeded
if err != nil {
s := err.Error()
errMsg = &s
status = StatusFailed
j.Fail(s)
} else {
j.mergeMeta(map[string]any{"maintenance": detail, "audit_id": auditID})
j.Succeed()
}
if auditID != "" {
_ = pgmonitor.FinishMaintenanceAudit(ctx, w.PgPool, auditID, status, detail, errMsg)
}
}
func (w *Worker) runPostgresCleanup(j *Job) {
if w == nil || w.PgPool == nil {
j.Fail("postgresql not configured")
return
}
policy, _ := j.Meta["policy"].(string)
dryRun, _ := j.Meta["dry_run"].(bool)
limit := 0
if v, ok := j.Meta["limit"].(float64); ok {
limit = int(v)
}
actor, _ := j.Meta["actor_prefix"].(string)
ctx, cancel := j.workContext()
defer cancel()
auditID, _ := pgmonitor.InsertMaintenanceAudit(ctx, w.PgPool, j.TenantID, actor, "cleanup", policy, dryRun)
detail, err := pgmonitor.RunCleanup(ctx, w.PgPool, strings.TrimSpace(policy), dryRun, limit)
var errMsg *string
status := StatusSucceeded
if err != nil {
s := err.Error()
errMsg = &s
status = StatusFailed
j.Fail(s)
} else {
j.mergeMeta(map[string]any{"cleanup": detail, "audit_id": auditID})
j.Succeed()
}
if auditID != "" {
_ = pgmonitor.FinishMaintenanceAudit(ctx, w.PgPool, auditID, status, detail, errMsg)
}
}
// EnqueuePostgresAnalyzerJobs enqueues periodic analyzer jobs (global tenant id).
func EnqueuePostgresAnalyzerJobs(reg *Registry, tenantID string) {
if reg == nil || tenantID == "" {
return
}
kinds := []string{
KindPostgresMetricsRefresh,
KindPostgresSlowQueryAgg,
KindPostgresTableBloat,
KindPostgresIndexUsage,
KindPostgresAutovacuumLag,
}
for _, k := range kinds {
key := fmt.Sprintf("pgmon-%s-%s", k, tenantID)
idem := key
_, _, _ = reg.Enqueue(tenantID, k, &idem, nil, map[string]any{"trigger": "scheduler"})
}
}
+44 -7
View File
@@ -13,10 +13,13 @@ import (
"evobgp/internal/birddeploy"
"evobgp/internal/birdfmt"
"evobgp/internal/httpclient"
"evobgp/internal/nodedispatch"
"evobgp/internal/observability"
"evobgp/internal/pipeline"
"evobgp/internal/store"
"github.com/jackc/pgx/v5/pgxpool"
)
// mergeBirdPostApplyMeta attaches a birdc snapshot after deploy/reload (best-effort).
@@ -42,17 +45,28 @@ func mergeBirdPostApplyMeta(j *Job) {
}
const (
KindModuleRefresh = "module_refresh"
KindTenantRefresh = "tenant_refresh"
KindPeerReconcile = "peer_reconcile"
KindDeployApply = "deploy_apply"
KindRevisionRollback = "revision_rollback"
KindBirdReload = "bird_reload"
KindModuleRefresh = "module_refresh"
KindTenantRefresh = "tenant_refresh"
KindPeerReconcile = "peer_reconcile"
KindDeployApply = "deploy_apply"
KindRevisionRollback = "revision_rollback"
KindBirdReload = "bird_reload"
KindPostgresMetricsRefresh = "postgres_metrics_refresh"
KindPostgresSlowQueryAgg = "postgres_slow_query_aggregate"
KindPostgresTableBloat = "postgres_table_bloat_estimate"
KindPostgresIndexUsage = "postgres_index_usage_analyze"
KindPostgresAutovacuumLag = "postgres_autovacuum_lag_detect"
KindPostgresVacuum = "postgres_vacuum"
KindPostgresVacuumAnalyze = "postgres_vacuum_analyze"
KindPostgresAnalyze = "postgres_analyze"
KindPostgresReindex = "postgres_reindex"
KindPostgresCleanup = "postgres_cleanup"
)
// Worker executes queued jobs against store.Backend (memory or SQL).
type Worker struct {
Store store.Backend
PgPool *pgxpool.Pool
HTTPClient *http.Client // optional; CDN refresh uses this (default 45s timeout).
// Registry is set after BootstrapWorkers creates the job queue; used to chain deploy_apply after refresh/rollback.
Registry *Registry
@@ -70,7 +84,7 @@ type revisionLogEntry struct {
Message string `json:"message"`
}
var defaultWorkerHTTP = &http.Client{Timeout: 45 * time.Second}
var defaultWorkerHTTP = httpclient.New(httpclient.DefaultTimeout)
func (w *Worker) httpClient() *http.Client {
if w != nil && w.HTTPClient != nil {
@@ -94,6 +108,9 @@ func (w *Worker) Process(j *Job) {
return
}
j.MarkRunning()
if w != nil && w.Registry != nil {
w.Registry.fireRunning(j)
}
if j.IsCancelRequested() {
j.MarkCancelled()
return
@@ -151,6 +168,26 @@ func (w *Worker) Process(j *Job) {
}
mergeBirdPostApplyMeta(j)
j.Succeed()
case KindPostgresMetricsRefresh:
w.runPostgresMetricsRefresh(j)
case KindPostgresSlowQueryAgg:
w.runPostgresSlowQueryAgg(j)
case KindPostgresTableBloat:
w.runPostgresTableBloat(j)
case KindPostgresIndexUsage:
w.runPostgresIndexUsage(j)
case KindPostgresAutovacuumLag:
w.runPostgresAutovacuumLag(j)
case KindPostgresVacuum:
w.runPostgresMaint(j, "vacuum")
case KindPostgresVacuumAnalyze:
w.runPostgresMaint(j, "vacuum_analyze")
case KindPostgresAnalyze:
w.runPostgresMaint(j, "analyze")
case KindPostgresReindex:
w.runPostgresMaint(j, "reindex")
case KindPostgresCleanup:
w.runPostgresCleanup(j)
default:
j.Fail("unknown job kind")
}
+11 -2
View File
@@ -16,6 +16,7 @@ import (
"evobgp/internal/birdfmt"
"evobgp/internal/bundle"
"evobgp/internal/httpclient"
"evobgp/internal/signing"
)
@@ -55,6 +56,10 @@ func CmdPullBundle(args []string) int {
return 0
}
func nodeHTTPClient() *http.Client {
return httpclient.New(60 * time.Second)
}
func fetchLatestRevision(base, token, speaker string) (string, error) {
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/revisions/latest"
req, err := http.NewRequest(http.MethodGet, u, nil)
@@ -62,7 +67,9 @@ func fetchLatestRevision(base, token, speaker string) (string, error) {
return "", err
}
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
resp, err := httpclient.DoWithRetry(ctx, nodeHTTPClient(), req, 3)
if err != nil {
return "", err
}
@@ -90,7 +97,9 @@ func fetchBundle(base, token, speaker, revision string) ([]byte, error) {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
ctx, cancel := context.WithTimeout(context.Background(), 120*time.Second)
defer cancel()
resp, err := httpclient.DoWithRetry(ctx, nodeHTTPClient(), req, 3)
if err != nil {
return nil, err
}
+69
View File
@@ -0,0 +1,69 @@
package nodedispatch
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"evobgp/internal/birdfmt"
"evobgp/internal/store"
)
// BirdProtocolsResult is agent birdc scrape outcome.
type BirdProtocolsResult struct {
SpeakerID string `json:"speaker_id,omitempty"`
Sessions []birdfmt.BGPSession `json:"sessions"`
Error string `json:"error,omitempty"`
}
// FetchBirdProtocols GETs /v1/agent/bird/protocols on a replica agent.
func FetchBirdProtocols(ctx context.Context, sp *store.Speaker, opts Options) BirdProtocolsResult {
res := BirdProtocolsResult{}
if sp != nil {
res.SpeakerID = sp.ID
}
if sp == nil {
res.Error = "nil speaker"
return res
}
meta := store.ParseSpeakerMeta(sp.MetaJSON)
url := store.AgentBirdProtocolsURL(meta)
if url == "" {
res.Error = "agent_domain not configured"
return res
}
secret := strings.TrimSpace(meta.AgentSecret)
if secret == "" {
res.Error = "agent_secret missing"
return res
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
res.Error = err.Error()
return res
}
req.Header.Set("Authorization", "Bearer "+secret)
resp, err := opts.client().Do(req)
if err != nil {
res.Error = err.Error()
return res
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
res.Error = fmt.Sprintf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(b)))
return res
}
var out struct {
Sessions []birdfmt.BGPSession `json:"sessions"`
}
if err := json.Unmarshal(b, &out); err != nil {
res.Error = err.Error()
return res
}
res.Sessions = out.Sessions
return res
}
+7 -26
View File
@@ -156,33 +156,14 @@ func WakeReplicas(ctx context.Context, st store.Backend, tenantID, revisionID st
return out
}
// CheckHealth GETs /v1/agent/health for UI Connected/Offline status.
// CheckHealth is deprecated; use FetchAgentHealth.
func CheckHealth(ctx context.Context, sp *store.Speaker, opts Options) (ok bool, detail string) {
if sp == nil {
return false, "nil speaker"
}
meta := store.ParseSpeakerMeta(sp.MetaJSON)
url := store.AgentHealthURL(meta)
if url == "" {
return false, "agent_domain not configured"
}
secret := strings.TrimSpace(meta.AgentSecret)
if secret == "" {
return false, "agent_secret missing"
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return false, err.Error()
}
req.Header.Set("Authorization", "Bearer "+secret)
resp, err := opts.client().Do(req)
if err != nil {
return false, err.Error()
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode >= 200 && resp.StatusCode < 300 {
res := FetchAgentHealth(ctx, sp, opts)
if res.OK {
return true, "connected"
}
b, _ := io.ReadAll(resp.Body)
return false, fmt.Sprintf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(b)))
if res.Error != "" {
return false, res.Error
}
return false, "agent unhealthy"
}
+69
View File
@@ -0,0 +1,69 @@
package nodedispatch
import (
"context"
"encoding/json"
"fmt"
"io"
"net/http"
"strings"
"evobgp/internal/store"
)
// AgentHealthResult is the parsed outcome of GET /v1/agent/health on a replica.
type AgentHealthResult struct {
OK bool
Error string
LastAppliedRevisionID string
LastSyncAt string
}
// FetchAgentHealth GETs /v1/agent/health for UI Connected/Offline status.
func FetchAgentHealth(ctx context.Context, sp *store.Speaker, opts Options) AgentHealthResult {
if sp == nil {
return AgentHealthResult{Error: "nil speaker"}
}
meta := store.ParseSpeakerMeta(sp.MetaJSON)
url := store.AgentHealthURL(meta)
if url == "" {
return AgentHealthResult{Error: "agent_domain not configured"}
}
secret := strings.TrimSpace(meta.AgentSecret)
if secret == "" {
return AgentHealthResult{Error: "agent_secret missing"}
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
if err != nil {
return AgentHealthResult{Error: err.Error()}
}
req.Header.Set("Authorization", "Bearer "+secret)
resp, err := opts.client().Do(req)
if err != nil {
return AgentHealthResult{Error: err.Error()}
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return AgentHealthResult{
Error: fmt.Sprintf("HTTP %d: %s", resp.StatusCode, strings.TrimSpace(string(b))),
}
}
var out struct {
OK bool `json:"ok"`
LastAppliedRevisionID string `json:"last_applied_revision_id"`
LastSyncAt string `json:"last_sync_at"`
}
if err := json.Unmarshal(b, &out); err != nil {
return AgentHealthResult{Error: err.Error()}
}
res := AgentHealthResult{
OK: out.OK,
LastAppliedRevisionID: strings.TrimSpace(out.LastAppliedRevisionID),
LastSyncAt: strings.TrimSpace(out.LastSyncAt),
}
if !res.OK {
res.Error = "agent reported ok=false"
}
return res
}
+37
View File
@@ -0,0 +1,37 @@
package pgmonitor
import (
"sync"
"time"
)
type cacheEntry struct {
at time.Time
data any
}
type ttlCache struct {
mu sync.RWMutex
ttl time.Duration
items map[string]cacheEntry
}
func newTTLCache(ttl time.Duration) *ttlCache {
return &ttlCache{ttl: ttl, items: make(map[string]cacheEntry)}
}
func (c *ttlCache) get(key string) (any, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
e, ok := c.items[key]
if !ok || time.Since(e.at) > c.ttl {
return nil, false
}
return e.data, true
}
func (c *ttlCache) set(key string, data any) {
c.mu.Lock()
defer c.mu.Unlock()
c.items[key] = cacheEntry{at: time.Now().UTC(), data: data}
}
+83
View File
@@ -0,0 +1,83 @@
package pgmonitor
import (
"context"
"fmt"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// Correlation builds aligned timeline points from job_audit and overview cache.
func (s *Service) Correlation(ctx context.Context, windowMinutes int) (CorrelationResponse, error) {
if s == nil || s.pool == nil {
return CorrelationResponse{}, fmt.Errorf("pgmonitor: postgres not configured")
}
if windowMinutes <= 0 {
windowMinutes = 60
}
if windowMinutes > 1440 {
windowMinutes = 1440
}
since := time.Now().UTC().Add(-time.Duration(windowMinutes) * time.Minute)
rows, err := s.pool.Query(ctx, `
SELECT date_trunc('minute', finished_at) AS bucket,
percentile_cont(0.99) WITHIN GROUP (ORDER BY
EXTRACT(EPOCH FROM (finished_at - started_at)) * 1000)
FROM job_audit
WHERE finished_at >= $1 AND kind IN ('module_refresh', 'tenant_refresh')
AND status = 'succeeded' AND started_at IS NOT NULL
GROUP BY 1
ORDER BY 1`, since)
if err != nil {
return CorrelationResponse{}, fmt.Errorf("pgmonitor: correlation jobs: %w", err)
}
defer rows.Close()
points := make(map[time.Time]*CorrelationPoint)
for rows.Next() {
var bucket time.Time
var p99 *float64
if err := rows.Scan(&bucket, &p99); err != nil {
return CorrelationResponse{}, err
}
bucket = bucket.UTC()
pt := points[bucket]
if pt == nil {
pt = &CorrelationPoint{Timestamp: bucket}
points[bucket] = pt
}
if p99 != nil {
pt.PipelineRefreshP99Ms = *p99
}
}
ov, err := s.Overview(ctx)
if err == nil && ov.Database.CacheHitPct > 0 {
now := time.Now().UTC().Truncate(time.Minute)
pt := points[now]
if pt == nil {
pt = &CorrelationPoint{Timestamp: now}
points[now] = pt
}
pt.CacheHitPct = ov.Database.CacheHitPct
}
out := make([]CorrelationPoint, 0, len(points))
for _, p := range points {
out = append(out, *p)
}
// simple sort by time
for i := 0; i < len(out); i++ {
for j := i + 1; j < len(out); j++ {
if out[j].Timestamp.Before(out[i].Timestamp) {
out[i], out[j] = out[j], out[i]
}
}
}
return CorrelationResponse{WindowMinutes: windowMinutes, Points: out}, nil
}
// RecordCorrelationSnapshot is a hook for future Prometheus samples (no-op placeholder).
func RecordCorrelationSnapshot(_ *pgxpool.Pool) {}
+154
View File
@@ -0,0 +1,154 @@
package pgmonitor
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgxpool"
)
// CleanupPolicy names safe retention policies.
type CleanupPolicy string
const (
PolicyJobAuditRetention CleanupPolicy = "job_audit_retention"
PolicyASNCacheRetention CleanupPolicy = "asn_cache_retention"
)
// CleanupRequest for POST /postgres/cleanup.
type CleanupRequest struct {
Policy string `json:"policy"`
DryRun bool `json:"dry_run"`
Limit int `json:"limit"`
}
// RunCleanup executes a named retention policy.
func RunCleanup(ctx context.Context, pool *pgxpool.Pool, policy string, dryRun bool, limit int) (map[string]any, error) {
if pool == nil {
return nil, fmt.Errorf("pgmonitor: postgres not configured")
}
if limit <= 0 {
limit = 10000
}
if limit > 100000 {
limit = 100000
}
detail := map[string]any{"policy": policy, "dry_run": dryRun, "limit": limit}
switch CleanupPolicy(policy) {
case PolicyJobAuditRetention:
cutoff := time.Now().UTC().Add(-90 * 24 * time.Hour)
if dryRun {
var n int64
err := pool.QueryRow(ctx, `
SELECT count(*) FROM job_audit
WHERE created_at < $1 AND status IN ('succeeded', 'failed', 'cancelled')`, cutoff).Scan(&n)
detail["would_delete"] = n
return detail, err
}
tag, err := pool.Exec(ctx, `
DELETE FROM job_audit
WHERE id IN (
SELECT id FROM job_audit
WHERE created_at < $1 AND status IN ('succeeded', 'failed', 'cancelled')
LIMIT $2
)`, cutoff, limit)
if err != nil {
return detail, err
}
detail["deleted"] = tag.RowsAffected()
return detail, nil
case PolicyASNCacheRetention:
cutoff := time.Now().UTC().Add(-7 * 24 * time.Hour)
if dryRun {
var n int64
err := pool.QueryRow(ctx, `SELECT count(*) FROM asn_prefix_cache WHERE fetched_at < $1`, cutoff).Scan(&n)
detail["would_delete"] = n
return detail, err
}
tag, err := pool.Exec(ctx, `
DELETE FROM asn_prefix_cache WHERE fetched_at < $1`, cutoff)
if err != nil {
return detail, err
}
detail["deleted"] = tag.RowsAffected()
return detail, nil
default:
return nil, fmt.Errorf("pgmonitor: unknown cleanup policy %q", policy)
}
}
// InsertMaintenanceAudit records an audit row at job start.
func InsertMaintenanceAudit(ctx context.Context, pool *pgxpool.Pool, tenantID, actorPrefix, kind, table string, dryRun bool) (string, error) {
id := uuid.New().String()
_, err := pool.Exec(ctx, `
INSERT INTO postgres_maintenance_audit
(id, tenant_id, actor_prefix, kind, target_table, dry_run, status, created_at)
VALUES ($1, NULLIF($2,''), NULLIF($3,''), $4, NULLIF($5,''), $6, 'running', now())`,
id, tenantID, actorPrefix, kind, table, dryRun)
return id, err
}
// FinishMaintenanceAudit updates terminal state.
func FinishMaintenanceAudit(ctx context.Context, pool *pgxpool.Pool, id, status string, detail map[string]any, errMsg *string) error {
var detailJSON []byte
if detail != nil {
detailJSON, _ = json.Marshal(detail)
}
_, err := pool.Exec(ctx, `
UPDATE postgres_maintenance_audit
SET status = $2, detail_json = $3::jsonb, error_message = $4,
finished_at = now(), started_at = COALESCE(started_at, now())
WHERE id = $1`,
id, status, string(detailJSON), errMsg)
return err
}
// ListMaintenanceLogs returns paginated audit rows.
func ListMaintenanceLogs(ctx context.Context, pool *pgxpool.Pool, cursor string, limit int) ([]MaintenanceLogRow, string, bool, error) {
limit = clampLimit(limit, 20, 100)
args := []any{limit + 1}
q := `
SELECT id, COALESCE(tenant_id,''), COALESCE(actor_prefix,''), kind,
COALESCE(target_table,''), dry_run, status,
detail_json, COALESCE(error_message,''), created_at, started_at, finished_at
FROM postgres_maintenance_audit`
if cursor != "" {
q += ` WHERE created_at < (SELECT created_at FROM postgres_maintenance_audit WHERE id = $2)`
args = append(args, cursor)
}
q += ` ORDER BY created_at DESC LIMIT $1`
rows, err := pool.Query(ctx, q, args...)
if err != nil {
return nil, "", false, err
}
defer rows.Close()
var out []MaintenanceLogRow
for rows.Next() {
var r MaintenanceLogRow
var detailRaw []byte
var started, finished *time.Time
if err := rows.Scan(&r.ID, &r.TenantID, &r.ActorPrefix, &r.Kind, &r.TargetTable,
&r.DryRun, &r.Status, &detailRaw, &r.Error, &r.CreatedAt, &started, &finished); err != nil {
return nil, "", false, err
}
r.StartedAt = started
r.FinishedAt = finished
if len(detailRaw) > 0 {
_ = json.Unmarshal(detailRaw, &r.Detail)
}
out = append(out, r)
}
hasMore := len(out) > limit
if hasMore {
out = out[:limit]
}
next := ""
if hasMore && len(out) > 0 {
next = out[len(out)-1].ID
}
return out, next, hasMore, rows.Err()
}
+350
View File
@@ -0,0 +1,350 @@
package pgmonitor
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
)
func clampLimit(limit, def, max int) int {
if limit <= 0 {
return def
}
if limit > max {
return max
}
return limit
}
func (s *Service) fetchOverview(ctx context.Context) (Overview, error) {
now := time.Now().UTC()
out := Overview{CollectedAt: now}
var active, idle, total, maxConn int
err := s.pool.QueryRow(ctx, `
SELECT
count(*) FILTER (WHERE state = 'active'),
count(*) FILTER (WHERE state = 'idle'),
count(*),
(SELECT setting::int FROM pg_settings WHERE name = 'max_connections')
FROM pg_stat_activity
WHERE datname = current_database()`).Scan(&active, &idle, &total, &maxConn)
if err != nil {
return out, fmt.Errorf("pgmonitor: connections: %w", err)
}
out.Connections = Connections{Active: active, Idle: idle, Total: total, MaxConnections: maxConn}
var cachePct *float64
err = s.pool.QueryRow(ctx, `
SELECT numbackends, xact_commit, xact_rollback, deadlocks, blks_hit, blks_read,
CASE WHEN blks_hit + blks_read > 0
THEN round(100.0 * blks_hit::numeric / (blks_hit + blks_read), 2) END
FROM pg_stat_database WHERE datname = current_database()`).Scan(
&out.Database.Backends,
&out.Database.XactCommit,
&out.Database.XactRollback,
&out.Database.Deadlocks,
&out.Database.BlksHit,
&out.Database.BlksRead,
&cachePct,
)
if err != nil {
return out, fmt.Errorf("pgmonitor: database stats: %w", err)
}
if cachePct != nil {
out.Database.CacheHitPct = *cachePct
}
_ = s.pool.QueryRow(ctx, `
SELECT checkpoints_timed, checkpoints_req, buffers_checkpoint, buffers_clean,
maxwritten_clean, buffers_backend, buffers_alloc
FROM pg_stat_bgwriter`).Scan(
&out.Bgwriter.CheckpointsTimed,
&out.Bgwriter.CheckpointsReq,
&out.Bgwriter.BuffersCheckpoint,
&out.Bgwriter.BuffersClean,
&out.Bgwriter.MaxWrittenClean,
&out.Bgwriter.BuffersBackend,
&out.Bgwriter.BuffersAlloc,
)
_ = s.pool.QueryRow(ctx, `SELECT pg_database_size(current_database())`).Scan(&out.SizeBytes)
_ = s.pool.QueryRow(ctx, `
SELECT
(SELECT setting FROM pg_settings WHERE name = 'shared_buffers'),
(SELECT setting FROM pg_settings WHERE name = 'work_mem'),
(SELECT setting FROM pg_settings WHERE name = 'effective_cache_size')`).Scan(
&out.MemorySettings.SharedBuffers,
&out.MemorySettings.WorkMem,
&out.MemorySettings.EffectiveCacheSize,
)
rows, err := s.pool.Query(ctx, `
SELECT client_addr::text, state, sync_state,
EXTRACT(EPOCH FROM COALESCE(write_lag, flush_lag, replay_lag)) * 1000
FROM pg_stat_replication`)
if err == nil {
defer rows.Close()
for rows.Next() {
var peer ReplicationPeer
var lagMs *float64
if err := rows.Scan(&peer.ClientAddr, &peer.State, &peer.SyncState, &lagMs); err != nil {
continue
}
if lagMs != nil {
v := int64(*lagMs)
peer.LagMs = &v
}
out.Replication = append(out.Replication, peer)
}
}
out.StatementsEnabled = s.statementsQueryable(ctx)
return out, nil
}
func queryLocks(ctx context.Context, pool *pgxpool.Pool) ([]LockRow, error) {
rows, err := pool.Query(ctx, `
SELECT l.locktype, l.mode, l.granted, a.pid, COALESCE(a.usename, ''),
COALESCE(a.state, ''), COALESCE(left(a.query, 300), ''),
NOT l.granted AS blocked
FROM pg_locks l
JOIN pg_stat_activity a ON a.pid = l.pid
WHERE a.datname = current_database()
AND (NOT l.granted OR l.mode LIKE '%Exclusive%')
ORDER BY l.granted ASC, a.query_start NULLS LAST
LIMIT 200`)
if err != nil {
return nil, fmt.Errorf("pgmonitor: locks: %w", err)
}
defer rows.Close()
var out []LockRow
for rows.Next() {
var r LockRow
if err := rows.Scan(&r.Locktype, &r.Mode, &r.Granted, &r.PID, &r.User, &r.State, &r.Query, &r.Blocked); err != nil {
return nil, err
}
out = append(out, r)
}
return out, rows.Err()
}
func queryTables(ctx context.Context, pool *pgxpool.Pool, limit int) ([]TableStat, error) {
limit = clampLimit(limit, 20, 100)
rows, err := pool.Query(ctx, `
SELECT t.relname,
pg_total_relation_size(t.relid),
s.heap_blks_read, s.heap_blks_hit,
t.idx_scan, t.seq_scan, t.n_dead_tup, t.last_autovacuum,
CASE WHEN t.n_live_tup + t.n_dead_tup > 0
THEN round(t.n_dead_tup::numeric / (t.n_live_tup + t.n_dead_tup), 4)
ELSE 0 END
FROM pg_statio_user_tables s
JOIN pg_stat_user_tables t ON t.relid = s.relid
WHERE t.schemaname = 'public'
ORDER BY pg_total_relation_size(t.relid) DESC
LIMIT $1`, limit)
if err != nil {
return nil, fmt.Errorf("pgmonitor: tables: %w", err)
}
defer rows.Close()
var out []TableStat
for rows.Next() {
var r TableStat
var last *time.Time
if err := rows.Scan(&r.Relname, &r.TotalBytes, &r.HeapBlksRead, &r.HeapBlksHit,
&r.IdxScan, &r.SeqScan, &r.DeadTuples, &last, &r.BloatRatio); err != nil {
return nil, err
}
r.LastAutovacuum = last
out = append(out, r)
}
return out, rows.Err()
}
// TopQueries loads from pg_stat_statements when available.
func (s *Service) TopQueries(ctx context.Context, limit int) (QueriesResponse, error) {
if s == nil || s.pool == nil {
return QueriesResponse{}, errors.New("pgmonitor: postgres not configured")
}
limit = clampLimit(limit, 20, 100)
now := time.Now().UTC()
if snap, ok, err := s.loadSnapshot(ctx, "slow_queries", 15*time.Minute); err == nil && ok {
var items []QueryStat
if err := decodePayload(snap.Payload, &items); err == nil {
return QueriesResponse{
CollectedAt: snap.CollectedAt,
Source: "snapshot",
Items: items,
StatementsAvailable: true,
}, nil
}
}
if !s.statementsQueryable(ctx) {
return queriesUnavailable(now), nil
}
items, err := queryTopStatements(ctx, s.pool, limit)
if err != nil {
if isPgStatStatementsUnavailable(err) {
s.markStatementsUnavailable()
return queriesUnavailable(now), nil
}
return QueriesResponse{}, err
}
return QueriesResponse{
CollectedAt: now,
Source: "live",
Items: items,
StatementsAvailable: true,
}, nil
}
func queriesUnavailable(at time.Time) QueriesResponse {
return QueriesResponse{
CollectedAt: at,
Source: "unavailable",
Items: nil,
StatementsAvailable: false,
StatementsHint: statementsUnavailableHint,
}
}
const statementsUnavailableHint = "pg_stat_statements requires shared_preload_libraries and PostgreSQL restart (see docs/db-diagnostics.md)"
// statementsQueryable returns true only when pg_stat_statements can be queried (not merely installed).
func (s *Service) statementsQueryable(ctx context.Context) bool {
if s == nil || s.pool == nil {
return false
}
if v, ok := s.cache.get("stmt_queryable"); ok {
if b, ok := v.(bool); ok {
return b
}
}
ok := probePgStatStatements(ctx, s.pool)
s.cache.set("stmt_queryable", ok)
return ok
}
func (s *Service) markStatementsUnavailable() {
s.cache.set("stmt_queryable", false)
}
func probePgStatStatements(ctx context.Context, pool *pgxpool.Pool) bool {
var dummy int64
err := pool.QueryRow(ctx, `
SELECT COALESCE(SUM(calls), 0)::bigint FROM pg_stat_statements LIMIT 1`).Scan(&dummy)
if err == nil {
return true
}
return !isPgStatStatementsUnavailable(err)
}
func queryTopStatements(ctx context.Context, pool *pgxpool.Pool, limit int) ([]QueryStat, error) {
rows, err := pool.Query(ctx, `
SELECT queryid, left(query, 500), calls, total_exec_time, mean_exec_time, rows
FROM pg_stat_statements
WHERE dbid = (SELECT oid FROM pg_database WHERE datname = current_database())
ORDER BY mean_exec_time DESC
LIMIT $1`, limit)
if err != nil {
if isPgStatStatementsUnavailable(err) {
return nil, nil
}
return nil, fmt.Errorf("pgmonitor: pg_stat_statements: %w", err)
}
defer rows.Close()
var out []QueryStat
for rows.Next() {
var r QueryStat
if err := rows.Scan(&r.QueryID, &r.Query, &r.Calls, &r.TotalExecMs, &r.MeanExecMs, &r.Rows); err != nil {
return nil, err
}
out = append(out, r)
}
return out, rows.Err()
}
// isPgStatStatementsUnavailable reports extension missing or not loaded via shared_preload_libraries.
func isPgStatStatementsUnavailable(err error) bool {
if err == nil {
return false
}
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) {
switch pgErr.Code {
case "42P01", "42704", "55000":
return true
}
msg := strings.ToLower(pgErr.Message)
if strings.Contains(msg, "shared_preload_libraries") || strings.Contains(msg, "pg_stat_statements") {
return true
}
}
low := strings.ToLower(err.Error())
return strings.Contains(low, "shared_preload_libraries") || strings.Contains(low, "pg_stat_statements")
}
func isSafeIdent(name string) bool {
if name == "" {
return true
}
for _, r := range name {
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
continue
}
return false
}
return true
}
// ExecMaintenance runs VACUUM/ANALYZE/REINDEX with optional dry-run (returns SQL executed or planned).
func ExecMaintenance(ctx context.Context, pool *pgxpool.Pool, kind, table string, dryRun bool) (detail map[string]any, err error) {
if pool == nil {
return nil, errors.New("pgmonitor: postgres not configured")
}
table = strings.TrimSpace(table)
if table != "" && !isSafeIdent(table) {
return nil, errors.New("pgmonitor: invalid table name")
}
qual := ""
if table != "" {
qual = " " + pgx.Identifier{table}.Sanitize()
}
var sql string
switch kind {
case "vacuum":
sql = "VACUUM" + qual
case "vacuum_analyze":
sql = "VACUUM ANALYZE" + qual
case "analyze":
sql = "ANALYZE" + qual
case "reindex":
if table == "" {
return nil, errors.New("pgmonitor: reindex requires table")
}
sql = "REINDEX TABLE" + qual
default:
return nil, fmt.Errorf("pgmonitor: unknown maintenance kind %q", kind)
}
detail = map[string]any{"sql": sql, "dry_run": dryRun}
if dryRun {
return detail, nil
}
_, err = pool.Exec(ctx, sql)
if err != nil {
return detail, fmt.Errorf("pgmonitor: %s: %w", kind, err)
}
detail["executed"] = true
return detail, nil
}
+104
View File
@@ -0,0 +1,104 @@
package pgmonitor
import (
"context"
"time"
)
// Recommendations builds heuristic items from live stats and snapshots.
func (s *Service) Recommendations(ctx context.Context) (RecommendationsResponse, error) {
now := time.Now().UTC()
var items []RecommendationItem
ov, err := s.Overview(ctx)
if err == nil {
if ov.Database.CacheHitPct > 0 && ov.Database.CacheHitPct < 90 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "low_cache_hit",
Title: "Низкий cache hit ratio",
Detail: "Buffer cache hit ниже 90%; проверьте shared_buffers и горячие seq scan.",
})
}
if ov.Database.Deadlocks > 0 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "deadlocks",
Title: "Зафиксированы deadlocks",
Detail: "Проверьте конкурирующие транзакции и порядок блокировок.",
})
}
if ov.Connections.MaxConnections > 0 &&
float64(ov.Connections.Total)/float64(ov.Connections.MaxConnections) > 0.8 {
items = append(items, RecommendationItem{
Severity: "critical",
Code: "connections_high",
Title: "Много подключений к PostgreSQL",
Detail: "Использование max_connections выше 80%; увеличьте pool tuning или лимит.",
})
}
}
tables, err := s.Tables(ctx, 30)
if err == nil {
for _, t := range tables {
if t.SeqScan > 1000 && t.IdxScan < t.SeqScan/10 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "missing_index",
Title: "Высокий seq_scan",
Detail: "Таблица часто сканируется последовательно; рассмотрите индекс.",
Refs: []string{t.Relname},
})
}
if t.BloatRatio > 0.2 && t.DeadTuples > 5000 {
items = append(items, RecommendationItem{
Severity: "info",
Code: "autovacuum_lag",
Title: "Возможный bloat / мёртвые строки",
Detail: "Высокая доля n_dead_tup; запланируйте VACUUM.",
Refs: []string{t.Relname},
})
}
}
}
if snap, ok, _ := s.loadSnapshot(ctx, "unused_indexes", 30*time.Minute); ok {
type unused struct {
Index string `json:"index"`
SizeBytes int64 `json:"size_bytes"`
}
var list []unused
if decodePayload(snap.Payload, &list) == nil {
for _, u := range list {
if u.SizeBytes < 1024*1024 {
continue
}
items = append(items, RecommendationItem{
Severity: "info",
Code: "unused_index",
Title: "Неиспользуемый индекс",
Detail: "idx_scan=0; проверьте перед удалением.",
Refs: []string{u.Index},
})
}
}
}
q, err := s.TopQueries(ctx, 5)
if err == nil {
for _, qs := range q.Items {
if qs.MeanExecMs > 500 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "slow_query",
Title: "Медленный запрос",
Detail: "Среднее время выполнения выше 500ms.",
Refs: []string{qs.Query},
})
}
}
}
return RecommendationsResponse{CollectedAt: now, Items: items}, nil
}
+59
View File
@@ -0,0 +1,59 @@
package pgmonitor
import (
"context"
"log"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// StartScheduler runs periodic PostgreSQL analyzer snapshots until ctx is cancelled.
func StartScheduler(ctx context.Context, pool *pgxpool.Pool) {
if pool == nil {
return
}
go func() {
t5 := time.NewTicker(5 * time.Minute)
t15 := time.NewTicker(15 * time.Minute)
defer t5.Stop()
defer t15.Stop()
s := NewService(pool)
runLight := func() {
c, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
if err := s.RefreshMetricsSnapshot(c); err != nil {
log.Printf("pgmonitor: metrics refresh: %v", err)
}
if err := s.DetectAutovacuumLag(c); err != nil {
log.Printf("pgmonitor: autovacuum lag: %v", err)
}
}
runHeavy := func() {
c, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
if err := s.AggregateSlowQueries(c, 30); err != nil {
log.Printf("pgmonitor: slow queries snapshot: %v", err)
}
if err := s.EstimateTableBloat(c); err != nil {
log.Printf("pgmonitor: bloat: %v", err)
}
if err := s.AnalyzeIndexUsage(c); err != nil {
log.Printf("pgmonitor: index usage: %v", err)
}
}
runLight()
runHeavy()
for {
select {
case <-ctx.Done():
return
case <-t5.C:
runLight()
case <-t15.C:
runHeavy()
}
}
}()
log.Printf("pgmonitor: scheduler started (5m light / 15m heavy)")
}
+90
View File
@@ -0,0 +1,90 @@
package pgmonitor
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// Service provides PostgreSQL observability and maintenance helpers (control plane instance scope).
type Service struct {
pool *pgxpool.Pool
cache *ttlCache
}
// NewService constructs a metrics service for the API PostgreSQL pool.
func NewService(pool *pgxpool.Pool) *Service {
if pool == nil {
return nil
}
return &Service{
pool: pool,
cache: newTTLCache(10 * time.Second),
}
}
// Pool exposes the underlying pool for job workers.
func (s *Service) Pool() *pgxpool.Pool {
if s == nil {
return nil
}
return s.pool
}
// Overview returns cached instance-level stats.
func (s *Service) Overview(ctx context.Context) (Overview, error) {
if s == nil || s.pool == nil {
return Overview{}, errors.New("pgmonitor: postgres not configured")
}
if v, ok := s.cache.get("overview"); ok {
if o, ok := v.(Overview); ok {
return o, nil
}
}
o, err := s.fetchOverview(ctx)
if err != nil {
return Overview{}, err
}
s.cache.set("overview", o)
return o, nil
}
// Locks returns active / blocking locks.
func (s *Service) Locks(ctx context.Context) ([]LockRow, error) {
if s == nil || s.pool == nil {
return nil, errors.New("pgmonitor: postgres not configured")
}
if v, ok := s.cache.get("locks"); ok {
if rows, ok := v.([]LockRow); ok {
return rows, nil
}
}
rows, err := queryLocks(ctx, s.pool)
if err != nil {
return nil, err
}
s.cache.set("locks", rows)
return rows, nil
}
// Tables returns top tables by size with I/O stats.
func (s *Service) Tables(ctx context.Context, limit int) ([]TableStat, error) {
if s == nil || s.pool == nil {
return nil, errors.New("pgmonitor: postgres not configured")
}
key := fmt.Sprintf("tables:%d", limit)
if v, ok := s.cache.get(key); ok {
if rows, ok := v.([]TableStat); ok {
return rows, nil
}
}
rows, err := queryTables(ctx, s.pool, limit)
if err != nil {
return nil, err
}
s.cache.set(key, rows)
return rows, nil
}
+48
View File
@@ -0,0 +1,48 @@
package pgmonitor
import (
"errors"
"testing"
"github.com/jackc/pgx/v5/pgconn"
)
func TestClampLimit(t *testing.T) {
if clampLimit(0, 20, 100) != 20 {
t.Fatal("default")
}
if clampLimit(200, 20, 100) != 100 {
t.Fatal("max")
}
if clampLimit(5, 20, 100) != 5 {
t.Fatal("value")
}
}
func TestIsSafeIdent(t *testing.T) {
if !isSafeIdent("revision_materialized_prefix") {
t.Fatal("valid")
}
if isSafeIdent("bad-name") {
t.Fatal("invalid")
}
if !isSafeIdent("") {
t.Fatal("empty ok")
}
}
func TestNewServiceNilPool(t *testing.T) {
if NewService(nil) != nil {
t.Fatal("expected nil service")
}
}
func TestIsPgStatStatementsUnavailable(t *testing.T) {
err := &pgconn.PgError{Code: "55000", Message: "pg_stat_statements must be loaded via shared_preload_libraries"}
if !isPgStatStatementsUnavailable(err) {
t.Fatal("55000")
}
if isPgStatStatementsUnavailable(errors.New("other")) {
t.Fatal("unrelated")
}
}
+162
View File
@@ -0,0 +1,162 @@
package pgmonitor
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
type snapshotRow struct {
ID string
CollectedAt time.Time
Payload json.RawMessage
}
func (s *Service) loadSnapshot(ctx context.Context, id string, maxAge time.Duration) (snapshotRow, bool, error) {
var row snapshotRow
err := s.pool.QueryRow(ctx, `
SELECT id, collected_at, payload_json
FROM postgres_monitor_snapshot
WHERE id = $1 AND collected_at >= $2`,
id, time.Now().UTC().Add(-maxAge)).Scan(&row.ID, &row.CollectedAt, &row.Payload)
if err != nil {
return snapshotRow{}, false, nil
}
return row, true, nil
}
func (s *Service) UpsertSnapshot(ctx context.Context, id string, payload any) error {
if s == nil || s.pool == nil {
return fmt.Errorf("pgmonitor: postgres not configured")
}
b, err := json.Marshal(payload)
if err != nil {
return err
}
_, err = s.pool.Exec(ctx, `
INSERT INTO postgres_monitor_snapshot (id, collected_at, payload_json)
VALUES ($1, now(), $2::jsonb)
ON CONFLICT (id) DO UPDATE SET collected_at = EXCLUDED.collected_at, payload_json = EXCLUDED.payload_json`,
id, string(b))
return err
}
func decodePayload(raw json.RawMessage, dest any) error {
return json.Unmarshal(raw, dest)
}
// RefreshMetricsSnapshot stores overview and tables for heavy reads.
func (s *Service) RefreshMetricsSnapshot(ctx context.Context) error {
ov, err := s.fetchOverview(ctx)
if err != nil {
return err
}
if err := s.UpsertSnapshot(ctx, "overview", ov); err != nil {
return err
}
tables, err := queryTables(ctx, s.pool, 50)
if err != nil {
return err
}
return s.UpsertSnapshot(ctx, "tables", tables)
}
// AggregateSlowQueries stores top statements snapshot.
func (s *Service) AggregateSlowQueries(ctx context.Context, limit int) error {
if !s.statementsQueryable(ctx) {
return s.UpsertSnapshot(ctx, "slow_queries", []QueryStat{})
}
items, err := queryTopStatements(ctx, s.pool, clampLimit(limit, 20, 100))
if err != nil {
if isPgStatStatementsUnavailable(err) {
s.markStatementsUnavailable()
return s.UpsertSnapshot(ctx, "slow_queries", []QueryStat{})
}
return err
}
return s.UpsertSnapshot(ctx, "slow_queries", items)
}
// EstimateTableBloat refreshes bloat heuristics on tables snapshot.
func (s *Service) EstimateTableBloat(ctx context.Context) error {
tables, err := queryTables(ctx, s.pool, 100)
if err != nil {
return err
}
return s.UpsertSnapshot(ctx, "table_bloat", tables)
}
// AnalyzeIndexUsage stores unused indexes.
func (s *Service) AnalyzeIndexUsage(ctx context.Context) error {
rows, err := s.pool.Query(ctx, `
SELECT indexrelname, idx_scan, pg_relation_size(indexrelid)
FROM pg_stat_user_indexes
WHERE schemaname = 'public' AND idx_scan = 0
ORDER BY pg_relation_size(indexrelid) DESC
LIMIT 50`)
if err != nil {
return fmt.Errorf("pgmonitor: index usage: %w", err)
}
defer rows.Close()
type unused struct {
Index string `json:"index"`
IdxScan int64 `json:"idx_scan"`
SizeBytes int64 `json:"size_bytes"`
}
var items []unused
for rows.Next() {
var u unused
if err := rows.Scan(&u.Index, &u.IdxScan, &u.SizeBytes); err != nil {
return err
}
items = append(items, u)
}
return s.UpsertSnapshot(ctx, "unused_indexes", items)
}
// DetectAutovacuumLag stores tables with high dead tuple ratio.
func (s *Service) DetectAutovacuumLag(ctx context.Context) error {
rows, err := s.pool.Query(ctx, `
SELECT relname, n_dead_tup, last_autovacuum,
CASE WHEN n_live_tup + n_dead_tup > 0
THEN round(n_dead_tup::numeric / (n_live_tup + n_dead_tup), 4) ELSE 0 END
FROM pg_stat_user_tables
WHERE schemaname = 'public' AND n_dead_tup > 1000
ORDER BY n_dead_tup DESC
LIMIT 30`)
if err != nil {
return fmt.Errorf("pgmonitor: autovacuum lag: %w", err)
}
defer rows.Close()
type lagRow struct {
Relname string `json:"relname"`
DeadTuples int64 `json:"n_dead_tup"`
LastAutovacuum *time.Time `json:"last_autovacuum,omitempty"`
Ratio float64 `json:"ratio"`
}
var items []lagRow
for rows.Next() {
var r lagRow
if err := rows.Scan(&r.Relname, &r.DeadTuples, &r.LastAutovacuum, &r.Ratio); err != nil {
return err
}
items = append(items, r)
}
return s.UpsertSnapshot(ctx, "autovacuum_lag", items)
}
// RunPeriodicAnalyzerJobs runs all snapshot analyzers (for scheduler).
func RunPeriodicAnalyzerJobs(ctx context.Context, pool *pgxpool.Pool) {
s := NewService(pool)
if s == nil {
return
}
_ = s.RefreshMetricsSnapshot(ctx)
_ = s.AggregateSlowQueries(ctx, 30)
_ = s.EstimateTableBloat(ctx)
_ = s.AnalyzeIndexUsage(ctx)
_ = s.DetectAutovacuumLag(ctx)
}
+150
View File
@@ -0,0 +1,150 @@
package pgmonitor
import "time"
// Overview is instance-level PostgreSQL health snapshot.
type Overview struct {
CollectedAt time.Time `json:"collected_at"`
Connections Connections `json:"connections"`
Database DatabaseStats `json:"database"`
Bgwriter BgwriterStats `json:"bgwriter"`
SizeBytes int64 `json:"database_size_bytes"`
MemorySettings MemorySettings `json:"memory_settings"`
Replication []ReplicationPeer `json:"replication"`
StatementsEnabled bool `json:"pg_stat_statements_enabled"`
}
// Connections summarizes pg_stat_activity for current database.
type Connections struct {
Active int `json:"active"`
Idle int `json:"idle"`
Total int `json:"total"`
MaxConnections int `json:"max_connections"`
}
// DatabaseStats from pg_stat_database.
type DatabaseStats struct {
Backends int `json:"backends"`
XactCommit int64 `json:"xact_commit"`
XactRollback int64 `json:"xact_rollback"`
Deadlocks int64 `json:"deadlocks"`
BlksHit int64 `json:"blks_hit"`
BlksRead int64 `json:"blks_read"`
CacheHitPct float64 `json:"cache_hit_pct"`
}
// BgwriterStats from pg_stat_bgwriter.
type BgwriterStats struct {
CheckpointsTimed int64 `json:"checkpoints_timed"`
CheckpointsReq int64 `json:"checkpoints_req"`
BuffersCheckpoint int64 `json:"buffers_checkpoint"`
BuffersClean int64 `json:"buffers_clean"`
MaxWrittenClean int64 `json:"maxwritten_clean"`
BuffersBackend int64 `json:"buffers_backend"`
BuffersAlloc int64 `json:"buffers_alloc"`
}
// MemorySettings is best-effort from pg_settings (not RSS).
type MemorySettings struct {
SharedBuffers string `json:"shared_buffers"`
WorkMem string `json:"work_mem"`
EffectiveCacheSize string `json:"effective_cache_size"`
}
// ReplicationPeer from pg_stat_replication.
type ReplicationPeer struct {
ClientAddr string `json:"client_addr,omitempty"`
State string `json:"state"`
SyncState string `json:"sync_state,omitempty"`
LagMs *int64 `json:"lag_ms,omitempty"`
}
// QueryStat is a row from pg_stat_statements or snapshot.
type QueryStat struct {
QueryID int64 `json:"queryid,omitempty"`
Query string `json:"query"`
Calls int64 `json:"calls"`
TotalExecMs float64 `json:"total_exec_ms"`
MeanExecMs float64 `json:"mean_exec_ms"`
Rows int64 `json:"rows"`
}
// QueriesResponse for GET /monitoring/postgres/queries.
type QueriesResponse struct {
CollectedAt time.Time `json:"collected_at"`
Source string `json:"source"` // live | snapshot | unavailable
Items []QueryStat `json:"items"`
StatementsAvailable bool `json:"statements_available"`
StatementsHint string `json:"statements_hint,omitempty"`
}
// LockRow describes a lock / blocking session.
type LockRow struct {
Locktype string `json:"locktype"`
Mode string `json:"mode"`
Granted bool `json:"granted"`
PID int32 `json:"pid"`
User string `json:"usename,omitempty"`
State string `json:"state,omitempty"`
Query string `json:"query,omitempty"`
Blocked bool `json:"blocked"`
}
// TableStat combines size and scan stats for a user table.
type TableStat struct {
Relname string `json:"relname"`
TotalBytes int64 `json:"total_bytes"`
HeapBlksRead int64 `json:"heap_blks_read"`
HeapBlksHit int64 `json:"heap_blks_hit"`
IdxScan int64 `json:"idx_scan"`
SeqScan int64 `json:"seq_scan"`
DeadTuples int64 `json:"n_dead_tup"`
LastAutovacuum *time.Time `json:"last_autovacuum,omitempty"`
BloatRatio float64 `json:"bloat_ratio,omitempty"`
}
// RecommendationItem is a heuristic ops hint.
type RecommendationItem struct {
Severity string `json:"severity"` // info | warn | critical
Code string `json:"code"`
Title string `json:"title"`
Detail string `json:"detail"`
Refs []string `json:"refs,omitempty"`
}
// RecommendationsResponse for GET recommendations.
type RecommendationsResponse struct {
CollectedAt time.Time `json:"collected_at"`
Items []RecommendationItem `json:"items"`
}
// CorrelationPoint is one aligned sample for overlay charts.
type CorrelationPoint struct {
Timestamp time.Time `json:"timestamp"`
PipelineRefreshP99Ms float64 `json:"pipeline_refresh_p99_ms,omitempty"`
BirdScrapeOK *float64 `json:"bird_scrape_ok,omitempty"`
HTTPRequestRate float64 `json:"http_request_rate,omitempty"`
CacheHitPct float64 `json:"cache_hit_pct,omitempty"`
}
// CorrelationResponse for GET /monitoring/correlation.
type CorrelationResponse struct {
WindowMinutes int `json:"window_minutes"`
Points []CorrelationPoint `json:"points"`
}
// MaintenanceLogRow is an audit entry.
type MaintenanceLogRow struct {
ID string `json:"id"`
TenantID string `json:"tenant_id,omitempty"`
ActorPrefix string `json:"actor_prefix,omitempty"`
Kind string `json:"kind"`
TargetTable string `json:"target_table,omitempty"`
DryRun bool `json:"dry_run"`
Status string `json:"status"`
Detail map[string]any `json:"detail,omitempty"`
Error string `json:"error,omitempty"`
CreatedAt time.Time `json:"created_at"`
StartedAt *time.Time `json:"started_at,omitempty"`
FinishedAt *time.Time `json:"finished_at,omitempty"`
}
+8 -4
View File
@@ -11,6 +11,7 @@ import (
)
func TestCollectModulePrefixRows_CDNSendsIfNoneMatch(t *testing.T) {
t.Setenv("EVOBGP_CDN_ALLOW_PRIVATE", "1")
m := store.NewMemory()
m.SeedDemo()
tenant, _, _, _, _ := m.DemoIDs()
@@ -25,7 +26,7 @@ func TestCollectModulePrefixRows_CDNSendsIfNoneMatch(t *testing.T) {
}
var gotIfNoneMatch string
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
gotIfNoneMatch = strings.TrimSpace(r.Header.Get("If-None-Match"))
w.Header().Set("ETag", "etag-new")
_, _ = w.Write([]byte("198.51.100.0/24\n"))
@@ -54,6 +55,7 @@ func TestCollectModulePrefixRows_CDNSendsIfNoneMatch(t *testing.T) {
}
func TestCollectModulePrefixRows_CDN304UsesSnapshot(t *testing.T) {
t.Setenv("EVOBGP_CDN_ALLOW_PRIVATE", "1")
m := store.NewMemory()
m.SeedDemo()
tenant, _, _, _, _ := m.DemoIDs()
@@ -67,7 +69,7 @@ func TestCollectModulePrefixRows_CDN304UsesSnapshot(t *testing.T) {
t.Fatal(err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotModified)
}))
defer srv.Close()
@@ -99,6 +101,7 @@ func TestCollectModulePrefixRows_CDN304UsesSnapshot(t *testing.T) {
}
func TestRefreshModuleIngest_CDN304UsesStoredSnapshot(t *testing.T) {
t.Setenv("EVOBGP_CDN_ALLOW_PRIVATE", "1")
m := store.NewMemory()
m.SeedDemo()
tenant, _, _, _, _ := m.DemoIDs()
@@ -112,7 +115,7 @@ func TestRefreshModuleIngest_CDN304UsesStoredSnapshot(t *testing.T) {
t.Fatal(err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotModified)
}))
defer srv.Close()
@@ -140,6 +143,7 @@ func TestRefreshModuleIngest_CDN304UsesStoredSnapshot(t *testing.T) {
}
func TestCollectModulePrefixRows_CDN304RetriesWithoutETag(t *testing.T) {
t.Setenv("EVOBGP_CDN_ALLOW_PRIVATE", "1")
m := store.NewMemory()
m.SeedDemo()
tenant, _, _, _, _ := m.DemoIDs()
@@ -154,7 +158,7 @@ func TestCollectModulePrefixRows_CDN304RetriesWithoutETag(t *testing.T) {
}
var calls int
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
if calls == 1 {
if got := strings.TrimSpace(r.Header.Get("If-None-Match")); got != "etag-stable" {
+16 -4
View File
@@ -119,6 +119,12 @@ func applyCDNSourceHTTPResult(ctx context.Context, st store.Backend, hc *http.Cl
if u == "" {
return nil, nil
}
if _, err := ValidateCDNURL(u); err != nil {
return nil, err
}
if err := ResolveCDNURLHost(ctx, u); err != nil {
return nil, err
}
sourceKey := cdnSourceKey(src.ID)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
@@ -127,7 +133,7 @@ func applyCDNSourceHTTPResult(ctx context.Context, st store.Backend, hc *http.Cl
if etag := strings.TrimSpace(src.Etag); etag != "" {
req.Header.Set("If-None-Match", etag)
}
resp, err := hc.Do(req)
resp, err := upstreamHTTPDo(ctx, hc, req)
if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
}
@@ -143,7 +149,7 @@ func applyCDNSourceHTTPResult(ctx context.Context, st store.Backend, hc *http.Cl
if err != nil {
return nil, err
}
resp, err = hc.Do(req2)
resp, err = upstreamHTTPDo(ctx, hc, req2)
if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
}
@@ -189,6 +195,12 @@ func fetchCDNSourceRows(ctx context.Context, st store.Backend, hc *http.Client,
if u == "" {
return nil, nil
}
if _, err := ValidateCDNURL(u); err != nil {
return nil, err
}
if err := ResolveCDNURLHost(ctx, u); err != nil {
return nil, err
}
sourceKey := cdnSourceKey(src.ID)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
@@ -197,7 +209,7 @@ func fetchCDNSourceRows(ctx context.Context, st store.Backend, hc *http.Client,
if etag := strings.TrimSpace(src.Etag); etag != "" {
req.Header.Set("If-None-Match", etag)
}
resp, err := hc.Do(req)
resp, err := upstreamHTTPDo(ctx, hc, req)
if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
}
@@ -212,7 +224,7 @@ func fetchCDNSourceRows(ctx context.Context, st store.Backend, hc *http.Client,
if err != nil {
return nil, err
}
resp, err = hc.Do(req2)
resp, err = upstreamHTTPDo(ctx, hc, req2)
if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
}
+116
View File
@@ -0,0 +1,116 @@
package pipeline
import (
"context"
"fmt"
"net"
"net/netip"
"net/url"
"os"
"strings"
"time"
)
func isBlockedCDNIP(ip netip.Addr) bool {
if allowPrivateCDNURLs() {
return false
}
if !ip.IsValid() {
return true
}
return ip.IsLoopback() || ip.IsPrivate() || ip.IsLinkLocalUnicast() || ip.IsMulticast() ||
ip.IsUnspecified() || ip == netip.MustParseAddr("169.254.169.254")
}
func allowPrivateCDNURLs() bool {
v := strings.TrimSpace(os.Getenv("EVOBGP_CDN_ALLOW_PRIVATE"))
return v == "1" || strings.EqualFold(v, "true")
}
func isBlockedCDNHostname(host string) bool {
if allowPrivateCDNURLs() {
return false
}
h := strings.ToLower(strings.TrimSpace(host))
if h == "" || h == "localhost" {
return true
}
if strings.HasSuffix(h, ".local") || strings.HasSuffix(h, ".internal") || strings.HasSuffix(h, ".localhost") {
return true
}
return false
}
// ValidateCDNURL checks CDN source URLs for SSRF-safe HTTPS endpoints (hostname only; no DNS resolve).
func ValidateCDNURL(raw string) (string, error) {
raw = strings.TrimSpace(raw)
if raw == "" {
return "", fmt.Errorf("pipeline: cdn url is required")
}
u, err := url.Parse(raw)
if err != nil {
return "", fmt.Errorf("pipeline: cdn url invalid: %w", err)
}
if u.Scheme != "https" {
return "", fmt.Errorf("pipeline: cdn url must use https")
}
if u.User != nil {
return "", fmt.Errorf("pipeline: cdn url must not include credentials")
}
host := strings.TrimSpace(u.Hostname())
if host == "" {
return "", fmt.Errorf("pipeline: cdn url missing host")
}
if isBlockedCDNHostname(host) {
return "", fmt.Errorf("pipeline: cdn url blocked host")
}
if ip, err := netip.ParseAddr(host); err == nil {
if isBlockedCDNIP(ip) {
return "", fmt.Errorf("pipeline: cdn url blocked host")
}
}
return u.String(), nil
}
// ResolveCDNURLHost resolves a CDN hostname and rejects private/link-local targets (SSRF at fetch time).
func ResolveCDNURLHost(ctx context.Context, raw string) error {
u, err := url.Parse(strings.TrimSpace(raw))
if err != nil {
return err
}
host := strings.TrimSpace(u.Hostname())
if host == "" {
return fmt.Errorf("pipeline: cdn url missing host")
}
if ip, err := netip.ParseAddr(host); err == nil {
if isBlockedCDNIP(ip) {
return fmt.Errorf("pipeline: cdn url blocked host")
}
return nil
}
if isBlockedCDNHostname(host) {
return fmt.Errorf("pipeline: cdn url blocked host")
}
if ctx == nil {
ctx = context.Background()
}
resolveCtx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()
ips, err := net.DefaultResolver.LookupIP(resolveCtx, "ip", host)
if err != nil {
return fmt.Errorf("pipeline: cdn url dns lookup: %w", err)
}
if len(ips) == 0 {
return fmt.Errorf("pipeline: cdn url dns lookup: no addresses")
}
for _, ip := range ips {
addr, ok := netip.AddrFromSlice(ip)
if !ok {
continue
}
if isBlockedCDNIP(addr) {
return fmt.Errorf("pipeline: cdn url resolves to blocked address")
}
}
return nil
}
+34
View File
@@ -0,0 +1,34 @@
package pipeline
import "testing"
func TestValidateCDNURL(t *testing.T) {
tests := []struct {
raw string
ok bool
want string
}{
{"https://cdn.example.com/prefixes.txt", true, "https://cdn.example.com/prefixes.txt"},
{"http://cdn.example.com/x", false, ""},
{"https://127.0.0.1/x", false, ""},
{"https://10.0.0.1/x", false, ""},
{"https://169.254.169.254/latest/meta-data", false, ""},
{"https://localhost/x", false, ""},
{"file:///etc/passwd", false, ""},
{"https://user:pass@cdn.example.com/x", false, ""},
}
for _, tc := range tests {
got, err := ValidateCDNURL(tc.raw)
if tc.ok && err != nil {
t.Errorf("%q: unexpected err %v", tc.raw, err)
continue
}
if !tc.ok && err == nil {
t.Errorf("%q: expected error", tc.raw)
continue
}
if tc.ok && got != tc.want {
t.Errorf("%q: got %q want %q", tc.raw, got, tc.want)
}
}
}
+57 -2
View File
@@ -25,7 +25,7 @@ func prefixRowsForSource(rows []store.PrefixRow, sourceKey string) []store.Prefi
return out
}
func collectASPrefixRows(ctx context.Context, st store.Backend, hc *http.Client, tenantID string, mod *store.Module, list []*store.ASEntry) ([]store.PrefixRow, error) {
func collectASPrefixRows(ctx context.Context, st store.Backend, hc *http.Client, tenantID string, mod *store.Module, list []*store.ASEntry, priorSnapshot []store.PrefixRow) ([]store.PrefixRow, error) {
moduleID := mod.ID
legacy := strings.TrimSpace(os.Getenv("EVOBGP_ASN_RESOLVE")) == "0"
if legacy {
@@ -76,6 +76,41 @@ func collectASPrefixRows(ctx context.Context, st store.Backend, hc *http.Client,
}
pfxs, holder, err := resolveASNForEntry(ctx, st, hc, entry.ASN)
if err != nil {
if staleOnUpstreamError() {
if staleRows, staleHolder, ok := staleASNPrefixes(st, priorSnapshot, entry.ASN); ok {
logStaleUpstream("asn", fmt.Sprintf("AS%d: %v", entry.ASN, err))
src := fmt.Sprintf("as:%d", entry.ASN)
rows := append([]store.PrefixRow(nil), staleRows...)
for i := range rows {
rows[i].CommunityID = comm
rows[i].Source = src
}
results[idx] = entryResult{
rows: rows,
metaID: entry.ID,
asn: entry.ASN,
holder: staleHolder,
count: int64(len(rows)),
}
return
}
if pfxs2, holder2, ok := asnCacheExpired(st, entry.ASN); ok {
logStaleUpstream("asn", fmt.Sprintf("AS%d expired cache: %v", entry.ASN, err))
src := fmt.Sprintf("as:%d", entry.ASN)
var rows []store.PrefixRow
for _, pfx := range pfxs2 {
rows = append(rows, store.PrefixRow{Prefix: pfx.String(), CommunityID: comm, Source: src})
}
results[idx] = entryResult{
rows: rows,
metaID: entry.ID,
asn: entry.ASN,
holder: holder2,
count: int64(len(pfxs2)),
}
return
}
}
results[idx] = entryResult{err: fmt.Errorf("resolve AS%d: %w", entry.ASN, err)}
return
}
@@ -167,6 +202,13 @@ func collectCDNPrefixRows(ctx context.Context, st store.Backend, hc *http.Client
}
rows, err := fetchCDNSourceRows(ctx, st, hc, tenantID, moduleID, mod, src, priorSnapshot, now)
if err != nil {
if staleOnUpstreamError() {
if cached, ok := staleCDNPrefixes(st, tenantID, moduleID, priorSnapshot, src.ID); ok {
logStaleUpstream("cdn", fmt.Sprintf("source %s: %v", src.ID, err))
results[idx] = srcResult{rows: cached}
return
}
}
results[idx] = srcResult{err: err}
return
}
@@ -190,7 +232,7 @@ func collectCDNPrefixRows(ctx context.Context, st store.Backend, hc *http.Client
return out, nil
}
func collectDomainPrefixRows(ctx context.Context, hc *http.Client, mod *store.Module, profiles []*store.DohProfile, policy string, entries []*store.DomainEntry) ([]store.PrefixRow, error) {
func collectDomainPrefixRows(ctx context.Context, hc *http.Client, mod *store.Module, profiles []*store.DohProfile, policy string, entries []*store.DomainEntry, priorSnapshot []store.PrefixRow) ([]store.PrefixRow, error) {
var validDom []*store.DomainEntry
for _, e := range entries {
if e != nil {
@@ -219,6 +261,19 @@ func collectDomainPrefixRows(ctx context.Context, hc *http.Client, mod *store.Mo
}
addrs, err := resolveDomainIPsWithPolicy(ctx, hc, profiles, policy, entry.FQDN)
if err != nil {
if staleOnUpstreamError() {
if cached, ok := staleDomainPrefixes(priorSnapshot, entry.FQDN); ok {
logStaleUpstream("domain", fmt.Sprintf("%q: %v", entry.FQDN, err))
rows := append([]store.PrefixRow(nil), cached...)
for i := range rows {
if rows[i].CommunityID == nil {
rows[i].CommunityID = comm
}
}
results[idx] = domResult{rows: rows}
return
}
}
results[idx] = domResult{err: fmt.Errorf("resolve domain %q: %w", entry.FQDN, err)}
return
}
+86
View File
@@ -0,0 +1,86 @@
package pipeline
import (
"fmt"
"log"
"net/netip"
"os"
"strings"
"evobgp/internal/store"
)
// staleOnUpstreamError reports whether ingest should keep last-known prefixes when an upstream fetch fails.
// Enabled by default; set EVOBGP_STALE_ON_UPSTREAM_ERROR=0 to restore fail-fast behavior.
func staleOnUpstreamError() bool {
v := strings.TrimSpace(os.Getenv("EVOBGP_STALE_ON_UPSTREAM_ERROR"))
if v == "" || v == "1" || strings.EqualFold(v, "true") {
return true
}
return false
}
func logStaleUpstream(kind, detail string) {
log.Printf("pipeline: stale upstream fallback (%s): %s", kind, detail)
}
func staleASNPrefixes(st store.Backend, priorSnapshot []store.PrefixRow, asn int64) ([]store.PrefixRow, string, bool) {
sourceKey := fmt.Sprintf("as:%d", asn)
if cached := prefixRowsForSource(priorSnapshot, sourceKey); len(cached) > 0 {
return cached, "", true
}
if st == nil {
return nil, "", false
}
ent, ok, err := st.GetASNPrefixCache(asn)
if err != nil || !ok || ent == nil || len(ent.Prefixes) == 0 {
return nil, "", false
}
var rows []store.PrefixRow
for _, p := range ent.Prefixes {
pfx, perr := netip.ParsePrefix(strings.TrimSpace(p))
if perr != nil {
continue
}
rows = append(rows, store.PrefixRow{Prefix: pfx.Masked().String(), Source: sourceKey})
}
if len(rows) == 0 {
return nil, "", false
}
return rows, ent.Holder, true
}
func staleDomainPrefixes(priorSnapshot []store.PrefixRow, fqdn string) ([]store.PrefixRow, bool) {
sourceKey := "domain:" + strings.TrimSpace(fqdn)
cached := prefixRowsForSource(priorSnapshot, sourceKey)
return cached, len(cached) > 0
}
func staleCDNPrefixes(st store.Backend, tenantID, moduleID string, priorSnapshot []store.PrefixRow, sourceID string) ([]store.PrefixRow, bool) {
sourceKey := cdnSourceKey(sourceID)
cached := cachedCDNPrefixRows(st, tenantID, moduleID, priorSnapshot, sourceKey)
return cached, len(cached) > 0
}
// asnCacheExpired returns cached ASN prefixes even past TTL (for stale fallback only).
func asnCacheExpired(st store.Backend, asn int64) ([]netip.Prefix, string, bool) {
if st == nil {
return nil, "", false
}
ent, ok, err := st.GetASNPrefixCache(asn)
if err != nil || !ok || ent == nil || len(ent.Prefixes) == 0 {
return nil, "", false
}
out := make([]netip.Prefix, 0, len(ent.Prefixes))
for _, p := range ent.Prefixes {
pfx, perr := netip.ParsePrefix(strings.TrimSpace(p))
if perr != nil {
continue
}
out = append(out, pfx.Masked())
}
if len(out) == 0 {
return nil, "", false
}
return out, ent.Holder, true
}
+72
View File
@@ -0,0 +1,72 @@
package pipeline
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"evobgp/internal/store"
)
func TestCollectCDNPrefixRows_StaleOnFetchError(t *testing.T) {
t.Setenv("EVOBGP_STALE_ON_UPSTREAM_ERROR", "1")
t.Setenv("EVOBGP_CDN_ALLOW_PRIVATE", "1")
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "upstream down", http.StatusServiceUnavailable)
}))
defer srv.Close()
st := store.NewMemory()
st.SeedDemo()
tenant, _, _, _, _ := st.DemoIDs()
mod, err := st.CreateModule(tenant, &store.Module{Type: "CDN_CIDRS", Name: "cdn", Enabled: true})
if err != nil {
t.Fatal(err)
}
if _, err := st.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
ID: "s1", URL: srv.URL, SourceKind: "plain",
}); err != nil {
t.Fatal(err)
}
prior := []store.PrefixRow{
{Prefix: "203.0.113.0/24", Source: "cdn:s1"},
}
rows, err := collectCDNPrefixRows(context.Background(), st, srv.Client(), tenant, mod, []*store.CDNSource{{ID: "s1", URL: srv.URL, SourceKind: "plain"}}, prior)
if err != nil {
t.Fatalf("expected stale fallback, got err: %v", err)
}
if len(rows) != 1 || rows[0].Prefix != "203.0.113.0/24" {
t.Fatalf("unexpected rows: %+v", rows)
}
}
func TestCollectCDNPrefixRows_FailFastWhenNoStale(t *testing.T) {
t.Setenv("EVOBGP_STALE_ON_UPSTREAM_ERROR", "0")
t.Setenv("EVOBGP_CDN_ALLOW_PRIVATE", "1")
srv := httptest.NewTLSServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
http.Error(w, "upstream down", http.StatusServiceUnavailable)
}))
defer srv.Close()
st := store.NewMemory()
st.SeedDemo()
tenant, _, _, _, _ := st.DemoIDs()
mod, err := st.CreateModule(tenant, &store.Module{Type: "CDN_CIDRS", Name: "cdn", Enabled: true})
if err != nil {
t.Fatal(err)
}
if _, err := st.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
ID: "s1", URL: srv.URL, SourceKind: "plain",
}); err != nil {
t.Fatal(err)
}
_, err = collectCDNPrefixRows(context.Background(), st, srv.Client(), tenant, mod, []*store.CDNSource{{ID: "s1", URL: srv.URL, SourceKind: "plain"}}, nil)
if err == nil {
t.Fatal("expected error when stale disabled and no cache")
}
}
+84 -54
View File
@@ -5,86 +5,116 @@ import (
"io"
"net/http"
"strings"
"sync"
"time"
"evobgp/internal/httpclient"
"evobgp/internal/store"
)
type prefetchTask struct {
tenantID string
mod *store.Module
src *store.CDNSource
}
// PrefetchCDNSourceETags performs conditional GETs for CDN sources; on 200 parses CIDRs into module_prefix_snapshot.
func PrefetchCDNSourceETags(ctx context.Context, st store.Backend, hc *http.Client) error {
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
if ctx == nil {
ctx = context.Background()
}
tenants, err := st.ListTenantIDs()
if err != nil {
return err
}
now := time.Now().UTC()
var tasks []prefetchTask
for _, tid := range tenants {
for _, mod := range st.ListModules(tid) {
if mod == nil || !mod.Enabled || mod.Type != "CDN_CIDRS" {
continue
}
omod, err := st.GetModule(tid, mod.ID)
if err != nil {
continue
}
sources, err := st.ListCDNSources(tid, mod.ID)
if err != nil {
continue
}
var prior []store.PrefixRow
if snap, ok, _ := st.GetModulePrefixSnapshot(tid, mod.ID); ok && snap != nil {
prior = snap.Prefixes
}
for _, src := range sources {
if src == nil || strings.TrimSpace(src.URL) == "" {
continue
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, strings.TrimSpace(src.URL), nil)
if err != nil {
continue
}
if etag := strings.TrimSpace(src.Etag); etag != "" {
req.Header.Set("If-None-Match", etag)
}
resp, err := hc.Do(req)
if err != nil {
continue
}
if resp.StatusCode == http.StatusNotModified {
_ = resp.Body.Close()
continue
}
if resp.StatusCode != http.StatusOK {
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
continue
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
_ = resp.Body.Close()
if err != nil {
continue
}
prefixStrs, err := parseCDNBody(string(body), src)
if err != nil {
continue
}
newEtag := strings.TrimSpace(resp.Header.Get("ETag"))
patch := &store.CDNSourcePatch{LastRefreshedAt: &now}
if newEtag != "" && newEtag != strings.TrimSpace(src.Etag) {
e := newEtag
patch.Etag = &e
}
_, _ = st.UpdateCDNSource(tid, mod.ID, src.ID, patch)
rows := cdnRowsFromParsed(omod, src, prefixStrs)
_ = mergeCDNSourceIntoModuleSnapshot(st, tid, omod, src.ID, rows)
_ = prior // prior may be stale after merge; refresh for next source in loop
if snap, ok, _ := st.GetModulePrefixSnapshot(tid, mod.ID); ok && snap != nil {
prior = snap.Prefixes
if src != nil && strings.TrimSpace(src.URL) != "" {
tasks = append(tasks, prefetchTask{tenantID: tid, mod: mod, src: src})
}
}
}
}
if len(tasks) == 0 {
return nil
}
sem := make(chan struct{}, collectConcurrency())
var wg sync.WaitGroup
for _, task := range tasks {
wg.Add(1)
go func(t prefetchTask) {
defer wg.Done()
sem <- struct{}{}
defer func() { <-sem }()
prefetchOneCDNSource(ctx, st, hc, t)
}(task)
}
wg.Wait()
return nil
}
func prefetchOneCDNSource(ctx context.Context, st store.Backend, hc *http.Client, t prefetchTask) {
now := time.Now().UTC()
tid, mod, src := t.tenantID, t.mod, t.src
u := strings.TrimSpace(src.URL)
if _, err := ValidateCDNURL(u); err != nil {
return
}
if err := ResolveCDNURLHost(ctx, u); err != nil {
return
}
omod, err := st.GetModule(tid, mod.ID)
if err != nil {
return
}
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
return
}
if etag := strings.TrimSpace(src.Etag); etag != "" {
req.Header.Set("If-None-Match", etag)
}
resp, err := upstreamHTTPDo(ctx, hc, req)
if err != nil {
return
}
if resp.StatusCode == http.StatusNotModified {
_ = resp.Body.Close()
return
}
if resp.StatusCode != http.StatusOK {
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
return
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
_ = resp.Body.Close()
if err != nil {
return
}
prefixStrs, err := parseCDNBody(string(body), src)
if err != nil {
return
}
newEtag := strings.TrimSpace(resp.Header.Get("ETag"))
patch := &store.CDNSourcePatch{LastRefreshedAt: &now}
if newEtag != "" && newEtag != strings.TrimSpace(src.Etag) {
e := newEtag
patch.Etag = &e
}
_, _ = st.UpdateCDNSource(tid, mod.ID, src.ID, patch)
rows := cdnRowsFromParsed(omod, src, prefixStrs)
_ = mergeCDNSourceIntoModuleSnapshot(st, tid, omod, src.ID, rows)
}
+7 -17
View File
@@ -19,6 +19,7 @@ import (
"time"
"evobgp/internal/birdfmt"
"evobgp/internal/httpclient"
"evobgp/internal/observability"
"evobgp/internal/store"
@@ -50,7 +51,7 @@ func MaterializedASPrefixKey(asn int64) string {
// It does not create a new config revision.
func RefreshModuleIngest(ctx context.Context, st store.Backend, hc *http.Client, tenantID, moduleID string) error {
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
start := time.Now()
mod, err := st.GetModule(tenantID, moduleID)
@@ -84,7 +85,7 @@ func RefreshModuleIngest(ctx context.Context, st store.Backend, hc *http.Client,
// If materialized prefixes are unchanged, returns latest revision id without creating a duplicate.
func RenderTenantRevision(ctx context.Context, st store.Backend, hc *http.Client, tenantID, triggerModuleID string) (revisionID string, err error) {
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
agg, err := aggregateTenantPrefixRowsAll(ctx, st, hc, tenantID)
if err != nil {
@@ -117,7 +118,7 @@ func RenderTenantRevision(ctx context.Context, st store.Backend, hc *http.Client
func RenderTenantRevisionFromPrefixes(ctx context.Context, st store.Backend, hc *http.Client, tenantID, triggerModuleID string, rows []store.PrefixRow) (revisionID string, err error) {
_ = ctx
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
agg := append([]store.PrefixRow(nil), rows...)
rawCount := len(agg)
@@ -176,7 +177,7 @@ func collectModulePrefixRows(ctx context.Context, st store.Backend, hc *http.Cli
return nil, err
}
sort.Slice(list, func(i, j int) bool { return list[i].ASN < list[j].ASN })
return collectASPrefixRows(ctx, st, hc, tenantID, mod, list)
return collectASPrefixRows(ctx, st, hc, tenantID, mod, list, priorSnapshot)
case "CDN_CIDRS":
sources, err := st.ListCDNSources(tenantID, moduleID)
if err != nil {
@@ -192,7 +193,7 @@ func collectModulePrefixRows(ctx context.Context, st store.Backend, hc *http.Cli
if err != nil {
return nil, err
}
return collectDomainPrefixRows(ctx, hc, mod, profiles, policy, entries)
return collectDomainPrefixRows(ctx, hc, mod, profiles, policy, entries, priorSnapshot)
default:
return nil, fmt.Errorf("pipeline: unknown module type %q", mod.Type)
}
@@ -1029,7 +1030,7 @@ func renderPeersBirdFragment(st store.Backend, tenantID string, loc birdLocals)
}
pol := parsePeerPolicies(p.PoliciesJSON)
lv4, lv6, asn := effectivePeerLocals(loc, pol)
proto := peerProtocolName(p.ID)
proto := birdfmt.PeerProtocolName(p.ID)
ra := uint32(p.RemoteASN)
if addr.Is4() {
opts := birdfmt.BGPPeerFromTemplateOptions{
@@ -1083,17 +1084,6 @@ func parsePeerPolicies(raw string) peerPolicyJSON {
return pol
}
func peerProtocolName(peerID string) string {
s := strings.ReplaceAll(strings.TrimSpace(peerID), "-", "")
if len(s) > 16 {
s = s[:16]
}
if s == "" {
s = "x"
}
return "evobgp_p_" + s
}
// buildExpandedBirdText concatenates bird.conf and the contents of each standard include (for UI / preview).
func buildExpandedBirdText(main string, frags map[string]string) string {
var b strings.Builder
+2 -1
View File
@@ -7,13 +7,14 @@ import (
"strings"
"sync"
"evobgp/internal/httpclient"
"evobgp/internal/store"
)
// RefreshTenantModules ingests all listed modules in parallel and updates per-module snapshots.
func RefreshTenantModules(ctx context.Context, st store.Backend, hc *http.Client, tenantID string, moduleIDs []string) error {
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
var ids []string
seen := make(map[string]struct{})
+23
View File
@@ -0,0 +1,23 @@
package pipeline
import (
"context"
"fmt"
"net/http"
"evobgp/internal/httpclient"
)
func upstreamHTTPDo(ctx context.Context, hc *http.Client, req *http.Request) (*http.Response, error) {
if hc == nil {
hc = httpclient.New(httpclient.DefaultTimeout)
}
resp, err := httpclient.DoWithBreaker(ctx, hc, req, 3)
if err != nil {
if req.URL != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", req.URL.String(), err)
}
return nil, err
}
return resp, nil
}
+61 -14
View File
@@ -11,14 +11,22 @@ import (
"github.com/jackc/pgx/v5"
)
func asnCacheRowTableExists(ctx context.Context, q queryRower) bool {
var n int
err := q.QueryRow(ctx, `
SELECT 1 FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'asn_prefix_cache_row'
LIMIT 1`).Scan(&n)
return err == nil
}
func (p *Postgres) GetASNPrefixCache(asn int64) (*store.ASNPrefixCacheEntry, bool, error) {
ctx := context.Background()
var holder string
var fetchedAt time.Time
var raw []byte
err := p.pool.QueryRow(ctx, `
SELECT holder, fetched_at, prefixes_json FROM asn_prefix_cache WHERE asn = $1`, asn).
Scan(&holder, &fetchedAt, &raw)
SELECT holder, fetched_at FROM asn_prefix_cache WHERE asn = $1`, asn).
Scan(&holder, &fetchedAt)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, false, nil
@@ -26,8 +34,25 @@ func (p *Postgres) GetASNPrefixCache(asn int64) (*store.ASNPrefixCacheEntry, boo
return nil, false, err
}
var prefixes []string
if len(raw) > 0 {
_ = json.Unmarshal(raw, &prefixes)
if asnCacheRowTableExists(ctx, p.pool) {
rows, qerr := p.pool.Query(ctx, `
SELECT prefix::text FROM asn_prefix_cache_row WHERE asn = $1 ORDER BY prefix`, asn)
if qerr != nil {
return nil, false, qerr
}
defer rows.Close()
for rows.Next() {
var s string
if err := rows.Scan(&s); err != nil {
continue
}
prefixes = append(prefixes, s)
}
} else {
var raw []byte
if err := p.pool.QueryRow(ctx, `SELECT prefixes_json FROM asn_prefix_cache WHERE asn = $1`, asn).Scan(&raw); err == nil && len(raw) > 0 {
_ = json.Unmarshal(raw, &prefixes)
}
}
return &store.ASNPrefixCacheEntry{
ASN: asn,
@@ -38,18 +63,40 @@ func (p *Postgres) GetASNPrefixCache(asn int64) (*store.ASNPrefixCacheEntry, boo
}
func (p *Postgres) SetASNPrefixCache(asn int64, holder string, prefixes []string) error {
raw, err := json.Marshal(prefixes)
ctx := context.Background()
tx, err := p.pool.Begin(ctx)
if err != nil {
return err
}
ctx := context.Background()
_, err = p.pool.Exec(ctx, `
INSERT INTO asn_prefix_cache (asn, holder, prefixes_json, fetched_at)
VALUES ($1, $2, $3::jsonb, now())
defer func() { _ = tx.Rollback(ctx) }()
_, err = tx.Exec(ctx, `
INSERT INTO asn_prefix_cache (asn, holder, fetched_at)
VALUES ($1, $2, now())
ON CONFLICT (asn) DO UPDATE SET
holder = EXCLUDED.holder,
prefixes_json = EXCLUDED.prefixes_json,
fetched_at = EXCLUDED.fetched_at`,
asn, holder, string(raw))
return err
fetched_at = EXCLUDED.fetched_at`, asn, holder)
if err != nil {
return err
}
if asnCacheRowTableExists(ctx, tx) {
if _, err := tx.Exec(ctx, `DELETE FROM asn_prefix_cache_row WHERE asn = $1`, asn); err != nil {
return err
}
for _, pfx := range prefixes {
if _, err := tx.Exec(ctx, `
INSERT INTO asn_prefix_cache_row (asn, prefix) VALUES ($1, $2::cidr)`, asn, pfx); err != nil {
return err
}
}
} else {
raw, err := json.Marshal(prefixes)
if err != nil {
return err
}
if _, err := tx.Exec(ctx, `
UPDATE asn_prefix_cache SET prefixes_json = $2::jsonb WHERE asn = $1`, asn, string(raw)); err != nil {
return err
}
}
return tx.Commit(ctx)
}
+19
View File
@@ -0,0 +1,19 @@
package repository
import (
"context"
"time"
)
const defaultRepoTimeout = 60 * time.Second
// boundedRepoCtx returns a context with default repository I/O timeout.
func boundedRepoCtx(parent context.Context) (context.Context, context.CancelFunc) {
if parent == nil {
parent = context.Background()
}
if _, ok := parent.Deadline(); ok {
return parent, func() {}
}
return context.WithTimeout(parent, defaultRepoTimeout)
}
+23 -2
View File
@@ -20,6 +20,28 @@ func NewJobAuditWriter(pool *pgxpool.Pool) *JobAuditWriter {
return &JobAuditWriter{pool: pool}
}
// UpsertQueued inserts a queued job row (best-effort).
func (w *JobAuditWriter) UpsertQueued(ctx context.Context, tenantID, jobID, kind string, idempotencyKey *string, moduleID *string, meta map[string]any) {
if w == nil || w.pool == nil {
return
}
metaJSON, _ := json.Marshal(meta)
var idem any
if idempotencyKey != nil && *idempotencyKey != "" {
idem = *idempotencyKey
}
var mod any
if moduleID != nil && *moduleID != "" {
mod = *moduleID
}
_, _ = w.pool.Exec(ctx, `
INSERT INTO job_audit (id, tenant_id, kind, status, idempotency_key, module_id, meta_json, created_at)
VALUES ($1::uuid, $2::uuid, $3, 'queued', $4, $5::uuid, $6::jsonb, now())
ON CONFLICT (tenant_id, idempotency_key) WHERE idempotency_key IS NOT NULL
DO UPDATE SET status='queued', meta_json=EXCLUDED.meta_json, module_id=EXCLUDED.module_id`,
jobID, tenantID, kind, idem, mod, metaJSON)
}
// UpsertRunning inserts or updates a running job row (best-effort).
func (w *JobAuditWriter) UpsertRunning(ctx context.Context, tenantID, jobID, kind string, idempotencyKey *string, meta map[string]any) {
if w == nil || w.pool == nil {
@@ -33,8 +55,7 @@ func (w *JobAuditWriter) UpsertRunning(ctx context.Context, tenantID, jobID, kin
_, _ = w.pool.Exec(ctx, `
INSERT INTO job_audit (id, tenant_id, kind, status, idempotency_key, meta_json, created_at, started_at)
VALUES ($1::uuid, $2::uuid, $3, 'running', $4, $5::jsonb, now(), now())
ON CONFLICT (tenant_id, idempotency_key) WHERE idempotency_key IS NOT NULL
DO UPDATE SET status='running', started_at=now(), meta_json=EXCLUDED.meta_json`,
ON CONFLICT (id) DO UPDATE SET status='running', started_at=COALESCE(job_audit.started_at, now()), meta_json=EXCLUDED.meta_json`,
jobID, tenantID, kind, idem, metaJSON)
}
+29
View File
@@ -0,0 +1,29 @@
package repository
import (
"context"
"time"
)
const (
jobAuditRetentionDays = 90
asnCacheRetentionDays = 7
)
// RunPeriodicMaintenance prunes stale job_audit and asn_prefix_cache rows (PostgreSQL).
func (p *Postgres) RunPeriodicMaintenance(ctx context.Context) {
if p == nil || p.pool == nil {
return
}
if ctx == nil {
ctx = context.Background()
}
jobCutoff := time.Now().UTC().Add(-time.Duration(jobAuditRetentionDays) * 24 * time.Hour)
_, _ = p.pool.Exec(ctx, `
DELETE FROM job_audit
WHERE created_at < $1
AND status IN ('succeeded', 'failed', 'cancelled')`, jobCutoff)
asnCutoff := time.Now().UTC().Add(-time.Duration(asnCacheRetentionDays) * 24 * time.Hour)
_, _ = p.pool.Exec(ctx, `
DELETE FROM asn_prefix_cache WHERE fetched_at < $1`, asnCutoff)
}
+82 -15
View File
@@ -12,16 +12,24 @@ import (
"github.com/jackc/pgx/v5"
)
func moduleSnapshotRowTableExists(ctx context.Context, q queryRower) bool {
var n int
err := q.QueryRow(ctx, `
SELECT 1 FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'module_prefix_snapshot_row'
LIMIT 1`).Scan(&n)
return err == nil
}
func (p *Postgres) GetModulePrefixSnapshot(tenantID, moduleID string) (*store.ModulePrefixSnapshot, bool, error) {
ctx := context.Background()
var inputHash string
var collectedAt time.Time
var raw []byte
err := p.pool.QueryRow(ctx, `
SELECT input_hash, collected_at, prefixes_json
SELECT input_hash, collected_at
FROM module_prefix_snapshot
WHERE tenant_id = $1 AND module_id = $2`,
tenantID, moduleID).Scan(&inputHash, &collectedAt, &raw)
tenantID, moduleID).Scan(&inputHash, &collectedAt)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, false, nil
@@ -29,9 +37,31 @@ func (p *Postgres) GetModulePrefixSnapshot(tenantID, moduleID string) (*store.Mo
return nil, false, err
}
var prefixes []store.PrefixRow
if len(raw) > 0 {
if err := json.Unmarshal(raw, &prefixes); err != nil {
return nil, false, err
if moduleSnapshotRowTableExists(ctx, p.pool) {
rows, qerr := p.pool.Query(ctx, `
SELECT prefix, community_id::text, source
FROM module_prefix_snapshot_row
WHERE tenant_id = $1::uuid AND module_id = $2::uuid
ORDER BY ord`, tenantID, moduleID)
if qerr != nil {
return nil, false, qerr
}
defer rows.Close()
for rows.Next() {
var pr store.PrefixRow
var comm *string
if err := rows.Scan(&pr.Prefix, &comm, &pr.Source); err != nil {
continue
}
pr.CommunityID = comm
prefixes = append(prefixes, pr)
}
} else {
var raw []byte
if err := p.pool.QueryRow(ctx, `
SELECT prefixes_json FROM module_prefix_snapshot
WHERE tenant_id = $1 AND module_id = $2`, tenantID, moduleID).Scan(&raw); err == nil && len(raw) > 0 {
_ = json.Unmarshal(raw, &prefixes)
}
}
return &store.ModulePrefixSnapshot{
@@ -45,20 +75,57 @@ func (p *Postgres) SetModulePrefixSnapshot(tenantID, moduleID, inputHash string,
if strings.TrimSpace(tenantID) == "" || strings.TrimSpace(moduleID) == "" || strings.TrimSpace(inputHash) == "" {
return store.ErrInvalidInput
}
raw, err := json.Marshal(prefixes)
ctx := context.Background()
tx, err := p.pool.Begin(ctx)
if err != nil {
return err
}
ctx := context.Background()
_, err = p.pool.Exec(ctx, `
INSERT INTO module_prefix_snapshot (tenant_id, module_id, input_hash, collected_at, prefixes_json)
VALUES ($1::uuid, $2::uuid, $3, now(), $4::jsonb)
defer func() { _ = tx.Rollback(ctx) }()
_, err = tx.Exec(ctx, `
INSERT INTO module_prefix_snapshot (tenant_id, module_id, input_hash, collected_at)
VALUES ($1::uuid, $2::uuid, $3, now())
ON CONFLICT (tenant_id, module_id) DO UPDATE SET
input_hash = EXCLUDED.input_hash,
collected_at = EXCLUDED.collected_at,
prefixes_json = EXCLUDED.prefixes_json`,
tenantID, moduleID, inputHash, string(raw))
return err
collected_at = EXCLUDED.collected_at`,
tenantID, moduleID, inputHash)
if err != nil {
return err
}
if moduleSnapshotRowTableExists(ctx, tx) {
if _, err := tx.Exec(ctx, `
DELETE FROM module_prefix_snapshot_row
WHERE tenant_id = $1::uuid AND module_id = $2::uuid`, tenantID, moduleID); err != nil {
return err
}
for i, pr := range prefixes {
var comm any
if pr.CommunityID != nil && strings.TrimSpace(*pr.CommunityID) != "" {
comm = strings.TrimSpace(*pr.CommunityID)
}
src := pr.Source
if strings.TrimSpace(src) == "" {
src = "render"
}
if _, err := tx.Exec(ctx, `
INSERT INTO module_prefix_snapshot_row (tenant_id, module_id, ord, prefix, community_id, source)
VALUES ($1::uuid, $2::uuid, $3, $4, $5::uuid, $6)`,
tenantID, moduleID, i, strings.TrimSpace(pr.Prefix), comm, src); err != nil {
return err
}
}
} else {
raw, err := json.Marshal(prefixes)
if err != nil {
return err
}
if _, err := tx.Exec(ctx, `
UPDATE module_prefix_snapshot SET prefixes_json = $3::jsonb
WHERE tenant_id = $1::uuid AND module_id = $2::uuid`,
tenantID, moduleID, string(raw)); err != nil {
return err
}
}
return tx.Commit(ctx)
}
func (p *Postgres) DeleteModulePrefixSnapshot(tenantID, moduleID string) error {
+106 -67
View File
@@ -659,7 +659,8 @@ func (p *Postgres) DeleteSpeaker(tenantID, id string) error {
}
func (p *Postgres) GetRevision(tenantID, revisionID string) (*store.Revision, error) {
ctx := context.Background()
ctx, cancel := boundedRepoCtx(context.Background())
defer cancel()
var r store.Revision
var mod *string
var parent *string
@@ -686,11 +687,38 @@ func (p *Postgres) GetRevision(tenantID, revisionID string) (*store.Revision, er
if mj.PreviewFragments == nil {
mj.PreviewFragments = map[string]string{}
}
r.PreviewFragments = mj.PreviewFragments
r.PreviewFragments = loadRevisionPreview(ctx, p.pool, revisionID, mj.PreviewFragments)
r.MaterializedPrefixCount = mj.MaterializedPrefixCount
return &r, nil
}
func (p *Postgres) GetRevisionSummary(tenantID, revisionID string) (*store.Revision, error) {
ctx, cancel := boundedRepoCtx(context.Background())
defer cancel()
var r store.Revision
var mod *string
var parent *string
var prefixCount int
err := p.pool.QueryRow(ctx, `
SELECT id::text, tenant_id::text, module_id::text, content_hash, parent_revision_id::text,
COALESCE((meta_json->>'materialized_prefix_count')::int, 0), created_at
FROM config_revision WHERE id=$1 AND tenant_id=$2`, revisionID, tenantID).Scan(
&r.ID, &r.TenantID, &mod, &r.ContentHash, &parent, &prefixCount, &r.CreatedAt)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, store.ErrNotFound
}
return nil, err
}
if mod != nil {
r.ModuleID = *mod
}
r.ParentRevisionID = strOrNil(parent)
r.MaterializedPrefixCount = prefixCount
r.PreviewFragments = map[string]string{}
return &r, nil
}
func (p *Postgres) ListRevisions(tenantID, moduleID string, cursor string, limit int) ([]*store.Revision, string, bool) {
if limit <= 0 {
limit = 50
@@ -761,12 +789,7 @@ func (p *Postgres) ListRevisionPrefixes(tenantID, revisionID string, cursor stri
if limit <= 0 {
limit = 50
}
off := 0
if cursor != "" {
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
off = n
}
}
afterID, off, useOffset := store.ParsePrefixPageCursor(cursor)
ctx := context.Background()
var one int
if err := p.pool.QueryRow(ctx, `
@@ -777,34 +800,55 @@ func (p *Postgres) ListRevisionPrefixes(tenantID, revisionID string, cursor stri
}
return nil, "", false
}
rows, err := p.pool.Query(ctx, `
SELECT prefix::text, community_id::text, source FROM revision_materialized_prefix
WHERE revision_id=$1 ORDER BY id
LIMIT $2 OFFSET $3`, revisionID, limit+1, off)
if snapID, ok := p.revisionPrefixSnapshotID(ctx, revisionID); ok {
return p.listSnapshotPrefixes(ctx, snapID, cursor, limit)
}
var rows pgx.Rows
var err error
if useOffset {
rows, err = p.pool.Query(ctx, `
SELECT id, prefix::text, community_id::text, source FROM revision_materialized_prefix
WHERE revision_id=$1::uuid ORDER BY id
LIMIT $2 OFFSET $3`, revisionID, limit+1, off)
} else {
var afterArg any
if afterID != nil {
afterArg = *afterID
}
rows, err = p.pool.Query(ctx, `
SELECT id, prefix::text, community_id::text, source FROM revision_materialized_prefix
WHERE revision_id=$1::uuid AND ($2::bigint IS NULL OR id > $2::bigint)
ORDER BY id
LIMIT $3`, revisionID, afterArg, limit+1)
}
if err != nil {
return nil, "", false
}
defer rows.Close()
var all []store.PrefixRow
var ids []int64
for rows.Next() {
var rowID int64
var pr store.PrefixRow
var comm *string
if err := rows.Scan(&pr.Prefix, &comm, &pr.Source); err != nil {
if err := rows.Scan(&rowID, &pr.Prefix, &comm, &pr.Source); err != nil {
continue
}
pr.CommunityID = comm
ids = append(ids, rowID)
all = append(all, pr)
}
agentDebugNDJSON3214("B", "repository/postgres.go:ListRevisionPrefixes", "list_prefixes_fetched", map[string]any{
"rows": len(all), "limit": limit, "offset": off,
"rows": len(all), "limit": limit, "keyset": !useOffset,
})
more := len(all) > limit
if more {
all = all[:limit]
ids = ids[:limit]
}
next := ""
if more {
next = fmt.Sprintf("%d", off+limit)
if more && len(ids) > 0 {
next = store.FormatPrefixPageCursor(ids[len(ids)-1])
}
if len(all) == 0 {
return nil, "", false
@@ -820,26 +864,35 @@ func (p *Postgres) CreateRollbackRevision(tenantID, sourceRevisionID string) (st
ctx := context.Background()
newID := uuid.NewString()
parent := sourceRevisionID
meta, _ := json.Marshal(map[string]any{
"preview_fragments": src.PreviewFragments,
"materialized_prefix_count": src.MaterializedPrefixCount,
})
meta, err := revisionMetaWithoutPreview(src.MaterializedPrefixCount)
if err != nil {
return "", err
}
var modArg any
if strings.TrimSpace(src.ModuleID) != "" {
modArg = src.ModuleID
}
_, err = p.pool.Exec(ctx, `
INSERT INTO config_revision (id, tenant_id, module_id, content_hash, parent_revision_id, meta_json)
VALUES ($1,$2,$3,$4,$5::uuid,$6::jsonb)`,
newID, tenantID, modArg, src.ContentHash+":rollback", parent, string(meta))
tx, err := p.pool.Begin(ctx)
if err != nil {
return "", err
}
// copy materialized prefixes
_, _ = p.pool.Exec(ctx, `
INSERT INTO revision_materialized_prefix (revision_id, prefix, community_id, source, meta_json)
SELECT $1::uuid, prefix, community_id, source, meta_json FROM revision_materialized_prefix WHERE revision_id=$2::uuid`,
newID, sourceRevisionID)
defer func() { _ = tx.Rollback(ctx) }()
_, err = tx.Exec(ctx, `
INSERT INTO config_revision (id, tenant_id, module_id, content_hash, parent_revision_id, meta_json)
VALUES ($1,$2,$3,$4,$5::uuid,$6::jsonb)`,
newID, tenantID, modArg, src.ContentHash+":rollback", parent, meta)
if err != nil {
return "", err
}
if err := copyRevisionPreview(ctx, tx, newID, sourceRevisionID); err != nil {
return "", err
}
if err := p.copyRevisionPrefixSnapshotRef(ctx, tx, newID, sourceRevisionID); err != nil {
return "", err
}
if err := tx.Commit(ctx); err != nil {
return "", err
}
return newID, nil
}
@@ -856,20 +909,18 @@ func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, erro
var unchanged int
err := p.pool.QueryRow(ctx, `
SELECT COUNT(*)::int FROM (
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
INTERSECT
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
SELECT b.prefix FROM (`+sqlRevisionPrefixes("$2")+`) b
INNER JOIN (`+sqlRevisionPrefixes("$1")+`) a ON a.prefix = b.prefix
) t`, aID, bID).Scan(&unchanged)
if err != nil {
return nil, err
}
// added: в B, нет в A; removed: в A, нет в B — без загрузки полных снапшотов в память.
rowsAdded, err := p.pool.Query(ctx, `
SELECT prefix::text FROM (
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
EXCEPT
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
) s ORDER BY 1 LIMIT $3`, bID, aID, maxRevisionDiffRows+1)
SELECT b.prefix::text FROM (`+sqlRevisionPrefixes("$2")+`) b
LEFT JOIN (`+sqlRevisionPrefixes("$1")+`) a ON a.prefix = b.prefix
WHERE a.prefix IS NULL
ORDER BY b.prefix
LIMIT $3`, aID, bID, maxRevisionDiffRows+1)
if err != nil {
return nil, err
}
@@ -888,11 +939,11 @@ func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, erro
}
addedTruncated := len(added) >= maxRevisionDiffRows
rowsRem, err := p.pool.Query(ctx, `
SELECT prefix::text FROM (
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
EXCEPT
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
) s ORDER BY 1 LIMIT $3`, aID, bID, maxRevisionDiffRows+1)
SELECT a.prefix::text FROM (`+sqlRevisionPrefixes("$1")+`) a
LEFT JOIN (`+sqlRevisionPrefixes("$2")+`) b ON b.prefix = a.prefix
WHERE b.prefix IS NULL
ORDER BY a.prefix
LIMIT $3`, aID, bID, maxRevisionDiffRows+1)
if err != nil {
return nil, err
}
@@ -1028,10 +1079,7 @@ func (p *Postgres) CreateRenderRevision(revisionID, tenantID, moduleID string, p
if previewFragments == nil {
previewFragments = map[string]string{}
}
meta, err := json.Marshal(map[string]any{
"preview_fragments": previewFragments,
"materialized_prefix_count": len(prefixes),
})
meta, err := revisionMetaWithoutPreview(len(prefixes))
if err != nil {
return err
}
@@ -1045,32 +1093,23 @@ func (p *Postgres) CreateRenderRevision(revisionID, tenantID, moduleID string, p
if parentRevisionID != nil && strings.TrimSpace(*parentRevisionID) != "" {
parent = strings.TrimSpace(*parentRevisionID)
}
revID := strings.TrimSpace(revisionID)
_, err = tx.Exec(ctx, `
INSERT INTO config_revision (id, tenant_id, module_id, content_hash, parent_revision_id, meta_json)
VALUES ($1::uuid, $2::uuid, $3::uuid, $4, $5::uuid, $6::jsonb)`,
strings.TrimSpace(revisionID), tenantID, moduleID, strings.TrimSpace(contentHash), parent, string(meta))
revID, tenantID, moduleID, strings.TrimSpace(contentHash), parent, meta)
if err != nil {
return err
}
if len(prefixes) > 0 {
_, err = tx.CopyFrom(ctx,
pgx.Identifier{"revision_materialized_prefix"},
[]string{"revision_id", "prefix", "community_id", "source"},
pgx.CopyFromSlice(len(prefixes), func(i int) ([]any, error) {
pr := prefixes[i]
var comm any
if pr.CommunityID != nil && strings.TrimSpace(*pr.CommunityID) != "" {
comm = strings.TrimSpace(*pr.CommunityID)
}
src := pr.Source
if strings.TrimSpace(src) == "" {
src = "render"
}
return []any{strings.TrimSpace(revisionID), strings.TrimSpace(pr.Prefix), comm, src}, nil
}))
if err != nil {
return err
}
if err := insertRevisionPreview(ctx, tx, revID, previewFragments); err != nil {
return err
}
snapID, err := p.ensurePrefixSnapshot(ctx, tx, contentHash, prefixes)
if err != nil {
return err
}
if err := p.linkRevisionPrefixSnapshot(ctx, tx, revID, snapID); err != nil {
return err
}
if err := tx.Commit(ctx); err != nil {
return err
@@ -0,0 +1,186 @@
package repository
import (
"context"
"errors"
"strings"
"evobgp/internal/store"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
)
func prefixSnapshotTableExists(ctx context.Context, q queryRower) bool {
var n int
err := q.QueryRow(ctx, `
SELECT 1 FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'prefix_snapshot'
LIMIT 1`).Scan(&n)
return err == nil
}
func normalizeSnapshotHash(contentHash string) string {
h := strings.TrimSpace(contentHash)
if strings.HasPrefix(h, "sha256:") {
h = strings.TrimPrefix(h, "sha256:")
}
if len(h) > 64 {
h = h[:64]
}
if len(h) < 64 {
h = h + strings.Repeat("0", 64-len(h))
}
return h
}
func (p *Postgres) revisionPrefixSnapshotID(ctx context.Context, revisionID string) (string, bool) {
if !prefixSnapshotTableExists(ctx, p.pool) {
return "", false
}
var snap *string
err := p.pool.QueryRow(ctx, `
SELECT prefix_snapshot_id::text FROM config_revision
WHERE id = $1::uuid AND prefix_snapshot_id IS NOT NULL`, revisionID).Scan(&snap)
if err != nil || snap == nil || strings.TrimSpace(*snap) == "" {
return "", false
}
return *snap, true
}
func (p *Postgres) ensurePrefixSnapshot(ctx context.Context, db execQuerier, contentHash string, prefixes []store.PrefixRow) (string, error) {
if !prefixSnapshotTableExists(ctx, db) {
return "", nil
}
hash := normalizeSnapshotHash(contentHash)
var existing string
err := db.QueryRow(ctx, `SELECT id::text FROM prefix_snapshot WHERE content_hash = $1`, hash).Scan(&existing)
if err == nil && existing != "" {
return existing, nil
}
if err != nil && !errors.Is(err, pgx.ErrNoRows) {
return "", err
}
snapID := uuid.NewString()
if _, err := db.Exec(ctx, `
INSERT INTO prefix_snapshot (id, content_hash) VALUES ($1::uuid, $2)
ON CONFLICT (content_hash) DO NOTHING`, snapID, hash); err != nil {
return "", err
}
if err := db.QueryRow(ctx, `SELECT id::text FROM prefix_snapshot WHERE content_hash = $1`, hash).Scan(&snapID); err != nil {
return "", err
}
var rowCount int
_ = db.QueryRow(ctx, `SELECT COUNT(*)::int FROM prefix_snapshot_row WHERE snapshot_id = $1::uuid`, snapID).Scan(&rowCount)
if rowCount > 0 {
return snapID, nil
}
for i, pr := range prefixes {
var comm any
if pr.CommunityID != nil && strings.TrimSpace(*pr.CommunityID) != "" {
comm = strings.TrimSpace(*pr.CommunityID)
}
src := pr.Source
if strings.TrimSpace(src) == "" {
src = "render"
}
if _, err := db.Exec(ctx, `
INSERT INTO prefix_snapshot_row (snapshot_id, ord, prefix, community_id, source)
VALUES ($1::uuid, $2, $3, $4::uuid, $5)`,
snapID, i, strings.TrimSpace(pr.Prefix), comm, src); err != nil {
return "", err
}
}
return snapID, nil
}
func (p *Postgres) listSnapshotPrefixes(ctx context.Context, snapshotID, cursor string, limit int) ([]store.PrefixRow, string, bool) {
afterOrd, off, useOffset := store.ParsePrefixPageCursor(cursor)
var rows pgx.Rows
var err error
if useOffset {
rows, err = p.pool.Query(ctx, `
SELECT ord, prefix::text, community_id::text, source
FROM prefix_snapshot_row
WHERE snapshot_id = $1::uuid
ORDER BY ord
LIMIT $2 OFFSET $3`, snapshotID, limit+1, off)
} else {
var afterArg any
if afterOrd != nil {
afterArg = int(*afterOrd)
}
rows, err = p.pool.Query(ctx, `
SELECT ord, prefix::text, community_id::text, source
FROM prefix_snapshot_row
WHERE snapshot_id = $1::uuid AND ($2::int IS NULL OR ord > $2::int)
ORDER BY ord
LIMIT $3`, snapshotID, afterArg, limit+1)
}
if err != nil {
return nil, "", false
}
defer rows.Close()
var all []store.PrefixRow
var ords []int64
for rows.Next() {
var ord int
var pr store.PrefixRow
var comm *string
if err := rows.Scan(&ord, &pr.Prefix, &comm, &pr.Source); err != nil {
continue
}
pr.CommunityID = comm
ords = append(ords, int64(ord))
all = append(all, pr)
}
more := len(all) > limit
if more {
all = all[:limit]
ords = ords[:limit]
}
next := ""
if more && len(ords) > 0 {
next = store.FormatPrefixPageCursor(ords[len(ords)-1])
}
if len(all) == 0 {
return nil, "", false
}
return all, next, more
}
func (p *Postgres) linkRevisionPrefixSnapshot(ctx context.Context, db execQuerier, revisionID, snapshotID string) error {
if snapshotID == "" || !prefixSnapshotTableExists(ctx, db) {
return nil
}
_, err := db.Exec(ctx, `
UPDATE config_revision SET prefix_snapshot_id = $2::uuid WHERE id = $1::uuid`,
revisionID, snapshotID)
return err
}
func (p *Postgres) copyRevisionPrefixSnapshotRef(ctx context.Context, db execQuerier, dstRevisionID, srcRevisionID string) error {
if !prefixSnapshotTableExists(ctx, db) {
return nil
}
_, err := db.Exec(ctx, `
UPDATE config_revision dst
SET prefix_snapshot_id = src.prefix_snapshot_id
FROM config_revision src
WHERE dst.id = $1::uuid AND src.id = $2::uuid AND src.prefix_snapshot_id IS NOT NULL`,
dstRevisionID, srcRevisionID)
return err
}
func sqlRevisionPrefixes(revParam string) string {
return `SELECT psr.prefix FROM config_revision cr
JOIN prefix_snapshot_row psr ON psr.snapshot_id = cr.prefix_snapshot_id
WHERE cr.id = ` + revParam + `::uuid AND cr.prefix_snapshot_id IS NOT NULL
UNION ALL
SELECT rmp.prefix FROM revision_materialized_prefix rmp
WHERE rmp.revision_id = ` + revParam + `::uuid
AND NOT EXISTS (
SELECT 1 FROM config_revision cr2
WHERE cr2.id = ` + revParam + `::uuid AND cr2.prefix_snapshot_id IS NOT NULL
)`
}
@@ -0,0 +1,90 @@
package repository
import (
"context"
"encoding/json"
"errors"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
)
func revisionPreviewTableExists(ctx context.Context, q queryRower) bool {
var n int
err := q.QueryRow(ctx, `
SELECT 1 FROM information_schema.tables
WHERE table_schema = 'public' AND table_name = 'config_revision_preview'
LIMIT 1`).Scan(&n)
return err == nil
}
type queryRower interface {
QueryRow(ctx context.Context, sql string, args ...any) pgx.Row
}
func loadRevisionPreview(ctx context.Context, q queryRower, revisionID string, metaPreview map[string]string) map[string]string {
if revisionPreviewTableExists(ctx, q) {
var raw []byte
err := q.QueryRow(ctx, `
SELECT fragments FROM config_revision_preview WHERE revision_id = $1::uuid`,
revisionID).Scan(&raw)
if err == nil {
out := map[string]string{}
_ = json.Unmarshal(raw, &out)
if out == nil {
out = map[string]string{}
}
return out
}
if !errors.Is(err, pgx.ErrNoRows) {
return metaPreview
}
}
if metaPreview == nil {
return map[string]string{}
}
return metaPreview
}
type execQuerier interface {
queryRower
Exec(ctx context.Context, sql string, arguments ...any) (pgconn.CommandTag, error)
}
func insertRevisionPreview(ctx context.Context, db execQuerier, revisionID string, preview map[string]string) error {
if !revisionPreviewTableExists(ctx, db) {
return nil
}
if preview == nil {
preview = map[string]string{}
}
raw, err := json.Marshal(preview)
if err != nil {
return err
}
_, err = db.Exec(ctx, `
INSERT INTO config_revision_preview (revision_id, fragments)
VALUES ($1::uuid, $2::jsonb)
ON CONFLICT (revision_id) DO UPDATE SET fragments = EXCLUDED.fragments`,
revisionID, string(raw))
return err
}
func copyRevisionPreview(ctx context.Context, db execQuerier, dstRevisionID, srcRevisionID string) error {
if !revisionPreviewTableExists(ctx, db) {
return nil
}
_, err := db.Exec(ctx, `
INSERT INTO config_revision_preview (revision_id, fragments)
SELECT $1::uuid, fragments FROM config_revision_preview WHERE revision_id = $2::uuid
ON CONFLICT (revision_id) DO UPDATE SET fragments = EXCLUDED.fragments`,
dstRevisionID, srcRevisionID)
return err
}
func revisionMetaWithoutPreview(materializedPrefixCount int) (string, error) {
raw, err := json.Marshal(map[string]any{
"materialized_prefix_count": materializedPrefixCount,
})
return string(raw), err
}
+30 -15
View File
@@ -26,9 +26,8 @@ func (p *Postgres) seedDemo(ctx context.Context) error {
p1 := uuid.NewString()
p2 := uuid.NewString()
preview := map[string]any{
"preview_fragments": map[string]string{
"bird.conf": `# EvoBGP demo bundle
previewFrags := map[string]string{
"bird.conf": `# EvoBGP demo bundle
router id 192.0.2.1;
protocol device {
@@ -39,15 +38,12 @@ protocol direct {
ipv6;
}
`,
"bird.d/evobgp_demo.conf": "# static demo fragment\n",
},
"materialized_prefix_count": 128,
"bird.d/evobgp_demo.conf": "# static demo fragment\n",
}
previewB, _ := json.Marshal(preview)
parentMeta, _ := json.Marshal(map[string]any{
"preview_fragments": map[string]string{"bird.conf": "# parent revision\n"},
"materialized_prefix_count": 0,
})
previewMeta, _ := json.Marshal(map[string]any{"materialized_prefix_count": 128})
parentMeta, _ := json.Marshal(map[string]any{"materialized_prefix_count": 0})
parentPreview, _ := json.Marshal(map[string]string{"bird.conf": "# parent revision\n"})
previewFragsB, _ := json.Marshal(previewFrags)
tx, err := p.pool.Begin(ctx)
if err != nil {
@@ -77,13 +73,32 @@ protocol direct {
return err
}
if _, err := tx.Exec(ctx, `
INSERT INTO config_revision (id, tenant_id, content_hash, parent_revision_id, meta_json)
VALUES ($1,$2,'sha256:demo-rev-1',$3::uuid,$4::jsonb)`, rid, tid, parent, string(previewB)); err != nil {
INSERT INTO config_revision_preview (revision_id, fragments) VALUES ($1::uuid, $2::jsonb)`, parent, string(parentPreview)); err != nil {
return err
}
if _, err := tx.Exec(ctx, `
INSERT INTO revision_materialized_prefix (revision_id, prefix, community_id, source)
VALUES ($1::uuid,'203.0.113.0/24',$2::uuid,'demo'), ($1::uuid,'2001:db8::/32',$2::uuid,'demo')`, rid, cid); err != nil {
INSERT INTO config_revision (id, tenant_id, content_hash, parent_revision_id, meta_json)
VALUES ($1,$2,'sha256:demo-rev-1',$3::uuid,$4::jsonb)`, rid, tid, parent, string(previewMeta)); err != nil {
return err
}
if _, err := tx.Exec(ctx, `
INSERT INTO config_revision_preview (revision_id, fragments) VALUES ($1::uuid, $2::jsonb)`, rid, string(previewFragsB)); err != nil {
return err
}
snapID := uuid.NewString()
demoHash := normalizeSnapshotHash("sha256:demo-rev-1")
if _, err := tx.Exec(ctx, `
INSERT INTO prefix_snapshot (id, content_hash) VALUES ($1::uuid, $2)`, snapID, demoHash); err != nil {
return err
}
if _, err := tx.Exec(ctx, `
UPDATE config_revision SET prefix_snapshot_id = $2::uuid WHERE id = $1::uuid`, rid, snapID); err != nil {
return err
}
if _, err := tx.Exec(ctx, `
INSERT INTO prefix_snapshot_row (snapshot_id, ord, prefix, community_id, source)
VALUES ($1::uuid, 0, '203.0.113.0/24', $2::uuid, 'demo'),
($1::uuid, 1, '2001:db8::/32', $2::uuid, 'demo')`, snapID, cid); err != nil {
return err
}
if _, err := tx.Exec(ctx, `
+6 -2
View File
@@ -14,6 +14,7 @@ import (
"evobgp/internal/broker"
"evobgp/internal/config"
"evobgp/internal/httpclient"
"evobgp/internal/jobs"
"evobgp/internal/pipeline"
"evobgp/internal/store"
@@ -108,6 +109,9 @@ func postTenantRefresh(ctx context.Context, deps *Deps, moduleIDs []string, idem
if err != nil {
return err
}
req.GetBody = func() (io.ReadCloser, error) {
return io.NopCloser(bytes.NewReader(body)), nil
}
req.Header.Set("Authorization", "Bearer "+strings.TrimSpace(deps.APIToken))
req.Header.Set("Content-Type", "application/json")
if idempotencyKey != "" {
@@ -115,9 +119,9 @@ func postTenantRefresh(ctx context.Context, deps *Deps, moduleIDs []string, idem
}
hc := deps.HTTP
if hc == nil {
hc = http.DefaultClient
hc = httpclient.New(httpclient.DefaultTimeout)
}
resp, err := hc.Do(req)
resp, err := httpclient.DoWithRetry(ctx, hc, req, 3)
if err != nil {
return err
}
+8 -3
View File
@@ -76,6 +76,8 @@ type Backend interface {
DeleteSpeaker(tenantID, id string) error
GetRevision(tenantID, revisionID string) (*Revision, error)
// GetRevisionSummary returns revision metadata without preview_fragments payloads.
GetRevisionSummary(tenantID, revisionID string) (*Revision, error)
ListRevisions(tenantID, moduleID string, cursor string, limit int) (items []*Revision, nextCursor string, hasMore bool)
ListRevisionPrefixes(tenantID, revisionID string, cursor string, limit int) (prefixes []PrefixRow, next string, more bool)
CreateRollbackRevision(tenantID, sourceRevisionID string) (newID string, err error)
@@ -110,6 +112,9 @@ type Backend interface {
// Ping verifies backend connectivity (no-op for in-memory).
Ping(ctx context.Context) error
// RunPeriodicMaintenance prunes stale DB rows (no-op for in-memory).
RunPeriodicMaintenance(ctx context.Context)
}
// ASNPrefixCacheEntry is a cached RIPEstat response for one ASN.
@@ -309,7 +314,7 @@ type SpeakerPatch struct {
// PrefixRow is one materialized prefix for GET /revisions/.../prefixes.
type PrefixRow struct {
Prefix string
CommunityID *string
Source string
Prefix string `json:"prefix"`
CommunityID *string `json:"community_id,omitempty"`
Source string `json:"source,omitempty"`
}
+17 -7
View File
@@ -305,6 +305,11 @@ func (m *Memory) Ping(ctx context.Context) error {
return nil
}
// RunPeriodicMaintenance is a no-op for the in-memory backend.
func (m *Memory) RunPeriodicMaintenance(ctx context.Context) {
_ = ctx
}
// ListTenantIDs returns tenant ids sorted lexicographically.
func (m *Memory) ListTenantIDs() ([]string, error) {
m.mu.RLock()
@@ -424,14 +429,19 @@ func (m *Memory) GetModule(tenantID, moduleID string) (*Module, error) {
func (m *Memory) GetRevision(tenantID, revisionID string) (*Revision, error) {
m.mu.RLock()
defer m.mu.RUnlock()
rev, ok := m.revisions[revisionID]
if !ok {
return nil, ErrNotFound
return m.getRevisionLocked(tenantID, revisionID)
}
func (m *Memory) GetRevisionSummary(tenantID, revisionID string) (*Revision, error) {
m.mu.RLock()
defer m.mu.RUnlock()
rev, err := m.getRevisionLocked(tenantID, revisionID)
if err != nil {
return nil, err
}
if rev.TenantID != tenantID {
return nil, ErrTenantScope
}
return rev, nil
cp := *rev
cp.PreviewFragments = nil
return &cp, nil
}
func (m *Memory) GetSpeaker(tenantID, speakerID string) (*Speaker, error) {
+23 -19
View File
@@ -1,8 +1,6 @@
package store
import (
"fmt"
"strconv"
"strings"
"time"
@@ -795,31 +793,37 @@ func (m *Memory) ListRevisionPrefixes(tenantID, revisionID string, cursor string
if limit <= 0 {
limit = 50
}
afterID, off, useOffset := ParsePrefixPageCursor(cursor)
m.mu.RLock()
defer m.mu.RUnlock()
if _, err := m.getRevisionLocked(tenantID, revisionID); err != nil {
return nil, "", false
}
all := m.revPrefixes[revisionID]
off := 0
if cursor != "" {
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
off = n
}
allRows := m.revPrefixes[revisionID]
start := 0
if useOffset {
start = off
} else if afterID != nil {
start = int(*afterID) + 1
}
end := off + limit
next := ""
more := false
if end > len(all) {
end = len(all)
} else {
more = true
next = fmt.Sprintf("%d", end)
}
if off >= len(all) {
if start > len(allRows) {
return nil, "", false
}
return all[off:end], next, more
end := start + limit
next := ""
more := false
if end > len(allRows) {
end = len(allRows)
} else {
more = true
next = FormatPrefixPageCursor(int64(end - 1))
}
if start >= end {
return nil, "", false
}
out := make([]PrefixRow, end-start)
copy(out, allRows[start:end])
return out, next, more
}
func (m *Memory) ListGlobalSettings(tenantID string) (map[string]any, error) {
+30
View File
@@ -0,0 +1,30 @@
package store
import (
"strconv"
"strings"
)
// ParsePrefixPageCursor decodes opaque cursors for revision prefix pagination.
func ParsePrefixPageCursor(cursor string) (afterID *int64, offset int, useOffset bool) {
cursor = strings.TrimSpace(cursor)
if cursor == "" {
return nil, 0, false
}
if strings.HasPrefix(cursor, "o:") {
n, err := strconv.Atoi(strings.TrimPrefix(cursor, "o:"))
if err != nil || n < 0 {
return nil, 0, false
}
return nil, n, true
}
if n, err := strconv.ParseInt(cursor, 10, 64); err == nil && n >= 0 {
return &n, 0, false
}
return nil, 0, false
}
// FormatPrefixPageCursor encodes the keyset cursor (last row id or slice index).
func FormatPrefixPageCursor(lastID int64) string {
return strconv.FormatInt(lastID, 10)
}
+13 -2
View File
@@ -72,7 +72,9 @@ func MergeSpeakerMetaJSON(existing string, patch SpeakerMeta) string {
if patch.LastDispatchAt != "" {
cur.LastDispatchAt = patch.LastDispatchAt
}
if patch.LastDispatchError != "" {
if patch.LastDispatchStatus == "ok" {
cur.LastDispatchError = ""
} else if patch.LastDispatchError != "" {
cur.LastDispatchError = patch.LastDispatchError
}
if patch.LastDispatchStatus != "" {
@@ -124,11 +126,20 @@ func AgentSyncURL(meta SpeakerMeta) string {
// AgentHealthURL returns HTTPS health URL for agent_domain.
func AgentHealthURL(meta SpeakerMeta) string {
return agentHTTPSURL(meta, "/v1/agent/health")
}
// AgentBirdProtocolsURL returns HTTPS bird protocols URL for agent_domain.
func AgentBirdProtocolsURL(meta SpeakerMeta) string {
return agentHTTPSURL(meta, "/v1/agent/bird/protocols")
}
func agentHTTPSURL(meta SpeakerMeta, path string) string {
domain := strings.TrimSpace(meta.AgentDomain)
if domain == "" {
return ""
}
return "https://" + strings.TrimSuffix(domain, "/") + "/v1/agent/health"
return "https://" + strings.TrimSuffix(domain, "/") + path
}
// SpeakerNeedsRemoteDispatch reports whether deploy_apply should wake this speaker via agent HTTP.
+21
View File
@@ -34,3 +34,24 @@ func TestAgentSyncURL(t *testing.T) {
t.Fatalf("got %q", u)
}
}
func TestMergeSpeakerMetaJSON_clearsDispatchErrorOnOk(t *testing.T) {
t.Parallel()
existing := store.SpeakerMetaJSON(store.SpeakerMeta{
LastDispatchError: "HTTP 502: bundle 403",
LastDispatchStatus: "error",
SyncStatus: "error",
})
merged := store.MergeSpeakerMetaJSON(existing, store.SpeakerMeta{
LastDispatchStatus: "ok",
SyncStatus: "synced",
LastDispatchAt: "2026-05-21T15:06:43Z",
})
m := store.ParseSpeakerMeta(merged)
if m.LastDispatchError != "" {
t.Fatalf("LastDispatchError should clear on ok dispatch, got %q", m.LastDispatchError)
}
if m.LastDispatchStatus != "ok" || m.SyncStatus != "synced" {
t.Fatalf("status: dispatch=%q sync=%q", m.LastDispatchStatus, m.SyncStatus)
}
}
@@ -0,0 +1,6 @@
DROP INDEX IF EXISTS idx_rev_mat_prefix_uniq;
DROP INDEX IF EXISTS idx_rev_mat_prefix_rev_cover;
CREATE INDEX IF NOT EXISTS idx_rev_mat_prefix_rev_id
ON revision_materialized_prefix (revision_id, id);
DROP INDEX IF EXISTS idx_module_tenant_active_sort;
DROP INDEX IF EXISTS idx_rev_mat_prefix_rev_prefix;
@@ -0,0 +1,17 @@
-- Query indexes: revision diff/list and module sort (H4 + M4).
CREATE INDEX IF NOT EXISTS idx_rev_mat_prefix_rev_prefix
ON revision_materialized_prefix (revision_id, prefix);
CREATE INDEX IF NOT EXISTS idx_module_tenant_active_sort
ON module (tenant_id, priority, name)
WHERE deleted_at IS NULL;
-- Covering index for prefix list pagination (PostgreSQL 11+ INCLUDE).
DROP INDEX IF EXISTS idx_rev_mat_prefix_rev_id;
CREATE INDEX IF NOT EXISTS idx_rev_mat_prefix_rev_cover
ON revision_materialized_prefix (revision_id, id)
INCLUDE (prefix, community_id, source);
CREATE UNIQUE INDEX IF NOT EXISTS idx_rev_mat_prefix_uniq
ON revision_materialized_prefix (revision_id, prefix, community_id, source);
@@ -0,0 +1,6 @@
UPDATE config_revision cr
SET meta_json = cr.meta_json || jsonb_build_object('preview_fragments', COALESCE(p.fragments, '{}'::jsonb))
FROM config_revision_preview p
WHERE p.revision_id = cr.id;
DROP TABLE IF EXISTS config_revision_preview;
@@ -0,0 +1,15 @@
-- Split BIRD preview fragments out of config_revision.meta_json (H1).
CREATE TABLE config_revision_preview (
revision_id UUID PRIMARY KEY REFERENCES config_revision (id) ON DELETE CASCADE,
fragments JSONB NOT NULL DEFAULT '{}'
);
INSERT INTO config_revision_preview (revision_id, fragments)
SELECT id, COALESCE(meta_json->'preview_fragments', '{}'::jsonb)
FROM config_revision
WHERE meta_json ? 'preview_fragments';
UPDATE config_revision
SET meta_json = meta_json - 'preview_fragments'
WHERE meta_json ? 'preview_fragments';
@@ -0,0 +1,3 @@
ALTER TABLE config_revision DROP COLUMN IF EXISTS prefix_snapshot_id;
DROP TABLE IF EXISTS prefix_snapshot_row;
DROP TABLE IF EXISTS prefix_snapshot;
@@ -0,0 +1,22 @@
-- Content-addressed prefix snapshots (H2 expand).
CREATE TABLE prefix_snapshot (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
content_hash CHAR(64) NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT prefix_snapshot_hash_uniq UNIQUE (content_hash)
);
CREATE TABLE prefix_snapshot_row (
snapshot_id UUID NOT NULL REFERENCES prefix_snapshot (id) ON DELETE CASCADE,
ord INTEGER NOT NULL,
prefix TEXT NOT NULL,
community_id UUID REFERENCES bgp_community (id) ON DELETE SET NULL,
source TEXT NOT NULL DEFAULT '',
PRIMARY KEY (snapshot_id, ord)
);
CREATE INDEX idx_prefix_snapshot_row_snapshot ON prefix_snapshot_row (snapshot_id);
ALTER TABLE config_revision
ADD COLUMN prefix_snapshot_id UUID REFERENCES prefix_snapshot (id) ON DELETE RESTRICT;
@@ -0,0 +1,3 @@
DELETE FROM prefix_snapshot_row;
UPDATE config_revision SET prefix_snapshot_id = NULL WHERE prefix_snapshot_id IS NOT NULL;
DELETE FROM prefix_snapshot;
@@ -0,0 +1,35 @@
-- Backfill prefix snapshots from revision_materialized_prefix.
-- prefix_snapshot_row.prefix is TEXT (revision_materialized_prefix.prefix since 000003).
ALTER TABLE prefix_snapshot_row
ALTER COLUMN prefix TYPE TEXT USING prefix::text;
WITH new_snaps AS (
INSERT INTO prefix_snapshot (id, content_hash)
SELECT gen_random_uuid(),
substr(replace(cr.id::text, '-', '') || replace(cr.id::text, '-', ''), 1, 64)
FROM config_revision cr
WHERE cr.prefix_snapshot_id IS NULL
AND EXISTS (
SELECT 1 FROM revision_materialized_prefix rmp WHERE rmp.revision_id = cr.id
)
RETURNING id, content_hash
)
UPDATE config_revision cr
SET prefix_snapshot_id = ns.id
FROM new_snaps ns
WHERE cr.prefix_snapshot_id IS NULL
AND ns.content_hash = substr(replace(cr.id::text, '-', '') || replace(cr.id::text, '-', ''), 1, 64);
INSERT INTO prefix_snapshot_row (snapshot_id, ord, prefix, community_id, source)
SELECT cr.prefix_snapshot_id,
(row_number() OVER (PARTITION BY cr.id ORDER BY rmp.id) - 1)::int,
rmp.prefix,
rmp.community_id,
rmp.source
FROM config_revision cr
JOIN revision_materialized_prefix rmp ON rmp.revision_id = cr.id
WHERE cr.prefix_snapshot_id IS NOT NULL
AND NOT EXISTS (
SELECT 1 FROM prefix_snapshot_row psr WHERE psr.snapshot_id = cr.prefix_snapshot_id
);
@@ -0,0 +1 @@
COMMENT ON COLUMN config_revision.prefix_snapshot_id IS NULL;
@@ -0,0 +1,3 @@
-- H2 contract marker: new revisions use prefix_snapshot_id only (enforced in application code).
COMMENT ON COLUMN config_revision.prefix_snapshot_id IS 'Materialized prefixes; revision_materialized_prefix deprecated for new rows';
@@ -0,0 +1,16 @@
ALTER TABLE module_prefix_snapshot ADD COLUMN prefixes_json JSONB NOT NULL DEFAULT '[]';
UPDATE module_prefix_snapshot mps
SET prefixes_json = COALESCE((
SELECT jsonb_agg(
jsonb_build_object(
'prefix', psr.prefix::text,
'community_id', psr.community_id,
'source', psr.source
) ORDER BY psr.ord
)
FROM module_prefix_snapshot_row psr
WHERE psr.tenant_id = mps.tenant_id AND psr.module_id = mps.module_id
), '[]'::jsonb);
DROP TABLE IF EXISTS module_prefix_snapshot_row;
@@ -0,0 +1,46 @@
CREATE TABLE module_prefix_snapshot_row (
tenant_id UUID NOT NULL,
module_id UUID NOT NULL,
ord INTEGER NOT NULL,
prefix TEXT NOT NULL,
community_id UUID,
source TEXT NOT NULL DEFAULT '',
PRIMARY KEY (tenant_id, module_id, ord),
FOREIGN KEY (tenant_id, module_id)
REFERENCES module_prefix_snapshot (tenant_id, module_id) ON DELETE CASCADE
);
-- prefixes_json from Go json.Marshal(PrefixRow) used "Prefix"/"CommunityID"/"Source" before json tags.
INSERT INTO module_prefix_snapshot_row (tenant_id, module_id, ord, prefix, community_id, source)
SELECT tenant_id,
module_id,
(row_number() OVER (PARTITION BY tenant_id, module_id ORDER BY ordinality) - 1)::int,
prefix,
NULLIF(community_id, '')::uuid,
COALESCE(source, '')
FROM (
SELECT mps.tenant_id,
mps.module_id,
t.ordinality,
COALESCE(
NULLIF(trim(t.elem->>'prefix'), ''),
NULLIF(trim(t.elem->>'Prefix'), '')
) AS prefix,
COALESCE(
NULLIF(trim(t.elem->>'community_id'), ''),
NULLIF(trim(t.elem->>'CommunityID'), '')
) AS community_id,
COALESCE(
NULLIF(trim(t.elem->>'source'), ''),
NULLIF(trim(t.elem->>'Source'), ''),
''
) AS source
FROM module_prefix_snapshot mps
CROSS JOIN LATERAL jsonb_array_elements(mps.prefixes_json) WITH ORDINALITY AS t(elem, ordinality)
WHERE jsonb_typeof(mps.prefixes_json) = 'array'
AND jsonb_array_length(mps.prefixes_json) > 0
) parsed
WHERE parsed.prefix IS NOT NULL
AND trim(parsed.prefix) <> '';
ALTER TABLE module_prefix_snapshot DROP COLUMN prefixes_json;
@@ -0,0 +1,10 @@
ALTER TABLE asn_prefix_cache ADD COLUMN prefixes_json JSONB NOT NULL DEFAULT '[]';
UPDATE asn_prefix_cache apc
SET prefixes_json = COALESCE((
SELECT jsonb_agg(apcr.prefix::text ORDER BY apcr.prefix::text)
FROM asn_prefix_cache_row apcr
WHERE apcr.asn = apc.asn
), '[]'::jsonb);
DROP TABLE IF EXISTS asn_prefix_cache_row;
@@ -0,0 +1,14 @@
CREATE TABLE asn_prefix_cache_row (
asn BIGINT NOT NULL REFERENCES asn_prefix_cache (asn) ON DELETE CASCADE,
prefix CIDR NOT NULL,
PRIMARY KEY (asn, prefix)
);
INSERT INTO asn_prefix_cache_row (asn, prefix)
SELECT apc.asn, t.elem::cidr
FROM asn_prefix_cache apc
CROSS JOIN LATERAL jsonb_array_elements_text(apc.prefixes_json) AS t(elem)
WHERE jsonb_typeof(apc.prefixes_json) = 'array'
AND jsonb_array_length(apc.prefixes_json) > 0;
ALTER TABLE asn_prefix_cache DROP COLUMN prefixes_json;
@@ -0,0 +1 @@
DROP INDEX IF EXISTS idx_job_audit_created_brin;
@@ -0,0 +1,2 @@
CREATE INDEX IF NOT EXISTS idx_job_audit_created_brin
ON job_audit USING BRIN (created_at);
@@ -0,0 +1,18 @@
CREATE TABLE IF NOT EXISTS module_cdn_fetch_log (
id BIGSERIAL PRIMARY KEY,
module_id UUID NOT NULL REFERENCES module (id) ON DELETE CASCADE,
source_id UUID REFERENCES module_cdn_source (id) ON DELETE SET NULL,
http_status INTEGER,
bytes BIGINT,
error TEXT,
fetched_at TIMESTAMPTZ NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS idx_module_cdn_fetch_log_module ON module_cdn_fetch_log (module_id, fetched_at DESC);
ALTER TABLE revision_materialized_prefix ADD COLUMN IF NOT EXISTS meta_json JSONB NOT NULL DEFAULT '{}';
ALTER TABLE module_domain_entry ADD COLUMN IF NOT EXISTS resolve_meta JSONB NOT NULL DEFAULT '{}';
CREATE INDEX IF NOT EXISTS idx_rev_mat_prefix_revision ON revision_materialized_prefix (revision_id);
CREATE INDEX IF NOT EXISTS idx_module_tenant ON module (tenant_id);
CREATE INDEX IF NOT EXISTS idx_module_prefix_snapshot_collected ON module_prefix_snapshot (collected_at);
CREATE INDEX IF NOT EXISTS idx_rev_mat_prefix_value ON revision_materialized_prefix (prefix);
@@ -0,0 +1,8 @@
ALTER TABLE revision_materialized_prefix DROP COLUMN IF EXISTS meta_json;
ALTER TABLE module_domain_entry DROP COLUMN IF EXISTS resolve_meta;
DROP TABLE IF EXISTS module_cdn_fetch_log;
DROP INDEX IF EXISTS idx_rev_mat_prefix_revision;
DROP INDEX IF EXISTS idx_module_tenant;
DROP INDEX IF EXISTS idx_module_prefix_snapshot_collected;
DROP INDEX IF EXISTS idx_rev_mat_prefix_value;

Some files were not shown because too many files have changed in this diff Show More