Compare commits

...
14 Commits
Author SHA1 Message Date
Denozordec aa4e3d0180 docs(agents): update engineering rules and add Context7 documentation references
CI / changes (push) Successful in 9s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 25s
CI / web (push) Successful in 31s
CI / go (push) Successful in 2m14s
CI / bird2 (push) Successful in 15s
CI / release (push) Successful in 3m53s
Added Context7 documentation links for stack IDs and skills to the agents guide, enhancing clarity on library usage and integration.
2026-06-12 13:48:37 +07:00
DenozordecandCursor 1ccffc85da test(maintenance): add policy executor and handler tests
Табличные тесты PolicyExecutor, ConfigProvider reload, memory CRUD политик и 503 для /v1/maintenance/* на memory-бэкенде без PostgreSQL.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:32:01 +07:00
DenozordecandCursor d38ee68c4e feat(web): add maintenance policies UI
Вкладка политик обслуживания PostgreSQL: CRUD через /v1/maintenance/policies, run/dry-run, форма с Zod. Hardcoded кнопки vacuum/cleanup в MonitoringPostgresTab заменены на MaintenancePoliciesTab.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:30:43 +07:00
DenozordecandCursor aaef47c7a7 feat(observability): add maintenance policy metrics
Prometheus: runs, duration, rows_deleted, config_changes; инкремент при CRUD и Execute.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:28:20 +07:00
DenozordecandCursor cbf345b25f refactor(maintenance): remove hardcoded retention and wire scheduler
RunPeriodicMaintenance и RunCleanup удалены; scheduler политик в StartBackground; deprecated /postgres/cleanup принимает policy_id.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:27:49 +07:00
DenozordecandCursor f548d0671f feat(api): add /v1/maintenance policies and run endpoints
OpenAPI, httpapi CRUD/run/dry-run, job maintenance_policy_run и audit с policy_id.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:23:10 +07:00
DenozordecandCursor 6510a9ca22 feat(maintenance): add policy executor and config provider
ConfigProvider, PolicyExecutor, DBStatsProvider, scheduler и safety; зависимость robfig/cron/v3.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:21:32 +07:00
DenozordecandCursor 07c3de4939 feat(store): add MaintenancePolicy CRUD backend
Типы maintenance_policy, методы store.Backend и реализации для PostgreSQL и in-memory.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:20:27 +07:00
DenozordecandCursor 948dac34fd feat(db): add maintenance_policy migration 000025
Добавлены таблицы maintenance_policy и maintenance_policy_config_audit для postgres и sqlite; в postgres_maintenance_audit — колонка policy_id.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-06-12 13:19:27 +07:00
Denozordec 480756d832 feat(settings): enhance revision retention minutes validation
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 29s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / release (push) Successful in 3m14s
Introduced a preprocessing function to normalize input for the revision retention minutes field, ensuring it handles various input types correctly. Updated the schema to utilize this new validation method, improving data integrity and user experience.
2026-06-01 14:35:51 +07:00
Denozordec 135fb34e00 fix(pgmonitor): standardize field alignment in QueriesResponse struct
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 54s
CI / bird2 (push) Successful in 15s
CI / release (push) Successful in 3m46s
2026-06-01 14:18:03 +07:00
Denozordec 9efa3bbc8a feat(db): enhance PostgreSQL statistics monitoring and error handling
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 30s
CI / go (push) Failing after 24s
CI / bird2 (push) Has been skipped
CI / release (push) Has been skipped
Updated the PostgreSQL monitoring service to improve handling of `pg_stat_statements` availability. Introduced a new method to check if the extension is queryable and updated the response structure to include availability status and hints. Enhanced the documentation to clarify the requirements for enabling `pg_stat_statements`. Adjusted related components to reflect these changes, ensuring better user feedback in the monitoring interface.
2026-06-01 14:15:38 +07:00
Denozordec fad2bd3353 feat(db): implement PostgreSQL monitoring and maintenance features
CI / changes (push) Successful in 9s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 26s
CI / web (push) Successful in 33s
CI / go (push) Successful in 2m11s
CI / bird2 (push) Successful in 16s
CI / release (push) Successful in 3m27s
Added PostgreSQL monitoring and maintenance capabilities to the API, including new endpoints for instance-level metrics, maintenance operations, and job scheduling. Updated the HTTP API to support PostgreSQL monitoring routes and integrated a background scheduler for metrics collection. Enhanced the CLI with database commands for maintenance tasks. Updated documentation to reflect these changes.
2026-06-01 13:43:33 +07:00
Denozordec 930e42b0b0 fix(db): update prefix handling in module_prefix_snapshot_row
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 55s
CI / bird2 (push) Successful in 14s
CI / release (push) Successful in 3m21s
Modified the prefix column type in the module_prefix_snapshot_row table to TEXT, allowing for more flexible input. Adjusted related SQL queries and Go struct tags to ensure compatibility with JSON serialization. Cleaned up migration logic to handle prefix and community_id fields more robustly.
2026-05-25 11:16:03 +07:00
81 changed files with 6842 additions and 399 deletions
+14
View File
@@ -0,0 +1,14 @@
{
"permissions": {
"allow": [
"mcp__codegraph__codegraph_explore",
"mcp__codegraph__codegraph_search",
"mcp__codegraph__codegraph_node",
"mcp__codegraph__codegraph_callers",
"mcp__codegraph__codegraph_callees",
"mcp__codegraph__codegraph_impact",
"mcp__codegraph__codegraph_files",
"mcp__codegraph__codegraph_status"
]
}
}
+16
View File
@@ -0,0 +1,16 @@
# CodeGraph data files
# These are local to each machine and should not be committed
# Database
*.db
*.db-wal
*.db-shm
# Cache
cache/
# Logs
*.log
# Hook markers
.dirty
+6
View File
@@ -0,0 +1,6 @@
{
"pid": 44608,
"version": "0.9.9",
"socketPath": "\\\\.\\pipe\\codegraph-97b92efdcc5351da",
"startedAt": 1781240018712
}
+14
View File
@@ -0,0 +1,14 @@
{
"mcpServers": {
"codegraph": {
"type": "stdio",
"command": "codegraph",
"args": [
"serve",
"--mcp",
"--path",
"C:\\Users\\shats\\Dev\\EvoBGP"
]
}
}
}
@@ -0,0 +1,408 @@
---
name: Технический аудит EvoBGP
overview: "Полный технический аудит EvoBGP для production-сценария (10+ клиентов, нестабильная сеть). Архитектура — hybrid control plane; сильные стороны: stale fallback, CDN/RIPEstat resilience, Ed25519 bundles. Критичные риски: in-process jobs, DoH без retry, misconfiguration demo-seed, отсутствие HA API."
todos:
- id: quick-ops-checklist
content: Применить production-checklist (SEED_DEMO=0, BUNDLE_SEED_HEX, DB/JOB/CONCURRENCY tuning, TLS)
status: pending
- id: fix-doh-retry
content: Добавить DoWithRetry для DoH в internal/pipeline/refresh.go
status: pending
- id: fix-job-meta-race
content: Исправить чтение j.Meta в worker.go через Snapshot() или locked accessor
status: pending
- id: cdn-preview-resilience
content: Перевести CDN preview на upstreamHTTPDo в routes_crud.go
status: pending
- id: partial-cdn-failure
content: "Partial CDN source failure: skip/degrade вместо fail всего модуля"
status: pending
- id: graceful-shutdown
content: Cancel/drain jobs при SIGTERM в cmd/evobgp-api и evobgp-all
status: pending
- id: ha-job-queue
content: "Roadmap: распределённая очередь jobs (PG claim или NATS) для HA API"
status: pending
isProject: false
---
# Технический аудит EvoBGP
## Executive summary
EvoBGP — **hybrid control plane**: один процесс [`evobgp-all`](cmd/evobgp-all/main.go) (monolith) или **reference Compose** с разделёнными воркерами ([`docs/architecture.md`](docs/architecture.md)). Data plane (BIRD + agent) отделён от control plane (API + PostgreSQL + jobs).
**Сильные стороны для нестабильной сети:**
- Stale snapshot fallback по умолчанию (`EVOBGP_STALE_ON_UPSTREAM_ERROR=1`) — [`internal/pipeline/collect_stale.go`](internal/pipeline/collect_stale.go)
- CDN/RIPEstat: retry (3×) + per-host circuit breaker — [`internal/httpclient/httpclient.go`](internal/httpclient/httpclient.go), [`circuit.go`](internal/httpclient/circuit.go)
- ETag conditional GET, ASN TTL-кэш, parallel collect с cap
- Подписанные бандлы Ed25519, verify перед apply
**Главные риски для 10+ клиентов:**
1. `jobs.Registry`**in-memory, только в процессе API** (ARCH-04)
2. DoH — **без retry/breaker** (критично при блокировках провайдеров)
3. Один failed CDN source **без stale cache валит весь модуль**
4. Production misconfiguration: `Bearer dev`, HTTP API, ephemeral bundle key
5. Data race на `Job.Meta` и alias pointers в `store.Memory`
---
## 1. Архитектура
### Стиль
```mermaid
flowchart TB
subgraph hybrid [Hybrid deployment]
All[evobgp_all monolith]
Split[evobgp_api + workers]
end
subgraph cp [Control plane]
API[HTTP API]
Jobs[jobs.Registry in-process]
PG[(PostgreSQL)]
end
subgraph dp [Data plane per speaker]
Agent[evobgp_agent]
BIRD[BIRD2]
NodeCLI[evobgp_node]
end
All --> API
Split --> API
API --> Jobs
API --> PG
NodeCLI --> API
Agent --> API
Agent --> BIRD
```
| Профиль | Стиль | Когда |
|---------|-------|-------|
| `microvps` / `evobgp-all` | Monolith | 1 VPS, shared Registry |
| reference Compose | Microservices-lite | API + scheduler/ingest/render/deploy |
| Remote speakers | Edge agents | Panel→Node dispatch |
### Узкие места (bottlenecks)
| # | Bottleneck | Где | Impact |
|---|------------|-----|--------|
| B1 | **In-process job queue** | [`internal/jobs/job.go:175-177`](internal/jobs/job.go) | HA API невозможен без потери/дублирования jobs; scheduler без `EVOBGP_CONTROL_PLANE_URL` создаёт **отдельный Registry** — [`cmd/evobgp-scheduler/main.go:58-60`](cmd/evobgp-scheduler/main.go) |
| B2 | **Module refresh = sync upstream fan-out** | [`internal/pipeline/collect_parallel.go`](internal/pipeline/collect_parallel.go) | До `EVOBGP_COLLECT_CONCURRENCY` (8 default, max 32) параллельных HTTP; worst case ~45s × retries на источник |
| B3 | **Default job concurrency = 8** | [`internal/jobs/job.go:264-268`](internal/jobs/job.go) | При burst refresh 10+ tenants — очередь растёт, goroutine блокируются на sem |
| B4 | **PostgreSQL pool default** | [`internal/db/open.go:28-38`](internal/db/open.go) | pgx default ~4 conns; при `JOB_MAX=16` + HTTP — contention без `EVOBGP_DB_MAX_CONNS=25` |
| B5 | **Live endpoints fan-out** | [`internal/httpapi/peers_live.go`](internal/httpapi/peers_live.go) | N goroutines × N speakers, 12s timeout каждый |
| B6 | **Broker — заглушка** | [`internal/broker`](internal/broker) | NATS URL логируется, очередь не распределена |
### Масштабируемость
- **Вертикальная:** хорошо до ~10–20 tenants при `evobgp-all` + tuning ([`docs/production-checklist.md`](docs/production-checklist.md))
- **Горизонтальная API:** **не поддерживается** — два `evobgp-api` = два независимых Registry; `job_audit` в PG — audit only, не очередь исполнения
- **Workers (ingest/render/deploy):** координируются через **общую БД**, не через jobs — OK для prefetch/drift
### Отказоустойчивость
| Сценарий | Поведение | Оценка |
|----------|-----------|--------|
| CDN/RIPEstat недоступен | Stale snapshot + circuit breaker | **Хорошо** (если был prior snapshot) |
| DoH недоступен | Fail модуля или stale domain snapshot | **Средне** (нет HTTP retry) |
| API restart mid-job | Job теряется из Registry; audit может быть inconsistent | **Плохо** |
| PG недоступен | API `/v1/ready` → 503 | **OK** |
| Agent unreachable | Deploy job succeed, drift в `evobgp-deploy` | **Частичный fail** (by design) |
**Рекомендация:** для 10+ клиентов — **`evobgp-all` на каждом CP** или один CP + tuning; HA API требует **распределённой очереди** (NATS/Redis + worker pool) — задокументировано как future work.
---
## 2. Анализ кода
### Антипаттерны
| ID | Проблема | Файл | Критичность |
|----|----------|------|-------------|
| A1 | **Concurrent map read/write** — worker читает `j.Meta` без lock, handler пишет через `mergeMeta`/`Snapshot` | [`worker.go:108,263,379`](internal/jobs/worker.go), [`job.go:103-111`](internal/jobs/job.go) | **high** |
| A2 | **Escape internal pointers** из Memory store | [`store/memory.go:416-475`](internal/store/memory.go) | **high** (tests/dev); **low** (prod PG) |
| A3 | **Fire-and-forget goroutine** на каждый auth | [`auth.go:79-81`](internal/httpapi/auth.go) | **medium** |
| A4 | **Silent error swallow** в prefetch | [`internal/ingest/run.go`](internal/ingest/run.go), `prefetch.go` | **medium** |
| A5 | **Bypass resilience layer** — CDN preview прямой `Do` | [`routes_crud.go:267`](internal/httpapi/routes_crud.go) | **medium** |
| A6 | **`EVOBGP_DEV_INSECURE` — dead code** | compose + [`server.go`](internal/httpapi/server.go) | **low** (misleading ops) |
| A7 | **Unused Registry** в ingest/render/deploy binaries | [`cmd/evobgp-ingest/main.go`](cmd/evobgp-ingest/main.go) | **low** (resource waste) |
### Maintainability
**Плюсы:** чёткое разделение слоёв (ARCH-01..10), `store.Backend`, OpenAPI как контракт, engineering rules, table-driven tests в birdfmt/pipeline.
**Минусы:**
- Дублирование retry-логики (httpclient vs nodedispatch inline loop)
- Env-tuning разбросан (`EVOBGP_*` в 15+ местах без central config struct для pipeline)
- `Job` comment «персистенция в БД пока не подключена» устарел — hooks есть в [`bootstrap.go:67-92`](internal/httpapi/bootstrap.go)
### Потенциальные баги и race conditions
1. **`j.Meta` data race** — `-race` на `TestParallelModuleRefresh_*` + concurrent `GET /v1/jobs/{id}` polling
2. **Memory store alias**`deploy.Run` читает `LastAppliedRevisionID` пока worker пишет
3. **peerLiveCache** возвращает slice без копии — [`peers_live.go:82-84`](internal/httpapi/peers_live.go)
4. **TOCTOU idempotency** — terminal job удаляется из `byIdempo`, повторный POST создаст новый job (by design, но клиент должен знать)
### Error handling
**Хорошо:**
- Префиксы ошибок (`httpclient:`, `birdfmt:`)
- HTTP 5xx через `writeProblem`, без raw `err.Error()` (ERR-01)
- `context.Context` в pipeline workers
**Пробелы:**
- `runRollback` без `workContext` — не отменяется — [`worker.go:500+`](internal/jobs/worker.go)
- Prefetch/ingest: ошибки не логируются
- `mergeBirdPostApplyMeta``context.Background()` 8s, игнорирует job cancel
---
## 3. Производительность
### Блокирующие операции
| Участок | Блокировка | Риск |
|---------|------------|------|
| `POST .../cdn-sources/preview` | Sync CDN fetch до 45s в HTTP handler | UI timeout, worker starvation |
| `GET /v1/peers/live` | N × agent HTTP, wg.Wait | Slow при многих speakers |
| Module refresh job | Sequential: ingest → render revision → optional deploy | Long job chain |
| `bird -p` / `birdc configure` | Subprocess в deploy | Disk I/O на ноде |
### Неэффективные алгоритмы / лишние запросы
- **Tenant refresh:** `aggregateTenantPrefixRowsAll` — parallel по модулям, но каждый модуль может refetch все CDN/ASN/DoH — [`aggregate.go:28+`](internal/pipeline/aggregate.go). Snapshot skip есть через `module_hash` — проверять hit rate в meta.
- **ASN resolve:** `PolitePause()` 150ms между AS — [`asnresolve/ripestat.go`](internal/asnresolve/ripestat.go) — при 50 AS = +7.5s minimum.
- **GetModulePrefixSnapshot** вызывается многократно в одном refresh (cdn_snapshot, collect_parallel) — potential duplicate DB reads.
- **Auth TouchAPIKeyLastUsed:** UPDATE на каждый request (async) — load на PG при high RPS.
### Кэширование
| Кэш | TTL | Gap |
|-----|-----|-----|
| ASN prefix cache | 1800s (`EVOBGP_ASN_CACHE_TTL_SEC`) | OK |
| CDN ETag in DB | Until 304/change | OK |
| Module prefix snapshot | Content-hash based skip | OK |
| peerLiveCache | In-memory, per-process | Не shared между API replicas; нет defensive copy |
| Circuit breaker state | Per-process | Не shared |
### Конкретные улучшения
```go
// 1. CDN preview — использовать upstreamHTTPDo вместо прямого Do
resp, err := pipeline.UpstreamHTTPDo(r.Context(), s.cdnHTTP, req) // extract upstreamHTTPDo
// 2. Job.Meta — читать под lock или через Snapshot()
st := j.Snapshot()
mid, _ := st["meta"].(map[string]any)["module_id"].(string)
// 3. Memory store — возвращать копии (как Postgres)
modCopy := *mod
return &modCopy, nil
```
---
## 4. Сетевое взаимодействие (критично)
### Текущее состояние
```mermaid
flowchart LR
subgraph resilient [Resilient path]
CDN[CDN fetch]
RIPE[RIPEstat]
CDN --> Breaker[Circuit breaker]
RIPE --> Breaker
Breaker --> Retry[DoWithRetry 3x linear 2s]
end
subgraph fragile [Fragile path]
DoH[DoH resolve]
Preview[CDN preview API]
AgentHealth[Agent health/bird]
DoH --> SingleDo[Single hc.Do]
Preview --> SingleDo
AgentHealth --> SingleDo
end
subgraph fallback [App-level fallback]
Stale[Stale snapshot]
SysDNS[System DNS]
DoH --> SysDNS
CDN --> Stale
RIPE --> Stale
end
```
| Upstream | Timeout | Retry | Breaker | Stale fallback |
|----------|---------|-------|---------|----------------|
| CDN ingest | 45s | 3× linear | per-host | yes |
| RIPEstat | 45s | 3× | per-host | yes + cache |
| DoH | 10s/profile | **no** | **no** | domain snapshot |
| CDN preview | 45s | **no** | **no** | N/A |
| Scheduler→API | 45s | 3× | no | N/A |
| Node dispatch | 30s | inline 3× | no | N/A |
### Пробелы для блокировок провайдеров
1. **DoH без retry** — transient timeout = fail; failover между profiles есть, но каждый profile — single shot
2. **429/408 не ретраятся** — только `>= 500`
3. **Нет jitter** — thundering herd при mass tenant refresh
4. **DNS rebinding TOCTOU** — SSRF check до fetch, HTTP dial без pinned IP — [`cdn_url.go:75-115`](internal/pipeline/cdn_url.go)
5. **Circuit breaker без half-open** — после 30s cooldown сразу full traffic — [`circuit.go:29-33`](internal/httpclient/circuit.go)
6. **Breaker per-process** — ingest container ≠ API container
### Рекомендации для нестабильной сети
| # | Изменение | Effort | Effect |
|---|-----------|--------|--------|
| N1 | DoH через `DoWithRetry` + optional breaker | Low | **High** для DOMAINS modules |
| N2 | Retry 429/503 с `Retry-After` + exponential backoff + jitter | Medium | **High** при rate limits |
| N3 | **Partial CDN failure** — continue с stale per-source, не fail whole module | Medium | **High** |
| N4 | Multiple DoH profiles + `failover` policy (already exists) — **документировать ops playbook** | Low | **High** (config, not code) |
| N5 | Pinned dialer / custom `Transport.DialContext` после SSRF resolve | Medium | **Medium** (SSRF hardening) |
| N6 | Proxy support (`HTTP_PROXY` / `EVOBGP_HTTP_PROXY`) для CDN/DoH | Medium | **High** в censored networks |
| N7 | Unify CDN preview на `upstreamHTTPDo` | Low | **Medium** |
---
## 5. Устойчивость и надёжность
### Graceful degradation
**Работает:**
- `EVOBGP_STALE_ON_UPSTREAM_ERROR=1` — ASN/CDN/domain stale — [`collect_stale.go`](internal/pipeline/collect_stale.go)
- CDN 304 без local cache → forced full GET — [`cdn_snapshot.go:141-159`](internal/pipeline/cdn_snapshot.go)
- DoH → system DNS fallback — [`doh_resolve.go:75-93`](internal/pipeline/doh_resolve.go)
- Deploy: job succeed even if agent wake fails (drift detection)
**Не работает / частично:**
- Один CDN source fail без cache → **весь module_refresh failed** — [`collect_parallel.go:221-223`](internal/pipeline/collect_parallel.go)
- Circuit open → immediate error, stale only if prior data exists
- API shutdown: HTTP drain 15s, **jobs не cancel/drain** — [`cmd/evobgp-api/main.go:67-72`](cmd/evobgp-api/main.go)
### Сценарии отказов
| Событие | Что произойдёт |
|---------|----------------|
| **Потеря CP↔PG** | Ready=false; running jobs fail; no new jobs persist audit reliably |
| **Потеря CP↔CDN** | Stale prefixes если были; иначе job fail; breaker opens 30s |
| **Потеря CP↔agent** | Deploy meta `dispatch_failed`; BIRD на старой ревизии; drift logs |
| **RIPEstat rate limit** | 429 → no retry → stale or fail |
| **Рост нагрузки** | Job queue; goroutine pile-up; PG pool exhaustion; `/metrics` shows queue depth |
| **API restart** | In-flight jobs lost; clients poll 404 or stale terminal state |
---
## 6. Безопасность
| ID | Finding | Severity | Fix |
|----|---------|----------|-----|
| S1 | `Bearer dev` → operator при demo-seed | **high** (misconfig) | `EVOBGP_SEED_DEMO=0` — [`auth.go:66-92`](internal/httpapi/auth.go) |
| S2 | API plain HTTP | **high** (ops) | TLS на edge (Traefik/nginx) |
| S3 | Ephemeral bundle key без `EVOBGP_BUNDLE_SEED_HEX` | **high** (ops) | Stable seed + pubkey на нодах |
| S4 | Compose defaults: weak PG password, `sslmode=disable` | **high** (ops) | Secrets manager, `sslmode=require` |
| S5 | `/metrics` без auth | **medium** | Network policy / mTLS |
| S6 | No rate limiting on auth | **medium** | Middleware limiter (e.g. per-IP) |
| S7 | CDN SSRF DNS rebinding | **medium** | Pinned dialer after resolve |
| S8 | `EVOBGP_CDN_ALLOW_PRIVATE=1` | **medium** | Never in prod |
| S9 | `EVOBGP_NODE_DISPATCH_INSECURE_TLS=1` | **medium** | Valid TLS to agent |
| S10 | Plaintext `EVOBGP_API_KEYS` in env | **medium** | DB keys via API |
| S11 | `editor` can cancel jobs | **low** | Restrict to operator |
| S12 | agent_secret `==` compare | **low** | `subtle.ConstantTimeCompare` |
**SQL injection:** не обнаружено — параметризованные запросы в [`repository/`](internal/repository/).
**Bundle crypto:** Ed25519 корректно; path traversal blocked в tar extract.
---
## 7. Конкретные рекомендации (prioritized backlog)
### High
| # | Описание | Как исправить |
|---|----------|---------------|
| H1 | DoH без retry | Обернуть `hc.Do` в `DoWithRetry(ctx, hc, req, 3)` в [`refresh.go:288,360`](internal/pipeline/refresh.go) |
| H2 | Data race `Job.Meta` | Читать через `Snapshot()` или добавить `MetaLocked()` accessor |
| H3 | CDN source partial failure | В `collectCDNPrefixRows`: при err без stale — log warning + skip source вместо `return nil, r.err` (config flag `EVOBGP_CDN_PARTIAL_OK=1`) |
| H4 | Production checklist enforcement | CI/deploy validation: reject `SEED_DEMO=1`, require `BUNDLE_SEED_HEX` |
| H5 | Job queue HA roadmap | Persist queued jobs in PG + worker claim (`SELECT FOR UPDATE SKIP LOCKED`) или NATS — ARCH-04 |
### Medium
| # | Описание | Как исправить |
|---|----------|---------------|
| M1 | CDN preview bypass | [`routes_crud.go:267`](internal/httpapi/routes_crud.go) → `upstreamHTTPDo` |
| M2 | Retry 429/503 | Extend `DoWithRetry` status check + parse `Retry-After` |
| M3 | Graceful shutdown | On SIGTERM: `Registry.RequestCancelAll()` + wait workers with timeout |
| M4 | Auth goroutine storm | Worker pool или sync touch with debounce |
| M5 | HTTP proxy support | Custom Transport reading `EVOBGP_HTTP_PROXY` |
| M6 | Memory store copies | Defensive copy in Get/List (dev/test safety) |
| M7 | Rate limiting | `golang.org/x/time/rate` on auth middleware |
### Low
| # | Описание | Как исправить |
|---|----------|---------------|
| L1 | Jitter in backoff | `wait + rand.Intn(wait/2)` in DoWithRetry |
| L2 | Half-open breaker | Single probe request after cooldown |
| L3 | Remove dead `EVOBGP_DEV_INSECURE` from compose | Docs + compose cleanup |
| L4 | Prefetch error logging | `log.Printf` or structured log in prefetch |
| L5 | peerLiveCache defensive copy | `append([]T(nil), views...)` on store |
---
## 8. Quick wins (максимальный эффект / минимум усилий)
1. **Ops (0 code):** [`docs/production-checklist.md`](docs/production-checklist.md) — `SEED_DEMO=0`, `BUNDLE_SEED_HEX`, `DB_MAX_CONNS=25`, `JOB_MAX=16`, `COLLECT_CONCURRENCY=16`, TLS edge, restrict metrics
2. **DoH retry** — 510 строк в `refresh.go`, reuse existing `DoWithRetry`
3. **CDN preview → upstreamHTTPDo** — 1 line change in handler
4. **Job.Meta read fix** — replace 4 reads in `worker.go` with `Snapshot()` parsing
5. **Log prefetch failures** — visibility без изменения behavior
6. **Document DoH failover playbook** — multiple profiles (Cloudflare, Google, Quad9) + `failover` policy for censored regions
7. **Run `go test -race ./internal/jobs/...`** in CI — catch Meta race
8. **Prefer `evobgp-all`** over split reference for &lt;20 tenants — eliminates Registry split bug
---
## Диаграмма: refresh под сетевым stress
```mermaid
sequenceDiagram
participant Op as Operator
participant API as evobgp_api
participant Job as module_refresh
participant CDN as CDN_upstream
participant PG as PostgreSQL
Op->>API: POST /modules/id/refresh
API->>Job: Enqueue
Job->>CDN: GET with ETag
alt CDN timeout or 5xx
CDN-->>Job: error after 3 retries
Job->>PG: load prior snapshot
alt stale exists
Job->>PG: CreateRenderRevision stale
Job-->>API: succeeded degraded
else no stale
Job-->>API: failed
end
else CDN 200
CDN-->>Job: new prefixes
Job->>PG: CreateRenderRevision
end
```
---
## Итоговая оценка зрелости
| Область | Оценка | Комментарий |
|---------|--------|-------------|
| Архитектура | 7/10 | Чистые слои; HA/API scaling — слабое место |
| Сеть/resilience | 6/10 | CDN/ASN хорошо; DoH/preview — пробелы |
| Concurrency | 6/10 | Registry продуман; Meta race, shutdown |
| Performance | 7/10 | Parallel collect, caching; tuning needed at scale |
| Security | 6/10 | Crypto OK; ops/config risks dominate |
| Maintainability | 8/10 | Docs, rules, OpenAPI, tests |
**Вердикт:** проект **готов для 10+ клиентов в single-CP deployment** (`evobgp-all` + PostgreSQL + production checklist) при условии ops discipline. Для **multi-CP HA** и **агрессивных сетевых блокировок** — приоритет: DoH retry, partial CDN failure, distributed job queue, HTTP proxy.
+93
View File
@@ -0,0 +1,93 @@
---
description: Context7 — закреплённые ID библиотек и документации стека EvoBGP
alwaysApply: true
---
# Context7 — стек EvoBGP
При вопросах об API, синтаксисе, конфигурации и миграциях библиотек **сначала** `query-docs` с ID из таблицы ниже. Шаг `resolve-library-id` **пропускать**, если библиотека уже перечислена (кроме неоднозначного случая).
Локальные версии: `go.mod`, `web/package.json`. При расхождении с ID — предпочитать версию из репозитория.
---
## Backend (Go)
| Библиотека | Context7 ID | Версия в проекте | Когда |
|------------|-------------|------------------|-------|
| Go stdlib | `/golang/go/go1_24_6` | Go 1.24 | `net/http`, `context`, тесты, concurrency |
| pgx | `/websites/pkg_go_dev_github_com_jackc_pgx_v5` | v5.7.2 | PostgreSQL, pool, транзакции, типы |
| Prometheus Go client | `/prometheus/client_golang` | v1.20.5 | метрики, `/metrics`, middleware |
| modernc SQLite | `/websites/pkg_go_dev_modernc_org_sqlite` | v1.34.5 | SQLite-бэкенд, миграции sqlite |
| miekg/dns | `/miekg/dns` | v1.1.72 | DNS-запросы, DoH, pipeline |
---
## HTTP-контракт и спецификации
| Библиотека | Context7 ID | Версия в проекте | Когда |
|------------|-------------|------------------|-------|
| OpenAPI | `/oai/openapi-specification` | 3.x в `docs/openapi.yaml` | схемы, operationId, problem+json |
| Redocly CLI | `/redocly/redocly-cli` | CI `@redocly/cli` | lint OpenAPI, `npx @redocly/cli lint` |
---
## Web UI (`web/`)
| Библиотека | Context7 ID | Версия в проекте | Когда |
|------------|-------------|------------------|-------|
| Svelte | `/websites/svelte_dev` | ^5.54 | runes, компоненты, реактивность |
| SvelteKit | `/sveltejs/kit` | ^2.50 | routing, `load`, adapters, SSR |
| Vite | `/vitejs/vite/v7.3.1` | ^7.3.1 | dev server, build, plugins |
| TypeScript | `/microsoft/typescript/v5.9.3` | ^5.9.3 | типы, strict, tsconfig |
| Tailwind CSS | `/tailwindlabs/tailwindcss.com` | ^4.1 | v4, `@tailwindcss/vite`, утилиты |
| shadcn-svelte | `/websites/shadcn-svelte` | CLI | примитивы `ui/core`, theming |
| Bits UI | `/llmstxt/bits-ui_llms_txt` | ^2.17 | headless-примитивы под shadcn |
| sveltekit-superforms | `/ciscoheat/sveltekit-superforms` | ^2.30 | формы, server actions |
| Formsnap | `/svecosystem/formsnap` | ^2.0 | доступные поля форм |
| Zod | `/websites/zod_dev_v4` | ^4.4 | схемы валидации |
| TanStack Table | `/websites/tanstack_table` | table-core ^8.21 | `AppDataTable`, колонки, сортировка |
UI-правила репозитория: `.cursor/rules/web-shadcn.mdc` (shadcn-svelte docs — первичный источник для компонентов).
---
## Data plane / BGP
| Библиотека | Context7 ID | Версия в проекте | Когда |
|------------|-------------|------------------|-------|
| BIRD 2 | `/llmstxt/bird_xmsl_dev_llms_txt` | BIRD2 в compose | `birdfmt`, фильтры, протоколы |
| BIRD (исходники) | `/cz-nic/bird` | — | низкоуровневый синтаксис daemon |
Сетевые правила: `.cursor/rules/networking-bird.mdc`.
---
## DevOps
| Библиотека | Context7 ID | Версия в проекте | Когда |
|------------|-------------|------------------|-------|
| Docker Compose | `/docker/compose` | `deploy/compose/` | сервисы, profiles, volumes |
| Docker | `/docker/docs` | — | образы, bake, networking |
---
## Приоритет источников
1. **Контракт HTTP** — `docs/openapi.yaml` (не Context7).
2. **Context7** — синтаксис и API библиотек из таблицы.
3. **Локальные docs** — `docs/`, `web/README.md`, `AGENTS.md`.
4. **Официальный сайт** — BIRD: https://bird.network.cz/?get_doc (если Context7 не покрыл кейс).
## Примеры запросов
```
/docs /websites/svelte_dev runes $state $derived
/docs /golang/go/go1_24_6 net/http ServeMux pattern matching
/docs /websites/pkg_go_dev_github_com_jackc_pgx_v5 pool acquire rows
/docs /llmstxt/bird_xmsl_dev_llms_txt filter bgp import
```
## Не через Context7
Рефакторинг `internal/*`, бизнес-логика EvoBGP, code review — код репозитория и `codegraph`. Context7 — только внешние библиотеки и инструменты.
+3
View File
@@ -2,6 +2,9 @@
"plugins": {
"svelte": {
"enabled": true
},
"claude-plugins-official/gopls-lsp": {
"enabled": true
}
}
}
+31
View File
@@ -0,0 +1,31 @@
---
name: context7-evobgp
description: Context7 lookup для стека EvoBGP — использовать закреплённые library ID из .cursor/rules/context7-stack.mdc вместо resolve-library-id.
---
# Context7 — EvoBGP stack
Перед `query-docs` открой `.cursor/rules/context7-stack.mdc` и выбери ID из таблицы по области задачи.
## Workflow
1. Определи область: `internal/` (Go), `web/` (Svelte), `docs/openapi.yaml`, `birdfmt`/`pipeline` (BIRD), `deploy/compose` (Docker).
2. Найди строку в таблице `context7-stack.mdc`.
3. Вызови `query-docs` с `libraryId` из таблицы и полным вопросом пользователя.
4. `resolve-library-id` — только если библиотеки нет в таблице или нужна другая major-версия.
## Быстрые ID (частые)
| Задача | libraryId |
|--------|-----------|
| Svelte 5 runes | `/websites/svelte_dev` |
| SvelteKit load/forms | `/sveltejs/kit` |
| shadcn-svelte компонент | `/websites/shadcn-svelte` |
| pgx pool/query | `/websites/pkg_go_dev_github_com_jackc_pgx_v5` |
| Go net/http | `/golang/go/go1_24_6` |
| OpenAPI lint | `/redocly/redocly-cli` |
| BIRD config | `/llmstxt/bird_xmsl_dev_llms_txt` |
| Tailwind v4 | `/tailwindlabs/tailwindcss.com` |
| Zod 4 schema | `/websites/zod_dev_v4` |
Полный список и версии — в `context7-stack.mdc`.
+12
View File
@@ -0,0 +1,12 @@
{
"mcpServers": {
"codegraph": {
"type": "stdio",
"command": "codegraph",
"args": [
"serve",
"--mcp"
]
}
}
}
+12
View File
@@ -0,0 +1,12 @@
{
"mcpServers": {
"codegraph": {
"type": "stdio",
"command": "codegraph",
"args": [
"serve",
"--mcp"
]
}
}
}
+1 -1
View File
@@ -4,7 +4,7 @@
## С чего начать (минимум чтения)
0. **Инженерные правила** — при изменении кода следовать [.cursor/rules/engineering.mdc](.cursor/rules/engineering.mdc); для `web/` — [.cursor/rules/web-shadcn.mdc](.cursor/rules/web-shadcn.mdc); для `birdfmt` / `pipeline` / BIRD — [.cursor/rules/networking-bird.mdc](.cursor/rules/networking-bird.mdc).
0. **Инженерные правила** — при изменении кода следовать [.cursor/rules/engineering.mdc](.cursor/rules/engineering.mdc); для `web/` — [.cursor/rules/web-shadcn.mdc](.cursor/rules/web-shadcn.mdc); для `birdfmt` / `pipeline` / BIRD — [.cursor/rules/networking-bird.mdc](.cursor/rules/networking-bird.mdc). **Context7 (документация библиотек)** — закреплённые ID стека: [.cursor/rules/context7-stack.mdc](.cursor/rules/context7-stack.mdc); скилл [.cursor/skills/context7-evobgp/SKILL.md](.cursor/skills/context7-evobgp/SKILL.md).
1. **[docs/README.md](docs/README.md)** — оглавление и роли читателя.
2. **[docs/architecture.md](docs/architecture.md)** — компоненты `cmd/`, карта `internal/`, потоки данных (одного этого файла обычно достаточно для ориентации).
3. Задача-специфично: [docs/api.md](docs/api.md), [docs/access.md](docs/access.md), [web/README.md](web/README.md) — только если меняете API, доступ или фронт.
+5
View File
@@ -12,6 +12,7 @@ import (
"evobgp/internal/birdfmt"
"evobgp/internal/config"
"evobgp/internal/dbcli"
"evobgp/internal/deploy"
"evobgp/internal/httpapi"
"evobgp/internal/ingest"
@@ -24,6 +25,9 @@ import (
// microVPS entrypoint: один процесс — HTTP API и фоновые воркеры scheduler, ingest, render, deploy (общий store и jobs.Registry).
func main() {
if len(os.Args) > 1 && os.Args[1] == "db" {
os.Exit(dbcli.Run(os.Args[2:]))
}
cfg := config.Load()
opts := httpapi.Options{
APIKeys: os.Getenv("EVOBGP_API_KEYS"),
@@ -52,6 +56,7 @@ func main() {
go render.Run(ctx, renderDeps)
go deploy.Run(ctx, deployDeps)
srv.StartBackground(ctx)
startBirdMetricsPoller(ctx)
httpSrv := &http.Server{
+5
View File
@@ -12,6 +12,7 @@ import (
"evobgp/internal/birdfmt"
"evobgp/internal/config"
"evobgp/internal/dbcli"
"evobgp/internal/httpapi"
"evobgp/internal/observability"
"evobgp/internal/platform"
@@ -19,6 +20,9 @@ import (
)
func main() {
if len(os.Args) > 1 && os.Args[1] == "db" {
os.Exit(dbcli.Run(os.Args[2:]))
}
cfg := config.Load()
seedDemo := os.Getenv("EVOBGP_SEED_DEMO") != "0"
opts := httpapi.Options{
@@ -39,6 +43,7 @@ func main() {
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop()
srv.StartBackground(ctx)
startBirdMetricsPoller(ctx)
httpSrv := &http.Server{
+12
View File
@@ -52,6 +52,18 @@ opkey|01ARZ3NDEKTSV4RRFFQ69G5FAV|operator,nodekey|01ARZ3NDEKTSV4RRFFQ69G5FAV|nod
**Запрещено** в продакшене: не оставляйте demo-seed с известным токеном `dev` на боевых данных. Переменная `EVOBGP_DEV_INSECURE` в текущей версии **не влияет** на аутентификацию (оставлена в compose для совместимости; не включайте в production — см. SEC-02 в инженерных правилах).
### PostgreSQL monitoring и maintenance (control plane)
При `EVOBGP_DATABASE_URL` (не memory backend):
| Операция | Минимальная роль |
|----------|------------------|
| `GET /v1/monitoring/postgres/*`, `GET /v1/monitoring/correlation` | viewer |
| `POST /v1/postgres/vacuum`, `vacuum-analyze`, `analyze`, `reindex`, `cleanup` | **operator** (async job, rate limit 60s на kind) |
| `GET /v1/postgres/maintenance/logs` | viewer |
Метрики **instance-level** (не per-tenant). CLI: `evobgp-api db …` / `evobgp-all db …`.
### Синхронные «тяжёлые» GET (control plane)
- `POST /v1/modules/{module_id}/cdn-sources/preview` — загрузка CDN в том же HTTP-запросе (лимит тела ~8 MiB, см. OpenAPI).
+22
View File
@@ -8,6 +8,28 @@ Runbook для оценки объёма БД и узких мест **пере
psql "$EVOBGP_DATABASE_URL"
```
## HTTP API (панель / мониторинг)
При подключённом PostgreSQL control plane отдаёт instance-level метрики (роль **viewer+**):
- `GET /v1/monitoring/postgres/overview` — подключения, TPS, cache hit, размер БД
- `GET /v1/monitoring/postgres/queries` — top queries (`pg_stat_statements`, если extension включён)
- `GET /v1/monitoring/postgres/locks`, `/tables`, `/recommendations`
- `GET /v1/monitoring/correlation?window=60` — корреляция refresh jobs и cache hit
Обслуживание (**operator**, async `202` + `job_id`): `POST /v1/postgres/vacuum`, `vacuum-analyze`, `analyze`, `reindex`, `cleanup`; журнал `GET /v1/postgres/maintenance/logs`.
CLI на CP: `evobgp-api db report|vacuum|analyze|cleanup` (см. `internal/dbcli`).
Миграция `000023` создаёт `pg_stat_statements`; для сбора статистики **обязательно** preload и перезапуск Postgres:
```text
# postgresql.conf или command в compose
shared_preload_libraries = 'pg_stat_statements'
```
После изменения — restart контейнера/сервиса Postgres. Без этого API `/v1/monitoring/postgres/queries` вернёт пустой список (`statements_available: false`), без 5xx.
## 1. Размеры таблиц и индексов
```sql
+626
View File
@@ -49,6 +49,10 @@ tags:
description: Управление API-ключами tenant (operator). Секрет возвращается только при создании и ротации.
- name: Auth
description: Сессия текущего API-ключа (tenant и роль).
- name: Monitoring
description: Наблюдаемость PostgreSQL и корреляция (instance-level, viewer+). Maintenance — operator.
- name: Maintenance
description: Политики обслуживания PostgreSQL (instance-scoped). CRUD и запуск — operator.
security:
- bearerAuth: []
@@ -910,6 +914,152 @@ components:
type: string
additionalProperties: true
PostgresOverview:
type: object
description: Instance-level PostgreSQL snapshot (GET /v1/monitoring/postgres/overview).
additionalProperties: true
PostgresQueriesResponse:
type: object
properties:
collected_at:
type: string
format: date-time
source:
type: string
enum: [live, snapshot]
items:
type: array
items:
type: object
additionalProperties: true
PostgresRecommendations:
type: object
properties:
collected_at:
type: string
format: date-time
items:
type: array
items:
type: object
properties:
severity:
type: string
code:
type: string
title:
type: string
detail:
type: string
refs:
type: array
items:
type: string
PostgresMaintenanceBody:
type: object
properties:
table:
type: string
dry_run:
type: boolean
default: false
policy:
type: string
description: Deprecated; use maintenance policies API.
limit:
type: integer
MaintenancePolicy:
type: object
required: [name, table_name, schedule, vacuum_strategy]
properties:
id:
$ref: "#/components/schemas/ResourceId"
name:
type: string
table_name:
type: string
condition:
type: string
default: "true"
retention_period_sec:
type: integer
minimum: 1
max_rows:
type: integer
minimum: 1
maximum: 100000
vacuum_strategy:
type: string
enum: [none, vacuum, analyze, vacuum_analyze, reindex]
schedule:
type: string
description: Cron expression (5-field, UTC).
enabled:
type: boolean
default: true
dry_run_enabled:
type: boolean
default: false
last_run_at:
type: string
format: date-time
last_status:
type: string
last_error:
type: string
created_at:
type: string
format: date-time
updated_at:
type: string
format: date-time
MaintenancePolicyPatch:
type: object
properties:
name:
type: string
table_name:
type: string
condition:
type: string
retention_period_sec:
type: integer
max_rows:
type: integer
vacuum_strategy:
type: string
enum: [none, vacuum, analyze, vacuum_analyze, reindex]
schedule:
type: string
enabled:
type: boolean
dry_run_enabled:
type: boolean
MaintenanceRunBody:
type: object
required: [policy_id]
properties:
policy_id:
$ref: "#/components/schemas/ResourceId"
MaintenancePolicyList:
type: object
properties:
items:
type: array
items:
$ref: "#/components/schemas/MaintenancePolicy"
next_cursor:
type: string
has_more:
type: boolean
BirdLocalStatus:
type: object
description: Статус локального BIRD на хосте API (GET /v1/bird/status).
@@ -3135,6 +3285,482 @@ paths:
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/overview:
get:
tags: [Monitoring]
summary: PostgreSQL overview (instance-level)
operationId: getPostgresOverview
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresOverview"
"503":
description: PostgreSQL backend не подключён.
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/queries:
get:
tags: [Monitoring]
summary: Top queries (pg_stat_statements or snapshot)
operationId: getPostgresQueries
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresQueriesResponse"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/locks:
get:
tags: [Monitoring]
summary: Active locks
operationId: getPostgresLocks
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
properties:
items:
type: array
items:
type: object
additionalProperties: true
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/tables:
get:
tags: [Monitoring]
summary: Table sizes and scan stats
operationId: getPostgresTables
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
properties:
items:
type: array
items:
type: object
additionalProperties: true
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/postgres/recommendations:
get:
tags: [Monitoring]
summary: Heuristic optimization recommendations
operationId: getPostgresRecommendations
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresRecommendations"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/monitoring/correlation:
get:
tags: [Monitoring]
summary: Timeline correlation (jobs vs cache hit)
operationId: getMonitoringCorrelation
parameters:
- $ref: "#/components/parameters/TenantId"
- name: window
in: query
schema:
type: integer
default: 60
description: Window in minutes (max 1440).
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
additionalProperties: true
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/vacuum:
post:
tags: [Monitoring]
summary: VACUUM (async job, operator)
operationId: postPostgresVacuum
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
"403":
$ref: "#/components/responses/Forbidden"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/vacuum-analyze:
post:
tags: [Monitoring]
summary: VACUUM ANALYZE (async job, operator)
operationId: postPostgresVacuumAnalyze
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/analyze:
post:
tags: [Monitoring]
summary: ANALYZE (async job, operator)
operationId: postPostgresAnalyze
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/reindex:
post:
tags: [Monitoring]
summary: REINDEX TABLE (async job, operator)
operationId: postPostgresReindex
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/cleanup:
post:
tags: [Monitoring]
summary: Retention cleanup (async job, operator)
operationId: postPostgresCleanup
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/PostgresMaintenanceBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/postgres/maintenance/logs:
get:
tags: [Monitoring]
summary: Maintenance audit log
operationId: listPostgresMaintenanceLogs
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Cursor"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
properties:
items:
type: array
items:
type: object
additionalProperties: true
next_cursor:
type: string
has_more:
type: boolean
default:
$ref: "#/components/responses/DefaultProblem"
/v1/maintenance/policies:
get:
tags: [Maintenance]
summary: List maintenance policies
operationId: listMaintenancePolicies
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Cursor"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenancePolicyList"
default:
$ref: "#/components/responses/DefaultProblem"
post:
tags: [Maintenance]
summary: Create maintenance policy
operationId: createMaintenancePolicy
parameters:
- $ref: "#/components/parameters/TenantId"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenancePolicy"
responses:
"201":
description: Создано.
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenancePolicy"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/maintenance/policies/{id}:
get:
tags: [Maintenance]
summary: Get maintenance policy
operationId: getMaintenancePolicy
parameters:
- $ref: "#/components/parameters/TenantId"
- name: id
in: path
required: true
schema:
$ref: "#/components/schemas/ResourceId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenancePolicy"
default:
$ref: "#/components/responses/DefaultProblem"
patch:
tags: [Maintenance]
summary: Update maintenance policy
operationId: patchMaintenancePolicy
parameters:
- $ref: "#/components/parameters/TenantId"
- name: id
in: path
required: true
schema:
$ref: "#/components/schemas/ResourceId"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenancePolicyPatch"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenancePolicy"
default:
$ref: "#/components/responses/DefaultProblem"
delete:
tags: [Maintenance]
summary: Delete maintenance policy
operationId: deleteMaintenancePolicy
parameters:
- $ref: "#/components/parameters/TenantId"
- name: id
in: path
required: true
schema:
$ref: "#/components/schemas/ResourceId"
responses:
"204":
description: Удалено.
default:
$ref: "#/components/responses/DefaultProblem"
/v1/maintenance/policies/{id}/hints:
get:
tags: [Maintenance]
summary: PostgreSQL hints for policy table
operationId: getMaintenancePolicyHints
parameters:
- $ref: "#/components/parameters/TenantId"
- name: id
in: path
required: true
schema:
$ref: "#/components/schemas/ResourceId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
additionalProperties: true
default:
$ref: "#/components/responses/DefaultProblem"
/v1/maintenance/config-audit:
get:
tags: [Maintenance]
summary: Maintenance policy configuration audit log
operationId: listMaintenanceConfigAudit
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Cursor"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
properties:
items:
type: array
items:
type: object
additionalProperties: true
next_cursor:
type: string
has_more:
type: boolean
default:
$ref: "#/components/responses/DefaultProblem"
/v1/maintenance/run:
post:
tags: [Maintenance]
summary: Run maintenance policy (async job)
operationId: postMaintenanceRun
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenanceRunBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/maintenance/dry-run:
post:
tags: [Maintenance]
summary: Dry-run maintenance policy (async job)
operationId: postMaintenanceDryRun
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/MaintenanceRunBody"
responses:
"202":
description: Задача поставлена.
content:
application/json:
schema:
$ref: "#/components/schemas/AsyncJobAccepted"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/settings:
get:
tags: [Settings]
+1
View File
@@ -25,6 +25,7 @@ require (
github.com/prometheus/common v0.55.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
github.com/robfig/cron/v3 v3.0.1 // indirect
golang.org/x/crypto v0.46.0 // indirect
golang.org/x/mod v0.31.0 // indirect
golang.org/x/net v0.48.0 // indirect
+2
View File
@@ -46,6 +46,8 @@ github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0leargg
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec h1:W09IVJc94icq4NjY3clb7Lk8O1qJ8BdBEF8z0ibU0rE=
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec/go.mod h1:qqbHyh8v60DhA7CoWK5oRCqLrMHRGoxYCSS9EjAz6Eo=
github.com/robfig/cron/v3 v3.0.1 h1:WdRxkvbJztn8LMz/QEvLN5sBU+xKpSqwwUO1Pjr4qDs=
github.com/robfig/cron/v3 v3.0.1/go.mod h1:eQICP3HwyT7UooqI/z+Ov+PtYAWygg1TEWWzGIFLtro=
github.com/stretchr/objx v0.1.0/go.mod h1:HFkY916IF+rwdDfMAkV7OtwuqBVzrE8GR6GFx+wExME=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
+212
View File
@@ -0,0 +1,212 @@
// Package dbcli implements control-plane PostgreSQL maintenance CLI (HTTP or local DSN).
package dbcli
import (
"bytes"
"context"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"os"
"strings"
"time"
"evobgp/internal/db"
"evobgp/internal/httpclient"
"evobgp/internal/pgmonitor"
)
// Run executes db subcommands; args exclude program name and "db".
func Run(args []string) int {
if len(args) == 0 {
printUsage()
return 2
}
switch args[0] {
case "report":
return cmdReport(args[1:])
case "vacuum":
return cmdMaint(args[1:], "vacuum", "/v1/postgres/vacuum")
case "analyze":
return cmdMaint(args[1:], "analyze", "/v1/postgres/analyze")
case "cleanup":
return cmdCleanup(args[1:])
default:
fmt.Fprintf(os.Stderr, "dbcli: unknown command %q\n", args[0])
printUsage()
return 2
}
}
func printUsage() {
fmt.Fprintln(os.Stderr, `usage:
evobgp-api db report [--api-url URL] [--token TOKEN] [--format json]
evobgp-api db vacuum [--table NAME] [--dry-run] [--api-url URL] [--token TOKEN]
evobgp-api db analyze [--table NAME] [--dry-run] [--api-url URL] [--token TOKEN]
evobgp-api db cleanup --policy NAME [--dry-run] [--limit N] [--api-url URL] [--token TOKEN]
Local break-glass: set EVOBGP_DATABASE_URL (report only uses direct SQL).`)
}
func cmdReport(args []string) int {
fs := flag.NewFlagSet("report", flag.ExitOnError)
apiURL := fs.String("api-url", "", "control plane base URL")
token := fs.String("token", "", "Bearer token (operator)")
format := fs.String("format", "json", "output format (json)")
_ = fs.Parse(args)
if dsn := strings.TrimSpace(os.Getenv("EVOBGP_DATABASE_URL")); dsn != "" && *apiURL == "" {
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel()
pool, err := db.OpenPostgresPool(ctx, dsn)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
defer pool.Close()
svc := pgmonitor.NewService(pool)
ov, err := svc.Overview(ctx)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeJSONStdout(ov, *format)
}
if *apiURL == "" || *token == "" {
fmt.Fprintln(os.Stderr, "report: --api-url and --token required without EVOBGP_DATABASE_URL")
return 2
}
body, err := apiGET(*apiURL, *token, "/v1/monitoring/postgres/overview")
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
var pretty any
if err := json.Unmarshal(body, &pretty); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeJSONStdout(pretty, *format)
}
func cmdMaint(args []string, _ string, path string) int {
fs := flag.NewFlagSet("maint", flag.ExitOnError)
table := fs.String("table", "", "table name")
dryRun := fs.Bool("dry-run", false, "dry run only")
apiURL := fs.String("api-url", "", "control plane base URL")
token := fs.String("token", "", "Bearer token (operator)")
_ = fs.Parse(args)
if *apiURL == "" || *token == "" {
fmt.Fprintln(os.Stderr, "maintenance: --api-url and --token are required")
return 2
}
payload := map[string]any{"dry_run": *dryRun}
if *table != "" {
payload["table"] = *table
}
body, err := apiPOST(*apiURL, *token, path, payload)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeRawJSON(body)
}
func cmdCleanup(args []string) int {
fs := flag.NewFlagSet("cleanup", flag.ExitOnError)
policyID := fs.String("policy-id", "", "maintenance policy UUID")
dryRun := fs.Bool("dry-run", true, "dry run")
apiURL := fs.String("api-url", "", "control plane base URL")
token := fs.String("token", "", "Bearer token (operator)")
_ = fs.Parse(args)
if *policyID == "" {
fmt.Fprintln(os.Stderr, "cleanup: --policy-id is required")
return 2
}
if *apiURL == "" || *token == "" {
fmt.Fprintln(os.Stderr, "cleanup: --api-url and --token are required")
return 2
}
path := "/v1/maintenance/run"
if *dryRun {
path = "/v1/maintenance/dry-run"
}
payload := map[string]any{"policy_id": *policyID}
body, err := apiPOST(*apiURL, *token, path, payload)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return writeRawJSON(body)
}
func apiGET(base, token, path string) ([]byte, error) {
u := strings.TrimRight(base, "/") + path
req, err := http.NewRequest(http.MethodGet, u, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
resp, err := httpclient.DoWithRetry(ctx, httpclient.New(60*time.Second), req, 3)
if err != nil {
return nil, err
}
defer func() { _ = resp.Body.Close() }()
b, _ := io.ReadAll(resp.Body)
if resp.StatusCode >= 300 {
return nil, fmt.Errorf("dbcli: GET %s: %s: %s", path, resp.Status, strings.TrimSpace(string(b)))
}
return b, nil
}
func apiPOST(base, token, path string, payload map[string]any) ([]byte, error) {
b, err := json.Marshal(payload)
if err != nil {
return nil, err
}
u := strings.TrimRight(base, "/") + path
req, err := http.NewRequest(http.MethodPost, u, bytes.NewReader(b))
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Content-Type", "application/json")
ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second)
defer cancel()
resp, err := httpclient.DoWithRetry(ctx, httpclient.New(60*time.Second), req, 3)
if err != nil {
return nil, err
}
defer func() { _ = resp.Body.Close() }()
out, _ := io.ReadAll(resp.Body)
if resp.StatusCode != http.StatusAccepted && resp.StatusCode != http.StatusOK {
return nil, fmt.Errorf("dbcli: POST %s: %s: %s", path, resp.Status, strings.TrimSpace(string(out)))
}
return out, nil
}
func writeJSONStdout(v any, format string) int {
if format != "json" {
fmt.Fprintln(os.Stderr, "only json format supported")
return 2
}
enc := json.NewEncoder(os.Stdout)
enc.SetIndent("", " ")
if err := enc.Encode(v); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
return 0
}
func writeRawJSON(b []byte) int {
var v any
if err := json.Unmarshal(b, &v); err != nil {
_, _ = os.Stdout.Write(b)
return 0
}
return writeJSONStdout(v, "json")
}
+1 -1
View File
@@ -51,7 +51,7 @@ func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.R
}
cdnHTTP := NewCDNHTTPClient()
wk := &jobs.Worker{Store: backend, HTTPClient: cdnHTTP}
wk := &jobs.Worker{Store: backend, PgPool: pool, HTTPClient: cdnHTTP}
reg := jobs.NewRegistry(wk.Process)
wk.Registry = reg
if pool != nil {
+3
View File
@@ -76,6 +76,9 @@ func (s *Server) registerV1(m *http.ServeMux) {
m.HandleFunc("GET /speakers/{speaker_id}/bundle/{revision_id}", s.handleNodeBundle)
m.HandleFunc("POST /nodes/enroll", s.handleNodeEnroll)
s.registerCRUDRoutes(m)
s.registerPostgresMonitoringRoutes(m)
s.registerPostgresMaintenanceRoutes(m)
s.registerMaintenanceRoutes(m)
}
func (s *Server) handleHealth(w http.ResponseWriter, r *http.Request) {
+274
View File
@@ -0,0 +1,274 @@
package httpapi
import (
"encoding/json"
"net/http"
"strings"
"evobgp/internal/jobs"
"evobgp/internal/observability"
"evobgp/internal/store"
)
func (s *Server) registerMaintenanceRoutes(m *http.ServeMux) {
m.HandleFunc("GET /maintenance/policies", s.handleListMaintenancePolicies)
m.HandleFunc("POST /maintenance/policies", s.handleCreateMaintenancePolicy)
m.HandleFunc("GET /maintenance/policies/{id}", s.handleGetMaintenancePolicy)
m.HandleFunc("PATCH /maintenance/policies/{id}", s.handlePatchMaintenancePolicy)
m.HandleFunc("DELETE /maintenance/policies/{id}", s.handleDeleteMaintenancePolicy)
m.HandleFunc("GET /maintenance/policies/{id}/hints", s.handleMaintenancePolicyHints)
m.HandleFunc("GET /maintenance/config-audit", s.handleListMaintenanceConfigAudit)
m.HandleFunc("POST /maintenance/run", s.handleMaintenanceRun)
m.HandleFunc("POST /maintenance/dry-run", s.handleMaintenanceDryRun)
}
func maintenancePolicyJSON(p *store.MaintenancePolicy) map[string]any {
if p == nil {
return map[string]any{}
}
out := map[string]any{
"id": p.ID,
"name": p.Name,
"table_name": p.TableName,
"condition": p.Condition,
"vacuum_strategy": p.VacuumStrategy,
"schedule": p.Schedule,
"enabled": p.Enabled,
"dry_run_enabled": p.DryRunEnabled,
}
if p.RetentionPeriodSec != nil {
out["retention_period_sec"] = *p.RetentionPeriodSec
}
if p.MaxRows != nil {
out["max_rows"] = *p.MaxRows
}
if p.LastRunAt != nil {
out["last_run_at"] = p.LastRunAt.UTC().Format("2006-01-02T15:04:05Z")
}
if p.LastStatus != "" {
out["last_status"] = p.LastStatus
}
if p.LastError != "" {
out["last_error"] = p.LastError
}
if !p.CreatedAt.IsZero() {
out["created_at"] = p.CreatedAt.UTC().Format("2006-01-02T15:04:05Z")
}
if !p.UpdatedAt.IsZero() {
out["updated_at"] = p.UpdatedAt.UTC().Format("2006-01-02T15:04:05Z")
}
return out
}
func (s *Server) handleListMaintenancePolicies(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
cursor := r.URL.Query().Get("cursor")
limit := parseLimitQuery(r, 20, 100)
items, next, hasMore, err := s.store.ListMaintenancePolicies(cursor, limit)
if err != nil {
writeInternalError(w, "maintenance_policies_list", err)
return
}
out := make([]map[string]any, 0, len(items))
for _, p := range items {
out = append(out, maintenancePolicyJSON(p))
}
writeJSON(w, http.StatusOK, map[string]any{"items": out, "next_cursor": next, "has_more": hasMore})
}
func (s *Server) handleGetMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
p, err := s.store.GetMaintenancePolicy(r.PathValue("id"))
if err != nil {
writeStoreErr(w, err)
return
}
writeJSON(w, http.StatusOK, maintenancePolicyJSON(p))
}
func (s *Server) handleCreateMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
return
}
var body store.MaintenancePolicy
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
p, err := s.store.CreateMaintenancePolicy(&body)
if err != nil {
writeStoreErr(w, err)
return
}
_ = s.store.AppendMaintenancePolicyConfigAudit(actorPrefix(a), p.ID, "create", nil, maintenancePolicyJSON(p))
observability.IncMaintenanceConfigChange("create")
s.reloadMaintenanceConfig(r)
writeJSON(w, http.StatusCreated, maintenancePolicyJSON(p))
}
func (s *Server) handlePatchMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
return
}
id := r.PathValue("id")
before, err := s.store.GetMaintenancePolicy(id)
if err != nil {
writeStoreErr(w, err)
return
}
var patch store.MaintenancePolicyPatch
if err := json.NewDecoder(r.Body).Decode(&patch); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
updated, err := s.store.UpdateMaintenancePolicy(id, &patch)
if err != nil {
writeStoreErr(w, err)
return
}
_ = s.store.AppendMaintenancePolicyConfigAudit(actorPrefix(a), id, "update", maintenancePolicyJSON(before), maintenancePolicyJSON(updated))
observability.IncMaintenanceConfigChange("update")
s.reloadMaintenanceConfig(r)
writeJSON(w, http.StatusOK, maintenancePolicyJSON(updated))
}
func (s *Server) handleDeleteMaintenancePolicy(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
return
}
id := r.PathValue("id")
before, err := s.store.GetMaintenancePolicy(id)
if err != nil {
writeStoreErr(w, err)
return
}
if err := s.store.DeleteMaintenancePolicy(id); err != nil {
writeStoreErr(w, err)
return
}
_ = s.store.AppendMaintenancePolicyConfigAudit(actorPrefix(a), id, "delete", maintenancePolicyJSON(before), nil)
observability.IncMaintenanceConfigChange("delete")
s.reloadMaintenanceConfig(r)
w.WriteHeader(http.StatusNoContent)
}
func (s *Server) handleMaintenancePolicyHints(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
if s.maintStats == nil {
writeProblem(w, http.StatusServiceUnavailable, "Unavailable", "postgresql backend required")
return
}
p, err := s.store.GetMaintenancePolicy(r.PathValue("id"))
if err != nil {
writeStoreErr(w, err)
return
}
hints, err := s.maintStats.Hints(r.Context(), p.TableName)
if err != nil {
writeInternalError(w, "maintenance_policy_hints", err)
return
}
writeJSON(w, http.StatusOK, hints)
}
func (s *Server) handleListMaintenanceConfigAudit(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
cursor := r.URL.Query().Get("cursor")
limit := parseLimitQuery(r, 20, 100)
items, next, hasMore, err := s.store.ListMaintenancePolicyConfigAudit(cursor, limit)
if err != nil {
writeInternalError(w, "maintenance_config_audit", err)
return
}
out := make([]map[string]any, 0, len(items))
for _, row := range items {
out = append(out, map[string]any{
"id": row.ID,
"policy_id": row.PolicyID,
"actor_prefix": row.ActorPrefix,
"action": row.Action,
"before": row.Before,
"after": row.After,
"created_at": row.CreatedAt.UTC().Format("2006-01-02T15:04:05Z"),
})
}
writeJSON(w, http.StatusOK, map[string]any{"items": out, "next_cursor": next, "has_more": hasMore})
}
type maintenanceRunBody struct {
PolicyID string `json:"policy_id"`
}
func (s *Server) handleMaintenanceRun(w http.ResponseWriter, r *http.Request) {
s.enqueueMaintenancePolicy(w, r, false)
}
func (s *Server) handleMaintenanceDryRun(w http.ResponseWriter, r *http.Request) {
s.enqueueMaintenancePolicy(w, r, true)
}
func (s *Server) enqueueMaintenancePolicy(w http.ResponseWriter, r *http.Request, dryRun bool) {
a, ok := authFromContext(r.Context())
if !ok || !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
return
}
var body maintenanceRunBody
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
policyID := strings.TrimSpace(body.PolicyID)
if policyID == "" {
writeProblem(w, http.StatusBadRequest, "Bad Request", "policy_id is required")
return
}
if _, err := s.store.GetMaintenancePolicy(policyID); err != nil {
writeStoreErr(w, err)
return
}
kind := "maintenance_policy_run"
if !s.checkPgMaintRateLimit(a.TenantID, kind+":"+policyID) {
writeProblem(w, http.StatusTooManyRequests, "Too Many Requests", "wait before repeating this maintenance operation")
return
}
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
var idemPtr *string
if idem != "" {
idemPtr = &idem
}
title := "Maintenance policy run"
if dryRun {
title = "Maintenance policy dry-run"
}
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindMaintenancePolicyRun, idemPtr, nil, map[string]any{
"policy_id": policyID, "dry_run": dryRun, "actor_prefix": actorPrefix(a), "job_title": title,
})
if err != nil {
writeInternalError(w, "maintenance_policy_enqueue", err)
return
}
w.Header().Set("Location", "/v1/jobs/"+j.ID)
snap := j.Snapshot()
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
}
func (s *Server) reloadMaintenanceConfig(r *http.Request) {
if s.maintConfig != nil {
_ = s.maintConfig.Reload(r.Context())
}
}
@@ -0,0 +1,46 @@
package httpapi
import (
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestMaintenancePoliciesMemoryBackend503(t *testing.T) {
srv, err := New(Options{SeedDemo: true, InsecureDev: true})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
handler := srv.Handler()
tests := []struct {
method string
path string
body string
}{
{http.MethodGet, "/v1/maintenance/policies", ""},
{http.MethodPost, "/v1/maintenance/policies", `{"name":"x","table_name":"job_audit","schedule":"0 3 * * *"}`},
{http.MethodPost, "/v1/maintenance/run", `{"policy_id":"00000000-0000-0000-0000-000000000001"}`},
{http.MethodGet, "/v1/maintenance/config-audit", ""},
}
for _, tc := range tests {
t.Run(tc.method+" "+tc.path, func(t *testing.T) {
var req *http.Request
if tc.body != "" {
req = httptest.NewRequest(tc.method, tc.path, strings.NewReader(tc.body))
req.Header.Set("Content-Type", "application/json")
} else {
req = httptest.NewRequest(tc.method, tc.path, nil)
}
req.Header.Set("Authorization", "Bearer dev")
rec := httptest.NewRecorder()
handler.ServeHTTP(rec, req)
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
})
}
}
@@ -0,0 +1,224 @@
package httpapi
import (
"context"
"encoding/json"
"io"
"net/http"
"strings"
"sync"
"time"
"evobgp/internal/jobs"
"evobgp/internal/pgmonitor"
)
var (
pgMaintRateMu sync.Mutex
pgMaintLastByTK = map[string]time.Time{}
)
func (s *Server) registerPostgresMaintenanceRoutes(m *http.ServeMux) {
m.HandleFunc("POST /postgres/vacuum", s.handlePostgresVacuum)
m.HandleFunc("POST /postgres/vacuum-analyze", s.handlePostgresVacuumAnalyze)
m.HandleFunc("POST /postgres/analyze", s.handlePostgresAnalyze)
m.HandleFunc("POST /postgres/reindex", s.handlePostgresReindex)
m.HandleFunc("POST /postgres/cleanup", s.handlePostgresCleanup)
m.HandleFunc("GET /postgres/maintenance/logs", s.handlePostgresMaintenanceLogs)
}
func (s *Server) requireOperatorStrict(w http.ResponseWriter, a Auth) bool {
if strings.ToLower(a.Role) != "operator" {
writeProblem(w, http.StatusForbidden, "Forbidden", "operator role required")
return false
}
return true
}
func (s *Server) checkPgMaintRateLimit(tenantID, kind string) bool {
key := tenantID + ":" + kind
pgMaintRateMu.Lock()
defer pgMaintRateMu.Unlock()
if t, ok := pgMaintLastByTK[key]; ok && time.Since(t) < 60*time.Second {
return false
}
pgMaintLastByTK[key] = time.Now().UTC()
return true
}
type pgMaintBody struct {
Table string `json:"table"`
DryRun bool `json:"dry_run"`
Index string `json:"index"`
Policy string `json:"policy"`
PolicyID string `json:"policy_id"`
Limit int `json:"limit"`
}
func (s *Server) decodePgMaintBody(r *http.Request) (pgMaintBody, bool) {
var body pgMaintBody
if r.Body == nil || r.ContentLength == 0 {
return body, true
}
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&body); err != nil && err != io.EOF {
return body, false
}
return body, true
}
func (s *Server) enqueuePostgresMaint(w http.ResponseWriter, r *http.Request, a Auth, kind string, meta map[string]any) {
if !s.requirePostgres(w) || !s.requireOperatorStrict(w, a) {
return
}
if !s.checkPgMaintRateLimit(a.TenantID, kind) {
writeProblem(w, http.StatusTooManyRequests, "Too Many Requests", "wait before repeating this maintenance operation")
return
}
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
var idemPtr *string
if idem != "" {
idemPtr = &idem
}
meta["actor_prefix"] = actorPrefix(a)
j, _, err := s.jobs.Enqueue(a.TenantID, kind, idemPtr, nil, meta)
if err != nil {
writeInternalError(w, "postgres_maint_enqueue", err)
return
}
w.Header().Set("Location", "/v1/jobs/"+j.ID)
snap := j.Snapshot()
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
}
func (s *Server) handlePostgresVacuum(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresVacuum, map[string]any{
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL VACUUM",
})
}
func (s *Server) handlePostgresVacuumAnalyze(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresVacuumAnalyze, map[string]any{
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL VACUUM ANALYZE",
})
}
func (s *Server) handlePostgresAnalyze(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresAnalyze, map[string]any{
"table": body.Table, "dry_run": body.DryRun, "job_title": "PostgreSQL ANALYZE",
})
}
func (s *Server) handlePostgresReindex(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
table := body.Table
if table == "" {
table = body.Index
}
s.enqueuePostgresMaint(w, r, a, jobs.KindPostgresReindex, map[string]any{
"table": table, "dry_run": body.DryRun, "job_title": "PostgreSQL REINDEX",
})
}
func (s *Server) handlePostgresCleanup(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
body, ok2 := s.decodePgMaintBody(r)
if !ok2 {
writeProblem(w, http.StatusBadRequest, "Bad Request", invalidInputDetail)
return
}
policyID := strings.TrimSpace(body.PolicyID)
if policyID == "" {
policyID = strings.TrimSpace(body.Policy)
}
if policyID == "" {
writeProblem(w, http.StatusBadRequest, "Bad Request", "policy_id is required")
return
}
if _, err := s.store.GetMaintenancePolicy(policyID); err != nil {
writeStoreErr(w, err)
return
}
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
var idemPtr *string
if idem != "" {
idemPtr = &idem
}
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindMaintenancePolicyRun, idemPtr, nil, map[string]any{
"policy_id": policyID, "dry_run": body.DryRun, "actor_prefix": actorPrefix(a),
"job_title": "PostgreSQL cleanup (deprecated path)",
})
if err != nil {
writeInternalError(w, "postgres_maint_enqueue", err)
return
}
w.Header().Set("Location", "/v1/jobs/"+j.ID)
snap := j.Snapshot()
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
}
func (s *Server) handlePostgresMaintenanceLogs(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
cursor := r.URL.Query().Get("cursor")
limit := parseLimitQuery(r, 20, 100)
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
items, next, hasMore, err := pgmonitor.ListMaintenanceLogs(ctx, s.pgMonitor.Pool(), cursor, limit)
if err != nil {
writeInternalError(w, "postgres_maint_logs", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "next_cursor": next, "has_more": hasMore})
}
func actorPrefix(a Auth) string {
if len(a.Token) >= 8 {
return a.Token[:8]
}
return a.Role
}
@@ -0,0 +1,130 @@
package httpapi
import (
"context"
"net/http"
"strconv"
"time"
)
func (s *Server) registerPostgresMonitoringRoutes(m *http.ServeMux) {
m.HandleFunc("GET /monitoring/postgres/overview", s.handlePostgresOverview)
m.HandleFunc("GET /monitoring/postgres/queries", s.handlePostgresQueries)
m.HandleFunc("GET /monitoring/postgres/locks", s.handlePostgresLocks)
m.HandleFunc("GET /monitoring/postgres/tables", s.handlePostgresTables)
m.HandleFunc("GET /monitoring/postgres/recommendations", s.handlePostgresRecommendations)
m.HandleFunc("GET /monitoring/correlation", s.handleMonitoringCorrelation)
}
func (s *Server) requirePostgres(w http.ResponseWriter) bool {
if s.pgMonitor == nil {
writeProblem(w, http.StatusServiceUnavailable, "Unavailable", "postgresql backend required")
return false
}
return true
}
func parseLimitQuery(r *http.Request, def, max int) int {
if v := r.URL.Query().Get("limit"); v != "" {
if n, err := strconv.Atoi(v); err == nil {
return n
}
}
return def
}
func (s *Server) handlePostgresOverview(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.Overview(ctx)
if err != nil {
writeInternalError(w, "postgres_overview", err)
return
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handlePostgresQueries(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.TopQueries(ctx, parseLimitQuery(r, 20, 100))
if err != nil {
writeInternalError(w, "postgres_queries", err)
return
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handlePostgresLocks(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.Locks(ctx)
if err != nil {
writeInternalError(w, "postgres_locks", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": out})
}
func (s *Server) handlePostgresTables(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 3*time.Second)
defer cancel()
out, err := s.pgMonitor.Tables(ctx, parseLimitQuery(r, 20, 100))
if err != nil {
writeInternalError(w, "postgres_tables", err)
return
}
writeJSON(w, http.StatusOK, map[string]any{"items": out})
}
func (s *Server) handlePostgresRecommendations(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
out, err := s.pgMonitor.Recommendations(ctx)
if err != nil {
writeInternalError(w, "postgres_recommendations", err)
return
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handleMonitoringCorrelation(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") || !s.requirePostgres(w) {
return
}
window := 60
if v := r.URL.Query().Get("window"); v != "" {
if n, err := strconv.Atoi(v); err == nil {
window = n
}
}
ctx, cancel := context.WithTimeout(r.Context(), 5*time.Second)
defer cancel()
out, err := s.pgMonitor.Correlation(ctx, window)
if err != nil {
writeInternalError(w, "monitoring_correlation", err)
return
}
writeJSON(w, http.StatusOK, out)
}
+21
View File
@@ -0,0 +1,21 @@
package httpapi
import (
"net/http"
"net/http/httptest"
"testing"
)
func TestPostgresOverviewMemoryBackend503(t *testing.T) {
srv, err := New(Options{SeedDemo: true})
if err != nil {
t.Fatal(err)
}
req := httptest.NewRequest(http.MethodGet, "/v1/monitoring/postgres/overview", nil)
req.Header.Set("Authorization", "Bearer dev")
rec := httptest.NewRecorder()
srv.Handler().ServeHTTP(rec, req)
if rec.Code != http.StatusServiceUnavailable {
t.Fatalf("status=%d body=%s", rec.Code, rec.Body.String())
}
}
+35
View File
@@ -8,8 +8,11 @@ import (
"errors"
"net/http"
"strings"
"time"
"evobgp/internal/jobs"
"evobgp/internal/maintenance"
"evobgp/internal/pgmonitor"
"evobgp/internal/store"
"github.com/jackc/pgx/v5/pgxpool"
@@ -19,6 +22,9 @@ import (
type Server struct {
store store.Backend
pgPool *pgxpool.Pool
pgMonitor *pgmonitor.Service
maintConfig *maintenance.ConfigProvider
maintStats *maintenance.DBStatsProvider
jobs *jobs.Registry
bundlePriv ed25519.PrivateKey
keyResolver *apiKeyResolver
@@ -63,9 +69,21 @@ func New(opts Options) (*Server, error) {
if err != nil {
return nil, err
}
var pgMon *pgmonitor.Service
var maintCfg *maintenance.ConfigProvider
var maintStats *maintenance.DBStatsProvider
if pool != nil {
pgMon = pgmonitor.NewService(pool)
maintCfg = maintenance.NewConfigProvider(backend)
_ = maintCfg.Reload(context.Background())
maintStats = maintenance.NewDBStatsProvider(pgMon)
}
s := &Server{
store: backend,
pgPool: pool,
pgMonitor: pgMon,
maintConfig: maintCfg,
maintStats: maintStats,
jobs: reg,
bundlePriv: priv,
keyResolver: resolver,
@@ -89,3 +107,20 @@ func (s *Server) Store() store.Backend { return s.store }
// Jobs exposes the in-process async job registry (for scheduler / evobgp-all).
func (s *Server) Jobs() *jobs.Registry { return s.jobs }
// StartBackground starts PostgreSQL monitoring and maintenance schedulers until ctx is cancelled.
func (s *Server) StartBackground(ctx context.Context) {
if s != nil && s.pgPool != nil {
pgmonitor.StartScheduler(ctx, s.pgPool)
}
if s != nil && s.maintConfig != nil && s.jobs != nil {
maintenance.StartScheduler(ctx, s.maintConfig, func(policyID string, dryRun bool, idem string) {
key := idem
_, _, _ = s.jobs.Enqueue("", jobs.KindMaintenancePolicyRun, &key, nil, map[string]any{
"policy_id": policyID,
"dry_run": dryRun,
"trigger": "scheduler",
})
}, 30*time.Second)
}
}
-6
View File
@@ -17,8 +17,6 @@ type Deps struct {
Store store.Backend
}
var lastMaintenance time.Time
// Run blocks until ctx is cancelled.
func Run(ctx context.Context, deps *Deps) {
cfg := config.Load()
@@ -36,10 +34,6 @@ func Run(ctx context.Context, deps *Deps) {
log.Printf("evobgp-ingest: stopped")
return
case <-t.C:
if deps.Store != nil && time.Since(lastMaintenance) > time.Hour {
deps.Store.RunPeriodicMaintenance(ctx)
lastMaintenance = time.Now()
}
prefetchCtx, cancel := context.WithTimeout(ctx, 50*time.Second)
err := pipeline.PrefetchCDNSourceETags(prefetchCtx, deps.Store, hc)
cancel()
+63
View File
@@ -0,0 +1,63 @@
package jobs
import (
"errors"
"strings"
"evobgp/internal/maintenance"
"evobgp/internal/pgmonitor"
"evobgp/internal/store"
)
func (w *Worker) maintenanceExecutor() *maintenance.PolicyExecutor {
if w == nil {
return nil
}
return &maintenance.PolicyExecutor{Store: w.Store, Pool: w.PgPool}
}
func (w *Worker) runMaintenancePolicy(j *Job) {
if w == nil || w.PgPool == nil {
j.Fail("postgresql not configured")
return
}
policyID, _ := j.Meta["policy_id"].(string)
policyID = strings.TrimSpace(policyID)
if policyID == "" {
j.Fail("missing policy_id in job meta")
return
}
dryRun, _ := j.Meta["dry_run"].(bool)
actor, _ := j.Meta["actor_prefix"].(string)
ctx, cancel := j.workContext()
defer cancel()
pol, err := w.Store.GetMaintenancePolicy(policyID)
if err != nil {
if errors.Is(err, store.ErrNotFound) {
j.Fail("maintenance policy not found")
return
}
j.Fail(err.Error())
return
}
auditID, _ := pgmonitor.InsertMaintenanceAuditWithPolicy(ctx, w.PgPool, j.TenantID, actor, "maintenance_policy_run", pol.TableName, policyID, dryRun)
exec := w.maintenanceExecutor()
detail, err := exec.Execute(ctx, pol, dryRun)
var errMsg *string
status := StatusSucceeded
if err != nil {
s := err.Error()
errMsg = &s
status = StatusFailed
_ = w.Store.TouchMaintenancePolicyRun(policyID, status, s)
j.Fail(s)
} else {
j.mergeMeta(map[string]any{"maintenance": detail, "audit_id": auditID, "policy_id": policyID})
j.Succeed()
}
if auditID != "" {
_ = pgmonitor.FinishMaintenanceAudit(ctx, w.PgPool, auditID, status, detail, errMsg)
}
}
+140
View File
@@ -0,0 +1,140 @@
package jobs
import (
"fmt"
"evobgp/internal/pgmonitor"
)
func (w *Worker) pgService() *pgmonitor.Service {
if w == nil || w.PgPool == nil {
return nil
}
return pgmonitor.NewService(w.PgPool)
}
func (w *Worker) runPostgresMetricsRefresh(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.RefreshMetricsSnapshot(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresSlowQueryAgg(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.AggregateSlowQueries(ctx, 30); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresTableBloat(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.EstimateTableBloat(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresIndexUsage(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.AnalyzeIndexUsage(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresAutovacuumLag(j *Job) {
s := w.pgService()
if s == nil {
j.Fail("postgresql not configured")
return
}
ctx, cancel := j.workContext()
defer cancel()
if err := s.DetectAutovacuumLag(ctx); err != nil {
j.Fail(err.Error())
return
}
j.Succeed()
}
func (w *Worker) runPostgresMaint(j *Job, kind string) {
if w == nil || w.PgPool == nil {
j.Fail("postgresql not configured")
return
}
table, _ := j.Meta["table"].(string)
dryRun, _ := j.Meta["dry_run"].(bool)
actor, _ := j.Meta["actor_prefix"].(string)
ctx, cancel := j.workContext()
defer cancel()
auditID, _ := pgmonitor.InsertMaintenanceAudit(ctx, w.PgPool, j.TenantID, actor, kind, table, dryRun)
detail, err := pgmonitor.ExecMaintenance(ctx, w.PgPool, kind, table, dryRun)
var errMsg *string
status := StatusSucceeded
if err != nil {
s := err.Error()
errMsg = &s
status = StatusFailed
j.Fail(s)
} else {
j.mergeMeta(map[string]any{"maintenance": detail, "audit_id": auditID})
j.Succeed()
}
if auditID != "" {
_ = pgmonitor.FinishMaintenanceAudit(ctx, w.PgPool, auditID, status, detail, errMsg)
}
}
func (w *Worker) runPostgresCleanup(j *Job) {
j.Fail("postgres_cleanup deprecated: configure maintenance_policy in UI and use maintenance_policy_run")
}
// EnqueuePostgresAnalyzerJobs enqueues periodic analyzer jobs (global tenant id).
func EnqueuePostgresAnalyzerJobs(reg *Registry, tenantID string) {
if reg == nil || tenantID == "" {
return
}
kinds := []string{
KindPostgresMetricsRefresh,
KindPostgresSlowQueryAgg,
KindPostgresTableBloat,
KindPostgresIndexUsage,
KindPostgresAutovacuumLag,
}
for _, k := range kinds {
key := fmt.Sprintf("pgmon-%s-%s", k, tenantID)
idem := key
_, _, _ = reg.Enqueue(tenantID, k, &idem, nil, map[string]any{"trigger": "scheduler"})
}
}
+42 -6
View File
@@ -18,6 +18,8 @@ import (
"evobgp/internal/observability"
"evobgp/internal/pipeline"
"evobgp/internal/store"
"github.com/jackc/pgx/v5/pgxpool"
)
// mergeBirdPostApplyMeta attaches a birdc snapshot after deploy/reload (best-effort).
@@ -43,17 +45,29 @@ func mergeBirdPostApplyMeta(j *Job) {
}
const (
KindModuleRefresh = "module_refresh"
KindTenantRefresh = "tenant_refresh"
KindPeerReconcile = "peer_reconcile"
KindDeployApply = "deploy_apply"
KindRevisionRollback = "revision_rollback"
KindBirdReload = "bird_reload"
KindModuleRefresh = "module_refresh"
KindTenantRefresh = "tenant_refresh"
KindPeerReconcile = "peer_reconcile"
KindDeployApply = "deploy_apply"
KindRevisionRollback = "revision_rollback"
KindBirdReload = "bird_reload"
KindPostgresMetricsRefresh = "postgres_metrics_refresh"
KindPostgresSlowQueryAgg = "postgres_slow_query_aggregate"
KindPostgresTableBloat = "postgres_table_bloat_estimate"
KindPostgresIndexUsage = "postgres_index_usage_analyze"
KindPostgresAutovacuumLag = "postgres_autovacuum_lag_detect"
KindPostgresVacuum = "postgres_vacuum"
KindPostgresVacuumAnalyze = "postgres_vacuum_analyze"
KindPostgresAnalyze = "postgres_analyze"
KindPostgresReindex = "postgres_reindex"
KindPostgresCleanup = "postgres_cleanup"
KindMaintenancePolicyRun = "maintenance_policy_run"
)
// Worker executes queued jobs against store.Backend (memory or SQL).
type Worker struct {
Store store.Backend
PgPool *pgxpool.Pool
HTTPClient *http.Client // optional; CDN refresh uses this (default 45s timeout).
// Registry is set after BootstrapWorkers creates the job queue; used to chain deploy_apply after refresh/rollback.
Registry *Registry
@@ -155,6 +169,28 @@ func (w *Worker) Process(j *Job) {
}
mergeBirdPostApplyMeta(j)
j.Succeed()
case KindPostgresMetricsRefresh:
w.runPostgresMetricsRefresh(j)
case KindPostgresSlowQueryAgg:
w.runPostgresSlowQueryAgg(j)
case KindPostgresTableBloat:
w.runPostgresTableBloat(j)
case KindPostgresIndexUsage:
w.runPostgresIndexUsage(j)
case KindPostgresAutovacuumLag:
w.runPostgresAutovacuumLag(j)
case KindPostgresVacuum:
w.runPostgresMaint(j, "vacuum")
case KindPostgresVacuumAnalyze:
w.runPostgresMaint(j, "vacuum_analyze")
case KindPostgresAnalyze:
w.runPostgresMaint(j, "analyze")
case KindPostgresReindex:
w.runPostgresMaint(j, "reindex")
case KindPostgresCleanup:
w.runPostgresCleanup(j)
case KindMaintenancePolicyRun:
w.runMaintenancePolicy(j)
default:
j.Fail("unknown job kind")
}
+67
View File
@@ -0,0 +1,67 @@
package maintenance
import (
"context"
"sync"
"evobgp/internal/store"
)
// ConfigProvider caches maintenance policies from store.Backend with hot reload.
type ConfigProvider struct {
store store.Backend
mu sync.RWMutex
items []*store.MaintenancePolicy
}
// NewConfigProvider constructs a provider; call Reload before use.
func NewConfigProvider(st store.Backend) *ConfigProvider {
return &ConfigProvider{store: st}
}
// Reload loads all policies from the database into memory.
func (c *ConfigProvider) Reload(ctx context.Context) error {
if c == nil || c.store == nil {
return nil
}
_ = ctx
items, _, _, err := c.store.ListMaintenancePolicies("", 1000)
if err != nil {
return err
}
cp := make([]*store.MaintenancePolicy, len(items))
copy(cp, items)
c.mu.Lock()
c.items = cp
c.mu.Unlock()
return nil
}
// Snapshot returns a copy of cached policies.
func (c *ConfigProvider) Snapshot() []*store.MaintenancePolicy {
if c == nil {
return nil
}
c.mu.RLock()
defer c.mu.RUnlock()
out := make([]*store.MaintenancePolicy, len(c.items))
copy(out, c.items)
return out
}
// Get returns one policy by id from cache or store.
func (c *ConfigProvider) Get(ctx context.Context, id string) (*store.MaintenancePolicy, error) {
if c == nil || c.store == nil {
return nil, store.ErrNotFound
}
c.mu.RLock()
for _, p := range c.items {
if p.ID == id {
cp := *p
c.mu.RUnlock()
return &cp, nil
}
}
c.mu.RUnlock()
return c.store.GetMaintenancePolicy(id)
}
@@ -0,0 +1,43 @@
package maintenance
import (
"context"
"testing"
"evobgp/internal/store"
)
func TestConfigProviderReloadAndSnapshot(t *testing.T) {
mem := store.NewMemory()
ret := 3600
if _, err := mem.CreateMaintenancePolicy(&store.MaintenancePolicy{
Name: "p1",
TableName: "job_audit",
Schedule: "0 3 * * *",
RetentionPeriodSec: &ret,
Enabled: true,
}); err != nil {
t.Fatal(err)
}
cp := NewConfigProvider(mem)
if err := cp.Reload(context.Background()); err != nil {
t.Fatal(err)
}
snap := cp.Snapshot()
if len(snap) != 1 || snap[0].Name != "p1" {
t.Fatalf("snapshot: %+v", snap)
}
newName := "p1-updated"
if _, err := mem.UpdateMaintenancePolicy(snap[0].ID, &store.MaintenancePolicyPatch{Name: &newName}); err != nil {
t.Fatal(err)
}
if err := cp.Reload(context.Background()); err != nil {
t.Fatal(err)
}
snap2 := cp.Snapshot()
if len(snap2) != 1 || snap2[0].Name != newName {
t.Fatalf("after reload: %+v", snap2)
}
}
+62
View File
@@ -0,0 +1,62 @@
package maintenance
import (
"context"
"fmt"
"evobgp/internal/pgmonitor"
)
// TableHints are PostgreSQL statistics hints for UI recommendations.
type TableHints struct {
TableName string `json:"table_name"`
DeadTuples int64 `json:"n_dead_tup"`
BloatRatio float64 `json:"bloat_ratio,omitempty"`
LastAutovacuum string `json:"last_autovacuum,omitempty"`
RecommendVacuum bool `json:"recommend_vacuum"`
Detail string `json:"detail,omitempty"`
Refs []string `json:"refs,omitempty"`
}
// DBStatsProvider wraps pgmonitor for maintenance policy hints.
type DBStatsProvider struct {
pg *pgmonitor.Service
}
// NewDBStatsProvider constructs a stats provider.
func NewDBStatsProvider(pg *pgmonitor.Service) *DBStatsProvider {
return &DBStatsProvider{pg: pg}
}
// Hints returns table-level vacuum/bloat hints.
func (d *DBStatsProvider) Hints(ctx context.Context, tableName string) (TableHints, error) {
out := TableHints{TableName: tableName}
if d == nil || d.pg == nil {
return out, fmt.Errorf("maintenance: postgres monitoring not configured")
}
if err := ValidateTableName(tableName); err != nil {
return out, err
}
tables, err := d.pg.Tables(ctx, 100)
if err != nil {
return out, err
}
for _, t := range tables {
if t.Relname != tableName {
continue
}
out.DeadTuples = t.DeadTuples
out.BloatRatio = t.BloatRatio
if t.LastAutovacuum != nil {
out.LastAutovacuum = t.LastAutovacuum.UTC().Format("2006-01-02T15:04:05Z")
}
if t.BloatRatio > 0.2 && t.DeadTuples > 5000 {
out.RecommendVacuum = true
out.Detail = "Высокая доля n_dead_tup; рекомендуется VACUUM."
out.Refs = []string{t.Relname}
}
return out, nil
}
out.Detail = "Таблица не найдена в pg_stat_user_tables (top by size)."
return out, nil
}
+2
View File
@@ -0,0 +1,2 @@
// Package maintenance implements PostgreSQL maintenance policies loaded from the database.
package maintenance
+195
View File
@@ -0,0 +1,195 @@
package maintenance
import (
"context"
"fmt"
"hash/fnv"
"strings"
"time"
"evobgp/internal/observability"
"evobgp/internal/pgmonitor"
"evobgp/internal/store"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgxpool"
)
// PolicyExecutor runs maintenance policies against PostgreSQL.
type PolicyExecutor struct {
Store store.Backend
Pool *pgxpool.Pool
}
// Execute runs cleanup and/or vacuum steps for a policy.
func (e *PolicyExecutor) Execute(ctx context.Context, policy *store.MaintenancePolicy, dryRun bool) (map[string]any, error) {
start := time.Now()
if policy == nil {
return nil, store.ErrInvalidInput
}
action := policyAction(policy)
record := func(status string, detail map[string]any) {
observability.RecordMaintenancePolicyRun(policy.ID, action, status, dryRun, time.Since(start), rowsDeletedFromDetail(detail))
}
if e == nil || e.Pool == nil {
record("failed", nil)
return nil, fmt.Errorf("maintenance: postgres not configured")
}
if err := ValidateTableName(policy.TableName); err != nil {
record("failed", nil)
return nil, err
}
if err := ValidateCondition(policy.Condition); err != nil {
record("failed", nil)
return nil, err
}
if !store.ValidVacuumStrategy(policy.VacuumStrategy) {
record("failed", nil)
return nil, store.ErrInvalidInput
}
detail := map[string]any{
"policy_id": policy.ID,
"table": policy.TableName,
"dry_run": dryRun,
}
if policy.RetentionPeriodSec != nil || policy.MaxRows != nil {
cleanupDetail, err := e.runCleanup(ctx, policy, dryRun)
for k, v := range cleanupDetail {
detail[k] = v
}
if err != nil {
record("failed", detail)
return detail, err
}
}
if policy.VacuumStrategy != store.VacuumStrategyNone {
kind := vacuumKind(policy.VacuumStrategy)
vacDetail, err := pgmonitor.ExecMaintenance(ctx, e.Pool, kind, policy.TableName, dryRun)
if vacDetail != nil {
detail["vacuum"] = vacDetail
}
if err != nil {
record("failed", detail)
return detail, err
}
}
if !dryRun && e.Store != nil {
_ = e.Store.TouchMaintenancePolicyRun(policy.ID, "succeeded", "")
}
record("succeeded", detail)
return detail, nil
}
func policyAction(p *store.MaintenancePolicy) string {
if p == nil {
return "run"
}
if p.RetentionPeriodSec != nil || p.MaxRows != nil {
if p.VacuumStrategy != store.VacuumStrategyNone {
return "cleanup_vacuum"
}
return "cleanup"
}
if p.VacuumStrategy != store.VacuumStrategyNone {
return p.VacuumStrategy
}
return "run"
}
func rowsDeletedFromDetail(detail map[string]any) int64 {
if detail == nil {
return 0
}
switch v := detail["deleted"].(type) {
case int64:
return v
case int:
return int64(v)
case float64:
return int64(v)
default:
return 0
}
}
func (e *PolicyExecutor) runCleanup(ctx context.Context, policy *store.MaintenancePolicy, dryRun bool) (map[string]any, error) {
detail := map[string]any{"cleanup": true}
limit := NormalizeBatchLimit(policy.MaxRows)
qualTable := pgx.Identifier{policy.TableName}.Sanitize()
cond := store.NormalizeMaintenancePolicyCondition(policy.Condition)
tx, err := e.Pool.Begin(ctx)
if err != nil {
return detail, fmt.Errorf("maintenance: begin tx: %w", err)
}
defer func() { _ = tx.Rollback(ctx) }()
lockKey := advisoryKey(policy.ID)
if _, err := tx.Exec(ctx, `SELECT pg_advisory_xact_lock($1)`, lockKey); err != nil {
return detail, fmt.Errorf("maintenance: advisory lock: %w", err)
}
if _, err := tx.Exec(ctx, fmt.Sprintf(`SET LOCAL statement_timeout = '%ds'`, DefaultStatementTimeoutSec)); err != nil {
return detail, fmt.Errorf("maintenance: statement_timeout: %w", err)
}
var args []any
where := cond
argN := 1
if policy.RetentionPeriodSec != nil && *policy.RetentionPeriodSec > 0 {
cutoff := time.Now().UTC().Add(-time.Duration(*policy.RetentionPeriodSec) * time.Second)
where = fmt.Sprintf("(%s) AND created_at < $%d", cond, argN)
args = append(args, cutoff)
argN++
}
countSQL := fmt.Sprintf(`SELECT count(*) FROM %s WHERE %s`, qualTable, where)
var wouldDelete int64
if err := tx.QueryRow(ctx, countSQL, args...).Scan(&wouldDelete); err != nil {
return detail, fmt.Errorf("maintenance: count: %w", err)
}
detail["would_delete"] = wouldDelete
if dryRun {
return detail, nil
}
deleteSQL := fmt.Sprintf(`
DELETE FROM %s WHERE ctid IN (
SELECT ctid FROM %s WHERE %s LIMIT $%d
)`, qualTable, qualTable, where, argN)
args = append(args, limit)
tag, err := tx.Exec(ctx, deleteSQL, args...)
if err != nil {
return detail, fmt.Errorf("maintenance: delete: %w", err)
}
if err := tx.Commit(ctx); err != nil {
return detail, fmt.Errorf("maintenance: commit: %w", err)
}
detail["deleted"] = tag.RowsAffected()
return detail, nil
}
func vacuumKind(strategy string) string {
switch strings.TrimSpace(strategy) {
case store.VacuumStrategyVacuum:
return "vacuum"
case store.VacuumStrategyAnalyze:
return "analyze"
case store.VacuumStrategyVacuumAnalyze:
return "vacuum_analyze"
case store.VacuumStrategyReindex:
return "reindex"
default:
return "vacuum"
}
}
func advisoryKey(policyID string) int64 {
h := fnv.New64a()
_, _ = h.Write([]byte("maint:" + policyID))
return int64(h.Sum64())
}
@@ -0,0 +1,161 @@
package maintenance
import (
"context"
"errors"
"strings"
"testing"
"evobgp/internal/store"
)
func validPolicy() *store.MaintenancePolicy {
ret := 86400
return &store.MaintenancePolicy{
ID: "11111111-1111-1111-1111-111111111111",
Name: "job audit",
TableName: "job_audit",
Condition: "true",
RetentionPeriodSec: &ret,
VacuumStrategy: store.VacuumStrategyNone,
Schedule: "0 3 * * *",
Enabled: true,
}
}
func TestPolicyExecutorExecuteValidation(t *testing.T) {
ctx := context.Background()
mem := store.NewMemory()
base := validPolicy()
tests := []struct {
name string
exec *PolicyExecutor
policy *store.MaintenancePolicy
wantErr error
contains string
}{
{
name: "nil policy",
exec: &PolicyExecutor{Store: mem},
policy: nil,
wantErr: store.ErrInvalidInput,
},
{
name: "nil pool",
exec: &PolicyExecutor{Store: mem, Pool: nil},
policy: base,
contains: "postgres not configured",
},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
_, err := tc.exec.Execute(ctx, tc.policy, false)
if tc.wantErr != nil {
if !errors.Is(err, tc.wantErr) {
t.Fatalf("Execute() err=%v want %v", err, tc.wantErr)
}
return
}
if err == nil {
t.Fatal("Execute() expected error")
}
if tc.contains != "" && !strings.Contains(err.Error(), tc.contains) {
t.Fatalf("Execute() err=%q want substring %q", err, tc.contains)
}
})
}
}
func TestPolicyExecutorPreExecuteValidation(t *testing.T) {
p := validPolicy()
p.TableName = "tenant"
if err := ValidateTableName(p.TableName); err == nil {
t.Fatal("expected blocked table error")
}
p = validPolicy()
p.Condition = "1=1; DROP TABLE job_audit"
if err := ValidateCondition(p.Condition); err == nil {
t.Fatal("expected unsafe condition error")
}
p = validPolicy()
p.VacuumStrategy = "invalid"
if !store.ValidVacuumStrategy(p.VacuumStrategy) {
return
}
t.Fatal("expected invalid vacuum strategy")
}
func TestPolicyAction(t *testing.T) {
ret := 3600
tests := []struct {
name string
p *store.MaintenancePolicy
want string
}{
{"nil", nil, "run"},
{"cleanup only", &store.MaintenancePolicy{RetentionPeriodSec: &ret, VacuumStrategy: store.VacuumStrategyNone}, "cleanup"},
{"vacuum only", &store.MaintenancePolicy{VacuumStrategy: store.VacuumStrategyVacuum}, "vacuum"},
{"cleanup+vacuum", &store.MaintenancePolicy{MaxRows: &ret, VacuumStrategy: store.VacuumStrategyAnalyze}, "cleanup_vacuum"},
{"noop run", &store.MaintenancePolicy{VacuumStrategy: store.VacuumStrategyNone}, "run"},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := policyAction(tc.p); got != tc.want {
t.Fatalf("policyAction()=%q want %q", got, tc.want)
}
})
}
}
func TestVacuumKind(t *testing.T) {
tests := []struct {
strategy string
want string
}{
{store.VacuumStrategyVacuum, "vacuum"},
{store.VacuumStrategyAnalyze, "analyze"},
{store.VacuumStrategyVacuumAnalyze, "vacuum_analyze"},
{store.VacuumStrategyReindex, "reindex"},
{"unknown", "vacuum"},
}
for _, tc := range tests {
if got := vacuumKind(tc.strategy); got != tc.want {
t.Fatalf("vacuumKind(%q)=%q want %q", tc.strategy, got, tc.want)
}
}
}
func TestRowsDeletedFromDetail(t *testing.T) {
tests := []struct {
name string
detail map[string]any
want int64
}{
{"nil", nil, 0},
{"int64", map[string]any{"deleted": int64(42)}, 42},
{"int", map[string]any{"deleted": 7}, 7},
{"float64", map[string]any{"deleted": float64(3)}, 3},
{"missing", map[string]any{"other": 1}, 0},
}
for _, tc := range tests {
t.Run(tc.name, func(t *testing.T) {
if got := rowsDeletedFromDetail(tc.detail); got != tc.want {
t.Fatalf("rowsDeletedFromDetail()=%d want %d", got, tc.want)
}
})
}
}
func TestAdvisoryKeyStable(t *testing.T) {
a := advisoryKey("policy-a")
b := advisoryKey("policy-a")
c := advisoryKey("policy-b")
if a != b {
t.Fatal("advisory key not stable for same id")
}
if a == c {
t.Fatal("advisory key collision for different ids")
}
}
+72
View File
@@ -0,0 +1,72 @@
package maintenance
import (
"fmt"
"regexp"
"strings"
)
const (
DefaultBatchRows = 10000
MaxBatchRows = 100000
DefaultStatementTimeoutSec = 30
)
var (
blockedTableNames = map[string]struct{}{
"schema_migrations": {},
"tenant": {},
"maintenance_policy": {},
"maintenance_policy_config_audit": {},
}
sqlForbidden = regexp.MustCompile(`(?i)(;|--|/\*|\b(drop|truncate|insert|update|alter|create|grant|revoke|copy)\b)`)
)
// ValidateTableName ensures table is a safe identifier and not blocked.
func ValidateTableName(name string) error {
name = strings.TrimSpace(name)
if name == "" || !isSafeIdent(name) {
return fmt.Errorf("maintenance: invalid table name")
}
if _, blocked := blockedTableNames[strings.ToLower(name)]; blocked {
return fmt.Errorf("maintenance: table %q is not allowed", name)
}
return nil
}
// ValidateCondition ensures the WHERE fragment is safe for parameterized cleanup.
func ValidateCondition(condition string) error {
c := strings.TrimSpace(condition)
if c == "" {
return nil
}
if sqlForbidden.MatchString(c) {
return fmt.Errorf("maintenance: unsafe condition")
}
return nil
}
// NormalizeBatchLimit clamps delete batch size.
func NormalizeBatchLimit(maxRows *int) int {
if maxRows == nil || *maxRows <= 0 {
return DefaultBatchRows
}
if *maxRows > MaxBatchRows {
return MaxBatchRows
}
return *maxRows
}
func isSafeIdent(name string) bool {
if name == "" {
return false
}
for _, r := range name {
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
continue
}
return false
}
return true
}
+46
View File
@@ -0,0 +1,46 @@
package maintenance
import "testing"
func TestValidateCondition(t *testing.T) {
tests := []struct {
cond string
ok bool
}{
{"true", true},
{"status IN ('succeeded', 'failed')", true},
{"1=1; DROP TABLE tenant", false},
{"x -- comment", false},
}
for _, tc := range tests {
err := ValidateCondition(tc.cond)
if tc.ok && err != nil {
t.Fatalf("cond %q: want ok, got %v", tc.cond, err)
}
if !tc.ok && err == nil {
t.Fatalf("cond %q: want error", tc.cond)
}
}
}
func TestValidateTableName(t *testing.T) {
if err := ValidateTableName("job_audit"); err != nil {
t.Fatal(err)
}
if err := ValidateTableName("tenant"); err == nil {
t.Fatal("expected blocked table")
}
if err := ValidateTableName("bad-name"); err == nil {
t.Fatal("expected invalid ident")
}
}
func TestNormalizeBatchLimit(t *testing.T) {
if got := NormalizeBatchLimit(nil); got != DefaultBatchRows {
t.Fatalf("default=%d got=%d", DefaultBatchRows, got)
}
max := 200000
if got := NormalizeBatchLimit(&max); got != MaxBatchRows {
t.Fatalf("max=%d got=%d", MaxBatchRows, got)
}
}
+85
View File
@@ -0,0 +1,85 @@
package maintenance
import (
"context"
"fmt"
"log"
"strings"
"sync"
"time"
"github.com/robfig/cron/v3"
)
// StartScheduler enqueues maintenance_policy_run jobs when cron schedules match.
func StartScheduler(ctx context.Context, provider *ConfigProvider, enqueue func(policyID string, dryRun bool, idempotencyKey string), tick time.Duration) {
if provider == nil || enqueue == nil {
return
}
if tick <= 0 {
tick = 30 * time.Second
}
go func() {
parser := cron.NewParser(cron.Minute | cron.Hour | cron.Dom | cron.Month | cron.Dow)
var mu sync.Mutex
schedules := map[string]cron.Schedule{}
lastFired := map[string]time.Time{}
rebuild := func() {
mu.Lock()
defer mu.Unlock()
schedules = map[string]cron.Schedule{}
for _, p := range provider.Snapshot() {
if p == nil || !p.Enabled || strings.TrimSpace(p.Schedule) == "" {
continue
}
sched, err := parser.Parse(p.Schedule)
if err != nil {
log.Printf("maintenance: invalid cron for policy %s: %v", p.ID, err)
continue
}
schedules[p.ID] = sched
}
}
rebuild()
t := time.NewTicker(tick)
defer t.Stop()
for {
select {
case <-ctx.Done():
return
case <-t.C:
rebuild()
now := time.Now().UTC()
mu.Lock()
for _, p := range provider.Snapshot() {
if p == nil || !p.Enabled {
continue
}
sched, ok := schedules[p.ID]
if !ok {
continue
}
prev := lastFired[p.ID]
if prev.IsZero() {
prev = now.Add(-time.Minute)
}
next := sched.Next(prev)
if next.After(now) {
continue
}
slot := next.Unix() / 60
if lf, ok := lastFired[p.ID]; ok && lf.Unix()/60 == slot {
continue
}
lastFired[p.ID] = next
idem := fmt.Sprintf("maint-%s-%d", p.ID, slot)
enqueue(p.ID, p.DryRunEnabled, idem)
}
mu.Unlock()
}
}
}()
log.Printf("maintenance: policy scheduler started (tick=%s)", tick)
}
@@ -0,0 +1,72 @@
package observability
import (
"strconv"
"time"
"github.com/prometheus/client_golang/prometheus"
"github.com/prometheus/client_golang/prometheus/promauto"
)
var (
maintenancePolicyRuns = promauto.NewCounterVec(
prometheus.CounterOpts{
Namespace: namespace,
Name: "maintenance_policy_runs_total",
Help: "Maintenance policy executions by outcome.",
},
[]string{"policy_id", "action", "status", "dry_run"},
)
maintenancePolicyDuration = promauto.NewHistogramVec(
prometheus.HistogramOpts{
Namespace: namespace,
Name: "maintenance_policy_duration_seconds",
Help: "Maintenance policy execution duration.",
Buckets: prometheus.ExponentialBuckets(0.05, 2, 12),
},
[]string{"policy_id", "action"},
)
maintenanceRowsDeleted = promauto.NewCounterVec(
prometheus.CounterOpts{
Namespace: namespace,
Name: "maintenance_policy_rows_deleted_total",
Help: "Rows deleted by maintenance cleanup policies.",
},
[]string{"policy_id"},
)
maintenanceConfigChanges = promauto.NewCounterVec(
prometheus.CounterOpts{
Namespace: namespace,
Name: "maintenance_config_changes_total",
Help: "Maintenance policy configuration changes from UI/API.",
},
[]string{"action"},
)
)
// RecordMaintenancePolicyRun updates run counters and histograms.
func RecordMaintenancePolicyRun(policyID, action, status string, dryRun bool, duration time.Duration, rowsDeleted int64) {
if policyID == "" {
policyID = "unknown"
}
if action == "" {
action = "run"
}
dry := strconv.FormatBool(dryRun)
maintenancePolicyRuns.WithLabelValues(policyID, action, status, dry).Inc()
maintenancePolicyDuration.WithLabelValues(policyID, action).Observe(duration.Seconds())
if rowsDeleted > 0 && !dryRun {
maintenanceRowsDeleted.WithLabelValues(policyID).Add(float64(rowsDeleted))
}
}
// IncMaintenanceConfigChange increments config audit metric.
func IncMaintenanceConfigChange(action string) {
if action == "" {
action = "unknown"
}
maintenanceConfigChanges.WithLabelValues(action).Inc()
}
+37
View File
@@ -0,0 +1,37 @@
package pgmonitor
import (
"sync"
"time"
)
type cacheEntry struct {
at time.Time
data any
}
type ttlCache struct {
mu sync.RWMutex
ttl time.Duration
items map[string]cacheEntry
}
func newTTLCache(ttl time.Duration) *ttlCache {
return &ttlCache{ttl: ttl, items: make(map[string]cacheEntry)}
}
func (c *ttlCache) get(key string) (any, bool) {
c.mu.RLock()
defer c.mu.RUnlock()
e, ok := c.items[key]
if !ok || time.Since(e.at) > c.ttl {
return nil, false
}
return e.data, true
}
func (c *ttlCache) set(key string, data any) {
c.mu.Lock()
defer c.mu.Unlock()
c.items[key] = cacheEntry{at: time.Now().UTC(), data: data}
}
+83
View File
@@ -0,0 +1,83 @@
package pgmonitor
import (
"context"
"fmt"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// Correlation builds aligned timeline points from job_audit and overview cache.
func (s *Service) Correlation(ctx context.Context, windowMinutes int) (CorrelationResponse, error) {
if s == nil || s.pool == nil {
return CorrelationResponse{}, fmt.Errorf("pgmonitor: postgres not configured")
}
if windowMinutes <= 0 {
windowMinutes = 60
}
if windowMinutes > 1440 {
windowMinutes = 1440
}
since := time.Now().UTC().Add(-time.Duration(windowMinutes) * time.Minute)
rows, err := s.pool.Query(ctx, `
SELECT date_trunc('minute', finished_at) AS bucket,
percentile_cont(0.99) WITHIN GROUP (ORDER BY
EXTRACT(EPOCH FROM (finished_at - started_at)) * 1000)
FROM job_audit
WHERE finished_at >= $1 AND kind IN ('module_refresh', 'tenant_refresh')
AND status = 'succeeded' AND started_at IS NOT NULL
GROUP BY 1
ORDER BY 1`, since)
if err != nil {
return CorrelationResponse{}, fmt.Errorf("pgmonitor: correlation jobs: %w", err)
}
defer rows.Close()
points := make(map[time.Time]*CorrelationPoint)
for rows.Next() {
var bucket time.Time
var p99 *float64
if err := rows.Scan(&bucket, &p99); err != nil {
return CorrelationResponse{}, err
}
bucket = bucket.UTC()
pt := points[bucket]
if pt == nil {
pt = &CorrelationPoint{Timestamp: bucket}
points[bucket] = pt
}
if p99 != nil {
pt.PipelineRefreshP99Ms = *p99
}
}
ov, err := s.Overview(ctx)
if err == nil && ov.Database.CacheHitPct > 0 {
now := time.Now().UTC().Truncate(time.Minute)
pt := points[now]
if pt == nil {
pt = &CorrelationPoint{Timestamp: now}
points[now] = pt
}
pt.CacheHitPct = ov.Database.CacheHitPct
}
out := make([]CorrelationPoint, 0, len(points))
for _, p := range points {
out = append(out, *p)
}
// simple sort by time
for i := 0; i < len(out); i++ {
for j := i + 1; j < len(out); j++ {
if out[j].Timestamp.Before(out[i].Timestamp) {
out[i], out[j] = out[j], out[i]
}
}
}
return CorrelationResponse{WindowMinutes: windowMinutes, Points: out}, nil
}
// RecordCorrelationSnapshot is a hook for future Prometheus samples (no-op placeholder).
func RecordCorrelationSnapshot(_ *pgxpool.Pool) {}
+96
View File
@@ -0,0 +1,96 @@
package pgmonitor
import (
"context"
"encoding/json"
"time"
"github.com/google/uuid"
"github.com/jackc/pgx/v5/pgxpool"
)
// CleanupRequest for deprecated POST /postgres/cleanup (use /v1/maintenance/run).
type CleanupRequest struct {
PolicyID string `json:"policy_id"`
Policy string `json:"policy"`
DryRun bool `json:"dry_run"`
Limit int `json:"limit"`
}
// InsertMaintenanceAudit records an audit row at job start.
func InsertMaintenanceAudit(ctx context.Context, pool *pgxpool.Pool, tenantID, actorPrefix, kind, table string, dryRun bool) (string, error) {
return InsertMaintenanceAuditWithPolicy(ctx, pool, tenantID, actorPrefix, kind, table, "", dryRun)
}
// InsertMaintenanceAuditWithPolicy records an audit row linked to maintenance_policy.
func InsertMaintenanceAuditWithPolicy(ctx context.Context, pool *pgxpool.Pool, tenantID, actorPrefix, kind, table, policyID string, dryRun bool) (string, error) {
id := uuid.New().String()
_, err := pool.Exec(ctx, `
INSERT INTO postgres_maintenance_audit
(id, tenant_id, actor_prefix, kind, target_table, policy_id, dry_run, status, created_at)
VALUES ($1, NULLIF($2,''), NULLIF($3,''), $4, NULLIF($5,''), NULLIF($6,''), $7, 'running', now())`,
id, tenantID, actorPrefix, kind, table, policyID, dryRun)
return id, err
}
// FinishMaintenanceAudit updates terminal state.
func FinishMaintenanceAudit(ctx context.Context, pool *pgxpool.Pool, id, status string, detail map[string]any, errMsg *string) error {
var detailJSON []byte
if detail != nil {
detailJSON, _ = json.Marshal(detail)
}
_, err := pool.Exec(ctx, `
UPDATE postgres_maintenance_audit
SET status = $2, detail_json = $3::jsonb, error_message = $4,
finished_at = now(), started_at = COALESCE(started_at, now())
WHERE id = $1`,
id, status, string(detailJSON), errMsg)
return err
}
// ListMaintenanceLogs returns paginated audit rows.
func ListMaintenanceLogs(ctx context.Context, pool *pgxpool.Pool, cursor string, limit int) ([]MaintenanceLogRow, string, bool, error) {
limit = clampLimit(limit, 20, 100)
args := []any{limit + 1}
q := `
SELECT id, COALESCE(tenant_id,''), COALESCE(actor_prefix,''), kind,
COALESCE(target_table,''), dry_run, status,
detail_json, COALESCE(error_message,''), created_at, started_at, finished_at
FROM postgres_maintenance_audit`
if cursor != "" {
q += ` WHERE created_at < (SELECT created_at FROM postgres_maintenance_audit WHERE id = $2)`
args = append(args, cursor)
}
q += ` ORDER BY created_at DESC LIMIT $1`
rows, err := pool.Query(ctx, q, args...)
if err != nil {
return nil, "", false, err
}
defer rows.Close()
var out []MaintenanceLogRow
for rows.Next() {
var r MaintenanceLogRow
var detailRaw []byte
var started, finished *time.Time
if err := rows.Scan(&r.ID, &r.TenantID, &r.ActorPrefix, &r.Kind, &r.TargetTable,
&r.DryRun, &r.Status, &detailRaw, &r.Error, &r.CreatedAt, &started, &finished); err != nil {
return nil, "", false, err
}
r.StartedAt = started
r.FinishedAt = finished
if len(detailRaw) > 0 {
_ = json.Unmarshal(detailRaw, &r.Detail)
}
out = append(out, r)
}
hasMore := len(out) > limit
if hasMore {
out = out[:limit]
}
next := ""
if hasMore && len(out) > 0 {
next = out[len(out)-1].ID
}
return out, next, hasMore, rows.Err()
}
+350
View File
@@ -0,0 +1,350 @@
package pgmonitor
import (
"context"
"errors"
"fmt"
"strings"
"time"
"github.com/jackc/pgx/v5"
"github.com/jackc/pgx/v5/pgconn"
"github.com/jackc/pgx/v5/pgxpool"
)
func clampLimit(limit, def, max int) int {
if limit <= 0 {
return def
}
if limit > max {
return max
}
return limit
}
func (s *Service) fetchOverview(ctx context.Context) (Overview, error) {
now := time.Now().UTC()
out := Overview{CollectedAt: now}
var active, idle, total, maxConn int
err := s.pool.QueryRow(ctx, `
SELECT
count(*) FILTER (WHERE state = 'active'),
count(*) FILTER (WHERE state = 'idle'),
count(*),
(SELECT setting::int FROM pg_settings WHERE name = 'max_connections')
FROM pg_stat_activity
WHERE datname = current_database()`).Scan(&active, &idle, &total, &maxConn)
if err != nil {
return out, fmt.Errorf("pgmonitor: connections: %w", err)
}
out.Connections = Connections{Active: active, Idle: idle, Total: total, MaxConnections: maxConn}
var cachePct *float64
err = s.pool.QueryRow(ctx, `
SELECT numbackends, xact_commit, xact_rollback, deadlocks, blks_hit, blks_read,
CASE WHEN blks_hit + blks_read > 0
THEN round(100.0 * blks_hit::numeric / (blks_hit + blks_read), 2) END
FROM pg_stat_database WHERE datname = current_database()`).Scan(
&out.Database.Backends,
&out.Database.XactCommit,
&out.Database.XactRollback,
&out.Database.Deadlocks,
&out.Database.BlksHit,
&out.Database.BlksRead,
&cachePct,
)
if err != nil {
return out, fmt.Errorf("pgmonitor: database stats: %w", err)
}
if cachePct != nil {
out.Database.CacheHitPct = *cachePct
}
_ = s.pool.QueryRow(ctx, `
SELECT checkpoints_timed, checkpoints_req, buffers_checkpoint, buffers_clean,
maxwritten_clean, buffers_backend, buffers_alloc
FROM pg_stat_bgwriter`).Scan(
&out.Bgwriter.CheckpointsTimed,
&out.Bgwriter.CheckpointsReq,
&out.Bgwriter.BuffersCheckpoint,
&out.Bgwriter.BuffersClean,
&out.Bgwriter.MaxWrittenClean,
&out.Bgwriter.BuffersBackend,
&out.Bgwriter.BuffersAlloc,
)
_ = s.pool.QueryRow(ctx, `SELECT pg_database_size(current_database())`).Scan(&out.SizeBytes)
_ = s.pool.QueryRow(ctx, `
SELECT
(SELECT setting FROM pg_settings WHERE name = 'shared_buffers'),
(SELECT setting FROM pg_settings WHERE name = 'work_mem'),
(SELECT setting FROM pg_settings WHERE name = 'effective_cache_size')`).Scan(
&out.MemorySettings.SharedBuffers,
&out.MemorySettings.WorkMem,
&out.MemorySettings.EffectiveCacheSize,
)
rows, err := s.pool.Query(ctx, `
SELECT client_addr::text, state, sync_state,
EXTRACT(EPOCH FROM COALESCE(write_lag, flush_lag, replay_lag)) * 1000
FROM pg_stat_replication`)
if err == nil {
defer rows.Close()
for rows.Next() {
var peer ReplicationPeer
var lagMs *float64
if err := rows.Scan(&peer.ClientAddr, &peer.State, &peer.SyncState, &lagMs); err != nil {
continue
}
if lagMs != nil {
v := int64(*lagMs)
peer.LagMs = &v
}
out.Replication = append(out.Replication, peer)
}
}
out.StatementsEnabled = s.statementsQueryable(ctx)
return out, nil
}
func queryLocks(ctx context.Context, pool *pgxpool.Pool) ([]LockRow, error) {
rows, err := pool.Query(ctx, `
SELECT l.locktype, l.mode, l.granted, a.pid, COALESCE(a.usename, ''),
COALESCE(a.state, ''), COALESCE(left(a.query, 300), ''),
NOT l.granted AS blocked
FROM pg_locks l
JOIN pg_stat_activity a ON a.pid = l.pid
WHERE a.datname = current_database()
AND (NOT l.granted OR l.mode LIKE '%Exclusive%')
ORDER BY l.granted ASC, a.query_start NULLS LAST
LIMIT 200`)
if err != nil {
return nil, fmt.Errorf("pgmonitor: locks: %w", err)
}
defer rows.Close()
var out []LockRow
for rows.Next() {
var r LockRow
if err := rows.Scan(&r.Locktype, &r.Mode, &r.Granted, &r.PID, &r.User, &r.State, &r.Query, &r.Blocked); err != nil {
return nil, err
}
out = append(out, r)
}
return out, rows.Err()
}
func queryTables(ctx context.Context, pool *pgxpool.Pool, limit int) ([]TableStat, error) {
limit = clampLimit(limit, 20, 100)
rows, err := pool.Query(ctx, `
SELECT t.relname,
pg_total_relation_size(t.relid),
s.heap_blks_read, s.heap_blks_hit,
t.idx_scan, t.seq_scan, t.n_dead_tup, t.last_autovacuum,
CASE WHEN t.n_live_tup + t.n_dead_tup > 0
THEN round(t.n_dead_tup::numeric / (t.n_live_tup + t.n_dead_tup), 4)
ELSE 0 END
FROM pg_statio_user_tables s
JOIN pg_stat_user_tables t ON t.relid = s.relid
WHERE t.schemaname = 'public'
ORDER BY pg_total_relation_size(t.relid) DESC
LIMIT $1`, limit)
if err != nil {
return nil, fmt.Errorf("pgmonitor: tables: %w", err)
}
defer rows.Close()
var out []TableStat
for rows.Next() {
var r TableStat
var last *time.Time
if err := rows.Scan(&r.Relname, &r.TotalBytes, &r.HeapBlksRead, &r.HeapBlksHit,
&r.IdxScan, &r.SeqScan, &r.DeadTuples, &last, &r.BloatRatio); err != nil {
return nil, err
}
r.LastAutovacuum = last
out = append(out, r)
}
return out, rows.Err()
}
// TopQueries loads from pg_stat_statements when available.
func (s *Service) TopQueries(ctx context.Context, limit int) (QueriesResponse, error) {
if s == nil || s.pool == nil {
return QueriesResponse{}, errors.New("pgmonitor: postgres not configured")
}
limit = clampLimit(limit, 20, 100)
now := time.Now().UTC()
if snap, ok, err := s.loadSnapshot(ctx, "slow_queries", 15*time.Minute); err == nil && ok {
var items []QueryStat
if err := decodePayload(snap.Payload, &items); err == nil {
return QueriesResponse{
CollectedAt: snap.CollectedAt,
Source: "snapshot",
Items: items,
StatementsAvailable: true,
}, nil
}
}
if !s.statementsQueryable(ctx) {
return queriesUnavailable(now), nil
}
items, err := queryTopStatements(ctx, s.pool, limit)
if err != nil {
if isPgStatStatementsUnavailable(err) {
s.markStatementsUnavailable()
return queriesUnavailable(now), nil
}
return QueriesResponse{}, err
}
return QueriesResponse{
CollectedAt: now,
Source: "live",
Items: items,
StatementsAvailable: true,
}, nil
}
func queriesUnavailable(at time.Time) QueriesResponse {
return QueriesResponse{
CollectedAt: at,
Source: "unavailable",
Items: nil,
StatementsAvailable: false,
StatementsHint: statementsUnavailableHint,
}
}
const statementsUnavailableHint = "pg_stat_statements requires shared_preload_libraries and PostgreSQL restart (see docs/db-diagnostics.md)"
// statementsQueryable returns true only when pg_stat_statements can be queried (not merely installed).
func (s *Service) statementsQueryable(ctx context.Context) bool {
if s == nil || s.pool == nil {
return false
}
if v, ok := s.cache.get("stmt_queryable"); ok {
if b, ok := v.(bool); ok {
return b
}
}
ok := probePgStatStatements(ctx, s.pool)
s.cache.set("stmt_queryable", ok)
return ok
}
func (s *Service) markStatementsUnavailable() {
s.cache.set("stmt_queryable", false)
}
func probePgStatStatements(ctx context.Context, pool *pgxpool.Pool) bool {
var dummy int64
err := pool.QueryRow(ctx, `
SELECT COALESCE(SUM(calls), 0)::bigint FROM pg_stat_statements LIMIT 1`).Scan(&dummy)
if err == nil {
return true
}
return !isPgStatStatementsUnavailable(err)
}
func queryTopStatements(ctx context.Context, pool *pgxpool.Pool, limit int) ([]QueryStat, error) {
rows, err := pool.Query(ctx, `
SELECT queryid, left(query, 500), calls, total_exec_time, mean_exec_time, rows
FROM pg_stat_statements
WHERE dbid = (SELECT oid FROM pg_database WHERE datname = current_database())
ORDER BY mean_exec_time DESC
LIMIT $1`, limit)
if err != nil {
if isPgStatStatementsUnavailable(err) {
return nil, nil
}
return nil, fmt.Errorf("pgmonitor: pg_stat_statements: %w", err)
}
defer rows.Close()
var out []QueryStat
for rows.Next() {
var r QueryStat
if err := rows.Scan(&r.QueryID, &r.Query, &r.Calls, &r.TotalExecMs, &r.MeanExecMs, &r.Rows); err != nil {
return nil, err
}
out = append(out, r)
}
return out, rows.Err()
}
// isPgStatStatementsUnavailable reports extension missing or not loaded via shared_preload_libraries.
func isPgStatStatementsUnavailable(err error) bool {
if err == nil {
return false
}
var pgErr *pgconn.PgError
if errors.As(err, &pgErr) {
switch pgErr.Code {
case "42P01", "42704", "55000":
return true
}
msg := strings.ToLower(pgErr.Message)
if strings.Contains(msg, "shared_preload_libraries") || strings.Contains(msg, "pg_stat_statements") {
return true
}
}
low := strings.ToLower(err.Error())
return strings.Contains(low, "shared_preload_libraries") || strings.Contains(low, "pg_stat_statements")
}
func isSafeIdent(name string) bool {
if name == "" {
return true
}
for _, r := range name {
if (r >= 'a' && r <= 'z') || (r >= 'A' && r <= 'Z') || (r >= '0' && r <= '9') || r == '_' {
continue
}
return false
}
return true
}
// ExecMaintenance runs VACUUM/ANALYZE/REINDEX with optional dry-run (returns SQL executed or planned).
func ExecMaintenance(ctx context.Context, pool *pgxpool.Pool, kind, table string, dryRun bool) (detail map[string]any, err error) {
if pool == nil {
return nil, errors.New("pgmonitor: postgres not configured")
}
table = strings.TrimSpace(table)
if table != "" && !isSafeIdent(table) {
return nil, errors.New("pgmonitor: invalid table name")
}
qual := ""
if table != "" {
qual = " " + pgx.Identifier{table}.Sanitize()
}
var sql string
switch kind {
case "vacuum":
sql = "VACUUM" + qual
case "vacuum_analyze":
sql = "VACUUM ANALYZE" + qual
case "analyze":
sql = "ANALYZE" + qual
case "reindex":
if table == "" {
return nil, errors.New("pgmonitor: reindex requires table")
}
sql = "REINDEX TABLE" + qual
default:
return nil, fmt.Errorf("pgmonitor: unknown maintenance kind %q", kind)
}
detail = map[string]any{"sql": sql, "dry_run": dryRun}
if dryRun {
return detail, nil
}
_, err = pool.Exec(ctx, sql)
if err != nil {
return detail, fmt.Errorf("pgmonitor: %s: %w", kind, err)
}
detail["executed"] = true
return detail, nil
}
+104
View File
@@ -0,0 +1,104 @@
package pgmonitor
import (
"context"
"time"
)
// Recommendations builds heuristic items from live stats and snapshots.
func (s *Service) Recommendations(ctx context.Context) (RecommendationsResponse, error) {
now := time.Now().UTC()
var items []RecommendationItem
ov, err := s.Overview(ctx)
if err == nil {
if ov.Database.CacheHitPct > 0 && ov.Database.CacheHitPct < 90 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "low_cache_hit",
Title: "Низкий cache hit ratio",
Detail: "Buffer cache hit ниже 90%; проверьте shared_buffers и горячие seq scan.",
})
}
if ov.Database.Deadlocks > 0 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "deadlocks",
Title: "Зафиксированы deadlocks",
Detail: "Проверьте конкурирующие транзакции и порядок блокировок.",
})
}
if ov.Connections.MaxConnections > 0 &&
float64(ov.Connections.Total)/float64(ov.Connections.MaxConnections) > 0.8 {
items = append(items, RecommendationItem{
Severity: "critical",
Code: "connections_high",
Title: "Много подключений к PostgreSQL",
Detail: "Использование max_connections выше 80%; увеличьте pool tuning или лимит.",
})
}
}
tables, err := s.Tables(ctx, 30)
if err == nil {
for _, t := range tables {
if t.SeqScan > 1000 && t.IdxScan < t.SeqScan/10 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "missing_index",
Title: "Высокий seq_scan",
Detail: "Таблица часто сканируется последовательно; рассмотрите индекс.",
Refs: []string{t.Relname},
})
}
if t.BloatRatio > 0.2 && t.DeadTuples > 5000 {
items = append(items, RecommendationItem{
Severity: "info",
Code: "autovacuum_lag",
Title: "Возможный bloat / мёртвые строки",
Detail: "Высокая доля n_dead_tup; запланируйте VACUUM.",
Refs: []string{t.Relname},
})
}
}
}
if snap, ok, _ := s.loadSnapshot(ctx, "unused_indexes", 30*time.Minute); ok {
type unused struct {
Index string `json:"index"`
SizeBytes int64 `json:"size_bytes"`
}
var list []unused
if decodePayload(snap.Payload, &list) == nil {
for _, u := range list {
if u.SizeBytes < 1024*1024 {
continue
}
items = append(items, RecommendationItem{
Severity: "info",
Code: "unused_index",
Title: "Неиспользуемый индекс",
Detail: "idx_scan=0; проверьте перед удалением.",
Refs: []string{u.Index},
})
}
}
}
q, err := s.TopQueries(ctx, 5)
if err == nil {
for _, qs := range q.Items {
if qs.MeanExecMs > 500 {
items = append(items, RecommendationItem{
Severity: "warn",
Code: "slow_query",
Title: "Медленный запрос",
Detail: "Среднее время выполнения выше 500ms.",
Refs: []string{qs.Query},
})
}
}
}
return RecommendationsResponse{CollectedAt: now, Items: items}, nil
}
+59
View File
@@ -0,0 +1,59 @@
package pgmonitor
import (
"context"
"log"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// StartScheduler runs periodic PostgreSQL analyzer snapshots until ctx is cancelled.
func StartScheduler(ctx context.Context, pool *pgxpool.Pool) {
if pool == nil {
return
}
go func() {
t5 := time.NewTicker(5 * time.Minute)
t15 := time.NewTicker(15 * time.Minute)
defer t5.Stop()
defer t15.Stop()
s := NewService(pool)
runLight := func() {
c, cancel := context.WithTimeout(context.Background(), 2*time.Minute)
defer cancel()
if err := s.RefreshMetricsSnapshot(c); err != nil {
log.Printf("pgmonitor: metrics refresh: %v", err)
}
if err := s.DetectAutovacuumLag(c); err != nil {
log.Printf("pgmonitor: autovacuum lag: %v", err)
}
}
runHeavy := func() {
c, cancel := context.WithTimeout(context.Background(), 3*time.Minute)
defer cancel()
if err := s.AggregateSlowQueries(c, 30); err != nil {
log.Printf("pgmonitor: slow queries snapshot: %v", err)
}
if err := s.EstimateTableBloat(c); err != nil {
log.Printf("pgmonitor: bloat: %v", err)
}
if err := s.AnalyzeIndexUsage(c); err != nil {
log.Printf("pgmonitor: index usage: %v", err)
}
}
runLight()
runHeavy()
for {
select {
case <-ctx.Done():
return
case <-t5.C:
runLight()
case <-t15.C:
runHeavy()
}
}
}()
log.Printf("pgmonitor: scheduler started (5m light / 15m heavy)")
}
+90
View File
@@ -0,0 +1,90 @@
package pgmonitor
import (
"context"
"errors"
"fmt"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// Service provides PostgreSQL observability and maintenance helpers (control plane instance scope).
type Service struct {
pool *pgxpool.Pool
cache *ttlCache
}
// NewService constructs a metrics service for the API PostgreSQL pool.
func NewService(pool *pgxpool.Pool) *Service {
if pool == nil {
return nil
}
return &Service{
pool: pool,
cache: newTTLCache(10 * time.Second),
}
}
// Pool exposes the underlying pool for job workers.
func (s *Service) Pool() *pgxpool.Pool {
if s == nil {
return nil
}
return s.pool
}
// Overview returns cached instance-level stats.
func (s *Service) Overview(ctx context.Context) (Overview, error) {
if s == nil || s.pool == nil {
return Overview{}, errors.New("pgmonitor: postgres not configured")
}
if v, ok := s.cache.get("overview"); ok {
if o, ok := v.(Overview); ok {
return o, nil
}
}
o, err := s.fetchOverview(ctx)
if err != nil {
return Overview{}, err
}
s.cache.set("overview", o)
return o, nil
}
// Locks returns active / blocking locks.
func (s *Service) Locks(ctx context.Context) ([]LockRow, error) {
if s == nil || s.pool == nil {
return nil, errors.New("pgmonitor: postgres not configured")
}
if v, ok := s.cache.get("locks"); ok {
if rows, ok := v.([]LockRow); ok {
return rows, nil
}
}
rows, err := queryLocks(ctx, s.pool)
if err != nil {
return nil, err
}
s.cache.set("locks", rows)
return rows, nil
}
// Tables returns top tables by size with I/O stats.
func (s *Service) Tables(ctx context.Context, limit int) ([]TableStat, error) {
if s == nil || s.pool == nil {
return nil, errors.New("pgmonitor: postgres not configured")
}
key := fmt.Sprintf("tables:%d", limit)
if v, ok := s.cache.get(key); ok {
if rows, ok := v.([]TableStat); ok {
return rows, nil
}
}
rows, err := queryTables(ctx, s.pool, limit)
if err != nil {
return nil, err
}
s.cache.set(key, rows)
return rows, nil
}
+48
View File
@@ -0,0 +1,48 @@
package pgmonitor
import (
"errors"
"testing"
"github.com/jackc/pgx/v5/pgconn"
)
func TestClampLimit(t *testing.T) {
if clampLimit(0, 20, 100) != 20 {
t.Fatal("default")
}
if clampLimit(200, 20, 100) != 100 {
t.Fatal("max")
}
if clampLimit(5, 20, 100) != 5 {
t.Fatal("value")
}
}
func TestIsSafeIdent(t *testing.T) {
if !isSafeIdent("revision_materialized_prefix") {
t.Fatal("valid")
}
if isSafeIdent("bad-name") {
t.Fatal("invalid")
}
if !isSafeIdent("") {
t.Fatal("empty ok")
}
}
func TestNewServiceNilPool(t *testing.T) {
if NewService(nil) != nil {
t.Fatal("expected nil service")
}
}
func TestIsPgStatStatementsUnavailable(t *testing.T) {
err := &pgconn.PgError{Code: "55000", Message: "pg_stat_statements must be loaded via shared_preload_libraries"}
if !isPgStatStatementsUnavailable(err) {
t.Fatal("55000")
}
if isPgStatStatementsUnavailable(errors.New("other")) {
t.Fatal("unrelated")
}
}
+162
View File
@@ -0,0 +1,162 @@
package pgmonitor
import (
"context"
"encoding/json"
"fmt"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
type snapshotRow struct {
ID string
CollectedAt time.Time
Payload json.RawMessage
}
func (s *Service) loadSnapshot(ctx context.Context, id string, maxAge time.Duration) (snapshotRow, bool, error) {
var row snapshotRow
err := s.pool.QueryRow(ctx, `
SELECT id, collected_at, payload_json
FROM postgres_monitor_snapshot
WHERE id = $1 AND collected_at >= $2`,
id, time.Now().UTC().Add(-maxAge)).Scan(&row.ID, &row.CollectedAt, &row.Payload)
if err != nil {
return snapshotRow{}, false, nil
}
return row, true, nil
}
func (s *Service) UpsertSnapshot(ctx context.Context, id string, payload any) error {
if s == nil || s.pool == nil {
return fmt.Errorf("pgmonitor: postgres not configured")
}
b, err := json.Marshal(payload)
if err != nil {
return err
}
_, err = s.pool.Exec(ctx, `
INSERT INTO postgres_monitor_snapshot (id, collected_at, payload_json)
VALUES ($1, now(), $2::jsonb)
ON CONFLICT (id) DO UPDATE SET collected_at = EXCLUDED.collected_at, payload_json = EXCLUDED.payload_json`,
id, string(b))
return err
}
func decodePayload(raw json.RawMessage, dest any) error {
return json.Unmarshal(raw, dest)
}
// RefreshMetricsSnapshot stores overview and tables for heavy reads.
func (s *Service) RefreshMetricsSnapshot(ctx context.Context) error {
ov, err := s.fetchOverview(ctx)
if err != nil {
return err
}
if err := s.UpsertSnapshot(ctx, "overview", ov); err != nil {
return err
}
tables, err := queryTables(ctx, s.pool, 50)
if err != nil {
return err
}
return s.UpsertSnapshot(ctx, "tables", tables)
}
// AggregateSlowQueries stores top statements snapshot.
func (s *Service) AggregateSlowQueries(ctx context.Context, limit int) error {
if !s.statementsQueryable(ctx) {
return s.UpsertSnapshot(ctx, "slow_queries", []QueryStat{})
}
items, err := queryTopStatements(ctx, s.pool, clampLimit(limit, 20, 100))
if err != nil {
if isPgStatStatementsUnavailable(err) {
s.markStatementsUnavailable()
return s.UpsertSnapshot(ctx, "slow_queries", []QueryStat{})
}
return err
}
return s.UpsertSnapshot(ctx, "slow_queries", items)
}
// EstimateTableBloat refreshes bloat heuristics on tables snapshot.
func (s *Service) EstimateTableBloat(ctx context.Context) error {
tables, err := queryTables(ctx, s.pool, 100)
if err != nil {
return err
}
return s.UpsertSnapshot(ctx, "table_bloat", tables)
}
// AnalyzeIndexUsage stores unused indexes.
func (s *Service) AnalyzeIndexUsage(ctx context.Context) error {
rows, err := s.pool.Query(ctx, `
SELECT indexrelname, idx_scan, pg_relation_size(indexrelid)
FROM pg_stat_user_indexes
WHERE schemaname = 'public' AND idx_scan = 0
ORDER BY pg_relation_size(indexrelid) DESC
LIMIT 50`)
if err != nil {
return fmt.Errorf("pgmonitor: index usage: %w", err)
}
defer rows.Close()
type unused struct {
Index string `json:"index"`
IdxScan int64 `json:"idx_scan"`
SizeBytes int64 `json:"size_bytes"`
}
var items []unused
for rows.Next() {
var u unused
if err := rows.Scan(&u.Index, &u.IdxScan, &u.SizeBytes); err != nil {
return err
}
items = append(items, u)
}
return s.UpsertSnapshot(ctx, "unused_indexes", items)
}
// DetectAutovacuumLag stores tables with high dead tuple ratio.
func (s *Service) DetectAutovacuumLag(ctx context.Context) error {
rows, err := s.pool.Query(ctx, `
SELECT relname, n_dead_tup, last_autovacuum,
CASE WHEN n_live_tup + n_dead_tup > 0
THEN round(n_dead_tup::numeric / (n_live_tup + n_dead_tup), 4) ELSE 0 END
FROM pg_stat_user_tables
WHERE schemaname = 'public' AND n_dead_tup > 1000
ORDER BY n_dead_tup DESC
LIMIT 30`)
if err != nil {
return fmt.Errorf("pgmonitor: autovacuum lag: %w", err)
}
defer rows.Close()
type lagRow struct {
Relname string `json:"relname"`
DeadTuples int64 `json:"n_dead_tup"`
LastAutovacuum *time.Time `json:"last_autovacuum,omitempty"`
Ratio float64 `json:"ratio"`
}
var items []lagRow
for rows.Next() {
var r lagRow
if err := rows.Scan(&r.Relname, &r.DeadTuples, &r.LastAutovacuum, &r.Ratio); err != nil {
return err
}
items = append(items, r)
}
return s.UpsertSnapshot(ctx, "autovacuum_lag", items)
}
// RunPeriodicAnalyzerJobs runs all snapshot analyzers (for scheduler).
func RunPeriodicAnalyzerJobs(ctx context.Context, pool *pgxpool.Pool) {
s := NewService(pool)
if s == nil {
return
}
_ = s.RefreshMetricsSnapshot(ctx)
_ = s.AggregateSlowQueries(ctx, 30)
_ = s.EstimateTableBloat(ctx)
_ = s.AnalyzeIndexUsage(ctx)
_ = s.DetectAutovacuumLag(ctx)
}
+150
View File
@@ -0,0 +1,150 @@
package pgmonitor
import "time"
// Overview is instance-level PostgreSQL health snapshot.
type Overview struct {
CollectedAt time.Time `json:"collected_at"`
Connections Connections `json:"connections"`
Database DatabaseStats `json:"database"`
Bgwriter BgwriterStats `json:"bgwriter"`
SizeBytes int64 `json:"database_size_bytes"`
MemorySettings MemorySettings `json:"memory_settings"`
Replication []ReplicationPeer `json:"replication"`
StatementsEnabled bool `json:"pg_stat_statements_enabled"`
}
// Connections summarizes pg_stat_activity for current database.
type Connections struct {
Active int `json:"active"`
Idle int `json:"idle"`
Total int `json:"total"`
MaxConnections int `json:"max_connections"`
}
// DatabaseStats from pg_stat_database.
type DatabaseStats struct {
Backends int `json:"backends"`
XactCommit int64 `json:"xact_commit"`
XactRollback int64 `json:"xact_rollback"`
Deadlocks int64 `json:"deadlocks"`
BlksHit int64 `json:"blks_hit"`
BlksRead int64 `json:"blks_read"`
CacheHitPct float64 `json:"cache_hit_pct"`
}
// BgwriterStats from pg_stat_bgwriter.
type BgwriterStats struct {
CheckpointsTimed int64 `json:"checkpoints_timed"`
CheckpointsReq int64 `json:"checkpoints_req"`
BuffersCheckpoint int64 `json:"buffers_checkpoint"`
BuffersClean int64 `json:"buffers_clean"`
MaxWrittenClean int64 `json:"maxwritten_clean"`
BuffersBackend int64 `json:"buffers_backend"`
BuffersAlloc int64 `json:"buffers_alloc"`
}
// MemorySettings is best-effort from pg_settings (not RSS).
type MemorySettings struct {
SharedBuffers string `json:"shared_buffers"`
WorkMem string `json:"work_mem"`
EffectiveCacheSize string `json:"effective_cache_size"`
}
// ReplicationPeer from pg_stat_replication.
type ReplicationPeer struct {
ClientAddr string `json:"client_addr,omitempty"`
State string `json:"state"`
SyncState string `json:"sync_state,omitempty"`
LagMs *int64 `json:"lag_ms,omitempty"`
}
// QueryStat is a row from pg_stat_statements or snapshot.
type QueryStat struct {
QueryID int64 `json:"queryid,omitempty"`
Query string `json:"query"`
Calls int64 `json:"calls"`
TotalExecMs float64 `json:"total_exec_ms"`
MeanExecMs float64 `json:"mean_exec_ms"`
Rows int64 `json:"rows"`
}
// QueriesResponse for GET /monitoring/postgres/queries.
type QueriesResponse struct {
CollectedAt time.Time `json:"collected_at"`
Source string `json:"source"` // live | snapshot | unavailable
Items []QueryStat `json:"items"`
StatementsAvailable bool `json:"statements_available"`
StatementsHint string `json:"statements_hint,omitempty"`
}
// LockRow describes a lock / blocking session.
type LockRow struct {
Locktype string `json:"locktype"`
Mode string `json:"mode"`
Granted bool `json:"granted"`
PID int32 `json:"pid"`
User string `json:"usename,omitempty"`
State string `json:"state,omitempty"`
Query string `json:"query,omitempty"`
Blocked bool `json:"blocked"`
}
// TableStat combines size and scan stats for a user table.
type TableStat struct {
Relname string `json:"relname"`
TotalBytes int64 `json:"total_bytes"`
HeapBlksRead int64 `json:"heap_blks_read"`
HeapBlksHit int64 `json:"heap_blks_hit"`
IdxScan int64 `json:"idx_scan"`
SeqScan int64 `json:"seq_scan"`
DeadTuples int64 `json:"n_dead_tup"`
LastAutovacuum *time.Time `json:"last_autovacuum,omitempty"`
BloatRatio float64 `json:"bloat_ratio,omitempty"`
}
// RecommendationItem is a heuristic ops hint.
type RecommendationItem struct {
Severity string `json:"severity"` // info | warn | critical
Code string `json:"code"`
Title string `json:"title"`
Detail string `json:"detail"`
Refs []string `json:"refs,omitempty"`
}
// RecommendationsResponse for GET recommendations.
type RecommendationsResponse struct {
CollectedAt time.Time `json:"collected_at"`
Items []RecommendationItem `json:"items"`
}
// CorrelationPoint is one aligned sample for overlay charts.
type CorrelationPoint struct {
Timestamp time.Time `json:"timestamp"`
PipelineRefreshP99Ms float64 `json:"pipeline_refresh_p99_ms,omitempty"`
BirdScrapeOK *float64 `json:"bird_scrape_ok,omitempty"`
HTTPRequestRate float64 `json:"http_request_rate,omitempty"`
CacheHitPct float64 `json:"cache_hit_pct,omitempty"`
}
// CorrelationResponse for GET /monitoring/correlation.
type CorrelationResponse struct {
WindowMinutes int `json:"window_minutes"`
Points []CorrelationPoint `json:"points"`
}
// MaintenanceLogRow is an audit entry.
type MaintenanceLogRow struct {
ID string `json:"id"`
TenantID string `json:"tenant_id,omitempty"`
ActorPrefix string `json:"actor_prefix,omitempty"`
Kind string `json:"kind"`
TargetTable string `json:"target_table,omitempty"`
DryRun bool `json:"dry_run"`
Status string `json:"status"`
Detail map[string]any `json:"detail,omitempty"`
Error string `json:"error,omitempty"`
CreatedAt time.Time `json:"created_at"`
StartedAt *time.Time `json:"started_at,omitempty"`
FinishedAt *time.Time `json:"finished_at,omitempty"`
}
+3 -24
View File
@@ -1,29 +1,8 @@
package repository
import (
"context"
"time"
)
import "context"
const (
jobAuditRetentionDays = 90
asnCacheRetentionDays = 7
)
// RunPeriodicMaintenance prunes stale job_audit and asn_prefix_cache rows (PostgreSQL).
// RunPeriodicMaintenance is a no-op; retention is driven by maintenance_policy rows (UI-configured).
func (p *Postgres) RunPeriodicMaintenance(ctx context.Context) {
if p == nil || p.pool == nil {
return
}
if ctx == nil {
ctx = context.Background()
}
jobCutoff := time.Now().UTC().Add(-time.Duration(jobAuditRetentionDays) * 24 * time.Hour)
_, _ = p.pool.Exec(ctx, `
DELETE FROM job_audit
WHERE created_at < $1
AND status IN ('succeeded', 'failed', 'cancelled')`, jobCutoff)
asnCutoff := time.Now().UTC().Add(-time.Duration(asnCacheRetentionDays) * 24 * time.Hour)
_, _ = p.pool.Exec(ctx, `
DELETE FROM asn_prefix_cache WHERE fetched_at < $1`, asnCutoff)
_ = ctx
}
+2 -2
View File
@@ -39,7 +39,7 @@ func (p *Postgres) GetModulePrefixSnapshot(tenantID, moduleID string) (*store.Mo
var prefixes []store.PrefixRow
if moduleSnapshotRowTableExists(ctx, p.pool) {
rows, qerr := p.pool.Query(ctx, `
SELECT prefix::text, community_id::text, source
SELECT prefix, community_id::text, source
FROM module_prefix_snapshot_row
WHERE tenant_id = $1::uuid AND module_id = $2::uuid
ORDER BY ord`, tenantID, moduleID)
@@ -108,7 +108,7 @@ func (p *Postgres) SetModulePrefixSnapshot(tenantID, moduleID, inputHash string,
}
if _, err := tx.Exec(ctx, `
INSERT INTO module_prefix_snapshot_row (tenant_id, module_id, ord, prefix, community_id, source)
VALUES ($1::uuid, $2::uuid, $3, $4::cidr, $5::uuid, $6)`,
VALUES ($1::uuid, $2::uuid, $3, $4, $5::uuid, $6)`,
tenantID, moduleID, i, strings.TrimSpace(pr.Prefix), comm, src); err != nil {
return err
}
@@ -0,0 +1,304 @@
package repository
import (
"context"
"encoding/json"
"errors"
"strconv"
"strings"
"time"
"evobgp/internal/store"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
)
const maintenancePolicySelect = `
SELECT id, name, table_name, condition_sql, retention_period_sec, max_rows,
vacuum_strategy, schedule_cron, enabled, dry_run_enabled,
last_run_at, COALESCE(last_status, ''), COALESCE(last_error, ''),
created_at, updated_at
FROM maintenance_policy`
func scanMaintenancePolicy(row pgx.Row) (*store.MaintenancePolicy, error) {
var p store.MaintenancePolicy
var retention, maxRows *int32
var lastRun *time.Time
err := row.Scan(
&p.ID, &p.Name, &p.TableName, &p.Condition, &retention, &maxRows,
&p.VacuumStrategy, &p.Schedule, &p.Enabled, &p.DryRunEnabled,
&lastRun, &p.LastStatus, &p.LastError, &p.CreatedAt, &p.UpdatedAt,
)
if err != nil {
return nil, err
}
if retention != nil {
v := int(*retention)
p.RetentionPeriodSec = &v
}
if maxRows != nil {
v := int(*maxRows)
p.MaxRows = &v
}
if lastRun != nil {
t := lastRun.UTC()
p.LastRunAt = &t
}
return &p, nil
}
func (p *Postgres) ListMaintenancePolicies(cursor string, limit int) ([]*store.MaintenancePolicy, string, bool, error) {
if limit <= 0 {
limit = 50
}
off := 0
if cursor != "" {
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
off = n
}
}
ctx := context.Background()
rows, err := p.pool.Query(ctx, maintenancePolicySelect+`
ORDER BY created_at DESC, id DESC
LIMIT $1 OFFSET $2`, limit+1, off)
if err != nil {
return nil, "", false, err
}
defer rows.Close()
var out []*store.MaintenancePolicy
for rows.Next() {
pol, err := scanMaintenancePolicy(rows)
if err != nil {
continue
}
out = append(out, pol)
}
more := len(out) > limit
if more {
out = out[:limit]
}
next := ""
if more {
next = strconv.Itoa(off + limit)
}
return out, next, more, rows.Err()
}
func (p *Postgres) GetMaintenancePolicy(id string) (*store.MaintenancePolicy, error) {
ctx := context.Background()
row := p.pool.QueryRow(ctx, maintenancePolicySelect+` WHERE id=$1`, id)
pol, err := scanMaintenancePolicy(row)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, store.ErrNotFound
}
return nil, err
}
return pol, nil
}
func (p *Postgres) CreateMaintenancePolicy(in *store.MaintenancePolicy) (*store.MaintenancePolicy, error) {
if in == nil {
return nil, store.ErrInvalidInput
}
vacuum := in.VacuumStrategy
if vacuum == "" {
vacuum = store.VacuumStrategyNone
}
if err := store.ValidateMaintenancePolicyInput(in.Name, in.TableName, vacuum, in.Schedule); err != nil {
return nil, err
}
ctx := context.Background()
id := uuid.NewString()
now := time.Now().UTC()
condition := store.NormalizeMaintenancePolicyCondition(in.Condition)
var retention, maxRows *int32
if in.RetentionPeriodSec != nil {
v := int32(*in.RetentionPeriodSec)
retention = &v
}
if in.MaxRows != nil {
v := int32(*in.MaxRows)
maxRows = &v
}
_, err := p.pool.Exec(ctx, `
INSERT INTO maintenance_policy (
id, name, table_name, condition_sql, retention_period_sec, max_rows,
vacuum_strategy, schedule_cron, enabled, dry_run_enabled, created_at, updated_at
) VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$11)`,
id, strings.TrimSpace(in.Name), strings.TrimSpace(in.TableName), condition,
retention, maxRows, vacuum, strings.TrimSpace(in.Schedule),
in.Enabled, in.DryRunEnabled, now)
if err != nil {
return nil, err
}
return p.GetMaintenancePolicy(id)
}
func (p *Postgres) UpdateMaintenancePolicy(id string, patch *store.MaintenancePolicyPatch) (*store.MaintenancePolicy, error) {
if patch == nil {
return nil, store.ErrInvalidInput
}
cur, err := p.GetMaintenancePolicy(id)
if err != nil {
return nil, err
}
if patch.Name != nil {
cur.Name = strings.TrimSpace(*patch.Name)
}
if patch.TableName != nil {
cur.TableName = strings.TrimSpace(*patch.TableName)
}
if patch.Condition != nil {
cur.Condition = store.NormalizeMaintenancePolicyCondition(*patch.Condition)
}
if patch.RetentionPeriodSec != nil {
cur.RetentionPeriodSec = patch.RetentionPeriodSec
}
if patch.MaxRows != nil {
cur.MaxRows = patch.MaxRows
}
if patch.VacuumStrategy != nil {
if !store.ValidVacuumStrategy(*patch.VacuumStrategy) {
return nil, store.ErrInvalidInput
}
cur.VacuumStrategy = strings.TrimSpace(*patch.VacuumStrategy)
}
if patch.Schedule != nil {
cur.Schedule = strings.TrimSpace(*patch.Schedule)
}
if patch.Enabled != nil {
cur.Enabled = *patch.Enabled
}
if patch.DryRunEnabled != nil {
cur.DryRunEnabled = *patch.DryRunEnabled
}
if err := store.ValidateMaintenancePolicyInput(cur.Name, cur.TableName, cur.VacuumStrategy, cur.Schedule); err != nil {
return nil, err
}
var retention, maxRows *int32
if cur.RetentionPeriodSec != nil {
v := int32(*cur.RetentionPeriodSec)
retention = &v
}
if cur.MaxRows != nil {
v := int32(*cur.MaxRows)
maxRows = &v
}
ctx := context.Background()
tag, err := p.pool.Exec(ctx, `
UPDATE maintenance_policy SET
name=$2, table_name=$3, condition_sql=$4, retention_period_sec=$5, max_rows=$6,
vacuum_strategy=$7, schedule_cron=$8, enabled=$9, dry_run_enabled=$10, updated_at=now()
WHERE id=$1`,
id, cur.Name, cur.TableName, cur.Condition, retention, maxRows,
cur.VacuumStrategy, cur.Schedule, cur.Enabled, cur.DryRunEnabled)
if err != nil {
return nil, err
}
if tag.RowsAffected() == 0 {
return nil, store.ErrNotFound
}
return p.GetMaintenancePolicy(id)
}
func (p *Postgres) DeleteMaintenancePolicy(id string) error {
ctx := context.Background()
tag, err := p.pool.Exec(ctx, `DELETE FROM maintenance_policy WHERE id=$1`, id)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return store.ErrNotFound
}
return nil
}
func (p *Postgres) TouchMaintenancePolicyRun(id, status, errMsg string) error {
ctx := context.Background()
tag, err := p.pool.Exec(ctx, `
UPDATE maintenance_policy SET
last_run_at=now(), last_status=$2, last_error=NULLIF($3,''), updated_at=now()
WHERE id=$1`, id, status, errMsg)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return store.ErrNotFound
}
return nil
}
func (p *Postgres) AppendMaintenancePolicyConfigAudit(actor, policyID, action string, before, after map[string]any) error {
ctx := context.Background()
id := uuid.NewString()
var beforeJSON, afterJSON []byte
if before != nil {
beforeJSON, _ = json.Marshal(before)
}
if after != nil {
afterJSON, _ = json.Marshal(after)
}
_, err := p.pool.Exec(ctx, `
INSERT INTO maintenance_policy_config_audit
(id, policy_id, actor_prefix, action, before_json, after_json, created_at)
VALUES ($1, NULLIF($2,''), $3, $4, $5::jsonb, $6::jsonb, now())`,
id, policyID, strings.TrimSpace(actor), action,
nullJSONBytes(beforeJSON), nullJSONBytes(afterJSON))
return err
}
func nullJSONBytes(b []byte) any {
if len(b) == 0 {
return nil
}
return string(b)
}
func (p *Postgres) ListMaintenancePolicyConfigAudit(cursor string, limit int) ([]*store.MaintenancePolicyConfigAudit, string, bool, error) {
if limit <= 0 {
limit = 50
}
off := 0
if cursor != "" {
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
off = n
}
}
ctx := context.Background()
rows, err := p.pool.Query(ctx, `
SELECT id, COALESCE(policy_id,''), actor_prefix, action,
before_json, after_json, created_at
FROM maintenance_policy_config_audit
ORDER BY created_at DESC, id DESC
LIMIT $1 OFFSET $2`, limit+1, off)
if err != nil {
return nil, "", false, err
}
defer rows.Close()
var out []*store.MaintenancePolicyConfigAudit
for rows.Next() {
var r store.MaintenancePolicyConfigAudit
var beforeRaw, afterRaw []byte
if err := rows.Scan(&r.ID, &r.PolicyID, &r.ActorPrefix, &r.Action, &beforeRaw, &afterRaw, &r.CreatedAt); err != nil {
continue
}
if len(beforeRaw) > 0 {
_ = json.Unmarshal(beforeRaw, &r.Before)
}
if len(afterRaw) > 0 {
_ = json.Unmarshal(afterRaw, &r.After)
}
out = append(out, &r)
}
more := len(out) > limit
if more {
out = out[:limit]
}
next := ""
if more {
next = strconv.Itoa(off + limit)
}
return out, next, more, rows.Err()
}
+13 -3
View File
@@ -115,6 +115,16 @@ type Backend interface {
// RunPeriodicMaintenance prunes stale DB rows (no-op for in-memory).
RunPeriodicMaintenance(ctx context.Context)
// Maintenance policies (instance-scoped PostgreSQL maintenance configuration).
ListMaintenancePolicies(cursor string, limit int) ([]*MaintenancePolicy, string, bool, error)
GetMaintenancePolicy(id string) (*MaintenancePolicy, error)
CreateMaintenancePolicy(in *MaintenancePolicy) (*MaintenancePolicy, error)
UpdateMaintenancePolicy(id string, patch *MaintenancePolicyPatch) (*MaintenancePolicy, error)
DeleteMaintenancePolicy(id string) error
TouchMaintenancePolicyRun(id, status, errMsg string) error
AppendMaintenancePolicyConfigAudit(actor, policyID, action string, before, after map[string]any) error
ListMaintenancePolicyConfigAudit(cursor string, limit int) ([]*MaintenancePolicyConfigAudit, string, bool, error)
}
// ASNPrefixCacheEntry is a cached RIPEstat response for one ASN.
@@ -314,7 +324,7 @@ type SpeakerPatch struct {
// PrefixRow is one materialized prefix for GET /revisions/.../prefixes.
type PrefixRow struct {
Prefix string
CommunityID *string
Source string
Prefix string `json:"prefix"`
CommunityID *string `json:"community_id,omitempty"`
Source string `json:"source,omitempty"`
}
+89
View File
@@ -0,0 +1,89 @@
package store
import (
"strings"
"time"
)
// Vacuum strategy values for maintenance_policy.vacuum_strategy.
const (
VacuumStrategyNone = "none"
VacuumStrategyVacuum = "vacuum"
VacuumStrategyAnalyze = "analyze"
VacuumStrategyVacuumAnalyze = "vacuum_analyze"
VacuumStrategyReindex = "reindex"
)
// MaintenancePolicy is an instance-scoped PostgreSQL maintenance policy (control plane DB).
type MaintenancePolicy struct {
ID string `json:"id,omitempty"`
Name string `json:"name"`
TableName string `json:"table_name"`
Condition string `json:"condition"`
RetentionPeriodSec *int `json:"retention_period_sec,omitempty"`
MaxRows *int `json:"max_rows,omitempty"`
VacuumStrategy string `json:"vacuum_strategy"`
Schedule string `json:"schedule"`
Enabled bool `json:"enabled"`
DryRunEnabled bool `json:"dry_run_enabled"`
LastRunAt *time.Time `json:"last_run_at,omitempty"`
LastStatus string `json:"last_status,omitempty"`
LastError string `json:"last_error,omitempty"`
CreatedAt time.Time `json:"created_at,omitempty"`
UpdatedAt time.Time `json:"updated_at,omitempty"`
}
// MaintenancePolicyPatch is a partial update for maintenance_policy.
type MaintenancePolicyPatch struct {
Name *string `json:"name,omitempty"`
TableName *string `json:"table_name,omitempty"`
Condition *string `json:"condition,omitempty"`
RetentionPeriodSec *int `json:"retention_period_sec,omitempty"`
MaxRows *int `json:"max_rows,omitempty"`
VacuumStrategy *string `json:"vacuum_strategy,omitempty"`
Schedule *string `json:"schedule,omitempty"`
Enabled *bool `json:"enabled,omitempty"`
DryRunEnabled *bool `json:"dry_run_enabled,omitempty"`
}
// MaintenancePolicyConfigAudit is a configuration change log entry.
type MaintenancePolicyConfigAudit struct {
ID string `json:"id"`
PolicyID string `json:"policy_id,omitempty"`
ActorPrefix string `json:"actor_prefix"`
Action string `json:"action"`
Before map[string]any `json:"before,omitempty"`
After map[string]any `json:"after,omitempty"`
CreatedAt time.Time `json:"created_at"`
}
// ValidVacuumStrategy reports whether s is an allowed vacuum_strategy value.
func ValidVacuumStrategy(s string) bool {
switch strings.TrimSpace(s) {
case VacuumStrategyNone, VacuumStrategyVacuum, VacuumStrategyAnalyze,
VacuumStrategyVacuumAnalyze, VacuumStrategyReindex:
return true
default:
return false
}
}
// NormalizeMaintenancePolicyCondition returns a safe default WHERE fragment.
func NormalizeMaintenancePolicyCondition(condition string) string {
c := strings.TrimSpace(condition)
if c == "" {
return "true"
}
return c
}
// ValidateMaintenancePolicyInput checks required fields for create/update payloads.
func ValidateMaintenancePolicyInput(name, tableName, vacuumStrategy, schedule string) error {
if strings.TrimSpace(name) == "" || strings.TrimSpace(tableName) == "" || strings.TrimSpace(schedule) == "" {
return ErrInvalidInput
}
if !ValidVacuumStrategy(vacuumStrategy) {
return ErrInvalidInput
}
return nil
}
+32 -28
View File
@@ -33,17 +33,19 @@ type Memory struct {
peers map[string]*BGPPeer
dohProfiles map[string]*DohProfile
communities map[string]*Community
cdnSources map[string]*CDNSource
asEntries map[string]*ASEntry
domainEnt map[string]*DomainEntry
ipRanges map[string]*IPRangeEntry
settings map[string]map[string]any // tenantID -> key -> JSON-compatible value
revPrefixes map[string][]PrefixRow
moduleSnapshots map[string]*moduleSnapshotRec
asnPrefixCache map[int64]*ASNPrefixCacheEntry
apiKeys map[string]*apiKeyRec
dohProfiles map[string]*DohProfile
communities map[string]*Community
cdnSources map[string]*CDNSource
asEntries map[string]*ASEntry
domainEnt map[string]*DomainEntry
ipRanges map[string]*IPRangeEntry
settings map[string]map[string]any // tenantID -> key -> JSON-compatible value
revPrefixes map[string][]PrefixRow
moduleSnapshots map[string]*moduleSnapshotRec
asnPrefixCache map[int64]*ASNPrefixCacheEntry
apiKeys map[string]*apiKeyRec
maintenancePolicies map[string]*MaintenancePolicy
maintConfigAudit []*MaintenancePolicyConfigAudit
// DemoIDs valid after SeedDemo()
demoTenantID string
@@ -123,23 +125,25 @@ type Speaker struct {
func NewMemory() *Memory {
return &Memory{
tenants: make(map[string]*Tenant),
modules: make(map[string]*Module),
revisions: make(map[string]*Revision),
speakers: make(map[string]*Speaker),
publishedRevision: make(map[string]publishedInfo),
peers: make(map[string]*BGPPeer),
dohProfiles: make(map[string]*DohProfile),
communities: make(map[string]*Community),
cdnSources: make(map[string]*CDNSource),
asEntries: make(map[string]*ASEntry),
domainEnt: make(map[string]*DomainEntry),
ipRanges: make(map[string]*IPRangeEntry),
settings: make(map[string]map[string]any),
revPrefixes: make(map[string][]PrefixRow),
moduleSnapshots: make(map[string]*moduleSnapshotRec),
asnPrefixCache: make(map[int64]*ASNPrefixCacheEntry),
apiKeys: make(map[string]*apiKeyRec),
tenants: make(map[string]*Tenant),
modules: make(map[string]*Module),
revisions: make(map[string]*Revision),
speakers: make(map[string]*Speaker),
publishedRevision: make(map[string]publishedInfo),
peers: make(map[string]*BGPPeer),
dohProfiles: make(map[string]*DohProfile),
communities: make(map[string]*Community),
cdnSources: make(map[string]*CDNSource),
asEntries: make(map[string]*ASEntry),
domainEnt: make(map[string]*DomainEntry),
ipRanges: make(map[string]*IPRangeEntry),
settings: make(map[string]map[string]any),
revPrefixes: make(map[string][]PrefixRow),
moduleSnapshots: make(map[string]*moduleSnapshotRec),
asnPrefixCache: make(map[int64]*ASNPrefixCacheEntry),
apiKeys: make(map[string]*apiKeyRec),
maintenancePolicies: make(map[string]*MaintenancePolicy),
maintConfigAudit: nil,
}
}
+244
View File
@@ -0,0 +1,244 @@
package store
import (
"sort"
"strconv"
"strings"
"time"
"github.com/google/uuid"
)
func (m *Memory) ListMaintenancePolicies(cursor string, limit int) ([]*MaintenancePolicy, string, bool, error) {
if limit <= 0 {
limit = 50
}
m.mu.RLock()
defer m.mu.RUnlock()
all := make([]*MaintenancePolicy, 0, len(m.maintenancePolicies))
for _, p := range m.maintenancePolicies {
all = append(all, p)
}
sort.Slice(all, func(i, j int) bool {
if all[i].CreatedAt.Equal(all[j].CreatedAt) {
return all[i].ID > all[j].ID
}
return all[i].CreatedAt.After(all[j].CreatedAt)
})
off := parseMaintCursor(cursor)
end := off + limit
next := ""
hasMore := false
if end > len(all) {
end = len(all)
} else if end < len(all) {
hasMore = true
next = formatMaintCursor(end)
}
if off >= len(all) {
return nil, "", false, nil
}
out := make([]*MaintenancePolicy, end-off)
copy(out, all[off:end])
return out, next, hasMore, nil
}
func (m *Memory) GetMaintenancePolicy(id string) (*MaintenancePolicy, error) {
m.mu.RLock()
defer m.mu.RUnlock()
p, ok := m.maintenancePolicies[id]
if !ok {
return nil, ErrNotFound
}
return cloneMaintenancePolicy(p), nil
}
func (m *Memory) CreateMaintenancePolicy(in *MaintenancePolicy) (*MaintenancePolicy, error) {
if in == nil {
return nil, ErrInvalidInput
}
vacuum := in.VacuumStrategy
if vacuum == "" {
vacuum = VacuumStrategyNone
}
if err := ValidateMaintenancePolicyInput(in.Name, in.TableName, vacuum, in.Schedule); err != nil {
return nil, err
}
m.mu.Lock()
defer m.mu.Unlock()
now := time.Now().UTC()
id := uuid.NewString()
p := &MaintenancePolicy{
ID: id,
Name: strings.TrimSpace(in.Name),
TableName: strings.TrimSpace(in.TableName),
Condition: NormalizeMaintenancePolicyCondition(in.Condition),
RetentionPeriodSec: in.RetentionPeriodSec,
MaxRows: in.MaxRows,
VacuumStrategy: vacuum,
Schedule: strings.TrimSpace(in.Schedule),
Enabled: in.Enabled,
DryRunEnabled: in.DryRunEnabled,
CreatedAt: now,
UpdatedAt: now,
}
m.maintenancePolicies[id] = p
return cloneMaintenancePolicy(p), nil
}
func (m *Memory) UpdateMaintenancePolicy(id string, patch *MaintenancePolicyPatch) (*MaintenancePolicy, error) {
if patch == nil {
return nil, ErrInvalidInput
}
m.mu.Lock()
defer m.mu.Unlock()
p, ok := m.maintenancePolicies[id]
if !ok {
return nil, ErrNotFound
}
if patch.Name != nil {
p.Name = strings.TrimSpace(*patch.Name)
}
if patch.TableName != nil {
p.TableName = strings.TrimSpace(*patch.TableName)
}
if patch.Condition != nil {
p.Condition = NormalizeMaintenancePolicyCondition(*patch.Condition)
}
if patch.RetentionPeriodSec != nil {
p.RetentionPeriodSec = patch.RetentionPeriodSec
}
if patch.MaxRows != nil {
p.MaxRows = patch.MaxRows
}
if patch.VacuumStrategy != nil {
if !ValidVacuumStrategy(*patch.VacuumStrategy) {
return nil, ErrInvalidInput
}
p.VacuumStrategy = strings.TrimSpace(*patch.VacuumStrategy)
}
if patch.Schedule != nil {
p.Schedule = strings.TrimSpace(*patch.Schedule)
}
if patch.Enabled != nil {
p.Enabled = *patch.Enabled
}
if patch.DryRunEnabled != nil {
p.DryRunEnabled = *patch.DryRunEnabled
}
if err := ValidateMaintenancePolicyInput(p.Name, p.TableName, p.VacuumStrategy, p.Schedule); err != nil {
return nil, err
}
p.UpdatedAt = time.Now().UTC()
return cloneMaintenancePolicy(p), nil
}
func (m *Memory) DeleteMaintenancePolicy(id string) error {
m.mu.Lock()
defer m.mu.Unlock()
if _, ok := m.maintenancePolicies[id]; !ok {
return ErrNotFound
}
delete(m.maintenancePolicies, id)
return nil
}
func (m *Memory) TouchMaintenancePolicyRun(id, status, errMsg string) error {
m.mu.Lock()
defer m.mu.Unlock()
p, ok := m.maintenancePolicies[id]
if !ok {
return ErrNotFound
}
now := time.Now().UTC()
p.LastRunAt = &now
p.LastStatus = status
p.LastError = errMsg
p.UpdatedAt = now
return nil
}
func (m *Memory) AppendMaintenancePolicyConfigAudit(actor, policyID, action string, before, after map[string]any) error {
m.mu.Lock()
defer m.mu.Unlock()
row := &MaintenancePolicyConfigAudit{
ID: uuid.NewString(),
PolicyID: policyID,
ActorPrefix: strings.TrimSpace(actor),
Action: action,
Before: before,
After: after,
CreatedAt: time.Now().UTC(),
}
m.maintConfigAudit = append(m.maintConfigAudit, row)
return nil
}
func (m *Memory) ListMaintenancePolicyConfigAudit(cursor string, limit int) ([]*MaintenancePolicyConfigAudit, string, bool, error) {
if limit <= 0 {
limit = 50
}
m.mu.RLock()
defer m.mu.RUnlock()
all := append([]*MaintenancePolicyConfigAudit(nil), m.maintConfigAudit...)
sort.Slice(all, func(i, j int) bool {
if all[i].CreatedAt.Equal(all[j].CreatedAt) {
return all[i].ID > all[j].ID
}
return all[i].CreatedAt.After(all[j].CreatedAt)
})
off := parseMaintCursor(cursor)
end := off + limit
next := ""
hasMore := false
if end > len(all) {
end = len(all)
} else if end < len(all) {
hasMore = true
next = formatMaintCursor(end)
}
if off >= len(all) {
return nil, "", false, nil
}
out := make([]*MaintenancePolicyConfigAudit, end-off)
copy(out, all[off:end])
return out, next, hasMore, nil
}
func cloneMaintenancePolicy(p *MaintenancePolicy) *MaintenancePolicy {
if p == nil {
return nil
}
cp := *p
if p.RetentionPeriodSec != nil {
v := *p.RetentionPeriodSec
cp.RetentionPeriodSec = &v
}
if p.MaxRows != nil {
v := *p.MaxRows
cp.MaxRows = &v
}
if p.LastRunAt != nil {
t := *p.LastRunAt
cp.LastRunAt = &t
}
return &cp
}
func parseMaintCursor(cursor string) int {
if cursor == "" {
return 0
}
var off int
for _, r := range cursor {
if r < '0' || r > '9' {
return 0
}
off = off*10 + int(r-'0')
}
return off
}
func formatMaintCursor(off int) string {
return strconv.Itoa(off)
}
@@ -0,0 +1,96 @@
package store
import "testing"
func TestMemoryMaintenancePolicyCRUD(t *testing.T) {
m := NewMemory()
ret := int(86400)
max := 5000
created, err := m.CreateMaintenancePolicy(&MaintenancePolicy{
Name: "audit cleanup",
TableName: "job_audit",
Condition: "status = 'succeeded'",
RetentionPeriodSec: &ret,
MaxRows: &max,
VacuumStrategy: VacuumStrategyVacuumAnalyze,
Schedule: "0 4 * * *",
Enabled: true,
DryRunEnabled: true,
})
if err != nil {
t.Fatal(err)
}
if created.ID == "" {
t.Fatal("missing id")
}
items, _, hasMore, err := m.ListMaintenancePolicies("", 10)
if err != nil {
t.Fatal(err)
}
if len(items) != 1 || hasMore {
t.Fatalf("list: len=%d hasMore=%v", len(items), hasMore)
}
got, err := m.GetMaintenancePolicy(created.ID)
if err != nil {
t.Fatal(err)
}
if got.Name != "audit cleanup" || got.VacuumStrategy != VacuumStrategyVacuumAnalyze {
t.Fatalf("get: %+v", got)
}
newName := "renamed"
disabled := false
updated, err := m.UpdateMaintenancePolicy(created.ID, &MaintenancePolicyPatch{
Name: &newName,
Enabled: &disabled,
})
if err != nil {
t.Fatal(err)
}
if updated.Name != newName || updated.Enabled {
t.Fatalf("update: %+v", updated)
}
if err := m.TouchMaintenancePolicyRun(created.ID, "succeeded", ""); err != nil {
t.Fatal(err)
}
afterTouch, err := m.GetMaintenancePolicy(created.ID)
if err != nil {
t.Fatal(err)
}
if afterTouch.LastStatus != "succeeded" || afterTouch.LastRunAt == nil {
t.Fatalf("touch: %+v", afterTouch)
}
if err := m.AppendMaintenancePolicyConfigAudit("op:test", created.ID, "update", map[string]any{"name": "old"}, map[string]any{"name": newName}); err != nil {
t.Fatal(err)
}
audit, next, hasMore, err := m.ListMaintenancePolicyConfigAudit("", 10)
if err != nil {
t.Fatal(err)
}
if len(audit) != 1 || audit[0].Action != "update" || next != "" || hasMore {
t.Fatalf("audit: %+v next=%q hasMore=%v", audit, next, hasMore)
}
if err := m.DeleteMaintenancePolicy(created.ID); err != nil {
t.Fatal(err)
}
if _, err := m.GetMaintenancePolicy(created.ID); err != ErrNotFound {
t.Fatalf("after delete: %v", err)
}
}
func TestCreateMaintenancePolicyInvalid(t *testing.T) {
m := NewMemory()
_, err := m.CreateMaintenancePolicy(&MaintenancePolicy{Name: "", TableName: "job_audit", Schedule: "0 3 * * *"})
if err != ErrInvalidInput {
t.Fatalf("want ErrInvalidInput got %v", err)
}
_, err = m.CreateMaintenancePolicy(&MaintenancePolicy{Name: "x", TableName: "job_audit", Schedule: "0 3 * * *", VacuumStrategy: "bad"})
if err != ErrInvalidInput {
t.Fatalf("want ErrInvalidInput got %v", err)
}
}
@@ -2,7 +2,7 @@ CREATE TABLE module_prefix_snapshot_row (
tenant_id UUID NOT NULL,
module_id UUID NOT NULL,
ord INTEGER NOT NULL,
prefix CIDR NOT NULL,
prefix TEXT NOT NULL,
community_id UUID,
source TEXT NOT NULL DEFAULT '',
PRIMARY KEY (tenant_id, module_id, ord),
@@ -10,16 +10,37 @@ CREATE TABLE module_prefix_snapshot_row (
REFERENCES module_prefix_snapshot (tenant_id, module_id) ON DELETE CASCADE
);
-- prefixes_json from Go json.Marshal(PrefixRow) used "Prefix"/"CommunityID"/"Source" before json tags.
INSERT INTO module_prefix_snapshot_row (tenant_id, module_id, ord, prefix, community_id, source)
SELECT mps.tenant_id,
mps.module_id,
(t.ordinality - 1)::int,
(t.elem->>'prefix')::cidr,
NULLIF(t.elem->>'community_id', '')::uuid,
COALESCE(NULLIF(t.elem->>'source', ''), '')
FROM module_prefix_snapshot mps
CROSS JOIN LATERAL jsonb_array_elements(mps.prefixes_json) WITH ORDINALITY AS t(elem, ordinality)
WHERE jsonb_typeof(mps.prefixes_json) = 'array'
AND jsonb_array_length(mps.prefixes_json) > 0;
SELECT tenant_id,
module_id,
(row_number() OVER (PARTITION BY tenant_id, module_id ORDER BY ordinality) - 1)::int,
prefix,
NULLIF(community_id, '')::uuid,
COALESCE(source, '')
FROM (
SELECT mps.tenant_id,
mps.module_id,
t.ordinality,
COALESCE(
NULLIF(trim(t.elem->>'prefix'), ''),
NULLIF(trim(t.elem->>'Prefix'), '')
) AS prefix,
COALESCE(
NULLIF(trim(t.elem->>'community_id'), ''),
NULLIF(trim(t.elem->>'CommunityID'), '')
) AS community_id,
COALESCE(
NULLIF(trim(t.elem->>'source'), ''),
NULLIF(trim(t.elem->>'Source'), ''),
''
) AS source
FROM module_prefix_snapshot mps
CROSS JOIN LATERAL jsonb_array_elements(mps.prefixes_json) WITH ORDINALITY AS t(elem, ordinality)
WHERE jsonb_typeof(mps.prefixes_json) = 'array'
AND jsonb_array_length(mps.prefixes_json) > 0
) parsed
WHERE parsed.prefix IS NOT NULL
AND trim(parsed.prefix) <> '';
ALTER TABLE module_prefix_snapshot DROP COLUMN prefixes_json;
@@ -0,0 +1 @@
DROP EXTENSION IF EXISTS pg_stat_statements;
@@ -0,0 +1,2 @@
-- pg_stat_statements requires shared_preload_libraries on the server; extension may fail on dev without restart.
CREATE EXTENSION IF NOT EXISTS pg_stat_statements;
@@ -0,0 +1,2 @@
DROP TABLE IF EXISTS postgres_maintenance_audit;
DROP TABLE IF EXISTS postgres_monitor_snapshot;
@@ -0,0 +1,29 @@
CREATE TABLE IF NOT EXISTS postgres_monitor_snapshot (
id TEXT PRIMARY KEY,
collected_at TIMESTAMPTZ NOT NULL,
payload_json JSONB NOT NULL
);
CREATE INDEX IF NOT EXISTS idx_postgres_monitor_snapshot_collected
ON postgres_monitor_snapshot (collected_at DESC);
CREATE TABLE IF NOT EXISTS postgres_maintenance_audit (
id TEXT PRIMARY KEY,
tenant_id TEXT,
actor_prefix TEXT,
kind TEXT NOT NULL,
target_table TEXT,
dry_run BOOLEAN NOT NULL DEFAULT false,
status TEXT NOT NULL,
detail_json JSONB,
error_message TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
started_at TIMESTAMPTZ,
finished_at TIMESTAMPTZ,
CONSTRAINT postgres_maintenance_audit_status_chk CHECK (
status IN ('queued', 'running', 'succeeded', 'failed', 'cancelled')
)
);
CREATE INDEX IF NOT EXISTS idx_postgres_maintenance_audit_created
ON postgres_maintenance_audit (created_at DESC);
@@ -0,0 +1,7 @@
DROP INDEX IF EXISTS idx_postgres_maintenance_audit_policy;
ALTER TABLE postgres_maintenance_audit DROP COLUMN IF EXISTS policy_id;
DROP TABLE IF EXISTS maintenance_policy_config_audit;
DROP TABLE IF EXISTS maintenance_policy;
@@ -0,0 +1,47 @@
CREATE TABLE IF NOT EXISTS maintenance_policy (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
table_name TEXT NOT NULL,
condition_sql TEXT NOT NULL DEFAULT 'true',
retention_period_sec INTEGER,
max_rows INTEGER,
vacuum_strategy TEXT NOT NULL DEFAULT 'none',
schedule_cron TEXT NOT NULL,
enabled BOOLEAN NOT NULL DEFAULT true,
dry_run_enabled BOOLEAN NOT NULL DEFAULT false,
last_run_at TIMESTAMPTZ,
last_status TEXT,
last_error TEXT,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT maintenance_policy_name_chk CHECK (length(trim(name)) > 0),
CONSTRAINT maintenance_policy_table_name_chk CHECK (length(trim(table_name)) > 0),
CONSTRAINT maintenance_policy_vacuum_strategy_chk CHECK (
vacuum_strategy IN ('none', 'vacuum', 'analyze', 'vacuum_analyze', 'reindex')
)
);
CREATE INDEX IF NOT EXISTS idx_maintenance_policy_enabled
ON maintenance_policy (enabled);
CREATE TABLE IF NOT EXISTS maintenance_policy_config_audit (
id TEXT PRIMARY KEY,
policy_id TEXT,
actor_prefix TEXT NOT NULL,
action TEXT NOT NULL,
before_json JSONB,
after_json JSONB,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
CONSTRAINT maintenance_policy_config_audit_action_chk CHECK (
action IN ('create', 'update', 'delete')
)
);
CREATE INDEX IF NOT EXISTS idx_maintenance_policy_config_audit_created
ON maintenance_policy_config_audit (created_at DESC);
ALTER TABLE postgres_maintenance_audit
ADD COLUMN IF NOT EXISTS policy_id TEXT REFERENCES maintenance_policy (id) ON DELETE SET NULL;
CREATE INDEX IF NOT EXISTS idx_postgres_maintenance_audit_policy
ON postgres_maintenance_audit (policy_id);
@@ -0,0 +1 @@
-- no-op
@@ -0,0 +1 @@
-- no-op: pg_stat_statements is PostgreSQL-only
@@ -0,0 +1,2 @@
DROP TABLE IF EXISTS postgres_maintenance_audit;
DROP TABLE IF EXISTS postgres_monitor_snapshot;
@@ -0,0 +1,20 @@
CREATE TABLE IF NOT EXISTS postgres_monitor_snapshot (
id TEXT PRIMARY KEY,
collected_at TEXT NOT NULL,
payload_json TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS postgres_maintenance_audit (
id TEXT PRIMARY KEY,
tenant_id TEXT,
actor_prefix TEXT,
kind TEXT NOT NULL,
target_table TEXT,
dry_run INTEGER NOT NULL DEFAULT 0,
status TEXT NOT NULL,
detail_json TEXT,
error_message TEXT,
created_at TEXT NOT NULL,
started_at TEXT,
finished_at TEXT
);
@@ -0,0 +1,5 @@
ALTER TABLE postgres_maintenance_audit DROP COLUMN policy_id;
DROP TABLE IF EXISTS maintenance_policy_config_audit;
DROP TABLE IF EXISTS maintenance_policy;
@@ -0,0 +1,29 @@
CREATE TABLE IF NOT EXISTS maintenance_policy (
id TEXT PRIMARY KEY,
name TEXT NOT NULL,
table_name TEXT NOT NULL,
condition_sql TEXT NOT NULL DEFAULT 'true',
retention_period_sec INTEGER,
max_rows INTEGER,
vacuum_strategy TEXT NOT NULL DEFAULT 'none',
schedule_cron TEXT NOT NULL,
enabled INTEGER NOT NULL DEFAULT 1,
dry_run_enabled INTEGER NOT NULL DEFAULT 0,
last_run_at TEXT,
last_status TEXT,
last_error TEXT,
created_at TEXT NOT NULL,
updated_at TEXT NOT NULL
);
CREATE TABLE IF NOT EXISTS maintenance_policy_config_audit (
id TEXT PRIMARY KEY,
policy_id TEXT,
actor_prefix TEXT NOT NULL,
action TEXT NOT NULL,
before_json TEXT,
after_json TEXT,
created_at TEXT NOT NULL
);
ALTER TABLE postgres_maintenance_audit ADD COLUMN policy_id TEXT;
+14
View File
@@ -0,0 +1,14 @@
{
"$schema": "https://opencode.ai/config.json",
"mcp": {
"codegraph": {
"type": "local",
"command": [
"codegraph",
"serve",
"--mcp"
],
"enabled": true
}
}
}
@@ -0,0 +1,367 @@
<script lang="ts">
import { onMount } from 'svelte';
import type { AuthSession } from '$lib/api/types.js';
import type { PostgresTableRow } from '$lib/monitoring/postgres.js';
import {
createMaintenancePolicy,
deleteMaintenancePolicy,
fetchPolicyHints,
listMaintenancePolicies,
runMaintenancePolicy,
updateMaintenancePolicy,
type MaintenancePolicy,
type MaintenancePolicyHints
} from '$lib/maintenance/policy-api.js';
import {
emptyMaintenancePolicyForm,
formToPayload,
vacuumStrategies,
type MaintenancePolicyForm
} from '$lib/maintenance/policy.schema.js';
import { Button } from '$lib/ui/core/button/index.js';
import {
Card,
CardContent,
CardHeader,
CardTitle,
CardDescription
} from '$lib/ui/core/card/index.js';
import { Badge } from '$lib/ui/core/badge/index.js';
import { Alert, AlertDescription, AlertTitle } from '$lib/ui/core/alert/index.js';
import {
Dialog,
DialogContent,
DialogHeader,
DialogTitle,
DialogFooter
} from '$lib/ui/core/dialog/index.js';
import { Switch } from '$lib/ui/core/switch/index.js';
import { Label } from '$lib/ui/core/label/index.js';
import FormField from '$lib/ui/patterns/form/form-field.svelte';
import AppInput from '$lib/ui/patterns/form/app-input.svelte';
import AppDataTable from '$lib/ui/patterns/data-table/app-data-table.svelte';
import type { DataTableColumn } from '$lib/ui/patterns/data-table/types.js';
import { confirm } from '$lib/ui/patterns/confirm/confirm-state.svelte.js';
import { notify, notifyApiError } from '$lib/ui/app/toast.js';
import Plus from '@lucide/svelte/icons/plus';
import Pencil from '@lucide/svelte/icons/pencil';
import Trash2 from '@lucide/svelte/icons/trash-2';
import Play from '@lucide/svelte/icons/play';
import FlaskConical from '@lucide/svelte/icons/flask-conical';
import Info from '@lucide/svelte/icons/info';
type Props = {
session: AuthSession | null;
tables: PostgresTableRow[];
onJobQueued?: () => void | Promise<void>;
};
let { session, tables = [], onJobQueued }: Props = $props();
let policies = $state<MaintenancePolicy[]>([]);
let loading = $state(true);
let dialogOpen = $state(false);
let editTarget = $state<MaintenancePolicy | null>(null);
let form = $state<MaintenancePolicyForm>(emptyMaintenancePolicyForm());
let saving = $state(false);
let hints = $state<MaintenancePolicyHints | null>(null);
let hintsLoading = $state(false);
const isOperator = $derived(session?.role === 'operator');
const tableOptions = $derived.by(() => {
const names = new Set(tables.map((t) => t.relname));
if (form.table_name.trim()) names.add(form.table_name.trim());
return [...names].sort();
});
const columns: DataTableColumn<MaintenancePolicy>[] = [
{ id: 'name', label: 'Название', sortable: true, sortValue: (p) => p.name },
{ id: 'table_name', label: 'Таблица', sortable: true, sortValue: (p) => p.table_name },
{ id: 'schedule', label: 'Cron (UTC)' },
{ id: 'status', label: 'Статус' },
{ id: 'actions', label: '', class: 'w-40' }
];
async function loadPolicies() {
loading = true;
try {
policies = await listMaintenancePolicies();
} catch (e) {
notifyApiError(e, 'Не удалось загрузить политики');
} finally {
loading = false;
}
}
function openCreate() {
editTarget = null;
form = emptyMaintenancePolicyForm();
hints = null;
dialogOpen = true;
}
function openEdit(p: MaintenancePolicy) {
editTarget = p;
form = {
name: p.name,
table_name: p.table_name,
condition: p.condition || 'true',
retention_period_sec: p.retention_period_sec ? String(p.retention_period_sec) : '',
max_rows: p.max_rows ? String(p.max_rows) : '',
vacuum_strategy: (vacuumStrategies.includes(
p.vacuum_strategy as (typeof vacuumStrategies)[number]
)
? p.vacuum_strategy
: 'none') as MaintenancePolicyForm['vacuum_strategy'],
schedule: p.schedule,
enabled: p.enabled,
dry_run_enabled: p.dry_run_enabled
};
hints = null;
dialogOpen = true;
void loadHints(p.id);
}
async function loadHints(id: string) {
hintsLoading = true;
try {
hints = await fetchPolicyHints(id);
} catch {
hints = null;
} finally {
hintsLoading = false;
}
}
function requestDelete(p: MaintenancePolicy) {
void confirm({
title: `Удалить политику «${p.name}»?`,
description: 'Расписание и очистка по этой политике прекратятся.',
confirmLabel: 'Удалить',
destructive: true,
onConfirm: async () => {
await deleteMaintenancePolicy(p.id);
notify.success('Политика удалена');
await loadPolicies();
}
});
}
async function save() {
if (!form.name.trim() || !form.table_name.trim() || !form.schedule.trim()) {
notify.error('Заполните обязательные поля');
return;
}
saving = true;
try {
const payload = formToPayload(form);
if (editTarget) {
await updateMaintenancePolicy(editTarget.id, payload);
notify.success('Политика обновлена');
} else {
await createMaintenancePolicy(payload);
notify.success('Политика создана');
}
dialogOpen = false;
await loadPolicies();
} catch (e) {
notifyApiError(e);
} finally {
saving = false;
}
}
function queueRun(p: MaintenancePolicy, dryRun: boolean) {
void confirm({
title: dryRun ? `Dry-run: ${p.name}` : `Запуск: ${p.name}`,
description: dryRun
? 'Изменения в БД не применяются — только оценка.'
: 'Задача будет поставлена в очередь jobs.',
confirmLabel: dryRun ? 'Dry-run' : 'Запустить',
destructive: !dryRun,
onConfirm: async () => {
const res = await runMaintenancePolicy(p.id, dryRun);
notify.success(`Задача ${res.job_id}`);
await onJobQueued?.();
}
});
}
function statusBadge(p: MaintenancePolicy) {
if (!p.enabled) return 'выкл';
if (p.dry_run_enabled) return 'dry-run sched';
return p.last_status || '—';
}
onMount(() => {
void loadPolicies();
});
</script>
{#if !isOperator}
<Alert>
<AlertTitle>Только operator</AlertTitle>
<AlertDescription>Политики обслуживания БД настраиваются с ролью operator.</AlertDescription>
</Alert>
{/if}
<Card>
<CardHeader
class="flex flex-col gap-3 border-b py-3 sm:flex-row sm:items-center sm:justify-between"
>
<div>
<CardTitle>Политики обслуживания</CardTitle>
<CardDescription>
Единственный источник конфигурации retention, vacuum и расписания (UTC cron).
</CardDescription>
</div>
{#if isOperator}
<Button size="sm" onclick={openCreate}><Plus class="size-4" /> Новая политика</Button>
{/if}
</CardHeader>
<CardContent class="pt-4">
<AppDataTable
{columns}
rows={policies}
rowKey={(p) => p.id}
{loading}
emptyTitle="Политики не созданы"
emptyDescription="Добавьте первую политику через UI — это единственный способ настройки."
>
{#snippet cell({ row, column })}
{#if column.id === 'status'}
<Badge variant={row.enabled ? 'secondary' : 'outline'}>{statusBadge(row)}</Badge>
{#if row.last_run_at}
<p class="mt-1 text-xs text-muted-foreground">{row.last_run_at}</p>
{/if}
{:else if column.id === 'actions' && isOperator}
<div class="flex flex-wrap gap-1">
<Button
variant="ghost"
size="icon-sm"
onclick={() => openEdit(row)}
aria-label="Изменить"
>
<Pencil class="size-4" />
</Button>
<Button
variant="ghost"
size="icon-sm"
onclick={() => queueRun(row, true)}
aria-label="Dry-run"
>
<FlaskConical class="size-4" />
</Button>
<Button
variant="ghost"
size="icon-sm"
onclick={() => queueRun(row, false)}
aria-label="Run"
>
<Play class="size-4" />
</Button>
<Button
variant="ghost"
size="icon-sm"
onclick={() => requestDelete(row)}
aria-label="Удалить"
>
<Trash2 class="size-4" />
</Button>
</div>
{:else if column.id === 'name'}
{row.name}
{:else if column.id === 'table_name'}
{row.table_name}
{:else if column.id === 'schedule'}
<span class="font-mono text-xs">{row.schedule}</span>
{:else if column.id !== 'actions'}
{/if}
{/snippet}
</AppDataTable>
</CardContent>
</Card>
<Dialog bind:open={dialogOpen}>
<DialogContent class="max-h-[90vh] overflow-y-auto sm:max-w-lg">
<DialogHeader>
<DialogTitle>{editTarget ? 'Изменить политику' : 'Новая политика'}</DialogTitle>
</DialogHeader>
{#if hints?.recommend_vacuum}
<Alert class="border-warning/30 bg-warning/5">
<Info class="text-warning" />
<AlertTitle>Рекомендация</AlertTitle>
<AlertDescription>{hints.detail ?? 'Рекомендуется VACUUM.'}</AlertDescription>
</Alert>
{:else if hintsLoading}
<p class="text-sm text-muted-foreground">Загрузка подсказок pg_stat…</p>
{/if}
<div class="grid gap-4 py-2">
<FormField label="Название" id="mp-name" required>
<AppInput bind:value={form.name} disabled={!isOperator} />
</FormField>
<FormField label="Таблица" id="mp-table" required>
<select
class="flex h-9 w-full rounded-md border border-input bg-background px-3 text-sm"
bind:value={form.table_name}
disabled={!isOperator}
>
<option value="">— выберите —</option>
{#each tableOptions as name (name)}
<option value={name}>{name}</option>
{/each}
</select>
</FormField>
<FormField label="Condition (SQL WHERE)" id="mp-condition" required>
<textarea
class="min-h-20 w-full rounded-md border border-input bg-background px-3 py-2 font-mono text-xs"
bind:value={form.condition}
disabled={!isOperator}
></textarea>
</FormField>
<div class="grid gap-4 sm:grid-cols-2">
<FormField label="Retention (сек)" id="mp-retention">
<AppInput bind:value={form.retention_period_sec} type="number" disabled={!isOperator} />
</FormField>
<FormField label="Max rows (batch)" id="mp-max-rows">
<AppInput bind:value={form.max_rows} type="number" disabled={!isOperator} />
</FormField>
</div>
<FormField label="Vacuum strategy" id="mp-vacuum">
<select
class="flex h-9 w-full rounded-md border border-input bg-background px-3 text-sm"
bind:value={form.vacuum_strategy}
disabled={!isOperator}
>
{#each vacuumStrategies as s (s)}
<option value={s}>{s}</option>
{/each}
</select>
</FormField>
<FormField label="Schedule (cron, UTC)" id="mp-schedule" required>
<AppInput bind:value={form.schedule} class="font-mono" disabled={!isOperator} />
</FormField>
<div class="flex flex-wrap gap-6">
<div class="flex items-center gap-2">
<Switch id="mp-enabled" bind:checked={form.enabled} disabled={!isOperator} />
<Label for="mp-enabled">Включена</Label>
</div>
<div class="flex items-center gap-2">
<Switch id="mp-dry" bind:checked={form.dry_run_enabled} disabled={!isOperator} />
<Label for="mp-dry">Scheduler только dry-run</Label>
</div>
</div>
</div>
<DialogFooter>
<Button variant="outline" onclick={() => (dialogOpen = false)}>Отмена</Button>
{#if isOperator}
<Button onclick={save} disabled={saving}>{saving ? 'Сохранение…' : 'Сохранить'}</Button>
{/if}
</DialogFooter>
</DialogContent>
</Dialog>
@@ -0,0 +1,436 @@
<script lang="ts">
import { onMount } from 'svelte';
import { apiJSON } from '$lib/api/client.js';
import type { AuthSession } from '$lib/api/types.js';
import { notifyApiError } from '$lib/ui/app/toast.js';
import MaintenancePoliciesTab from '$lib/components/monitoring/MaintenancePoliciesTab.svelte';
import {
POSTGRES_POLL_MS,
POSTGRES_SLOW_POLL_MS,
formatBytes,
connUsagePct,
type PostgresOverview,
type PostgresQueriesResponse,
type PostgresLockRow,
type PostgresTableRow,
type PostgresRecommendationsResponse,
type PostgresMaintLog,
type CorrelationResponse
} from '$lib/monitoring/postgres.js';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '$lib/ui/core/tabs/index.js';
import {
Card,
CardContent,
CardHeader,
CardTitle,
CardDescription
} from '$lib/ui/core/card/index.js';
import { Button } from '$lib/ui/core/button/index.js';
import { Badge } from '$lib/ui/core/badge/index.js';
import { Alert, AlertDescription, AlertTitle } from '$lib/ui/core/alert/index.js';
import {
Table,
TableBody,
TableCell,
TableHead,
TableHeader,
TableRow
} from '$lib/ui/core/table/index.js';
import { Switch } from '$lib/ui/core/switch/index.js';
import { Label } from '$lib/ui/core/label/index.js';
import Database from '@lucide/svelte/icons/database';
import RefreshCw from '@lucide/svelte/icons/refresh-cw';
let pgTab = $state('overview');
let autoRefresh = $state(true);
let session = $state<AuthSession | null>(null);
let unavailable = $state(false);
let overview = $state<PostgresOverview | null>(null);
let queries = $state<PostgresQueriesResponse | null>(null);
let locks = $state<PostgresLockRow[]>([]);
let tables = $state<PostgresTableRow[]>([]);
let recommendations = $state<PostgresRecommendationsResponse | null>(null);
let maintLogs = $state<PostgresMaintLog[]>([]);
let correlation = $state<CorrelationResponse | null>(null);
let loading = $state(true);
async function loadCore() {
try {
overview = await apiJSON<PostgresOverview>('/v1/monitoring/postgres/overview');
locks = (await apiJSON<{ items: PostgresLockRow[] }>('/v1/monitoring/postgres/locks')).items;
unavailable = false;
} catch (e) {
unavailable = true;
overview = null;
throw e;
}
}
async function loadSlow() {
queries = await apiJSON<PostgresQueriesResponse>('/v1/monitoring/postgres/queries?limit=20');
tables = (
await apiJSON<{ items: PostgresTableRow[] }>('/v1/monitoring/postgres/tables?limit=30')
).items;
recommendations = await apiJSON<PostgresRecommendationsResponse>(
'/v1/monitoring/postgres/recommendations'
);
correlation = await apiJSON<CorrelationResponse>('/v1/monitoring/correlation?window=60');
maintLogs = (
await apiJSON<{ items: PostgresMaintLog[] }>('/v1/postgres/maintenance/logs?limit=20')
).items;
}
async function loadAll() {
loading = true;
try {
await loadCore();
await loadSlow();
} catch (e) {
notifyApiError(e, 'PostgreSQL monitoring');
} finally {
loading = false;
}
}
onMount(() => {
void (async () => {
try {
session = await apiJSON<AuthSession>('/v1/auth/session');
} catch {
session = null;
}
await loadAll();
})();
});
$effect(() => {
if (!autoRefresh || unavailable) return;
const fast = setInterval(() => {
void loadCore().catch(() => {});
}, POSTGRES_POLL_MS);
const slow = setInterval(() => {
void loadSlow().catch(() => {});
}, POSTGRES_SLOW_POLL_MS);
return () => {
clearInterval(fast);
clearInterval(slow);
};
});
</script>
<div class="flex flex-wrap items-center justify-between gap-3">
<div class="flex items-center gap-2 text-sm text-muted-foreground">
<Database class="size-4" />
<span>Instance-level PostgreSQL (control plane)</span>
</div>
<div class="flex items-center gap-4">
<div class="flex items-center gap-2">
<Switch id="pg-auto" bind:checked={autoRefresh} />
<Label for="pg-auto">Автообновление</Label>
</div>
<Button variant="outline" size="sm" onclick={() => loadAll()} disabled={loading}>
<RefreshCw class="mr-1 size-4 {loading ? 'animate-spin' : ''}" />
Обновить
</Button>
</div>
</div>
{#if unavailable}
<Alert variant="destructive" class="mt-4">
<AlertTitle>PostgreSQL недоступен</AlertTitle>
<AlertDescription>
Мониторинг требует <code class="text-xs">EVOBGP_DATABASE_URL</code> (не memory backend).
</AlertDescription>
</Alert>
{:else}
<Tabs bind:value={pgTab} class="mt-4">
<TabsList>
<TabsTrigger value="overview">Обзор</TabsTrigger>
<TabsTrigger value="queries">Запросы</TabsTrigger>
<TabsTrigger value="locks">Блокировки</TabsTrigger>
<TabsTrigger value="tables">Таблицы</TabsTrigger>
<TabsTrigger value="maintenance">Обслуживание</TabsTrigger>
<TabsTrigger value="correlation">Корреляция</TabsTrigger>
</TabsList>
<TabsContent value="overview" class="mt-4 space-y-4">
{#if overview}
<div class="grid gap-4 md:grid-cols-2 lg:grid-cols-4">
<Card>
<CardHeader class="pb-2">
<CardTitle class="text-sm font-medium">Подключения</CardTitle>
</CardHeader>
<CardContent>
<p class="text-2xl font-semibold tabular-nums">
{overview.connections.active} / {overview.connections.max_connections}
</p>
<div class="mt-2 h-2 overflow-hidden rounded-full bg-muted">
<div
class="h-full bg-chart-1 transition-all"
style="width: {connUsagePct(overview)}%"
></div>
</div>
<p class="mt-1 text-xs text-muted-foreground">
idle {overview.connections.idle}, total {overview.connections.total}
</p>
</CardContent>
</Card>
<Card>
<CardHeader class="pb-2">
<CardTitle class="text-sm font-medium">Cache hit</CardTitle>
</CardHeader>
<CardContent>
<p class="text-2xl font-semibold tabular-nums">
{overview.database.cache_hit_pct ?? '—'}%
</p>
<div class="mt-2 h-2 overflow-hidden rounded-full bg-muted">
<div
class="h-full bg-chart-2 transition-all"
style="width: {overview.database.cache_hit_pct ?? 0}%"
></div>
</div>
</CardContent>
</Card>
<Card>
<CardHeader class="pb-2">
<CardTitle class="text-sm font-medium">TPS (commits)</CardTitle>
</CardHeader>
<CardContent>
<p class="text-2xl font-semibold tabular-nums">
{overview.database.xact_commit.toLocaleString()}
</p>
<p class="text-xs text-muted-foreground">
rollback {overview.database.xact_rollback.toLocaleString()}, deadlocks {overview
.database.deadlocks}
</p>
</CardContent>
</Card>
<Card>
<CardHeader class="pb-2">
<CardTitle class="text-sm font-medium">Размер БД</CardTitle>
</CardHeader>
<CardContent>
<p class="text-2xl font-semibold">{formatBytes(overview.database_size_bytes)}</p>
<p class="text-xs text-muted-foreground">
shared_buffers {overview.memory_settings.shared_buffers}
</p>
</CardContent>
</Card>
</div>
{#if overview.replication?.length}
<Card>
<CardHeader>
<CardTitle>Репликация</CardTitle>
</CardHeader>
<CardContent>
<Table>
<TableHeader>
<TableRow>
<TableHead>Адрес</TableHead>
<TableHead>Состояние</TableHead>
<TableHead>Lag ms</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{#each overview.replication as r (r.client_addr ?? r.state)}
<TableRow>
<TableCell>{r.client_addr ?? '—'}</TableCell>
<TableCell>{r.state}</TableCell>
<TableCell>{r.lag_ms ?? '—'}</TableCell>
</TableRow>
{/each}
</TableBody>
</Table>
</CardContent>
</Card>
{/if}
{/if}
</TabsContent>
<TabsContent value="queries" class="mt-4">
<Card>
<CardHeader>
<CardTitle>Медленные запросы</CardTitle>
<CardDescription>
Источник: {queries?.source ?? '—'}
{#if queries?.statements_available === false || (overview && !overview.pg_stat_statements_enabled)}
· pg_stat_statements недоступен
{/if}
</CardDescription>
</CardHeader>
<CardContent>
{#if queries?.statements_hint}
<Alert class="mb-4">
<AlertTitle>Нет статистики запросов</AlertTitle>
<AlertDescription>{queries.statements_hint}</AlertDescription>
</Alert>
{/if}
<Table>
<TableHeader>
<TableRow>
<TableHead>mean ms</TableHead>
<TableHead>calls</TableHead>
<TableHead>query</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{#each queries?.items ?? [] as q (q.queryid ?? q.query)}
<TableRow>
<TableCell class="tabular-nums">{q.mean_exec_ms.toFixed(1)}</TableCell>
<TableCell>{q.calls}</TableCell>
<TableCell class="max-w-md truncate font-mono text-xs">{q.query}</TableCell>
</TableRow>
{:else}
<TableRow>
<TableCell colspan={3} class="text-muted-foreground">Нет данных</TableCell>
</TableRow>
{/each}
</TableBody>
</Table>
</CardContent>
</Card>
</TabsContent>
<TabsContent value="locks" class="mt-4">
<Card>
<CardHeader>
<CardTitle>Блокировки</CardTitle>
</CardHeader>
<CardContent>
<Table>
<TableHeader>
<TableRow>
<TableHead>pid</TableHead>
<TableHead>mode</TableHead>
<TableHead>granted</TableHead>
<TableHead>query</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{#each locks as l (l.pid)}
<TableRow>
<TableCell>{l.pid}</TableCell>
<TableCell>
<Badge variant={l.blocked ? 'destructive' : 'secondary'}>{l.mode}</Badge>
</TableCell>
<TableCell>{l.granted ? 'да' : 'нет'}</TableCell>
<TableCell class="max-w-lg truncate font-mono text-xs">{l.query ?? '—'}</TableCell
>
</TableRow>
{:else}
<TableRow>
<TableCell colspan={4} class="text-muted-foreground"
>Нет активных блокировок</TableCell
>
</TableRow>
{/each}
</TableBody>
</Table>
</CardContent>
</Card>
</TabsContent>
<TabsContent value="tables" class="mt-4 space-y-4">
<Card>
<CardHeader>
<CardTitle>Таблицы и хранилище</CardTitle>
</CardHeader>
<CardContent>
<Table>
<TableHeader>
<TableRow>
<TableHead>table</TableHead>
<TableHead>size</TableHead>
<TableHead>seq_scan</TableHead>
<TableHead>idx_scan</TableHead>
<TableHead>bloat</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{#each tables as t (t.relname)}
<TableRow>
<TableCell class="font-mono text-xs">{t.relname}</TableCell>
<TableCell>{formatBytes(t.total_bytes)}</TableCell>
<TableCell>{t.seq_scan}</TableCell>
<TableCell>{t.idx_scan}</TableCell>
<TableCell>{(t.bloat_ratio ?? 0).toFixed(2)}</TableCell>
</TableRow>
{/each}
</TableBody>
</Table>
</CardContent>
</Card>
{#if recommendations?.items?.length}
<Card>
<CardHeader>
<CardTitle>Рекомендации</CardTitle>
</CardHeader>
<CardContent class="space-y-2">
{#each recommendations.items as item (item.code + item.title)}
<Alert>
<AlertTitle>{item.title}</AlertTitle>
<AlertDescription>{item.detail}</AlertDescription>
</Alert>
{/each}
</CardContent>
</Card>
{/if}
</TabsContent>
<TabsContent value="maintenance" class="mt-4 space-y-4">
<MaintenancePoliciesTab {session} {tables} onJobQueued={loadSlow} />
<Card>
<CardHeader>
<CardTitle>Журнал обслуживания</CardTitle>
</CardHeader>
<CardContent>
<Table>
<TableHeader>
<TableRow>
<TableHead>время</TableHead>
<TableHead>kind</TableHead>
<TableHead>status</TableHead>
<TableHead>dry_run</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{#each maintLogs as log (log.id)}
<TableRow>
<TableCell class="text-xs">{log.created_at}</TableCell>
<TableCell>{log.kind}</TableCell>
<TableCell>{log.status}</TableCell>
<TableCell>{log.dry_run ? 'да' : 'нет'}</TableCell>
</TableRow>
{:else}
<TableRow>
<TableCell colspan={4} class="text-muted-foreground">Пусто</TableCell>
</TableRow>
{/each}
</TableBody>
</Table>
</CardContent>
</Card>
</TabsContent>
<TabsContent value="correlation" class="mt-4">
<Card>
<CardHeader>
<CardTitle>Корреляция (1ч)</CardTitle>
<CardDescription>Pipeline refresh p99 vs cache hit по минутам</CardDescription>
</CardHeader>
<CardContent class="space-y-3">
{#each correlation?.points ?? [] as p (p.timestamp)}
<div class="grid gap-2 rounded-md border p-2 text-xs md:grid-cols-3">
<span>{p.timestamp}</span>
<span>p99 refresh: {p.pipeline_refresh_p99_ms?.toFixed(0) ?? '—'} ms</span>
<span>cache hit: {p.cache_hit_pct?.toFixed(1) ?? '—'}%</span>
</div>
{:else}
<p class="text-muted-foreground">Нет точек за окно</p>
{/each}
</CardContent>
</Card>
</TabsContent>
</Tabs>
{/if}
+68
View File
@@ -0,0 +1,68 @@
import { apiJSON, apiMutate } from '$lib/api/client.js';
export type MaintenancePolicy = {
id: string;
name: string;
table_name: string;
condition: string;
retention_period_sec?: number;
max_rows?: number;
vacuum_strategy: string;
schedule: string;
enabled: boolean;
dry_run_enabled: boolean;
last_run_at?: string;
last_status?: string;
last_error?: string;
created_at?: string;
updated_at?: string;
};
export type MaintenancePolicyHints = {
table_name: string;
n_dead_tup: number;
bloat_ratio?: number;
last_autovacuum?: string;
recommend_vacuum: boolean;
detail?: string;
};
export type MaintenancePoliciesResponse = {
items: MaintenancePolicy[];
next_cursor?: string;
has_more?: boolean;
};
export async function listMaintenancePolicies(limit = 100): Promise<MaintenancePolicy[]> {
const r = await apiJSON<MaintenancePoliciesResponse>(`/v1/maintenance/policies?limit=${limit}`);
return r.items ?? [];
}
export async function createMaintenancePolicy(
body: Record<string, unknown>
): Promise<MaintenancePolicy> {
return apiMutate<MaintenancePolicy>('/v1/maintenance/policies', 'POST', body);
}
export async function updateMaintenancePolicy(
id: string,
body: Record<string, unknown>
): Promise<MaintenancePolicy> {
return apiMutate<MaintenancePolicy>(`/v1/maintenance/policies/${id}`, 'PATCH', body);
}
export async function deleteMaintenancePolicy(id: string): Promise<void> {
await apiMutate(`/v1/maintenance/policies/${id}`, 'DELETE', undefined, { idempotent: false });
}
export async function runMaintenancePolicy(
id: string,
dryRun: boolean
): Promise<{ job_id: string }> {
const path = dryRun ? '/v1/maintenance/dry-run' : '/v1/maintenance/run';
return apiMutate<{ job_id: string; status: string }>(path, 'POST', { policy_id: id });
}
export async function fetchPolicyHints(id: string): Promise<MaintenancePolicyHints> {
return apiJSON<MaintenancePolicyHints>(`/v1/maintenance/policies/${id}/hints`);
}
+58
View File
@@ -0,0 +1,58 @@
import { z } from 'zod';
export const vacuumStrategies = ['none', 'vacuum', 'analyze', 'vacuum_analyze', 'reindex'] as const;
export type VacuumStrategy = (typeof vacuumStrategies)[number];
export const maintenancePolicySchema = z.object({
name: z.string().trim().min(1, 'Укажите название'),
table_name: z.string().trim().min(1, 'Укажите таблицу'),
condition: z
.string()
.trim()
.min(1, 'Укажите условие')
.refine((v) => !/[;]|--|\/\*/.test(v), 'Недопустимые символы в condition'),
retention_period_sec: z.string().optional(),
max_rows: z.string().optional(),
vacuum_strategy: z.enum(vacuumStrategies),
schedule: z.string().trim().min(1, 'Укажите cron (UTC)'),
enabled: z.boolean(),
dry_run_enabled: z.boolean()
});
export type MaintenancePolicyForm = z.infer<typeof maintenancePolicySchema>;
export function emptyMaintenancePolicyForm(): MaintenancePolicyForm {
return {
name: '',
table_name: '',
condition: 'true',
retention_period_sec: '',
max_rows: '10000',
vacuum_strategy: 'none',
schedule: '0 3 * * *',
enabled: true,
dry_run_enabled: true
};
}
export function parseOptionalInt(raw: string | undefined): number | undefined {
const v = String(raw ?? '').trim();
if (!v) return undefined;
const n = Number(v);
return Number.isFinite(n) && n > 0 ? Math.floor(n) : undefined;
}
export function formToPayload(form: MaintenancePolicyForm) {
return {
name: form.name.trim(),
table_name: form.table_name.trim(),
condition: form.condition.trim() || 'true',
retention_period_sec: parseOptionalInt(form.retention_period_sec),
max_rows: parseOptionalInt(form.max_rows),
vacuum_strategy: form.vacuum_strategy,
schedule: form.schedule.trim(),
enabled: form.enabled,
dry_run_enabled: form.dry_run_enabled
};
}
+118
View File
@@ -0,0 +1,118 @@
/** Types and helpers for PostgreSQL monitoring API. */
export const POSTGRES_POLL_MS = 20_000;
export const POSTGRES_SLOW_POLL_MS = 60_000;
export type PostgresOverview = {
collected_at: string;
connections: {
active: number;
idle: number;
total: number;
max_connections: number;
};
database: {
backends: number;
xact_commit: number;
xact_rollback: number;
deadlocks: number;
blks_hit: number;
blks_read: number;
cache_hit_pct: number;
};
database_size_bytes: number;
memory_settings: {
shared_buffers: string;
work_mem: string;
effective_cache_size: string;
};
replication: Array<{
client_addr?: string;
state: string;
sync_state?: string;
lag_ms?: number;
}>;
pg_stat_statements_enabled: boolean;
};
export type PostgresQueryRow = {
queryid?: number;
query: string;
calls: number;
total_exec_ms: number;
mean_exec_ms: number;
rows: number;
};
export type PostgresQueriesResponse = {
collected_at: string;
source: string;
items: PostgresQueryRow[];
statements_available?: boolean;
statements_hint?: string;
};
export type PostgresLockRow = {
locktype: string;
mode: string;
granted: boolean;
pid: number;
usename?: string;
state?: string;
query?: string;
blocked: boolean;
};
export type PostgresTableRow = {
relname: string;
total_bytes: number;
idx_scan: number;
seq_scan: number;
n_dead_tup: number;
bloat_ratio?: number;
last_autovacuum?: string;
};
export type PostgresRecommendation = {
severity: string;
code: string;
title: string;
detail: string;
refs?: string[];
};
export type PostgresRecommendationsResponse = {
collected_at: string;
items: PostgresRecommendation[];
};
export type PostgresMaintLog = {
id: string;
kind: string;
target_table?: string;
dry_run: boolean;
status: string;
error?: string;
created_at: string;
};
export type CorrelationResponse = {
window_minutes: number;
points: Array<{
timestamp: string;
pipeline_refresh_p99_ms?: number;
cache_hit_pct?: number;
}>;
};
export function formatBytes(n: number): string {
if (n >= 1 << 30) return `${(n / (1 << 30)).toFixed(1)} GiB`;
if (n >= 1 << 20) return `${(n / (1 << 20)).toFixed(1)} MiB`;
if (n >= 1 << 10) return `${(n / (1 << 10)).toFixed(1)} KiB`;
return `${n} B`;
}
export function connUsagePct(ov: PostgresOverview | null): number {
if (!ov?.connections.max_connections) return 0;
return Math.min(100, (ov.connections.total / ov.connections.max_connections) * 100);
}
+2
View File
@@ -23,6 +23,8 @@ export function jobKindTitle(job: JobRow, moduleNameById?: ReadonlyMap<string, s
return 'Откат ревизии';
case 'bird_reload':
return 'Перезагрузка BIRD';
case 'maintenance_policy_run':
return 'Обслуживание PostgreSQL (политика)';
default:
return job.kind;
}
@@ -1,7 +1,18 @@
import { z } from 'zod';
export const revisionSettingsSchema = z.object({
revision_retention_minutes: z.string().refine(
/** HTML type=number binds number; API/store may return number — normalize to string for validation. */
function retentionMinutesInput(val: unknown): string {
if (val === undefined || val === null) return '';
if (typeof val === 'number') {
if (!Number.isFinite(val)) return '';
return String(Math.trunc(val));
}
return String(val);
}
const revisionRetentionMinutes = z.preprocess(
retentionMinutesInput,
z.string().refine(
(v) => {
const s = v.trim();
if (s === '') return true;
@@ -10,6 +21,10 @@ export const revisionSettingsSchema = z.object({
},
{ message: 'TTL ревизий должен быть целым числом от 15 до 43200 минут' }
)
);
export const revisionSettingsSchema = z.object({
revision_retention_minutes: revisionRetentionMinutes
});
export type RevisionSettingsForm = z.infer<typeof revisionSettingsSchema>;
+343 -315
View File
@@ -49,6 +49,8 @@
import KpiMetricsGrid from '$lib/ui/patterns/kpi/kpi-metrics-grid.svelte';
import ScrollPreBlock from '$lib/components/app/scroll-pre-block.svelte';
import PageHeader from '$lib/ui/app/page-header/page-header.svelte';
import MonitoringPostgresTab from '$lib/components/monitoring/MonitoringPostgresTab.svelte';
import { Tabs, TabsContent, TabsList, TabsTrigger } from '$lib/ui/core/tabs/index.js';
import { cn } from '$lib/utils.js';
import RefreshCw from '@lucide/svelte/icons/refresh-cw';
import Gauge from '@lucide/svelte/icons/gauge';
@@ -80,6 +82,7 @@
let lastUpdated = $state<Date | null>(null);
let initialLoading = $state(true);
let refreshing = $state(false);
let mainTab = $state('system');
const statAccents = [
{
@@ -322,337 +325,362 @@
{/snippet}
</PageHeader>
{#if !initialLoading}
{#if overallStatus === 'ok'}
<Alert class="border-success/30 bg-success/5">
<CheckCircle class="text-success" />
<AlertTitle>Система в норме</AlertTitle>
<AlertDescription>{overallHint}</AlertDescription>
</Alert>
{:else if overallStatus === 'warn'}
<Alert class="border-warning/30 bg-warning/5">
<AlertTriangle class="text-warning" />
<AlertTitle>Требуется внимание</AlertTitle>
<AlertDescription>{overallHint}</AlertDescription>
</Alert>
{:else if overallStatus === 'error'}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Обнаружена проблема</AlertTitle>
<AlertDescription>{overallHint}</AlertDescription>
</Alert>
{/if}
{/if}
<Tabs bind:value={mainTab}>
<TabsList>
<TabsTrigger value="system">Система</TabsTrigger>
<TabsTrigger value="postgres">PostgreSQL</TabsTrigger>
</TabsList>
<KpiMetricsGrid
cards={kpiCards}
loading={initialLoading}
skeletonCount={4}
class="sm:grid-cols-2 xl:grid-cols-4"
/>
<TabsContent value="system" class="mt-4 flex flex-col gap-6">
{#if !initialLoading}
{#if overallStatus === 'ok'}
<Alert class="border-success/30 bg-success/5">
<CheckCircle class="text-success" />
<AlertTitle>Система в норме</AlertTitle>
<AlertDescription>{overallHint}</AlertDescription>
</Alert>
{:else if overallStatus === 'warn'}
<Alert class="border-warning/30 bg-warning/5">
<AlertTriangle class="text-warning" />
<AlertTitle>Требуется внимание</AlertTitle>
<AlertDescription>{overallHint}</AlertDescription>
</Alert>
{:else if overallStatus === 'error'}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Обнаружена проблема</AlertTitle>
<AlertDescription>{overallHint}</AlertDescription>
</Alert>
{/if}
{/if}
<div class="grid gap-4 lg:grid-cols-2">
{#if initialLoading}
<CardSkeleton />
<CardSkeleton />
{:else}
<Card>
<CardHeader>
<CardTitle class="text-base">Доступность и готовность</CardTitle>
<CardDescription>GET /v1/health · GET /v1/ready</CardDescription>
</CardHeader>
<CardContent class="space-y-4">
{#if health?.error || readyError}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Ошибка проверки</AlertTitle>
<AlertDescription>
{#if health?.error}{health.error}{/if}
{#if health?.error && readyError}<br />{/if}
{#if readyError}{readyError}{/if}
</AlertDescription>
</Alert>
{/if}
<KpiMetricsGrid
cards={kpiCards}
loading={initialLoading}
skeletonCount={4}
class="sm:grid-cols-2 xl:grid-cols-4"
/>
<Table>
<TableHeader>
<TableRow>
<TableHead class="w-[55%]">Проверка</TableHead>
<TableHead>Статус</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{@const liveBadge = livenessBadge(health)}
<TableRow>
<TableCell>
<div class="flex items-center gap-2">
<HeartPulse class="size-4 shrink-0 text-muted-foreground" aria-hidden="true" />
<div>
<p class="text-sm font-medium">Liveness</p>
<p class="text-xs text-muted-foreground">/v1/health</p>
</div>
</div>
</TableCell>
<TableCell>
<Badge variant={liveBadge.variant} class={liveBadge.class}
>{liveBadge.label}</Badge
>
</TableCell>
</TableRow>
{@const readyBadge = readinessBadge(ready)}
<TableRow>
<TableCell>
<div class="flex items-center gap-2">
<ShieldCheck class="size-4 shrink-0 text-muted-foreground" aria-hidden="true" />
<div>
<p class="text-sm font-medium">Readiness</p>
<p class="text-xs text-muted-foreground">/v1/ready</p>
</div>
</div>
</TableCell>
<TableCell>
<Badge variant={readyBadge.variant} class={readyBadge.class}
>{readyBadge.label}</Badge
>
</TableCell>
</TableRow>
{#if ready?.checks && Object.keys(ready.checks).length > 0}
<TableRow>
<TableCell colspan={2} class="bg-muted/30 py-2">
<p class="text-xs font-medium text-muted-foreground">Зависимости</p>
</TableCell>
</TableRow>
{#each Object.entries(ready.checks) as [key, value] (key)}
{@const badge = checkStatusBadge(value)}
{@const CheckIcon = checkIconByKey[key] ?? ListTodo}
<div class="grid gap-4 lg:grid-cols-2">
{#if initialLoading}
<CardSkeleton />
<CardSkeleton />
{:else}
<Card>
<CardHeader>
<CardTitle class="text-base">Доступность и готовность</CardTitle>
<CardDescription>GET /v1/health · GET /v1/ready</CardDescription>
</CardHeader>
<CardContent class="space-y-4">
{#if health?.error || readyError}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Ошибка проверки</AlertTitle>
<AlertDescription>
{#if health?.error}{health.error}{/if}
{#if health?.error && readyError}<br />{/if}
{#if readyError}{readyError}{/if}
</AlertDescription>
</Alert>
{/if}
<Table>
<TableHeader>
<TableRow>
<TableHead class="w-[55%]">Проверка</TableHead>
<TableHead>Статус</TableHead>
</TableRow>
</TableHeader>
<TableBody>
{@const liveBadge = livenessBadge(health)}
<TableRow>
<TableCell>
<div class="flex items-center gap-2">
<CheckIcon
<HeartPulse
class="size-4 shrink-0 text-muted-foreground"
aria-hidden="true"
/>
<div>
<p class="text-sm font-medium">{checkDisplayName(key)}</p>
<p class="text-xs text-muted-foreground">{key}</p>
<p class="text-sm font-medium">Liveness</p>
<p class="text-xs text-muted-foreground">/v1/health</p>
</div>
</div>
</TableCell>
<TableCell>
<div class="space-y-1">
<Badge variant={badge.variant} class={badge.class}>{badge.label}</Badge>
{#if badge.hint}
<p class="text-xs text-muted-foreground">{badge.hint}</p>
{/if}
</div>
<Badge variant={liveBadge.variant} class={liveBadge.class}
>{liveBadge.label}</Badge
>
</TableCell>
</TableRow>
{/each}
{/if}
</TableBody>
</Table>
<p class="text-xs text-muted-foreground">
HTTP 503 на readiness означает недоступность одной из зависимостей в checks.
</p>
</CardContent>
</Card>
<Card>
<CardHeader>
<div class="flex items-start justify-between gap-2">
<div>
<CardTitle class="flex items-center gap-2 text-base">
<Bird class="size-4" />
BGP на API-хосте
</CardTitle>
<CardDescription>GET /v1/bird/status</CardDescription>
</div>
<Button variant="outline" size="sm" href={resolve('/network?tab=overview')}
>Пиры и спикеры</Button
>
</div>
</CardHeader>
<CardContent class="space-y-4">
{#if birdError}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Ошибка birdc</AlertTitle>
<AlertDescription>{birdError}</AlertDescription>
</Alert>
{:else if bird && !bird.birdc_configured}
<p class="text-sm text-muted-foreground">
{bird.message ?? 'birdc не настроен на API-хосте (EVOBGP_BIRDC_SOCKET).'}
</p>
{:else if bird}
<div class="space-y-2">
<div class="flex items-center justify-between text-sm">
<span class="text-muted-foreground">Established / total</span>
<span class="font-medium tabular-nums">
{bird.bgp_established} / {bird.bgp_sessions_total}
{#if bgpRatio !== null}
<span class="text-muted-foreground">({bgpRatio}%)</span>
{/if}
</span>
</div>
{#if bgpRatio !== null}
<div class="h-2 overflow-hidden rounded-full bg-muted">
<div
class={cn(
'h-full rounded-full transition-all',
bgpRatio >= 100
? 'bg-success'
: bgpRatio >= 50
? 'bg-warning'
: 'bg-destructive'
)}
style="width: {bgpRatio}%"
></div>
</div>
{/if}
{#if bird.error}
<p class="text-xs text-destructive">{bird.error}</p>
{/if}
</div>
{#if bird.protocols_excerpt}
<Separator />
<div class="space-y-2">
<p class="text-sm font-medium">Вывод birdc (protocols)</p>
<ScrollPreBlock variant="preserve" text={bird.protocols_excerpt} class="max-h-48" />
</div>
{/if}
{/if}
</CardContent>
</Card>
{/if}
</div>
<div class="grid gap-4 lg:grid-cols-2">
{#if initialLoading}
<CardSkeleton />
<CardSkeleton />
{:else}
<Card>
<CardHeader>
<div class="flex items-start justify-between gap-2">
<div>
<CardTitle class="flex items-center gap-2 text-base">
<Activity class="size-4" />
Задачи
</CardTitle>
<CardDescription>Последние 100 задач · GET /v1/jobs</CardDescription>
</div>
<Button variant="outline" size="sm" href={resolve('/operations')}>Все операции</Button>
</div>
</CardHeader>
<CardContent class="space-y-4">
{#if jobsError}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Не удалось загрузить задачи</AlertTitle>
<AlertDescription>{jobsError}</AlertDescription>
</Alert>
{:else if jobs}
<div class="flex flex-wrap gap-4 text-sm">
<div>
<p class="text-muted-foreground">Активных</p>
<p class="text-2xl font-bold tabular-nums">{jobs.running}</p>
</div>
<div>
<p class="text-muted-foreground">С ошибками</p>
<p
class={cn(
'text-2xl font-bold tabular-nums',
jobs.failed > 0 ? 'text-warning' : 'text-success'
)}
>
{jobs.failed}
</p>
</div>
<div>
<p class="text-muted-foreground">В выборке</p>
<p class="text-2xl font-bold tabular-nums">{jobs.total}</p>
</div>
</div>
<Separator />
{#if failedJobs.length > 0}
<div class="space-y-3">
<p class="text-sm font-medium">Последние ошибки</p>
<ul class="space-y-2">
{#each failedJobs as job (job.job_id)}
<li class="rounded-lg border px-3 py-2 text-sm">
<div class="flex items-start justify-between gap-2">
<p class="font-medium">{jobKindTitle(job)}</p>
<Badge variant="destructive">{jobStatusRu(job.status)}</Badge>
{@const readyBadge = readinessBadge(ready)}
<TableRow>
<TableCell>
<div class="flex items-center gap-2">
<ShieldCheck
class="size-4 shrink-0 text-muted-foreground"
aria-hidden="true"
/>
<div>
<p class="text-sm font-medium">Readiness</p>
<p class="text-xs text-muted-foreground">/v1/ready</p>
</div>
</div>
{#if job.error}
<p class="mt-1 text-xs text-muted-foreground">
{truncateError(job.error)}
</p>
{/if}
</li>
{/each}
</ul>
</div>
{:else}
<p class="text-sm text-muted-foreground">
Критичных сбоев в последних 100 задачах нет.
</p>
{/if}
{/if}
</CardContent>
</Card>
</TableCell>
<TableCell>
<Badge variant={readyBadge.variant} class={readyBadge.class}
>{readyBadge.label}</Badge
>
</TableCell>
</TableRow>
{#if ready?.checks && Object.keys(ready.checks).length > 0}
<TableRow>
<TableCell colspan={2} class="bg-muted/30 py-2">
<p class="text-xs font-medium text-muted-foreground">Зависимости</p>
</TableCell>
</TableRow>
{#each Object.entries(ready.checks) as [key, value] (key)}
{@const badge = checkStatusBadge(value)}
{@const CheckIcon = checkIconByKey[key] ?? ListTodo}
<TableRow>
<TableCell>
<div class="flex items-center gap-2">
<CheckIcon
class="size-4 shrink-0 text-muted-foreground"
aria-hidden="true"
/>
<div>
<p class="text-sm font-medium">{checkDisplayName(key)}</p>
<p class="text-xs text-muted-foreground">{key}</p>
</div>
</div>
</TableCell>
<TableCell>
<div class="space-y-1">
<Badge variant={badge.variant} class={badge.class}>{badge.label}</Badge>
{#if badge.hint}
<p class="text-xs text-muted-foreground">{badge.hint}</p>
{/if}
</div>
</TableCell>
</TableRow>
{/each}
{/if}
</TableBody>
</Table>
<Card>
<CardHeader>
<CardTitle class="flex items-center gap-2 text-base">
<AlertTriangle class="size-4 text-muted-foreground" />
Что проверять при деградации
</CardTitle>
<CardDescription>Короткая шпаргалка для triage</CardDescription>
</CardHeader>
<CardContent class="space-y-3">
<Alert>
<HeartPulse class="size-4" />
<AlertTitle>API недоступен</AlertTitle>
<AlertDescription>
Если <code class="text-xs">/v1/health</code> возвращает ошибку — проверьте процесс API и
его логи.
</AlertDescription>
</Alert>
<Alert>
<Database class="size-4" />
<AlertTitle>Readiness не «Готов»</AlertTitle>
<AlertDescription>
Сначала <code class="text-xs">postgres</code>, затем
<code class="text-xs">store</code> и <code class="text-xs">jobs</code> в checks.
</AlertDescription>
</Alert>
<Alert>
<Bird class="size-4" />
<AlertTitle>Низкий ratio BGP</AlertTitle>
<AlertDescription>
Проверьте <code class="text-xs">/v1/bird/status</code>, затем состояние пиров в
<Button variant="link" class="h-auto p-0" href={resolve('/network?tab=overview')}
>Сети</Button
>.
</AlertDescription>
</Alert>
<Alert>
<ListTodo class="size-4" />
<AlertTitle>Ошибки задач</AlertTitle>
<AlertDescription>
Откройте
<Button variant="link" class="h-auto p-0" href={resolve('/operations')}
>Операции</Button
>
и проверьте последние неуспешные jobs.
</AlertDescription>
</Alert>
</CardContent>
</Card>
{/if}
</div>
<p class="text-xs text-muted-foreground">
HTTP 503 на readiness означает недоступность одной из зависимостей в checks.
</p>
</CardContent>
</Card>
<Card>
<CardHeader>
<div class="flex items-start justify-between gap-2">
<div>
<CardTitle class="flex items-center gap-2 text-base">
<Bird class="size-4" />
BGP на API-хосте
</CardTitle>
<CardDescription>GET /v1/bird/status</CardDescription>
</div>
<Button variant="outline" size="sm" href={resolve('/network?tab=overview')}
>Пиры и спикеры</Button
>
</div>
</CardHeader>
<CardContent class="space-y-4">
{#if birdError}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Ошибка birdc</AlertTitle>
<AlertDescription>{birdError}</AlertDescription>
</Alert>
{:else if bird && !bird.birdc_configured}
<p class="text-sm text-muted-foreground">
{bird.message ?? 'birdc не настроен на API-хосте (EVOBGP_BIRDC_SOCKET).'}
</p>
{:else if bird}
<div class="space-y-2">
<div class="flex items-center justify-between text-sm">
<span class="text-muted-foreground">Established / total</span>
<span class="font-medium tabular-nums">
{bird.bgp_established} / {bird.bgp_sessions_total}
{#if bgpRatio !== null}
<span class="text-muted-foreground">({bgpRatio}%)</span>
{/if}
</span>
</div>
{#if bgpRatio !== null}
<div class="h-2 overflow-hidden rounded-full bg-muted">
<div
class={cn(
'h-full rounded-full transition-all',
bgpRatio >= 100
? 'bg-success'
: bgpRatio >= 50
? 'bg-warning'
: 'bg-destructive'
)}
style="width: {bgpRatio}%"
></div>
</div>
{/if}
{#if bird.error}
<p class="text-xs text-destructive">{bird.error}</p>
{/if}
</div>
{#if bird.protocols_excerpt}
<Separator />
<div class="space-y-2">
<p class="text-sm font-medium">Вывод birdc (protocols)</p>
<ScrollPreBlock
variant="preserve"
text={bird.protocols_excerpt}
class="max-h-48"
/>
</div>
{/if}
{/if}
</CardContent>
</Card>
{/if}
</div>
<div class="grid gap-4 lg:grid-cols-2">
{#if initialLoading}
<CardSkeleton />
<CardSkeleton />
{:else}
<Card>
<CardHeader>
<div class="flex items-start justify-between gap-2">
<div>
<CardTitle class="flex items-center gap-2 text-base">
<Activity class="size-4" />
Задачи
</CardTitle>
<CardDescription>Последние 100 задач · GET /v1/jobs</CardDescription>
</div>
<Button variant="outline" size="sm" href={resolve('/operations')}
>Все операции</Button
>
</div>
</CardHeader>
<CardContent class="space-y-4">
{#if jobsError}
<Alert variant="destructive">
<XCircle />
<AlertTitle>Не удалось загрузить задачи</AlertTitle>
<AlertDescription>{jobsError}</AlertDescription>
</Alert>
{:else if jobs}
<div class="flex flex-wrap gap-4 text-sm">
<div>
<p class="text-muted-foreground">Активных</p>
<p class="text-2xl font-bold tabular-nums">{jobs.running}</p>
</div>
<div>
<p class="text-muted-foreground">С ошибками</p>
<p
class={cn(
'text-2xl font-bold tabular-nums',
jobs.failed > 0 ? 'text-warning' : 'text-success'
)}
>
{jobs.failed}
</p>
</div>
<div>
<p class="text-muted-foreground">В выборке</p>
<p class="text-2xl font-bold tabular-nums">{jobs.total}</p>
</div>
</div>
<Separator />
{#if failedJobs.length > 0}
<div class="space-y-3">
<p class="text-sm font-medium">Последние ошибки</p>
<ul class="space-y-2">
{#each failedJobs as job (job.job_id)}
<li class="rounded-lg border px-3 py-2 text-sm">
<div class="flex items-start justify-between gap-2">
<p class="font-medium">{jobKindTitle(job)}</p>
<Badge variant="destructive">{jobStatusRu(job.status)}</Badge>
</div>
{#if job.error}
<p class="mt-1 text-xs text-muted-foreground">
{truncateError(job.error)}
</p>
{/if}
</li>
{/each}
</ul>
</div>
{:else}
<p class="text-sm text-muted-foreground">
Критичных сбоев в последних 100 задачах нет.
</p>
{/if}
{/if}
</CardContent>
</Card>
<Card>
<CardHeader>
<CardTitle class="flex items-center gap-2 text-base">
<AlertTriangle class="size-4 text-muted-foreground" />
Что проверять при деградации
</CardTitle>
<CardDescription>Короткая шпаргалка для triage</CardDescription>
</CardHeader>
<CardContent class="space-y-3">
<Alert>
<HeartPulse class="size-4" />
<AlertTitle>API недоступен</AlertTitle>
<AlertDescription>
Если <code class="text-xs">/v1/health</code> возвращает ошибку — проверьте процесс API
и его логи.
</AlertDescription>
</Alert>
<Alert>
<Database class="size-4" />
<AlertTitle>Readiness не «Готов»</AlertTitle>
<AlertDescription>
Сначала <code class="text-xs">postgres</code>, затем
<code class="text-xs">store</code> и <code class="text-xs">jobs</code> в checks.
</AlertDescription>
</Alert>
<Alert>
<Bird class="size-4" />
<AlertTitle>Низкий ratio BGP</AlertTitle>
<AlertDescription>
Проверьте <code class="text-xs">/v1/bird/status</code>, затем состояние пиров в
<Button variant="link" class="h-auto p-0" href={resolve('/network?tab=overview')}
>Сети</Button
>.
</AlertDescription>
</Alert>
<Alert>
<ListTodo class="size-4" />
<AlertTitle>Ошибки задач</AlertTitle>
<AlertDescription>
Откройте
<Button variant="link" class="h-auto p-0" href={resolve('/operations')}
>Операции</Button
>
и проверьте последние неуспешные jobs.
</AlertDescription>
</Alert>
</CardContent>
</Card>
{/if}
</div>
</TabsContent>
<TabsContent value="postgres" class="mt-4">
<MonitoringPostgresTab />
</TabsContent>
</Tabs>
</div>