Compare commits
111
Commits
07f2ce562e
...
v1.4.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
c66cc9317d | ||
|
|
9ea4a68ccf | ||
|
|
8204105fd6 | ||
|
|
0c11ecfa48 | ||
|
|
ab8660ac42 | ||
|
|
ce747673fd | ||
|
|
e558570967 | ||
|
|
b6b081b04f | ||
|
|
13b4c20148 | ||
|
|
700d415422 | ||
|
|
6ca34b9008 | ||
|
|
bb334b10f5 | ||
|
|
4c23232c4e | ||
|
|
c263fd5c7e | ||
|
|
6bee5adc34 | ||
|
|
64516c316d | ||
|
|
ea65451160 | ||
|
|
250cdea35d | ||
|
|
1dd713514c | ||
|
|
aff27e8f7b | ||
|
|
07b97bddc1 | ||
|
|
34ecc5c235 | ||
|
|
a536a2d5eb | ||
|
|
765a8052da | ||
|
|
5e59a1c334 | ||
|
|
92f6de8695 | ||
|
|
87309cfcec | ||
|
|
1f9930d467 | ||
|
|
9808aca44f | ||
|
|
3a3056ba76 | ||
|
|
57dcf306f3 | ||
|
|
b992301c99 | ||
|
|
77a2cfd426 | ||
|
|
ac635e91eb | ||
|
|
2d78c8156f | ||
|
|
00a2a44630 | ||
|
|
35ed79bdd6 | ||
|
|
0cb5a7d4cb | ||
|
|
fefadac1bf | ||
|
|
7d52a4af63 | ||
|
|
9ab7de65ce | ||
|
|
6025f383f1 | ||
|
|
27e01d010f | ||
|
|
b5061a58d9 | ||
|
|
80229b33fc | ||
|
|
7a2b015b12 | ||
|
|
ee364c8b6d | ||
|
|
1a61ae2d71 | ||
|
|
458a885868 | ||
|
|
1e13ef9e70 | ||
|
|
47764345f6 | ||
|
|
8a50ba44b3 | ||
|
|
6107c3eb87 | ||
|
|
3b17228ef2 | ||
|
|
e700f90c47 | ||
|
|
1765ab89e7 | ||
|
|
b99fc179ab | ||
|
|
1f85654653 | ||
|
|
05ec176c7a | ||
|
|
25374a6919 | ||
|
|
dc1f4a59b0 | ||
|
|
ae9fbe30b4 | ||
|
|
9f021289c0 | ||
|
|
40c8a8d38a | ||
|
|
d91667f698 | ||
|
|
1386b63db2 | ||
|
|
4f50350990 | ||
|
|
10e2415cfa | ||
|
|
2260ecf74a | ||
|
|
6d0e214655 | ||
|
|
f6b94a44d0 | ||
|
|
39ba976454 | ||
|
|
9163aeca1e | ||
|
|
6c8e7c55ad | ||
|
|
c811c43bbc | ||
|
|
b7a8aab2e8 | ||
|
|
00bd03735a | ||
|
|
4acd6857c9 | ||
|
|
1e77e1a2b8 | ||
|
|
f877ec89a7 | ||
|
|
2da6d49368 | ||
|
|
53fd527a6d | ||
|
|
e1c4cbce93 | ||
|
|
4aaaa30b1b | ||
|
|
d83efe24ef | ||
|
|
149fb0a18c | ||
|
|
b140dcbcf2 | ||
|
|
b541794d84 | ||
|
|
0ca5542f6b | ||
|
|
6a7284245f | ||
|
|
67ce135b46 | ||
|
|
88226f6dd9 | ||
|
|
57c09acb3d | ||
|
|
97908b50c8 | ||
|
|
9f0b1281f2 | ||
|
|
1c5a9c91bb | ||
|
|
0428b83eb5 | ||
|
|
e816d175ea | ||
|
|
bae2d68803 | ||
|
|
233f29b437 | ||
|
|
4ba5b03f0b | ||
|
|
655d7507e5 | ||
|
|
05bc5c435e | ||
|
|
c48cc4e2d4 | ||
|
|
678bd27085 | ||
|
|
42d956d109 | ||
|
|
e20c9f3113 | ||
|
|
a2cbfacd45 | ||
|
|
482eeb122d | ||
|
|
dc5e20e9b2 | ||
|
|
20e674c1f2 |
No files matched your search
@@ -0,0 +1,3 @@
|
|||||||
|
{
|
||||||
|
"extends": ["@commitlint/config-conventional"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,37 @@
|
|||||||
|
# Сгенерировать commit message (EvoBGP)
|
||||||
|
|
||||||
|
Сгенерируй сообщение коммита для **текущих staged-изменений**. Не выполняй `git commit`, если пользователь явно не просил закоммитить.
|
||||||
|
|
||||||
|
## Обязательный workflow (MUST)
|
||||||
|
|
||||||
|
1. Прочитай скилл [`.cursor/skills/commit-message/SKILL.md`](../skills/commit-message/SKILL.md).
|
||||||
|
2. Следуй правилу [`.cursor/rules/conventional-commits.mdc`](../rules/conventional-commits.mdc).
|
||||||
|
3. **Первым вызовом Shell** из корня репозитория:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
powershell -NoProfile -File scripts/commit/staged-context.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
4. Строй текст **только** по JSON stdout (`groups`, `stat`, `diff_excerpt`). Exit `1` → index пуст, сообщи пользователю.
|
||||||
|
5. **Запрещено** обходить скрипт через один `git diff --cached`.
|
||||||
|
|
||||||
|
## Формат вывода
|
||||||
|
|
||||||
|
Для каждого коммита (при auto-split — по одному блоку):
|
||||||
|
|
||||||
|
```
|
||||||
|
<type>(<scope>): <summary in English>
|
||||||
|
|
||||||
|
<тело на русском>
|
||||||
|
```
|
||||||
|
|
||||||
|
Плюс пояснение (RU): semver impact (`minor`|`patch`|`none`|`major`), почему выбран type, был ли split.
|
||||||
|
|
||||||
|
## Semver (кратко)
|
||||||
|
|
||||||
|
- Новая пользовательская возможность → `feat` (minor)
|
||||||
|
- Починка ожидаемого поведения / баг → `fix` (patch)
|
||||||
|
- Follow-up баги после недавнего `feat` в том же scope → **`fix`**, не `feat`
|
||||||
|
- Только перестройка без нового поведения → `refactor` (none)
|
||||||
|
|
||||||
|
Заголовок — EN, императив, ≤72 символов. Тело — RU.
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
---
|
||||||
|
description: >-
|
||||||
|
Conventional Commits EvoBGP. Триггеры commit/коммит/закоммить/staged/commit message —
|
||||||
|
ОБЯЗАТЕЛЬНО сначала Shell scripts/commit/staged-context.ps1, скилл commit-message.
|
||||||
|
alwaysApply: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# Conventional Commits (EvoBGP)
|
||||||
|
|
||||||
|
Полное описание релизного пайплайна: [docs/releasing.md](../../docs/releasing.md).
|
||||||
|
|
||||||
|
## Триггеры (применить правило + скилл)
|
||||||
|
|
||||||
|
Любой запрос на коммит или сообщение коммита: `commit`, `коммит`, `закоммить`, `git commit`, `commit message`, `conventional commit`, `staged`, «сгенерируй коммит», **`/commit-message`** — в т.ч. если это указано в плане или [AGENTS.md](../../AGENTS.md).
|
||||||
|
|
||||||
|
### Не путать с кнопкой ✨ в Source Control
|
||||||
|
|
||||||
|
Команда **`cursor.generateGitCommitMessage`** (sparkle в поле commit message) **не** читает Rules, Skills и `staged-context.ps1` — только staged diff и история коммитов ([ограничение Cursor](https://forum.cursor.com/t/how-to-set-prompt-for-generate-commit-message/148606)).
|
||||||
|
|
||||||
|
**Замена для EvoBGP:** Agent → `/commit-message` или команда [`.cursor/commands/commit-message.md`](../commands/commit-message.md).
|
||||||
|
|
||||||
|
## Обязательный запуск скрипта (MUST)
|
||||||
|
|
||||||
|
1. Прочитать скилл [`.cursor/skills/commit-message/SKILL.md`](../skills/commit-message/SKILL.md).
|
||||||
|
2. **Первым действием** выполнить Shell (из корня репо):
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
powershell -NoProfile -File scripts/commit/staged-context.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Сообщение коммита строить **только** по JSON из stdout скрипта (`groups`, `diff_excerpt`, `stat`).
|
||||||
|
4. **Запрещено** генерировать commit message без успешного (exit 0) запуска скрипта; не заменять скрипт одним `git diff --cached`.
|
||||||
|
|
||||||
|
## Формат (строго)
|
||||||
|
|
||||||
|
```
|
||||||
|
<type>(<scope>): <short summary in English>
|
||||||
|
|
||||||
|
<тело на русском: что изменено и зачем>
|
||||||
|
```
|
||||||
|
|
||||||
|
- **Заголовок** — только английский; императив, без точки в конце; ≤72 символов.
|
||||||
|
- **Тело** — только русский; полные предложения; пустая строка после заголовка.
|
||||||
|
- Запрещены vague-сообщения: `fix bug`, `update code`, `wip`, `misc`.
|
||||||
|
|
||||||
|
## Типы (semantic-release)
|
||||||
|
|
||||||
|
| type | Когда | Версия |
|
||||||
|
|------|--------|--------|
|
||||||
|
| `feat` | **новая** пользовательская возможность (раньше нельзя было) | minor |
|
||||||
|
| `fix` | восстановление **ожидаемого** поведения; баг, регрессия, падение UI | patch |
|
||||||
|
| `perf` | ускорение без смены API | patch |
|
||||||
|
| `refactor` | реструктуризация **без** новой возможности и **без** исправления бага | — |
|
||||||
|
| `docs` | только документация | — |
|
||||||
|
| `test` | тесты | — |
|
||||||
|
| `ci` | CI/CD (`.gitea/`, workflows) | — |
|
||||||
|
| `chore` | обслуживание, deps, `.cursor/` | — |
|
||||||
|
|
||||||
|
### Выбор type: semver, а не «красивые слова»
|
||||||
|
|
||||||
|
**Главный вопрос:** что изменится для пользователя после релиза?
|
||||||
|
|
||||||
|
1. Появилось **новое** действие / экран / API / настройка, которых не было → `feat`
|
||||||
|
2. То, что **должно было работать**, не работало (кнопки, диалоги, сохранение, 500) → `fix`
|
||||||
|
3. Только перестройка кода или UI на другой паттерн, поведение для пользователя то же → `refactor`
|
||||||
|
4. Ускорение без изменения контракта → `perf`
|
||||||
|
|
||||||
|
**Не путать с формулировкой diff:**
|
||||||
|
|
||||||
|
| В diff / задаче часто пишут | Неверный type | Верный type, если… |
|
||||||
|
|-----------------------------|---------------|---------------------|
|
||||||
|
| enhance, improve, polish UI | `feat` | …только чиним сломанное после прошлого PR → `fix` |
|
||||||
|
| refactor pages, unify tables | `feat` | …новой возможности нет, лишь перенос на AppDataTable → `refactor` |
|
||||||
|
| follow-up после feat(web) | `feat` | …исправляем баги того же экрана → `fix` |
|
||||||
|
|
||||||
|
**Follow-up rule:** коммит сразу после `feat` в той же области, который **не добавляет** новую возможность, а устраняет дефект (effect loop, не открывается dialog, confirm не срабатывает) — **`fix`**, не `feat`.
|
||||||
|
|
||||||
|
**Split при смешанном diff:** новая страница/flow → `feat`; отдельным коммитом правки багов → `fix`. Не объединять в один `feat`.
|
||||||
|
|
||||||
|
**Breaking changes** — только `feat!` / `fix!` / `BREAKING CHANGE:` когда пользователь **обязан** менять конфиг, API или привычный workflow.
|
||||||
|
|
||||||
|
### Обязательно в пояснении агенту
|
||||||
|
|
||||||
|
При каждом предложении коммита указать:
|
||||||
|
|
||||||
|
- **Semver impact:** `minor` | `patch` | `none` | `major`
|
||||||
|
- **Почему не другой type** (одно предложение), если diff большой или формулировка двусмысленная
|
||||||
|
|
||||||
|
Пример неправильно / правильно:
|
||||||
|
|
||||||
|
```
|
||||||
|
# Плохо — patch-фикс, minor-bump
|
||||||
|
feat(web): enhance module entry dialogs and selection handling
|
||||||
|
|
||||||
|
# Хорошо
|
||||||
|
fix(web): stop effect loop breaking module action buttons
|
||||||
|
|
||||||
|
Исправлен effect_update_depth_exceeded и bind:open у Dialog; кнопки редактирования/удаления снова работают.
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
# Плохо — рефакторинг без новой фичи
|
||||||
|
feat(web): migrate modules list to AppDataTable
|
||||||
|
|
||||||
|
# Хорошо — если не было нового user-facing
|
||||||
|
refactor(web): migrate modules list to AppDataTable
|
||||||
|
|
||||||
|
Единый паттерн таблиц; поведение списка модулей без изменений.
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
# Хорошо feat — действительно новое
|
||||||
|
feat(web): add module create dialog on /modules
|
||||||
|
|
||||||
|
Диалог создания модуля с POST /v1/modules; раньше создание было только через API.
|
||||||
|
```
|
||||||
|
|
||||||
|
## Breaking changes
|
||||||
|
|
||||||
|
- Заголовок: `feat!` / `fix!` **или** в теле строка `BREAKING CHANGE:` (на английском ключевое слово) + описание impact **на русском**.
|
||||||
|
|
||||||
|
## Scope (EvoBGP)
|
||||||
|
|
||||||
|
Выбирать по доминирующему пути из staged diff:
|
||||||
|
|
||||||
|
| Префикс | scope |
|
||||||
|
|---------|-------|
|
||||||
|
| `internal/httpapi/` | `httpapi` |
|
||||||
|
| `internal/store/`, `internal/repository/`, `internal/db/` | `store` |
|
||||||
|
| `internal/jobs/` | `jobs` |
|
||||||
|
| `internal/pipeline/` | `pipeline` |
|
||||||
|
| `internal/birdfmt/` | `birdfmt` |
|
||||||
|
| `internal/birddeploy/` | `birddeploy` |
|
||||||
|
| `internal/bundle/`, `internal/signing/` | `bundle` |
|
||||||
|
| `cmd/` | `cmd` |
|
||||||
|
| `web/` | `web` |
|
||||||
|
| `docs/openapi.yaml`, `redocly.yaml` | `openapi` |
|
||||||
|
| `docs/` (остальное) | `docs` |
|
||||||
|
| `migrations/` | `db` |
|
||||||
|
| `.gitea/` | `ci` |
|
||||||
|
| `deploy/` | `deploy` |
|
||||||
|
| `.cursor/` | `chore` |
|
||||||
|
| прочее в корне | `chore` |
|
||||||
|
|
||||||
|
`type` определять по **содержимому diff**, не только по пути.
|
||||||
|
|
||||||
|
## Multi-change
|
||||||
|
|
||||||
|
- Несвязанные области → **отдельные коммиты** (auto-split по скиллу).
|
||||||
|
- Одна логическая фича через несколько scope (OpenAPI + httpapi + web) → **один** коммит, scope по главной области.
|
||||||
|
|
||||||
|
## Обязательный вывод агенту
|
||||||
|
|
||||||
|
Для каждого коммита:
|
||||||
|
|
||||||
|
**1. Готовое сообщение** (копировать в `git commit -m` / HEREDOC):
|
||||||
|
|
||||||
|
```
|
||||||
|
<type>(<scope>): <summary>
|
||||||
|
|
||||||
|
<тело RU>
|
||||||
|
```
|
||||||
|
|
||||||
|
**2. Пояснение (RU):** semver impact (`minor`|`patch`|`none`|`major`); почему выбран type; риск/impact; был ли split.
|
||||||
|
|
||||||
|
## Примеры
|
||||||
|
|
||||||
|
```
|
||||||
|
feat(httpapi): add version endpoint and wire web footer
|
||||||
|
|
||||||
|
Добавлен GET /v1/version. В Web UI версия в footer берётся из API вместо захардкоженного значения.
|
||||||
|
```
|
||||||
|
|
||||||
|
```
|
||||||
|
fix(auth): correct token validation edge case
|
||||||
|
|
||||||
|
Исправлена ложная 401 при истёкшем refresh-токене с валидной сессией.
|
||||||
|
```
|
||||||
@@ -0,0 +1,241 @@
|
|||||||
|
---
|
||||||
|
description: Инженерные правила EvoBGP — Go, API, migrations, общие стандарты
|
||||||
|
alwaysApply: true
|
||||||
|
---
|
||||||
|
|
||||||
|
# Engineering Guidelines — EvoBGP
|
||||||
|
|
||||||
|
Специализированные правила: **Web UI** → `.cursor/rules/web-shadcn.mdc`; **BIRD2 / BGP / IP** → `.cursor/rules/networking-bird.mdc`.
|
||||||
|
|
||||||
|
Карта репозитория: `AGENTS.md`, `docs/architecture.md`. HTTP-контракт: `docs/openapi.yaml`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
**ARCH-01** | MUST | Новая persistence-логика — метод `store.Backend` + реализации в `repository` и `store.Memory`; SQL не в `httpapi`.
|
||||||
|
*Rationale:* единая абстракция данных.
|
||||||
|
*Проверка:* grep SQL в `internal/httpapi` — отсутствие; review.
|
||||||
|
|
||||||
|
**ARCH-02** | MUST | HTTP-маршруты только в `internal/httpapi`; регистрация через `http.ServeMux` с паттернами `METHOD /v1/...`.
|
||||||
|
*Rationale:* один слой REST.
|
||||||
|
*Проверка:* маршруты только в `routes.go`, `routes_crud.go`.
|
||||||
|
|
||||||
|
**ARCH-03** | MUST | Долгие операции (refresh, apply, rollback) — `jobs.Registry`; ответ `202` + `job_id` где задано OpenAPI.
|
||||||
|
*Rationale:* не блокировать HTTP worker.
|
||||||
|
*Проверка:* OpenAPI + handlers.
|
||||||
|
|
||||||
|
**ARCH-04** | MUST | Кросс-процессные воркеры — HTTP к API или общая БД; не shared memory (кроме `evobgp-all`).
|
||||||
|
*Rationale:* `jobs.Registry` in-process only.
|
||||||
|
*Проверка:* `docs/architecture.md`; review.
|
||||||
|
|
||||||
|
**ARCH-05** | MUST | `birdfmt` не импортирует `httpapi` / `store`.
|
||||||
|
*Rationale:* направление зависимостей вниз.
|
||||||
|
*Проверка:* `go list -deps` / review imports.
|
||||||
|
|
||||||
|
**ARCH-06** | MUST | Точки входа `cmd/*` — тонкий `main`: config, wiring, signal/shutdown; без бизнес-логики.
|
||||||
|
*Rationale:* тестируемость `internal/`.
|
||||||
|
*Проверка:* review `main.go`.
|
||||||
|
|
||||||
|
**ARCH-07** | MUST | Инициализация store + jobs для API и воркеров — `httpapi.BootstrapWorkers`.
|
||||||
|
*Rationale:* единый wiring.
|
||||||
|
*Проверка:* `bootstrap.go`.
|
||||||
|
|
||||||
|
**ARCH-08** | MUST | Ingest+render префиксов — `internal/pipeline`; генерация BIRD-текста — `internal/birdfmt`.
|
||||||
|
*Rationale:* разделение data plane / control plane.
|
||||||
|
*Проверка:* review пакетов.
|
||||||
|
|
||||||
|
**ARCH-09** | NEVER | Прямой доступ handler'ов к `pgxpool` для CRUD; только `store.Backend`.
|
||||||
|
*Rationale:* абстракция бэкенда.
|
||||||
|
*Проверка:* review `httpapi`.
|
||||||
|
|
||||||
|
**ARCH-10** | MUST | Подпись бандлов — `internal/bundle` + `internal/signing`; не дублировать crypto в handlers.
|
||||||
|
*Rationale:* единая криптография артефактов.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Code Style
|
||||||
|
|
||||||
|
**STYLE-01** | MUST | Go-код после `gofmt`; перед PR — `go vet ./...`.
|
||||||
|
*Rationale:* единый стиль.
|
||||||
|
*Проверка:* CI job `go`.
|
||||||
|
|
||||||
|
**STYLE-02** | MUST | Экспортируемые типы/функции публичных пакетов — godoc-комментарий.
|
||||||
|
*Rationale:* навигация по API пакетов.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**STYLE-03** | NEVER | `panic` в `internal/*` вне `init` и тестов.
|
||||||
|
*Rationale:* предсказуемые ошибки.
|
||||||
|
*Проверка:* grep `panic(`.
|
||||||
|
|
||||||
|
**STYLE-04** | MUST | Ошибки пакетов с префиксом (`birdfmt:`, `db:`, `httpapi:`) и `%w` при оборачивании.
|
||||||
|
*Rationale:* трассировка.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**STYLE-05** | MUST | HTTP-ошибки — `writeProblem` / `writeJSON` (`application/problem+json` для 4xx/5xx).
|
||||||
|
*Rationale:* RFC 9457, OpenAPI.
|
||||||
|
*Проверка:* `problem.go`.
|
||||||
|
|
||||||
|
**STYLE-06** | MUST | JSON полей HTTP DTO согласованы с `docs/openapi.yaml`.
|
||||||
|
*Rationale:* контракт API.
|
||||||
|
*Проверка:* OpenAPI diff + review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Dependency Management
|
||||||
|
|
||||||
|
**DEP-01** | MUST | Go-зависимости — через `go get` / `go.mod`; версия Go как в `go.mod` и CI (1.24).
|
||||||
|
*Проверка:* `go.mod`, `.gitea/workflows/ci.yaml`.
|
||||||
|
|
||||||
|
**DEP-02** | NEVER | Vendor-копирование без явного решения в репозитории.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**DEP-03** | MUST | Миграции схемы — пары `.up.sql`/`.down.sql` для **postgres** и **sqlite**, синхронная нумерация.
|
||||||
|
*Проверка:* `migrations/postgres/`, `migrations/sqlite/`.
|
||||||
|
|
||||||
|
**DEP-04** | MUST | Web UI-библиотеки — только экосистема shadcn-svelte/bits-ui (см. `web-shadcn.mdc`).
|
||||||
|
*Проверка:* `web/package.json` review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Error Handling
|
||||||
|
|
||||||
|
**ERR-01** | MUST | HTTP 5xx — без сырого `err.Error()` клиенту; detail через `writeProblem`.
|
||||||
|
*Проверка:* review handlers.
|
||||||
|
|
||||||
|
**ERR-02** | MUST | Публичные функции I/O — `(T, error)`; `errors.Is`/`errors.As` для sentinel.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**ERR-03** | MUST | `context.Context` — первый аргумент для I/O; таймауты на внешние HTTP (CDN, DoH, RIPEstat).
|
||||||
|
*Проверка:* `pipeline`, `bootstrap.go`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Testing
|
||||||
|
|
||||||
|
**TEST-01** | MUST | Перед PR: `go test ./... -race -count=1`.
|
||||||
|
*Проверка:* CI job `go`.
|
||||||
|
|
||||||
|
**TEST-02** | MUST | Табличные тесты — эталон для `birdfmt`, `pipeline`, `bundle`.
|
||||||
|
*Проверка:* `*_test.go`.
|
||||||
|
|
||||||
|
**TEST-03** | MUST | Новые BIRD-сценарии в `internal/birdfmt/testdata/scenarios/*/bird.conf` + `bird -p`.
|
||||||
|
*Проверка:* CI job `bird2`.
|
||||||
|
|
||||||
|
**TEST-04** | MUST | Изменения `web/` — локально `npm run check` и `npm run lint` (CI web пока не в scope).
|
||||||
|
*Проверка:* локальные команды.
|
||||||
|
|
||||||
|
**TEST-05** | MUST | Изменения OpenAPI — `npx @redocly/cli lint docs/openapi.yaml`.
|
||||||
|
*Проверка:* CI job `openapi`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation
|
||||||
|
|
||||||
|
**DOC-01** | MUST | Пользовательская документация в `docs/` — на русском.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**DOC-02** | MUST | Изменение HTTP API: сначала `docs/openapi.yaml`, затем handlers/store, затем `docs/api.md` при необходимости.
|
||||||
|
*Проверка:* PR diff order.
|
||||||
|
|
||||||
|
**DOC-03** | MUST | Расхождение spec/код — явно в `docs/access.md` или description операции OpenAPI.
|
||||||
|
*Проверка:* review (напр. `X-Tenant-Id`).
|
||||||
|
|
||||||
|
**DOC-04** | NEVER | Дублировать длинные фрагменты OpenAPI в комментариях; ссылка на путь/operationId.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Naming Conventions
|
||||||
|
|
||||||
|
**NAME-01** | MUST | Бинарники: `cmd/evobgp-<role>/main.go`.
|
||||||
|
*Проверка:* `cmd/`.
|
||||||
|
|
||||||
|
**NAME-02** | MUST | Env-переменные: префикс `EVOBGP_`.
|
||||||
|
*Проверка:* `internal/config`, `docs/access.md`.
|
||||||
|
|
||||||
|
**NAME-03** | MUST | Job kinds — константы в `internal/jobs/worker.go` (`module_refresh`, …).
|
||||||
|
*Проверка:* grep `Kind`.
|
||||||
|
|
||||||
|
**NAME-04** | MUST | ID сущностей — UUID-строки; tenant только из API-ключа (auth context), не из недокументированных заголовков.
|
||||||
|
*Проверка:* `auth.go`, `docs/access.md`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Performance
|
||||||
|
|
||||||
|
**PERF-01** | MUST | Списки API — пагинация `cursor` + `limit`; не unbounded выборки в handlers.
|
||||||
|
*Проверка:* OpenAPI + store methods.
|
||||||
|
|
||||||
|
**PERF-02** | MUST | CDN HTTP — переиспользуемый `http.Client` с таймаутом (45s в bootstrap).
|
||||||
|
*Проверка:* `bootstrap.go`.
|
||||||
|
|
||||||
|
**PERF-03** | MUST | Новые метрики — `internal/observability`, экспорт `/metrics`.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Security
|
||||||
|
|
||||||
|
**SEC-01** | NEVER | Секреты, API-ключи, токены в Git.
|
||||||
|
*Проверка:* review; `docs/access.md`.
|
||||||
|
|
||||||
|
**SEC-02** | NEVER | `EVOBGP_DEV_INSECURE=1` в production.
|
||||||
|
*Проверка:* ops review.
|
||||||
|
|
||||||
|
**SEC-03** | MUST | Роль `node` — только node API; CRUD — `403` для `node`.
|
||||||
|
*Проверка:* `auth.go`, handlers.
|
||||||
|
|
||||||
|
**SEC-04** | MUST | apply/rollback/settings — operator (или выше по `roleLevel`).
|
||||||
|
*Проверка:* review handlers.
|
||||||
|
|
||||||
|
**SEC-05** | MUST | Бандл на ноде — `verify-bundle` перед `apply-bundle`.
|
||||||
|
*Проверка:* `docs/access.md`.
|
||||||
|
|
||||||
|
**SEC-06** | MUST | CORS — явный whitelist `EVOBGP_CORS_ORIGINS`.
|
||||||
|
*Проверка:* `cors.go`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation Sync Rules
|
||||||
|
|
||||||
|
| Область | Источник |
|
||||||
|
|---------|----------|
|
||||||
|
| Go / net/http | https://go.dev/doc/ |
|
||||||
|
| pgx v5 | https://pkg.go.dev/github.com/jackc/pgx/v5 |
|
||||||
|
| OpenAPI / problem+json | `docs/openapi.yaml`, RFC 9457 |
|
||||||
|
| Svelte / Kit | https://svelte.dev/docs , https://kit.svelte.dev/docs |
|
||||||
|
| shadcn-svelte | https://shadcn-svelte.com/docs |
|
||||||
|
| BIRD 2 | https://bird.network.cz/?get_doc |
|
||||||
|
| Prometheus Go | https://pkg.go.dev/github.com/prometheus/client_golang |
|
||||||
|
|
||||||
|
**DOC-SYNC-01** | MUST | Новый API библиотеки — сверка версии в `go.mod`/`package.json` с официальной документацией.
|
||||||
|
**DOC-SYNC-02** | NEVER | Устаревшие примеры (Svelte 4 `export let`, deprecated pgx).
|
||||||
|
**DOC-SYNC-03** | MUST | Конфликт docs: **OpenAPI (HTTP)** → **код** → обзорные `docs/`; `.cursor/plans/` не контракт.
|
||||||
|
**DOC-SYNC-04** | MUST | Сомнения по Svelte — Svelte MCP / `npm run check`.
|
||||||
|
**DOC-SYNC-05** | MUST | BIRD — официальная документация BIRD2 + `networking-bird.mdc` + `go test ./internal/birdfmt/...`.
|
||||||
|
|
||||||
|
Приоритет при сомнениях — **официальные источники**, не блоги и не «память модели».
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Enforcement Strategy
|
||||||
|
|
||||||
|
**CI (Gitea):** OpenAPI lint; `go vet`, `go test -race`, build `cmd/*`; `bird -p` на scenarios; Docker bake на push.
|
||||||
|
|
||||||
|
**Локально перед PR:**
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
go vet ./...
|
||||||
|
go test ./... -race -count=1
|
||||||
|
npx @redocly/cli lint docs/openapi.yaml
|
||||||
|
# web: cd web; npm run check; npm run lint
|
||||||
|
# birdfmt: go test ./internal/birdfmt/... -count=1
|
||||||
|
```
|
||||||
|
|
||||||
|
**Рекомендуется (частично внедрено):** CI job `web` (Gitea); `scripts/lint-httpapi.sh` в job `go`; `.golangci.yml` (локально); pre-commit gofmt/prettier.
|
||||||
|
|
||||||
|
**Только code review:** слои SQL; роли; idempotency; OpenAPI bodies; secrets в compose.
|
||||||
|
|
||||||
|
**Известные ограничения:** `X-Tenant-Id` в OpenAPI не реализован в handlers; `jobs.Registry` не shared между процессами API и отдельными воркерами без HTTP.
|
||||||
@@ -0,0 +1,219 @@
|
|||||||
|
---
|
||||||
|
description: EvoBGP — BIRD2, BGP policy, IP/CIDR, маршрутизация, валидация конфигов
|
||||||
|
globs:
|
||||||
|
- internal/birdfmt/**
|
||||||
|
- internal/birddeploy/**
|
||||||
|
- internal/pipeline/**
|
||||||
|
- deploy/bird/**
|
||||||
|
- "**/testdata/scenarios/**"
|
||||||
|
alwaysApply: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# Networking & Routing Rules
|
||||||
|
|
||||||
|
Общие правила: `.cursor/rules/engineering.mdc`. Пакет BIRD: `internal/birdfmt/doc.go`.
|
||||||
|
|
||||||
|
## Single source of truth
|
||||||
|
|
||||||
|
| Домен | SSOT |
|
||||||
|
|-------|------|
|
||||||
|
| Префиксы для анонса | Модули tenant → materialized snapshot → ревизия (`store`, `pipeline`, `CreateRenderRevision`) |
|
||||||
|
| Текст BIRD-фрагментов | `internal/birdfmt` — только генератор `evobgp_*.conf` |
|
||||||
|
| Каркас `bird.conf` | Operator skeleton + `include` (`StandardIncludeFragments`) |
|
||||||
|
| Параметры BIRD tenant | Global settings: `bird_router_id`, `bird_local_asn`, … (`docs/manual.md`) |
|
||||||
|
| BGP peers | `BGPPeer` + `ParsePeerNeighbor` |
|
||||||
|
|
||||||
|
**BIRD2 docs:** https://bird.network.cz/?get_doc
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## A. BIRD2 Configuration Rules
|
||||||
|
|
||||||
|
**BIRD-01** | MUST | Имена фрагментов — константы `Fragment*` (`evobgp_prefixes_v4.conf`, …); include `bird.d/<name>`.
|
||||||
|
*Rationale:* совместимость deploy.
|
||||||
|
*Проверка:* `layout.go`; `go test ./internal/birdfmt/...`
|
||||||
|
|
||||||
|
**BIRD-02** | MUST | Порядок include: filters v4/v6 → BGP template → prefixes v4/v6 → peers (`StandardIncludeFragments`).
|
||||||
|
*Rationale:* фильтры до ссылок.
|
||||||
|
*Проверка:* scenario `standard_layout`; `bird -p`
|
||||||
|
|
||||||
|
**BIRD-03** | MUST | `bird.conf` (router id, device, direct, includes) отдельно от `bird.d/evobgp_*` (политика, префиксы).
|
||||||
|
*Rationale:* атомарная подмена фрагментов.
|
||||||
|
*Проверка:* review layout.
|
||||||
|
|
||||||
|
**BIRD-04** | MUST | Новая логика фильтрации — функция в `birdfmt` + unit-тест; NEVER дублировать filter-блок в два `.conf` вручную.
|
||||||
|
*Rationale:* DRY v4/v6.
|
||||||
|
*Проверка:* `filter_test.go`; review.
|
||||||
|
|
||||||
|
**BIRD-05** | MUST | Filter: префикс `evobgp_`, суффикс `_v4`/`_v6`; templates `bgp_template` / `bgp_template_v6`.
|
||||||
|
*Проверка:* `bgp.go`, `layout.go`.
|
||||||
|
|
||||||
|
**BIRD-06** | MUST | `router id` — валидный IPv4 dotted-quad (`RenderMainBirdConf`).
|
||||||
|
*Проверка:* `birdfmt` error; `bird -p`
|
||||||
|
|
||||||
|
**BIRD-07** | MUST | Communities — `RouteCommunityAttrs`; синтаксис `((a,b))` в `add()`, не `add(65000,1)`.
|
||||||
|
*Проверка:* `community_bird_test.go`
|
||||||
|
|
||||||
|
**BIRD-08** | SHOULD | Комментарии `#` в preamble; в фрагментах — revision/module id где уместно.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**BIRD-09** | NEVER | Ручное редактирование `evobgp_*.conf` на ноде без ревизии/бандла в control plane.
|
||||||
|
*Rationale:* drift.
|
||||||
|
*Проверка:* `evobgp-deploy` logs.
|
||||||
|
|
||||||
|
**BIRD-10** | MUST | Новый scenario: `testdata/scenarios/<name>/bird.conf` (+ `bird.d/`) проходит `bird -p`.
|
||||||
|
*Проверка:* CI `bird2`
|
||||||
|
|
||||||
|
**BIRD-11** | MUST | Только BIRD **2.x**, не 1.6.
|
||||||
|
*Проверка:* `bird --version`
|
||||||
|
|
||||||
|
**BIRD-12** | MUST | Apply: staging → `bird -p` → atomic swap → `birdc configure`; NEVER configure без parse check.
|
||||||
|
*Проверка:* `birdctl.go`; `birddeploy` review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## B. Routing Policy Rules
|
||||||
|
|
||||||
|
**RTE-01** | MUST | BGP template: `import none` на каждый AFI, если задача явно не добавляет import policy.
|
||||||
|
*Rationale:* deny-by-default.
|
||||||
|
*Проверка:* `bgp.go`
|
||||||
|
|
||||||
|
**RTE-02** | MUST | Export filter завершается `reject;`.
|
||||||
|
*Проверка:* `filter.go`; `filter_export` scenario.
|
||||||
|
|
||||||
|
**RTE-03** | MUST | Анонс только префиксов materialized revision; NEVER static `route` вне pipeline/birdfmt без operator approval.
|
||||||
|
*Проверка:* store snapshot; review.
|
||||||
|
|
||||||
|
**RTE-04** | NEVER | Анонс `0.0.0.0/0`, `::/0`, RFC1918/ULA, loopback, link-local, multicast, reserved без documented exception.
|
||||||
|
*Проверка:* ingest validation; review.
|
||||||
|
|
||||||
|
**RTE-05** | NEVER | Префиксы чужих ASN без авторизации (IRR/RPKI — ops review).
|
||||||
|
*Проверка:* ops review.
|
||||||
|
|
||||||
|
**RTE-06** | MUST | Отдельные export filters IPv4 и IPv6.
|
||||||
|
*Проверка:* `RenderExportFilterIPv4/IPv6`
|
||||||
|
|
||||||
|
**RTE-07** | SHOULD | AS_PATH match — ASN из tenant modules; дедуп `filterUniqueASNs`.
|
||||||
|
*Проверка:* unit tests.
|
||||||
|
|
||||||
|
**RTE-08** | MUST | Peer import/export overrides — расширение `birdfmt`, не ad-hoc в operator `bird.conf`.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**RTE-09** | SHOULD | hold 90s / keepalive 30s — менять только с обоснованием в settings/docs.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**RTE-10** | MUST | После apply — post-check `birdc`/метрики (`mergeBirdPostApplyMeta`, `EVOBGP_BIRDC_SOCKET`).
|
||||||
|
*Проверка:* jobs meta; observability poller.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## C. IP Addressing Rules
|
||||||
|
|
||||||
|
**IP-01** | MUST | CIDR в Go — `netip.Prefix`; строка — `Masked().String()`.
|
||||||
|
*Проверка:* `pipeline/parse.go`
|
||||||
|
|
||||||
|
**IP-02** | MUST | IPv4/IPv6 — отдельные fragments/filters.
|
||||||
|
*Проверка:* `layout.go`
|
||||||
|
|
||||||
|
**IP-03** | MUST | BGP neighbor — `ParsePeerNeighbor`; `/32`/`/128` нормализуются в host addr.
|
||||||
|
*Проверка:* `peer_neighbor_test.go`
|
||||||
|
|
||||||
|
**IP-04** | NEVER | Hardcoded production CIDR вне testdata/fixtures.
|
||||||
|
*Проверка:* grep; review.
|
||||||
|
|
||||||
|
**IP-05** | MUST | Fixtures/docs — TEST-NET (RFC 5737, 3849): `203.0.113.0/24`, `198.51.100.0/24`, `2001:db8::/32`.
|
||||||
|
*Проверка:* scenarios.
|
||||||
|
|
||||||
|
**IP-06** | MUST | Domain→IP: host routes `/32`, `/128` via `ipToHostPrefix`; invalid отбрасывается.
|
||||||
|
*Проверка:* `collect_parallel.go`
|
||||||
|
|
||||||
|
**IP-07** | MUST | IP range entries — валидный CIDR на границе API/store.
|
||||||
|
*Проверка:* httpapi; review.
|
||||||
|
|
||||||
|
**IP-08** | SHOULD | Множество префиксов — без дубликатов; агрегация — policy decision.
|
||||||
|
*Проверка:* snapshot dedup.
|
||||||
|
|
||||||
|
**IP-09** | MUST | `router id`, `local as` — из tenant settings, не hardcode в callers `birdfmt`.
|
||||||
|
*Проверка:* `docs/manual.md`
|
||||||
|
|
||||||
|
**IP-10** | NEVER | Reserved blocks (`0.0.0.0/8`, `127.0.0.0/8`, `169.254.0.0/16`, `224.0.0.0/4`, IPv6 analogs) в materialized prefixes.
|
||||||
|
*Проверка:* review; future validator.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## D. Internet Compliance
|
||||||
|
|
||||||
|
**COMP-01** | SHOULD | Policy changes — сверка RFC 4271, 4760, 7454 (обзор).
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**COMP-02** | MUST | No excessive deaggregation без need.
|
||||||
|
*Проверка:* review prefix list.
|
||||||
|
|
||||||
|
**COMP-03** | MUST | Standard community 0..65535; large — schema store.
|
||||||
|
*Проверка:* `community_bird_test.go`
|
||||||
|
|
||||||
|
**COMP-04** | NEVER | AS_PATH/MED manipulation без product spec.
|
||||||
|
*Проверка:* review output.
|
||||||
|
|
||||||
|
**COMP-05** | SHOULD | Revision/job meta: source, counts, samples (`revisionLogEntry`).
|
||||||
|
*Проверка:* jobs.
|
||||||
|
|
||||||
|
**COMP-06** | MUST | Node: `evobgp-node verify-bundle` перед apply.
|
||||||
|
*Проверка:* `docs/access.md`
|
||||||
|
|
||||||
|
**COMP-07** | SHOULD | `bird_local_asn` = `local as` в templates/peers.
|
||||||
|
*Проверка:* settings + birdfmt.
|
||||||
|
|
||||||
|
**COMP-08** | NEVER | Default route leak без named `default-originate` policy.
|
||||||
|
*Проверка:* export filter review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Validation & Observability
|
||||||
|
|
||||||
|
**VAL-01** | MUST | Перед деплоем: `bird -c <bird.conf> -p` (main + includes).
|
||||||
|
*Проверка:* CI `bird2`; `ParseCheck`
|
||||||
|
|
||||||
|
**VAL-02** | MUST | Preview ревизии read-only; apply — jobs/apply/bundle only.
|
||||||
|
*Проверка:* OpenAPI + httpapi.
|
||||||
|
|
||||||
|
**VAL-03** | MUST | Изменения birdfmt — `go test ./internal/birdfmt/...`.
|
||||||
|
*Проверка:* CI `go`
|
||||||
|
|
||||||
|
**VAL-04** | SHOULD | Dry-run apply где API поддерживает `dry_run`.
|
||||||
|
*Проверка:* routes.go.
|
||||||
|
|
||||||
|
**VAL-05** | MUST | `/metrics` Prometheus; `EVOBGP_BIRDC_SOCKET` — BGP session poller.
|
||||||
|
*Проверка:* `observability`
|
||||||
|
|
||||||
|
**VAL-06** | MUST | Логировать job_id, revision_id, speaker_id на apply/refresh/rollback.
|
||||||
|
*Проверка:* jobs API.
|
||||||
|
|
||||||
|
**VAL-07** | SHOULD | После apply — `birdc show protocols` / parsers в `birdfmt`.
|
||||||
|
*Проверка:* `protocols.go`
|
||||||
|
|
||||||
|
**VAL-08** | MUST | Мониторить drift: `last_applied_revision_id` vs published (`evobgp-deploy`).
|
||||||
|
*Проверка:* deploy worker.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation Sync
|
||||||
|
|
||||||
|
**DOC-SYNC-05** | MUST | BIRD — https://bird.network.cz/?get_doc
|
||||||
|
**DOC-SYNC-08** | MUST | BGP policy — RFC 4271, 4760, 7454 + BIRD docs + `birdfmt`
|
||||||
|
**DOC-SYNC-09** | MUST | CIDR — https://pkg.go.dev/net/netip ; примеры — RFC 5737, 3849
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Enforcement
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
go test ./internal/birdfmt/... ./internal/pipeline/... -count=1
|
||||||
|
bird -c internal/birdfmt/testdata/scenarios/<name>/bird.conf -p
|
||||||
|
```
|
||||||
|
|
||||||
|
**PR checklist `birdfmt/**`, `pipeline/**`, `deploy/bird/**`:**
|
||||||
|
- [ ] filter + unit-test + `bird -p`
|
||||||
|
- [ ] нет дубли filter logic v4/v6 без общей Go-функции
|
||||||
|
- [ ] префиксы только store/revision path
|
||||||
|
- [ ] `import none` на templates сохранён
|
||||||
@@ -0,0 +1,109 @@
|
|||||||
|
---
|
||||||
|
description: EvoBGP WebUI — shadcn-svelte, Svelte 5, слои ui/core|patterns|app
|
||||||
|
globs:
|
||||||
|
- web/**
|
||||||
|
alwaysApply: false
|
||||||
|
---
|
||||||
|
|
||||||
|
# Web UI — shadcn-svelte
|
||||||
|
|
||||||
|
**Источник правды:** https://shadcn-svelte.com/docs (не React shadcn/ui, не Legacy Docs).
|
||||||
|
|
||||||
|
Общие правила Go/API: `.cursor/rules/engineering.mdc`. Локальная карта: `web/README.md`.
|
||||||
|
|
||||||
|
## Слои UI
|
||||||
|
|
||||||
|
| Слой | Путь | Назначение |
|
||||||
|
|------|------|------------|
|
||||||
|
| Примитивы | `src/lib/ui/core/` | shadcn-svelte (только CLI `add`) |
|
||||||
|
| Паттерны | `src/lib/ui/patterns/` | FormField, AppDataTable, ConfirmDialog, EmptyState |
|
||||||
|
| App chrome | `src/lib/ui/app/` | Layout, PageHeader, `notify` |
|
||||||
|
| Legacy | `src/lib/components/ui/` | Re-export; **не добавлять новые файлы** |
|
||||||
|
|
||||||
|
Тема: `src/routes/layout.css`, `src/lib/ui/app/tokens.md`. CLI из `web/`: `npx shadcn-svelte@latest add <component> -y -o`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Правила
|
||||||
|
|
||||||
|
**WEB-01** | MUST | Перед новым UI — проверить https://shadcn-svelte.com/docs/components; использовать компонент, не HTML+CSS с нуля.
|
||||||
|
*Rationale:* Open Code + единый дизайн.
|
||||||
|
*Проверка:* review; нет голых `<button class=…>`.
|
||||||
|
|
||||||
|
**WEB-02** | MUST | Отсутствующий примитив — `npx shadcn-svelte@latest add <component> -y -o` → `src/lib/ui/core/`.
|
||||||
|
*Rationale:* Distribution через CLI и `components.json`.
|
||||||
|
*Проверка:* файлы только в `ui/core`.
|
||||||
|
|
||||||
|
**WEB-03** | NEVER | Альтернативные UI-kit'ы (Material, Vuetify, DaisyUI-only без shadcn-примитива).
|
||||||
|
*Проверка:* `package.json` review.
|
||||||
|
|
||||||
|
**WEB-04** | MUST | Комозиция по docs: все sub-компоненты (`DialogHeader`, `TableRow`, `Field`, …).
|
||||||
|
*Проверка:* сверка со страницей компонента в docs.
|
||||||
|
|
||||||
|
**WEB-05** | MUST | Формы — Formsnap + `sveltekit-superforms`; UI в `ui/patterns/form`, не ad-hoc валидация на странице.
|
||||||
|
*Проверка:* https://shadcn-svelte.com/docs/components/form
|
||||||
|
|
||||||
|
**WEB-06** | MUST | Таблицы — Data Table + `@tanstack/table-core`; на страницах — `AppDataTable` из patterns.
|
||||||
|
*Проверка:* https://shadcn-svelte.com/docs/components/data-table
|
||||||
|
|
||||||
|
**WEB-07** | MUST | Toast — Sonner через `notify` из `ui/app/toast.js`.
|
||||||
|
*Проверка:* https://shadcn-svelte.com/docs/components/sonner
|
||||||
|
|
||||||
|
**WEB-08** | MUST | Иконки — `@lucide/svelte` (`components.json` → `iconLibrary: lucide`).
|
||||||
|
*Проверка:* imports.
|
||||||
|
|
||||||
|
**WEB-09** | MUST | Цвета — CSS-переменные `layout.css` и токены `tokens.md`; не hex/rgb на страницах.
|
||||||
|
*Проверка:* grep `#[0-9a-f]{3,6}` в `routes/`.
|
||||||
|
|
||||||
|
**WEB-10** | SHOULD | Кастомизация — правка `ui/core` (Open Code), не `!important` поверх API.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**WEB-11** | MUST | `routes/**` — композиция `ui/core` + `ui/patterns` + `ui/app`; не копировать целые примитивы shadcn в route.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**WEB-12** | NEVER | Примеры React shadcn/ui или Svelte 4 Legacy без адаптации под https://shadcn-svelte.com/docs/migration/svelte-5
|
||||||
|
*Проверка:* `npm run check`.
|
||||||
|
|
||||||
|
**WEB-13** | MUST | Реактивность — Svelte 5 runes (`$state`, `$derived`, `$effect`); не `export let` для локального state страниц.
|
||||||
|
*Проверка:* `npm run check`; Svelte MCP.
|
||||||
|
|
||||||
|
**WEB-14** | SHOULD | Нетривиальный UI — прочитать страницу компонента (props, a11y).
|
||||||
|
*Проверка:* PR description.
|
||||||
|
|
||||||
|
**WEB-15** | MUST | Сомнения — https://shadcn-svelte.com/llms.txt , Svelte MCP, `npm run check`.
|
||||||
|
*Проверка:* локально.
|
||||||
|
|
||||||
|
**WEB-16** | MUST | Подтверждение удаления — `ConfirmDialog` из patterns, не `window.confirm`.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**WEB-17** | MUST | Пустые списки — `EmptyState` из patterns.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
**WEB-18** | SHOULD | Повторяемая комбинация core (≥2 раза) — вынести в `ui/patterns/`.
|
||||||
|
*Проверка:* review.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Documentation Sync (Web)
|
||||||
|
|
||||||
|
**DOC-SYNC-06** | MUST | UI — первично https://shadcn-svelte.com/docs; при конфликте с блогами/Stack Overflow побеждает официальная страница компонента.
|
||||||
|
**DOC-SYNC-07** | MUST | Перед `add` — сверить Installation/Theming с `web/components.json` и `src/routes/layout.css`.
|
||||||
|
|
||||||
|
Tailwind v4: https://shadcn-svelte.com/docs/migration/tailwind-v4
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Enforcement
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
cd web
|
||||||
|
npm run check
|
||||||
|
npm run lint
|
||||||
|
```
|
||||||
|
|
||||||
|
**PR checklist `web/**`:**
|
||||||
|
- [ ] `ui/core` / `ui/patterns`, не дубли примитивов
|
||||||
|
- [ ] Новые примитивы через shadcn CLI
|
||||||
|
- [ ] Ссылка на docs компонента (если новый паттерн)
|
||||||
|
|
||||||
|
**CI:** job `web` рекомендован; пока обязательно локально.
|
||||||
@@ -0,0 +1,143 @@
|
|||||||
|
---
|
||||||
|
name: commit-message
|
||||||
|
description: >-
|
||||||
|
ОБЯЗАТЕЛЬНО при commit, коммит, закоммить, commit message, conventional commit,
|
||||||
|
staged, semantic-release, «сгенерируй коммит», git commit, /commit-message: ПЕРВЫМ
|
||||||
|
делом Shell — scripts/commit/staged-context.ps1; затем Conventional Commit (заголовок EN, тело RU).
|
||||||
|
---
|
||||||
|
|
||||||
|
> **Кнопка ✨ Generate commit message в Source Control** не использует этот скилл и Rules.
|
||||||
|
> Эквивалент: Agent Chat → **`/commit-message`** или «сгенерируй коммит по staged».
|
||||||
|
> См. [docs/README.md](../../docs/README.md#сообщения-коммитов-cursor).
|
||||||
|
|
||||||
|
# Commit message (EvoBGP)
|
||||||
|
|
||||||
|
## Когда применять (сразу читать этот скилл)
|
||||||
|
|
||||||
|
Триггеры в промпте пользователя или в плане/AGENTS.md:
|
||||||
|
|
||||||
|
- `commit`, `коммит`, `закоммить`, `commit message`, `conventional commit`
|
||||||
|
- `git commit`, `staged`, `сообщение коммита`, `сгенерируй коммит`
|
||||||
|
- агент собирается выполнить `git commit` или предложить текст коммита
|
||||||
|
|
||||||
|
## Шаг 0 — ОБЯЗАТЕЛЬНО (до любого текста коммита)
|
||||||
|
|
||||||
|
**Первый вызов инструментов** в этой задаче — Shell из корня репозитория:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
powershell -NoProfile -File scripts/commit/staged-context.ps1
|
||||||
|
```
|
||||||
|
|
||||||
|
| Правило | Деталь |
|
||||||
|
|---------|--------|
|
||||||
|
| **MUST** | Запустить скрипт до генерации заголовка/тела коммита |
|
||||||
|
| **MUST NOT** | Строить сообщение только по `git diff --cached` / `git status` без скрипта |
|
||||||
|
| **MUST NOT** | Пропускать скрипт, даже если diff «и так понятен» |
|
||||||
|
| Exit `1` | Index пуст — сообщить пользователю, **не коммитить** |
|
||||||
|
| Exit `0` | Разобрать JSON stdout: `staged_count`, `groups[]` (`scope`, `files`, `stat`, `diff_excerpt`) |
|
||||||
|
|
||||||
|
Дополнительно: [.cursor/rules/conventional-commits.mdc](../../rules/conventional-commits.mdc).
|
||||||
|
|
||||||
|
## Слияние групп (одна фича)
|
||||||
|
|
||||||
|
Скрипт группирует **только по путям**. Перед split проверьте логическую связность:
|
||||||
|
|
||||||
|
**Объединить в один коммит**, если это одна задача:
|
||||||
|
|
||||||
|
- `docs/openapi.yaml` + `internal/httpapi/` (+ опционально `web/`) — один endpoint/контракт;
|
||||||
|
- `migrations/` + `internal/store/` / `repository/` — одна схема;
|
||||||
|
- правки теста рядом с кодом той же фичи.
|
||||||
|
|
||||||
|
При объединении: один scope (доминирующий пакет, часто `httpapi` или `openapi`), один type, одно тело RU.
|
||||||
|
|
||||||
|
**Auto-split** — если группы **не связаны** (например `web/` + `internal/birdfmt/` без общего смысла): **отдельный коммит на группу**, без вопроса пользователю.
|
||||||
|
|
||||||
|
## Порядок коммитов при split
|
||||||
|
|
||||||
|
1. `openapi`
|
||||||
|
2. `httpapi`, `store`, `jobs`
|
||||||
|
3. `pipeline`, `birdfmt`, `birddeploy`, `bundle`
|
||||||
|
4. `web`
|
||||||
|
5. `ci`, `deploy`, `db`
|
||||||
|
6. `docs`
|
||||||
|
7. `chore`, `cmd`
|
||||||
|
|
||||||
|
## Алгоритм auto-split (уровень файлов)
|
||||||
|
|
||||||
|
1. `git status` и `git diff --cached --stat` — зафиксировать полный список staged-файлов.
|
||||||
|
2. `git reset HEAD` — снять всё из index (working tree не трогать).
|
||||||
|
3. Для каждой (объединённой) группы по порядку выше:
|
||||||
|
- `git add -- <paths…>`
|
||||||
|
- Сгенерировать сообщение по правилу conventional-commits.
|
||||||
|
- Закоммитить (см. ниже).
|
||||||
|
4. После всех коммитов — `git status` для проверки.
|
||||||
|
|
||||||
|
**Ограничение:** частичный stage одного файла с разной семантикой — предупредить; split по hunk'ам не делать; предложить разнести правки по файлам.
|
||||||
|
|
||||||
|
## Создание коммита
|
||||||
|
|
||||||
|
Только если пользователь **явно** просил закоммитить. Иначе — только вывести готовые сообщения.
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
git commit -m "$( @'
|
||||||
|
<type>(<scope>): <summary in English>
|
||||||
|
|
||||||
|
<Тело на русском.>
|
||||||
|
'@ )"
|
||||||
|
```
|
||||||
|
|
||||||
|
В PowerShell для многострочного тела используйте here-string как выше или `-m` для заголовка и `-m` для тела.
|
||||||
|
|
||||||
|
Перед коммитом: `git status`, `git diff --cached` для группы — убедиться, что stage соответствует сообщению.
|
||||||
|
|
||||||
|
## Генерация текста
|
||||||
|
|
||||||
|
По `groups[].diff_excerpt`, `stat`, `files`:
|
||||||
|
|
||||||
|
### Шаг A — semver (до выбора type)
|
||||||
|
|
||||||
|
| Вопрос | Если «да» → |
|
||||||
|
|--------|-------------|
|
||||||
|
| Пользователь получает **новую** возможность? | `feat` (minor) |
|
||||||
|
| Восстанавливается **ожидаемое** поведение / устранён баг? | `fix` (patch) |
|
||||||
|
| Только скорость, контракт тот же? | `perf` (patch) |
|
||||||
|
| Только структура кода/UI, поведение то же? | `refactor` (none) |
|
||||||
|
|
||||||
|
**Follow-up:** правки сразу после `feat` в том же scope без новой возможности → **`fix`**, не `feat` (слова *enhance/improve/refactor* в задаче не делают commit `feat`).
|
||||||
|
|
||||||
|
**Запрещено** по умолчанию ставить `feat` для «большого diff» в `web/` — type по **semver impact**, не по объёму.
|
||||||
|
|
||||||
|
### Шаг B — type, scope, текст
|
||||||
|
|
||||||
|
- **type** — результат шага A, не «chore по умолчанию» и не `feat` из-за слова enhance.
|
||||||
|
- **scope** — из JSON группы или доминирующий при merge.
|
||||||
|
- **summary** — конкретный, английский, императив; для `fix` — что **починено** (`fix broken …`, `prevent … loop`).
|
||||||
|
- **body** — русский: что, зачем, edge cases, breaking impact.
|
||||||
|
|
||||||
|
## Вывод пользователю
|
||||||
|
|
||||||
|
Для **каждого** коммита:
|
||||||
|
|
||||||
|
### 1. Готовое сообщение
|
||||||
|
|
||||||
|
```
|
||||||
|
<type>(<scope>): <summary>
|
||||||
|
|
||||||
|
<тело>
|
||||||
|
```
|
||||||
|
|
||||||
|
### 2. Пояснение (RU)
|
||||||
|
|
||||||
|
- **Semver impact:** `minor` | `patch` | `none` | `major` — и почему не другой type.
|
||||||
|
- Риски и impact.
|
||||||
|
- Split: сколько коммитов и почему.
|
||||||
|
|
||||||
|
## Не смешивать
|
||||||
|
|
||||||
|
Один commit message — одна primary intent. Не объединять несвязанный `fix` и `feat` в один заголовок.
|
||||||
|
|
||||||
|
## Ссылки
|
||||||
|
|
||||||
|
- Правило: `.cursor/rules/conventional-commits.mdc`
|
||||||
|
- Скрипт: `scripts/commit/staged-context.ps1`
|
||||||
|
- Просмотр групп: `powershell -File scripts/commit/staged-context.ps1 | ConvertFrom-Json`
|
||||||
+29
-51
@@ -1,75 +1,53 @@
|
|||||||
# Gitea Actions
|
# Gitea Actions
|
||||||
|
|
||||||
В репозитории включён workflow [workflows/ci.yaml](workflows/ci.yaml).
|
Workflow: [workflows/ci.yaml](workflows/ci.yaml).
|
||||||
|
|
||||||
Job **changes** вычисляет два флага:
|
| Job | PR | push в main |
|
||||||
|
|-----|-----|-------------|
|
||||||
|
| changes, openapi, web, go, bird2 | quality gates | quality gates |
|
||||||
|
| commitlint | да | — |
|
||||||
|
| **release** | — | semantic-release + docker push (один run) |
|
||||||
|
|
||||||
- **`openapi`** — в диффе есть `docs/openapi.yaml` или `redocly.yaml` → запускается **openapi** (Redocly lint).
|
Подробнее: [docs/releasing.md](../docs/releasing.md).
|
||||||
- **`code`** — в диффе есть **хотя бы один файл вне «только документация»** → запускаются **go**, **bird2** и при push в main/master **docker-images**.
|
|
||||||
|
|
||||||
Если изменены **только** файлы из списка ниже, **`code=false`**: Go-тесты, bird2 и сборка образов **не** запускаются; при необходимости всё ещё выполняется **openapi**, если трогали OpenAPI или `redocly.yaml`.
|
## CI (quality gates)
|
||||||
|
|
||||||
**Список путей «только документация» (без полной сборки):** всё под `docs/`, корневой `README.md`, `web/README.md`, `.gitea/README.md`, `redocly.yaml`.
|
Job **changes** вычисляет флаги по путям в diff. Изменение `.gitea/workflows/*` поднимает полный прогон.
|
||||||
|
|
||||||
Любой другой путь (в т.ч. `.gitea/workflows/ci.yaml`, `go.mod`, `web/package.json`, `deploy/**`, `**/*.go`) включает полный пайплайн.
|
На **pull request** — **commitlint** (Conventional Commits).
|
||||||
|
|
||||||
Требования к runner:
|
Runner: `ubuntu-latest`, **bird2** из apt, Docker для job **release**.
|
||||||
|
|
||||||
- Метка `ubuntu-latest` (или измените `runs-on` под свои labels).
|
## Release (job в ci.yaml)
|
||||||
- Job `bird2` ставит пакет **bird2** из репозитория **Ubuntu** на runner (как образ `evobgp-bird2` на `ubuntu:noble`) и вызывает `bird -c … -p` по файлам в checkout (без вложенного Docker: иначе bind-mount `/workspace/...` часто не виден демону Docker на хосте).
|
|
||||||
- Job **docker-images** требует **Docker** и (для push) доступ к Container Registry Gitea с runner’а.
|
|
||||||
|
|
||||||
## Сборка и публикация образов (`docker-images`)
|
После успешных quality gates на **push в main** job **release**:
|
||||||
|
|
||||||
Запускается только при **push** в ветки `main` или `master` (после успешного job `go`). Собираются образы из `deploy/docker/*` (все бинарники через `gobinary`, плюс `evobgp-web`, `evobgp-agent`, `evobgp-bird2`).
|
1. `npx semantic-release` — тег `vX.Y.Z` на **текущий commit** (без дополнительного commit в main).
|
||||||
|
2. Gitea Release + `CHANGELOG.md` как attachment (не в git).
|
||||||
|
3. `docker buildx bake default --push` с `VERSION=X.Y.Z` — в том же job.
|
||||||
|
|
||||||
### Секреты репозитория (для push в реестр)
|
Если releasable-коммитов нет — semantic-release no-op, образы не публикуются.
|
||||||
|
|
||||||
Хост Container Registry в workflow **зашит**: **`git.shts.su`** (`docker/login-action` и теги образов).
|
### Секреты
|
||||||
|
|
||||||
В настройках репозитория Gitea: **Settings → Actions → Secrets** (или аналог):
|
**`ACTIONS_PAT`**: push tags, releases, Container Registry. Fallback: **`gitea.token`**.
|
||||||
|
|
||||||
| Секрет | Назначение |
|
### Теги образов
|
||||||
|--------|------------|
|
|
||||||
| `ACTIONS_PAT` | (Опционально) PAT с правом **записи пакетов**. Если не задан, для `docker login` используется **`gitea.token`** (нужны права у workflow на пакеты). |
|
|
||||||
|
|
||||||
Вход в реестр в workflow:
|
|
||||||
|
|
||||||
- **registry:** `git.shts.su`;
|
|
||||||
- **username:** `${{ gitea.actor }}`;
|
|
||||||
- **password:** `${{ secrets.ACTIONS_PAT || gitea.token }}`.
|
|
||||||
|
|
||||||
### Почему шаг «Log in to Gitea Registry» мог быть skipped
|
|
||||||
|
|
||||||
Раньше в `if:` стояло **`secrets.ACTIONS_PAT != ''`**. В **Gitea act_runner** такая проверка секрета в условии шага часто **всегда ложна**, из‑за этого логин **всегда skipped**. Сейчас условие только `docker_push == 'true'` (всегда при нормальном шаге **Prepare**), пароль: **`secrets.ACTIONS_PAT || gitea.token`**.
|
|
||||||
|
|
||||||
Если после правки `docker login` падает с 401/403 — выдайте job-токену или PAT право **записи в Container Registry** (в настройках Gitea / организации / репозитория, в зависимости от версии).
|
|
||||||
|
|
||||||
Если **`gitea.token`** в выражениях недоступен, в [ci.yaml](workflows/ci.yaml) замените пароль на `${{ secrets.ACTIONS_PAT || github.token }}`.
|
|
||||||
|
|
||||||
### Имена образов и теги
|
|
||||||
|
|
||||||
Формат:
|
|
||||||
|
|
||||||
```text
|
```text
|
||||||
git.shts.su/<owner_lowercase>/<имя>:<тег>
|
git.shts.su/<owner>/<имя>:latest
|
||||||
|
git.shts.su/<owner>/<имя>:v1.2.3
|
||||||
|
git.shts.su/<owner>/<имя>:1.2.3
|
||||||
|
git.shts.su/<owner>/<имя>:<short-sha>
|
||||||
|
git.shts.su/<owner>/<имя>:sha-<full-sha>
|
||||||
```
|
```
|
||||||
|
|
||||||
где `<имя>` — одно из: `evobgp-api`, `evobgp-all`, `evobgp-scheduler`, `evobgp-ingest`, `evobgp-render`, `evobgp-deploy`, `evobgp-node`, `evobgp-web`, `evobgp-web-all` (UI с прокси на `evobgp-all` для профиля Compose `microvps-full`), `evobgp-agent`, `evobgp-bird2`.
|
Имена образов: `evobgp-api`, `evobgp-all`, `evobgp-scheduler`, `evobgp-ingest`, `evobgp-render`, `evobgp-deploy`, `evobgp-node`, `evobgp-web`, `evobgp-web-all`, `evobgp-agent`, `evobgp-bird2`.
|
||||||
|
|
||||||
Теги: `latest`, короткий SHA (7 символов), `sha-<полный_commit_sha>`.
|
Пример:
|
||||||
|
|
||||||
Пример pull:
|
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
docker pull git.shts.su/myuser/evobgp-api:latest
|
docker pull git.shts.su/myuser/evobgp-api:1.2.3
|
||||||
```
|
```
|
||||||
|
|
||||||
Полная таблица образов, тегов, ссылок на страницы пакетов в Gitea и минимальный `docker run` для сервера без сборки — в [docs/quickstart.md](../docs/quickstart.md) (раздел «Готовые образы без сборки»).
|
См. [deploy/docker/README.md](../deploy/docker/README.md), [docs/quickstart.md](../docs/quickstart.md).
|
||||||
|
|
||||||
Убедитесь, что в **Gitea** включены пакеты (Container Registry) и у токена есть права на создание/обновление образов. Другой хост реестра — правьте **`registry:`** и префикс **`IMG=`** в [ci.yaml](workflows/ci.yaml).
|
|
||||||
|
|
||||||
### Примечания
|
|
||||||
|
|
||||||
- Сборка с платформой `linux/amd64`. При необходимости добавьте `linux/arm64` и multi-arch в `docker buildx build`.
|
|
||||||
- Для PR job **docker-images** не выполняется (только push в main/master), чтобы не публиковать образы из чужих форков без явной политики.
|
|
||||||
+100
-194
@@ -83,6 +83,7 @@ jobs:
|
|||||||
deploy/bird/*) bird_conf=true ;;
|
deploy/bird/*) bird_conf=true ;;
|
||||||
deploy/docker/bird/*) docker_bird=true; docker_go=true ;;
|
deploy/docker/bird/*) docker_bird=true; docker_go=true ;;
|
||||||
deploy/docker/gobinary/*) docker_go=true ;;
|
deploy/docker/gobinary/*) docker_go=true ;;
|
||||||
|
deploy/docker/docker-bake.hcl) docker_go=true; docker_web=true ;;
|
||||||
deploy/docker/evobgp-agent/*) docker_go=true ;;
|
deploy/docker/evobgp-agent/*) docker_go=true ;;
|
||||||
deploy/docker/evobgp-web/*) docker_web=true ;;
|
deploy/docker/evobgp-web/*) docker_web=true ;;
|
||||||
deploy/docker/bird2/*) docker_bird=true ;;
|
deploy/docker/bird2/*) docker_bird=true ;;
|
||||||
@@ -120,6 +121,26 @@ jobs:
|
|||||||
- name: Lint OpenAPI (Redocly)
|
- name: Lint OpenAPI (Redocly)
|
||||||
run: npx --yes @redocly/cli@1 lint docs/openapi.yaml
|
run: npx --yes @redocly/cli@1 lint docs/openapi.yaml
|
||||||
|
|
||||||
|
# ---------------------------------------------------------------------------
|
||||||
|
web:
|
||||||
|
needs: [changes]
|
||||||
|
if: needs.changes.outputs.web == 'true'
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: "20"
|
||||||
|
cache: npm
|
||||||
|
cache-dependency-path: web/package-lock.json
|
||||||
|
- name: npm ci, check, lint
|
||||||
|
run: |
|
||||||
|
set -euxo pipefail
|
||||||
|
cd web
|
||||||
|
npm ci
|
||||||
|
npm run check
|
||||||
|
npm run lint
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
go:
|
go:
|
||||||
needs: [changes]
|
needs: [changes]
|
||||||
@@ -129,9 +150,13 @@ jobs:
|
|||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
- uses: actions/setup-go@v5
|
- uses: actions/setup-go@v5
|
||||||
with:
|
with:
|
||||||
go-version: "1.22"
|
go-version: "1.24"
|
||||||
|
cache: true
|
||||||
|
cache-dependency-path: go.sum
|
||||||
- name: Vet
|
- name: Vet
|
||||||
run: go vet ./...
|
run: go vet ./...
|
||||||
|
- name: Lint httpapi (ERR-01 / ARCH-01)
|
||||||
|
run: sh scripts/lint-httpapi.sh
|
||||||
- name: Test
|
- name: Test
|
||||||
run: go test ./... -race -count=1
|
run: go test ./... -race -count=1
|
||||||
- name: Build all commands
|
- name: Build all commands
|
||||||
@@ -183,224 +208,105 @@ jobs:
|
|||||||
done
|
done
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# Docker: Go-бинарники (api, all, scheduler, ingest, render, deploy, node, agent).
|
commitlint:
|
||||||
# Запускается если изменился Go-код или Go-Dockerfile.
|
if: github.event_name == 'pull_request'
|
||||||
# Если Go-код менялся — требуем успех go-тестов; если только Dockerfile — go skipped, ОК.
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: "22"
|
||||||
|
cache: npm
|
||||||
|
cache-dependency-path: package-lock.json
|
||||||
|
- name: Lint commit messages
|
||||||
|
run: |
|
||||||
|
set -euxo pipefail
|
||||||
|
npm ci
|
||||||
|
npx commitlint --from "${{ github.event.pull_request.base.sha }}" --to "${{ github.event.pull_request.head.sha }}"
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
docker-go:
|
# Один push в main: semantic-release (тег на текущий commit, без доп. commit) + docker push.
|
||||||
needs: [changes, go]
|
# ---------------------------------------------------------------------------
|
||||||
|
release:
|
||||||
|
needs: [changes, openapi, web, go, bird2]
|
||||||
if: >-
|
if: >-
|
||||||
always() &&
|
always() &&
|
||||||
|
github.event_name == 'push' &&
|
||||||
|
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') &&
|
||||||
needs.changes.result == 'success' &&
|
needs.changes.result == 'success' &&
|
||||||
needs.go.result != 'failure' &&
|
(needs.openapi.result == 'success' || needs.openapi.result == 'skipped') &&
|
||||||
github.event_name == 'push' &&
|
(needs.web.result == 'success' || needs.web.result == 'skipped') &&
|
||||||
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') &&
|
(needs.go.result == 'success' || needs.go.result == 'skipped') &&
|
||||||
(needs.changes.outputs.go == 'true' || needs.changes.outputs.docker_go == 'true')
|
(needs.bird2.result == 'success' || needs.bird2.result == 'skipped')
|
||||||
runs-on: ubuntu-latest
|
runs-on: ubuntu-latest
|
||||||
strategy:
|
|
||||||
fail-fast: false
|
|
||||||
matrix:
|
|
||||||
include:
|
|
||||||
- image: evobgp-api
|
|
||||||
dockerfile: deploy/docker/gobinary/Dockerfile
|
|
||||||
bin: evobgp-api
|
|
||||||
birdc: "1"
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-all
|
|
||||||
dockerfile: deploy/docker/gobinary/Dockerfile
|
|
||||||
bin: evobgp-all
|
|
||||||
birdc: "1"
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-scheduler
|
|
||||||
dockerfile: deploy/docker/gobinary/Dockerfile
|
|
||||||
bin: evobgp-scheduler
|
|
||||||
birdc: "0"
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-ingest
|
|
||||||
dockerfile: deploy/docker/gobinary/Dockerfile
|
|
||||||
bin: evobgp-ingest
|
|
||||||
birdc: "0"
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-render
|
|
||||||
dockerfile: deploy/docker/gobinary/Dockerfile
|
|
||||||
bin: evobgp-render
|
|
||||||
birdc: "0"
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-deploy
|
|
||||||
dockerfile: deploy/docker/gobinary/Dockerfile
|
|
||||||
bin: evobgp-deploy
|
|
||||||
birdc: "0"
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-node
|
|
||||||
dockerfile: deploy/docker/gobinary/Dockerfile
|
|
||||||
bin: evobgp-node
|
|
||||||
birdc: "0"
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-agent
|
|
||||||
dockerfile: deploy/docker/evobgp-agent/Dockerfile
|
|
||||||
evobgp_upstream: ""
|
|
||||||
steps:
|
steps:
|
||||||
- uses: actions/checkout@v4
|
- uses: actions/checkout@v4
|
||||||
|
with:
|
||||||
|
fetch-depth: 0
|
||||||
|
token: ${{ secrets.ACTIONS_PAT || gitea.token }}
|
||||||
|
persist-credentials: true
|
||||||
|
- uses: actions/setup-node@v4
|
||||||
|
with:
|
||||||
|
node-version: "22"
|
||||||
|
cache: npm
|
||||||
|
cache-dependency-path: package-lock.json
|
||||||
|
- name: Install release tooling
|
||||||
|
run: npm ci
|
||||||
|
- name: Semantic release
|
||||||
|
run: npx semantic-release
|
||||||
|
env:
|
||||||
|
GITEA_URL: https://git.shts.su
|
||||||
|
GITEA_TOKEN: ${{ secrets.ACTIONS_PAT || gitea.token }}
|
||||||
|
- name: Detect new release
|
||||||
|
id: rel
|
||||||
|
run: |
|
||||||
|
set -euo pipefail
|
||||||
|
if [ -f .release-version ]; then
|
||||||
|
echo "version=$(tr -d '[:space:]' < .release-version)" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "released=true" >> "$GITHUB_OUTPUT"
|
||||||
|
else
|
||||||
|
echo "released=false" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "No releasable commits — skipping image publish"
|
||||||
|
fi
|
||||||
- name: Set up Docker Buildx
|
- name: Set up Docker Buildx
|
||||||
|
if: steps.rel.outputs.released == 'true'
|
||||||
uses: docker/setup-buildx-action@v3
|
uses: docker/setup-buildx-action@v3
|
||||||
- name: Prepare image metadata
|
- name: Prepare image metadata
|
||||||
|
if: steps.rel.outputs.released == 'true'
|
||||||
id: meta
|
id: meta
|
||||||
run: |
|
run: |
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
echo "version=${{ steps.rel.outputs.version }}" >> "$GITHUB_OUTPUT"
|
||||||
owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
|
owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
|
||||||
echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT"
|
echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT"
|
||||||
short_sha="$(echo '${{ github.sha }}' | cut -c1-7)"
|
short_sha="$(echo '${{ github.sha }}' | cut -c1-7)"
|
||||||
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
|
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
|
||||||
|
echo "build_time=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT"
|
||||||
- name: Log in to Gitea Registry
|
- name: Log in to Gitea Registry
|
||||||
|
if: steps.rel.outputs.released == 'true'
|
||||||
uses: docker/login-action@v3
|
uses: docker/login-action@v3
|
||||||
with:
|
with:
|
||||||
registry: git.shts.su
|
registry: git.shts.su
|
||||||
username: ${{ gitea.actor }}
|
username: ${{ gitea.actor }}
|
||||||
password: ${{ secrets.ACTIONS_PAT || gitea.token }}
|
password: ${{ secrets.ACTIONS_PAT || gitea.token }}
|
||||||
- name: Build and push ${{ matrix.image }}
|
- name: Build and push images (bake)
|
||||||
|
if: steps.rel.outputs.released == 'true'
|
||||||
env:
|
env:
|
||||||
OWNER_LC: ${{ steps.meta.outputs.owner_lc }}
|
REGISTRY: git.shts.su/${{ steps.meta.outputs.owner_lc }}
|
||||||
|
IMAGE_TAG: latest
|
||||||
|
VERSION: ${{ steps.meta.outputs.version }}
|
||||||
SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
|
SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
|
||||||
DF: ${{ matrix.dockerfile }}
|
SHA_FULL: ${{ github.sha }}
|
||||||
IMAGE: ${{ matrix.image }}
|
BUILD_TIME: ${{ steps.meta.outputs.build_time }}
|
||||||
BIN: ${{ matrix.bin }}
|
CACHE_REF_GO: git.shts.su/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:go-buildcache
|
||||||
BIRDC: ${{ matrix.birdc }}
|
CACHE_REF_WEB: git.shts.su/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:web-buildcache
|
||||||
EVOBGP_UPSTREAM: ${{ matrix.evobgp_upstream }}
|
BUILDX_BAKE_ENTITLEMENTS_FS: "0"
|
||||||
run: |
|
run: |
|
||||||
set -euxo pipefail
|
set -euxo pipefail
|
||||||
WS="${{ github.workspace }}"
|
cd "${{ github.workspace }}/deploy/docker"
|
||||||
cd "$WS"
|
sh write-bake-override.sh
|
||||||
|
docker buildx bake --allow=fs.read="${{ github.workspace }}" \
|
||||||
|
-f docker-bake.hcl -f docker-bake.override.hcl default --push
|
||||||
|
|
||||||
IMG="git.shts.su/${OWNER_LC}/${IMAGE}"
|
|
||||||
BUILD_ARGS=()
|
|
||||||
if [ -n "${BIN:-}" ]; then
|
|
||||||
BUILD_ARGS+=(--build-arg "BIN=${BIN}")
|
|
||||||
BUILD_ARGS+=(--build-arg "INSTALL_BIRDC=${BIRDC:-0}")
|
|
||||||
fi
|
|
||||||
if [ -n "${EVOBGP_UPSTREAM:-}" ]; then
|
|
||||||
BUILD_ARGS+=(--build-arg "EVOBGP_UPSTREAM=${EVOBGP_UPSTREAM}")
|
|
||||||
fi
|
|
||||||
|
|
||||||
docker buildx build \
|
|
||||||
--platform linux/amd64 \
|
|
||||||
--file "$DF" \
|
|
||||||
"${BUILD_ARGS[@]}" \
|
|
||||||
--tag "${IMG}:latest" \
|
|
||||||
--tag "${IMG}:${SHORT_SHA}" \
|
|
||||||
--tag "${IMG}:sha-${{ github.sha }}" \
|
|
||||||
--push \
|
|
||||||
"$WS"
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Docker: Web-фронтенд (evobgp-web, evobgp-web-all).
|
|
||||||
# Не зависит от Go-тестов — SvelteKit собирается отдельно.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
docker-web:
|
|
||||||
needs: [changes]
|
|
||||||
if: >-
|
|
||||||
github.event_name == 'push' &&
|
|
||||||
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') &&
|
|
||||||
(needs.changes.outputs.web == 'true' || needs.changes.outputs.docker_web == 'true')
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
strategy:
|
|
||||||
fail-fast: false
|
|
||||||
matrix:
|
|
||||||
include:
|
|
||||||
- image: evobgp-web
|
|
||||||
dockerfile: deploy/docker/evobgp-web/Dockerfile
|
|
||||||
evobgp_upstream: ""
|
|
||||||
- image: evobgp-web-all
|
|
||||||
dockerfile: deploy/docker/evobgp-web/Dockerfile
|
|
||||||
evobgp_upstream: evobgp-all
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
- name: Set up Docker Buildx
|
|
||||||
uses: docker/setup-buildx-action@v3
|
|
||||||
- name: Prepare image metadata
|
|
||||||
id: meta
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
|
|
||||||
echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT"
|
|
||||||
short_sha="$(echo '${{ github.sha }}' | cut -c1-7)"
|
|
||||||
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
|
|
||||||
- name: Log in to Gitea Registry
|
|
||||||
uses: docker/login-action@v3
|
|
||||||
with:
|
|
||||||
registry: git.shts.su
|
|
||||||
username: ${{ gitea.actor }}
|
|
||||||
password: ${{ secrets.ACTIONS_PAT || gitea.token }}
|
|
||||||
- name: Build and push ${{ matrix.image }}
|
|
||||||
env:
|
|
||||||
OWNER_LC: ${{ steps.meta.outputs.owner_lc }}
|
|
||||||
SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
|
|
||||||
DF: ${{ matrix.dockerfile }}
|
|
||||||
IMAGE: ${{ matrix.image }}
|
|
||||||
EVOBGP_UPSTREAM: ${{ matrix.evobgp_upstream }}
|
|
||||||
run: |
|
|
||||||
set -euxo pipefail
|
|
||||||
WS="${{ github.workspace }}"
|
|
||||||
cd "$WS"
|
|
||||||
|
|
||||||
IMG="git.shts.su/${OWNER_LC}/${IMAGE}"
|
|
||||||
BUILD_ARGS=()
|
|
||||||
if [ -n "${EVOBGP_UPSTREAM:-}" ]; then
|
|
||||||
BUILD_ARGS+=(--build-arg "EVOBGP_UPSTREAM=${EVOBGP_UPSTREAM}")
|
|
||||||
fi
|
|
||||||
|
|
||||||
docker buildx build \
|
|
||||||
--platform linux/amd64 \
|
|
||||||
--file "$DF" \
|
|
||||||
"${BUILD_ARGS[@]}" \
|
|
||||||
--tag "${IMG}:latest" \
|
|
||||||
--tag "${IMG}:${SHORT_SHA}" \
|
|
||||||
--tag "${IMG}:sha-${{ github.sha }}" \
|
|
||||||
--push \
|
|
||||||
"$WS"
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
# Docker: BIRD2 (evobgp-bird2).
|
|
||||||
# Собирается только при изменении deploy/bird/ или Dockerfile bird2.
|
|
||||||
# ---------------------------------------------------------------------------
|
|
||||||
docker-bird:
|
|
||||||
needs: [changes]
|
|
||||||
if: >-
|
|
||||||
github.event_name == 'push' &&
|
|
||||||
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') &&
|
|
||||||
(needs.changes.outputs.bird_conf == 'true' || needs.changes.outputs.docker_bird == 'true')
|
|
||||||
runs-on: ubuntu-latest
|
|
||||||
steps:
|
|
||||||
- uses: actions/checkout@v4
|
|
||||||
- name: Set up Docker Buildx
|
|
||||||
uses: docker/setup-buildx-action@v3
|
|
||||||
- name: Prepare image metadata
|
|
||||||
id: meta
|
|
||||||
run: |
|
|
||||||
set -euo pipefail
|
|
||||||
owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
|
|
||||||
echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT"
|
|
||||||
short_sha="$(echo '${{ github.sha }}' | cut -c1-7)"
|
|
||||||
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
|
|
||||||
- name: Log in to Gitea Registry
|
|
||||||
uses: docker/login-action@v3
|
|
||||||
with:
|
|
||||||
registry: git.shts.su
|
|
||||||
username: ${{ gitea.actor }}
|
|
||||||
password: ${{ secrets.ACTIONS_PAT || gitea.token }}
|
|
||||||
- name: Build and push evobgp-bird2
|
|
||||||
env:
|
|
||||||
OWNER_LC: ${{ steps.meta.outputs.owner_lc }}
|
|
||||||
SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
|
|
||||||
run: |
|
|
||||||
set -euxo pipefail
|
|
||||||
WS="${{ github.workspace }}"
|
|
||||||
IMG="git.shts.su/${OWNER_LC}/evobgp-bird2"
|
|
||||||
|
|
||||||
docker buildx build \
|
|
||||||
--platform linux/amd64 \
|
|
||||||
--file deploy/docker/bird2/Dockerfile \
|
|
||||||
--tag "${IMG}:latest" \
|
|
||||||
--tag "${IMG}:${SHORT_SHA}" \
|
|
||||||
--tag "${IMG}:sha-${{ github.sha }}" \
|
|
||||||
--push \
|
|
||||||
"$WS"
|
|
||||||
+18
@@ -0,0 +1,18 @@
|
|||||||
|
# Root npm (semantic-release, commitlint) — npm ci in CI, never commit deps
|
||||||
|
node_modules/
|
||||||
|
|
||||||
|
# Generated by deploy/docker/write-bake-override.sh (CI/local bake)
|
||||||
|
deploy/docker/docker-bake.override.hcl
|
||||||
|
|
||||||
|
# CI-only markers (not committed)
|
||||||
|
.release-version
|
||||||
|
CHANGELOG.md
|
||||||
|
|
||||||
|
# OS
|
||||||
|
.DS_Store
|
||||||
|
Thumbs.db
|
||||||
|
|
||||||
|
# Env / secrets (local only)
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
!.env.example
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
run:
|
||||||
|
timeout: 5m
|
||||||
|
|
||||||
|
linters:
|
||||||
|
enable:
|
||||||
|
- gofmt
|
||||||
|
- govet
|
||||||
|
- errcheck
|
||||||
|
|
||||||
|
issues:
|
||||||
|
exclude-use-default: false
|
||||||
|
max-issues-per-linter: 50
|
||||||
|
max-same-issues: 20
|
||||||
|
|
||||||
|
linters-settings:
|
||||||
|
gofmt:
|
||||||
|
simplify: true
|
||||||
@@ -0,0 +1,48 @@
|
|||||||
|
{
|
||||||
|
"branches": ["main", "master"],
|
||||||
|
"tagFormat": "v${version}",
|
||||||
|
"plugins": [
|
||||||
|
[
|
||||||
|
"@semantic-release/commit-analyzer",
|
||||||
|
{
|
||||||
|
"preset": "conventionalcommits",
|
||||||
|
"releaseRules": [
|
||||||
|
{ "type": "feat", "release": "minor" },
|
||||||
|
{ "type": "fix", "release": "patch" },
|
||||||
|
{ "type": "perf", "release": "patch" },
|
||||||
|
{ "breaking": true, "release": "major" }
|
||||||
|
]
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"@semantic-release/release-notes-generator",
|
||||||
|
{
|
||||||
|
"preset": "conventionalcommits"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"@semantic-release/changelog",
|
||||||
|
{
|
||||||
|
"changelogFile": "CHANGELOG.md"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"@semantic-release/exec",
|
||||||
|
{
|
||||||
|
"successCmd": "echo ${nextRelease.version} > .release-version"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
[
|
||||||
|
"@markwylde/semantic-release-gitea",
|
||||||
|
{
|
||||||
|
"giteaUrl": "https://git.shts.su",
|
||||||
|
"assets": [
|
||||||
|
{
|
||||||
|
"path": "CHANGELOG.md",
|
||||||
|
"label": "Changelog"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
]
|
||||||
|
]
|
||||||
|
}
|
||||||
@@ -0,0 +1,61 @@
|
|||||||
|
# Руководство для ИИ-агентов (экономия контекста)
|
||||||
|
|
||||||
|
Краткие ориентиры по репозиторию **EvoBGP**, чтобы не тратить токены на полное сканирование дерева и повторное чтение одних и тех же файлов.
|
||||||
|
|
||||||
|
## С чего начать (минимум чтения)
|
||||||
|
|
||||||
|
0. **Инженерные правила** — при изменении кода следовать [.cursor/rules/engineering.mdc](.cursor/rules/engineering.mdc); для `web/` — [.cursor/rules/web-shadcn.mdc](.cursor/rules/web-shadcn.mdc); для `birdfmt` / `pipeline` / BIRD — [.cursor/rules/networking-bird.mdc](.cursor/rules/networking-bird.mdc).
|
||||||
|
1. **[docs/README.md](docs/README.md)** — оглавление и роли читателя.
|
||||||
|
2. **[docs/architecture.md](docs/architecture.md)** — компоненты `cmd/`, карта `internal/`, потоки данных (одного этого файла обычно достаточно для ориентации).
|
||||||
|
3. Задача-специфично: [docs/api.md](docs/api.md), [docs/access.md](docs/access.md), [web/README.md](web/README.md) — только если меняете API, доступ или фронт.
|
||||||
|
|
||||||
|
Источник правды по HTTP-контракту: **[docs/openapi.yaml](docs/openapi.yaml)**. Не дублируйте длинные фрагменты спецификации в ответах — ссылайтесь на путь и тег/операцию.
|
||||||
|
|
||||||
|
## Карта кода (куда смотреть)
|
||||||
|
|
||||||
|
| Область | Где искать |
|
||||||
|
|---------|------------|
|
||||||
|
| REST, auth, CORS | `internal/httpapi/` |
|
||||||
|
| Бизнес-слой и абстракция хранилища | `internal/store/` |
|
||||||
|
| PostgreSQL | `internal/repository/`, `internal/db/`, `migrations/` |
|
||||||
|
| Фоновые задачи | `internal/jobs/` |
|
||||||
|
| Цепочка refresh модуля (ingest+render, BIRD preview) | `internal/pipeline/` |
|
||||||
|
| Конфиг BIRD, `birdc` | `internal/birdfmt/`, `internal/birddeploy/` |
|
||||||
|
| Бандлы и подписи | `internal/bundle/`, `internal/signing/` |
|
||||||
|
| Точки входа процессов | `cmd/*/` |
|
||||||
|
| Веб (SvelteKit) | `web/` |
|
||||||
|
| Compose, деплой | `deploy/compose/` |
|
||||||
|
|
||||||
|
Точки входа бинарников и их роли — в таблице в начале [docs/architecture.md](docs/architecture.md).
|
||||||
|
|
||||||
|
## Как не раздувать контекст
|
||||||
|
|
||||||
|
- **Сначала узкий поиск:** `grep`/поиск по символу или короткий семантический запрос по одной папке (`internal/httpapi/`, `internal/pipeline/`, …), а не чтение всех `.go` подряд.
|
||||||
|
- **Читайте файлы целиком только при необходимости:** большие файлы — с `offset`/`limit` или по найденным строкам.
|
||||||
|
- **Не подтягивайте в контекст:** `web/node_modules/`, сгенерированные артефакты сборки, бинарники, полный `openapi.html`, если достаточно `openapi.yaml`.
|
||||||
|
- **Повторное использование:** если [docs/architecture.md](docs/architecture.md) уже описывает поток — не пересказывайте его длинно; укажите документ и конкретный подпункт задачи.
|
||||||
|
- **Длинные планы:** `.cursor/plans/*.plan.md` — для истории решений; для навигации пользователю достаточно `docs/`; не читайте план целиком без причины.
|
||||||
|
|
||||||
|
## Коммиты (Conventional Commits)
|
||||||
|
|
||||||
|
Если пользователь просит **коммит**, **commit message**, **закоммить**, **git commit**, **`/commit-message`** или это следует из плана — **сразу**:
|
||||||
|
|
||||||
|
1. Shell: `powershell -NoProfile -File scripts/commit/staged-context.ps1` (первый вызов, до текста коммита).
|
||||||
|
2. Скилл [.cursor/skills/commit-message/SKILL.md](.cursor/skills/commit-message/SKILL.md) и правило [.cursor/rules/conventional-commits.mdc](.cursor/rules/conventional-commits.mdc).
|
||||||
|
|
||||||
|
Без вывода скрипта (exit 0) **не** придумывать сообщение коммита. Заголовок — EN, тело — RU; несвязанные области — auto-split (скилл).
|
||||||
|
|
||||||
|
**Кнопка ✨ Generate commit message в Source Control** skill/rule **не** использует. Для сообщений по правилам EvoBGP — Agent Chat → **`/commit-message`** (см. [.cursor/commands/commit-message.md](.cursor/commands/commit-message.md)).
|
||||||
|
|
||||||
|
## Команды и среда
|
||||||
|
|
||||||
|
- Консоль пользователя: **PowerShell**; пути в стиле `deploy\compose`.
|
||||||
|
- Быстрый старт и переменные: [docs/quickstart.md](docs/quickstart.md), [README.md](README.md).
|
||||||
|
|
||||||
|
## Язык документации проекта
|
||||||
|
|
||||||
|
Пользовательская документация в `docs/` — преимущественно на русском. Комментарии и имена в коде — в существующем стиле репозитория.
|
||||||
|
|
||||||
|
## Svelte / фронтенд
|
||||||
|
|
||||||
|
При правках `web/**/*.svelte` или Svelte-модулей следуйте навыкам/инструментам проекта (официальный Svelte MCP и скиллы Cursor, если подключены).
|
||||||
+150
@@ -0,0 +1,150 @@
|
|||||||
|
## 1.0.0 (2026-05-19)
|
||||||
|
|
||||||
|
* feat: add @tanstack/table-core dependency and integrate module name mapping in operations page. Upda ([b140dcb](http://192.168.100.56:3000/denozord/EvoBGP/commit/b140dcb))
|
||||||
|
* feat: add @types/node and undici-types to devDependencies in package.json and package-lock.json for ([d92a06c](http://192.168.100.56:3000/denozord/EvoBGP/commit/d92a06c))
|
||||||
|
* feat: add aggregated router-lists catalog endpoint and update module listing filters ([c811c43](http://192.168.100.56:3000/denozord/EvoBGP/commit/c811c43))
|
||||||
|
* feat: add auto-updater service for Docker stack with configurable settings ([ae9fbe3](http://192.168.100.56:3000/denozord/EvoBGP/commit/ae9fbe3))
|
||||||
|
* feat: add bird configuration preview functionality with tabbed interface for fragments and prefixes. ([1969a00](http://192.168.100.56:3000/denozord/EvoBGP/commit/1969a00))
|
||||||
|
* feat: add BIRD status endpoint and integrate birdc checks into job processing. Enhance UI to display ([94a4c6a](http://192.168.100.56:3000/denozord/EvoBGP/commit/94a4c6a))
|
||||||
|
* feat: add CSV import and export functionality for module entries. Implement endpoints for exporting ([bae2d68](http://192.168.100.56:3000/denozord/EvoBGP/commit/bae2d68))
|
||||||
|
* feat: add debug logging and optimize database queries in Postgres repository ([6d0e214](http://192.168.100.56:3000/denozord/EvoBGP/commit/6d0e214))
|
||||||
|
* feat: add diagnostic log endpoint for revisions and implement download functionality ([2260ecf](http://192.168.100.56:3000/denozord/EvoBGP/commit/2260ecf))
|
||||||
|
* feat: add enabled state to BGP peer types and UI. Implement peer enabling toggle in the network page ([80965ad](http://192.168.100.56:3000/denozord/EvoBGP/commit/80965ad))
|
||||||
|
* feat: add last refreshed timestamp to modules and CDN sources ([1e13ef9](http://192.168.100.56:3000/denozord/EvoBGP/commit/1e13ef9))
|
||||||
|
* feat: add live BGP protocol state tracking to peer listing ([8a50ba4](http://192.168.100.56:3000/denozord/EvoBGP/commit/8a50ba4))
|
||||||
|
* feat: add optional name field to BGP peer creation and update UI ([1a61ae2](http://192.168.100.56:3000/denozord/EvoBGP/commit/1a61ae2))
|
||||||
|
* feat: add optional name field to DoH profile schema and types ([6bee5ad](http://192.168.100.56:3000/denozord/EvoBGP/commit/6bee5ad))
|
||||||
|
* feat: add smart aggregation for IPv4 CIDR prefixes in refresh pipeline. Implement functionality to a ([0428b83](http://192.168.100.56:3000/denozord/EvoBGP/commit/0428b83))
|
||||||
|
* feat: add strict bind configuration to BGP templates. Enhance BGP template rendering by including 's ([07f2ce5](http://192.168.100.56:3000/denozord/EvoBGP/commit/07f2ce5))
|
||||||
|
* feat: add tenant refresh functionality to HTTP API and job processing ([7a2b015](http://192.168.100.56:3000/denozord/EvoBGP/commit/7a2b015))
|
||||||
|
* feat: auto-refresh non-IP module updates and add full project manual ([1e77e1a](http://192.168.100.56:3000/denozord/EvoBGP/commit/1e77e1a))
|
||||||
|
* feat: enhance API client with pagination and new data fetching functions. Implement `apiPageAll` for ([9f0b128](http://192.168.100.56:3000/denozord/EvoBGP/commit/9f0b128))
|
||||||
|
* feat: enhance BGP template and peer configuration handling. Introduce policies_json parameter for lo ([be8e7b5](http://192.168.100.56:3000/denozord/EvoBGP/commit/be8e7b5))
|
||||||
|
* feat: enhance CDN source management with last refreshed timestamp ([4776434](http://192.168.100.56:3000/denozord/EvoBGP/commit/4776434))
|
||||||
|
* feat: enhance community attribute handling in BIRD format. Add validation for community route bodies ([b541794](http://192.168.100.56:3000/denozord/EvoBGP/commit/b541794))
|
||||||
|
* feat: enhance community label handling and improve UI localization across operations pages. Introduc ([53fd527](http://192.168.100.56:3000/denozord/EvoBGP/commit/53fd527))
|
||||||
|
* feat: enhance deployment configuration for secure Web UI with Traefik integration ([4acd685](http://192.168.100.56:3000/denozord/EvoBGP/commit/4acd685))
|
||||||
|
* feat: enhance DoH profile management and resolver policy in modules ([34ecc5c](http://192.168.100.56:3000/denozord/EvoBGP/commit/34ecc5c))
|
||||||
|
* feat: enhance EvoBGP with new command-line options for the evobgp-agent, including a watch command f ([6a55f72](http://192.168.100.56:3000/denozord/EvoBGP/commit/6a55f72))
|
||||||
|
* feat: enhance evobgp with new command-line tools for bundle management, including pull, verify, and ([bf52b21](http://192.168.100.56:3000/denozord/EvoBGP/commit/bf52b21))
|
||||||
|
* feat: enhance job detail handling in UI with caching and loading states. Introduce SvelteMap for job ([05bc5c4](http://192.168.100.56:3000/denozord/EvoBGP/commit/05bc5c4))
|
||||||
|
* feat: enhance job detail UI with improved overflow handling and layout adjustments. Update card comp ([67ce135](http://192.168.100.56:3000/denozord/EvoBGP/commit/67ce135))
|
||||||
|
* feat: enhance job filtering and display functionality in operations page. Introduce job search and f ([e1c4cbc](http://192.168.100.56:3000/denozord/EvoBGP/commit/e1c4cbc))
|
||||||
|
* feat: enhance job processing with detailed logging and UI improvements. Introduce revision log entri ([c48cc4e](http://192.168.100.56:3000/denozord/EvoBGP/commit/c48cc4e))
|
||||||
|
* feat: enhance monitoring page with detailed status tracking and error handling ([458a885](http://192.168.100.56:3000/denozord/EvoBGP/commit/458a885))
|
||||||
|
* feat: enhance observability with prefix aggregation metrics ([07b97bd](http://192.168.100.56:3000/denozord/EvoBGP/commit/07b97bd))
|
||||||
|
* feat: enhance peer neighbor handling by introducing NormalizePeerNeighborString function. Update Cre ([80f3a8d](http://192.168.100.56:3000/denozord/EvoBGP/commit/80f3a8d))
|
||||||
|
* feat: enhance RefreshModule to aggregate prefixes from all enabled tenant modules. Update documentat ([431c6a9](http://192.168.100.56:3000/denozord/EvoBGP/commit/431c6a9))
|
||||||
|
* feat: enhance RefreshModule to support legacy ASN resolution and improve prefix handling. Introduce ([e1aaa18](http://192.168.100.56:3000/denozord/EvoBGP/commit/e1aaa18))
|
||||||
|
* feat: enhance settings page layout and section management ([d91667f](http://192.168.100.56:3000/denozord/EvoBGP/commit/d91667f))
|
||||||
|
* feat: enhance UI components with improved styling and layout adjustments. Update button sizes, text ([6a72842](http://192.168.100.56:3000/denozord/EvoBGP/commit/6a72842))
|
||||||
|
* feat: enhance worker functionality by adding job queuing for deploy_apply. Introduce a Registry fiel ([42d956d](http://192.168.100.56:3000/denozord/EvoBGP/commit/42d956d))
|
||||||
|
* feat: implement AS holder name resolution and enhance ASEntry structure. Add ASHolderName function t ([c958d5a](http://192.168.100.56:3000/denozord/EvoBGP/commit/c958d5a))
|
||||||
|
* feat: implement bulk selection and deletion for modules and entries. Add functionality for selecting ([e816d17](http://192.168.100.56:3000/denozord/EvoBGP/commit/e816d17))
|
||||||
|
* feat: implement CDN source preview functionality and enhance data handling. Add a new endpoint for p ([e20c9f3](http://192.168.100.56:3000/denozord/EvoBGP/commit/e20c9f3))
|
||||||
|
* feat: implement countAll function for API data retrieval ([40c8a8d](http://192.168.100.56:3000/denozord/EvoBGP/commit/40c8a8d))
|
||||||
|
* feat: implement domain resolution via DoH in refresh pipeline. Enhance the collectModulePrefixRows f ([678bd27](http://192.168.100.56:3000/denozord/EvoBGP/commit/678bd27))
|
||||||
|
* feat: implement effective peer enabling logic during peer creation. Update CreatePeer methods in Pos ([3e85431](http://192.168.100.56:3000/denozord/EvoBGP/commit/3e85431))
|
||||||
|
* feat: implement module refresh job management with concurrency control. Add CountOtherActiveModuleRe ([149fb0a](http://192.168.100.56:3000/denozord/EvoBGP/commit/149fb0a))
|
||||||
|
* feat: implement peer reconciliation job for BGP peers ([3b17228](http://192.168.100.56:3000/denozord/EvoBGP/commit/3b17228))
|
||||||
|
* feat: implement revision retention settings and pruning logic ([4f50350](http://192.168.100.56:3000/denozord/EvoBGP/commit/4f50350))
|
||||||
|
* feat: implement theme selection and automatic theme application based on user preference. Update App ([b70a643](http://192.168.100.56:3000/denozord/EvoBGP/commit/b70a643))
|
||||||
|
* feat: implement versioning and release management in the project ([bb334b1](http://192.168.100.56:3000/denozord/EvoBGP/commit/bb334b1))
|
||||||
|
* feat: improve BGP protocol summary handling by introducing isBGPProtocolSummaryRow function. Update ([399622c](http://192.168.100.56:3000/denozord/EvoBGP/commit/399622c))
|
||||||
|
* feat: improve job detail UI with enhanced spacing and layout adjustments. Update table cell classes ([97908b5](http://192.168.100.56:3000/denozord/EvoBGP/commit/97908b5))
|
||||||
|
* feat: improve peer speaker assignment UX in network settings ([f877ec8](http://192.168.100.56:3000/denozord/EvoBGP/commit/f877ec8))
|
||||||
|
* feat: improve UI components with enhanced overflow handling and layout adjustments. Update card and ([0ca5542](http://192.168.100.56:3000/denozord/EvoBGP/commit/0ca5542))
|
||||||
|
* feat: improve UI layout and responsiveness in module and operations pages. Refactor component struct ([233f29b](http://192.168.100.56:3000/denozord/EvoBGP/commit/233f29b))
|
||||||
|
* feat: redesign settings page with structured controls ([2da6d49](http://192.168.100.56:3000/denozord/EvoBGP/commit/2da6d49))
|
||||||
|
* feat: refactor materialization logic to group static routes by community in the refresh pipeline. In ([1c5a9c9](http://192.168.100.56:3000/denozord/EvoBGP/commit/1c5a9c9))
|
||||||
|
* feat: refactor operations page to enhance UI and component structure. Remove unused card components ([88226f6](http://192.168.100.56:3000/denozord/EvoBGP/commit/88226f6))
|
||||||
|
* feat: refine BGP template handling and documentation updates. Remove local IP requirements in BGP te ([6266a39](http://192.168.100.56:3000/denozord/EvoBGP/commit/6266a39))
|
||||||
|
* feat: refine job detail UI with improved layout and scrolling capabilities. Enhance spacing and over ([57c09ac](http://192.168.100.56:3000/denozord/EvoBGP/commit/57c09ac))
|
||||||
|
* feat: update BGP community structure and related documentation. Refactor BGP community handling by r ([a2cbfac](http://192.168.100.56:3000/denozord/EvoBGP/commit/a2cbfac))
|
||||||
|
* feat: update BGP template configuration and OpenAPI documentation. Change import settings from 'all' ([133ea93](http://192.168.100.56:3000/denozord/EvoBGP/commit/133ea93))
|
||||||
|
* feat: update BIRD installation process to build from source. Modify CI workflow to ensure BIRD 2.14 ([e2002ba](http://192.168.100.56:3000/denozord/EvoBGP/commit/e2002ba))
|
||||||
|
* feat: update CI configuration and Dockerfiles for improved build efficiency ([80229b3](http://192.168.100.56:3000/denozord/EvoBGP/commit/80229b3))
|
||||||
|
* feat: update collectModulePrefixRows to support prior snapshots and enhance CDN/domain prefix collec ([ee364c8](http://192.168.100.56:3000/denozord/EvoBGP/commit/ee364c8))
|
||||||
|
* feat: update Go module dependencies and enhance DNS resolution in refresh pipeline. Upgrade Go versi ([655d750](http://192.168.100.56:3000/denozord/EvoBGP/commit/655d750))
|
||||||
|
* feat: update OpenAPI documentation and internal logic for BIRD settings. Enhance the description of ([62bf987](http://192.168.100.56:3000/denozord/EvoBGP/commit/62bf987))
|
||||||
|
* feat: update RevisionDiff type and enhance OperationsDiffTab component for better diff handling. Int ([4aaaa30](http://192.168.100.56:3000/denozord/EvoBGP/commit/4aaaa30))
|
||||||
|
* feat: update Traefik deployment configuration for Let's Encrypt and logging ([39ba976](http://192.168.100.56:3000/denozord/EvoBGP/commit/39ba976))
|
||||||
|
* docs: add router-lists-ui integration documentation to README ([b7a8aab](http://192.168.100.56:3000/denozord/EvoBGP/commit/b7a8aab))
|
||||||
|
* docs: add Web UI section to EvoBGP architecture plan, detailing Svelte and shadcn-svelte integration ([272542b](http://192.168.100.56:3000/denozord/EvoBGP/commit/272542b))
|
||||||
|
* docs: enhance EvoBGP architecture plan with new risk mitigation strategies, testing matrix for BIRD2 ([3992d01](http://192.168.100.56:3000/denozord/EvoBGP/commit/3992d01))
|
||||||
|
* docs: enhance README and CI workflow documentation for Docker image building and publishing process, ([9eb3e33](http://192.168.100.56:3000/denozord/EvoBGP/commit/9eb3e33))
|
||||||
|
* docs: enhance README and CI workflow to clarify Docker image build conditions and Gitea Container Re ([c6687b2](http://192.168.100.56:3000/denozord/EvoBGP/commit/c6687b2))
|
||||||
|
* docs: openapi.html со встроенным Redoc; инструкция для Gitea и скрипт сборки ([a754e1d](http://192.168.100.56:3000/denozord/EvoBGP/commit/a754e1d))
|
||||||
|
* docs: update AGENTS and README to include guidelines for Conventional Commits ([4c23232](http://192.168.100.56:3000/denozord/EvoBGP/commit/4c23232))
|
||||||
|
* docs: update AGENTS and README with engineering rules and guidelines ([aff27e8](http://192.168.100.56:3000/denozord/EvoBGP/commit/aff27e8))
|
||||||
|
* docs: update architecture and API documentation to clarify job processing, in-memory registry usage, ([acc0608](http://192.168.100.56:3000/denozord/EvoBGP/commit/acc0608))
|
||||||
|
* docs: update EvoBGP architecture plan with detailed REST API section, including general agreements, ([1a86d6c](http://192.168.100.56:3000/denozord/EvoBGP/commit/1a86d6c))
|
||||||
|
* docs: update EvoBGP architecture plan with new repository structure, refined sections, and detailed ([4480d64](http://192.168.100.56:3000/denozord/EvoBGP/commit/4480d64))
|
||||||
|
* docs: update README and CI workflow to clarify Docker workspace mounting for bird2 job ([6be46db](http://192.168.100.56:3000/denozord/EvoBGP/commit/6be46db))
|
||||||
|
* docs: update README and CI workflow to clarify Gitea Registry login process and improve Docker image ([f1b33b9](http://192.168.100.56:3000/denozord/EvoBGP/commit/f1b33b9))
|
||||||
|
* docs: update README and Docker Compose configurations to clarify usage of pre-built images from the ([7fac79c](http://192.168.100.56:3000/denozord/EvoBGP/commit/7fac79c))
|
||||||
|
* docs: update README to include information about the EvoBGP web interface, linking to quickstart and ([5d21f01](http://192.168.100.56:3000/denozord/EvoBGP/commit/5d21f01))
|
||||||
|
* docs: статическая Redoc-страница OpenAPI (для просмотра в Gitea без встроенного рендера) ([5c9d2e2](http://192.168.100.56:3000/denozord/EvoBGP/commit/5c9d2e2))
|
||||||
|
* docs: убрать типографское тире и невидимый Unicode из текстов API ([fbe9cd3](http://192.168.100.56:3000/denozord/EvoBGP/commit/fbe9cd3))
|
||||||
|
* refactor: adjust BootstrapWorkers function call in server.go to change the order of returned values, ([032fa99](http://192.168.100.56:3000/denozord/EvoBGP/commit/032fa99))
|
||||||
|
* refactor: consolidate module interval display logic and improve edit form handling ([1f85654](http://192.168.100.56:3000/denozord/EvoBGP/commit/1f85654))
|
||||||
|
* refactor: enhance bird metrics polling with context support ([f6b94a4](http://192.168.100.56:3000/denozord/EvoBGP/commit/f6b94a4))
|
||||||
|
* refactor: enhance CI workflow for granular change detection by module and improve idempotency key ge ([54eaad3](http://192.168.100.56:3000/denozord/EvoBGP/commit/54eaad3))
|
||||||
|
* refactor: enhance job status checks in worker tests ([6107c3e](http://192.168.100.56:3000/denozord/EvoBGP/commit/6107c3e))
|
||||||
|
* refactor: enhance JSON struct tags for various data models in backend and memory store. Update field ([4a860a0](http://192.168.100.56:3000/denozord/EvoBGP/commit/4a860a0))
|
||||||
|
* refactor: enhance module display and input handling in module pages ([25374a6](http://192.168.100.56:3000/denozord/EvoBGP/commit/25374a6))
|
||||||
|
* refactor: enhance module interval label logic and add reset buttons in edit forms ([b99fc17](http://192.168.100.56:3000/denozord/EvoBGP/commit/b99fc17))
|
||||||
|
* refactor: improve code consistency and formatting across components ([c263fd5](http://192.168.100.56:3000/denozord/EvoBGP/commit/c263fd5))
|
||||||
|
* refactor: improve handling of deployable fragments in ApplyRevision and buildPreviewFragments functi ([6932271](http://192.168.100.56:3000/denozord/EvoBGP/commit/6932271))
|
||||||
|
* refactor: improve module save functionality and nullable value handling ([05ec176](http://192.168.100.56:3000/denozord/EvoBGP/commit/05ec176))
|
||||||
|
* refactor: improve PATCH handling for module updates with nullable field support ([e700f90](http://192.168.100.56:3000/denozord/EvoBGP/commit/e700f90))
|
||||||
|
* refactor: optimize job listing and counting logic for concurrency safety. Separate tenant filtering ([d83efe2](http://192.168.100.56:3000/denozord/EvoBGP/commit/d83efe2))
|
||||||
|
* refactor: phase 2 structural alignment — reports, importer, nodecli, pagination ([250cdea](http://192.168.100.56:3000/denozord/EvoBGP/commit/250cdea))
|
||||||
|
* refactor: remove strict bind configuration from BGP templates and update related logic. Eliminate 's ([482eeb1](http://192.168.100.56:3000/denozord/EvoBGP/commit/482eeb1))
|
||||||
|
* refactor: streamline Docker build process in CI configuration and documentation ([6025f38](http://192.168.100.56:3000/denozord/EvoBGP/commit/6025f38))
|
||||||
|
* refactor: streamline edit form handling for community and DoH profile selections ([1765ab8](http://192.168.100.56:3000/denozord/EvoBGP/commit/1765ab8))
|
||||||
|
* refactor: streamline module refresh process and enhance revision handling ([10e2415](http://192.168.100.56:3000/denozord/EvoBGP/commit/10e2415))
|
||||||
|
* refactor: switch BIRD installation from source to Ubuntu package. Update CI workflow and Dockerfiles ([0b3ceaa](http://192.168.100.56:3000/denozord/EvoBGP/commit/0b3ceaa))
|
||||||
|
* refactor: update AS entry handling in API and database to support ASN-only entries. Remove prefix fi ([1db6b5b](http://192.168.100.56:3000/denozord/EvoBGP/commit/1db6b5b))
|
||||||
|
* refactor: update Docker build context and paths in CI configuration and documentation ([7d52a4a](http://192.168.100.56:3000/denozord/EvoBGP/commit/7d52a4a))
|
||||||
|
* refactor: update Docker build variable syntax in CI configuration and documentation ([27e01d0](http://192.168.100.56:3000/denozord/EvoBGP/commit/27e01d0))
|
||||||
|
* refactor: update job kind display logic in schedule page ([dc1f4a5](http://192.168.100.56:3000/denozord/EvoBGP/commit/dc1f4a5))
|
||||||
|
* refactor: update ListModules and GetModule methods in Postgres repository to include cron expression ([73d0003](http://192.168.100.56:3000/denozord/EvoBGP/commit/73d0003))
|
||||||
|
* refactor: update Switch component styles and structure for improved accessibility and responsiveness ([d4282c2](http://192.168.100.56:3000/denozord/EvoBGP/commit/d4282c2))
|
||||||
|
* refactor: update worker processes in EvoBGP to accept dependencies for shared store and job registry ([b7968db](http://192.168.100.56:3000/denozord/EvoBGP/commit/b7968db))
|
||||||
|
* ci: add conditional job for OpenAPI linting based on file changes; update README with workflow detai ([3adfc49](http://192.168.100.56:3000/denozord/EvoBGP/commit/3adfc49))
|
||||||
|
* ci: phase 4 automation — web job, lint-httpapi, golangci ([64516c3](http://192.168.100.56:3000/denozord/EvoBGP/commit/64516c3))
|
||||||
|
* ci: update README and CI workflow to improve bird2 job installation process and clarify Docker usage ([eddc6b8](http://192.168.100.56:3000/denozord/EvoBGP/commit/eddc6b8))
|
||||||
|
* chore: phase 3 code quality — pipeline prefixes, godoc baseline ([ea65451](http://192.168.100.56:3000/denozord/EvoBGP/commit/ea65451))
|
||||||
|
* chore: update CI configuration to disable filesystem entitlement for Docker builds ([9ab7de6](http://192.168.100.56:3000/denozord/EvoBGP/commit/9ab7de6))
|
||||||
|
* chore: update package dependencies and enhance AppShell component with new navigation items and tool ([7d85fef](http://192.168.100.56:3000/denozord/EvoBGP/commit/7d85fef))
|
||||||
|
* chore: upgrade Go version to 1.24 in Dockerfiles for evobgp-agent and gobinary. This change ensures ([4ba5b03](http://192.168.100.56:3000/denozord/EvoBGP/commit/4ba5b03))
|
||||||
|
* fix: adjust Postgres resource limits and healthcheck parameters. Increase memory limit to 256M and m ([20e674c](http://192.168.100.56:3000/denozord/EvoBGP/commit/20e674c))
|
||||||
|
* fix: improve input handling and validation for settings fields ([1386b63](http://192.168.100.56:3000/denozord/EvoBGP/commit/1386b63))
|
||||||
|
* fix: improve sidebar and navigation layout in AppShell component ([9f02128](http://192.168.100.56:3000/denozord/EvoBGP/commit/9f02128))
|
||||||
|
* fix: update BGP configuration in test scenarios to use new IPv6 addresses ([9fd9745](http://192.168.100.56:3000/denozord/EvoBGP/commit/9fd9745))
|
||||||
|
* fix: update bird build script and Dockerfile to include libncurses dependency ([0cb5a7d](http://192.168.100.56:3000/denozord/EvoBGP/commit/0cb5a7d))
|
||||||
|
* fix: update bird build script and Dockerfile to include readline dependency ([35ed79b](http://192.168.100.56:3000/denozord/EvoBGP/commit/35ed79b))
|
||||||
|
* fix: update bird build script and Dockerfile to remove readline dependency ([fefadac](http://192.168.100.56:3000/denozord/EvoBGP/commit/fefadac))
|
||||||
|
* fix: update deployment configuration for Traefik and secure Web UI ([00bd037](http://192.168.100.56:3000/denozord/EvoBGP/commit/00bd037))
|
||||||
|
* fix: update Docker build commands in CI configuration and documentation ([b5061a5](http://192.168.100.56:3000/denozord/EvoBGP/commit/b5061a5))
|
||||||
|
* fix(httpapi): phase 1 compliance — ERR-01, SEC-04, CDN client, scheduler docs ([1dd7135](http://192.168.100.56:3000/denozord/EvoBGP/commit/1dd7135))
|
||||||
|
* test: add CDN prefetch tests for 304 responses and snapshot handling ([a536a2d](http://192.168.100.56:3000/denozord/EvoBGP/commit/a536a2d))
|
||||||
|
* test: add validation for 'strict bind on' in BGP rendering tests. Enhance BGP test cases to ensure ' ([dc5e20e](http://192.168.100.56:3000/denozord/EvoBGP/commit/dc5e20e))
|
||||||
|
* docker: refactor cache handling in Docker bake configuration ([5e59a1c](http://192.168.100.56:3000/denozord/EvoBGP/commit/5e59a1c))
|
||||||
|
* docker: simplify web image targets in Docker bake configuration ([765a805](http://192.168.100.56:3000/denozord/EvoBGP/commit/765a805))
|
||||||
|
* docker: update web image build process and README ([92f6de8](http://192.168.100.56:3000/denozord/EvoBGP/commit/92f6de8))
|
||||||
|
* web(ui): app layout — sidebar, mobile sheet, topbar, PageHeader ([2d78c81](http://192.168.100.56:3000/denozord/EvoBGP/commit/2d78c81))
|
||||||
|
* web(ui): AppDataTable pattern and modules pilot ([b992301](http://192.168.100.56:3000/denozord/EvoBGP/commit/b992301))
|
||||||
|
* web(ui): design tokens — spacing and component scale ([ac635e9](http://192.168.100.56:3000/denozord/EvoBGP/commit/ac635e9))
|
||||||
|
* web(ui): feedback — confirm dialog, skeletons, README ([87309cf](http://192.168.100.56:3000/denozord/EvoBGP/commit/87309cf))
|
||||||
|
* web(ui): form patterns — FormField, formsnap, directories pilot ([77a2cfd](http://192.168.100.56:3000/denozord/EvoBGP/commit/77a2cfd))
|
||||||
|
* web(ui): foundation — ui/core layout, shadcn primitives, notify ([00a2a44](http://192.168.100.56:3000/denozord/EvoBGP/commit/00a2a44))
|
||||||
|
* web(ui): migrate dashboard, schedule, monitoring to ui patterns ([57dcf30](http://192.168.100.56:3000/denozord/EvoBGP/commit/57dcf30))
|
||||||
|
* web(ui): migrate network to ui patterns ([3a3056b](http://192.168.100.56:3000/denozord/EvoBGP/commit/3a3056b))
|
||||||
|
* web(ui): migrate operations and module detail to ui patterns ([1f9930d](http://192.168.100.56:3000/denozord/EvoBGP/commit/1f9930d))
|
||||||
|
* web(ui): migrate settings to ui patterns ([9808aca](http://192.168.100.56:3000/denozord/EvoBGP/commit/9808aca))
|
||||||
|
* Enhance EvoBGP architecture plan by refining the overview, updating todos with detailed descriptions ([30efdbf](http://192.168.100.56:3000/denozord/EvoBGP/commit/30efdbf))
|
||||||
|
* init ([cbf3aaa](http://192.168.100.56:3000/denozord/EvoBGP/commit/cbf3aaa))
|
||||||
|
* Refine EvoBGP architecture plan with updated database schema details, improved descriptions for comp ([431ef3e](http://192.168.100.56:3000/denozord/EvoBGP/commit/431ef3e))
|
||||||
|
* Update EvoBGP architecture plan with new overview and todos. Refined descriptions for control-plane ([31ff5bb](http://192.168.100.56:3000/denozord/EvoBGP/commit/31ff5bb))
|
||||||
@@ -6,6 +6,7 @@ Control plane для управления префиксами, модулями
|
|||||||
|
|
||||||
| Документ | Содержание |
|
| Документ | Содержание |
|
||||||
|----------|------------|
|
|----------|------------|
|
||||||
|
| [AGENTS.md](AGENTS.md) | Краткая карта репозитория и советы для ИИ-агентов (экономия контекста) |
|
||||||
| [docs/README.md](docs/README.md) | Оглавление и навигация по разделам |
|
| [docs/README.md](docs/README.md) | Оглавление и навигация по разделам |
|
||||||
| [docs/overview.md](docs/overview.md) | Ключевые возможности продукта |
|
| [docs/overview.md](docs/overview.md) | Ключевые возможности продукта |
|
||||||
| [docs/quickstart.md](docs/quickstart.md) | Быстрый запуск (Docker, локально, фронтенд) |
|
| [docs/quickstart.md](docs/quickstart.md) | Быстрый запуск (Docker, локально, фронтенд) |
|
||||||
@@ -43,4 +44,12 @@ docker compose --profile reference up -d
|
|||||||
- **Go:** модуль `evobgp`, точки входа в `cmd/`.
|
- **Go:** модуль `evobgp`, точки входа в `cmd/`.
|
||||||
- **Веб:** SvelteKit в каталоге `web/` (см. [web/README.md](web/README.md)).
|
- **Веб:** SvelteKit в каталоге `web/` (см. [web/README.md](web/README.md)).
|
||||||
|
|
||||||
|
## Версионирование и релизы
|
||||||
|
|
||||||
|
Версии определяются автоматически из Conventional Commits при merge в `main` (semantic-release на Gitea Actions). Первая версия — **1.0.0**; ручной bump не нужен.
|
||||||
|
|
||||||
|
- [docs/releasing.md](docs/releasing.md) — пайплайн, commit conventions, секреты CI
|
||||||
|
- API: `GET /version` и `GET /v1/version` (поле `version`)
|
||||||
|
- Docker-образы: теги `latest`, `vX.Y.Z`, `X.Y.Z` — в **том же CI run**, что и релиз (job `release`)
|
||||||
|
|
||||||
Лицензия и условия использования — по политике владельца репозитория.
|
Лицензия и условия использования — по политике владельца репозитория.
|
||||||
@@ -19,6 +19,7 @@ import (
|
|||||||
"evobgp/internal/platform"
|
"evobgp/internal/platform"
|
||||||
"evobgp/internal/render"
|
"evobgp/internal/render"
|
||||||
"evobgp/internal/scheduler"
|
"evobgp/internal/scheduler"
|
||||||
|
"evobgp/internal/version"
|
||||||
)
|
)
|
||||||
|
|
||||||
// microVPS entrypoint: один процесс — HTTP API и фоновые воркеры scheduler, ingest, render, deploy (общий store и jobs.Registry).
|
// microVPS entrypoint: один процесс — HTTP API и фоновые воркеры scheduler, ingest, render, deploy (общий store и jobs.Registry).
|
||||||
@@ -37,7 +38,7 @@ func main() {
|
|||||||
log.Fatal(err)
|
log.Fatal(err)
|
||||||
}
|
}
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
observability.SetBuildInfo("0.1.0", strings.TrimSpace(os.Getenv("EVOBGP_GIT_SHA")))
|
observability.SetBuildInfo(version.Version, version.GitSHA)
|
||||||
|
|
||||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
defer stop()
|
defer stop()
|
||||||
@@ -51,7 +52,7 @@ func main() {
|
|||||||
go render.Run(ctx, renderDeps)
|
go render.Run(ctx, renderDeps)
|
||||||
go deploy.Run(ctx, deployDeps)
|
go deploy.Run(ctx, deployDeps)
|
||||||
|
|
||||||
startBirdMetricsPoller()
|
startBirdMetricsPoller(ctx)
|
||||||
|
|
||||||
httpSrv := &http.Server{
|
httpSrv := &http.Server{
|
||||||
Addr: cfg.HTTPAddr,
|
Addr: cfg.HTTPAddr,
|
||||||
@@ -75,7 +76,7 @@ func main() {
|
|||||||
log.Printf("%s stopped", platform.ServiceName("evobgp-all"))
|
log.Printf("%s stopped", platform.ServiceName("evobgp-all"))
|
||||||
}
|
}
|
||||||
|
|
||||||
func startBirdMetricsPoller() {
|
func startBirdMetricsPoller(ctx context.Context) {
|
||||||
sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET"))
|
sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET"))
|
||||||
if sock == "" {
|
if sock == "" {
|
||||||
return
|
return
|
||||||
@@ -85,7 +86,7 @@ func startBirdMetricsPoller() {
|
|||||||
interval = d
|
interval = d
|
||||||
}
|
}
|
||||||
bin := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN"))
|
bin := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN"))
|
||||||
observability.StartBirdProtocolsPoller(sock, bin, interval,
|
observability.StartBirdProtocolsPoller(ctx, sock, bin, interval,
|
||||||
func(ctx context.Context, socket, birdcBin string) (string, error) {
|
func(ctx context.Context, socket, birdcBin string) (string, error) {
|
||||||
return birdfmt.ShowProtocols(ctx, socket, birdcBin)
|
return birdfmt.ShowProtocols(ctx, socket, birdcBin)
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -15,6 +15,7 @@ import (
|
|||||||
"evobgp/internal/httpapi"
|
"evobgp/internal/httpapi"
|
||||||
"evobgp/internal/observability"
|
"evobgp/internal/observability"
|
||||||
"evobgp/internal/platform"
|
"evobgp/internal/platform"
|
||||||
|
"evobgp/internal/version"
|
||||||
)
|
)
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
@@ -34,12 +35,12 @@ func main() {
|
|||||||
}
|
}
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
|
|
||||||
observability.SetBuildInfo("0.1.0", strings.TrimSpace(os.Getenv("EVOBGP_GIT_SHA")))
|
observability.SetBuildInfo(version.Version, version.GitSHA)
|
||||||
startBirdMetricsPoller()
|
|
||||||
|
|
||||||
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
|
||||||
defer stop()
|
defer stop()
|
||||||
|
|
||||||
|
startBirdMetricsPoller(ctx)
|
||||||
|
|
||||||
httpSrv := &http.Server{
|
httpSrv := &http.Server{
|
||||||
Addr: cfg.HTTPAddr,
|
Addr: cfg.HTTPAddr,
|
||||||
Handler: srv.Handler(),
|
Handler: srv.Handler(),
|
||||||
@@ -68,7 +69,7 @@ func main() {
|
|||||||
log.Printf("%s stopped", platform.ServiceName("evobgp-api"))
|
log.Printf("%s stopped", platform.ServiceName("evobgp-api"))
|
||||||
}
|
}
|
||||||
|
|
||||||
func startBirdMetricsPoller() {
|
func startBirdMetricsPoller(ctx context.Context) {
|
||||||
sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET"))
|
sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET"))
|
||||||
if sock == "" {
|
if sock == "" {
|
||||||
return
|
return
|
||||||
@@ -78,7 +79,7 @@ func startBirdMetricsPoller() {
|
|||||||
interval = d
|
interval = d
|
||||||
}
|
}
|
||||||
bin := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN"))
|
bin := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN"))
|
||||||
observability.StartBirdProtocolsPoller(sock, bin, interval,
|
observability.StartBirdProtocolsPoller(ctx, sock, bin, interval,
|
||||||
func(ctx context.Context, socket, birdcBin string) (string, error) {
|
func(ctx context.Context, socket, birdcBin string) (string, error) {
|
||||||
return birdfmt.ShowProtocols(ctx, socket, birdcBin)
|
return birdfmt.ShowProtocols(ctx, socket, birdcBin)
|
||||||
},
|
},
|
||||||
|
|||||||
+3
-228
@@ -1,240 +1,15 @@
|
|||||||
package main
|
package main
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
|
||||||
"crypto/ed25519"
|
|
||||||
"encoding/base64"
|
|
||||||
"encoding/json"
|
|
||||||
"flag"
|
|
||||||
"fmt"
|
|
||||||
"io"
|
|
||||||
"net/http"
|
|
||||||
"os"
|
"os"
|
||||||
"path/filepath"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"evobgp/internal/birdfmt"
|
"evobgp/internal/nodecli"
|
||||||
"evobgp/internal/bundle"
|
|
||||||
"evobgp/internal/signing"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
func main() {
|
func main() {
|
||||||
if len(os.Args) < 2 {
|
if len(os.Args) < 2 {
|
||||||
usage()
|
nodecli.Usage(os.Stderr)
|
||||||
os.Exit(2)
|
|
||||||
}
|
|
||||||
switch os.Args[1] {
|
|
||||||
case "pull-bundle":
|
|
||||||
os.Exit(cmdPullBundle(os.Args[2:]))
|
|
||||||
case "verify-bundle":
|
|
||||||
os.Exit(cmdVerifyBundle(os.Args[2:]))
|
|
||||||
case "apply-bundle":
|
|
||||||
os.Exit(cmdApplyBundle(os.Args[2:]))
|
|
||||||
default:
|
|
||||||
usage()
|
|
||||||
os.Exit(2)
|
os.Exit(2)
|
||||||
}
|
}
|
||||||
}
|
os.Exit(nodecli.Run(os.Args[1:]))
|
||||||
|
|
||||||
func usage() {
|
|
||||||
fmt.Fprintf(os.Stderr, `Usage:
|
|
||||||
%s pull-bundle -base-url URL -token TOKEN -speaker-id ID [-revision-id ID] [-o path]
|
|
||||||
%s verify-bundle -f bundle.tar.gz (-pubkey-base64 B64 | -pubkey-hex HEX)
|
|
||||||
%s apply-bundle -f bundle.tar.gz -extract-dir DIR (-pubkey-base64 B64 | -pubkey-hex HEX)
|
|
||||||
[-bird PATH] [-birdc PATH] [-socket PATH] [-timeout DURATION]
|
|
||||||
|
|
||||||
apply-bundle verifies, extracts, runs bird -p on main bird.conf, then birdc configure.
|
|
||||||
`, os.Args[0], os.Args[0], os.Args[0])
|
|
||||||
}
|
|
||||||
|
|
||||||
func cmdPullBundle(args []string) int {
|
|
||||||
fs := flag.NewFlagSet("pull-bundle", flag.ExitOnError)
|
|
||||||
base := fs.String("base-url", "", "control plane base URL, e.g. http://localhost:8080")
|
|
||||||
token := fs.String("token", "", "Bearer token (node role)")
|
|
||||||
speaker := fs.String("speaker-id", "", "bgp_speaker id")
|
|
||||||
revision := fs.String("revision-id", "", "revision to fetch (empty = latest pointer)")
|
|
||||||
out := fs.String("o", "bundle.tar.gz", "output file")
|
|
||||||
_ = fs.Parse(args)
|
|
||||||
|
|
||||||
if strings.TrimSpace(*base) == "" || *token == "" || *speaker == "" {
|
|
||||||
fmt.Fprintln(os.Stderr, "pull-bundle: -base-url, -token, -speaker-id are required")
|
|
||||||
return 2
|
|
||||||
}
|
|
||||||
rev := strings.TrimSpace(*revision)
|
|
||||||
if rev == "" {
|
|
||||||
var err error
|
|
||||||
rev, err = fetchLatestRevision(*base, *token, *speaker)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
}
|
|
||||||
body, err := fetchBundle(*base, *token, *speaker, rev)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
if err := os.WriteFile(*out, body, 0o644); err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
fmt.Fprintf(os.Stderr, "wrote %s (revision %s)\n", *out, rev)
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
func fetchLatestRevision(base, token, speaker string) (string, error) {
|
|
||||||
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/revisions/latest"
|
|
||||||
req, err := http.NewRequest(http.MethodGet, u, nil)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+token)
|
|
||||||
resp, err := http.DefaultClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
b, _ := io.ReadAll(resp.Body)
|
|
||||||
return "", fmt.Errorf("latest revision: %s: %s", resp.Status, strings.TrimSpace(string(b)))
|
|
||||||
}
|
|
||||||
var out struct {
|
|
||||||
RevisionID string `json:"revision_id"`
|
|
||||||
}
|
|
||||||
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
|
|
||||||
return "", err
|
|
||||||
}
|
|
||||||
if out.RevisionID == "" {
|
|
||||||
return "", fmt.Errorf("empty revision_id in response")
|
|
||||||
}
|
|
||||||
return out.RevisionID, nil
|
|
||||||
}
|
|
||||||
|
|
||||||
func fetchBundle(base, token, speaker, revision string) ([]byte, error) {
|
|
||||||
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/bundle/" + revision
|
|
||||||
req, err := http.NewRequest(http.MethodGet, u, nil)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
req.Header.Set("Authorization", "Bearer "+token)
|
|
||||||
resp, err := http.DefaultClient.Do(req)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
defer resp.Body.Close()
|
|
||||||
if resp.StatusCode != http.StatusOK {
|
|
||||||
b, _ := io.ReadAll(resp.Body)
|
|
||||||
return nil, fmt.Errorf("bundle: %s: %s", resp.Status, strings.TrimSpace(string(b)))
|
|
||||||
}
|
|
||||||
return io.ReadAll(resp.Body)
|
|
||||||
}
|
|
||||||
|
|
||||||
func loadPubKey(pubB64, pubHex string) (ed25519.PublicKey, error) {
|
|
||||||
switch {
|
|
||||||
case strings.TrimSpace(pubB64) != "":
|
|
||||||
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(pubB64))
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
if len(raw) != ed25519.PublicKeySize {
|
|
||||||
return nil, fmt.Errorf("pubkey-base64 must decode to %d bytes", ed25519.PublicKeySize)
|
|
||||||
}
|
|
||||||
return ed25519.PublicKey(raw), nil
|
|
||||||
case strings.TrimSpace(pubHex) != "":
|
|
||||||
return bundle.ParsePublicKeyHex(pubHex)
|
|
||||||
default:
|
|
||||||
return nil, fmt.Errorf("public key required")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func cmdVerifyBundle(args []string) int {
|
|
||||||
fs := flag.NewFlagSet("verify-bundle", flag.ExitOnError)
|
|
||||||
path := fs.String("f", "", "path to bundle.tar.gz")
|
|
||||||
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
|
|
||||||
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
|
|
||||||
_ = fs.Parse(args)
|
|
||||||
|
|
||||||
if *path == "" {
|
|
||||||
fmt.Fprintln(os.Stderr, "verify-bundle: -f required")
|
|
||||||
return 2
|
|
||||||
}
|
|
||||||
pub, err := loadPubKey(*pubB64, *pubHex)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 2
|
|
||||||
}
|
|
||||||
raw, err := os.ReadFile(*path)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
v, err := signing.VerifyGzippedTar(raw, pub)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
fmt.Fprintf(os.Stderr, "ok: revision %s, %d payload files\n", v.Manifest.RevisionID, len(v.Files))
|
|
||||||
return 0
|
|
||||||
}
|
|
||||||
|
|
||||||
func cmdApplyBundle(args []string) int {
|
|
||||||
fs := flag.NewFlagSet("apply-bundle", flag.ExitOnError)
|
|
||||||
path := fs.String("f", "", "path to bundle.tar.gz")
|
|
||||||
dir := fs.String("extract-dir", "", "directory to extract into")
|
|
||||||
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
|
|
||||||
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
|
|
||||||
bird := fs.String("bird", "", "bird binary (default PATH)")
|
|
||||||
birdc := fs.String("birdc", "", "birdc binary (default PATH)")
|
|
||||||
socket := fs.String("socket", "", "birdc -s socket path")
|
|
||||||
timeout := fs.Duration("timeout", 30*time.Second, "timeout for bird/birdc")
|
|
||||||
_ = fs.Parse(args)
|
|
||||||
|
|
||||||
if *path == "" || *dir == "" {
|
|
||||||
fmt.Fprintln(os.Stderr, "apply-bundle: -f and -extract-dir required")
|
|
||||||
return 2
|
|
||||||
}
|
|
||||||
pub, err := loadPubKey(*pubB64, *pubHex)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 2
|
|
||||||
}
|
|
||||||
raw, err := os.ReadFile(*path)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
v, err := signing.VerifyGzippedTar(raw, pub)
|
|
||||||
if err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
root := filepath.Clean(*dir)
|
|
||||||
if err := os.MkdirAll(root, 0o755); err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
if err := bundle.WriteExtractedFiles(root, v); err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
mainRel := v.FindMainBirdConf()
|
|
||||||
if mainRel == "" {
|
|
||||||
fmt.Fprintln(os.Stderr, "bundle has no bird.conf path in manifest")
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
mainPath := filepath.Join(root, filepath.FromSlash(strings.TrimPrefix(mainRel, "/")))
|
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), *timeout)
|
|
||||||
defer cancel()
|
|
||||||
ctl := &birdfmt.BirdCtl{Bird: *bird, Birdc: *birdc, Socket: *socket}
|
|
||||||
if err := ctl.ParseCheck(ctx, mainPath); err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
if err := ctl.Configure(ctx); err != nil {
|
|
||||||
fmt.Fprintln(os.Stderr, err)
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
fmt.Fprintf(os.Stderr, "applied revision %s (main config %s)\n", v.Manifest.RevisionID, mainPath)
|
|
||||||
return 0
|
|
||||||
}
|
}
|
||||||
@@ -41,6 +41,9 @@ func main() {
|
|||||||
|
|
||||||
apiBase := strings.TrimSpace(os.Getenv("EVOBGP_CONTROL_PLANE_URL"))
|
apiBase := strings.TrimSpace(os.Getenv("EVOBGP_CONTROL_PLANE_URL"))
|
||||||
apiTok := strings.TrimSpace(os.Getenv("EVOBGP_SCHEDULER_BEARER"))
|
apiTok := strings.TrimSpace(os.Getenv("EVOBGP_SCHEDULER_BEARER"))
|
||||||
|
if os.Getenv("EVOBGP_SCHEDULER_STANDALONE") == "1" && apiBase == "" {
|
||||||
|
log.Fatal("evobgp-scheduler: EVOBGP_SCHEDULER_STANDALONE=1 requires EVOBGP_CONTROL_PLANE_URL and EVOBGP_SCHEDULER_BEARER (split deploy must not use in-process jobs.Registry)")
|
||||||
|
}
|
||||||
deps := &scheduler.Deps{
|
deps := &scheduler.Deps{
|
||||||
Store: st,
|
Store: st,
|
||||||
HTTP: &http.Client{Timeout: 45 * time.Second},
|
HTTP: &http.Client{Timeout: 45 * time.Second},
|
||||||
|
|||||||
@@ -3,3 +3,6 @@
|
|||||||
|
|
||||||
EVOBGP_REGISTRY=git.shts.su/denozord
|
EVOBGP_REGISTRY=git.shts.su/denozord
|
||||||
EVOBGP_IMAGE_TAG=latest
|
EVOBGP_IMAGE_TAG=latest
|
||||||
|
|
||||||
|
# Для защищенного Web UI (Traefik + Let's Encrypt + Cloudflare DNS challenge)
|
||||||
|
# используйте второй файл рядом: .env.web-sec (шаблон: .env.web-sec.example)
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
# Standalone stack env (Linux): stack.microvps-full.yaml
|
||||||
|
# Использование:
|
||||||
|
# cp .env.stack.microvps-full.example .env.stack.microvps-full
|
||||||
|
# docker compose --env-file .env.stack.microvps-full -f stack.microvps-full.yaml up -d
|
||||||
|
|
||||||
|
# Образы EvoBGP
|
||||||
|
EVOBGP_REGISTRY=git.shts.su/denozord
|
||||||
|
EVOBGP_IMAGE_TAG=latest
|
||||||
|
|
||||||
|
# Защищенный Web UI (Traefik + Let's Encrypt + Cloudflare DNS challenge)
|
||||||
|
WEBUI_DOMAIN=bgp.shz.su
|
||||||
|
WEBUI_IP_WHITELIST=109.174.26.78/32,87.103.241.8/32
|
||||||
|
LETSENCRYPT_EMAIL=admin@shz.su
|
||||||
|
CF_DNS_API_TOKEN=cfut_3D1i0MNBVX6MNSsjcFytsfSlyB4B15t7H3DYXuzq22011588
|
||||||
|
|
||||||
|
# Auto-updater (проверка registry образов и точечный restart контейнеров)
|
||||||
|
# 0 = выключен, 1 = включен
|
||||||
|
AUTO_UPDATE_ENABLED=0
|
||||||
|
# Интервал проверки обновлений (сек), минимум 30
|
||||||
|
AUTO_UPDATE_INTERVAL_SEC=300
|
||||||
|
# Сервисы для автообновления (через запятую)
|
||||||
|
AUTO_UPDATE_SERVICES=evobgp-all,evobgp-web
|
||||||
|
# Защищенные сервисы, которые updater никогда не перезапускает
|
||||||
|
AUTO_UPDATE_PROTECTED_SERVICES=bird2
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
# Параметры защищенного Web UI (Traefik + Let's Encrypt DNS challenge через Cloudflare).
|
||||||
|
# Используйте вместе с deploy/compose/.env:
|
||||||
|
# docker compose --env-file .env --env-file .env.web-sec ...
|
||||||
|
|
||||||
|
# FQDN для Web UI (A/AAAA запись должна указывать на VPS; Cloudflare: DNS only).
|
||||||
|
WEBUI_DOMAIN=bgp.example.com
|
||||||
|
|
||||||
|
# Разрешенные IP/CIDR (через запятую).
|
||||||
|
# Пример: один IP + подсеть офиса.
|
||||||
|
WEBUI_IP_WHITELIST=203.0.113.10/32,198.51.100.0/24
|
||||||
|
|
||||||
|
# Email для ACME аккаунта Let's Encrypt.
|
||||||
|
LETSENCRYPT_EMAIL=admin@example.com
|
||||||
|
|
||||||
|
# Cloudflare API token с правами Zone:DNS:Edit только на нужную зону.
|
||||||
|
CF_DNS_API_TOKEN=replace_me
|
||||||
@@ -7,6 +7,7 @@
|
|||||||
# docker compose -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full down
|
# docker compose -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full down
|
||||||
#
|
#
|
||||||
# Лимиты под хост ~1 ГиБ: сумма лимитов контейнеров ~800 МиБ + запас ОС/Docker. При OOM увеличьте RAM VPS или уберите prometheus/nats из профиля (отдельная правка compose).
|
# Лимиты под хост ~1 ГиБ: сумма лимитов контейнеров ~800 МиБ + запас ОС/Docker. При OOM увеличьте RAM VPS или уберите prometheus/nats из профиля (отдельная правка compose).
|
||||||
|
# Postgres ниже ~240–256 МиБ часто не успевает пройти healthcheck / падает по памяти — не уменьшайте postgres без tuned postgresql.conf.
|
||||||
|
|
||||||
services:
|
services:
|
||||||
evobgp-all:
|
evobgp-all:
|
||||||
@@ -23,7 +24,7 @@ services:
|
|||||||
resources:
|
resources:
|
||||||
limits:
|
limits:
|
||||||
cpus: "0.45"
|
cpus: "0.45"
|
||||||
memory: 180M
|
memory: 256M
|
||||||
|
|
||||||
bird2:
|
bird2:
|
||||||
deploy:
|
deploy:
|
||||||
|
|||||||
@@ -45,6 +45,8 @@ services:
|
|||||||
profiles: ["reference", "microvps", "microvps-full"]
|
profiles: ["reference", "microvps", "microvps-full"]
|
||||||
image: postgres:16-alpine
|
image: postgres:16-alpine
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
|
# Дефолтный /dev/shm в Docker (64 МиБ) мало для Postgres; иначе ошибки shared memory при старте.
|
||||||
|
shm_size: "256mb"
|
||||||
environment:
|
environment:
|
||||||
POSTGRES_USER: evobgp
|
POSTGRES_USER: evobgp
|
||||||
POSTGRES_PASSWORD: evobgp
|
POSTGRES_PASSWORD: evobgp
|
||||||
@@ -55,7 +57,8 @@ services:
|
|||||||
test: ["CMD-SHELL", "pg_isready -U evobgp -d evobgp"]
|
test: ["CMD-SHELL", "pg_isready -U evobgp -d evobgp"]
|
||||||
interval: 10s
|
interval: 10s
|
||||||
timeout: 5s
|
timeout: 5s
|
||||||
retries: 5
|
retries: 12
|
||||||
|
start_period: 45s
|
||||||
logging: *default-logging
|
logging: *default-logging
|
||||||
deploy:
|
deploy:
|
||||||
resources:
|
resources:
|
||||||
@@ -123,6 +126,8 @@ services:
|
|||||||
condition: service_started
|
condition: service_started
|
||||||
environment:
|
environment:
|
||||||
<<: *env-ref
|
<<: *env-ref
|
||||||
|
# Split deploy: scheduler must enqueue jobs via API, not in-process Registry (ARCH-04).
|
||||||
|
EVOBGP_SCHEDULER_STANDALONE: "1"
|
||||||
EVOBGP_CONTROL_PLANE_URL: http://evobgp-api:8080
|
EVOBGP_CONTROL_PLANE_URL: http://evobgp-api:8080
|
||||||
EVOBGP_SCHEDULER_BEARER: dev
|
EVOBGP_SCHEDULER_BEARER: dev
|
||||||
logging: *default-logging
|
logging: *default-logging
|
||||||
@@ -240,8 +245,15 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
depends_on:
|
depends_on:
|
||||||
- evobgp-all
|
- evobgp-all
|
||||||
ports:
|
labels:
|
||||||
- "3000:80"
|
- traefik.enable=true
|
||||||
|
- traefik.http.routers.evobgp-web.rule=Host(`${WEBUI_DOMAIN}`)
|
||||||
|
- traefik.http.routers.evobgp-web.entrypoints=websecure
|
||||||
|
- traefik.http.routers.evobgp-web.tls=true
|
||||||
|
- traefik.http.routers.evobgp-web.tls.certresolver=letsencrypt
|
||||||
|
- traefik.http.routers.evobgp-web.middlewares=webui-ipwhitelist@docker
|
||||||
|
- traefik.http.middlewares.webui-ipwhitelist.ipallowlist.sourcerange=${WEBUI_IP_WHITELIST}
|
||||||
|
- traefik.http.services.evobgp-web.loadbalancer.server.port=80
|
||||||
logging: *default-logging
|
logging: *default-logging
|
||||||
deploy:
|
deploy:
|
||||||
resources:
|
resources:
|
||||||
@@ -249,6 +261,41 @@ services:
|
|||||||
cpus: "0.5"
|
cpus: "0.5"
|
||||||
memory: 128M
|
memory: 128M
|
||||||
|
|
||||||
|
evobgp-edge:
|
||||||
|
profiles: ["microvps-full"]
|
||||||
|
image: traefik:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
- evobgp-web-microvps
|
||||||
|
ports:
|
||||||
|
- "80:80"
|
||||||
|
- "443:443"
|
||||||
|
environment:
|
||||||
|
DOCKER_API_VERSION: "1.44"
|
||||||
|
CF_DNS_API_TOKEN: ${CF_DNS_API_TOKEN}
|
||||||
|
command:
|
||||||
|
- --api.dashboard=false
|
||||||
|
- --providers.docker=true
|
||||||
|
- --providers.docker.exposedbydefault=false
|
||||||
|
- --entrypoints.web.address=:80
|
||||||
|
- --entrypoints.websecure.address=:443
|
||||||
|
- --entrypoints.web.http.redirections.entrypoint.to=websecure
|
||||||
|
- --entrypoints.web.http.redirections.entrypoint.scheme=https
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL}
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.dnschallenge=true
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.dnschallenge.delaybeforecheck=15
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
- traefik_letsencrypt:/letsencrypt
|
||||||
|
logging: *default-logging
|
||||||
|
deploy:
|
||||||
|
resources:
|
||||||
|
limits:
|
||||||
|
cpus: "0.35"
|
||||||
|
memory: 128M
|
||||||
|
|
||||||
prometheus-microvps:
|
prometheus-microvps:
|
||||||
profiles: ["microvps-full"]
|
profiles: ["microvps-full"]
|
||||||
image: prom/prometheus:v2.54.1
|
image: prom/prometheus:v2.54.1
|
||||||
@@ -337,3 +384,4 @@ volumes:
|
|||||||
pgdata:
|
pgdata:
|
||||||
bird_etc:
|
bird_etc:
|
||||||
bird_run:
|
bird_run:
|
||||||
|
traefik_letsencrypt:
|
||||||
@@ -0,0 +1,342 @@
|
|||||||
|
# Standalone stack (без profiles): Portainer / `docker compose -f stack.microvps-full.yaml up -d`
|
||||||
|
# Эквивалент microvps-full из docker-compose.yaml + docker-compose.microvps-full.yaml,
|
||||||
|
# с встроенными configs для Prometheus (как у вас).
|
||||||
|
#
|
||||||
|
# Postgres: shm_size + start_period — иначе часто «dependency postgres failed» на слабом хосте.
|
||||||
|
# Для защищенного Web UI задайте переменные окружения перед запуском:
|
||||||
|
# WEBUI_DOMAIN, WEBUI_IP_WHITELIST, LETSENCRYPT_EMAIL, CF_DNS_API_TOKEN
|
||||||
|
# Cloudflare запись для WEBUI_DOMAIN: DNS only (серый облачок).
|
||||||
|
#
|
||||||
|
# ACME / Let's Encrypt: путь задан в evobgp-edge (--certificatesresolvers...storage).
|
||||||
|
# Имя тома traefik_letsencrypt зафиксировано (volumes.traefik_letsencrypt.name), иначе при
|
||||||
|
# другом имени стека в Portainer или смене project name создаётся новый пустой том —
|
||||||
|
# Traefik не находит acme.json и отдаёт дефолтный сертификат до новой выдачи LE.
|
||||||
|
# Не использовать `docker compose down -v` без бэкапа. Если раньше был том с префиксом
|
||||||
|
# проекта, перенесите acme.json в том evobgp_traefik_letsencrypt.
|
||||||
|
#
|
||||||
|
# Долгий сбор логов в файлы на хосте: сервис stack-runtime-logs пишет в каталог
|
||||||
|
# ./runtime-logs/ (рядом с этим compose-файлом) по одному файлу на сервис.
|
||||||
|
# Автообновление сервисов из registry (без перезапуска bird2): stack-auto-updater.
|
||||||
|
# Настройки через .env: AUTO_UPDATE_ENABLED, AUTO_UPDATE_INTERVAL_SEC,
|
||||||
|
# AUTO_UPDATE_SERVICES, AUTO_UPDATE_PROTECTED_SERVICES.
|
||||||
|
# Имя проекта в Docker должно совпадать с label com.docker.compose.project:
|
||||||
|
# при смене `name:` или имени стека в Portainer задайте COMPOSE_PROJECT_NAME.
|
||||||
|
# Требуется доступ к docker.sock (полные права на демон — осознанно).
|
||||||
|
name: evobgp-microvps-full
|
||||||
|
|
||||||
|
configs:
|
||||||
|
prometheus_yml:
|
||||||
|
content: |
|
||||||
|
global:
|
||||||
|
scrape_interval: 15s
|
||||||
|
evaluation_interval: 15s
|
||||||
|
rule_files:
|
||||||
|
- /etc/prometheus/alerts.yml
|
||||||
|
scrape_configs:
|
||||||
|
- job_name: evobgp-all
|
||||||
|
metrics_path: /metrics
|
||||||
|
static_configs:
|
||||||
|
- targets: ["evobgp-all:8080"]
|
||||||
|
prometheus_alerts_yml:
|
||||||
|
content: |
|
||||||
|
groups: []
|
||||||
|
|
||||||
|
services:
|
||||||
|
postgres:
|
||||||
|
image: postgres:16-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
shm_size: "256mb"
|
||||||
|
environment:
|
||||||
|
POSTGRES_USER: evobgp
|
||||||
|
POSTGRES_PASSWORD: evobgp
|
||||||
|
POSTGRES_DB: evobgp
|
||||||
|
volumes:
|
||||||
|
- pgdata:/var/lib/postgresql/data
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD-SHELL", "pg_isready -U evobgp -d evobgp"]
|
||||||
|
interval: 10s
|
||||||
|
timeout: 5s
|
||||||
|
retries: 12
|
||||||
|
start_period: 45s
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
nats:
|
||||||
|
image: nats:2.10-alpine
|
||||||
|
restart: unless-stopped
|
||||||
|
command: ["-js", "-m", "8222"]
|
||||||
|
ports:
|
||||||
|
- "4222:4222"
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
bird2:
|
||||||
|
image: ${EVOBGP_REGISTRY:-git.shts.su/denozord}/evobgp-bird2:${EVOBGP_IMAGE_TAG:-latest}
|
||||||
|
restart: unless-stopped
|
||||||
|
cap_add:
|
||||||
|
- NET_ADMIN
|
||||||
|
sysctls:
|
||||||
|
net.ipv4.ip_forward: "1"
|
||||||
|
net.ipv6.conf.all.forwarding: "1"
|
||||||
|
volumes:
|
||||||
|
- bird_etc:/etc/bird
|
||||||
|
- bird_run:/run/bird
|
||||||
|
ports:
|
||||||
|
- "179:179/tcp"
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
evobgp-agent:
|
||||||
|
image: ${EVOBGP_REGISTRY:-git.shts.su/denozord}/evobgp-agent:${EVOBGP_IMAGE_TAG:-latest}
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
bird2:
|
||||||
|
condition: service_started
|
||||||
|
cap_add:
|
||||||
|
- NET_ADMIN
|
||||||
|
volumes:
|
||||||
|
- bird_etc:/etc/bird
|
||||||
|
- bird_run:/run/bird
|
||||||
|
entrypoint: ["/usr/local/bin/evobgp-agent"]
|
||||||
|
command: ["watch", "-socket=/run/bird/bird.ctl", "-watch-interval=30s"]
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
evobgp-all:
|
||||||
|
image: ${EVOBGP_REGISTRY:-git.shts.su/denozord}/evobgp-all:${EVOBGP_IMAGE_TAG:-latest}
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
postgres:
|
||||||
|
condition: service_healthy
|
||||||
|
nats:
|
||||||
|
condition: service_started
|
||||||
|
bird2:
|
||||||
|
condition: service_started
|
||||||
|
ports:
|
||||||
|
- "8080:8080"
|
||||||
|
environment:
|
||||||
|
EVOBGP_DATABASE_URL: postgres://evobgp:evobgp@postgres:5432/evobgp?sslmode=disable
|
||||||
|
EVOBGP_BROKER_URL: nats://nats:4222
|
||||||
|
EVOBGP_HTTP_ADDR: ":8080"
|
||||||
|
EVOBGP_SEED_DEMO: "1"
|
||||||
|
EVOBGP_BIRDC_SOCKET: /run/bird/bird.ctl
|
||||||
|
EVOBGP_BIRDC_INTERVAL: 30s
|
||||||
|
EVOBGP_BIRD_ACTIVE_DIR: /etc/bird
|
||||||
|
EVOBGP_BIRD_STAGING_DIR: /tmp/evobgp-bird-staging
|
||||||
|
EVOBGP_DEV_INSECURE: "1"
|
||||||
|
volumes:
|
||||||
|
- bird_etc:/etc/bird
|
||||||
|
- bird_run:/run/bird:ro
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
evobgp-web:
|
||||||
|
image: ${EVOBGP_REGISTRY:-git.shts.su/denozord}/evobgp-web-all:${EVOBGP_IMAGE_TAG:-latest}
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
evobgp-all:
|
||||||
|
condition: service_started
|
||||||
|
labels:
|
||||||
|
- traefik.enable=true
|
||||||
|
- traefik.http.routers.evobgp-web.rule=Host(`${WEBUI_DOMAIN}`)
|
||||||
|
- traefik.http.routers.evobgp-web.entrypoints=websecure
|
||||||
|
- traefik.http.routers.evobgp-web.tls=true
|
||||||
|
- traefik.http.routers.evobgp-web.tls.certresolver=letsencrypt
|
||||||
|
- traefik.http.routers.evobgp-web.middlewares=webui-ipwhitelist@docker
|
||||||
|
- traefik.http.middlewares.webui-ipwhitelist.ipallowlist.sourcerange=${WEBUI_IP_WHITELIST}
|
||||||
|
- traefik.http.services.evobgp-web.loadbalancer.server.port=80
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
evobgp-edge:
|
||||||
|
image: traefik:latest
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
evobgp-web:
|
||||||
|
condition: service_started
|
||||||
|
ports:
|
||||||
|
- "80:80"
|
||||||
|
- "443:443"
|
||||||
|
environment:
|
||||||
|
DOCKER_API_VERSION: "1.44"
|
||||||
|
CF_DNS_API_TOKEN: ${CF_DNS_API_TOKEN}
|
||||||
|
command:
|
||||||
|
- --log.level=INFO
|
||||||
|
- --api.dashboard=false
|
||||||
|
- --providers.docker=true
|
||||||
|
- --providers.docker.exposedbydefault=false
|
||||||
|
- --entrypoints.web.address=:80
|
||||||
|
- --entrypoints.websecure.address=:443
|
||||||
|
- --entrypoints.web.http.redirections.entrypoint.to=websecure
|
||||||
|
- --entrypoints.web.http.redirections.entrypoint.scheme=https
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.email=${LETSENCRYPT_EMAIL}
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.storage=/letsencrypt/acme.json
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.dnschallenge=true
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.dnschallenge.provider=cloudflare
|
||||||
|
- --certificatesresolvers.letsencrypt.acme.dnschallenge.delaybeforecheck=15
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
- traefik_letsencrypt:/letsencrypt
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
prometheus:
|
||||||
|
image: prom/prometheus:v2.54.1
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
evobgp-all:
|
||||||
|
condition: service_started
|
||||||
|
ports:
|
||||||
|
- "9090:9090"
|
||||||
|
configs:
|
||||||
|
- source: prometheus_yml
|
||||||
|
target: /etc/prometheus/prometheus.yml
|
||||||
|
- source: prometheus_alerts_yml
|
||||||
|
target: /etc/prometheus/alerts.yml
|
||||||
|
command:
|
||||||
|
- --config.file=/etc/prometheus/prometheus.yml
|
||||||
|
- --storage.tsdb.path=/prometheus
|
||||||
|
- --storage.tsdb.retention.time=7d
|
||||||
|
- --web.enable-lifecycle
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
stack-runtime-logs:
|
||||||
|
image: docker:27-cli
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
evobgp-all:
|
||||||
|
condition: service_started
|
||||||
|
environment:
|
||||||
|
COMPOSE_PROJECT_NAME: ${COMPOSE_PROJECT_NAME:-evobgp-microvps-full}
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock
|
||||||
|
- type: bind
|
||||||
|
source: ./runtime-logs
|
||||||
|
target: /logs
|
||||||
|
entrypoint: ["/bin/sh", "-c"]
|
||||||
|
command:
|
||||||
|
- |
|
||||||
|
set -eu
|
||||||
|
mkdir -p /logs
|
||||||
|
PROJECT=$$COMPOSE_PROJECT_NAME
|
||||||
|
SERVICES="postgres nats bird2 evobgp-agent evobgp-all evobgp-web evobgp-edge prometheus"
|
||||||
|
log_one() {
|
||||||
|
svc=$$1
|
||||||
|
f="/logs/$$svc.log"
|
||||||
|
while true; do
|
||||||
|
cid=$$(docker ps -q \
|
||||||
|
-f "label=com.docker.compose.service=$$svc" \
|
||||||
|
-f "label=com.docker.compose.project=$$PROJECT" | head -n1)
|
||||||
|
if [ -n "$$cid" ]; then
|
||||||
|
echo "---- $$(date -u +"%Y-%m-%dT%H:%M:%SZ") attach $$svc $$cid ----" >> "$$f"
|
||||||
|
docker logs -f --timestamps "$$cid" >> "$$f" 2>&1 || true
|
||||||
|
fi
|
||||||
|
sleep 3
|
||||||
|
done
|
||||||
|
}
|
||||||
|
for s in $$SERVICES; do log_one "$$s" & done
|
||||||
|
wait
|
||||||
|
|
||||||
|
stack-auto-updater:
|
||||||
|
image: docker:27-cli
|
||||||
|
restart: unless-stopped
|
||||||
|
depends_on:
|
||||||
|
evobgp-all:
|
||||||
|
condition: service_started
|
||||||
|
environment:
|
||||||
|
COMPOSE_PROJECT_NAME: ${COMPOSE_PROJECT_NAME:-evobgp-microvps-full}
|
||||||
|
AUTO_UPDATE_ENABLED: ${AUTO_UPDATE_ENABLED:-0}
|
||||||
|
AUTO_UPDATE_INTERVAL_SEC: ${AUTO_UPDATE_INTERVAL_SEC:-300}
|
||||||
|
AUTO_UPDATE_SERVICES: ${AUTO_UPDATE_SERVICES:-evobgp-all,evobgp-web}
|
||||||
|
AUTO_UPDATE_PROTECTED_SERVICES: ${AUTO_UPDATE_PROTECTED_SERVICES:-bird2}
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock
|
||||||
|
- type: bind
|
||||||
|
source: ./stack.microvps-full.yaml
|
||||||
|
target: /stack/stack.microvps-full.yaml
|
||||||
|
read_only: true
|
||||||
|
entrypoint: ["/bin/sh", "-c"]
|
||||||
|
command:
|
||||||
|
- |
|
||||||
|
set -eu
|
||||||
|
PROJECT=$$COMPOSE_PROJECT_NAME
|
||||||
|
INTERVAL=$$AUTO_UPDATE_INTERVAL_SEC
|
||||||
|
ENABLED=$$AUTO_UPDATE_ENABLED
|
||||||
|
if [ "$$ENABLED" != "1" ]; then
|
||||||
|
echo "auto-updater disabled: AUTO_UPDATE_ENABLED=$$ENABLED"
|
||||||
|
while true; do sleep 3600; done
|
||||||
|
fi
|
||||||
|
if [ "$$INTERVAL" -lt 30 ]; then
|
||||||
|
echo "auto-updater interval too low ($$INTERVAL), forcing to 30"
|
||||||
|
INTERVAL=30
|
||||||
|
fi
|
||||||
|
echo "auto-updater started: project=$$PROJECT interval=$$INTERVAL services=$$AUTO_UPDATE_SERVICES protected=$$AUTO_UPDATE_PROTECTED_SERVICES"
|
||||||
|
while true; do
|
||||||
|
TS=$$(date -u +"%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
SERVICES_LIST=$$(echo "$$AUTO_UPDATE_SERVICES" | tr ',' ' ')
|
||||||
|
PROTECTED_LIST=$$(echo "$$AUTO_UPDATE_PROTECTED_SERVICES" | tr ',' ' ')
|
||||||
|
for svc in $$SERVICES_LIST; do
|
||||||
|
if [ -z "$$svc" ]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
skip=0
|
||||||
|
for protected in $$PROTECTED_LIST; do
|
||||||
|
if [ "$$svc" = "$$protected" ]; then
|
||||||
|
echo "$$TS skip protected service '$$svc'"
|
||||||
|
skip=1
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ "$$skip" = "1" ]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
before=$$(docker image inspect "$$(docker compose -p "$$PROJECT" -f /stack/stack.microvps-full.yaml config --images $$svc | tail -n1)" --format '{{.Id}}' 2>/dev/null || true)
|
||||||
|
pull_out=$$(docker compose -p "$$PROJECT" -f /stack/stack.microvps-full.yaml pull $$svc 2>&1 || true)
|
||||||
|
echo "$$TS pull $$svc: $$pull_out"
|
||||||
|
after=$$(docker image inspect "$$(docker compose -p "$$PROJECT" -f /stack/stack.microvps-full.yaml config --images $$svc | tail -n1)" --format '{{.Id}}' 2>/dev/null || true)
|
||||||
|
if [ -n "$$after" ] && [ "$$before" != "$$after" ]; then
|
||||||
|
echo "$$TS update detected for $$svc, restarting container"
|
||||||
|
docker compose -p "$$PROJECT" -f /stack/stack.microvps-full.yaml up -d --no-deps $$svc
|
||||||
|
else
|
||||||
|
echo "$$TS no update for $$svc"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
sleep "$$INTERVAL"
|
||||||
|
done
|
||||||
|
logging:
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: "10m"
|
||||||
|
max-file: "3"
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
pgdata:
|
||||||
|
bird_etc:
|
||||||
|
bird_run:
|
||||||
|
traefik_letsencrypt:
|
||||||
|
name: evobgp_traefik_letsencrypt
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# Docker-образы EvoBGP
|
||||||
|
|
||||||
|
## CI (Gitea Actions)
|
||||||
|
|
||||||
|
Сборка образов — **`docker buildx bake`** (`deploy/docker/docker-bake.hcl`), не отдельные `docker build`.
|
||||||
|
|
||||||
|
**Publish:** push в `main` после quality gates — job **release** в [.gitea/workflows/ci.yaml](../.gitea/workflows/ci.yaml): semantic-release + bake с `VERSION` из релиза.
|
||||||
|
|
||||||
|
Локально BuildKit также кэширует `/go/pkg/mod` и `~/.cache/go-build` через `RUN --mount=type=cache`.
|
||||||
|
|
||||||
|
| Было | Стало |
|
||||||
|
|------|--------|
|
||||||
|
| 8× `go mod download` + 8× `go build` (разные BIN) | 1× download + 1× компиляция всех `cmd/*` |
|
||||||
|
| 2× сборка BIRD из исходников (api, all) | 1× stage `birdc`, копируется в runtime |
|
||||||
|
| 2× `npm ci` (web, web-all) | 1× `web-deps` + 1× `web-build` + два nginx-образа (общий артефакт) |
|
||||||
|
| 8 runner'ов с checkout/login | 1 job `docker-go`, 1 job `docker-web` |
|
||||||
|
|
||||||
|
Кэш registry (переменные bake):
|
||||||
|
|
||||||
|
- `git.shts.su/<owner>/evobgp-buildcache:go-buildcache` — **запись** только из target `go-build-all`
|
||||||
|
- `git.shts.su/<owner>/evobgp-buildcache:web-buildcache` — **запись** только из target `web-build`
|
||||||
|
|
||||||
|
Остальные bake-target’ы только `cache-from` (чтение). Параллельный `cache-to` в один ref ломает manifest в registry (`content descriptor … not found`).
|
||||||
|
|
||||||
|
Локально BuildKit также кэширует `/go/pkg/mod` и `~/.cache/go-build` через `RUN --mount=type=cache`.
|
||||||
|
|
||||||
|
Если CI падает на «not found» после смены схемы кэша — один раз удалите теги `evobgp-buildcache:go-buildcache` и `:web-buildcache` в registry и пересоберите.
|
||||||
|
|
||||||
|
## Локальная сборка
|
||||||
|
|
||||||
|
Из корня репозитория:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd deploy/docker
|
||||||
|
export REGISTRY=git.shts.su/<owner>
|
||||||
|
export IMAGE_TAG=latest
|
||||||
|
export SHORT_SHA=$(git rev-parse --short HEAD)
|
||||||
|
export VERSION=dev
|
||||||
|
export BUILD_TIME=
|
||||||
|
sh write-bake-override.sh
|
||||||
|
docker buildx bake --allow=fs.read=../.. -f docker-bake.hcl -f docker-bake.override.hcl go-images
|
||||||
|
docker buildx bake --allow=fs.read=../.. -f docker-bake.hcl -f docker-bake.override.hcl web-images
|
||||||
|
```
|
||||||
|
|
||||||
|
В `docker-bake.hcl`: `context = "../.."` (корень репо), `dockerfile = "deploy/docker/…"` (путь от корня репо). **CI:** `write-bake-override.sh` + `--allow=fs.read=$GITHUB_WORKSPACE` в `.gitea/workflows/ci.yaml`.
|
||||||
|
|
||||||
|
Один образ (legacy):
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker build -f deploy/docker/gobinary/Dockerfile \
|
||||||
|
--target build-all \
|
||||||
|
--build-arg BIN=evobgp-api \
|
||||||
|
-t evobgp-api:local .
|
||||||
|
```
|
||||||
|
|
||||||
|
Runtime-образы в bake ожидают stage `build-all` (через bake contexts), не отдельный `--build-arg BIN` на старый target `build`.
|
||||||
@@ -1,6 +1,7 @@
|
|||||||
#!/bin/sh
|
#!/bin/sh
|
||||||
# Сборка BIRD из официального tarball (версия задаётся BIRD_VERSION, по умолчанию 2.14).
|
# Сборка BIRD из официального tarball (версия задаётся BIRD_VERSION, по умолчанию 2.14).
|
||||||
# Запуск от root (Docker) или через sudo на хосте. Префикс установки: /usr/local.
|
# Запуск от root (Docker) или через sudo на хосте. Префикс установки: /usr/local.
|
||||||
|
# birdc требует ncurses + readline; в runtime-образе gobinary — libncurses6 + libreadline8.
|
||||||
set -eu
|
set -eu
|
||||||
BIRD_VERSION="${BIRD_VERSION:-2.14}"
|
BIRD_VERSION="${BIRD_VERSION:-2.14}"
|
||||||
export DEBIAN_FRONTEND=noninteractive
|
export DEBIAN_FRONTEND=noninteractive
|
||||||
@@ -27,5 +28,4 @@ rm -rf "/tmp/bird-${BIRD_VERSION}"
|
|||||||
|
|
||||||
apt-get purge -y build-essential flex bison libncurses-dev libreadline-dev
|
apt-get purge -y build-essential flex bison libncurses-dev libreadline-dev
|
||||||
apt-get autoremove -y
|
apt-get autoremove -y
|
||||||
apt-get install -y --no-install-recommends libreadline8 libtinfo6
|
|
||||||
rm -rf /var/lib/apt/lists/*
|
rm -rf /var/lib/apt/lists/*
|
||||||
@@ -0,0 +1,265 @@
|
|||||||
|
# Единая сборка образов EvoBGP (buildx bake: -f deploy/docker/docker-bake.hcl).
|
||||||
|
# context = "../.." — корень репозитория (относительно этого файла).
|
||||||
|
# dockerfile — путь от корня репозитория (относительно context).
|
||||||
|
# Переменные: REGISTRY, IMAGE_TAG, CACHE_REF_GO, CACHE_REF_WEB
|
||||||
|
|
||||||
|
variable "REGISTRY" {
|
||||||
|
default = "git.shts.su/evobgp"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "IMAGE_TAG" {
|
||||||
|
default = "latest"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "SHORT_SHA" {
|
||||||
|
default = "dev"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "SHA_FULL" {
|
||||||
|
default = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "VERSION" {
|
||||||
|
default = "dev"
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "BUILD_TIME" {
|
||||||
|
default = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "CACHE_REF_GO" {
|
||||||
|
default = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
variable "CACHE_REF_WEB" {
|
||||||
|
default = ""
|
||||||
|
}
|
||||||
|
|
||||||
|
function "go-cache-from" {
|
||||||
|
params = []
|
||||||
|
result = notequal("", CACHE_REF_GO) ? ["type=registry,ref=${CACHE_REF_GO}"] : []
|
||||||
|
}
|
||||||
|
|
||||||
|
# Экспорт кэша — только go-build-all / web-build (один writer на ref).
|
||||||
|
# Несколько target с cache-to в один ref и mode=max дают гонку в registry
|
||||||
|
# (content descriptor not found при параллельном bake).
|
||||||
|
function "go-cache-to-export" {
|
||||||
|
params = []
|
||||||
|
result = notequal("", CACHE_REF_GO) ? ["type=registry,ref=${CACHE_REF_GO},mode=max"] : []
|
||||||
|
}
|
||||||
|
|
||||||
|
function "web-cache-from" {
|
||||||
|
params = []
|
||||||
|
result = notequal("", CACHE_REF_WEB) ? ["type=registry,ref=${CACHE_REF_WEB}"] : []
|
||||||
|
}
|
||||||
|
|
||||||
|
function "web-cache-to-export" {
|
||||||
|
params = []
|
||||||
|
result = notequal("", CACHE_REF_WEB) ? ["type=registry,ref=${CACHE_REF_WEB},mode=max"] : []
|
||||||
|
}
|
||||||
|
|
||||||
|
group "default" {
|
||||||
|
targets = ["go-images", "web-images", "evobgp-bird2"]
|
||||||
|
}
|
||||||
|
|
||||||
|
group "go-images" {
|
||||||
|
targets = [
|
||||||
|
"evobgp-api",
|
||||||
|
"evobgp-all",
|
||||||
|
"evobgp-scheduler",
|
||||||
|
"evobgp-ingest",
|
||||||
|
"evobgp-render",
|
||||||
|
"evobgp-deploy",
|
||||||
|
"evobgp-node",
|
||||||
|
"evobgp-agent",
|
||||||
|
]
|
||||||
|
}
|
||||||
|
|
||||||
|
# web-deps / web-build — только зависимости (contexts), без tags; при --push в группе не указывать.
|
||||||
|
group "web-images" {
|
||||||
|
targets = ["evobgp-web", "evobgp-web-all"]
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- Go: go mod download → все cmd/* → birdc (один раз) → runtime-образы ---
|
||||||
|
|
||||||
|
target "go-deps" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/gobinary/Dockerfile"
|
||||||
|
target = "deps"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
cache-from = go-cache-from()
|
||||||
|
}
|
||||||
|
|
||||||
|
target "go-build-all" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/gobinary/Dockerfile"
|
||||||
|
target = "build-all"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
args = {
|
||||||
|
VERSION = VERSION
|
||||||
|
GIT_SHA = notequal("", SHA_FULL) ? SHA_FULL : SHORT_SHA
|
||||||
|
BUILD_TIME = BUILD_TIME
|
||||||
|
}
|
||||||
|
contexts = {
|
||||||
|
deps = "target:go-deps"
|
||||||
|
}
|
||||||
|
cache-from = go-cache-from()
|
||||||
|
cache-to = go-cache-to-export()
|
||||||
|
}
|
||||||
|
|
||||||
|
target "go-birdc" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/gobinary/Dockerfile"
|
||||||
|
target = "birdc"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
cache-from = go-cache-from()
|
||||||
|
}
|
||||||
|
|
||||||
|
target "_go-runtime" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/gobinary/Dockerfile"
|
||||||
|
target = "runtime"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
contexts = {
|
||||||
|
build-all = "target:go-build-all"
|
||||||
|
}
|
||||||
|
cache-from = go-cache-from()
|
||||||
|
}
|
||||||
|
|
||||||
|
target "_go-runtime-birdc" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/gobinary/Dockerfile"
|
||||||
|
target = "runtime-birdc"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
contexts = {
|
||||||
|
build-all = "target:go-build-all"
|
||||||
|
birdc = "target:go-birdc"
|
||||||
|
}
|
||||||
|
cache-from = go-cache-from()
|
||||||
|
}
|
||||||
|
|
||||||
|
function "image-tags" {
|
||||||
|
params = [name]
|
||||||
|
result = concat(
|
||||||
|
notequal("", SHA_FULL) ? [
|
||||||
|
"${REGISTRY}/${name}:${IMAGE_TAG}",
|
||||||
|
"${REGISTRY}/${name}:${SHORT_SHA}",
|
||||||
|
"${REGISTRY}/${name}:sha-${SHA_FULL}",
|
||||||
|
] : [
|
||||||
|
"${REGISTRY}/${name}:${IMAGE_TAG}",
|
||||||
|
"${REGISTRY}/${name}:${SHORT_SHA}",
|
||||||
|
],
|
||||||
|
notequal(VERSION, "dev") && notequal(VERSION, "") ? [
|
||||||
|
"${REGISTRY}/${name}:v${VERSION}",
|
||||||
|
"${REGISTRY}/${name}:${VERSION}",
|
||||||
|
] : []
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-api" {
|
||||||
|
inherits = ["_go-runtime-birdc"]
|
||||||
|
args = { BIN = "evobgp-api" }
|
||||||
|
tags = image-tags("evobgp-api")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-all" {
|
||||||
|
inherits = ["_go-runtime-birdc"]
|
||||||
|
args = { BIN = "evobgp-all" }
|
||||||
|
tags = image-tags("evobgp-all")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-scheduler" {
|
||||||
|
inherits = ["_go-runtime"]
|
||||||
|
args = { BIN = "evobgp-scheduler" }
|
||||||
|
tags = image-tags("evobgp-scheduler")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-ingest" {
|
||||||
|
inherits = ["_go-runtime"]
|
||||||
|
args = { BIN = "evobgp-ingest" }
|
||||||
|
tags = image-tags("evobgp-ingest")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-render" {
|
||||||
|
inherits = ["_go-runtime"]
|
||||||
|
args = { BIN = "evobgp-render" }
|
||||||
|
tags = image-tags("evobgp-render")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-deploy" {
|
||||||
|
inherits = ["_go-runtime"]
|
||||||
|
args = { BIN = "evobgp-deploy" }
|
||||||
|
tags = image-tags("evobgp-deploy")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-node" {
|
||||||
|
inherits = ["_go-runtime"]
|
||||||
|
args = { BIN = "evobgp-node" }
|
||||||
|
tags = image-tags("evobgp-node")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-agent" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/evobgp-agent/Dockerfile"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
contexts = {
|
||||||
|
build-all = "target:go-build-all"
|
||||||
|
}
|
||||||
|
cache-from = go-cache-from()
|
||||||
|
tags = image-tags("evobgp-agent")
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- Web: npm ci (кэш) → build → nginx ---
|
||||||
|
|
||||||
|
target "web-deps" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/evobgp-web/Dockerfile"
|
||||||
|
target = "deps"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
cache-from = web-cache-from()
|
||||||
|
}
|
||||||
|
|
||||||
|
target "web-build" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/evobgp-web/Dockerfile"
|
||||||
|
target = "build"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
contexts = {
|
||||||
|
deps = "target:web-deps"
|
||||||
|
}
|
||||||
|
cache-from = web-cache-from()
|
||||||
|
cache-to = web-cache-to-export()
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-web" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/evobgp-web/Dockerfile"
|
||||||
|
target = "web"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
contexts = {
|
||||||
|
web-artifacts = "target:web-build"
|
||||||
|
}
|
||||||
|
args = { EVOBGP_UPSTREAM = "evobgp-api" }
|
||||||
|
cache-from = web-cache-from()
|
||||||
|
tags = image-tags("evobgp-web")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-web-all" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/evobgp-web/Dockerfile"
|
||||||
|
target = "web"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
contexts = {
|
||||||
|
web-artifacts = "target:web-build"
|
||||||
|
}
|
||||||
|
args = { EVOBGP_UPSTREAM = "evobgp-all" }
|
||||||
|
cache-from = web-cache-from()
|
||||||
|
tags = image-tags("evobgp-web-all")
|
||||||
|
}
|
||||||
|
|
||||||
|
target "evobgp-bird2" {
|
||||||
|
context = "../.."
|
||||||
|
dockerfile = "deploy/docker/bird2/Dockerfile"
|
||||||
|
platforms = ["linux/amd64"]
|
||||||
|
tags = image-tags("evobgp-bird2")
|
||||||
|
}
|
||||||
@@ -1,15 +1,10 @@
|
|||||||
# evobgp-agent + bird2 из репозитория Ubuntu Noble (тот же стек, что evobgp-bird2).
|
# syntax=docker/dockerfile:1.7
|
||||||
# См. gobinary/Dockerfile — ECR Public вместо прямого pull с Docker Hub.
|
# Агент: бинарь из общего build-all (docker-bake.hcl → contexts.build-all), bird2 из apt.
|
||||||
FROM public.ecr.aws/docker/library/golang:1.22-bookworm AS build
|
|
||||||
WORKDIR /src
|
|
||||||
COPY go.mod go.sum ./
|
|
||||||
COPY . .
|
|
||||||
RUN go build -trimpath -ldflags="-s -w" -o /out/evobgp-agent ./cmd/evobgp-agent
|
|
||||||
|
|
||||||
FROM public.ecr.aws/docker/library/ubuntu:noble
|
FROM public.ecr.aws/docker/library/ubuntu:noble
|
||||||
RUN apt-get update \
|
RUN apt-get update \
|
||||||
&& apt-get install -y --no-install-recommends bird2 ca-certificates \
|
&& apt-get install -y --no-install-recommends bird2 ca-certificates \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
COPY --from=build /out/evobgp-agent /usr/local/bin/evobgp-agent
|
ARG BIN=evobgp-agent
|
||||||
|
COPY --from=build-all /out/${BIN} /usr/local/bin/evobgp-agent
|
||||||
WORKDIR /etc/bird
|
WORKDIR /etc/bird
|
||||||
ENTRYPOINT ["/usr/local/bin/evobgp-agent"]
|
ENTRYPOINT ["/usr/local/bin/evobgp-agent"]
|
||||||
@@ -1,14 +1,19 @@
|
|||||||
# Статическая панель EvoBGP (SvelteKit) + nginx как reverse-proxy к evobgp-api (/v1, /metrics).
|
# syntax=docker/dockerfile:1.7
|
||||||
# ECR Public — то же содержимое, что library/node и library/nginx на Docker Hub.
|
# Статическая панель EvoBGP (SvelteKit) + nginx.
|
||||||
FROM public.ecr.aws/docker/library/node:22-alpine AS build
|
# Финальный stage `web` ожидает bake-контекст web-artifacts (= target:web-build).
|
||||||
|
FROM public.ecr.aws/docker/library/node:22-alpine AS deps
|
||||||
WORKDIR /web
|
WORKDIR /web
|
||||||
COPY web/package.json web/package-lock.json ./
|
COPY web/package.json web/package-lock.json ./
|
||||||
RUN npm ci
|
RUN --mount=type=cache,target=/root/.npm,sharing=locked \
|
||||||
|
npm ci
|
||||||
|
|
||||||
|
FROM deps AS build
|
||||||
COPY web/ ./
|
COPY web/ ./
|
||||||
RUN npm run build
|
RUN npm run build
|
||||||
|
|
||||||
FROM public.ecr.aws/docker/library/nginx:1.27-alpine
|
FROM public.ecr.aws/docker/library/nginx:1.27-alpine AS web
|
||||||
ARG EVOBGP_UPSTREAM=evobgp-api
|
ARG EVOBGP_UPSTREAM=evobgp-api
|
||||||
COPY deploy/docker/evobgp-web/nginx.conf /tmp/nginx-default.conf
|
COPY deploy/docker/evobgp-web/nginx.conf /tmp/nginx-default.conf
|
||||||
RUN sed -e "s/evobgp-api/${EVOBGP_UPSTREAM}/g" /tmp/nginx-default.conf > /etc/nginx/conf.d/default.conf && rm -f /tmp/nginx-default.conf
|
RUN sed -e "s/evobgp-api/${EVOBGP_UPSTREAM}/g" /tmp/nginx-default.conf > /etc/nginx/conf.d/default.conf \
|
||||||
COPY --from=build /web/build /usr/share/nginx/html
|
&& rm -f /tmp/nginx-default.conf
|
||||||
|
COPY --from=web-artifacts /web/build /usr/share/nginx/html
|
||||||
@@ -1,27 +1,69 @@
|
|||||||
# Универсальная сборка бинаря из cmd/* (ARG BIN=evobgp-api | evobgp-all).
|
# syntax=docker/dockerfile:1.7
|
||||||
# INSTALL_BIRDC=1 собирает BIRD 2.14 из исходников (birdc) для EVOBGP_BIRDC_SOCKET; иначе пакет Debian не используется.
|
# Универсальная сборка бинарей cmd/* (ARG BIN) или всех сразу (target build-all).
|
||||||
# Базовые образы из ECR Public (официальное зеркало library/*), чтобы CI не зависел от auth.docker.io.
|
# INSTALL_BIRDC=1 — birdc из stage birdc (собирается один раз, переиспользуется api/all).
|
||||||
FROM public.ecr.aws/docker/library/golang:1.22-bookworm AS build
|
# CI: docker buildx bake -f deploy/docker/docker-bake.hcl
|
||||||
|
FROM public.ecr.aws/docker/library/golang:1.24-bookworm AS deps
|
||||||
WORKDIR /src
|
WORKDIR /src
|
||||||
COPY go.mod go.sum ./
|
COPY go.mod go.sum ./
|
||||||
RUN go mod download
|
RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \
|
||||||
|
go mod download
|
||||||
|
|
||||||
|
FROM deps AS build-all
|
||||||
|
COPY . .
|
||||||
|
ARG VERSION=dev
|
||||||
|
ARG GIT_SHA=unknown
|
||||||
|
ARG BUILD_TIME=
|
||||||
|
RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \
|
||||||
|
--mount=type=cache,target=/root/.cache/go-build,sharing=locked \
|
||||||
|
set -eux; \
|
||||||
|
mkdir -p /out; \
|
||||||
|
for d in cmd/*/; do \
|
||||||
|
name="$(basename "$d")"; \
|
||||||
|
CGO_ENABLED=0 go build -trimpath \
|
||||||
|
-ldflags="-s -w \
|
||||||
|
-X evobgp/internal/version.Version=${VERSION} \
|
||||||
|
-X evobgp/internal/version.GitSHA=${GIT_SHA} \
|
||||||
|
-X evobgp/internal/version.BuildTime=${BUILD_TIME}" \
|
||||||
|
-o "/out/${name}" "./cmd/${name}"; \
|
||||||
|
done
|
||||||
|
|
||||||
|
# Один бинарь (локальная сборка); в CI — build-all + runtime.
|
||||||
|
FROM deps AS build
|
||||||
COPY . .
|
COPY . .
|
||||||
ARG BIN=evobgp-api
|
ARG BIN=evobgp-api
|
||||||
RUN CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/evobgp ./cmd/${BIN}
|
ARG VERSION=dev
|
||||||
|
ARG GIT_SHA=unknown
|
||||||
|
ARG BUILD_TIME=
|
||||||
|
RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \
|
||||||
|
--mount=type=cache,target=/root/.cache/go-build,sharing=locked \
|
||||||
|
CGO_ENABLED=0 go build -trimpath \
|
||||||
|
-ldflags="-s -w \
|
||||||
|
-X evobgp/internal/version.Version=${VERSION} \
|
||||||
|
-X evobgp/internal/version.GitSHA=${GIT_SHA} \
|
||||||
|
-X evobgp/internal/version.BuildTime=${BUILD_TIME}" \
|
||||||
|
-o /out/evobgp "./cmd/${BIN}"
|
||||||
|
|
||||||
FROM public.ecr.aws/docker/library/debian:bookworm-slim
|
FROM public.ecr.aws/docker/library/debian:bookworm-slim AS birdc
|
||||||
ARG INSTALL_BIRDC=0
|
|
||||||
ARG BIRD_VERSION=2.14
|
ARG BIRD_VERSION=2.14
|
||||||
COPY deploy/docker/bird/bird-from-source.sh /tmp/bird-from-source.sh
|
COPY deploy/docker/bird/bird-from-source.sh /tmp/bird-from-source.sh
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends ca-certificates \
|
RUN chmod +x /tmp/bird-from-source.sh \
|
||||||
&& if [ "$INSTALL_BIRDC" = "1" ]; then \
|
|
||||||
chmod +x /tmp/bird-from-source.sh \
|
|
||||||
&& BIRD_VERSION="${BIRD_VERSION}" /tmp/bird-from-source.sh \
|
&& BIRD_VERSION="${BIRD_VERSION}" /tmp/bird-from-source.sh \
|
||||||
&& rm -f /tmp/bird-from-source.sh; \
|
&& rm -f /tmp/bird-from-source.sh
|
||||||
else \
|
|
||||||
rm -f /tmp/bird-from-source.sh; \
|
FROM public.ecr.aws/docker/library/debian:bookworm-slim AS runtime-base
|
||||||
fi \
|
RUN apt-get update \
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
&& apt-get install -y --no-install-recommends ca-certificates \
|
||||||
COPY --from=build /out/evobgp /usr/local/bin/evobgp
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
FROM runtime-base AS runtime
|
||||||
|
ARG BIN=evobgp-api
|
||||||
|
COPY --from=build-all /out/${BIN} /usr/local/bin/evobgp
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
ENTRYPOINT ["/usr/local/bin/evobgp"]
|
ENTRYPOINT ["/usr/local/bin/evobgp"]
|
||||||
|
|
||||||
|
FROM runtime AS runtime-birdc
|
||||||
|
# birdc: динамическая линковка readline + ncurses (debian bookworm).
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends libreadline8 libncurses6 \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
COPY --from=birdc /usr/local/sbin/bird /usr/local/sbin/birdc /usr/local/sbin/
|
||||||
@@ -0,0 +1,39 @@
|
|||||||
|
#!/usr/bin/env sh
|
||||||
|
# Writes docker-bake.override.hcl for CI (buildx without --var support).
|
||||||
|
set -eu
|
||||||
|
OUT="${1:-docker-bake.override.hcl}"
|
||||||
|
REGISTRY="${REGISTRY:?REGISTRY required}"
|
||||||
|
IMAGE_TAG="${IMAGE_TAG:-latest}"
|
||||||
|
SHORT_SHA="${SHORT_SHA:-dev}"
|
||||||
|
SHA_FULL="${SHA_FULL:-}"
|
||||||
|
VERSION="${VERSION:-dev}"
|
||||||
|
BUILD_TIME="${BUILD_TIME:-}"
|
||||||
|
CACHE_REF_GO="${CACHE_REF_GO:-}"
|
||||||
|
CACHE_REF_WEB="${CACHE_REF_WEB:-}"
|
||||||
|
cat >"$OUT" <<EOF
|
||||||
|
# Generated by deploy/docker/write-bake-override.sh — do not commit.
|
||||||
|
variable "REGISTRY" {
|
||||||
|
default = "${REGISTRY}"
|
||||||
|
}
|
||||||
|
variable "IMAGE_TAG" {
|
||||||
|
default = "${IMAGE_TAG}"
|
||||||
|
}
|
||||||
|
variable "SHORT_SHA" {
|
||||||
|
default = "${SHORT_SHA}"
|
||||||
|
}
|
||||||
|
variable "SHA_FULL" {
|
||||||
|
default = "${SHA_FULL}"
|
||||||
|
}
|
||||||
|
variable "VERSION" {
|
||||||
|
default = "${VERSION}"
|
||||||
|
}
|
||||||
|
variable "BUILD_TIME" {
|
||||||
|
default = "${BUILD_TIME}"
|
||||||
|
}
|
||||||
|
variable "CACHE_REF_GO" {
|
||||||
|
default = "${CACHE_REF_GO}"
|
||||||
|
}
|
||||||
|
variable "CACHE_REF_WEB" {
|
||||||
|
default = "${CACHE_REF_WEB}"
|
||||||
|
}
|
||||||
|
EOF
|
||||||
@@ -4,6 +4,7 @@
|
|||||||
|
|
||||||
## По роли читателя
|
## По роли читателя
|
||||||
|
|
||||||
|
- **ИИ-агент / ассистент в репозитории** — [../AGENTS.md](../AGENTS.md): с чего начать чтение, карта `internal/` и `cmd/`, что не тащить в контекст. **Инженерные правила:** [../.cursor/rules/engineering.mdc](../.cursor/rules/engineering.mdc) (общие), [../.cursor/rules/web-shadcn.mdc](../.cursor/rules/web-shadcn.mdc) (Web UI), [../.cursor/rules/networking-bird.mdc](../.cursor/rules/networking-bird.mdc) (BIRD2/BGP/IP).
|
||||||
- **Оператор / DevOps** — [quickstart.md](quickstart.md), [architecture.md](architecture.md), [access.md](access.md), [deploy/compose/docker-compose.yaml](../deploy/compose/docker-compose.yaml).
|
- **Оператор / DevOps** — [quickstart.md](quickstart.md), [architecture.md](architecture.md), [access.md](access.md), [deploy/compose/docker-compose.yaml](../deploy/compose/docker-compose.yaml).
|
||||||
- **Разработчик бэкенда или интегратор API** — [api.md](api.md), [access.md](access.md), [openapi.yaml](openapi.yaml), исходники маршрутов в `internal/httpapi/`.
|
- **Разработчик бэкенда или интегратор API** — [api.md](api.md), [access.md](access.md), [openapi.yaml](openapi.yaml), исходники маршрутов в `internal/httpapi/`.
|
||||||
- **Разработчик фронтенда** — [quickstart.md](quickstart.md) (раздел про `web/` и CORS), [api.md](api.md), [../web/README.md](../web/README.md).
|
- **Разработчик фронтенда** — [quickstart.md](quickstart.md) (раздел про `web/` и CORS), [api.md](api.md), [../web/README.md](../web/README.md).
|
||||||
@@ -15,12 +16,38 @@
|
|||||||
| [overview.md](overview.md) | Ключевые возможности системы |
|
| [overview.md](overview.md) | Ключевые возможности системы |
|
||||||
| [quickstart.md](quickstart.md) | Быстрый запуск: готовые образы из `git.shts.su`, Docker Compose, локальный Go, веб |
|
| [quickstart.md](quickstart.md) | Быстрый запуск: готовые образы из `git.shts.su`, Docker Compose, локальный Go, веб |
|
||||||
| [architecture.md](architecture.md) | Компоненты, потоки данных, пакеты |
|
| [architecture.md](architecture.md) | Компоненты, потоки данных, пакеты |
|
||||||
|
| [manual.md](manual.md) | Подробное руководство по модулям, процессам и API |
|
||||||
| [api.md](api.md) | REST: префикс `/v1`, публичные маршруты, ссылки на OpenAPI |
|
| [api.md](api.md) | REST: префикс `/v1`, публичные маршруты, ссылки на OpenAPI |
|
||||||
|
| [router-lists-ui-integration.md](router-lists-ui-integration.md) | Интеграция `router-lists-ui` с EvoBGP API (`DOMAINS/IP_RANGES/AS_PREFIXES/communities`) |
|
||||||
| [access.md](access.md) | Выдача доступа: API-ключи, роли, нода, CORS |
|
| [access.md](access.md) | Выдача доступа: API-ключи, роли, нода, CORS |
|
||||||
|
| [releasing.md](releasing.md) | Автоматические релизы, Conventional Commits, CI |
|
||||||
| [openapi.yaml](openapi.yaml) | Источник правды по контракту API |
|
| [openapi.yaml](openapi.yaml) | Источник правды по контракту API |
|
||||||
| [OPENAPI-GITEA.md](OPENAPI-GITEA.md) | Как открыть HTML-документацию API (в т.ч. из Gitea) |
|
| [OPENAPI-GITEA.md](OPENAPI-GITEA.md) | Как открыть HTML-документацию API (в т.ч. из Gitea) |
|
||||||
| [evobgp-api-sketches.md](evobgp-api-sketches.md) | Ранний черновик идей API (контекст, не замена OpenAPI) |
|
| [evobgp-api-sketches.md](evobgp-api-sketches.md) | Ранний черновик идей API (контекст, не замена OpenAPI) |
|
||||||
|
|
||||||
|
## Инженерные правила (Cursor / разработка)
|
||||||
|
|
||||||
|
| Файл | Область |
|
||||||
|
|------|---------|
|
||||||
|
| [../.cursor/rules/engineering.mdc](../.cursor/rules/engineering.mdc) | Go, API, migrations, security, enforcement |
|
||||||
|
| [../.cursor/rules/web-shadcn.mdc](../.cursor/rules/web-shadcn.mdc) | SvelteKit, shadcn-svelte |
|
||||||
|
| [../.cursor/rules/networking-bird.mdc](../.cursor/rules/networking-bird.mdc) | BIRD2, BGP policy, IP/CIDR |
|
||||||
|
| [../.cursor/rules/conventional-commits.mdc](../.cursor/rules/conventional-commits.mdc) | Conventional Commits (заголовок EN, тело RU) |
|
||||||
|
|
||||||
|
### Сообщения коммитов (Cursor)
|
||||||
|
|
||||||
|
После `git add`:
|
||||||
|
|
||||||
|
| Способ | Skill + rule + `staged-context.ps1` |
|
||||||
|
|--------|-------------------------------------|
|
||||||
|
| Agent → **`/commit-message`** или «сгенерируй коммит по staged» | **Да** |
|
||||||
|
| Кнопка **✨ Generate commit message** в Source Control | **Нет** (только diff + история; [ограничение Cursor](https://forum.cursor.com/t/how-to-set-prompt-for-generate-commit-message/148606)) |
|
||||||
|
|
||||||
|
Рекомендуемый workflow: Agent Chat → **`/commit-message`** ([команда](../.cursor/commands/commit-message.md), [скилл](../.cursor/skills/commit-message/SKILL.md), [правило](../.cursor/rules/conventional-commits.mdc)). Агент **обязан первым делом** запустить `scripts/commit/staged-context.ps1` (заголовок EN, тело RU, auto-split).
|
||||||
|
|
||||||
|
Просмотр групп вручную: `powershell -NoProfile -File scripts/commit/staged-context.ps1 | ConvertFrom-Json`.
|
||||||
|
|
||||||
## Репозиторий и CI
|
## Репозиторий и CI
|
||||||
|
|
||||||
- [../.gitea/README.md](../.gitea/README.md) — Gitea Actions, runner, сборка образов и push в Container Registry.
|
- [../.gitea/README.md](../.gitea/README.md) — Gitea Actions, runner, сборка образов и push в Container Registry.
|
||||||
|
- [releasing.md](releasing.md) — semantic-release, commit conventions, теги образов.
|
||||||
@@ -48,6 +48,7 @@
|
|||||||
|
|
||||||
- `GET /v1/peers`, `POST /v1/peers`
|
- `GET /v1/peers`, `POST /v1/peers`
|
||||||
- `GET|PATCH|DELETE /v1/peers/{id}`
|
- `GET|PATCH|DELETE /v1/peers/{id}`
|
||||||
|
- Для `POST|PATCH|DELETE` peer запускается быстрый job `peer_reconcile` (без module ingest/сбора префиксов); после него автоматически ставится apply на спикеры.
|
||||||
|
|
||||||
### Speakers
|
### Speakers
|
||||||
|
|
||||||
|
|||||||
+142
@@ -0,0 +1,142 @@
|
|||||||
|
# Полное руководство по EvoBGP
|
||||||
|
|
||||||
|
Этот документ объединяет эксплуатационное и разработческое описание системы: что делает каждый процесс, как двигаются данные, какие API использовать и где искать причины инцидентов.
|
||||||
|
|
||||||
|
## 1. Назначение системы
|
||||||
|
|
||||||
|
EvoBGP управляет генерацией и применением BGP-конфигураций на основе модулей источников префиксов (`AS_PREFIXES`, `CDN_CIDRS`, `DOMAINS`, `IP_RANGES`).
|
||||||
|
|
||||||
|
Система разделена на:
|
||||||
|
- **control plane**: API, БД, jobs, рендер ревизий, публикация и подписание бандлов;
|
||||||
|
- **data plane**: BIRD и связанный агент/нода для применения ревизий.
|
||||||
|
|
||||||
|
## 2. Компоненты и роли бинарников (`cmd/*`)
|
||||||
|
|
||||||
|
### `evobgp-api`
|
||||||
|
- Основной HTTP API.
|
||||||
|
- Поднимает маршруты из `internal/httpapi`.
|
||||||
|
- Работает с `store`/`repository`, jobs и аутентификацией.
|
||||||
|
|
||||||
|
### `evobgp-all`
|
||||||
|
- Монолитный режим: API + scheduler + ingest + render + deploy в одном процессе.
|
||||||
|
- Удобен для компактных окружений (`microvps`).
|
||||||
|
|
||||||
|
### `evobgp-scheduler`
|
||||||
|
- Периодически запускает `module_refresh` по расписанию/интервалам.
|
||||||
|
- В reference-профиле может стучаться в API и/или работать через store.
|
||||||
|
|
||||||
|
### `evobgp-ingest`
|
||||||
|
- Периодически делает prefetch внешних CDN-источников (ETag/доступность).
|
||||||
|
|
||||||
|
### `evobgp-render`
|
||||||
|
- Ведёт рендер-цикл; в режиме autopublish может назначать последнюю ревизию на спикеры.
|
||||||
|
|
||||||
|
### `evobgp-deploy`
|
||||||
|
- Диагностирует drift: различия между опубликованной и применённой ревизией.
|
||||||
|
|
||||||
|
### `evobgp-node`
|
||||||
|
- CLI-нода для edge: загрузка бандла, верификация подписи, применение.
|
||||||
|
|
||||||
|
### `evobgp-agent`
|
||||||
|
- Локальный агент рядом с BIRD (наблюдение и служебные операции).
|
||||||
|
|
||||||
|
## 3. Карта внутренних модулей (`internal/*`)
|
||||||
|
|
||||||
|
### API и доступ
|
||||||
|
- `internal/httpapi`: маршруты, auth, CORS, problem+json, CRUD, jobs endpoints.
|
||||||
|
|
||||||
|
### Данные
|
||||||
|
- `internal/store`: бизнес-контракты бэкенда.
|
||||||
|
- `internal/repository`: PostgreSQL-реализация.
|
||||||
|
- `internal/db`: коннект и миграции.
|
||||||
|
|
||||||
|
### Jobs/pipeline
|
||||||
|
- `internal/jobs`: очередь задач и worker.
|
||||||
|
- `internal/pipeline`: `module_refresh`, сбор источников, материализация, рендер-превью.
|
||||||
|
- `internal/scheduler`, `internal/ingest`, `internal/render`, `internal/deploy`: фоновые циклы.
|
||||||
|
|
||||||
|
### BIRD и бандлы
|
||||||
|
- `internal/birdfmt`: генерация конфигурации BIRD.
|
||||||
|
- `internal/birddeploy`: применение конфигурации и интеграция с `birdc`.
|
||||||
|
- `internal/bundle`, `internal/signing`: упаковка и криптографическая проверка.
|
||||||
|
|
||||||
|
### Наблюдаемость и служебные
|
||||||
|
- `internal/observability`: метрики/middleware.
|
||||||
|
- `internal/asnresolve`: внешние резолвы ASN.
|
||||||
|
- `internal/broker`: задел под внешний брокер.
|
||||||
|
- `internal/config`, `internal/platform`: параметры среды и платформенные адаптеры.
|
||||||
|
|
||||||
|
## 4. Сквозной поток данных
|
||||||
|
|
||||||
|
1. Оператор меняет данные модуля (CRUD источников, peers/speakers, настройки).
|
||||||
|
2. Включённый модуль триггерит `module_refresh`.
|
||||||
|
3. `jobs.Worker` вызывает `pipeline.RefreshModule`.
|
||||||
|
4. Pipeline собирает префиксы всех enabled-модулей tenant, строит materialized snapshot.
|
||||||
|
5. Создаётся ревизия и BIRD preview.
|
||||||
|
6. По операциям deploy/apply ревизия применяется на спикере.
|
||||||
|
7. Нода получает бандл, проверяет подпись, применяет локально.
|
||||||
|
|
||||||
|
## 5. API: ключевые группы и сценарии
|
||||||
|
|
||||||
|
Источник истины контракта: `docs/openapi.yaml`.
|
||||||
|
|
||||||
|
### Основные группы endpoint-ов
|
||||||
|
- `Modules`: модули и их общие параметры.
|
||||||
|
- `AS Entries`, `CDN Sources`, `Domain Entries`, `IP Range Entries`: источники префиксов.
|
||||||
|
- `Peers`, `Speakers`: сетевая топология применения.
|
||||||
|
- `Revisions`, `Deploy`, `Jobs`: жизненный цикл ревизий и фоновых задач.
|
||||||
|
- `Settings`: глобальные KV-настройки.
|
||||||
|
- `Node`: edge-флоу бандлов/enrollment.
|
||||||
|
|
||||||
|
### Типовой сценарий оператора
|
||||||
|
1. Создать/обновить модуль и его источники.
|
||||||
|
2. Дождаться или инициировать refresh.
|
||||||
|
3. Проверить ревизию и diff.
|
||||||
|
4. Выполнить apply на целевой спикер.
|
||||||
|
5. Проверить health/monitoring/jobs.
|
||||||
|
|
||||||
|
## 6. Настройки и доступ
|
||||||
|
|
||||||
|
### Аутентификация/роли
|
||||||
|
- Роли и правила доступа: `docs/access.md`.
|
||||||
|
- Для мутаций критичных сущностей требуется `editor`/`operator`.
|
||||||
|
|
||||||
|
### Настройки (`/v1/settings`)
|
||||||
|
- KV c ключами BIRD и дополнительными feature flags.
|
||||||
|
- Ключевые параметры BIRD: `bird_router_id`, `bird_local_ipv4`, `bird_local_ipv6`, `bird_local_asn`, `bird_bgp_source_ipv4`, `bird_bgp_source_ipv6`.
|
||||||
|
|
||||||
|
## 7. Эксплуатация и runbook
|
||||||
|
|
||||||
|
### Что проверять при инцидентах
|
||||||
|
1. Статус API и БД.
|
||||||
|
2. Состояние jobs (`module_refresh`, `deploy_apply`), ошибки в job meta.
|
||||||
|
3. Состояние внешних источников (CDN/DoH/ASN).
|
||||||
|
4. Состояние BIRD и применённой ревизии на спикере.
|
||||||
|
|
||||||
|
### Частые причины проблем
|
||||||
|
- Невалидные данные источников (непарсящийся JSON/plaintext для CDN).
|
||||||
|
- Неполные настройки BIRD.
|
||||||
|
- Ошибки внешних upstream (RIPEstat/DoH/CDN).
|
||||||
|
- Расхождение published/applied ревизии.
|
||||||
|
|
||||||
|
## 8. Разработка и расширение
|
||||||
|
|
||||||
|
### Где вносить изменения
|
||||||
|
- Новый endpoint: `internal/httpapi` + обновление `docs/openapi.yaml`.
|
||||||
|
- Новая логика источника: `internal/pipeline` + соответствующие CRUD/store/repository.
|
||||||
|
- Новая операция UI: `web/src/routes/*` и `web/src/lib/components/*`.
|
||||||
|
|
||||||
|
### Рекомендации по качеству
|
||||||
|
- Для изменений API всегда обновлять `docs/openapi.yaml`.
|
||||||
|
- Для изменений UI держаться единого набора компонентов shadcn-svelte и Lucide.
|
||||||
|
- Для pipeline-изменений добавлять метрики стадии и явные trigger-метки job.
|
||||||
|
|
||||||
|
## 9. Индекс исходников (быстрый вход)
|
||||||
|
|
||||||
|
- Архитектура: `docs/architecture.md`
|
||||||
|
- API обзор: `docs/api.md`
|
||||||
|
- Контракт API: `docs/openapi.yaml`
|
||||||
|
- Доступ/роли: `docs/access.md`
|
||||||
|
- Web запуск: `web/README.md`
|
||||||
|
- Compose: `deploy/compose/docker-compose.yaml`
|
||||||
|
|
||||||
+327
-16
@@ -1,10 +1,12 @@
|
|||||||
openapi: 3.1.0
|
openapi: 3.1.0
|
||||||
info:
|
info:
|
||||||
title: EvoBGP Control Plane API
|
title: EvoBGP Control Plane API
|
||||||
version: 0.1.0
|
version: 1.0.0
|
||||||
description: |
|
description: |
|
||||||
REST API управления префиксами, модулями ingest, ревизиями конфигурации BIRD и задачами (async jobs).
|
REST API управления префиксами, модулями ingest, ревизиями конфигурации BIRD и задачами (async jobs).
|
||||||
|
|
||||||
|
**Актуальная semver-сборка:** `GET /version` или `GET /v1/version` (поле `version`; совпадает с git-тегом `vX.Y.Z`).
|
||||||
|
|
||||||
**Соглашения:** префикс `/v1`; идентификаторы - UUID v7 или ULID (строки); время - ISO 8601 UTC.
|
**Соглашения:** префикс `/v1`; идентификаторы - UUID v7 или ULID (строки); время - ISO 8601 UTC.
|
||||||
Ошибки - `application/problem+json` ([RFC 9457](https://www.rfc-editor.org/rfc/rfc9457)).
|
Ошибки - `application/problem+json` ([RFC 9457](https://www.rfc-editor.org/rfc/rfc9457)).
|
||||||
Пагинация списков - `cursor` + `limit`; ответ содержит `items`, `next_cursor`, `has_more`.
|
Пагинация списков - `cursor` + `limit`; ответ содержит `items`, `next_cursor`, `has_more`.
|
||||||
@@ -185,7 +187,9 @@ components:
|
|||||||
in: query
|
in: query
|
||||||
schema:
|
schema:
|
||||||
type: string
|
type: string
|
||||||
description: Фильтр по виду задачи; точный перечень расширяем.
|
description: >
|
||||||
|
Фильтр по виду задачи; точный перечень расширяем.
|
||||||
|
Основные значения: `module_refresh`, `peer_reconcile`, `deploy_apply`, `revision_rollback`, `bird_reload`.
|
||||||
|
|
||||||
responses:
|
responses:
|
||||||
Unauthorized:
|
Unauthorized:
|
||||||
@@ -293,6 +297,18 @@ components:
|
|||||||
- DOMAINS
|
- DOMAINS
|
||||||
- IP_RANGES
|
- IP_RANGES
|
||||||
|
|
||||||
|
DohResolverPolicy:
|
||||||
|
type: string
|
||||||
|
description: |
|
||||||
|
Политика резолва доменов через DoH при нескольких профилях.
|
||||||
|
`primary_only` — только первый профиль; `failover` — по порядку до первого успешного;
|
||||||
|
`union` — объединение A/AAAA со всех профилей.
|
||||||
|
enum:
|
||||||
|
- primary_only
|
||||||
|
- failover
|
||||||
|
- union
|
||||||
|
default: primary_only
|
||||||
|
|
||||||
JobStatus:
|
JobStatus:
|
||||||
type: string
|
type: string
|
||||||
description: Статус задачи; перечень может расширяться.
|
description: Статус задачи; перечень может расширяться.
|
||||||
@@ -330,6 +346,14 @@ components:
|
|||||||
type: integer
|
type: integer
|
||||||
doh_profile_id:
|
doh_profile_id:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
|
description: Первый DoH-профиль (legacy); предпочтительно `doh_profile_ids`.
|
||||||
|
doh_profile_ids:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: "#/components/schemas/ResourceId"
|
||||||
|
description: Упорядоченный список DoH-профилей для модулей `DOMAINS`.
|
||||||
|
doh_resolver_policy:
|
||||||
|
$ref: "#/components/schemas/DohResolverPolicy"
|
||||||
refresh_interval_sec:
|
refresh_interval_sec:
|
||||||
type: ["integer", "null"]
|
type: ["integer", "null"]
|
||||||
minimum: 0
|
minimum: 0
|
||||||
@@ -337,6 +361,10 @@ components:
|
|||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
default_community_id:
|
default_community_id:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
|
last_refreshed_at:
|
||||||
|
type: ["string", "null"]
|
||||||
|
format: date-time
|
||||||
|
description: Время последнего успешного обновления данных модуля.
|
||||||
additionalProperties: true
|
additionalProperties: true
|
||||||
|
|
||||||
ModuleCreate:
|
ModuleCreate:
|
||||||
@@ -357,8 +385,20 @@ components:
|
|||||||
default: 0
|
default: 0
|
||||||
doh_profile_id:
|
doh_profile_id:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
|
description: Первый DoH-профиль (legacy); предпочтительно `doh_profile_ids`.
|
||||||
|
doh_profile_ids:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: "#/components/schemas/ResourceId"
|
||||||
|
description: Упорядоченный список DoH-профилей для модулей `DOMAINS`.
|
||||||
|
doh_resolver_policy:
|
||||||
|
$ref: "#/components/schemas/DohResolverPolicy"
|
||||||
refresh_interval_sec:
|
refresh_interval_sec:
|
||||||
type: ["integer", "null"]
|
type: ["integer", "null"]
|
||||||
|
last_refreshed_at:
|
||||||
|
type: ["string", "null"]
|
||||||
|
format: date-time
|
||||||
|
description: Время последнего успешного обновления этого CDN-источника.
|
||||||
cron_expr:
|
cron_expr:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
default_community_id:
|
default_community_id:
|
||||||
@@ -375,6 +415,14 @@ components:
|
|||||||
type: integer
|
type: integer
|
||||||
doh_profile_id:
|
doh_profile_id:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
|
description: Первый DoH-профиль (legacy); предпочтительно `doh_profile_ids`.
|
||||||
|
doh_profile_ids:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: "#/components/schemas/ResourceId"
|
||||||
|
description: Упорядоченный список DoH-профилей для модулей `DOMAINS`.
|
||||||
|
doh_resolver_policy:
|
||||||
|
$ref: "#/components/schemas/DohResolverPolicy"
|
||||||
refresh_interval_sec:
|
refresh_interval_sec:
|
||||||
type: ["integer", "null"]
|
type: ["integer", "null"]
|
||||||
cron_expr:
|
cron_expr:
|
||||||
@@ -397,6 +445,9 @@ components:
|
|||||||
source_kind:
|
source_kind:
|
||||||
type: string
|
type: string
|
||||||
description: Формат скачанного списка / парсер.
|
description: Формат скачанного списка / парсер.
|
||||||
|
prefix_path:
|
||||||
|
type: string
|
||||||
|
description: Путь до поля с префиксами для source_kind=json (например data.items[].cidr).
|
||||||
community_id:
|
community_id:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
refresh_interval_sec:
|
refresh_interval_sec:
|
||||||
@@ -413,6 +464,8 @@ components:
|
|||||||
format: uri
|
format: uri
|
||||||
source_kind:
|
source_kind:
|
||||||
type: string
|
type: string
|
||||||
|
prefix_path:
|
||||||
|
type: string
|
||||||
community_id:
|
community_id:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
|
|
||||||
@@ -424,11 +477,40 @@ components:
|
|||||||
format: uri
|
format: uri
|
||||||
source_kind:
|
source_kind:
|
||||||
type: string
|
type: string
|
||||||
|
prefix_path:
|
||||||
|
type: string
|
||||||
community_id:
|
community_id:
|
||||||
type: ["string", "null"]
|
type: ["string", "null"]
|
||||||
refresh_interval_sec:
|
refresh_interval_sec:
|
||||||
type: ["integer", "null"]
|
type: ["integer", "null"]
|
||||||
|
|
||||||
|
CdnPreviewRequest:
|
||||||
|
type: object
|
||||||
|
required: [url, source_kind]
|
||||||
|
properties:
|
||||||
|
url:
|
||||||
|
type: string
|
||||||
|
format: uri
|
||||||
|
source_kind:
|
||||||
|
type: string
|
||||||
|
prefix_path:
|
||||||
|
type: string
|
||||||
|
|
||||||
|
CdnPreviewResponse:
|
||||||
|
type: object
|
||||||
|
required: [items, total, truncated, source_url]
|
||||||
|
properties:
|
||||||
|
items:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: string
|
||||||
|
total:
|
||||||
|
type: integer
|
||||||
|
truncated:
|
||||||
|
type: boolean
|
||||||
|
source_url:
|
||||||
|
type: string
|
||||||
|
|
||||||
AsEntry:
|
AsEntry:
|
||||||
type: object
|
type: object
|
||||||
required:
|
required:
|
||||||
@@ -526,6 +608,9 @@ components:
|
|||||||
properties:
|
properties:
|
||||||
id:
|
id:
|
||||||
$ref: "#/components/schemas/ResourceId"
|
$ref: "#/components/schemas/ResourceId"
|
||||||
|
name:
|
||||||
|
type: string
|
||||||
|
description: Отображаемое имя профиля (опционально).
|
||||||
url:
|
url:
|
||||||
type: string
|
type: string
|
||||||
format: uri
|
format: uri
|
||||||
@@ -541,6 +626,9 @@ components:
|
|||||||
required:
|
required:
|
||||||
- url
|
- url
|
||||||
properties:
|
properties:
|
||||||
|
name:
|
||||||
|
type: string
|
||||||
|
description: Отображаемое имя профиля (опционально).
|
||||||
url:
|
url:
|
||||||
type: string
|
type: string
|
||||||
format: uri
|
format: uri
|
||||||
@@ -556,10 +644,12 @@ components:
|
|||||||
properties:
|
properties:
|
||||||
id:
|
id:
|
||||||
$ref: "#/components/schemas/ResourceId"
|
$ref: "#/components/schemas/ResourceId"
|
||||||
name:
|
community:
|
||||||
type: string
|
type: string
|
||||||
kind:
|
description: Техническое значение BGP community (строка для BIRD, например 65001:120 или large JSON в value_json).
|
||||||
|
title:
|
||||||
type: string
|
type: string
|
||||||
|
description: Человекочитаемое название для UI и фильтров.
|
||||||
additionalProperties: true
|
additionalProperties: true
|
||||||
|
|
||||||
BgpPeer:
|
BgpPeer:
|
||||||
@@ -670,8 +760,14 @@ components:
|
|||||||
VersionInfo:
|
VersionInfo:
|
||||||
type: object
|
type: object
|
||||||
properties:
|
properties:
|
||||||
|
version:
|
||||||
|
type: string
|
||||||
|
description: Semver сборки (git tag без префикса v).
|
||||||
|
example: "1.2.3"
|
||||||
api_version:
|
api_version:
|
||||||
type: string
|
type: string
|
||||||
|
deprecated: true
|
||||||
|
description: Alias поля `version` (сохранён для обратной совместимости).
|
||||||
git_sha:
|
git_sha:
|
||||||
type: string
|
type: string
|
||||||
build_time:
|
build_time:
|
||||||
@@ -704,8 +800,8 @@ components:
|
|||||||
KV настройки tenant/глобальные лимиты и feature flags.
|
KV настройки tenant/глобальные лимиты и feature flags.
|
||||||
Параметры BIRD (строки в `global_settings`, JSON-значения — обычно строка или число):
|
Параметры BIRD (строки в `global_settings`, JSON-значения — обычно строка или число):
|
||||||
`bird_router_id`, `bird_local_ipv4`, `bird_local_ipv6`, `bird_local_asn`;
|
`bird_router_id`, `bird_local_ipv4`, `bird_local_ipv6`, `bird_local_asn`;
|
||||||
`bird_bgp_source_ipv4` / `bird_bgp_source_ipv6` — для `source address` у BGP-пиров (если не заданы — эффективный `bird_local_ipv4` / политика пира).
|
если задан `bird_bgp_source_ipv4`, он подставляется как BIRD `router id` (перекрывает `bird_router_id`).
|
||||||
Если задан `bird_bgp_source_ipv4`, он же подставляется как BIRD `router id` (перекрывает `bird_router_id`).
|
Ключи `bird_bgp_source_ipv4` / `bird_bgp_source_ipv6` в сгенерированном BGP для пиров не используются (оставлены для совместимости API).
|
||||||
Шаблон BGP в конфиге: `local as <bird_local_asn>;` без локального IP.
|
Шаблон BGP в конфиге: `local as <bird_local_asn>;` без локального IP.
|
||||||
properties:
|
properties:
|
||||||
bird_router_id:
|
bird_router_id:
|
||||||
@@ -720,11 +816,10 @@ components:
|
|||||||
bird_bgp_source_ipv4:
|
bird_bgp_source_ipv4:
|
||||||
type: string
|
type: string
|
||||||
description: |
|
description: |
|
||||||
Исходящий IPv4 для `source address` у пиров; также задаёт `router id`, если указан.
|
Если задан — используется как BIRD `router id` (IPv4). В блоках `protocol bgp … from bgp_template` строка `source address` не генерируется.
|
||||||
На стороне MikroTik в BGP Connection поле **Remote address** должно совпадать с этим адресом (или с тем, с которого реально устанавливается TCP-сессия к 179/tcp), иначе сессия не поднимется.
|
|
||||||
bird_bgp_source_ipv6:
|
bird_bgp_source_ipv6:
|
||||||
type: string
|
type: string
|
||||||
description: Исходящий IPv6 для `source address` у пиров IPv6.
|
description: Зарезервировано; в текущей генерации BGP не используется.
|
||||||
additionalProperties: true
|
additionalProperties: true
|
||||||
|
|
||||||
RevisionDiff:
|
RevisionDiff:
|
||||||
@@ -759,6 +854,8 @@ components:
|
|||||||
DohProfilePatch:
|
DohProfilePatch:
|
||||||
type: object
|
type: object
|
||||||
properties:
|
properties:
|
||||||
|
name:
|
||||||
|
type: string
|
||||||
url:
|
url:
|
||||||
type: string
|
type: string
|
||||||
format: uri
|
format: uri
|
||||||
@@ -769,20 +866,20 @@ components:
|
|||||||
|
|
||||||
BgpCommunityCreate:
|
BgpCommunityCreate:
|
||||||
type: object
|
type: object
|
||||||
required: [name]
|
required: [community]
|
||||||
properties:
|
properties:
|
||||||
name:
|
community:
|
||||||
type: string
|
type: string
|
||||||
kind:
|
title:
|
||||||
type: string
|
type: string
|
||||||
additionalProperties: true
|
additionalProperties: true
|
||||||
|
|
||||||
BgpCommunityPatch:
|
BgpCommunityPatch:
|
||||||
type: object
|
type: object
|
||||||
properties:
|
properties:
|
||||||
name:
|
community:
|
||||||
type: string
|
type: string
|
||||||
kind:
|
title:
|
||||||
type: string
|
type: string
|
||||||
additionalProperties: true
|
additionalProperties: true
|
||||||
|
|
||||||
@@ -878,11 +975,31 @@ paths:
|
|||||||
default:
|
default:
|
||||||
$ref: "#/components/responses/DefaultProblem"
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
|
||||||
|
/version:
|
||||||
|
get:
|
||||||
|
tags: [System]
|
||||||
|
summary: Версия сборки (корневой путь)
|
||||||
|
description: |
|
||||||
|
Аналог `GET /v1/version`. Публичный маршрут без аутентификации.
|
||||||
|
Semver в поле `version` задаётся при сборке Docker-образов из git-тега.
|
||||||
|
operationId: getVersionRoot
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Метаданные сборки.
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: "#/components/schemas/VersionInfo"
|
||||||
|
default:
|
||||||
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
|
||||||
/v1/version:
|
/v1/version:
|
||||||
get:
|
get:
|
||||||
tags: [System]
|
tags: [System]
|
||||||
summary: Версия сборки
|
summary: Версия сборки
|
||||||
description: Версия API и control-plane (`git_sha`, `build_time` и др.).
|
description: |
|
||||||
|
Semver control-plane и метаданные сборки (`git_sha`, `build_time`).
|
||||||
|
Дублирует `GET /version`.
|
||||||
operationId: getVersion
|
operationId: getVersion
|
||||||
responses:
|
responses:
|
||||||
"200":
|
"200":
|
||||||
@@ -898,7 +1015,7 @@ paths:
|
|||||||
get:
|
get:
|
||||||
tags: [Modules]
|
tags: [Modules]
|
||||||
summary: Список модулей
|
summary: Список модулей
|
||||||
description: Модули tenant с опциональными фильтрами по типу и флагу `enabled`.
|
description: Модули tenant с опциональными фильтрами по типу и флагу `enabled` (фильтры применяются сервером).
|
||||||
operationId: listModules
|
operationId: listModules
|
||||||
parameters:
|
parameters:
|
||||||
- $ref: "#/components/parameters/TenantId"
|
- $ref: "#/components/parameters/TenantId"
|
||||||
@@ -927,6 +1044,90 @@ paths:
|
|||||||
$ref: "#/components/responses/Unauthorized"
|
$ref: "#/components/responses/Unauthorized"
|
||||||
default:
|
default:
|
||||||
$ref: "#/components/responses/DefaultProblem"
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
|
||||||
|
/v1/router-lists/catalog:
|
||||||
|
get:
|
||||||
|
tags: [Modules]
|
||||||
|
summary: Агрегированный каталог для router-lists-ui
|
||||||
|
description: |
|
||||||
|
Возвращает в одном ответе:
|
||||||
|
- модули типов `DOMAINS`, `IP_RANGES`, `AS_PREFIXES`;
|
||||||
|
- entries по каждому модулю;
|
||||||
|
- справочник community (`id`, `community`, `title`).
|
||||||
|
operationId: getRouterListsCatalog
|
||||||
|
parameters:
|
||||||
|
- $ref: "#/components/parameters/TenantId"
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Агрегированные данные для списков UI.
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
required: [modules, domains, asns, ip_ranges, communities]
|
||||||
|
properties:
|
||||||
|
modules:
|
||||||
|
type: object
|
||||||
|
required: [items]
|
||||||
|
properties:
|
||||||
|
items:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: "#/components/schemas/Module"
|
||||||
|
domains:
|
||||||
|
type: object
|
||||||
|
required: [items]
|
||||||
|
properties:
|
||||||
|
items:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: object
|
||||||
|
required: [module_id, entry]
|
||||||
|
properties:
|
||||||
|
module_id:
|
||||||
|
$ref: "#/components/schemas/ResourceId"
|
||||||
|
entry:
|
||||||
|
$ref: "#/components/schemas/DomainEntry"
|
||||||
|
asns:
|
||||||
|
type: object
|
||||||
|
required: [items]
|
||||||
|
properties:
|
||||||
|
items:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: object
|
||||||
|
required: [module_id, entry]
|
||||||
|
properties:
|
||||||
|
module_id:
|
||||||
|
$ref: "#/components/schemas/ResourceId"
|
||||||
|
entry:
|
||||||
|
$ref: "#/components/schemas/AsEntry"
|
||||||
|
ip_ranges:
|
||||||
|
type: object
|
||||||
|
required: [items]
|
||||||
|
properties:
|
||||||
|
items:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
type: object
|
||||||
|
required: [module_id, entry]
|
||||||
|
properties:
|
||||||
|
module_id:
|
||||||
|
$ref: "#/components/schemas/ResourceId"
|
||||||
|
entry:
|
||||||
|
$ref: "#/components/schemas/IpRangeEntry"
|
||||||
|
communities:
|
||||||
|
type: object
|
||||||
|
required: [items]
|
||||||
|
properties:
|
||||||
|
items:
|
||||||
|
type: array
|
||||||
|
items:
|
||||||
|
$ref: "#/components/schemas/BgpCommunity"
|
||||||
|
"401":
|
||||||
|
$ref: "#/components/responses/Unauthorized"
|
||||||
|
default:
|
||||||
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
post:
|
post:
|
||||||
tags: [Modules]
|
tags: [Modules]
|
||||||
summary: Создать модуль
|
summary: Создать модуль
|
||||||
@@ -1480,6 +1681,74 @@ paths:
|
|||||||
default:
|
default:
|
||||||
$ref: "#/components/responses/DefaultProblem"
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
|
||||||
|
/v1/modules/{module_id}/entries.csv:
|
||||||
|
parameters:
|
||||||
|
- $ref: "#/components/parameters/TenantId"
|
||||||
|
- $ref: "#/components/parameters/ModuleId"
|
||||||
|
get:
|
||||||
|
tags: [Modules]
|
||||||
|
summary: Экспорт записей модуля в CSV
|
||||||
|
description: |
|
||||||
|
Доступно для типов модулей `AS_PREFIXES`, `DOMAINS`, `IP_RANGES`.
|
||||||
|
Возвращает CSV с колонками:
|
||||||
|
- AS: `asn,community`
|
||||||
|
- Домены: `domain,community`
|
||||||
|
- IP ranges: `ipRange,community`
|
||||||
|
operationId: exportModuleEntriesCsv
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: CSV-файл записей модуля.
|
||||||
|
content:
|
||||||
|
text/csv:
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
"404":
|
||||||
|
$ref: "#/components/responses/NotFound"
|
||||||
|
"422":
|
||||||
|
$ref: "#/components/responses/UnprocessableEntity"
|
||||||
|
default:
|
||||||
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
post:
|
||||||
|
tags: [Modules]
|
||||||
|
summary: Импорт записей модуля из CSV
|
||||||
|
description: |
|
||||||
|
Импортирует CSV в модуль типов `AS_PREFIXES`, `DOMAINS`, `IP_RANGES`.
|
||||||
|
Поддерживаемые заголовки:
|
||||||
|
- `asn,community`
|
||||||
|
- `domain,community`
|
||||||
|
- `ipRange,community`
|
||||||
|
|
||||||
|
В поле `community` можно передавать либо ID community, либо её значение.
|
||||||
|
operationId: importModuleEntriesCsv
|
||||||
|
parameters:
|
||||||
|
- $ref: "#/components/parameters/IdempotencyKey"
|
||||||
|
requestBody:
|
||||||
|
required: true
|
||||||
|
content:
|
||||||
|
text/csv:
|
||||||
|
schema:
|
||||||
|
type: string
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Импорт завершён.
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
type: object
|
||||||
|
required: [imported, module_type]
|
||||||
|
properties:
|
||||||
|
imported:
|
||||||
|
type: integer
|
||||||
|
minimum: 0
|
||||||
|
module_type:
|
||||||
|
type: string
|
||||||
|
"404":
|
||||||
|
$ref: "#/components/responses/NotFound"
|
||||||
|
"422":
|
||||||
|
$ref: "#/components/responses/UnprocessableEntity"
|
||||||
|
default:
|
||||||
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
|
||||||
/v1/doh-profiles:
|
/v1/doh-profiles:
|
||||||
get:
|
get:
|
||||||
tags: [DoH profiles]
|
tags: [DoH profiles]
|
||||||
@@ -1534,6 +1803,40 @@ paths:
|
|||||||
default:
|
default:
|
||||||
$ref: "#/components/responses/DefaultProblem"
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
|
||||||
|
/v1/modules/{module_id}/cdn-sources/preview:
|
||||||
|
parameters:
|
||||||
|
- $ref: "#/components/parameters/TenantId"
|
||||||
|
- $ref: "#/components/parameters/ModuleId"
|
||||||
|
post:
|
||||||
|
tags: [Modules]
|
||||||
|
summary: Предпросмотр префиксов из CDN-источника
|
||||||
|
description: >
|
||||||
|
Загружает URL, парсит как plaintext или json и возвращает список извлечённых префиксов (до 100 записей).
|
||||||
|
operationId: previewCdnSource
|
||||||
|
requestBody:
|
||||||
|
required: true
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: "#/components/schemas/CdnPreviewRequest"
|
||||||
|
responses:
|
||||||
|
"200":
|
||||||
|
description: Успешно.
|
||||||
|
content:
|
||||||
|
application/json:
|
||||||
|
schema:
|
||||||
|
$ref: "#/components/schemas/CdnPreviewResponse"
|
||||||
|
"401":
|
||||||
|
$ref: "#/components/responses/Unauthorized"
|
||||||
|
"403":
|
||||||
|
$ref: "#/components/responses/Forbidden"
|
||||||
|
"404":
|
||||||
|
$ref: "#/components/responses/NotFound"
|
||||||
|
"422":
|
||||||
|
$ref: "#/components/responses/UnprocessableEntity"
|
||||||
|
default:
|
||||||
|
$ref: "#/components/responses/DefaultProblem"
|
||||||
|
|
||||||
/v1/doh-profiles/{id}:
|
/v1/doh-profiles/{id}:
|
||||||
parameters:
|
parameters:
|
||||||
- $ref: "#/components/parameters/TenantId"
|
- $ref: "#/components/parameters/TenantId"
|
||||||
@@ -1736,6 +2039,9 @@ paths:
|
|||||||
post:
|
post:
|
||||||
tags: [Peers]
|
tags: [Peers]
|
||||||
summary: Создать пира
|
summary: Создать пира
|
||||||
|
description: >
|
||||||
|
Создаёт BGP-пира и инициирует быстрый reconcile пиров (job `peer_reconcile`) без module ingest.
|
||||||
|
После reconcile автоматически запускается apply на спикеры.
|
||||||
operationId: createPeer
|
operationId: createPeer
|
||||||
parameters:
|
parameters:
|
||||||
- $ref: "#/components/parameters/TenantId"
|
- $ref: "#/components/parameters/TenantId"
|
||||||
@@ -1781,6 +2087,8 @@ paths:
|
|||||||
description: >
|
description: >
|
||||||
Политики (`policies_json`: `local_ipv4`, `local_ipv6`, `local_asn`), neighbor, ASN,
|
Политики (`policies_json`: `local_ipv4`, `local_ipv6`, `local_asn`), neighbor, ASN,
|
||||||
привязка к `bgp_speaker_id` или `null` для всех спикеров.
|
привязка к `bgp_speaker_id` или `null` для всех спикеров.
|
||||||
|
Изменение инициирует быстрый reconcile пиров (job `peer_reconcile`) без module ingest
|
||||||
|
и затем авто-apply на спикеры.
|
||||||
operationId: patchPeer
|
operationId: patchPeer
|
||||||
parameters:
|
parameters:
|
||||||
- $ref: "#/components/parameters/IdempotencyKey"
|
- $ref: "#/components/parameters/IdempotencyKey"
|
||||||
@@ -1804,6 +2112,9 @@ paths:
|
|||||||
delete:
|
delete:
|
||||||
tags: [Peers]
|
tags: [Peers]
|
||||||
summary: Удалить или отключить пира
|
summary: Удалить или отключить пира
|
||||||
|
description: >
|
||||||
|
Удаление/отключение инициирует быстрый reconcile пиров (job `peer_reconcile`) без module ingest
|
||||||
|
и затем авто-apply на спикеры.
|
||||||
operationId: deletePeer
|
operationId: deletePeer
|
||||||
parameters:
|
parameters:
|
||||||
- $ref: "#/components/parameters/IdempotencyKey"
|
- $ref: "#/components/parameters/IdempotencyKey"
|
||||||
|
|||||||
+44
-7
@@ -109,35 +109,72 @@ docker compose --profile microvps down -v
|
|||||||
|
|
||||||
Из каталога `deploy\compose` (как в варианте 1: `.env` из `.env.example`, при необходимости `docker login git.shts.su`):
|
Из каталога `deploy\compose` (как в варианте 1: `.env` из `.env.example`, при необходимости `docker login git.shts.su`):
|
||||||
|
|
||||||
|
1. Подготовьте отдельный файл параметров защищенного UI:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
Copy-Item .env.web-sec.example .env.web-sec -Force
|
||||||
|
```
|
||||||
|
|
||||||
|
2. Заполните в `.env.web-sec`:
|
||||||
|
- `WEBUI_DOMAIN` — домен панели;
|
||||||
|
- `WEBUI_IP_WHITELIST` — список разрешенных IP/CIDR через запятую;
|
||||||
|
- `LETSENCRYPT_EMAIL` — email для ACME;
|
||||||
|
- `CF_DNS_API_TOKEN` — Cloudflare token для `DNS challenge` (минимальные права `Zone:DNS:Edit` на нужной зоне).
|
||||||
|
|
||||||
|
3. В Cloudflare для `WEBUI_DOMAIN` используйте запись в режиме **DNS only** (серый облачок), указывающую на публичный IP VPS.
|
||||||
|
|
||||||
|
4. Запускайте compose с двумя env-файлами:
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
# С ужатыми лимитами и EVOBGP_BROKER_URL=nats://… (ориентир под ~1 ГиБ RAM на хосте)
|
# С ужатыми лимитами и EVOBGP_BROKER_URL=nats://… (ориентир под ~1 ГиБ RAM на хосте)
|
||||||
docker compose -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full pull
|
docker compose --env-file .env --env-file .env.web-sec -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full pull
|
||||||
docker compose -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full up -d
|
docker compose --env-file .env --env-file .env.web-sec -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
Только профиль `microvps-full` и **без** файла `docker-compose.microvps-full.yaml` (лимиты как в базовом compose, без принудительной подстановки брокера в `evobgp-all`):
|
Только профиль `microvps-full` и **без** файла `docker-compose.microvps-full.yaml` (лимиты как в базовом compose, без принудительной подстановки брокера в `evobgp-all`):
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
docker compose --profile microvps-full pull
|
docker compose --env-file .env --env-file .env.web-sec --profile microvps-full pull
|
||||||
docker compose --profile microvps-full up -d
|
docker compose --env-file .env --env-file .env.web-sec --profile microvps-full up -d
|
||||||
```
|
```
|
||||||
|
|
||||||
При необходимости задайте брокер вручную для `evobgp-all` (override или `.env` рядом с compose): `EVOBGP_BROKER_URL=nats://nats:4222`.
|
При необходимости задайте брокер вручную для `evobgp-all` (override или `.env` рядом с compose): `EVOBGP_BROKER_URL=nats://nats:4222`.
|
||||||
|
|
||||||
| Порт | Назначение |
|
| Порт | Назначение |
|
||||||
|------|------------|
|
|------|------------|
|
||||||
| **3000** | Web UI (nginx → API под капотом) |
|
| **80** | HTTP вход (редирект на HTTPS через Traefik) |
|
||||||
|
| **443** | HTTPS Web UI (Traefik + Let's Encrypt) |
|
||||||
| **8080** | Прямой HTTP API (`evobgp-all`) |
|
| **8080** | Прямой HTTP API (`evobgp-all`) |
|
||||||
| **9090** | Prometheus (`prometheus-microvps`) |
|
| **9090** | Prometheus (`prometheus-microvps`) |
|
||||||
| **4222** | NATS |
|
| **4222** | NATS |
|
||||||
| **179** | BGP (BIRD2), как в варианте 1 |
|
| **179** | BGP (BIRD2), как в варианте 1 |
|
||||||
|
|
||||||
Проверка UI: откройте `http://<IP-вашей-VPS>:3000`. Health API: `http://<IP>:8080/v1/health`.
|
Проверка UI:
|
||||||
|
- `http://<WEBUI_DOMAIN>` должен редиректить на `https://<WEBUI_DOMAIN>`;
|
||||||
|
- с IP из `WEBUI_IP_WHITELIST` UI доступен по HTTPS;
|
||||||
|
- с неразрешенного IP Traefik вернет `403`.
|
||||||
|
|
||||||
|
Health API: `http://<IP>:8080/v1/health`.
|
||||||
|
|
||||||
|
### Auto-updater для standalone stack (без рестарта BIRD2)
|
||||||
|
|
||||||
|
Для `stack.microvps-full.yaml` можно включить автообновление только выбранных сервисов (например, `evobgp-all,evobgp-web`) по digest образов в registry.
|
||||||
|
|
||||||
|
В файле `deploy\compose\.env.stack.microvps-full`:
|
||||||
|
|
||||||
|
- `AUTO_UPDATE_ENABLED=1` — включить updater;
|
||||||
|
- `AUTO_UPDATE_INTERVAL_SEC=300` — период проверки;
|
||||||
|
- `AUTO_UPDATE_SERVICES=evobgp-all,evobgp-web` — allowlist сервисов;
|
||||||
|
- `AUTO_UPDATE_PROTECTED_SERVICES=bird2` — список защищённых сервисов (по умолчанию `bird2`).
|
||||||
|
|
||||||
|
Updater делает `pull` и при изменении образа выполняет `up -d --no-deps` только для сервиса из allowlist. `bird2` в обновление не попадает, пока явно указан в protected-списке.
|
||||||
|
|
||||||
|
Важно: updater использует доступ к `docker.sock` (высокие привилегии), включайте осознанно.
|
||||||
|
|
||||||
Остановка (если поднимали с двумя `-f`):
|
Остановка (если поднимали с двумя `-f`):
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
docker compose -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full down
|
docker compose --env-file .env --env-file .env.web-sec -f docker-compose.yaml -f docker-compose.microvps-full.yaml --profile microvps-full down
|
||||||
```
|
```
|
||||||
|
|
||||||
## Вариант 2: Docker, профиль reference
|
## Вариант 2: Docker, профиль reference
|
||||||
|
|||||||
@@ -0,0 +1,74 @@
|
|||||||
|
# Релизы и версионирование EvoBGP
|
||||||
|
|
||||||
|
EvoBGP использует [Conventional Commits](https://www.conventionalcommits.org/) и [semantic-release](https://semantic-release.gitbook.io/) для полностью автоматических релизов на Gitea (`git.shts.su`). Ручное повышение версии в коде не требуется.
|
||||||
|
|
||||||
|
## Как определяется версия
|
||||||
|
|
||||||
|
| Тип коммита | Bump |
|
||||||
|
|-------------|------|
|
||||||
|
| `feat` | minor (1.0.0 → 1.1.0) |
|
||||||
|
| `fix`, `perf` | patch (1.0.0 → 1.0.1) |
|
||||||
|
| `feat!`, `fix!` или `BREAKING CHANGE:` в теле | major (1.0.0 → 2.0.0) |
|
||||||
|
| `docs`, `chore`, `ci`, `test`, `refactor` | без релиза |
|
||||||
|
|
||||||
|
Первый релиз при отсутствии git-тегов — **1.0.0**, если есть releasable-коммиты.
|
||||||
|
|
||||||
|
**Как не перепутать `feat` и `fix`:** см. раздел «Выбор type: semver, а не «красивые слова»» в [.cursor/rules/conventional-commits.mdc](../.cursor/rules/conventional-commits.mdc). Кратко: `feat` — новая возможность (minor); `fix` — починка ожидаемого поведения (patch); follow-up баги после недавнего `feat` — всегда `fix`, даже если diff большой.
|
||||||
|
|
||||||
|
Подробные правила сообщений коммитов: [.cursor/rules/conventional-commits.mdc](../.cursor/rules/conventional-commits.mdc).
|
||||||
|
|
||||||
|
## CI-пайплайн (один push в main)
|
||||||
|
|
||||||
|
```text
|
||||||
|
push/merge в main
|
||||||
|
→ CI: openapi, web, go, bird2 (параллельно)
|
||||||
|
→ job release (в том же workflow, после quality gates):
|
||||||
|
→ semantic-release: git tag vX.Y.Z на текущий commit (без доп. commit)
|
||||||
|
→ Gitea Release + CHANGELOG.md как attachment
|
||||||
|
→ docker buildx bake с VERSION=X.Y.Z
|
||||||
|
→ образы: latest, vX.Y.Z, X.Y.Z, sha-*, короткий SHA
|
||||||
|
```
|
||||||
|
|
||||||
|
Pull request: только quality gates + commitlint; релиз и образы **не** публикуются.
|
||||||
|
|
||||||
|
Workflow: [.gitea/workflows/ci.yaml](../.gitea/workflows/ci.yaml) (job **release**).
|
||||||
|
|
||||||
|
Конфиг semantic-release: [.releaserc.json](../.releaserc.json) — без `@semantic-release/git` (CHANGELOG не коммитится в репозиторий).
|
||||||
|
|
||||||
|
## Секреты Gitea
|
||||||
|
|
||||||
|
Один PAT — **`ACTIONS_PAT`** (Settings → Actions → Secrets).
|
||||||
|
|
||||||
|
| Право PAT | Зачем |
|
||||||
|
|-----------|--------|
|
||||||
|
| push tags | git tag `vX.Y.Z` на commit merge |
|
||||||
|
| releases | Gitea Release + notes |
|
||||||
|
| packages (Container Registry) | push образов |
|
||||||
|
|
||||||
|
Fallback: **`gitea.token`** (нужны права на releases и packages).
|
||||||
|
|
||||||
|
## Источник правды для версии в runtime
|
||||||
|
|
||||||
|
Semver из git-тега пробрасывается в Go-бинарники через `-ldflags` при сборке Docker (`deploy/docker/gobinary/Dockerfile`). Пакет [`internal/version`](../internal/version/version.go):
|
||||||
|
|
||||||
|
- локально (`go run`) — `version: "dev"`
|
||||||
|
- в образе после релиза — совпадает с тегом (например `1.2.3`)
|
||||||
|
|
||||||
|
API: `GET /version`, `GET /v1/version` — поля `version`, `git_sha`, `build_time`.
|
||||||
|
|
||||||
|
Web UI показывает версию из API (footer sidebar, страница «Мониторинг»).
|
||||||
|
|
||||||
|
## CHANGELOG
|
||||||
|
|
||||||
|
Release notes — в Gitea Release; файл `CHANGELOG.md` генерируется в CI и прикрепляется как asset, **не** попадает в git history.
|
||||||
|
|
||||||
|
## Проверка после релиза
|
||||||
|
|
||||||
|
1. Один run workflow **CI** на push в main: job **release** зелёный.
|
||||||
|
2. Gitea: тег `vX.Y.Z` на том же commit, что и merge; Release с notes.
|
||||||
|
3. Container Registry: `evobgp-api:vX.Y.Z`, `evobgp-api:X.Y.Z`, `evobgp-api:latest`.
|
||||||
|
4. `curl http://localhost:8080/version` → `"version":"X.Y.Z"`.
|
||||||
|
|
||||||
|
## Первый релиз (bootstrap)
|
||||||
|
|
||||||
|
Merge в `main` с `feat(release): ...` → **v1.0.0** в том же CI run.
|
||||||
@@ -0,0 +1,119 @@
|
|||||||
|
# Интеграция router-lists-ui с EvoBGP API
|
||||||
|
|
||||||
|
Документ описывает подключение разделов `Домены`, `IP-диапазоны`, `AS`, `Community` из проекта `router-lists-ui` напрямую к EvoBGP API (`/v1`).
|
||||||
|
|
||||||
|
Источник правды по контракту: [openapi.yaml](openapi.yaml).
|
||||||
|
|
||||||
|
## 1. Базовые требования
|
||||||
|
|
||||||
|
- Base URL API: `https://<evobgp-host>/v1`
|
||||||
|
- Аутентификация: `Authorization: Bearer <api_key>`
|
||||||
|
- CORS на стороне EvoBGP: переменная `EVOBGP_CORS_ORIGINS`
|
||||||
|
- Роли:
|
||||||
|
- чтение: `viewer+`
|
||||||
|
- изменение данных: `editor+`
|
||||||
|
- apply/деплой: `operator`
|
||||||
|
|
||||||
|
## 2. Маппинг legacy -> EvoBGP
|
||||||
|
|
||||||
|
| Раздел UI | Legacy endpoint | EvoBGP endpoint |
|
||||||
|
|---|---|---|
|
||||||
|
| Домены | `/api/domains-new` | `/v1/modules?type=DOMAINS` + `/v1/modules/{module_id}/domain-entries` |
|
||||||
|
| IP-диапазоны | `/api/ip-ranges` | `/v1/modules?type=IP_RANGES` + `/v1/modules/{module_id}/ip-range-entries` |
|
||||||
|
| AS | `/api/asns` | `/v1/modules?type=AS_PREFIXES` + `/v1/modules/{module_id}/as-entries` |
|
||||||
|
| Community | `/api/communities` | `/v1/communities` |
|
||||||
|
|
||||||
|
Для упрощённой интеграции доступен агрегированный endpoint: `GET /v1/router-lists/catalog` (модули + entries + communities в одном ответе).
|
||||||
|
|
||||||
|
## 3. Маппинг полей
|
||||||
|
|
||||||
|
| Legacy модель | EvoBGP модель | Комментарий |
|
||||||
|
|---|---|---|
|
||||||
|
| `{ domain, community }` | `DomainEntry { fqdn, community_id }` | `community` в UI резолвится в `community_id` через `/v1/communities` |
|
||||||
|
| `{ ipRange, community }` | `IpRangeEntry { prefix, community_id }` | Для `IP_RANGES` `community_id` обязателен |
|
||||||
|
| `{ domain, type }` (AS) | `AsEntry { asn, community_id }` | `domain` legacy = ASN |
|
||||||
|
| `{ value, name }` (Community dict) | `BgpCommunity { community, title }` | `value -> community`, `name -> title` |
|
||||||
|
|
||||||
|
## 4. Алгоритм работы по разделам
|
||||||
|
|
||||||
|
### 4.1 Домены / IP / AS (общая схема)
|
||||||
|
|
||||||
|
1. Найти модуль нужного типа через `GET /v1/modules?type=...`.
|
||||||
|
2. Если модуль отсутствует, создать через `POST /v1/modules`.
|
||||||
|
3. Загрузить entries:
|
||||||
|
- домены: `GET /v1/modules/{module_id}/domain-entries`
|
||||||
|
- IP: `GET /v1/modules/{module_id}/ip-range-entries`
|
||||||
|
- AS: `GET /v1/modules/{module_id}/as-entries`
|
||||||
|
4. При сохранении:
|
||||||
|
- удалить отсутствующие записи (`DELETE .../{entry_id}`)
|
||||||
|
- обновить изменённые (`PATCH .../{entry_id}`)
|
||||||
|
- добавить новые (`POST ...`)
|
||||||
|
|
||||||
|
### 4.2 Community
|
||||||
|
|
||||||
|
1. Получить список: `GET /v1/communities`
|
||||||
|
2. Для сохранения diff:
|
||||||
|
- новые -> `POST /v1/communities`
|
||||||
|
- изменённые -> `PATCH /v1/communities/{id}`
|
||||||
|
- удалённые -> `DELETE /v1/communities/{id}`
|
||||||
|
|
||||||
|
## 5. Разрешение community
|
||||||
|
|
||||||
|
- Перед работой с entries загрузить `/v1/communities`.
|
||||||
|
- Основной путь: сопоставление по `id`.
|
||||||
|
- Для UI-формы использовать значение `community` (например, `65001:120`) и перед записью преобразовывать в `community_id`.
|
||||||
|
- Если community не найдена:
|
||||||
|
- для `IP_RANGES` считать ошибкой валидации;
|
||||||
|
- для `DOMAINS` и `AS_PREFIXES` можно передать `null` только если это допускается бизнес-логикой.
|
||||||
|
|
||||||
|
## 6. Пагинация и ошибки
|
||||||
|
|
||||||
|
- Списки используют `cursor + limit`.
|
||||||
|
- Формат ошибок: `application/problem+json` (RFC 9457).
|
||||||
|
- Базовая обработка:
|
||||||
|
- `401` — неверный/отсутствующий токен
|
||||||
|
- `403` — недостаточно прав
|
||||||
|
- `404` — ресурс не найден
|
||||||
|
- `422` — ошибка валидации
|
||||||
|
|
||||||
|
## 7. Примеры (PowerShell)
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
$base = "http://localhost:8080/v1"
|
||||||
|
$token = "YOUR_API_KEY"
|
||||||
|
$h = @{ Authorization = "Bearer $token" }
|
||||||
|
|
||||||
|
# 1) Найти модуль DOMAINS
|
||||||
|
$modules = Invoke-RestMethod -Uri "$base/modules?type=DOMAINS&limit=50" -Headers $h
|
||||||
|
$moduleId = $modules.items[0].id
|
||||||
|
|
||||||
|
# 2) Прочитать домены
|
||||||
|
Invoke-RestMethod -Uri "$base/modules/$moduleId/domain-entries?limit=100" -Headers $h
|
||||||
|
```
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
# Создать community
|
||||||
|
$body = @{
|
||||||
|
community = "65001:120"
|
||||||
|
title = "Video"
|
||||||
|
} | ConvertTo-Json
|
||||||
|
|
||||||
|
Invoke-RestMethod -Method Post -Uri "$base/communities" -Headers $h -ContentType "application/json" -Body $body
|
||||||
|
```
|
||||||
|
|
||||||
|
## 8. Настройки router-lists-ui для прямой интеграции
|
||||||
|
|
||||||
|
Рекомендуемые переменные frontend:
|
||||||
|
|
||||||
|
- `VITE_EVOBGP_API_URL` (по умолчанию `/v1`)
|
||||||
|
- `VITE_EVOBGP_API_TOKEN` (опционально; либо хранить токен в `localStorage` как `evobgp_api_token`)
|
||||||
|
|
||||||
|
Для dev-прокси Vite добавить маршрут `/v1 -> http://localhost:8080`.
|
||||||
|
|
||||||
|
## 9. Контрольный список интеграции
|
||||||
|
|
||||||
|
- [ ] `Домены`: list/create/update/delete работают через `/v1/modules/{id}/domain-entries`
|
||||||
|
- [ ] `IP-диапазоны`: list/create/update/delete работают через `/v1/modules/{id}/ip-range-entries`
|
||||||
|
- [ ] `AS`: list/create/update/delete работают через `/v1/modules/{id}/as-entries`
|
||||||
|
- [ ] `Community`: CRUD работает через `/v1/communities`
|
||||||
|
- [ ] ошибки `problem+json` корректно отображаются в UI
|
||||||
@@ -1,10 +1,11 @@
|
|||||||
module evobgp
|
module evobgp
|
||||||
|
|
||||||
go 1.22
|
go 1.24.0
|
||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/google/uuid v1.6.0
|
github.com/google/uuid v1.6.0
|
||||||
github.com/jackc/pgx/v5 v5.7.2
|
github.com/jackc/pgx/v5 v5.7.2
|
||||||
|
github.com/miekg/dns v1.1.72
|
||||||
github.com/prometheus/client_golang v1.20.5
|
github.com/prometheus/client_golang v1.20.5
|
||||||
modernc.org/sqlite v1.34.5
|
modernc.org/sqlite v1.34.5
|
||||||
)
|
)
|
||||||
@@ -24,10 +25,13 @@ require (
|
|||||||
github.com/prometheus/common v0.55.0 // indirect
|
github.com/prometheus/common v0.55.0 // indirect
|
||||||
github.com/prometheus/procfs v0.15.1 // indirect
|
github.com/prometheus/procfs v0.15.1 // indirect
|
||||||
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
||||||
golang.org/x/crypto v0.31.0 // indirect
|
golang.org/x/crypto v0.46.0 // indirect
|
||||||
golang.org/x/sync v0.10.0 // indirect
|
golang.org/x/mod v0.31.0 // indirect
|
||||||
golang.org/x/sys v0.28.0 // indirect
|
golang.org/x/net v0.48.0 // indirect
|
||||||
golang.org/x/text v0.21.0 // indirect
|
golang.org/x/sync v0.19.0 // indirect
|
||||||
|
golang.org/x/sys v0.39.0 // indirect
|
||||||
|
golang.org/x/text v0.32.0 // indirect
|
||||||
|
golang.org/x/tools v0.40.0 // indirect
|
||||||
google.golang.org/protobuf v1.34.2 // indirect
|
google.golang.org/protobuf v1.34.2 // indirect
|
||||||
modernc.org/libc v1.55.3 // indirect
|
modernc.org/libc v1.55.3 // indirect
|
||||||
modernc.org/mathutil v1.6.0 // indirect
|
modernc.org/mathutil v1.6.0 // indirect
|
||||||
|
|||||||
@@ -28,6 +28,8 @@ github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0
|
|||||||
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
||||||
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
|
||||||
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
|
||||||
|
github.com/miekg/dns v1.1.72 h1:vhmr+TF2A3tuoGNkLDFK9zi36F2LS+hKTRW0Uf8kbzI=
|
||||||
|
github.com/miekg/dns v1.1.72/go.mod h1:+EuEPhdHOsfk6Wk5TT2CzssZdqkmFhf8r+aVyDEToIs=
|
||||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
|
||||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
|
||||||
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
|
github.com/ncruces/go-strftime v0.1.9 h1:bY0MQC28UADQmHmaF5dgpLmImcShSi2kHU9XLdhx/f4=
|
||||||
@@ -49,19 +51,21 @@ github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UV
|
|||||||
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
|
||||||
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
github.com/stretchr/testify v1.9.0 h1:HtqpIVDClZ4nwg75+f6Lvsy/wHu+3BoSGCbBAcpTsTg=
|
||||||
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
github.com/stretchr/testify v1.9.0/go.mod h1:r2ic/lqez/lEtzL7wO/rwa5dbSLXVDPFyf8C91i36aY=
|
||||||
golang.org/x/crypto v0.31.0 h1:ihbySMvVjLAeSH1IbfcRTkD/iNscyz8rGzjF/E5hV6U=
|
golang.org/x/crypto v0.46.0 h1:cKRW/pmt1pKAfetfu+RCEvjvZkA9RimPbh7bhFjGVBU=
|
||||||
golang.org/x/crypto v0.31.0/go.mod h1:kDsLvtWBEx7MV9tJOj9bnXsPbxwJQ6csT/x4KIN4Ssk=
|
golang.org/x/crypto v0.46.0/go.mod h1:Evb/oLKmMraqjZ2iQTwDwvCtJkczlDuTmdJXoZVzqU0=
|
||||||
golang.org/x/mod v0.17.0 h1:zY54UmvipHiNd+pm+m0x9KhZ9hl1/7QNMyxXbc6ICqA=
|
golang.org/x/mod v0.31.0 h1:HaW9xtz0+kOcWKwli0ZXy79Ix+UW/vOfmWI5QVd2tgI=
|
||||||
golang.org/x/mod v0.17.0/go.mod h1:hTbmBsO62+eylJbnUtE2MGJUyE7QWk4xUqPFrRgJ+7c=
|
golang.org/x/mod v0.31.0/go.mod h1:43JraMp9cGx1Rx3AqioxrbrhNsLl2l/iNAvuBkrezpg=
|
||||||
golang.org/x/sync v0.10.0 h1:3NQrjDixjgGwUOCaF8w2+VYHv0Ve/vGYSbdkTa98gmQ=
|
golang.org/x/net v0.48.0 h1:zyQRTTrjc33Lhh0fBgT/H3oZq9WuvRR5gPC70xpDiQU=
|
||||||
golang.org/x/sync v0.10.0/go.mod h1:Czt+wKu1gCyEFDUtn0jG5QVvpJ6rzVqr5aXyt9drQfk=
|
golang.org/x/net v0.48.0/go.mod h1:+ndRgGjkh8FGtu1w1FGbEC31if4VrNVMuKTgcAAnQRY=
|
||||||
|
golang.org/x/sync v0.19.0 h1:vV+1eWNmZ5geRlYjzm2adRgW2/mcpevXNg50YZtPCE4=
|
||||||
|
golang.org/x/sync v0.19.0/go.mod h1:9KTHXmSnoGruLpwFjVSX0lNNA75CykiMECbovNTZqGI=
|
||||||
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
|
||||||
golang.org/x/sys v0.28.0 h1:Fksou7UEQUWlKvIdsqzJmUmCX3cZuD2+P3XyyzwMhlA=
|
golang.org/x/sys v0.39.0 h1:CvCKL8MeisomCi6qNZ+wbb0DN9E5AATixKsvNtMoMFk=
|
||||||
golang.org/x/sys v0.28.0/go.mod h1:/VUhepiaJMQUp4+oa/7Zr1D23ma6VTLIYjOOTFZPUcA=
|
golang.org/x/sys v0.39.0/go.mod h1:OgkHotnGiDImocRcuBABYBEXf8A9a87e/uXjp9XT3ks=
|
||||||
golang.org/x/text v0.21.0 h1:zyQAAkrwaneQ066sspRyJaG9VNi/YJ1NfzcGB3hZ/qo=
|
golang.org/x/text v0.32.0 h1:ZD01bjUt1FQ9WJ0ClOL5vxgxOI/sVCNgX1YtKwcY0mU=
|
||||||
golang.org/x/text v0.21.0/go.mod h1:4IBbMaMmOPCJ8SecivzSH54+73PCFmPWxNTLm+vZkEQ=
|
golang.org/x/text v0.32.0/go.mod h1:o/rUWzghvpD5TXrTIBuJU77MTaN0ljMWE47kxGJQ7jY=
|
||||||
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d h1:vU5i/LfpvrRCpgM/VPfJLg5KjxD3E+hfT1SH+d9zLwg=
|
golang.org/x/tools v0.40.0 h1:yLkxfA+Qnul4cs9QA3KnlFu0lVmd8JJfoq+E41uSutA=
|
||||||
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d/go.mod h1:aiJjzUbINMkxbQROHiO6hDPo2LHcIPhhQsa9DLh0yGk=
|
golang.org/x/tools v0.40.0/go.mod h1:Ik/tzLRlbscWpqqMRjyWYDisX8bG13FrdXp3o4Sr9lc=
|
||||||
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
|
google.golang.org/protobuf v1.34.2 h1:6xV6lTsCfpGD21XK49h7MhtcApnLqkfYgPcdHftf6hg=
|
||||||
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
|
google.golang.org/protobuf v1.34.2/go.mod h1:qYOHts0dSfpeUzUFpOMr/WGzszTmLH+DiWniOlNbLDw=
|
||||||
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
|
||||||
|
|||||||
+4
-11
@@ -12,7 +12,7 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// BGPTemplatesOptions holds ASN and export filters for template bgp bgp_template (+ v6 mirror).
|
// BGPTemplatesOptions holds ASN and export filters for template bgp bgp_template (+ v6 mirror).
|
||||||
// Templates use "local as <asn>;" only (no local IP); peers supply source address / optional local override.
|
// Templates use "local as <asn>;" only (no local IP); peers add neighbor / multihop / passive and optional "local … as …" override.
|
||||||
type BGPTemplatesOptions struct {
|
type BGPTemplatesOptions struct {
|
||||||
LocalASN uint32
|
LocalASN uint32
|
||||||
ExportFilterV4 string
|
ExportFilterV4 string
|
||||||
@@ -43,8 +43,6 @@ func RenderBGPTemplates(opts BGPTemplatesOptions) (string, error) {
|
|||||||
b.WriteString(" };\n")
|
b.WriteString(" };\n")
|
||||||
b.WriteString(" hold time 90;\n")
|
b.WriteString(" hold time 90;\n")
|
||||||
b.WriteString(" keepalive time 30;\n")
|
b.WriteString(" keepalive time 30;\n")
|
||||||
// Жёсткая привязка к source address (см. пир): полезно в Docker / bridge, иначе listen может уйти на 0.0.0.0.
|
|
||||||
b.WriteString(" strict bind on;\n")
|
|
||||||
b.WriteString("}\n\n")
|
b.WriteString("}\n\n")
|
||||||
fmt.Fprintf(&b, "template bgp %s {\n", BGPTemplateNameV6)
|
fmt.Fprintf(&b, "template bgp %s {\n", BGPTemplateNameV6)
|
||||||
fmt.Fprintf(&b, " local as %d;\n", opts.LocalASN)
|
fmt.Fprintf(&b, " local as %d;\n", opts.LocalASN)
|
||||||
@@ -56,7 +54,6 @@ func RenderBGPTemplates(opts BGPTemplatesOptions) (string, error) {
|
|||||||
b.WriteString(" };\n")
|
b.WriteString(" };\n")
|
||||||
b.WriteString(" hold time 90;\n")
|
b.WriteString(" hold time 90;\n")
|
||||||
b.WriteString(" keepalive time 30;\n")
|
b.WriteString(" keepalive time 30;\n")
|
||||||
b.WriteString(" strict bind on;\n")
|
|
||||||
b.WriteString("}\n")
|
b.WriteString("}\n")
|
||||||
return b.String(), nil
|
return b.String(), nil
|
||||||
}
|
}
|
||||||
@@ -67,13 +64,12 @@ type BGPPeerFromTemplateOptions struct {
|
|||||||
TemplateName string
|
TemplateName string
|
||||||
NeighborIP string
|
NeighborIP string
|
||||||
NeighborASN uint32
|
NeighborASN uint32
|
||||||
SourceAddress string
|
|
||||||
// If set, emits "local … as …" before neighbor (overrides template local/ASN for this peer).
|
// If set, emits "local … as …" before neighbor (overrides template local/ASN for this peer).
|
||||||
OverrideLocalIP string
|
OverrideLocalIP string
|
||||||
OverrideLocalASN uint32
|
OverrideLocalASN uint32
|
||||||
}
|
}
|
||||||
|
|
||||||
// RenderProtocolBGPFromTemplate renders protocol bgp … from TEMPLATE { neighbor; multihop; source address; passive; }.
|
// RenderProtocolBGPFromTemplate renders protocol bgp … from TEMPLATE { neighbor; multihop; passive; }.
|
||||||
func RenderProtocolBGPFromTemplate(opts BGPPeerFromTemplateOptions) (string, error) {
|
func RenderProtocolBGPFromTemplate(opts BGPPeerFromTemplateOptions) (string, error) {
|
||||||
if strings.TrimSpace(opts.ProtocolName) == "" {
|
if strings.TrimSpace(opts.ProtocolName) == "" {
|
||||||
return "", fmt.Errorf("birdfmt: protocol name is required")
|
return "", fmt.Errorf("birdfmt: protocol name is required")
|
||||||
@@ -81,8 +77,8 @@ func RenderProtocolBGPFromTemplate(opts BGPPeerFromTemplateOptions) (string, err
|
|||||||
if strings.TrimSpace(opts.TemplateName) == "" {
|
if strings.TrimSpace(opts.TemplateName) == "" {
|
||||||
return "", fmt.Errorf("birdfmt: template name is required")
|
return "", fmt.Errorf("birdfmt: template name is required")
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(opts.NeighborIP) == "" || strings.TrimSpace(opts.SourceAddress) == "" {
|
if strings.TrimSpace(opts.NeighborIP) == "" {
|
||||||
return "", fmt.Errorf("birdfmt: neighbor and source address are required")
|
return "", fmt.Errorf("birdfmt: neighbor is required")
|
||||||
}
|
}
|
||||||
if opts.NeighborASN == 0 {
|
if opts.NeighborASN == 0 {
|
||||||
return "", fmt.Errorf("birdfmt: neighbor ASN must be non-zero")
|
return "", fmt.Errorf("birdfmt: neighbor ASN must be non-zero")
|
||||||
@@ -106,9 +102,6 @@ func RenderProtocolBGPFromTemplate(opts BGPPeerFromTemplateOptions) (string, err
|
|||||||
b.WriteString(strings.TrimSpace(opts.NeighborIP))
|
b.WriteString(strings.TrimSpace(opts.NeighborIP))
|
||||||
fmt.Fprintf(&b, " as %d;\n", opts.NeighborASN)
|
fmt.Fprintf(&b, " as %d;\n", opts.NeighborASN)
|
||||||
b.WriteString(" multihop;\n")
|
b.WriteString(" multihop;\n")
|
||||||
b.WriteString(" source address ")
|
|
||||||
b.WriteString(strings.TrimSpace(opts.SourceAddress))
|
|
||||||
b.WriteString(";\n")
|
|
||||||
b.WriteString(" passive;\n")
|
b.WriteString(" passive;\n")
|
||||||
b.WriteString("}\n")
|
b.WriteString("}\n")
|
||||||
return b.String(), nil
|
return b.String(), nil
|
||||||
|
|||||||
@@ -37,11 +37,10 @@ func TestRenderBGPTemplates_Validation(t *testing.T) {
|
|||||||
|
|
||||||
func TestRenderProtocolBGPFromTemplate_MultihopPassive(t *testing.T) {
|
func TestRenderProtocolBGPFromTemplate_MultihopPassive(t *testing.T) {
|
||||||
got, err := RenderProtocolBGPFromTemplate(BGPPeerFromTemplateOptions{
|
got, err := RenderProtocolBGPFromTemplate(BGPPeerFromTemplateOptions{
|
||||||
ProtocolName: "evobgp_p_x",
|
ProtocolName: "evobgp_p_x",
|
||||||
TemplateName: BGPTemplateNameV4,
|
TemplateName: BGPTemplateNameV4,
|
||||||
NeighborIP: "94.142.140.141",
|
NeighborIP: "94.142.140.141",
|
||||||
NeighborASN: 65002,
|
NeighborASN: 65002,
|
||||||
SourceAddress: "77.232.38.173",
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
@@ -52,7 +51,7 @@ func TestRenderProtocolBGPFromTemplate_MultihopPassive(t *testing.T) {
|
|||||||
if !strings.Contains(got, "multihop;") || !strings.Contains(got, "passive;") {
|
if !strings.Contains(got, "multihop;") || !strings.Contains(got, "passive;") {
|
||||||
t.Fatal(got)
|
t.Fatal(got)
|
||||||
}
|
}
|
||||||
if !strings.Contains(got, "source address 77.232.38.173;") {
|
if strings.Contains(got, "source address") || strings.Contains(got, "strict bind") {
|
||||||
t.Fatal(got)
|
t.Fatal(got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -63,7 +62,6 @@ func TestRenderProtocolBGPFromTemplate_OverrideLocal(t *testing.T) {
|
|||||||
TemplateName: BGPTemplateNameV4,
|
TemplateName: BGPTemplateNameV4,
|
||||||
NeighborIP: "192.0.2.2",
|
NeighborIP: "192.0.2.2",
|
||||||
NeighborASN: 2,
|
NeighborASN: 2,
|
||||||
SourceAddress: "10.0.0.1",
|
|
||||||
OverrideLocalIP: "10.0.0.1",
|
OverrideLocalIP: "10.0.0.1",
|
||||||
OverrideLocalASN: 65099,
|
OverrideLocalASN: 65099,
|
||||||
})
|
})
|
||||||
|
|||||||
@@ -10,6 +10,10 @@ import (
|
|||||||
|
|
||||||
var standardPairRE = regexp.MustCompile(`^\s*(\d{1,5}):(\d{1,5})\s*$`)
|
var standardPairRE = regexp.MustCompile(`^\s*(\d{1,5}):(\d{1,5})\s*$`)
|
||||||
|
|
||||||
|
// badCommunityAddRE matches invalid BIRD filter calls like bgp_community.add(65000,1)
|
||||||
|
// (first argument is an int). Valid form is bgp_community.add((65000,1)); — note the pair tuple.
|
||||||
|
var badCommunityAddRE = regexp.MustCompile(`\b(?:bgp_community|bgp_large_community)\.add\(\s*\d`)
|
||||||
|
|
||||||
// RouteCommunityAttrs returns BIRD 2 lines (indented with 4 spaces) for use inside a static `route … { … }` block.
|
// RouteCommunityAttrs returns BIRD 2 lines (indented with 4 spaces) for use inside a static `route … { … }` block.
|
||||||
// Empty string means no community attributes.
|
// Empty string means no community attributes.
|
||||||
func RouteCommunityAttrs(kind, name, valueJSON string) (string, error) {
|
func RouteCommunityAttrs(kind, name, valueJSON string) (string, error) {
|
||||||
@@ -20,26 +24,46 @@ func RouteCommunityAttrs(kind, name, valueJSON string) (string, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if s, ok := parseLargeFromJSONLoose(raw); ok {
|
if s, ok := parseLargeFromJSONLoose(raw); ok {
|
||||||
|
if err := validateCommunityRouteBody(s); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
return s, nil
|
return s, nil
|
||||||
}
|
}
|
||||||
_ = kind // reserved for future kinds (e.g. extended communities)
|
_ = kind // reserved for future kinds (e.g. extended communities)
|
||||||
|
|
||||||
if m := standardPairRE.FindStringSubmatch(name); len(m) == 3 {
|
if m := standardPairRE.FindStringSubmatch(name); len(m) == 3 {
|
||||||
return standardAddLine(m[1], m[2])
|
s, err := standardAddLine(m[1], m[2])
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return s, validateCommunityRouteBody(s)
|
||||||
}
|
}
|
||||||
|
|
||||||
var obj map[string]any
|
var obj map[string]any
|
||||||
if json.Unmarshal([]byte(raw), &obj) == nil {
|
if json.Unmarshal([]byte(raw), &obj) == nil {
|
||||||
if v, ok := obj["standard"].(string); ok {
|
if v, ok := obj["standard"].(string); ok {
|
||||||
if m := standardPairRE.FindStringSubmatch(v); len(m) == 3 {
|
if m := standardPairRE.FindStringSubmatch(v); len(m) == 3 {
|
||||||
return standardAddLine(m[1], m[2])
|
s, err := standardAddLine(m[1], m[2])
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return s, validateCommunityRouteBody(s)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if arr, ok := obj["standard"].([]any); ok && len(arr) == 2 {
|
if arr, ok := obj["standard"].([]any); ok {
|
||||||
a, aok := numberToUint32String(arr[0])
|
if len(arr) == 1 {
|
||||||
b, bok := numberToUint32String(arr[1])
|
return "", fmt.Errorf(`birdfmt: value_json "standard" must be a pair [a,b] or "a:b" string, got single element`)
|
||||||
if aok && bok {
|
}
|
||||||
return standardAddLine(a, b)
|
if len(arr) == 2 {
|
||||||
|
a, aok := numberToUint32String(arr[0])
|
||||||
|
b, bok := numberToUint32String(arr[1])
|
||||||
|
if aok && bok {
|
||||||
|
s, err := standardAddLine(a, b)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return s, validateCommunityRouteBody(s)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -47,13 +71,36 @@ func RouteCommunityAttrs(kind, name, valueJSON string) (string, error) {
|
|||||||
var strVal string
|
var strVal string
|
||||||
if json.Unmarshal([]byte(raw), &strVal) == nil {
|
if json.Unmarshal([]byte(raw), &strVal) == nil {
|
||||||
if m := standardPairRE.FindStringSubmatch(strVal); len(m) == 3 {
|
if m := standardPairRE.FindStringSubmatch(strVal); len(m) == 3 {
|
||||||
return standardAddLine(m[1], m[2])
|
s, err := standardAddLine(m[1], m[2])
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
return s, validateCommunityRouteBody(s)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return "", nil
|
return "", nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// validateCommunityRouteBody rejects lines that would make BIRD 2 fail parse-check with
|
||||||
|
// "Argument 1 of 'add' expected clist / quad / ip / pair, got int" (wrong add() shape).
|
||||||
|
func validateCommunityRouteBody(body string) error {
|
||||||
|
body = strings.TrimSpace(body)
|
||||||
|
if body == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
for _, line := range strings.Split(body, "\n") {
|
||||||
|
line = strings.TrimSpace(line)
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if badCommunityAddRE.MatchString(line) {
|
||||||
|
return fmt.Errorf("birdfmt: invalid community line (expected pair/triple tuple inside add(), e.g. bgp_community.add((65000,1));): %q", line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
func standardAddLine(a, b string) (string, error) {
|
func standardAddLine(a, b string) (string, error) {
|
||||||
ai, err1 := strconv.ParseUint(strings.TrimSpace(a), 10, 16)
|
ai, err1 := strconv.ParseUint(strings.TrimSpace(a), 10, 16)
|
||||||
bi, err2 := strconv.ParseUint(strings.TrimSpace(b), 10, 16)
|
bi, err2 := strconv.ParseUint(strings.TrimSpace(b), 10, 16)
|
||||||
@@ -111,6 +158,11 @@ func numberToUint32String(v any) (string, bool) {
|
|||||||
return "", false
|
return "", false
|
||||||
}
|
}
|
||||||
return strconv.FormatUint(uint64(x), 10), true
|
return strconv.FormatUint(uint64(x), 10), true
|
||||||
|
case int64:
|
||||||
|
if x < 0 || x > 4294967295 {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
return strconv.FormatUint(uint64(x), 10), true
|
||||||
case string:
|
case string:
|
||||||
n, err := strconv.ParseUint(strings.TrimSpace(x), 10, 32)
|
n, err := strconv.ParseUint(strings.TrimSpace(x), 10, 32)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -21,3 +21,10 @@ func TestRouteCommunityAttrs_LargeJSON(t *testing.T) {
|
|||||||
t.Fatalf("got %q", s)
|
t.Fatalf("got %q", s)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRouteCommunityAttrs_StandardArrayOneElement(t *testing.T) {
|
||||||
|
_, err := RouteCommunityAttrs("", "x", `{"standard":[65000]}`)
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected error for single-element standard pair")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,3 +1,5 @@
|
|||||||
|
// Engineering rules for BIRD/BGP/IP: .cursor/rules/networking-bird.mdc
|
||||||
|
//
|
||||||
// Package birdfmt builds BIRD 2 configuration text: main bird.conf skeleton, include
|
// Package birdfmt builds BIRD 2 configuration text: main bird.conf skeleton, include
|
||||||
// fragments under bird.d/, export filters, static route protocols, BGP templates, and
|
// fragments under bird.d/, export filters, static route protocols, BGP templates, and
|
||||||
// BGP peer protocols (from template or standalone). Runtime helpers parse-check configs
|
// BGP peer protocols (from template or standalone). Runtime helpers parse-check configs
|
||||||
|
|||||||
@@ -40,11 +40,10 @@ func TestStandardLayout_GeneratorMatchesFixtures(t *testing.T) {
|
|||||||
assertFileEquals(t, "testdata/scenarios/standard_layout/bird.d/evobgp_bgp_template.conf", tpl)
|
assertFileEquals(t, "testdata/scenarios/standard_layout/bird.d/evobgp_bgp_template.conf", tpl)
|
||||||
|
|
||||||
peer, err := RenderProtocolBGPFromTemplate(BGPPeerFromTemplateOptions{
|
peer, err := RenderProtocolBGPFromTemplate(BGPPeerFromTemplateOptions{
|
||||||
ProtocolName: "evobgp_peer_ci",
|
ProtocolName: "evobgp_peer_ci",
|
||||||
TemplateName: BGPTemplateNameV4,
|
TemplateName: BGPTemplateNameV4,
|
||||||
NeighborIP: "192.0.2.2",
|
NeighborIP: "192.0.2.2",
|
||||||
NeighborASN: 65002,
|
NeighborASN: 65002,
|
||||||
SourceAddress: "192.0.2.1",
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
|
|||||||
-2
@@ -6,7 +6,6 @@ template bgp bgp_template {
|
|||||||
};
|
};
|
||||||
hold time 90;
|
hold time 90;
|
||||||
keepalive time 30;
|
keepalive time 30;
|
||||||
strict bind on;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
template bgp bgp_template_v6 {
|
template bgp bgp_template_v6 {
|
||||||
@@ -17,5 +16,4 @@ template bgp bgp_template_v6 {
|
|||||||
};
|
};
|
||||||
hold time 90;
|
hold time 90;
|
||||||
keepalive time 30;
|
keepalive time 30;
|
||||||
strict bind on;
|
|
||||||
}
|
}
|
||||||
@@ -1,6 +1,5 @@
|
|||||||
protocol bgp evobgp_peer_ci from bgp_template {
|
protocol bgp evobgp_peer_ci from bgp_template {
|
||||||
neighbor 192.0.2.2 as 65002;
|
neighbor 192.0.2.2 as 65002;
|
||||||
multihop;
|
multihop;
|
||||||
source address 192.0.2.1;
|
|
||||||
passive;
|
passive;
|
||||||
}
|
}
|
||||||
@@ -15,6 +15,11 @@ import (
|
|||||||
"github.com/jackc/pgx/v5/pgxpool"
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// NewCDNHTTPClient returns the shared HTTP client for CDN and preview fetches (PERF-02 / ERR-03).
|
||||||
|
func NewCDNHTTPClient() *http.Client {
|
||||||
|
return &http.Client{Timeout: 45 * time.Second}
|
||||||
|
}
|
||||||
|
|
||||||
// BootstrapWorkers opens the same store.Backend and jobs.Registry as New (without HTTP or bundle keys).
|
// BootstrapWorkers opens the same store.Backend and jobs.Registry as New (without HTTP or bundle keys).
|
||||||
// Used by standalone worker binaries (scheduler, ingest, …) that share PostgreSQL with the API.
|
// Used by standalone worker binaries (scheduler, ingest, …) that share PostgreSQL with the API.
|
||||||
func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.Registry, *pgxpool.Pool, error) {
|
func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.Registry, *pgxpool.Pool, error) {
|
||||||
@@ -44,9 +49,10 @@ func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.R
|
|||||||
backend = mem
|
backend = mem
|
||||||
}
|
}
|
||||||
|
|
||||||
cdnHTTP := &http.Client{Timeout: 45 * time.Second}
|
cdnHTTP := NewCDNHTTPClient()
|
||||||
wk := &jobs.Worker{Store: backend, HTTPClient: cdnHTTP}
|
wk := &jobs.Worker{Store: backend, HTTPClient: cdnHTTP}
|
||||||
reg := jobs.NewRegistry(wk.Process)
|
reg := jobs.NewRegistry(wk.Process)
|
||||||
|
wk.Registry = reg
|
||||||
observability.RegisterStoreBackend(backend)
|
observability.RegisterStoreBackend(backend)
|
||||||
return backend, reg, pool, nil
|
return backend, reg, pool, nil
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
package httpapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"strconv"
|
||||||
|
)
|
||||||
|
|
||||||
|
func parseListLimit(r *http.Request) int {
|
||||||
|
limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
|
||||||
|
if limit <= 0 {
|
||||||
|
return 50
|
||||||
|
}
|
||||||
|
if limit > 500 {
|
||||||
|
return 500
|
||||||
|
}
|
||||||
|
return limit
|
||||||
|
}
|
||||||
|
|
||||||
|
func strPtrOrNull(s string) any {
|
||||||
|
if s == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return s
|
||||||
|
}
|
||||||
@@ -2,9 +2,15 @@ package httpapi
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"log"
|
||||||
"net/http"
|
"net/http"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
const (
|
||||||
|
internalErrorDetail = "an internal error occurred"
|
||||||
|
badGatewayDetail = "upstream request failed"
|
||||||
|
)
|
||||||
|
|
||||||
// Problem is RFC 9457 application/problem+json.
|
// Problem is RFC 9457 application/problem+json.
|
||||||
type Problem struct {
|
type Problem struct {
|
||||||
Type string `json:"type,omitempty"`
|
Type string `json:"type,omitempty"`
|
||||||
@@ -34,3 +40,19 @@ func writeJSON(w http.ResponseWriter, status int, v any) {
|
|||||||
func writeNoContent(w http.ResponseWriter) {
|
func writeNoContent(w http.ResponseWriter) {
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// writeInternalError logs err server-side and returns a generic 500 problem (ERR-01).
|
||||||
|
func writeInternalError(w http.ResponseWriter, operation string, err error) {
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("httpapi: %s: %v", operation, err)
|
||||||
|
}
|
||||||
|
writeProblem(w, http.StatusInternalServerError, "Internal Error", internalErrorDetail)
|
||||||
|
}
|
||||||
|
|
||||||
|
// writeBadGateway logs err server-side and returns a generic 502 problem (ERR-01).
|
||||||
|
func writeBadGateway(w http.ResponseWriter, operation string, err error) {
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("httpapi: %s: %v", operation, err)
|
||||||
|
}
|
||||||
|
writeProblem(w, http.StatusBadGateway, "Bad Gateway", badGatewayDetail)
|
||||||
|
}
|
||||||
+259
-31
@@ -18,7 +18,10 @@ import (
|
|||||||
"evobgp/internal/bundle"
|
"evobgp/internal/bundle"
|
||||||
"evobgp/internal/jobs"
|
"evobgp/internal/jobs"
|
||||||
"evobgp/internal/observability"
|
"evobgp/internal/observability"
|
||||||
|
"evobgp/internal/pipeline"
|
||||||
|
"evobgp/internal/reports"
|
||||||
"evobgp/internal/store"
|
"evobgp/internal/store"
|
||||||
|
"evobgp/internal/version"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Handler returns the root HTTP handler (system routes public; rest under /v1/ authenticated).
|
// Handler returns the root HTTP handler (system routes public; rest under /v1/ authenticated).
|
||||||
@@ -28,6 +31,7 @@ func (s *Server) Handler() http.Handler {
|
|||||||
wrappedV1 := http.StripPrefix("/v1", v1)
|
wrappedV1 := http.StripPrefix("/v1", v1)
|
||||||
|
|
||||||
s.mux.Handle("GET /metrics", observability.MetricsHandler())
|
s.mux.Handle("GET /metrics", observability.MetricsHandler())
|
||||||
|
s.mux.HandleFunc("GET /version", s.handleVersion)
|
||||||
s.mux.HandleFunc("GET /v1/health", s.handleHealth)
|
s.mux.HandleFunc("GET /v1/health", s.handleHealth)
|
||||||
s.mux.HandleFunc("GET /v1/ready", s.handleReady)
|
s.mux.HandleFunc("GET /v1/ready", s.handleReady)
|
||||||
s.mux.HandleFunc("GET /v1/version", s.handleVersion)
|
s.mux.HandleFunc("GET /v1/version", s.handleVersion)
|
||||||
@@ -47,13 +51,16 @@ func (s *Server) registerRoutes() {
|
|||||||
|
|
||||||
func (s *Server) registerV1(m *http.ServeMux) {
|
func (s *Server) registerV1(m *http.ServeMux) {
|
||||||
m.HandleFunc("GET /modules", s.handleListModules)
|
m.HandleFunc("GET /modules", s.handleListModules)
|
||||||
|
m.HandleFunc("GET /router-lists/catalog", s.handleRouterListsCatalog)
|
||||||
m.HandleFunc("GET /modules/{module_id}", s.handleGetModule)
|
m.HandleFunc("GET /modules/{module_id}", s.handleGetModule)
|
||||||
m.HandleFunc("GET /peers", s.handleListPeers)
|
m.HandleFunc("GET /peers", s.handleListPeers)
|
||||||
m.HandleFunc("GET /speakers", s.handleListSpeakers)
|
m.HandleFunc("GET /speakers", s.handleListSpeakers)
|
||||||
m.HandleFunc("POST /modules/{module_id}/refresh", s.handleModuleRefresh)
|
m.HandleFunc("POST /modules/{module_id}/refresh", s.handleModuleRefresh)
|
||||||
|
m.HandleFunc("POST /tenant/refresh", s.handleTenantRefresh)
|
||||||
m.HandleFunc("GET /revisions", s.handleListRevisions)
|
m.HandleFunc("GET /revisions", s.handleListRevisions)
|
||||||
m.HandleFunc("GET /revisions/{revision_id}", s.handleGetRevision)
|
m.HandleFunc("GET /revisions/{revision_id}", s.handleGetRevision)
|
||||||
m.HandleFunc("GET /revisions/{revision_id}/preview", s.handleRevisionPreview)
|
m.HandleFunc("GET /revisions/{revision_id}/preview", s.handleRevisionPreview)
|
||||||
|
m.HandleFunc("GET /revisions/{revision_id}/diagnostic-log", s.handleRevisionDiagnosticLog)
|
||||||
m.HandleFunc("GET /revisions/{revision_a}/diff/{revision_b}", s.handleRevisionDiff)
|
m.HandleFunc("GET /revisions/{revision_a}/diff/{revision_b}", s.handleRevisionDiff)
|
||||||
m.HandleFunc("POST /revisions/{revision_id}/rollback", s.handleRevisionRollback)
|
m.HandleFunc("POST /revisions/{revision_id}/rollback", s.handleRevisionRollback)
|
||||||
m.HandleFunc("POST /apply", s.handleApply)
|
m.HandleFunc("POST /apply", s.handleApply)
|
||||||
@@ -79,7 +86,7 @@ func (s *Server) handleReady(w http.ResponseWriter, r *http.Request) {
|
|||||||
defer cancel()
|
defer cancel()
|
||||||
if s.pgPool != nil {
|
if s.pgPool != nil {
|
||||||
if err := s.pgPool.Ping(ctx); err != nil {
|
if err := s.pgPool.Ping(ctx); err != nil {
|
||||||
checks["postgres"] = err.Error()
|
checks["postgres"] = "unavailable"
|
||||||
writeJSON(w, http.StatusServiceUnavailable, map[string]any{"status": "not_ready", "checks": checks})
|
writeJSON(w, http.StatusServiceUnavailable, map[string]any{"status": "not_ready", "checks": checks})
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -91,11 +98,16 @@ func (s *Server) handleReady(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (s *Server) handleVersion(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleVersion(w http.ResponseWriter, r *http.Request) {
|
||||||
sha := strings.TrimSpace(os.Getenv("EVOBGP_GIT_SHA"))
|
ver, sha, buildTime := version.Info()
|
||||||
if sha == "" {
|
resp := map[string]string{
|
||||||
sha = "unknown"
|
"version": ver,
|
||||||
|
"api_version": ver,
|
||||||
|
"git_sha": sha,
|
||||||
}
|
}
|
||||||
writeJSON(w, http.StatusOK, map[string]string{"api_version": "0.1.0", "git_sha": sha})
|
if buildTime != "" {
|
||||||
|
resp["build_time"] = buildTime
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, resp)
|
||||||
}
|
}
|
||||||
|
|
||||||
func moduleJSON(mod *store.Module) map[string]any {
|
func moduleJSON(mod *store.Module) map[string]any {
|
||||||
@@ -108,11 +120,23 @@ func moduleJSON(mod *store.Module) map[string]any {
|
|||||||
"refresh_interval_sec": mod.RefreshIntervalSec,
|
"refresh_interval_sec": mod.RefreshIntervalSec,
|
||||||
"cron_expr": mod.CronExpr,
|
"cron_expr": mod.CronExpr,
|
||||||
}
|
}
|
||||||
|
if mod.LastRefreshedAt != nil {
|
||||||
|
m["last_refreshed_at"] = mod.LastRefreshedAt.UTC().Format(time.RFC3339Nano)
|
||||||
|
} else {
|
||||||
|
m["last_refreshed_at"] = nil
|
||||||
|
}
|
||||||
if mod.DefaultCommunityID != nil {
|
if mod.DefaultCommunityID != nil {
|
||||||
m["default_community_id"] = *mod.DefaultCommunityID
|
m["default_community_id"] = *mod.DefaultCommunityID
|
||||||
} else {
|
} else {
|
||||||
m["default_community_id"] = nil
|
m["default_community_id"] = nil
|
||||||
}
|
}
|
||||||
|
ids := mod.EffectiveDohProfileIDs()
|
||||||
|
if len(ids) > 0 {
|
||||||
|
m["doh_profile_ids"] = ids
|
||||||
|
} else {
|
||||||
|
m["doh_profile_ids"] = []string{}
|
||||||
|
}
|
||||||
|
m["doh_resolver_policy"] = store.NormalizeDohResolverPolicy(mod.DohResolverPolicy)
|
||||||
if mod.DohProfileID != nil {
|
if mod.DohProfileID != nil {
|
||||||
m["doh_profile_id"] = *mod.DohProfileID
|
m["doh_profile_id"] = *mod.DohProfileID
|
||||||
} else {
|
} else {
|
||||||
@@ -161,13 +185,58 @@ func (s *Server) handleListModules(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !s.requireAtLeast(w, a, "viewer") {
|
if !s.requireAtLeast(w, a, "viewer") {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
mods := s.store.ListModules(a.TenantID)
|
typeFilter := strings.TrimSpace(r.URL.Query().Get("type"))
|
||||||
items := make([]map[string]any, 0, len(mods))
|
enabledRaw := strings.TrimSpace(r.URL.Query().Get("enabled"))
|
||||||
for _, mod := range mods {
|
var enabledFilter *bool
|
||||||
|
if enabledRaw != "" {
|
||||||
|
v, err := strconv.ParseBool(enabledRaw)
|
||||||
|
if err != nil {
|
||||||
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "enabled must be boolean")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
enabledFilter = &v
|
||||||
|
}
|
||||||
|
|
||||||
|
filtered := make([]*store.Module, 0)
|
||||||
|
for _, mod := range s.store.ListModules(a.TenantID) {
|
||||||
|
if typeFilter != "" && mod.Type != typeFilter {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if enabledFilter != nil && mod.Enabled != *enabledFilter {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
filtered = append(filtered, mod)
|
||||||
|
}
|
||||||
|
page, next, more := store.PaginateOffset(filtered, r.URL.Query().Get("cursor"), parseListLimit(r))
|
||||||
|
items := make([]map[string]any, 0, len(page))
|
||||||
|
for _, mod := range page {
|
||||||
items = append(items, moduleJSON(mod))
|
items = append(items, moduleJSON(mod))
|
||||||
}
|
}
|
||||||
writeJSON(w, http.StatusOK, map[string]any{
|
writeJSON(w, http.StatusOK, map[string]any{
|
||||||
"items": items, "next_cursor": nil, "has_more": false,
|
"items": items, "next_cursor": strPtrOrNull(next), "has_more": more,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleRouterListsCatalog(w http.ResponseWriter, r *http.Request) {
|
||||||
|
a, ok := authFromContext(r.Context())
|
||||||
|
if !ok {
|
||||||
|
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.requireAtLeast(w, a, "viewer") {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
cat, err := reports.BuildRouterListsCatalog(s.store, a.TenantID)
|
||||||
|
if err != nil {
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, map[string]any{
|
||||||
|
"modules": map[string]any{"items": cat.Modules},
|
||||||
|
"domains": map[string]any{"items": cat.Domains},
|
||||||
|
"asns": map[string]any{"items": cat.ASNs},
|
||||||
|
"ip_ranges": map[string]any{"items": cat.IPRanges},
|
||||||
|
"communities": map[string]any{"items": cat.Communities},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -186,7 +255,7 @@ func (s *Server) handleGetModule(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusNotFound, "Not Found", "module not found")
|
writeProblem(w, http.StatusNotFound, "Not Found", "module not found")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
writeJSON(w, http.StatusOK, moduleJSON(mod))
|
writeJSON(w, http.StatusOK, moduleJSON(mod))
|
||||||
@@ -201,16 +270,91 @@ func (s *Server) handleListPeers(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !s.requireAtLeast(w, a, "viewer") {
|
if !s.requireAtLeast(w, a, "viewer") {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
peers := s.store.ListPeers(a.TenantID)
|
allPeers := s.store.ListPeers(a.TenantID)
|
||||||
items := make([]map[string]any, 0, len(peers))
|
page, next, more := store.PaginateOffset(allPeers, r.URL.Query().Get("cursor"), parseListLimit(r))
|
||||||
for _, p := range peers {
|
liveStates := s.liveBGPProtocolStates(r.Context())
|
||||||
items = append(items, peerJSON(p))
|
items := make([]map[string]any, 0, len(page))
|
||||||
|
for _, p := range page {
|
||||||
|
row := peerJSON(p)
|
||||||
|
if st, ok := liveStates[peerProtocolNameForID(p.ID)]; ok && strings.TrimSpace(st) != "" {
|
||||||
|
row["session_state"] = strings.TrimSpace(st)
|
||||||
|
}
|
||||||
|
items = append(items, row)
|
||||||
}
|
}
|
||||||
writeJSON(w, http.StatusOK, map[string]any{
|
writeJSON(w, http.StatusOK, map[string]any{
|
||||||
"items": items, "next_cursor": nil, "has_more": false,
|
"items": items, "next_cursor": strPtrOrNull(next), "has_more": more,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *Server) liveBGPProtocolStates(ctx context.Context) map[string]string {
|
||||||
|
sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET"))
|
||||||
|
if sock == "" {
|
||||||
|
return map[string]string{}
|
||||||
|
}
|
||||||
|
out, err := birdfmt.ShowProtocols(ctx, sock, strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN")))
|
||||||
|
if err != nil {
|
||||||
|
return map[string]string{}
|
||||||
|
}
|
||||||
|
return parseBGPProtocolStates(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
// parseBGPProtocolStates parses `birdc show protocols all` summary rows into protocol_name -> state.
|
||||||
|
func parseBGPProtocolStates(output string) map[string]string {
|
||||||
|
out := make(map[string]string)
|
||||||
|
for _, raw := range strings.Split(output, "\n") {
|
||||||
|
line := strings.TrimSpace(raw)
|
||||||
|
if line == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
low := strings.ToLower(line)
|
||||||
|
if strings.HasPrefix(low, "bird ") || strings.HasPrefix(low, "name ") || strings.HasPrefix(low, "table ") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
fields := strings.Fields(line)
|
||||||
|
if len(fields) < 4 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !strings.EqualFold(fields[1], "BGP") {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
state := extractBGPSessionState(line)
|
||||||
|
if state == "" {
|
||||||
|
state = fields[3]
|
||||||
|
}
|
||||||
|
out[fields[0]] = state
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func extractBGPSessionState(line string) string {
|
||||||
|
known := []string{
|
||||||
|
"Established",
|
||||||
|
"Idle",
|
||||||
|
"Connect",
|
||||||
|
"Active",
|
||||||
|
"OpenSent",
|
||||||
|
"OpenConfirm",
|
||||||
|
}
|
||||||
|
for _, st := range known {
|
||||||
|
if strings.Contains(line, st) {
|
||||||
|
return st
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return ""
|
||||||
|
}
|
||||||
|
|
||||||
|
// peerProtocolNameForID must stay in sync with pipeline peer protocol naming.
|
||||||
|
func peerProtocolNameForID(peerID string) string {
|
||||||
|
s := strings.ReplaceAll(strings.TrimSpace(peerID), "-", "")
|
||||||
|
if len(s) > 16 {
|
||||||
|
s = s[:16]
|
||||||
|
}
|
||||||
|
if s == "" {
|
||||||
|
s = "x"
|
||||||
|
}
|
||||||
|
return "evobgp_p_" + s
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleListSpeakers(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleListSpeakers(w http.ResponseWriter, r *http.Request) {
|
||||||
a, ok := authFromContext(r.Context())
|
a, ok := authFromContext(r.Context())
|
||||||
if !ok {
|
if !ok {
|
||||||
@@ -246,7 +390,7 @@ func (s *Server) handleModuleRefresh(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusNotFound, "Not Found", "module not found")
|
writeProblem(w, http.StatusNotFound, "Not Found", "module not found")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
idem := r.Header.Get("Idempotency-Key")
|
idem := r.Header.Get("Idempotency-Key")
|
||||||
@@ -258,7 +402,7 @@ func (s *Server) handleModuleRefresh(w http.ResponseWriter, r *http.Request) {
|
|||||||
mid := mod.ID
|
mid := mod.ID
|
||||||
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindModuleRefresh, idemPtr, &mid, map[string]any{"module_id": moduleID})
|
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindModuleRefresh, idemPtr, &mid, map[string]any{"module_id": moduleID})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||||
@@ -266,6 +410,60 @@ func (s *Server) handleModuleRefresh(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
|
writeJSON(w, http.StatusAccepted, map[string]any{"job_id": snap["job_id"], "status": snap["status"]})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleTenantRefresh(w http.ResponseWriter, r *http.Request) {
|
||||||
|
a, ok := authFromContext(r.Context())
|
||||||
|
if !ok {
|
||||||
|
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.requireAtLeast(w, a, "editor") {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var body struct {
|
||||||
|
ModuleIDs []string `json:"module_ids"`
|
||||||
|
}
|
||||||
|
if r.Body != nil && r.ContentLength != 0 {
|
||||||
|
if err := json.NewDecoder(io.LimitReader(r.Body, 1<<20)).Decode(&body); err != nil && err != io.EOF {
|
||||||
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid JSON body")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
moduleIDs := body.ModuleIDs
|
||||||
|
if len(moduleIDs) == 0 {
|
||||||
|
now := time.Now().UTC()
|
||||||
|
for _, mod := range s.store.ListModules(a.TenantID) {
|
||||||
|
if pipeline.ModuleDueForScheduler(mod, now) {
|
||||||
|
moduleIDs = append(moduleIDs, mod.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(moduleIDs) == 0 {
|
||||||
|
writeJSON(w, http.StatusNoContent, map[string]any{"message": "no modules due for refresh"})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
idem := strings.TrimSpace(r.Header.Get("Idempotency-Key"))
|
||||||
|
var idemPtr *string
|
||||||
|
if idem != "" {
|
||||||
|
idemPtr = &idem
|
||||||
|
}
|
||||||
|
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindTenantRefresh, idemPtr, nil, map[string]any{
|
||||||
|
"module_ids": moduleIDs,
|
||||||
|
"trigger": "api",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
writeInternalError(w, "internal", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||||
|
snap := j.Snapshot()
|
||||||
|
writeJSON(w, http.StatusAccepted, map[string]any{
|
||||||
|
"job_id": snap["job_id"],
|
||||||
|
"status": snap["status"],
|
||||||
|
"module_ids": moduleIDs,
|
||||||
|
"modules_due": len(moduleIDs),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleListRevisions(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleListRevisions(w http.ResponseWriter, r *http.Request) {
|
||||||
a, ok := authFromContext(r.Context())
|
a, ok := authFromContext(r.Context())
|
||||||
if !ok {
|
if !ok {
|
||||||
@@ -311,13 +509,6 @@ func revisionJSON(rev *store.Revision) map[string]any {
|
|||||||
return m
|
return m
|
||||||
}
|
}
|
||||||
|
|
||||||
func strPtrOrNull(s string) any {
|
|
||||||
if s == "" {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
return s
|
|
||||||
}
|
|
||||||
|
|
||||||
// enqueueModuleRefreshIfEnabled queues module_refresh when the module exists and is enabled (best-effort, no HTTP error).
|
// enqueueModuleRefreshIfEnabled queues module_refresh when the module exists and is enabled (best-effort, no HTTP error).
|
||||||
func (s *Server) enqueueModuleRefreshIfEnabled(tenantID, moduleID, trigger string) {
|
func (s *Server) enqueueModuleRefreshIfEnabled(tenantID, moduleID, trigger string) {
|
||||||
if s.jobs == nil {
|
if s.jobs == nil {
|
||||||
@@ -334,6 +525,17 @@ func (s *Server) enqueueModuleRefreshIfEnabled(tenantID, moduleID, trigger strin
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// enqueuePeerReconcile queues fast peer-only reconcile/render (best-effort, no HTTP error).
|
||||||
|
func (s *Server) enqueuePeerReconcile(tenantID, trigger string) {
|
||||||
|
if s.jobs == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _, _ = s.jobs.Enqueue(tenantID, jobs.KindPeerReconcile, nil, nil, map[string]any{
|
||||||
|
"trigger": trigger,
|
||||||
|
"job_title": "Обновление BGP пиров",
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleGetRevision(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleGetRevision(w http.ResponseWriter, r *http.Request) {
|
||||||
a, ok := authFromContext(r.Context())
|
a, ok := authFromContext(r.Context())
|
||||||
if !ok {
|
if !ok {
|
||||||
@@ -387,6 +589,32 @@ func (s *Server) handleRevisionPreview(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeJSON(w, http.StatusOK, obj)
|
writeJSON(w, http.StatusOK, obj)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleRevisionDiagnosticLog(w http.ResponseWriter, r *http.Request) {
|
||||||
|
a, ok := authFromContext(r.Context())
|
||||||
|
if !ok {
|
||||||
|
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if !s.requireAtLeast(w, a, "viewer") {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
revID := r.PathValue("revision_id")
|
||||||
|
rev, err := s.store.GetRevision(a.TenantID, revID)
|
||||||
|
if err != nil {
|
||||||
|
writeProblem(w, http.StatusNotFound, "Not Found", "revision not found")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
logOut, err := reports.BuildRevisionDiagnosticLog(s.store, a.TenantID, rev)
|
||||||
|
if err != nil {
|
||||||
|
writeInternalError(w, "revision diagnostic log", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||||
|
w.Header().Set("Content-Disposition", `attachment; filename="`+logOut.Filename+`"`)
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
_, _ = w.Write(logOut.Body)
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleRevisionDiff(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleRevisionDiff(w http.ResponseWriter, r *http.Request) {
|
||||||
a, ok := authFromContext(r.Context())
|
a, ok := authFromContext(r.Context())
|
||||||
if !ok {
|
if !ok {
|
||||||
@@ -410,7 +638,7 @@ func (s *Server) handleRevisionRollback(w http.ResponseWriter, r *http.Request)
|
|||||||
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if !s.requireAtLeast(w, a, "editor") {
|
if !s.requireAtLeast(w, a, "operator") {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
revID := r.PathValue("revision_id")
|
revID := r.PathValue("revision_id")
|
||||||
@@ -427,7 +655,7 @@ func (s *Server) handleRevisionRollback(w http.ResponseWriter, r *http.Request)
|
|||||||
"source_revision_id": revID,
|
"source_revision_id": revID,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||||
@@ -474,7 +702,7 @@ func (s *Server) handleApply(w http.ResponseWriter, r *http.Request) {
|
|||||||
"strategy": body.Strategy,
|
"strategy": body.Strategy,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||||
@@ -525,7 +753,7 @@ func (s *Server) handleSpeakerApply(w http.ResponseWriter, r *http.Request) {
|
|||||||
"speaker_id": spkID,
|
"speaker_id": spkID,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||||
@@ -550,7 +778,7 @@ func (s *Server) handleBirdReload(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindBirdReload, idemPtr, nil, nil)
|
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindBirdReload, idemPtr, nil, nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
w.Header().Set("Location", "/v1/jobs/"+j.ID)
|
||||||
@@ -685,7 +913,7 @@ func (s *Server) handleNodeBundle(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
tgz, err := bundle.BuildGzippedTar(rid, sid, rev.PreviewFragments, s.bundlePriv)
|
tgz, err := bundle.BuildGzippedTar(rid, sid, rev.PreviewFragments, s.bundlePriv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
w.Header().Set("Content-Type", "application/gzip")
|
w.Header().Set("Content-Type", "application/gzip")
|
||||||
@@ -735,7 +963,7 @@ func (s *Server) handleNodeEnroll(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
patch := &store.SpeakerPatch{MetaJSON: &meta}
|
patch := &store.SpeakerPatch{MetaJSON: &meta}
|
||||||
if _, err := s.store.UpdateSpeaker(a.TenantID, sid, patch); err != nil {
|
if _, err := s.store.UpdateSpeaker(a.TenantID, sid, patch); err != nil {
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "internal", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
writeJSON(w, http.StatusOK, map[string]any{
|
writeJSON(w, http.StatusOK, map[string]any{
|
||||||
|
|||||||
+330
-21
@@ -1,12 +1,18 @@
|
|||||||
package httpapi
|
package httpapi
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"encoding/csv"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/importer"
|
||||||
|
"evobgp/internal/pipeline"
|
||||||
"evobgp/internal/store"
|
"evobgp/internal/store"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -17,6 +23,7 @@ func (s *Server) registerCRUDRoutes(m *http.ServeMux) {
|
|||||||
|
|
||||||
m.HandleFunc("GET /modules/{module_id}/cdn-sources", s.handleListCDNSources)
|
m.HandleFunc("GET /modules/{module_id}/cdn-sources", s.handleListCDNSources)
|
||||||
m.HandleFunc("POST /modules/{module_id}/cdn-sources", s.handlePostCDNSource)
|
m.HandleFunc("POST /modules/{module_id}/cdn-sources", s.handlePostCDNSource)
|
||||||
|
m.HandleFunc("POST /modules/{module_id}/cdn-sources/preview", s.handlePreviewCDNSource)
|
||||||
m.HandleFunc("PATCH /modules/{module_id}/cdn-sources/{source_id}", s.handlePatchCDNSource)
|
m.HandleFunc("PATCH /modules/{module_id}/cdn-sources/{source_id}", s.handlePatchCDNSource)
|
||||||
m.HandleFunc("DELETE /modules/{module_id}/cdn-sources/{source_id}", s.handleDeleteCDNSource)
|
m.HandleFunc("DELETE /modules/{module_id}/cdn-sources/{source_id}", s.handleDeleteCDNSource)
|
||||||
|
|
||||||
@@ -34,6 +41,8 @@ func (s *Server) registerCRUDRoutes(m *http.ServeMux) {
|
|||||||
m.HandleFunc("POST /modules/{module_id}/ip-range-entries", s.handlePostIPRange)
|
m.HandleFunc("POST /modules/{module_id}/ip-range-entries", s.handlePostIPRange)
|
||||||
m.HandleFunc("PATCH /modules/{module_id}/ip-range-entries/{entry_id}", s.handlePatchIPRange)
|
m.HandleFunc("PATCH /modules/{module_id}/ip-range-entries/{entry_id}", s.handlePatchIPRange)
|
||||||
m.HandleFunc("DELETE /modules/{module_id}/ip-range-entries/{entry_id}", s.handleDeleteIPRange)
|
m.HandleFunc("DELETE /modules/{module_id}/ip-range-entries/{entry_id}", s.handleDeleteIPRange)
|
||||||
|
m.HandleFunc("GET /modules/{module_id}/entries.csv", s.handleExportModuleEntriesCSV)
|
||||||
|
m.HandleFunc("POST /modules/{module_id}/entries.csv", s.handleImportModuleEntriesCSV)
|
||||||
|
|
||||||
m.HandleFunc("GET /doh-profiles", s.handleListDoh)
|
m.HandleFunc("GET /doh-profiles", s.handleListDoh)
|
||||||
m.HandleFunc("POST /doh-profiles", s.handlePostDoh)
|
m.HandleFunc("POST /doh-profiles", s.handlePostDoh)
|
||||||
@@ -68,14 +77,16 @@ func (s *Server) handlePostModule(w http.ResponseWriter, r *http.Request) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
var body struct {
|
var body struct {
|
||||||
Type string `json:"type"`
|
Type string `json:"type"`
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
Enabled bool `json:"enabled"`
|
Enabled bool `json:"enabled"`
|
||||||
Priority int `json:"priority"`
|
Priority int `json:"priority"`
|
||||||
RefreshIntervalSec int `json:"refresh_interval_sec"`
|
RefreshIntervalSec int `json:"refresh_interval_sec"`
|
||||||
CronExpr string `json:"cron_expr"`
|
CronExpr string `json:"cron_expr"`
|
||||||
DefaultCommunityID *string `json:"default_community_id"`
|
DefaultCommunityID *string `json:"default_community_id"`
|
||||||
DohProfileID *string `json:"doh_profile_id"`
|
DohProfileID *string `json:"doh_profile_id"`
|
||||||
|
DohProfileIDs []string `json:"doh_profile_ids"`
|
||||||
|
DohResolverPolicy string `json:"doh_resolver_policy"`
|
||||||
}
|
}
|
||||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
@@ -85,6 +96,7 @@ func (s *Server) handlePostModule(w http.ResponseWriter, r *http.Request) {
|
|||||||
Type: body.Type, Name: body.Name, Enabled: body.Enabled, Priority: body.Priority,
|
Type: body.Type, Name: body.Name, Enabled: body.Enabled, Priority: body.Priority,
|
||||||
RefreshIntervalSec: body.RefreshIntervalSec, CronExpr: body.CronExpr,
|
RefreshIntervalSec: body.RefreshIntervalSec, CronExpr: body.CronExpr,
|
||||||
DefaultCommunityID: body.DefaultCommunityID, DohProfileID: body.DohProfileID,
|
DefaultCommunityID: body.DefaultCommunityID, DohProfileID: body.DohProfileID,
|
||||||
|
DohProfileIDs: body.DohProfileIDs, DohResolverPolicy: body.DohResolverPolicy,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
@@ -98,11 +110,47 @@ func (s *Server) handlePatchModule(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !ok || !s.requireAtLeast(w, a, "editor") {
|
if !ok || !s.requireAtLeast(w, a, "editor") {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
rawBody, err := io.ReadAll(r.Body)
|
||||||
|
if err != nil {
|
||||||
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid body")
|
||||||
|
return
|
||||||
|
}
|
||||||
var body store.ModulePatch
|
var body store.ModulePatch
|
||||||
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
if err := json.Unmarshal(rawBody, &body); err != nil {
|
||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
// NOTE:
|
||||||
|
// In Go, unmarshalling JSON `null` into pointer fields results in nil,
|
||||||
|
// which is indistinguishable from "field omitted". For PATCH we need to
|
||||||
|
// distinguish these cases so clients can explicitly clear nullable fields.
|
||||||
|
var raw map[string]json.RawMessage
|
||||||
|
if err := json.Unmarshal(rawBody, &raw); err == nil {
|
||||||
|
if v, ok := raw["default_community_id"]; ok && string(v) == "null" {
|
||||||
|
empty := ""
|
||||||
|
body.DefaultCommunityID = &empty
|
||||||
|
}
|
||||||
|
if v, ok := raw["doh_profile_id"]; ok && string(v) == "null" {
|
||||||
|
empty := ""
|
||||||
|
body.DohProfileID = &empty
|
||||||
|
}
|
||||||
|
if v, ok := raw["doh_profile_ids"]; ok && string(v) == "null" {
|
||||||
|
empty := []string{}
|
||||||
|
body.DohProfileIDs = &empty
|
||||||
|
}
|
||||||
|
if v, ok := raw["doh_resolver_policy"]; ok && string(v) == "null" {
|
||||||
|
p := store.DohPolicyPrimaryOnly
|
||||||
|
body.DohResolverPolicy = &p
|
||||||
|
}
|
||||||
|
if v, ok := raw["cron_expr"]; ok && string(v) == "null" {
|
||||||
|
empty := ""
|
||||||
|
body.CronExpr = &empty
|
||||||
|
}
|
||||||
|
if v, ok := raw["refresh_interval_sec"]; ok && string(v) == "null" {
|
||||||
|
zero := 0
|
||||||
|
body.RefreshIntervalSec = &zero
|
||||||
|
}
|
||||||
|
}
|
||||||
mod, err := s.store.UpdateModule(a.TenantID, r.PathValue("module_id"), &body)
|
mod, err := s.store.UpdateModule(a.TenantID, r.PathValue("module_id"), &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
@@ -132,7 +180,7 @@ func writeStoreErr(w http.ResponseWriter, err error) {
|
|||||||
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", err.Error())
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", err.Error())
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error())
|
writeInternalError(w, "store", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Server) handleListCDNSources(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleListCDNSources(w http.ResponseWriter, r *http.Request) {
|
||||||
@@ -153,7 +201,7 @@ func (s *Server) handleListCDNSources(w http.ResponseWriter, r *http.Request) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func cdnSourceJSON(x *store.CDNSource) map[string]any {
|
func cdnSourceJSON(x *store.CDNSource) map[string]any {
|
||||||
m := map[string]any{"id": x.ID, "source_kind": x.SourceKind, "url": x.URL, "etag": x.Etag}
|
m := map[string]any{"id": x.ID, "source_kind": x.SourceKind, "url": x.URL, "prefix_path": x.PrefixPath, "etag": x.Etag}
|
||||||
if x.RefreshIntervalSec != nil {
|
if x.RefreshIntervalSec != nil {
|
||||||
m["refresh_interval_sec"] = *x.RefreshIntervalSec
|
m["refresh_interval_sec"] = *x.RefreshIntervalSec
|
||||||
} else {
|
} else {
|
||||||
@@ -164,9 +212,84 @@ func cdnSourceJSON(x *store.CDNSource) map[string]any {
|
|||||||
} else {
|
} else {
|
||||||
m["community_id"] = nil
|
m["community_id"] = nil
|
||||||
}
|
}
|
||||||
|
if x.LastRefreshedAt != nil {
|
||||||
|
m["last_refreshed_at"] = x.LastRefreshedAt.UTC().Format(time.RFC3339Nano)
|
||||||
|
} else {
|
||||||
|
m["last_refreshed_at"] = nil
|
||||||
|
}
|
||||||
return m
|
return m
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *Server) handlePreviewCDNSource(w http.ResponseWriter, r *http.Request) {
|
||||||
|
a, ok := authFromContext(r.Context())
|
||||||
|
if !ok || !s.requireAtLeast(w, a, "editor") {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
var body struct {
|
||||||
|
URL string `json:"url"`
|
||||||
|
SourceKind string `json:"source_kind"`
|
||||||
|
PrefixPath string `json:"prefix_path"`
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
|
||||||
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
u := strings.TrimSpace(body.URL)
|
||||||
|
if u == "" {
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "url is required")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
mod, err := s.store.GetModule(a.TenantID, r.PathValue("module_id"))
|
||||||
|
if err != nil {
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if mod.Type != "CDN_CIDRS" {
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "module type must be CDN_CIDRS")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
req, err := http.NewRequestWithContext(r.Context(), http.MethodGet, u, nil)
|
||||||
|
if err != nil {
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "invalid url")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
resp, err := s.cdnHTTP.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
writeBadGateway(w, "cdn preview fetch", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
_, _ = io.Copy(io.Discard, resp.Body)
|
||||||
|
writeBadGateway(w, "cdn preview fetch", fmt.Errorf("upstream status: %s", resp.Status))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
raw, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
|
||||||
|
if err != nil {
|
||||||
|
writeBadGateway(w, "cdn preview read body", err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
pfxs, err := pipeline.ExtractCIDRs(string(raw), body.SourceKind, body.PrefixPath)
|
||||||
|
if err != nil {
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
items := make([]string, 0, len(pfxs))
|
||||||
|
const previewLimit = 100
|
||||||
|
for i, p := range pfxs {
|
||||||
|
if i >= previewLimit {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
items = append(items, p.String())
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, map[string]any{
|
||||||
|
"items": items,
|
||||||
|
"total": len(pfxs),
|
||||||
|
"truncated": len(pfxs) > previewLimit,
|
||||||
|
"source_url": u,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handlePostCDNSource(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handlePostCDNSource(w http.ResponseWriter, r *http.Request) {
|
||||||
a, ok := authFromContext(r.Context())
|
a, ok := authFromContext(r.Context())
|
||||||
if !ok || !s.requireAtLeast(w, a, "editor") {
|
if !ok || !s.requireAtLeast(w, a, "editor") {
|
||||||
@@ -177,11 +300,13 @@ func (s *Server) handlePostCDNSource(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
x, err := s.store.CreateCDNSource(a.TenantID, r.PathValue("module_id"), &body)
|
mid := r.PathValue("module_id")
|
||||||
|
x, err := s.store.CreateCDNSource(a.TenantID, mid, &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "cdn_source_create")
|
||||||
writeJSON(w, http.StatusCreated, cdnSourceJSON(x))
|
writeJSON(w, http.StatusCreated, cdnSourceJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -195,11 +320,13 @@ func (s *Server) handlePatchCDNSource(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
x, err := s.store.UpdateCDNSource(a.TenantID, r.PathValue("module_id"), r.PathValue("source_id"), &body)
|
mid := r.PathValue("module_id")
|
||||||
|
x, err := s.store.UpdateCDNSource(a.TenantID, mid, r.PathValue("source_id"), &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "cdn_source_patch")
|
||||||
writeJSON(w, http.StatusOK, cdnSourceJSON(x))
|
writeJSON(w, http.StatusOK, cdnSourceJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -208,10 +335,12 @@ func (s *Server) handleDeleteCDNSource(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !ok || !s.requireAtLeast(w, a, "editor") {
|
if !ok || !s.requireAtLeast(w, a, "editor") {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err := s.store.DeleteCDNSource(a.TenantID, r.PathValue("module_id"), r.PathValue("source_id")); err != nil {
|
mid := r.PathValue("module_id")
|
||||||
|
if err := s.store.DeleteCDNSource(a.TenantID, mid, r.PathValue("source_id")); err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "cdn_source_delete")
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -267,11 +396,13 @@ func (s *Server) handlePostAS(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
x, err := s.store.CreateASEntry(a.TenantID, r.PathValue("module_id"), &body)
|
mid := r.PathValue("module_id")
|
||||||
|
x, err := s.store.CreateASEntry(a.TenantID, mid, &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "as_entry_create")
|
||||||
writeJSON(w, http.StatusCreated, asEntryJSON(x))
|
writeJSON(w, http.StatusCreated, asEntryJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -285,11 +416,13 @@ func (s *Server) handlePatchAS(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
x, err := s.store.UpdateASEntry(a.TenantID, r.PathValue("module_id"), r.PathValue("entry_id"), &body)
|
mid := r.PathValue("module_id")
|
||||||
|
x, err := s.store.UpdateASEntry(a.TenantID, mid, r.PathValue("entry_id"), &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "as_entry_patch")
|
||||||
writeJSON(w, http.StatusOK, asEntryJSON(x))
|
writeJSON(w, http.StatusOK, asEntryJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -298,10 +431,12 @@ func (s *Server) handleDeleteAS(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !ok || !s.requireAtLeast(w, a, "editor") {
|
if !ok || !s.requireAtLeast(w, a, "editor") {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err := s.store.DeleteASEntry(a.TenantID, r.PathValue("module_id"), r.PathValue("entry_id")); err != nil {
|
mid := r.PathValue("module_id")
|
||||||
|
if err := s.store.DeleteASEntry(a.TenantID, mid, r.PathValue("entry_id")); err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "as_entry_delete")
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -342,11 +477,13 @@ func (s *Server) handlePostDomain(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
x, err := s.store.CreateDomainEntry(a.TenantID, r.PathValue("module_id"), &body)
|
mid := r.PathValue("module_id")
|
||||||
|
x, err := s.store.CreateDomainEntry(a.TenantID, mid, &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "domain_entry_create")
|
||||||
writeJSON(w, http.StatusCreated, domainEntryJSON(x))
|
writeJSON(w, http.StatusCreated, domainEntryJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -360,11 +497,13 @@ func (s *Server) handlePatchDomain(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
x, err := s.store.UpdateDomainEntry(a.TenantID, r.PathValue("module_id"), r.PathValue("entry_id"), &body)
|
mid := r.PathValue("module_id")
|
||||||
|
x, err := s.store.UpdateDomainEntry(a.TenantID, mid, r.PathValue("entry_id"), &body)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "domain_entry_patch")
|
||||||
writeJSON(w, http.StatusOK, domainEntryJSON(x))
|
writeJSON(w, http.StatusOK, domainEntryJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -373,10 +512,12 @@ func (s *Server) handleDeleteDomain(w http.ResponseWriter, r *http.Request) {
|
|||||||
if !ok || !s.requireAtLeast(w, a, "editor") {
|
if !ok || !s.requireAtLeast(w, a, "editor") {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
if err := s.store.DeleteDomainEntry(a.TenantID, r.PathValue("module_id"), r.PathValue("entry_id")); err != nil {
|
mid := r.PathValue("module_id")
|
||||||
|
if err := s.store.DeleteDomainEntry(a.TenantID, mid, r.PathValue("entry_id")); err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, mid, "domain_entry_delete")
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -461,6 +602,133 @@ func (s *Server) handleDeleteIPRange(w http.ResponseWriter, r *http.Request) {
|
|||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleExportModuleEntriesCSV(w http.ResponseWriter, r *http.Request) {
|
||||||
|
a, ok := authFromContext(r.Context())
|
||||||
|
if !ok || !s.requireAtLeast(w, a, "viewer") {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
moduleID := r.PathValue("module_id")
|
||||||
|
mod, err := s.store.GetModule(a.TenantID, moduleID)
|
||||||
|
if err != nil {
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
communities, err := s.store.ListCommunities(a.TenantID)
|
||||||
|
if err != nil {
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
communityByID := make(map[string]string, len(communities))
|
||||||
|
for _, c := range communities {
|
||||||
|
communityByID[c.ID] = strings.TrimSpace(c.Community)
|
||||||
|
}
|
||||||
|
|
||||||
|
records := make([][]string, 0, 64)
|
||||||
|
|
||||||
|
switch mod.Type {
|
||||||
|
case "AS_PREFIXES":
|
||||||
|
records = append(records, []string{"asn", "community"})
|
||||||
|
list, err := s.store.ListASEntries(a.TenantID, moduleID)
|
||||||
|
if err != nil {
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, x := range list {
|
||||||
|
community := ""
|
||||||
|
if x.CommunityID != nil {
|
||||||
|
community = communityByID[*x.CommunityID]
|
||||||
|
}
|
||||||
|
records = append(records, []string{strconv.FormatInt(x.ASN, 10), community})
|
||||||
|
}
|
||||||
|
case "DOMAINS":
|
||||||
|
records = append(records, []string{"domain", "community"})
|
||||||
|
list, err := s.store.ListDomainEntries(a.TenantID, moduleID)
|
||||||
|
if err != nil {
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, x := range list {
|
||||||
|
community := ""
|
||||||
|
if x.CommunityID != nil {
|
||||||
|
community = communityByID[*x.CommunityID]
|
||||||
|
}
|
||||||
|
records = append(records, []string{x.FQDN, community})
|
||||||
|
}
|
||||||
|
case "IP_RANGES":
|
||||||
|
records = append(records, []string{"ipRange", "community"})
|
||||||
|
list, err := s.store.ListIPRangeEntries(a.TenantID, moduleID)
|
||||||
|
if err != nil {
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
for _, x := range list {
|
||||||
|
community := ""
|
||||||
|
if x.CommunityID != nil {
|
||||||
|
community = communityByID[*x.CommunityID]
|
||||||
|
}
|
||||||
|
records = append(records, []string{x.Prefix, community})
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "csv import/export is supported only for AS_PREFIXES, DOMAINS, IP_RANGES")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
w.Header().Set("Content-Type", "text/csv; charset=utf-8")
|
||||||
|
w.Header().Set("Content-Disposition", `attachment; filename="module-entries.csv"`)
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
cw := csv.NewWriter(w)
|
||||||
|
for _, rec := range records {
|
||||||
|
if err := cw.Write(rec); err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
cw.Flush()
|
||||||
|
}
|
||||||
|
|
||||||
|
func (s *Server) handleImportModuleEntriesCSV(w http.ResponseWriter, r *http.Request) {
|
||||||
|
a, ok := authFromContext(r.Context())
|
||||||
|
if !ok || !s.requireAtLeast(w, a, "editor") {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
moduleID := r.PathValue("module_id")
|
||||||
|
res, err := importer.ImportModuleEntriesCSV(s.store, a.TenantID, moduleID, r.Body)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, store.ErrInvalidInput) {
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "csv is empty")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), "importer: invalid csv") {
|
||||||
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid csv")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), "importer: line") {
|
||||||
|
detail := strings.TrimPrefix(err.Error(), "importer: ")
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", detail)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if strings.Contains(err.Error(), "importer: csv import/export") {
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "csv import/export is supported only for AS_PREFIXES, DOMAINS, IP_RANGES")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
writeStoreErr(w, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if res.Imported > 0 {
|
||||||
|
switch res.ModuleType {
|
||||||
|
case "AS_PREFIXES":
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "as_entry_import_csv")
|
||||||
|
case "DOMAINS":
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "domain_entry_import_csv")
|
||||||
|
case "IP_RANGES":
|
||||||
|
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "ip_range_import_csv")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
writeJSON(w, http.StatusOK, map[string]any{
|
||||||
|
"imported": res.Imported,
|
||||||
|
"module_type": res.ModuleType,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
func (s *Server) handleListDoh(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleListDoh(w http.ResponseWriter, r *http.Request) {
|
||||||
a, ok := authFromContext(r.Context())
|
a, ok := authFromContext(r.Context())
|
||||||
if !ok || !s.requireAtLeast(w, a, "viewer") {
|
if !ok || !s.requireAtLeast(w, a, "viewer") {
|
||||||
@@ -576,7 +844,7 @@ func commJSON(x *store.Community) map[string]any {
|
|||||||
if err := json.Unmarshal([]byte(x.ValueJSON), &v); err != nil {
|
if err := json.Unmarshal([]byte(x.ValueJSON), &v); err != nil {
|
||||||
v = x.ValueJSON
|
v = x.ValueJSON
|
||||||
}
|
}
|
||||||
return map[string]any{"id": x.ID, "name": x.Name, "kind": x.Kind, "value_json": v}
|
return map[string]any{"id": x.ID, "community": x.Community, "title": x.Title, "value_json": v}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (s *Server) handleGetComm(w http.ResponseWriter, r *http.Request) {
|
func (s *Server) handleGetComm(w http.ResponseWriter, r *http.Request) {
|
||||||
@@ -656,6 +924,7 @@ func (s *Server) handlePostPeer(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueuePeerReconcile(a.TenantID, "peer_create")
|
||||||
writeJSON(w, http.StatusCreated, peerJSON(x))
|
writeJSON(w, http.StatusCreated, peerJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -687,6 +956,7 @@ func (s *Server) handlePatchPeer(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueuePeerReconcile(a.TenantID, "peer_patch")
|
||||||
writeJSON(w, http.StatusOK, peerJSON(x))
|
writeJSON(w, http.StatusOK, peerJSON(x))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -699,6 +969,7 @@ func (s *Server) handleDeletePeer(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
s.enqueuePeerReconcile(a.TenantID, "peer_delete")
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -798,9 +1069,47 @@ func (s *Server) handlePatchSettings(w http.ResponseWriter, r *http.Request) {
|
|||||||
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
if raw, ok := body["revision_retention_minutes"]; ok && raw != nil {
|
||||||
|
v, ok := parseRevisionRetentionMinutes(raw)
|
||||||
|
if !ok {
|
||||||
|
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "revision_retention_minutes must be an integer in range 15..43200")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
body["revision_retention_minutes"] = v
|
||||||
|
}
|
||||||
if err := s.store.PatchGlobalSettings(a.TenantID, body); err != nil {
|
if err := s.store.PatchGlobalSettings(a.TenantID, body); err != nil {
|
||||||
writeStoreErr(w, err)
|
writeStoreErr(w, err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
|
writeJSON(w, http.StatusOK, map[string]string{"status": "ok"})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func parseRevisionRetentionMinutes(v any) (int, bool) {
|
||||||
|
const minMinutes = 15
|
||||||
|
const maxMinutes = 30 * 24 * 60
|
||||||
|
|
||||||
|
var out int
|
||||||
|
switch x := v.(type) {
|
||||||
|
case float64:
|
||||||
|
if x != float64(int(x)) {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
out = int(x)
|
||||||
|
case int:
|
||||||
|
out = x
|
||||||
|
case int64:
|
||||||
|
out = int(x)
|
||||||
|
case string:
|
||||||
|
n, err := strconv.Atoi(strings.TrimSpace(x))
|
||||||
|
if err != nil {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
out = n
|
||||||
|
default:
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
if out < minMinutes || out > maxMinutes {
|
||||||
|
return 0, false
|
||||||
|
}
|
||||||
|
return out, true
|
||||||
|
}
|
||||||
@@ -0,0 +1,105 @@
|
|||||||
|
package httpapi
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/csv"
|
||||||
|
"encoding/json"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestModuleEntriesCSVImportExportIPRanges(t *testing.T) {
|
||||||
|
srv, err := New(Options{
|
||||||
|
InsecureDev: true,
|
||||||
|
SeedDemo: true,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
tenant, _, modIP, _, _ := srv.Store().DemoIDs()
|
||||||
|
srv.apiKeys = parseAPIKeysSpec("opkey|" + tenant + "|operator")
|
||||||
|
|
||||||
|
ts := httptest.NewServer(srv.Handler())
|
||||||
|
defer ts.Close()
|
||||||
|
client := ts.Client()
|
||||||
|
base := ts.URL
|
||||||
|
|
||||||
|
reqList, _ := http.NewRequest(http.MethodGet, base+"/v1/communities?limit=10", nil)
|
||||||
|
reqList.Header.Set("Authorization", "Bearer opkey")
|
||||||
|
respList, err := client.Do(reqList)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer respList.Body.Close()
|
||||||
|
if respList.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(respList.Body)
|
||||||
|
t.Fatalf("communities status %d: %s", respList.StatusCode, b)
|
||||||
|
}
|
||||||
|
var listBody struct {
|
||||||
|
Items []struct {
|
||||||
|
Community string `json:"community"`
|
||||||
|
} `json:"items"`
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(respList.Body).Decode(&listBody); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(listBody.Items) == 0 {
|
||||||
|
t.Fatal("expected seeded community")
|
||||||
|
}
|
||||||
|
|
||||||
|
csvBody := "\"ipRange\",\"community\"\n\"10.254.1.254/32\",\"" + listBody.Items[0].Community + "\"\n\"160.79.104.0/23\",\"" + listBody.Items[0].Community + "\"\n"
|
||||||
|
reqImport, _ := http.NewRequest(http.MethodPost, base+"/v1/modules/"+modIP+"/entries.csv", strings.NewReader(csvBody))
|
||||||
|
reqImport.Header.Set("Authorization", "Bearer opkey")
|
||||||
|
reqImport.Header.Set("Content-Type", "text/csv")
|
||||||
|
respImport, err := client.Do(reqImport)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer respImport.Body.Close()
|
||||||
|
if respImport.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(respImport.Body)
|
||||||
|
t.Fatalf("import status %d: %s", respImport.StatusCode, b)
|
||||||
|
}
|
||||||
|
var importBody struct {
|
||||||
|
Imported int `json:"imported"`
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(respImport.Body).Decode(&importBody); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if importBody.Imported != 2 {
|
||||||
|
t.Fatalf("expected imported=2, got %d", importBody.Imported)
|
||||||
|
}
|
||||||
|
|
||||||
|
reqExport, _ := http.NewRequest(http.MethodGet, base+"/v1/modules/"+modIP+"/entries.csv", nil)
|
||||||
|
reqExport.Header.Set("Authorization", "Bearer opkey")
|
||||||
|
respExport, err := client.Do(reqExport)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer respExport.Body.Close()
|
||||||
|
if respExport.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(respExport.Body)
|
||||||
|
t.Fatalf("export status %d: %s", respExport.StatusCode, b)
|
||||||
|
}
|
||||||
|
if got := respExport.Header.Get("Content-Type"); !strings.Contains(got, "text/csv") {
|
||||||
|
t.Fatalf("unexpected content-type: %q", got)
|
||||||
|
}
|
||||||
|
raw, err := io.ReadAll(respExport.Body)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
records, err := csv.NewReader(strings.NewReader(string(raw))).ReadAll()
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(records) < 3 {
|
||||||
|
t.Fatalf("expected at least 3 csv rows, got %d", len(records))
|
||||||
|
}
|
||||||
|
if records[0][0] != "ipRange" || records[0][1] != "community" {
|
||||||
|
t.Fatalf("unexpected header: %#v", records[0])
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -24,6 +24,7 @@ type Server struct {
|
|||||||
apiKeys []apiKeyRecord
|
apiKeys []apiKeyRecord
|
||||||
insecureDev bool
|
insecureDev bool
|
||||||
corsOrigins []string
|
corsOrigins []string
|
||||||
|
cdnHTTP *http.Client
|
||||||
mux *http.ServeMux
|
mux *http.ServeMux
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -67,6 +68,7 @@ func New(opts Options) (*Server, error) {
|
|||||||
apiKeys: parseAPIKeysSpec(opts.APIKeys),
|
apiKeys: parseAPIKeysSpec(opts.APIKeys),
|
||||||
insecureDev: opts.InsecureDev && opts.SeedDemo,
|
insecureDev: opts.InsecureDev && opts.SeedDemo,
|
||||||
corsOrigins: parseCORSOrigins(opts.CORSAllowedOrigins),
|
corsOrigins: parseCORSOrigins(opts.CORSAllowedOrigins),
|
||||||
|
cdnHTTP: NewCDNHTTPClient(),
|
||||||
}
|
}
|
||||||
s.mux = http.NewServeMux()
|
s.mux = http.NewServeMux()
|
||||||
s.registerRoutes()
|
s.registerRoutes()
|
||||||
|
|||||||
@@ -13,6 +13,7 @@ import (
|
|||||||
|
|
||||||
"evobgp/internal/jobs"
|
"evobgp/internal/jobs"
|
||||||
"evobgp/internal/signing"
|
"evobgp/internal/signing"
|
||||||
|
"evobgp/internal/version"
|
||||||
)
|
)
|
||||||
|
|
||||||
const testBundleSeed = "0101010101010101010101010101010101010101010101010101010101010101"
|
const testBundleSeed = "0101010101010101010101010101010101010101010101010101010101010101"
|
||||||
@@ -28,7 +29,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
|
|||||||
}
|
}
|
||||||
defer srv.Close()
|
defer srv.Close()
|
||||||
tenant, modCDN, modIP, rev, speaker := srv.Store().DemoIDs()
|
tenant, modCDN, modIP, rev, speaker := srv.Store().DemoIDs()
|
||||||
srv.apiKeys = parseAPIKeysSpec("nodekey|" + tenant + "|node,opkey|" + tenant + "|operator")
|
srv.apiKeys = parseAPIKeysSpec("nodekey|" + tenant + "|node,opkey|" + tenant + "|operator,edkey|" + tenant + "|editor")
|
||||||
|
|
||||||
ts := httptest.NewServer(srv.Handler())
|
ts := httptest.NewServer(srv.Handler())
|
||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
@@ -124,6 +125,34 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
t.Run("diagnostic log for revision", func(t *testing.T) {
|
||||||
|
req, _ := http.NewRequest(http.MethodGet, base+"/v1/revisions/"+rev+"/diagnostic-log", nil)
|
||||||
|
req.Header.Set("Authorization", "Bearer opkey")
|
||||||
|
resp, err := client.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(resp.Body)
|
||||||
|
t.Fatalf("status %d: %s", resp.StatusCode, b)
|
||||||
|
}
|
||||||
|
if got := resp.Header.Get("Content-Type"); !strings.Contains(got, "text/plain") {
|
||||||
|
t.Fatalf("unexpected content-type: %q", got)
|
||||||
|
}
|
||||||
|
raw, _ := io.ReadAll(resp.Body)
|
||||||
|
body := string(raw)
|
||||||
|
for _, needle := range []string{
|
||||||
|
"EvoBGP revision diagnostic log",
|
||||||
|
"revision_id=" + rev,
|
||||||
|
"## Raw rows",
|
||||||
|
} {
|
||||||
|
if !strings.Contains(body, needle) {
|
||||||
|
t.Fatalf("diagnostic log missing %q", needle)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
t.Run("list modules peers speakers", func(t *testing.T) {
|
t.Run("list modules peers speakers", func(t *testing.T) {
|
||||||
for _, path := range []string{"/v1/modules", "/v1/peers", "/v1/speakers"} {
|
for _, path := range []string{"/v1/modules", "/v1/peers", "/v1/speakers"} {
|
||||||
req, _ := http.NewRequest(http.MethodGet, base+path, nil)
|
req, _ := http.NewRequest(http.MethodGet, base+path, nil)
|
||||||
@@ -149,6 +178,90 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
|
|||||||
}
|
}
|
||||||
})
|
})
|
||||||
|
|
||||||
|
t.Run("modules filter by type", func(t *testing.T) {
|
||||||
|
req, _ := http.NewRequest(http.MethodGet, base+"/v1/modules?type=IP_RANGES", nil)
|
||||||
|
req.Header.Set("Authorization", "Bearer opkey")
|
||||||
|
resp, err := client.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(resp.Body)
|
||||||
|
t.Fatalf("status %d: %s", resp.StatusCode, b)
|
||||||
|
}
|
||||||
|
var body struct {
|
||||||
|
Items []struct {
|
||||||
|
Type string `json:"type"`
|
||||||
|
} `json:"items"`
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(body.Items) == 0 {
|
||||||
|
t.Fatalf("expected at least one IP_RANGES module")
|
||||||
|
}
|
||||||
|
for _, item := range body.Items {
|
||||||
|
if item.Type != "IP_RANGES" {
|
||||||
|
t.Fatalf("unexpected module type %q", item.Type)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("router lists catalog endpoint", func(t *testing.T) {
|
||||||
|
req, _ := http.NewRequest(http.MethodGet, base+"/v1/router-lists/catalog", nil)
|
||||||
|
req.Header.Set("Authorization", "Bearer opkey")
|
||||||
|
resp, err := client.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(resp.Body)
|
||||||
|
t.Fatalf("status %d: %s", resp.StatusCode, b)
|
||||||
|
}
|
||||||
|
var body struct {
|
||||||
|
Modules struct {
|
||||||
|
Items []map[string]any `json:"items"`
|
||||||
|
} `json:"modules"`
|
||||||
|
Domains struct {
|
||||||
|
Items []map[string]any `json:"items"`
|
||||||
|
} `json:"domains"`
|
||||||
|
ASNs struct {
|
||||||
|
Items []map[string]any `json:"items"`
|
||||||
|
} `json:"asns"`
|
||||||
|
IPRanges struct {
|
||||||
|
Items []map[string]any `json:"items"`
|
||||||
|
} `json:"ip_ranges"`
|
||||||
|
Communities struct {
|
||||||
|
Items []map[string]any `json:"items"`
|
||||||
|
} `json:"communities"`
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(body.Modules.Items) == 0 {
|
||||||
|
t.Fatalf("expected modules in catalog")
|
||||||
|
}
|
||||||
|
if body.Communities.Items == nil {
|
||||||
|
t.Fatalf("expected communities.items field in catalog")
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
|
t.Run("rollback forbidden for editor", func(t *testing.T) {
|
||||||
|
req, _ := http.NewRequest(http.MethodPost, base+"/v1/revisions/"+rev+"/rollback", nil)
|
||||||
|
req.Header.Set("Authorization", "Bearer edkey")
|
||||||
|
resp, err := client.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusForbidden {
|
||||||
|
b, _ := io.ReadAll(resp.Body)
|
||||||
|
t.Fatalf("status %d want 403: %s", resp.StatusCode, b)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
|
||||||
t.Run("rollback queues job", func(t *testing.T) {
|
t.Run("rollback queues job", func(t *testing.T) {
|
||||||
req, _ := http.NewRequest(http.MethodPost, base+"/v1/revisions/"+rev+"/rollback", nil)
|
req, _ := http.NewRequest(http.MethodPost, base+"/v1/revisions/"+rev+"/rollback", nil)
|
||||||
req.Header.Set("Authorization", "Bearer opkey")
|
req.Header.Set("Authorization", "Bearer opkey")
|
||||||
@@ -243,3 +356,76 @@ func waitJob(t *testing.T, client *http.Client, base, token, jobID string) {
|
|||||||
}
|
}
|
||||||
t.Fatalf("job %s did not complete", jobID)
|
t.Fatalf("job %s did not complete", jobID)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestParseBGPProtocolStates(t *testing.T) {
|
||||||
|
raw := `BIRD 2.16.2 ready.
|
||||||
|
name proto table state since info
|
||||||
|
device1 Device --- up 2026-04-09
|
||||||
|
evobgp_p_abcdef01 BGP master up 2026-04-09 Established
|
||||||
|
evobgp_p_01234567 BGP master start 2026-04-09 Connect
|
||||||
|
`
|
||||||
|
got := parseBGPProtocolStates(raw)
|
||||||
|
if got["evobgp_p_abcdef01"] != "Established" {
|
||||||
|
t.Fatalf("expected protocol state Established, got %q", got["evobgp_p_abcdef01"])
|
||||||
|
}
|
||||||
|
if got["evobgp_p_01234567"] != "Connect" {
|
||||||
|
t.Fatalf("expected protocol state Connect, got %q", got["evobgp_p_01234567"])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestVersionEndpoints(t *testing.T) {
|
||||||
|
srv, err := New(Options{
|
||||||
|
InsecureDev: true,
|
||||||
|
SeedDemo: true,
|
||||||
|
BundleSeedHex: testBundleSeed,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
oldVer := version.Version
|
||||||
|
oldSHA := version.GitSHA
|
||||||
|
oldTime := version.BuildTime
|
||||||
|
version.Version = "1.2.3"
|
||||||
|
version.GitSHA = "abc123def456"
|
||||||
|
version.BuildTime = "2026-05-20T12:00:00Z"
|
||||||
|
t.Cleanup(func() {
|
||||||
|
version.Version = oldVer
|
||||||
|
version.GitSHA = oldSHA
|
||||||
|
version.BuildTime = oldTime
|
||||||
|
})
|
||||||
|
|
||||||
|
ts := httptest.NewServer(srv.Handler())
|
||||||
|
defer ts.Close()
|
||||||
|
|
||||||
|
for _, path := range []string{"/version", "/v1/version"} {
|
||||||
|
t.Run(path, func(t *testing.T) {
|
||||||
|
resp, err := http.Get(ts.URL + path)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(resp.Body)
|
||||||
|
t.Fatalf("status %d: %s", resp.StatusCode, b)
|
||||||
|
}
|
||||||
|
var body map[string]string
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if body["version"] != "1.2.3" {
|
||||||
|
t.Fatalf("version=%q want 1.2.3", body["version"])
|
||||||
|
}
|
||||||
|
if body["api_version"] != "1.2.3" {
|
||||||
|
t.Fatalf("api_version=%q want 1.2.3", body["api_version"])
|
||||||
|
}
|
||||||
|
if body["git_sha"] != "abc123def456" {
|
||||||
|
t.Fatalf("git_sha=%q", body["git_sha"])
|
||||||
|
}
|
||||||
|
if body["build_time"] != "2026-05-20T12:00:00Z" {
|
||||||
|
t.Fatalf("build_time=%q", body["build_time"])
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,147 @@
|
|||||||
|
package importer
|
||||||
|
|
||||||
|
import (
|
||||||
|
"encoding/csv"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Result of importing module entries from CSV.
|
||||||
|
type Result struct {
|
||||||
|
Imported int
|
||||||
|
ModuleType string
|
||||||
|
}
|
||||||
|
|
||||||
|
// ImportModuleEntriesCSV parses and creates entries for AS_PREFIXES, DOMAINS, or IP_RANGES modules.
|
||||||
|
func ImportModuleEntriesCSV(st store.Backend, tenantID, moduleID string, body io.Reader) (*Result, error) {
|
||||||
|
mod, err := st.GetModule(tenantID, moduleID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
|
||||||
|
cr := csv.NewReader(io.LimitReader(body, 8<<20))
|
||||||
|
cr.TrimLeadingSpace = true
|
||||||
|
cr.FieldsPerRecord = -1
|
||||||
|
rows, err := cr.ReadAll()
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("importer: invalid csv: %w", err)
|
||||||
|
}
|
||||||
|
if len(rows) == 0 {
|
||||||
|
return nil, fmt.Errorf("importer: %w", store.ErrInvalidInput)
|
||||||
|
}
|
||||||
|
|
||||||
|
communities, err := st.ListCommunities(tenantID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
communityIDByID := make(map[string]string, len(communities))
|
||||||
|
communityIDByValue := make(map[string]string, len(communities))
|
||||||
|
for _, c := range communities {
|
||||||
|
communityIDByID[c.ID] = c.ID
|
||||||
|
communityIDByValue[strings.TrimSpace(c.Community)] = c.ID
|
||||||
|
}
|
||||||
|
|
||||||
|
resolveCommunity := func(raw string, required bool) (*string, error) {
|
||||||
|
v := strings.TrimSpace(raw)
|
||||||
|
if v == "" {
|
||||||
|
if required {
|
||||||
|
return nil, fmt.Errorf("community is required")
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
if id, ok := communityIDByID[v]; ok {
|
||||||
|
return &id, nil
|
||||||
|
}
|
||||||
|
if id, ok := communityIDByValue[v]; ok {
|
||||||
|
return &id, nil
|
||||||
|
}
|
||||||
|
return nil, fmt.Errorf("unknown community %q", v)
|
||||||
|
}
|
||||||
|
|
||||||
|
start := 0
|
||||||
|
if len(rows[0]) >= 2 {
|
||||||
|
key := strings.ToLower(strings.TrimSpace(rows[0][0]))
|
||||||
|
switch key {
|
||||||
|
case "asn", "domain", "iprange":
|
||||||
|
start = 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
imported := 0
|
||||||
|
switch mod.Type {
|
||||||
|
case "AS_PREFIXES":
|
||||||
|
for i := start; i < len(rows); i++ {
|
||||||
|
rec := rows[i]
|
||||||
|
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(rec) < 2 {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: expected 2 columns", i+1)
|
||||||
|
}
|
||||||
|
asn, err := strconv.ParseInt(strings.TrimSpace(rec[0]), 10, 64)
|
||||||
|
if err != nil || asn <= 0 {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: invalid asn", i+1)
|
||||||
|
}
|
||||||
|
cid, err := resolveCommunity(rec[1], false)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: %w", i+1, err)
|
||||||
|
}
|
||||||
|
if _, err := st.CreateASEntry(tenantID, moduleID, &store.ASEntry{ASN: asn, CommunityID: cid}); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
imported++
|
||||||
|
}
|
||||||
|
case "DOMAINS":
|
||||||
|
for i := start; i < len(rows); i++ {
|
||||||
|
rec := rows[i]
|
||||||
|
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(rec) < 2 {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: expected 2 columns", i+1)
|
||||||
|
}
|
||||||
|
fqdn := strings.TrimSpace(rec[0])
|
||||||
|
if fqdn == "" {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: domain is required", i+1)
|
||||||
|
}
|
||||||
|
cid, err := resolveCommunity(rec[1], false)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: %w", i+1, err)
|
||||||
|
}
|
||||||
|
if _, err := st.CreateDomainEntry(tenantID, moduleID, &store.DomainEntry{FQDN: fqdn, CommunityID: cid}); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
imported++
|
||||||
|
}
|
||||||
|
case "IP_RANGES":
|
||||||
|
for i := start; i < len(rows); i++ {
|
||||||
|
rec := rows[i]
|
||||||
|
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(rec) < 2 {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: expected 2 columns", i+1)
|
||||||
|
}
|
||||||
|
prefix := strings.TrimSpace(rec[0])
|
||||||
|
if prefix == "" {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: ipRange is required", i+1)
|
||||||
|
}
|
||||||
|
cid, err := resolveCommunity(rec[1], true)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("importer: line %d: %w", i+1, err)
|
||||||
|
}
|
||||||
|
if _, err := st.CreateIPRangeEntry(tenantID, moduleID, &store.IPRangeEntry{Prefix: prefix, CommunityID: cid}); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
imported++
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("importer: csv import/export is supported only for AS_PREFIXES, DOMAINS, IP_RANGES")
|
||||||
|
}
|
||||||
|
|
||||||
|
return &Result{Imported: imported, ModuleType: mod.Type}, nil
|
||||||
|
}
|
||||||
+134
-21
@@ -2,7 +2,10 @@ package jobs
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
|
"os"
|
||||||
"sort"
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
"sync"
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -20,17 +23,17 @@ const (
|
|||||||
|
|
||||||
// Job is the API-facing job model (поля согласованы со схемой job_audit в миграциях; персистенция в БД пока не подключена).
|
// Job is the API-facing job model (поля согласованы со схемой job_audit в миграциях; персистенция в БД пока не подключена).
|
||||||
type Job struct {
|
type Job struct {
|
||||||
ID string
|
ID string
|
||||||
TenantID string
|
TenantID string
|
||||||
Kind string
|
Kind string
|
||||||
Status string
|
Status string
|
||||||
IdempotencyKey *string
|
IdempotencyKey *string
|
||||||
ModuleID *string
|
ModuleID *string
|
||||||
CreatedAt time.Time
|
CreatedAt time.Time
|
||||||
StartedAt *time.Time
|
StartedAt *time.Time
|
||||||
FinishedAt *time.Time
|
FinishedAt *time.Time
|
||||||
Error *string
|
Error *string
|
||||||
ProgressPct *int16
|
ProgressPct *int16
|
||||||
Meta map[string]any
|
Meta map[string]any
|
||||||
cancelRequested bool
|
cancelRequested bool
|
||||||
mu sync.Mutex
|
mu sync.Mutex
|
||||||
@@ -148,6 +151,25 @@ func (j *Job) Snapshot() map[string]any {
|
|||||||
return m
|
return m
|
||||||
}
|
}
|
||||||
|
|
||||||
|
var jobRegistryMaxJobsOnce sync.Once
|
||||||
|
var jobRegistryMaxJobs int
|
||||||
|
|
||||||
|
// registryMaxJobsFromEnv returns EVOBGP_JOB_REGISTRY_MAX_JOBS once (0 = без лимита, только завершённые джобы вытесняются).
|
||||||
|
func registryMaxJobsFromEnv() int {
|
||||||
|
jobRegistryMaxJobsOnce.Do(func() {
|
||||||
|
s := strings.TrimSpace(os.Getenv("EVOBGP_JOB_REGISTRY_MAX_JOBS"))
|
||||||
|
if s == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
n, err := strconv.Atoi(s)
|
||||||
|
if err != nil || n <= 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
jobRegistryMaxJobs = n
|
||||||
|
})
|
||||||
|
return jobRegistryMaxJobs
|
||||||
|
}
|
||||||
|
|
||||||
// Registry — in-memory очередь и индекс по idempotency в процессе, где поднят HTTP API (evobgp-api и evobgp-all).
|
// Registry — in-memory очередь и индекс по idempotency в процессе, где поднят HTTP API (evobgp-api и evobgp-all).
|
||||||
// Отдельные воркеры в reference-профиле не разделяют память с API: scheduler дергает refresh по HTTP; см. docs/architecture.md.
|
// Отдельные воркеры в reference-профиле не разделяют память с API: scheduler дергает refresh по HTTP; см. docs/architecture.md.
|
||||||
// Запись задач в PostgreSQL job_audit + SKIP LOCKED / внешний брокер — планируемое расширение (архитектурный план §2, §7.10).
|
// Запись задач в PostgreSQL job_audit + SKIP LOCKED / внешний брокер — планируемое расширение (архитектурный план §2, §7.10).
|
||||||
@@ -171,11 +193,54 @@ func NewRegistry(workerStart func(j *Job)) *Registry {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// pruneTerminalIfOver удаляет самые старые завершённые джобы (succeeded/failed/cancelled), пока len(byID) > maxJobs.
|
||||||
|
func (r *Registry) pruneTerminalIfOver(maxJobs int) {
|
||||||
|
if r == nil || maxJobs <= 0 || len(r.byID) <= maxJobs {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
type fin struct {
|
||||||
|
j *Job
|
||||||
|
t time.Time
|
||||||
|
}
|
||||||
|
var cands []fin
|
||||||
|
for _, j := range r.byID {
|
||||||
|
st := j.statusLocked()
|
||||||
|
if st != StatusSucceeded && st != StatusFailed && st != StatusCancelled {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
j.mu.Lock()
|
||||||
|
ft := j.FinishedAt
|
||||||
|
j.mu.Unlock()
|
||||||
|
if ft == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
cands = append(cands, fin{j: j, t: *ft})
|
||||||
|
}
|
||||||
|
need := len(r.byID) - maxJobs
|
||||||
|
if need <= 0 || len(cands) == 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
sort.Slice(cands, func(i, j int) bool { return cands[i].t.Before(cands[j].t) })
|
||||||
|
if need > len(cands) {
|
||||||
|
need = len(cands)
|
||||||
|
}
|
||||||
|
for i := 0; i < need; i++ {
|
||||||
|
v := cands[i].j
|
||||||
|
delete(r.byID, v.ID)
|
||||||
|
if v.IdempotencyKey != nil && *v.IdempotencyKey != "" {
|
||||||
|
delete(r.byIdempo, idempoKey{tenant: v.TenantID, key: *v.IdempotencyKey})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Enqueue creates a job or returns an existing one for the same idempotency key.
|
// Enqueue creates a job or returns an existing one for the same idempotency key.
|
||||||
func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, moduleID *string, meta map[string]any) (*Job, bool, error) {
|
func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, moduleID *string, meta map[string]any) (*Job, bool, error) {
|
||||||
r.mu.Lock()
|
r.mu.Lock()
|
||||||
defer r.mu.Unlock()
|
defer r.mu.Unlock()
|
||||||
|
|
||||||
|
maxJobs := registryMaxJobsFromEnv()
|
||||||
|
r.pruneTerminalIfOver(maxJobs)
|
||||||
|
|
||||||
if idempotencyKey != nil && *idempotencyKey != "" {
|
if idempotencyKey != nil && *idempotencyKey != "" {
|
||||||
k := idempoKey{tenant: tenantID, key: *idempotencyKey}
|
k := idempoKey{tenant: tenantID, key: *idempotencyKey}
|
||||||
if existing, ok := r.byIdempo[k]; ok {
|
if existing, ok := r.byIdempo[k]; ok {
|
||||||
@@ -197,6 +262,7 @@ func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, module
|
|||||||
r.byIdempo[idempoKey{tenant: tenantID, key: *idempotencyKey}] = j
|
r.byIdempo[idempoKey{tenant: tenantID, key: *idempotencyKey}] = j
|
||||||
}
|
}
|
||||||
r.byID[j.ID] = j
|
r.byID[j.ID] = j
|
||||||
|
r.pruneTerminalIfOver(maxJobs)
|
||||||
|
|
||||||
if r.workerStart != nil {
|
if r.workerStart != nil {
|
||||||
go r.workerStart(j)
|
go r.workerStart(j)
|
||||||
@@ -229,22 +295,32 @@ func (r *Registry) List(tenantID, statusFilter, kindFilter, cursor string, limit
|
|||||||
if limit <= 0 {
|
if limit <= 0 {
|
||||||
limit = 50
|
limit = 50
|
||||||
}
|
}
|
||||||
|
// Build candidate set under r.mu, but read mutable status fields (j.Status) outside
|
||||||
|
// of r.mu so we don't race with Job.mu-protected writes (MarkRunning/Succeed/Fail).
|
||||||
r.mu.RLock()
|
r.mu.RLock()
|
||||||
var all []*Job
|
all := make([]*Job, 0, len(r.byID))
|
||||||
for _, j := range r.byID {
|
for _, j := range r.byID {
|
||||||
if j.TenantID != tenantID {
|
if j.TenantID == tenantID {
|
||||||
continue
|
all = append(all, j)
|
||||||
}
|
}
|
||||||
if statusFilter != "" && j.Status != statusFilter {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if kindFilter != "" && j.Kind != kindFilter {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
all = append(all, j)
|
|
||||||
}
|
}
|
||||||
r.mu.RUnlock()
|
r.mu.RUnlock()
|
||||||
|
|
||||||
|
// Apply filters outside of r.mu to synchronize with Job.mu.
|
||||||
|
if statusFilter != "" || kindFilter != "" {
|
||||||
|
filtered := all[:0]
|
||||||
|
for _, j := range all {
|
||||||
|
if kindFilter != "" && j.Kind != kindFilter {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if statusFilter != "" && j.statusLocked() != statusFilter {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
filtered = append(filtered, j)
|
||||||
|
}
|
||||||
|
all = filtered
|
||||||
|
}
|
||||||
|
|
||||||
sort.Slice(all, func(i, j int) bool {
|
sort.Slice(all, func(i, j int) bool {
|
||||||
return all[i].CreatedAt.After(all[j].CreatedAt)
|
return all[i].CreatedAt.After(all[j].CreatedAt)
|
||||||
})
|
})
|
||||||
@@ -267,6 +343,43 @@ func (r *Registry) List(tenantID, statusFilter, kindFilter, cursor string, limit
|
|||||||
return all[off:end], next, hasMore
|
return all[off:end], next, hasMore
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// CountOtherActiveModuleRefresh returns how many module_refresh jobs for the tenant are still
|
||||||
|
// queued or running, excluding excludeJobID (the current job). Used to batch deploy_apply.
|
||||||
|
func (r *Registry) CountOtherActiveModuleRefresh(tenantID, excludeJobID string) int {
|
||||||
|
return r.CountOtherActiveRefresh(tenantID, excludeJobID)
|
||||||
|
}
|
||||||
|
|
||||||
|
// CountOtherActiveRefresh counts queued/running module_refresh and tenant_refresh jobs for the tenant.
|
||||||
|
func (r *Registry) CountOtherActiveRefresh(tenantID, excludeJobID string) int {
|
||||||
|
if r == nil {
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
r.mu.RLock()
|
||||||
|
candidates := make([]*Job, 0, 8)
|
||||||
|
for _, j := range r.byID {
|
||||||
|
if j.TenantID != tenantID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if j.Kind != KindModuleRefresh && j.Kind != KindTenantRefresh {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if j.ID == excludeJobID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
candidates = append(candidates, j)
|
||||||
|
}
|
||||||
|
r.mu.RUnlock()
|
||||||
|
|
||||||
|
n := 0
|
||||||
|
for _, j := range candidates {
|
||||||
|
st := j.statusLocked()
|
||||||
|
if st == StatusQueued || st == StatusRunning {
|
||||||
|
n++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
func parseCursor(s string, off *int) error {
|
func parseCursor(s string, off *int) error {
|
||||||
_, err := fmt.Sscanf(s, "%d", off)
|
_, err := fmt.Sscanf(s, "%d", off)
|
||||||
return err
|
return err
|
||||||
|
|||||||
+407
-9
@@ -2,9 +2,13 @@ package jobs
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"fmt"
|
||||||
"net/http"
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
"os"
|
"os"
|
||||||
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"evobgp/internal/birddeploy"
|
"evobgp/internal/birddeploy"
|
||||||
@@ -37,16 +41,32 @@ func mergeBirdPostApplyMeta(j *Job) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const (
|
const (
|
||||||
KindModuleRefresh = "module_refresh"
|
KindModuleRefresh = "module_refresh"
|
||||||
KindDeployApply = "deploy_apply"
|
KindTenantRefresh = "tenant_refresh"
|
||||||
|
KindPeerReconcile = "peer_reconcile"
|
||||||
|
KindDeployApply = "deploy_apply"
|
||||||
KindRevisionRollback = "revision_rollback"
|
KindRevisionRollback = "revision_rollback"
|
||||||
KindBirdReload = "bird_reload"
|
KindBirdReload = "bird_reload"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Worker executes queued jobs against store.Backend (memory or SQL).
|
// Worker executes queued jobs against store.Backend (memory or SQL).
|
||||||
type Worker struct {
|
type Worker struct {
|
||||||
Store store.Backend
|
Store store.Backend
|
||||||
HTTPClient *http.Client // optional; CDN refresh uses this (default 45s timeout).
|
HTTPClient *http.Client // optional; CDN refresh uses this (default 45s timeout).
|
||||||
|
// Registry is set after BootstrapWorkers creates the job queue; used to chain deploy_apply after refresh/rollback.
|
||||||
|
Registry *Registry
|
||||||
|
// refreshGate serializes deploy_apply gating after module_refresh per tenant (see finishModuleRefreshSuccess).
|
||||||
|
refreshGate sync.Map // map[string]*sync.Mutex
|
||||||
|
}
|
||||||
|
|
||||||
|
type revisionLogEntry struct {
|
||||||
|
Kind string `json:"kind"`
|
||||||
|
Source string `json:"source"`
|
||||||
|
Community string `json:"community"`
|
||||||
|
CommunityLabel string `json:"community_label"`
|
||||||
|
PrefixCount int `json:"prefix_count"`
|
||||||
|
Sample []string `json:"sample,omitempty"`
|
||||||
|
Message string `json:"message"`
|
||||||
}
|
}
|
||||||
|
|
||||||
var defaultWorkerHTTP = &http.Client{Timeout: 45 * time.Second}
|
var defaultWorkerHTTP = &http.Client{Timeout: 45 * time.Second}
|
||||||
@@ -82,13 +102,15 @@ func (w *Worker) Process(j *Job) {
|
|||||||
j.Fail("missing module_id in job meta")
|
j.Fail("missing module_id in job meta")
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
rev, err := pipeline.RefreshModule(context.Background(), w.Store, w.httpClient(), j.TenantID, mid)
|
if err := pipeline.RefreshModuleIngest(context.Background(), w.Store, w.httpClient(), j.TenantID, mid); err != nil {
|
||||||
if err != nil {
|
|
||||||
j.Fail(err.Error())
|
j.Fail(err.Error())
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
j.mergeMeta(map[string]any{"revision_id": rev})
|
w.finishModuleRefreshSuccess(j, mid)
|
||||||
j.Succeed()
|
case KindTenantRefresh:
|
||||||
|
w.runTenantRefresh(j)
|
||||||
|
case KindPeerReconcile:
|
||||||
|
w.runPeerReconcile(j)
|
||||||
case KindDeployApply:
|
case KindDeployApply:
|
||||||
w.runDeployApply(j)
|
w.runDeployApply(j)
|
||||||
case KindRevisionRollback:
|
case KindRevisionRollback:
|
||||||
@@ -114,6 +136,198 @@ func (w *Worker) Process(j *Job) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (w *Worker) runPeerReconcile(j *Job) {
|
||||||
|
if w == nil || w.Store == nil {
|
||||||
|
j.Fail("worker not configured")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
const peerJobTitle = "Обновление BGP пиров"
|
||||||
|
j.mergeMeta(map[string]any{"job_title": peerJobTitle})
|
||||||
|
|
||||||
|
var revID string
|
||||||
|
latest, _, _ := w.Store.ListRevisions(j.TenantID, "", "", 1)
|
||||||
|
triggerModuleID, err := w.peerTriggerModuleID(j.TenantID, latest)
|
||||||
|
if err != nil {
|
||||||
|
j.Fail(err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if len(latest) == 0 {
|
||||||
|
// First run fallback: render full tenant state once if no baseline revision exists yet.
|
||||||
|
rid, err := pipeline.RenderTenantRevision(context.Background(), w.Store, w.httpClient(), j.TenantID, triggerModuleID)
|
||||||
|
if err != nil {
|
||||||
|
j.Fail(err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
revID = rid
|
||||||
|
} else {
|
||||||
|
baseRevID := latest[0].ID
|
||||||
|
rows := make([]store.PrefixRow, 0, 1024)
|
||||||
|
cursor := ""
|
||||||
|
for {
|
||||||
|
page, next, more := w.Store.ListRevisionPrefixes(j.TenantID, baseRevID, cursor, 2000)
|
||||||
|
rows = append(rows, page...)
|
||||||
|
if !more || strings.TrimSpace(next) == "" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
cursor = next
|
||||||
|
}
|
||||||
|
rid, err := pipeline.RenderTenantRevisionFromPrefixes(context.Background(), w.Store, w.httpClient(), j.TenantID, triggerModuleID, rows)
|
||||||
|
if err != nil {
|
||||||
|
j.Fail(err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
revID = rid
|
||||||
|
}
|
||||||
|
j.mergeMeta(map[string]any{"revision_id": revID})
|
||||||
|
if entries, total, err := w.buildRevisionLogEntries(j.TenantID, revID); err == nil {
|
||||||
|
j.mergeMeta(map[string]any{
|
||||||
|
"log_entries": entries,
|
||||||
|
"log_total": total,
|
||||||
|
"log_generated": time.Now().UTC().Format(time.RFC3339Nano),
|
||||||
|
})
|
||||||
|
} else {
|
||||||
|
j.mergeMeta(map[string]any{"log_build_error": err.Error()})
|
||||||
|
}
|
||||||
|
j.Succeed()
|
||||||
|
w.enqueueDeployAllSpeakers(j, j.TenantID, revID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *Worker) peerTriggerModuleID(tenantID string, latest []*store.Revision) (string, error) {
|
||||||
|
if len(latest) > 0 {
|
||||||
|
if mid := strings.TrimSpace(latest[0].ModuleID); mid != "" {
|
||||||
|
return mid, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, mod := range w.Store.ListModules(tenantID) {
|
||||||
|
if mod == nil || !mod.Enabled {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(mod.ID) != "" {
|
||||||
|
return mod.ID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, mod := range w.Store.ListModules(tenantID) {
|
||||||
|
if mod == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(mod.ID) != "" {
|
||||||
|
return mod.ID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("missing module_id for peer reconcile")
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *Worker) tenantRefreshMu(tenantID string) *sync.Mutex {
|
||||||
|
v, _ := w.refreshGate.LoadOrStore(tenantID, &sync.Mutex{})
|
||||||
|
return v.(*sync.Mutex)
|
||||||
|
}
|
||||||
|
|
||||||
|
// finishModuleRefreshSuccess marks the refresh job and, for the last active refresh in tenant,
|
||||||
|
// creates one aggregate revision and enqueues a single deploy_apply.
|
||||||
|
func (w *Worker) runTenantRefresh(j *Job) {
|
||||||
|
moduleIDs := moduleIDsFromJobMeta(j.Meta)
|
||||||
|
if len(moduleIDs) == 0 {
|
||||||
|
j.Fail("missing module_ids in job meta")
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if err := pipeline.RefreshTenantModules(context.Background(), w.Store, w.httpClient(), j.TenantID, moduleIDs); err != nil {
|
||||||
|
j.Fail(err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
trigger, err := pipeline.PickTenantRefreshTriggerModule(w.Store, j.TenantID, moduleIDs)
|
||||||
|
if err != nil {
|
||||||
|
j.Fail(err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
j.mergeMeta(map[string]any{"module_ids": moduleIDs, "modules_refreshed": len(moduleIDs)})
|
||||||
|
w.finishModuleRefreshSuccess(j, trigger)
|
||||||
|
}
|
||||||
|
|
||||||
|
func moduleIDsFromJobMeta(meta map[string]any) []string {
|
||||||
|
if meta == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
raw, ok := meta["module_ids"]
|
||||||
|
if !ok {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
switch v := raw.(type) {
|
||||||
|
case []string:
|
||||||
|
return v
|
||||||
|
case []any:
|
||||||
|
var out []string
|
||||||
|
for _, x := range v {
|
||||||
|
if s, ok := x.(string); ok && strings.TrimSpace(s) != "" {
|
||||||
|
out = append(out, strings.TrimSpace(s))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
default:
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *Worker) finishModuleRefreshSuccess(j *Job, triggerModuleID string) {
|
||||||
|
if w == nil || w.Store == nil {
|
||||||
|
j.Succeed()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
mu := w.tenantRefreshMu(j.TenantID)
|
||||||
|
mu.Lock()
|
||||||
|
defer mu.Unlock()
|
||||||
|
deferDeploy := false
|
||||||
|
if w.Registry != nil {
|
||||||
|
deferDeploy = w.Registry.CountOtherActiveRefresh(j.TenantID, j.ID) > 0
|
||||||
|
}
|
||||||
|
if deferDeploy {
|
||||||
|
j.mergeMeta(map[string]any{
|
||||||
|
"deploy_apply_deferred": true,
|
||||||
|
"deploy_apply_defer_reason": "parallel_module_refresh",
|
||||||
|
})
|
||||||
|
j.Succeed()
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
rev, err := pipeline.RenderTenantRevision(context.Background(), w.Store, w.httpClient(), j.TenantID, triggerModuleID)
|
||||||
|
if err != nil {
|
||||||
|
j.Fail(err.Error())
|
||||||
|
return
|
||||||
|
}
|
||||||
|
j.mergeMeta(map[string]any{"revision_id": rev})
|
||||||
|
if entries, total, err := w.buildRevisionLogEntries(j.TenantID, rev); err == nil {
|
||||||
|
j.mergeMeta(map[string]any{
|
||||||
|
"log_entries": entries,
|
||||||
|
"log_total": total,
|
||||||
|
"log_generated": time.Now().UTC().Format(time.RFC3339Nano),
|
||||||
|
})
|
||||||
|
} else {
|
||||||
|
j.mergeMeta(map[string]any{"log_build_error": err.Error()})
|
||||||
|
}
|
||||||
|
j.Succeed()
|
||||||
|
w.enqueueDeployAllSpeakers(j, j.TenantID, rev)
|
||||||
|
}
|
||||||
|
|
||||||
|
// enqueueDeployAllSpeakers queues the same work as POST /v1/apply (all speakers, no speaker_id).
|
||||||
|
func (w *Worker) enqueueDeployAllSpeakers(j *Job, tenantID, revID string) {
|
||||||
|
if w == nil || w.Registry == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
revID = strings.TrimSpace(revID)
|
||||||
|
if revID == "" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
applyJob, _, err := w.Registry.Enqueue(tenantID, KindDeployApply, nil, nil, map[string]any{
|
||||||
|
"revision_id": revID,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
j.mergeMeta(map[string]any{"deploy_apply_enqueue_error": err.Error()})
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if applyJob != nil {
|
||||||
|
j.mergeMeta(map[string]any{"deploy_apply_job_id": applyJob.ID})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func (w *Worker) runDeployApply(j *Job) {
|
func (w *Worker) runDeployApply(j *Job) {
|
||||||
revID, _ := j.Meta["revision_id"].(string)
|
revID, _ := j.Meta["revision_id"].(string)
|
||||||
spk, hasSpeaker := j.Meta["speaker_id"].(string)
|
spk, hasSpeaker := j.Meta["speaker_id"].(string)
|
||||||
@@ -145,6 +359,7 @@ func (w *Worker) runDeployApply(j *Job) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
applied := make([]string, 0, 8)
|
||||||
applyOne := func(speakerID string) error {
|
applyOne := func(speakerID string) error {
|
||||||
if err := w.Store.SetLastAppliedRevision(j.TenantID, speakerID, revID); err != nil {
|
if err := w.Store.SetLastAppliedRevision(j.TenantID, speakerID, revID); err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -153,6 +368,7 @@ func (w *Worker) runDeployApply(j *Job) {
|
|||||||
if err := w.Store.PublishRevisionForSpeaker(speakerID, revID); err != nil {
|
if err := w.Store.PublishRevisionForSpeaker(speakerID, revID); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
applied = append(applied, speakerID)
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
if hasSpeaker && spk != "" {
|
if hasSpeaker && spk != "" {
|
||||||
@@ -170,6 +386,14 @@ func (w *Worker) runDeployApply(j *Job) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
j.mergeMeta(map[string]any{
|
||||||
|
"apply_summary": map[string]any{
|
||||||
|
"revision_id": revID,
|
||||||
|
"speakers_count": len(applied),
|
||||||
|
"speaker_ids": applied,
|
||||||
|
"message": fmt.Sprintf("Ревизия %s применена на %d спикерах", shortID(revID), len(applied)),
|
||||||
|
},
|
||||||
|
})
|
||||||
mergeBirdPostApplyMeta(j)
|
mergeBirdPostApplyMeta(j)
|
||||||
j.Succeed()
|
j.Succeed()
|
||||||
}
|
}
|
||||||
@@ -186,5 +410,179 @@ func (w *Worker) runRollback(j *Job) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
j.mergeMeta(map[string]any{"new_revision_id": newID})
|
j.mergeMeta(map[string]any{"new_revision_id": newID})
|
||||||
|
j.mergeMeta(map[string]any{
|
||||||
|
"rollback_summary": map[string]any{
|
||||||
|
"source_revision_id": src,
|
||||||
|
"new_revision_id": newID,
|
||||||
|
"message": fmt.Sprintf("Откат %s → %s", shortID(src), shortID(newID)),
|
||||||
|
},
|
||||||
|
})
|
||||||
|
w.enqueueDeployAllSpeakers(j, j.TenantID, newID)
|
||||||
j.Succeed()
|
j.Succeed()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// buildCommunityLabelMap maps community UUID -> human-readable title (or BGP community string).
|
||||||
|
func buildCommunityLabelMap(st store.Backend, tenantID string) map[string]string {
|
||||||
|
out := make(map[string]string)
|
||||||
|
if st == nil {
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
list, err := st.ListCommunities(tenantID)
|
||||||
|
if err != nil || list == nil {
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
for _, c := range list {
|
||||||
|
if c == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
label := strings.TrimSpace(c.Title)
|
||||||
|
if label == "" {
|
||||||
|
label = strings.TrimSpace(c.Community)
|
||||||
|
}
|
||||||
|
if label == "" {
|
||||||
|
label = c.ID
|
||||||
|
}
|
||||||
|
out[c.ID] = label
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveCommunityLabel(commID string, byID map[string]string) string {
|
||||||
|
if commID == "" || commID == "none" {
|
||||||
|
return "без community"
|
||||||
|
}
|
||||||
|
if lbl, ok := byID[commID]; ok && strings.TrimSpace(lbl) != "" {
|
||||||
|
return strings.TrimSpace(lbl)
|
||||||
|
}
|
||||||
|
return commID
|
||||||
|
}
|
||||||
|
|
||||||
|
func (w *Worker) buildRevisionLogEntries(tenantID, revID string) ([]map[string]any, int, error) {
|
||||||
|
if w == nil || w.Store == nil {
|
||||||
|
return nil, 0, fmt.Errorf("store not configured")
|
||||||
|
}
|
||||||
|
commLabels := buildCommunityLabelMap(w.Store, tenantID)
|
||||||
|
type agg struct {
|
||||||
|
kind string
|
||||||
|
source string
|
||||||
|
community string
|
||||||
|
count int
|
||||||
|
sample []string
|
||||||
|
}
|
||||||
|
groups := map[string]*agg{}
|
||||||
|
total := 0
|
||||||
|
cursor := ""
|
||||||
|
for {
|
||||||
|
rows, next, more := w.Store.ListRevisionPrefixes(tenantID, revID, cursor, 1000)
|
||||||
|
for _, p := range rows {
|
||||||
|
total++
|
||||||
|
src := strings.TrimSpace(p.Source)
|
||||||
|
comm := "none"
|
||||||
|
if p.CommunityID != nil && strings.TrimSpace(*p.CommunityID) != "" {
|
||||||
|
comm = strings.TrimSpace(*p.CommunityID)
|
||||||
|
}
|
||||||
|
kind, sourceName := classifySource(src)
|
||||||
|
k := kind + "|" + sourceName + "|" + comm
|
||||||
|
g, ok := groups[k]
|
||||||
|
if !ok {
|
||||||
|
g = &agg{kind: kind, source: sourceName, community: comm}
|
||||||
|
groups[k] = g
|
||||||
|
}
|
||||||
|
g.count++
|
||||||
|
if len(g.sample) < 5 {
|
||||||
|
g.sample = append(g.sample, p.Prefix)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !more {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
cursor = next
|
||||||
|
if strings.TrimSpace(cursor) == "" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
}
|
||||||
|
keys := make([]string, 0, len(groups))
|
||||||
|
for k := range groups {
|
||||||
|
keys = append(keys, k)
|
||||||
|
}
|
||||||
|
sort.Strings(keys)
|
||||||
|
out := make([]map[string]any, 0, len(keys))
|
||||||
|
for _, k := range keys {
|
||||||
|
g := groups[k]
|
||||||
|
cl := resolveCommunityLabel(g.community, commLabels)
|
||||||
|
msg := humanLogMessage(g.kind, g.source, g.count, cl, g.sample)
|
||||||
|
out = append(out, map[string]any{
|
||||||
|
"kind": g.kind,
|
||||||
|
"source": g.source,
|
||||||
|
"community": g.community,
|
||||||
|
"community_label": cl,
|
||||||
|
"prefix_count": g.count,
|
||||||
|
"sample": g.sample,
|
||||||
|
"message": msg,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return out, total, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func classifySource(src string) (kind, name string) {
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(src, "as:"):
|
||||||
|
return "asn", strings.TrimPrefix(src, "as:")
|
||||||
|
case strings.HasPrefix(src, "domain:"):
|
||||||
|
return "domain", strings.TrimPrefix(src, "domain:")
|
||||||
|
case strings.HasPrefix(src, "cdn:"):
|
||||||
|
return "cdn", strings.TrimPrefix(src, "cdn:")
|
||||||
|
case src == "ip_range":
|
||||||
|
return "ip_range", "manual_ranges"
|
||||||
|
default:
|
||||||
|
if src == "" {
|
||||||
|
return "unknown", "unknown"
|
||||||
|
}
|
||||||
|
return "source", src
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// humanLogMessage builds a Russian log line; communityLabel is already resolved (title or BGP value).
|
||||||
|
func humanLogMessage(kind, source string, count int, communityLabel string, sample []string) string {
|
||||||
|
switch kind {
|
||||||
|
case "asn":
|
||||||
|
return fmt.Sprintf("AS%s: добавлено %d префиксов в сообщество «%s»", source, count, communityLabel)
|
||||||
|
case "domain":
|
||||||
|
ips := strings.Join(prettyDomainSample(sample), " ")
|
||||||
|
if ips == "" {
|
||||||
|
ips = "—"
|
||||||
|
}
|
||||||
|
return fmt.Sprintf("%s: IP (%s) → добавлено в сообщество «%s»", source, ips, communityLabel)
|
||||||
|
case "cdn":
|
||||||
|
return fmt.Sprintf("CDN «%s»: добавлено %d префиксов в сообщество «%s»", source, count, communityLabel)
|
||||||
|
case "ip_range":
|
||||||
|
return fmt.Sprintf("Статические диапазоны: добавлено %d префиксов в сообщество «%s»", count, communityLabel)
|
||||||
|
default:
|
||||||
|
return fmt.Sprintf("%s: добавлено %d префиксов в сообщество «%s»", source, count, communityLabel)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func prettyDomainSample(sample []string) []string {
|
||||||
|
out := make([]string, 0, len(sample))
|
||||||
|
for _, s := range sample {
|
||||||
|
p, err := netip.ParsePrefix(strings.TrimSpace(s))
|
||||||
|
if err != nil {
|
||||||
|
out = append(out, s)
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if (p.Addr().Is4() && p.Bits() == 32) || (p.Addr().Is6() && p.Bits() == 128) {
|
||||||
|
out = append(out, p.Addr().String())
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, s)
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func shortID(id string) string {
|
||||||
|
s := strings.TrimSpace(id)
|
||||||
|
if len(s) <= 8 {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
return s[:8]
|
||||||
|
}
|
||||||
@@ -0,0 +1,178 @@
|
|||||||
|
package jobs
|
||||||
|
|
||||||
|
import (
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestParallelModuleRefresh_CoalescesDeployApply(t *testing.T) {
|
||||||
|
t.Setenv("EVOBGP_ASN_RESOLVE", "0")
|
||||||
|
t.Setenv("EVOBGP_BIRD_ACTIVE_DIR", "") // skip bird binary path in deploy_apply
|
||||||
|
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, modIP, _, _ := m.DemoIDs()
|
||||||
|
for _, mod := range m.ListModules(tenant) {
|
||||||
|
if mod == nil || mod.ID == modIP {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
disabled := false
|
||||||
|
if _, err := m.UpdateModule(tenant, mod.ID, &store.ModulePatch{Enabled: &disabled}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
mod2, err := m.CreateModule(tenant, &store.Module{Type: "IP_RANGES", Name: "extra-ip", Enabled: true, Priority: 30})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := m.CreateIPRangeEntry(tenant, modIP, &store.IPRangeEntry{Prefix: "10.0.0.0/24"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := m.CreateIPRangeEntry(tenant, mod2.ID, &store.IPRangeEntry{Prefix: "192.168.0.0/24"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
wk := &Worker{Store: m}
|
||||||
|
reg := NewRegistry(wk.Process)
|
||||||
|
wk.Registry = reg
|
||||||
|
|
||||||
|
mid1 := modIP
|
||||||
|
mid2 := mod2.ID
|
||||||
|
if _, _, err := reg.Enqueue(tenant, KindModuleRefresh, nil, &mid1, map[string]any{"module_id": modIP}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := reg.Enqueue(tenant, KindModuleRefresh, nil, &mid2, map[string]any{"module_id": mod2.ID}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
waitSucceededJobsByKindCount(t, reg, tenant, KindModuleRefresh, 2)
|
||||||
|
|
||||||
|
deployJobs, _, _ := reg.List(tenant, "", KindDeployApply, "", 100)
|
||||||
|
if len(deployJobs) != 1 {
|
||||||
|
t.Fatalf("want exactly one deploy_apply job, got %d", len(deployJobs))
|
||||||
|
}
|
||||||
|
|
||||||
|
var deferred, withDeployID int
|
||||||
|
refreshJobs, _, _ := reg.List(tenant, "", KindModuleRefresh, "", 100)
|
||||||
|
for _, j := range refreshJobs {
|
||||||
|
if j == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
snap := j.Snapshot()
|
||||||
|
if st, _ := snap["status"].(string); st != StatusSucceeded {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
meta, _ := snap["meta"].(map[string]any)
|
||||||
|
if v, ok := meta["deploy_apply_deferred"].(bool); ok && v {
|
||||||
|
deferred++
|
||||||
|
}
|
||||||
|
if _, ok := meta["deploy_apply_job_id"].(string); ok {
|
||||||
|
withDeployID++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if deferred != 1 || withDeployID != 1 {
|
||||||
|
t.Fatalf("want one deferred and one with deploy_apply_job_id, got deferred=%d deploy_meta=%d", deferred, withDeployID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func waitSucceededModuleRefreshCount(t *testing.T, reg *Registry, tenant string, want int) {
|
||||||
|
t.Helper()
|
||||||
|
waitSucceededJobsByKindCount(t, reg, tenant, KindModuleRefresh, want)
|
||||||
|
}
|
||||||
|
|
||||||
|
func waitSucceededJobsByKindCount(t *testing.T, reg *Registry, tenant, kind string, want int) {
|
||||||
|
t.Helper()
|
||||||
|
deadline := time.Now().Add(30 * time.Second)
|
||||||
|
for time.Now().Before(deadline) {
|
||||||
|
jobs, _, _ := reg.List(tenant, StatusSucceeded, kind, "", 100)
|
||||||
|
if len(jobs) >= want {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
all, _, _ := reg.List(tenant, "", kind, "", 100)
|
||||||
|
for _, j := range all {
|
||||||
|
if j == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
snap := j.Snapshot()
|
||||||
|
if st, _ := snap["status"].(string); st != StatusFailed {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
t.Fatalf("%s job failed: %#v", kind, snap["error"])
|
||||||
|
}
|
||||||
|
time.Sleep(5 * time.Millisecond)
|
||||||
|
}
|
||||||
|
t.Fatalf("timeout waiting for %s jobs", kind)
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestPeerReconcile_RendersFromLatestRevisionAndQueuesDeploy(t *testing.T) {
|
||||||
|
t.Setenv("EVOBGP_ASN_RESOLVE", "0")
|
||||||
|
t.Setenv("EVOBGP_BIRD_ACTIVE_DIR", "") // skip bird binary path in deploy_apply
|
||||||
|
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, modIP, _, _ := m.DemoIDs()
|
||||||
|
for _, mod := range m.ListModules(tenant) {
|
||||||
|
if mod == nil || mod.ID == modIP {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
disabled := false
|
||||||
|
if _, err := m.UpdateModule(tenant, mod.ID, &store.ModulePatch{Enabled: &disabled}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if _, err := m.CreateIPRangeEntry(tenant, modIP, &store.IPRangeEntry{Prefix: "10.10.0.0/24"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
peer, err := m.CreatePeer(tenant, &store.BGPPeer{
|
||||||
|
Name: "peer-a",
|
||||||
|
Neighbor: "192.0.2.2",
|
||||||
|
RemoteASN: 64512,
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
wk := &Worker{Store: m}
|
||||||
|
reg := NewRegistry(wk.Process)
|
||||||
|
wk.Registry = reg
|
||||||
|
|
||||||
|
// Create baseline revision from enabled IP module.
|
||||||
|
mid := modIP
|
||||||
|
if _, _, err := reg.Enqueue(tenant, KindModuleRefresh, nil, &mid, map[string]any{"module_id": modIP}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
waitSucceededJobsByKindCount(t, reg, tenant, KindModuleRefresh, 1)
|
||||||
|
beforeRevs, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
|
|
||||||
|
enabled := false
|
||||||
|
if _, err := m.UpdatePeer(tenant, peer.ID, &store.PeerPatch{Enabled: &enabled}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, _, err := reg.Enqueue(tenant, KindPeerReconcile, nil, nil, map[string]any{"trigger": "peer_patch"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
waitSucceededJobsByKindCount(t, reg, tenant, KindPeerReconcile, 1)
|
||||||
|
|
||||||
|
afterRevs, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
|
if len(afterRevs) <= len(beforeRevs) {
|
||||||
|
t.Fatalf("expected new revision after peer_reconcile, before=%d after=%d", len(beforeRevs), len(afterRevs))
|
||||||
|
}
|
||||||
|
peerJobs, _, _ := reg.List(tenant, StatusSucceeded, KindPeerReconcile, "", 10)
|
||||||
|
if len(peerJobs) == 0 {
|
||||||
|
t.Fatal("expected succeeded peer_reconcile job")
|
||||||
|
}
|
||||||
|
meta := peerJobs[0].Snapshot()["meta"].(map[string]any)
|
||||||
|
if _, ok := meta["revision_id"].(string); !ok {
|
||||||
|
t.Fatalf("expected revision_id in peer_reconcile meta, got %#v", meta)
|
||||||
|
}
|
||||||
|
if title, ok := meta["job_title"].(string); !ok || title != "Обновление BGP пиров" {
|
||||||
|
t.Fatalf("expected job_title in peer_reconcile meta, got %#v", meta["job_title"])
|
||||||
|
}
|
||||||
|
if _, ok := meta["deploy_apply_job_id"].(string); !ok {
|
||||||
|
t.Fatalf("expected deploy_apply_job_id in peer_reconcile meta, got %#v", meta)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,214 @@
|
|||||||
|
package nodecli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"crypto/ed25519"
|
||||||
|
"encoding/base64"
|
||||||
|
"encoding/json"
|
||||||
|
"flag"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"path/filepath"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/birdfmt"
|
||||||
|
"evobgp/internal/bundle"
|
||||||
|
"evobgp/internal/signing"
|
||||||
|
)
|
||||||
|
|
||||||
|
// CmdPullBundle downloads a signed bundle from the control plane API.
|
||||||
|
func CmdPullBundle(args []string) int {
|
||||||
|
fs := flag.NewFlagSet("pull-bundle", flag.ExitOnError)
|
||||||
|
base := fs.String("base-url", "", "control plane base URL, e.g. http://localhost:8080")
|
||||||
|
token := fs.String("token", "", "Bearer token (node role)")
|
||||||
|
speaker := fs.String("speaker-id", "", "bgp_speaker id")
|
||||||
|
revision := fs.String("revision-id", "", "revision to fetch (empty = latest pointer)")
|
||||||
|
out := fs.String("o", "bundle.tar.gz", "output file")
|
||||||
|
_ = fs.Parse(args)
|
||||||
|
|
||||||
|
if strings.TrimSpace(*base) == "" || *token == "" || *speaker == "" {
|
||||||
|
fmt.Fprintln(os.Stderr, "pull-bundle: -base-url, -token, -speaker-id are required")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
rev := strings.TrimSpace(*revision)
|
||||||
|
if rev == "" {
|
||||||
|
var err error
|
||||||
|
rev, err = fetchLatestRevision(*base, *token, *speaker)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
}
|
||||||
|
body, err := fetchBundle(*base, *token, *speaker, rev)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if err := os.WriteFile(*out, body, 0o644); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
fmt.Fprintf(os.Stderr, "wrote %s (revision %s)\n", *out, rev)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
func fetchLatestRevision(base, token, speaker string) (string, error) {
|
||||||
|
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/revisions/latest"
|
||||||
|
req, err := http.NewRequest(http.MethodGet, u, nil)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
req.Header.Set("Authorization", "Bearer "+token)
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(resp.Body)
|
||||||
|
return "", fmt.Errorf("latest revision: %s: %s", resp.Status, strings.TrimSpace(string(b)))
|
||||||
|
}
|
||||||
|
var out struct {
|
||||||
|
RevisionID string `json:"revision_id"`
|
||||||
|
}
|
||||||
|
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if out.RevisionID == "" {
|
||||||
|
return "", fmt.Errorf("empty revision_id in response")
|
||||||
|
}
|
||||||
|
return out.RevisionID, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func fetchBundle(base, token, speaker, revision string) ([]byte, error) {
|
||||||
|
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/bundle/" + revision
|
||||||
|
req, err := http.NewRequest(http.MethodGet, u, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
req.Header.Set("Authorization", "Bearer "+token)
|
||||||
|
resp, err := http.DefaultClient.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
b, _ := io.ReadAll(resp.Body)
|
||||||
|
return nil, fmt.Errorf("bundle: %s: %s", resp.Status, strings.TrimSpace(string(b)))
|
||||||
|
}
|
||||||
|
return io.ReadAll(resp.Body)
|
||||||
|
}
|
||||||
|
|
||||||
|
func loadPubKey(pubB64, pubHex string) (ed25519.PublicKey, error) {
|
||||||
|
switch {
|
||||||
|
case strings.TrimSpace(pubB64) != "":
|
||||||
|
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(pubB64))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if len(raw) != ed25519.PublicKeySize {
|
||||||
|
return nil, fmt.Errorf("pubkey-base64 must decode to %d bytes", ed25519.PublicKeySize)
|
||||||
|
}
|
||||||
|
return ed25519.PublicKey(raw), nil
|
||||||
|
case strings.TrimSpace(pubHex) != "":
|
||||||
|
return bundle.ParsePublicKeyHex(pubHex)
|
||||||
|
default:
|
||||||
|
return nil, fmt.Errorf("public key required")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// CmdVerifyBundle checks bundle signature and manifest.
|
||||||
|
func CmdVerifyBundle(args []string) int {
|
||||||
|
fs := flag.NewFlagSet("verify-bundle", flag.ExitOnError)
|
||||||
|
path := fs.String("f", "", "path to bundle.tar.gz")
|
||||||
|
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
|
||||||
|
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
|
||||||
|
_ = fs.Parse(args)
|
||||||
|
|
||||||
|
if *path == "" {
|
||||||
|
fmt.Fprintln(os.Stderr, "verify-bundle: -f required")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
pub, err := loadPubKey(*pubB64, *pubHex)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
raw, err := os.ReadFile(*path)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
v, err := signing.VerifyGzippedTar(raw, pub)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
fmt.Fprintf(os.Stderr, "ok: revision %s, %d payload files\n", v.Manifest.RevisionID, len(v.Files))
|
||||||
|
return 0
|
||||||
|
}
|
||||||
|
|
||||||
|
// CmdApplyBundle verifies, extracts, parse-checks, and configures BIRD from a bundle.
|
||||||
|
func CmdApplyBundle(args []string) int {
|
||||||
|
fs := flag.NewFlagSet("apply-bundle", flag.ExitOnError)
|
||||||
|
path := fs.String("f", "", "path to bundle.tar.gz")
|
||||||
|
dir := fs.String("extract-dir", "", "directory to extract into")
|
||||||
|
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
|
||||||
|
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
|
||||||
|
bird := fs.String("bird", "", "bird binary (default PATH)")
|
||||||
|
birdc := fs.String("birdc", "", "birdc binary (default PATH)")
|
||||||
|
socket := fs.String("socket", "", "birdc -s socket path")
|
||||||
|
timeout := fs.Duration("timeout", 30*time.Second, "timeout for bird/birdc")
|
||||||
|
_ = fs.Parse(args)
|
||||||
|
|
||||||
|
if *path == "" || *dir == "" {
|
||||||
|
fmt.Fprintln(os.Stderr, "apply-bundle: -f and -extract-dir required")
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
pub, err := loadPubKey(*pubB64, *pubHex)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
raw, err := os.ReadFile(*path)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
v, err := signing.VerifyGzippedTar(raw, pub)
|
||||||
|
if err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
root := filepath.Clean(*dir)
|
||||||
|
if err := os.MkdirAll(root, 0o755); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if err := bundle.WriteExtractedFiles(root, v); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
mainRel := v.FindMainBirdConf()
|
||||||
|
if mainRel == "" {
|
||||||
|
fmt.Fprintln(os.Stderr, "bundle has no bird.conf path in manifest")
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
mainPath := filepath.Join(root, filepath.FromSlash(strings.TrimPrefix(mainRel, "/")))
|
||||||
|
ctx, cancel := context.WithTimeout(context.Background(), *timeout)
|
||||||
|
defer cancel()
|
||||||
|
ctl := &birdfmt.BirdCtl{Bird: *bird, Birdc: *birdc, Socket: *socket}
|
||||||
|
if err := ctl.ParseCheck(ctx, mainPath); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
if err := ctl.Configure(ctx); err != nil {
|
||||||
|
fmt.Fprintln(os.Stderr, err)
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
fmt.Fprintf(os.Stderr, "applied revision %s (main config %s)\n", v.Manifest.RevisionID, mainPath)
|
||||||
|
return 0
|
||||||
|
}
|
||||||
@@ -0,0 +1,38 @@
|
|||||||
|
// Package nodecli implements evobgp-node subcommands (pull/verify/apply bundle).
|
||||||
|
package nodecli
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"os"
|
||||||
|
)
|
||||||
|
|
||||||
|
// Run executes a subcommand from args (without program name). Returns exit code.
|
||||||
|
func Run(args []string) int {
|
||||||
|
if len(args) < 1 {
|
||||||
|
Usage(os.Stderr)
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
switch args[0] {
|
||||||
|
case "pull-bundle":
|
||||||
|
return CmdPullBundle(args[1:])
|
||||||
|
case "verify-bundle":
|
||||||
|
return CmdVerifyBundle(args[1:])
|
||||||
|
case "apply-bundle":
|
||||||
|
return CmdApplyBundle(args[1:])
|
||||||
|
default:
|
||||||
|
Usage(os.Stderr)
|
||||||
|
return 2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Usage prints CLI help to w.
|
||||||
|
func Usage(w interface{ Write([]byte) (int, error) }) {
|
||||||
|
fmt.Fprintf(w, `Usage:
|
||||||
|
evobgp-node pull-bundle -base-url URL -token TOKEN -speaker-id ID [-revision-id ID] [-o path]
|
||||||
|
evobgp-node verify-bundle -f bundle.tar.gz (-pubkey-base64 B64 | -pubkey-hex HEX)
|
||||||
|
evobgp-node apply-bundle -f bundle.tar.gz -extract-dir DIR (-pubkey-base64 B64 | -pubkey-hex HEX)
|
||||||
|
[-bird PATH] [-birdc PATH] [-socket PATH] [-timeout DURATION]
|
||||||
|
|
||||||
|
apply-bundle verifies, extracts, runs bird -p on main bird.conf, then birdc configure.
|
||||||
|
`)
|
||||||
|
}
|
||||||
@@ -59,8 +59,42 @@ var (
|
|||||||
Name: "build_info",
|
Name: "build_info",
|
||||||
Help: "Build metadata (value always 1).",
|
Help: "Build metadata (value always 1).",
|
||||||
}, []string{"version", "git_sha"})
|
}, []string{"version", "git_sha"})
|
||||||
|
|
||||||
|
prefixAggregationDuration = promauto.NewHistogram(prometheus.HistogramOpts{
|
||||||
|
Namespace: namespace,
|
||||||
|
Name: "prefix_aggregation_duration_seconds",
|
||||||
|
Help: "Time spent in smartAggregatePrefixRows during tenant render.",
|
||||||
|
Buckets: prometheus.ExponentialBuckets(0.0001, 2, 16),
|
||||||
|
})
|
||||||
|
|
||||||
|
prefixAggregationRawCount = promauto.NewHistogram(prometheus.HistogramOpts{
|
||||||
|
Namespace: namespace,
|
||||||
|
Name: "prefix_aggregation_raw_count",
|
||||||
|
Help: "Prefix row count before CIDR aggregation on tenant render.",
|
||||||
|
Buckets: prometheus.ExponentialBuckets(1, 2, 16),
|
||||||
|
})
|
||||||
|
|
||||||
|
prefixAggregationAggregatedCount = promauto.NewHistogram(prometheus.HistogramOpts{
|
||||||
|
Namespace: namespace,
|
||||||
|
Name: "prefix_aggregation_aggregated_count",
|
||||||
|
Help: "Prefix row count after CIDR aggregation on tenant render.",
|
||||||
|
Buckets: prometheus.ExponentialBuckets(1, 2, 16),
|
||||||
|
})
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// RecordPrefixAggregation records tenant render CIDR aggregation stats.
|
||||||
|
func RecordPrefixAggregation(rawCount, aggregatedCount int, duration time.Duration) {
|
||||||
|
if rawCount < 0 {
|
||||||
|
rawCount = 0
|
||||||
|
}
|
||||||
|
if aggregatedCount < 0 {
|
||||||
|
aggregatedCount = 0
|
||||||
|
}
|
||||||
|
prefixAggregationDuration.Observe(duration.Seconds())
|
||||||
|
prefixAggregationRawCount.Observe(float64(rawCount))
|
||||||
|
prefixAggregationAggregatedCount.Observe(float64(aggregatedCount))
|
||||||
|
}
|
||||||
|
|
||||||
// RecordJobTerminal increments jobs_finished_total for terminal statuses.
|
// RecordJobTerminal increments jobs_finished_total for terminal statuses.
|
||||||
func RecordJobTerminal(kind, status string) {
|
func RecordJobTerminal(kind, status string) {
|
||||||
switch status {
|
switch status {
|
||||||
@@ -196,14 +230,15 @@ func (s *statusRecorder) WriteHeader(code int) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// StartBirdProtocolsPoller runs birdc "show protocols" on interval when socket is non-empty.
|
// StartBirdProtocolsPoller runs birdc "show protocols" on interval when socket is non-empty.
|
||||||
func StartBirdProtocolsPoller(socket string, birdcPath string, interval time.Duration, showFn func(ctx context.Context, socket, birdcBin string) (string, error), countFn func(output string) int) {
|
// Горутина завершается при отмене ctx (корректное завершение вместе с процессом API).
|
||||||
|
func StartBirdProtocolsPoller(ctx context.Context, socket string, birdcPath string, interval time.Duration, showFn func(ctx context.Context, socket, birdcBin string) (string, error), countFn func(output string) int) {
|
||||||
socket = trimSpace(socket)
|
socket = trimSpace(socket)
|
||||||
if socket == "" || interval <= 0 || showFn == nil || countFn == nil {
|
if ctx == nil || socket == "" || interval <= 0 || showFn == nil || countFn == nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
scrape := func() {
|
scrape := func() {
|
||||||
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
sctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
|
||||||
out, err := showFn(ctx, socket, birdcPath)
|
out, err := showFn(sctx, socket, birdcPath)
|
||||||
cancel()
|
cancel()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
SetBirdSessionMetrics(0, false)
|
SetBirdSessionMetrics(0, false)
|
||||||
@@ -215,8 +250,13 @@ func StartBirdProtocolsPoller(socket string, birdcPath string, interval time.Dur
|
|||||||
scrape()
|
scrape()
|
||||||
t := time.NewTicker(interval)
|
t := time.NewTicker(interval)
|
||||||
defer t.Stop()
|
defer t.Stop()
|
||||||
for range t.C {
|
for {
|
||||||
scrape()
|
select {
|
||||||
|
case <-ctx.Done():
|
||||||
|
return
|
||||||
|
case <-t.C:
|
||||||
|
scrape()
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,106 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func collectConcurrency() int {
|
||||||
|
n := 8
|
||||||
|
if s := strings.TrimSpace(os.Getenv("EVOBGP_COLLECT_CONCURRENCY")); s != "" {
|
||||||
|
if v, err := strconv.Atoi(s); err == nil && v > 0 {
|
||||||
|
n = v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if n > 32 {
|
||||||
|
n = 32
|
||||||
|
}
|
||||||
|
return n
|
||||||
|
}
|
||||||
|
|
||||||
|
// aggregateTenantPrefixRowsAll builds the union of materialized prefixes for all enabled modules.
|
||||||
|
// Uses per-module snapshots when inputs are unchanged to avoid duplicate external fetches on render.
|
||||||
|
func aggregateTenantPrefixRowsAll(ctx context.Context, st store.Backend, hc *http.Client, tenantID string) ([]store.PrefixRow, error) {
|
||||||
|
mods := st.ListModules(tenantID)
|
||||||
|
type modRef struct {
|
||||||
|
id string
|
||||||
|
}
|
||||||
|
var enabled []modRef
|
||||||
|
for _, m := range mods {
|
||||||
|
if m != nil && m.Enabled {
|
||||||
|
enabled = append(enabled, modRef{id: m.ID})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(enabled) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
if len(enabled) == 1 {
|
||||||
|
omod, err := st.GetModule(tenantID, enabled[0].id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return rowsForModule(ctx, st, hc, tenantID, omod)
|
||||||
|
}
|
||||||
|
|
||||||
|
sem := make(chan struct{}, collectConcurrency())
|
||||||
|
results := make([][]store.PrefixRow, len(enabled))
|
||||||
|
errs := make([]error, len(enabled))
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i, ref := range enabled {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(idx int, moduleID string) {
|
||||||
|
defer wg.Done()
|
||||||
|
sem <- struct{}{}
|
||||||
|
defer func() { <-sem }()
|
||||||
|
omod, err := st.GetModule(tenantID, moduleID)
|
||||||
|
if err != nil {
|
||||||
|
errs[idx] = err
|
||||||
|
return
|
||||||
|
}
|
||||||
|
rows, err := rowsForModule(ctx, st, hc, tenantID, omod)
|
||||||
|
if err != nil {
|
||||||
|
errs[idx] = fmt.Errorf("module %s: %w", moduleID, err)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
results[idx] = rows
|
||||||
|
}(i, ref.id)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
for _, err := range errs {
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var out []store.PrefixRow
|
||||||
|
for _, part := range results {
|
||||||
|
out = append(out, part...)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func rowsForModule(ctx context.Context, st store.Backend, hc *http.Client, tenantID string, mod *store.Module) ([]store.PrefixRow, error) {
|
||||||
|
if rows, ok, err := moduleRowsFromSnapshot(st, tenantID, mod); err != nil {
|
||||||
|
return nil, err
|
||||||
|
} else if ok {
|
||||||
|
return rows, nil
|
||||||
|
}
|
||||||
|
var prior []store.PrefixRow
|
||||||
|
if snap, ok, _ := st.GetModulePrefixSnapshot(tenantID, mod.ID); ok && snap != nil {
|
||||||
|
prior = snap.Prefixes
|
||||||
|
}
|
||||||
|
rows, err := collectModulePrefixRows(ctx, st, hc, tenantID, mod, prior)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if err := persistModuleSnapshot(st, tenantID, mod, rows); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return rows, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
|
"os"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/asnresolve"
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func asnCacheTTL() time.Duration {
|
||||||
|
sec := 1800
|
||||||
|
if s := strings.TrimSpace(os.Getenv("EVOBGP_ASN_CACHE_TTL_SEC")); s != "" {
|
||||||
|
if v, err := strconv.Atoi(s); err == nil && v > 0 {
|
||||||
|
sec = v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return time.Duration(sec) * time.Second
|
||||||
|
}
|
||||||
|
|
||||||
|
// resolveASNForEntry fetches prefixes and holder with shared TTL cache (asn_prefix_cache).
|
||||||
|
func resolveASNForEntry(ctx context.Context, st store.Backend, hc *http.Client, asn int64) ([]netip.Prefix, string, error) {
|
||||||
|
ttl := asnCacheTTL()
|
||||||
|
if st != nil {
|
||||||
|
if ent, ok, err := st.GetASNPrefixCache(asn); err == nil && ok && ent != nil && time.Since(ent.FetchedAt) < ttl {
|
||||||
|
out := make([]netip.Prefix, 0, len(ent.Prefixes))
|
||||||
|
for _, p := range ent.Prefixes {
|
||||||
|
pfx, perr := netip.ParsePrefix(strings.TrimSpace(p))
|
||||||
|
if perr != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
out = append(out, pfx.Masked())
|
||||||
|
}
|
||||||
|
return out, ent.Holder, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
pfxs, err := asnresolve.AnnouncedPrefixes(ctx, hc, asn)
|
||||||
|
if err != nil {
|
||||||
|
return nil, "", err
|
||||||
|
}
|
||||||
|
holder, _ := asnresolve.ASHolderName(ctx, hc, asn)
|
||||||
|
if st != nil {
|
||||||
|
strs := make([]string, len(pfxs))
|
||||||
|
for i, p := range pfxs {
|
||||||
|
strs[i] = p.String()
|
||||||
|
}
|
||||||
|
if err := st.SetASNPrefixCache(asn, holder, strs); err != nil {
|
||||||
|
return nil, "", fmt.Errorf("asn cache AS%d: %w", asn, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return pfxs, holder, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,64 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"sync/atomic"
|
||||||
|
"testing"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResolveASNForEntry_UsesTTLCache(t *testing.T) {
|
||||||
|
m := store.NewMemory()
|
||||||
|
t.Setenv("EVOBGP_ASN_CACHE_TTL_SEC", "3600")
|
||||||
|
|
||||||
|
var calls atomic.Int32
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
calls.Add(1)
|
||||||
|
if strings.Contains(r.URL.Path, "/announced") {
|
||||||
|
_, _ = w.Write([]byte(`{"status":"ok","data":{"prefixes":[{"prefix":"203.0.113.0/24"}]}}`))
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = w.Write([]byte(`{"status":"ok","data":{"holder":"Test AS"}}`))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
t.Setenv("EVOBGP_RIPESTAT_ANNOUNCED_PREFIXES_URL", srv.URL+"/announced")
|
||||||
|
t.Setenv("EVOBGP_RIPESTAT_AS_OVERVIEW_URL", srv.URL+"/overview")
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
hc := srv.Client()
|
||||||
|
p1, h1, err := resolveASNForEntry(ctx, m, hc, 64512)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(p1) != 1 || h1 != "Test AS" {
|
||||||
|
t.Fatalf("unexpected first resolve: %+v holder=%q", p1, h1)
|
||||||
|
}
|
||||||
|
if calls.Load() < 2 {
|
||||||
|
t.Fatalf("expected ripestat calls on first resolve, got %d", calls.Load())
|
||||||
|
}
|
||||||
|
firstCalls := calls.Load()
|
||||||
|
_, _, err = resolveASNForEntry(ctx, m, hc, 64512)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if calls.Load() != firstCalls {
|
||||||
|
t.Fatalf("expected cache hit (no new HTTP), calls went from %d to %d", firstCalls, calls.Load())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestModuleDueForScheduler_BucketRollover(t *testing.T) {
|
||||||
|
mod := &store.Module{Enabled: true, Type: "CDN_CIDRS", RefreshIntervalSec: 300}
|
||||||
|
boundary := time.Unix(300, 0)
|
||||||
|
if !ModuleDueForScheduler(mod, boundary) {
|
||||||
|
t.Fatal("expected due when refresh bucket rolls")
|
||||||
|
}
|
||||||
|
mid := time.Unix(330, 0)
|
||||||
|
if ModuleDueForScheduler(mod, mid) {
|
||||||
|
t.Fatal("expected not due within same bucket")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -3,6 +3,7 @@ package pipeline
|
|||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
|
"sort"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
@@ -10,12 +11,18 @@ import (
|
|||||||
"evobgp/internal/store"
|
"evobgp/internal/store"
|
||||||
)
|
)
|
||||||
|
|
||||||
// materializeRowsForBird turns prefix rows into export-filter CIDR lists, optional AS_PATH rules, and static routes with per-prefix communities.
|
type staticCommunityRoutes struct {
|
||||||
// Duplicate CIDRs: first occurrence wins for static community; filters use unique CIDRs.
|
CommunityID string
|
||||||
|
RoutesV4 []birdfmt.StaticRoute
|
||||||
|
RoutesV6 []birdfmt.StaticRoute
|
||||||
|
}
|
||||||
|
|
||||||
|
// materializeRowsForBird turns prefix rows into export-filter CIDR lists, optional AS_PATH rules,
|
||||||
|
// and static routes grouped by community. The same prefix can appear in multiple community groups.
|
||||||
func materializeRowsForBird(st store.Backend, tenantID string, rows []store.PrefixRow) (
|
func materializeRowsForBird(st store.Backend, tenantID string, rows []store.PrefixRow) (
|
||||||
filterV4, filterV6 []netip.Prefix,
|
filterV4, filterV6 []netip.Prefix,
|
||||||
pathASNs []int64,
|
pathASNs []int64,
|
||||||
staticV4, staticV6 []birdfmt.StaticRoute,
|
staticByCommunity []staticCommunityRoutes,
|
||||||
err error,
|
err error,
|
||||||
) {
|
) {
|
||||||
var pathSeen map[int64]struct{}
|
var pathSeen map[int64]struct{}
|
||||||
@@ -31,7 +38,12 @@ func materializeRowsForBird(st store.Backend, tenantID string, rows []store.Pref
|
|||||||
}
|
}
|
||||||
|
|
||||||
filterSeen := make(map[string]struct{})
|
filterSeen := make(map[string]struct{})
|
||||||
staticChosen := make(map[string]birdfmt.StaticRoute)
|
type staticGroup struct {
|
||||||
|
v4 map[string]birdfmt.StaticRoute
|
||||||
|
v6 map[string]birdfmt.StaticRoute
|
||||||
|
}
|
||||||
|
staticGroups := make(map[string]*staticGroup)
|
||||||
|
routeBodyByCommunity := make(map[string]string)
|
||||||
|
|
||||||
for _, pr := range rows {
|
for _, pr := range rows {
|
||||||
p := strings.TrimSpace(pr.Prefix)
|
p := strings.TrimSpace(pr.Prefix)
|
||||||
@@ -58,24 +70,67 @@ func materializeRowsForBird(st store.Backend, tenantID string, rows []store.Pref
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if _, ok := staticChosen[key]; ok {
|
communityID := ""
|
||||||
continue
|
if pr.CommunityID != nil {
|
||||||
|
communityID = strings.TrimSpace(*pr.CommunityID)
|
||||||
}
|
}
|
||||||
body, err := communityRouteBody(st, tenantID, pr.CommunityID)
|
body, ok := routeBodyByCommunity[communityID]
|
||||||
if err != nil {
|
if !ok {
|
||||||
return nil, nil, nil, nil, nil, err
|
body, err = communityRouteBody(st, tenantID, pr.CommunityID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, nil, nil, nil, err
|
||||||
|
}
|
||||||
|
routeBodyByCommunity[communityID] = body
|
||||||
|
}
|
||||||
|
grp, ok := staticGroups[communityID]
|
||||||
|
if !ok {
|
||||||
|
grp = &staticGroup{
|
||||||
|
v4: make(map[string]birdfmt.StaticRoute),
|
||||||
|
v6: make(map[string]birdfmt.StaticRoute),
|
||||||
|
}
|
||||||
|
staticGroups[communityID] = grp
|
||||||
|
}
|
||||||
|
if pfx.Addr().Is4() {
|
||||||
|
if _, exists := grp.v4[key]; !exists {
|
||||||
|
grp.v4[key] = birdfmt.StaticRoute{Prefix: pfx, RouteBody: body}
|
||||||
|
}
|
||||||
|
} else if pfx.Addr().Is6() {
|
||||||
|
if _, exists := grp.v6[key]; !exists {
|
||||||
|
grp.v6[key] = birdfmt.StaticRoute{Prefix: pfx, RouteBody: body}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
staticChosen[key] = birdfmt.StaticRoute{Prefix: pfx, RouteBody: body}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, sr := range staticChosen {
|
groupKeys := make([]string, 0, len(staticGroups))
|
||||||
if sr.Prefix.Addr().Is4() {
|
for k := range staticGroups {
|
||||||
staticV4 = append(staticV4, sr)
|
groupKeys = append(groupKeys, k)
|
||||||
} else if sr.Prefix.Addr().Is6() {
|
|
||||||
staticV6 = append(staticV6, sr)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
return filterV4, filterV6, pathASNs, staticV4, staticV6, nil
|
sort.Strings(groupKeys)
|
||||||
|
|
||||||
|
for _, gid := range groupKeys {
|
||||||
|
grp := staticGroups[gid]
|
||||||
|
out := staticCommunityRoutes{CommunityID: gid}
|
||||||
|
|
||||||
|
v4Keys := make([]string, 0, len(grp.v4))
|
||||||
|
for k := range grp.v4 {
|
||||||
|
v4Keys = append(v4Keys, k)
|
||||||
|
}
|
||||||
|
sort.Strings(v4Keys)
|
||||||
|
for _, k := range v4Keys {
|
||||||
|
out.RoutesV4 = append(out.RoutesV4, grp.v4[k])
|
||||||
|
}
|
||||||
|
|
||||||
|
v6Keys := make([]string, 0, len(grp.v6))
|
||||||
|
for k := range grp.v6 {
|
||||||
|
v6Keys = append(v6Keys, k)
|
||||||
|
}
|
||||||
|
sort.Strings(v6Keys)
|
||||||
|
for _, k := range v6Keys {
|
||||||
|
out.RoutesV6 = append(out.RoutesV6, grp.v6[k])
|
||||||
|
}
|
||||||
|
staticByCommunity = append(staticByCommunity, out)
|
||||||
|
}
|
||||||
|
return filterV4, filterV6, pathASNs, staticByCommunity, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
func communityRouteBody(st store.Backend, tenantID string, cid *string) (string, error) {
|
func communityRouteBody(st store.Backend, tenantID string, cid *string) (string, error) {
|
||||||
@@ -90,5 +145,5 @@ func communityRouteBody(st store.Backend, tenantID string, cid *string) (string,
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return "", fmt.Errorf("community %s: %w", id, err)
|
return "", fmt.Errorf("community %s: %w", id, err)
|
||||||
}
|
}
|
||||||
return birdfmt.RouteCommunityAttrs(c.Kind, c.Name, c.ValueJSON)
|
return birdfmt.RouteCommunityAttrs("", c.Community, c.ValueJSON)
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,192 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestCollectModulePrefixRows_CDNSendsIfNoneMatch(t *testing.T) {
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, _, _, _ := m.DemoIDs()
|
||||||
|
|
||||||
|
mod, err := m.CreateModule(tenant, &store.Module{
|
||||||
|
Type: "CDN_CIDRS",
|
||||||
|
Name: "cdn-conditional",
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var gotIfNoneMatch string
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
gotIfNoneMatch = strings.TrimSpace(r.Header.Get("If-None-Match"))
|
||||||
|
w.Header().Set("ETag", "etag-new")
|
||||||
|
_, _ = w.Write([]byte("198.51.100.0/24\n"))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
etag := "etag-old"
|
||||||
|
if _, err := m.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
|
||||||
|
SourceKind: "txt",
|
||||||
|
URL: srv.URL,
|
||||||
|
Etag: etag,
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
collected, err := collectModulePrefixRows(context.Background(), m, srv.Client(), tenant, mod, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if gotIfNoneMatch != etag {
|
||||||
|
t.Fatalf("want If-None-Match %q, got %q", etag, gotIfNoneMatch)
|
||||||
|
}
|
||||||
|
if len(collected) != 1 || collected[0].Prefix != "198.51.100.0/24" {
|
||||||
|
t.Fatalf("unexpected collected rows: %+v", collected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCollectModulePrefixRows_CDN304UsesSnapshot(t *testing.T) {
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, _, _, _ := m.DemoIDs()
|
||||||
|
|
||||||
|
mod, err := m.CreateModule(tenant, &store.Module{
|
||||||
|
Type: "CDN_CIDRS",
|
||||||
|
Name: "cdn-304",
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusNotModified)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
etag := "etag-stable"
|
||||||
|
src, err := m.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
|
||||||
|
SourceKind: "txt",
|
||||||
|
URL: srv.URL,
|
||||||
|
Etag: etag,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
prior := []store.PrefixRow{{
|
||||||
|
Prefix: "203.0.113.0/24",
|
||||||
|
Source: cdnSourceKey(src.ID),
|
||||||
|
}}
|
||||||
|
if err := mergeCDNSourceIntoModuleSnapshot(m, tenant, mod, src.ID, prior); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
collected, err := collectModulePrefixRows(context.Background(), m, srv.Client(), tenant, mod, prior)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(collected) != 1 || collected[0].Prefix != "203.0.113.0/24" {
|
||||||
|
t.Fatalf("want cached prefix on 304, got %+v", collected)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRefreshModuleIngest_CDN304UsesStoredSnapshot(t *testing.T) {
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, _, _, _ := m.DemoIDs()
|
||||||
|
|
||||||
|
mod, err := m.CreateModule(tenant, &store.Module{
|
||||||
|
Type: "CDN_CIDRS",
|
||||||
|
Name: "cdn-304-ingest",
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusNotModified)
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
etag := "etag-stable"
|
||||||
|
src, err := m.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
|
||||||
|
SourceKind: "txt",
|
||||||
|
URL: srv.URL,
|
||||||
|
Etag: etag,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
prior := []store.PrefixRow{{
|
||||||
|
Prefix: "203.0.113.0/24",
|
||||||
|
Source: cdnSourceKey(src.ID),
|
||||||
|
}}
|
||||||
|
if err := mergeCDNSourceIntoModuleSnapshot(m, tenant, mod, src.ID, prior); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err := RefreshModuleIngest(context.Background(), m, srv.Client(), tenant, mod.ID); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCollectModulePrefixRows_CDN304RetriesWithoutETag(t *testing.T) {
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, _, _, _ := m.DemoIDs()
|
||||||
|
|
||||||
|
mod, err := m.CreateModule(tenant, &store.Module{
|
||||||
|
Type: "CDN_CIDRS",
|
||||||
|
Name: "cdn-304-retry",
|
||||||
|
Enabled: true,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
var calls int
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
calls++
|
||||||
|
if calls == 1 {
|
||||||
|
if got := strings.TrimSpace(r.Header.Get("If-None-Match")); got != "etag-stable" {
|
||||||
|
t.Fatalf("first request want If-None-Match etag-stable, got %q", got)
|
||||||
|
}
|
||||||
|
w.WriteHeader(http.StatusNotModified)
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if got := strings.TrimSpace(r.Header.Get("If-None-Match")); got != "" {
|
||||||
|
t.Fatalf("retry must omit If-None-Match, got %q", got)
|
||||||
|
}
|
||||||
|
w.Header().Set("ETag", "etag-stable")
|
||||||
|
_, _ = w.Write([]byte("198.51.100.0/24\n"))
|
||||||
|
}))
|
||||||
|
defer srv.Close()
|
||||||
|
|
||||||
|
if _, err := m.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
|
||||||
|
SourceKind: "txt",
|
||||||
|
URL: srv.URL,
|
||||||
|
Etag: "etag-stable",
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
collected, err := collectModulePrefixRows(context.Background(), m, srv.Client(), tenant, mod, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if calls != 2 {
|
||||||
|
t.Fatalf("want 2 HTTP calls (304 then 200), got %d", calls)
|
||||||
|
}
|
||||||
|
if len(collected) != 1 || collected[0].Prefix != "198.51.100.0/24" {
|
||||||
|
t.Fatalf("unexpected collected rows: %+v", collected)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,158 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func cdnSourceKey(sourceID string) string {
|
||||||
|
return "cdn:" + strings.TrimSpace(sourceID)
|
||||||
|
}
|
||||||
|
|
||||||
|
func cachedCDNPrefixRows(st store.Backend, tenantID, moduleID string, priorSnapshot []store.PrefixRow, sourceKey string) []store.PrefixRow {
|
||||||
|
if cached := prefixRowsForSource(priorSnapshot, sourceKey); len(cached) > 0 {
|
||||||
|
return cached
|
||||||
|
}
|
||||||
|
if st != nil {
|
||||||
|
if snap, ok, _ := st.GetModulePrefixSnapshot(tenantID, moduleID); ok && snap != nil {
|
||||||
|
if cached := prefixRowsForSource(snap.Prefixes, sourceKey); len(cached) > 0 {
|
||||||
|
return cached
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if cached := latestCDNRowsBySource(st, tenantID)[sourceKey]; len(cached) > 0 {
|
||||||
|
return cached
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func mergeSnapshotDropSource(rows []store.PrefixRow, sourceKey string) []store.PrefixRow {
|
||||||
|
if len(rows) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
out := make([]store.PrefixRow, 0, len(rows))
|
||||||
|
for _, row := range rows {
|
||||||
|
if row.Source != sourceKey {
|
||||||
|
out = append(out, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func cdnRowsFromParsed(mod *store.Module, src *store.CDNSource, pfxStrings []string) []store.PrefixRow {
|
||||||
|
var rows []store.PrefixRow
|
||||||
|
for _, p := range pfxStrings {
|
||||||
|
p = strings.TrimSpace(p)
|
||||||
|
if p == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
comm := src.CommunityID
|
||||||
|
if comm == nil && mod != nil && mod.DefaultCommunityID != nil {
|
||||||
|
c := *mod.DefaultCommunityID
|
||||||
|
comm = &c
|
||||||
|
}
|
||||||
|
rows = append(rows, store.PrefixRow{Prefix: p, CommunityID: comm, Source: cdnSourceKey(src.ID)})
|
||||||
|
}
|
||||||
|
return rows
|
||||||
|
}
|
||||||
|
|
||||||
|
// mergeCDNSourceIntoModuleSnapshot replaces rows for one CDN source in the module snapshot.
|
||||||
|
func mergeCDNSourceIntoModuleSnapshot(st store.Backend, tenantID string, mod *store.Module, sourceID string, newRows []store.PrefixRow) error {
|
||||||
|
if st == nil || mod == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
sourceKey := cdnSourceKey(sourceID)
|
||||||
|
var base []store.PrefixRow
|
||||||
|
if snap, ok, _ := st.GetModulePrefixSnapshot(tenantID, mod.ID); ok && snap != nil {
|
||||||
|
base = mergeSnapshotDropSource(snap.Prefixes, sourceKey)
|
||||||
|
}
|
||||||
|
merged := append(base, newRows...)
|
||||||
|
return persistModuleSnapshot(st, tenantID, mod, merged)
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseCDNBody(body string, src *store.CDNSource) ([]string, error) {
|
||||||
|
pfxs, err := ExtractCIDRs(body, src.SourceKind, src.PrefixPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out := make([]string, 0, len(pfxs))
|
||||||
|
for _, pfx := range pfxs {
|
||||||
|
out = append(out, pfx.String())
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func applyCDNSourceHTTPResult(ctx context.Context, st store.Backend, hc *http.Client, tenantID, moduleID string, mod *store.Module, src *store.CDNSource, priorSnapshot []store.PrefixRow, now time.Time) ([]store.PrefixRow, error) {
|
||||||
|
u := strings.TrimSpace(src.URL)
|
||||||
|
if u == "" {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
sourceKey := cdnSourceKey(src.ID)
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
if etag := strings.TrimSpace(src.Etag); etag != "" {
|
||||||
|
req.Header.Set("If-None-Match", etag)
|
||||||
|
}
|
||||||
|
resp, err := hc.Do(req)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
if resp.StatusCode == http.StatusNotModified {
|
||||||
|
if cached := cachedCDNPrefixRows(st, tenantID, moduleID, priorSnapshot, sourceKey); len(cached) > 0 {
|
||||||
|
_ = resp.Body.Close()
|
||||||
|
return cached, nil
|
||||||
|
}
|
||||||
|
// ETag is known but local snapshot is empty — force a full download.
|
||||||
|
_ = resp.Body.Close()
|
||||||
|
req2, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
resp, err = hc.Do(req2)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
|
||||||
|
}
|
||||||
|
if resp.StatusCode == http.StatusNotModified {
|
||||||
|
_ = resp.Body.Close()
|
||||||
|
return nil, fmt.Errorf("cdn url %s: 304 without cached prefixes", u)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
defer resp.Body.Close()
|
||||||
|
|
||||||
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
_, _ = io.Copy(io.Discard, resp.Body)
|
||||||
|
return nil, fmt.Errorf("cdn url %s: %s", u, resp.Status)
|
||||||
|
}
|
||||||
|
body, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
prefixStrs, err := parseCDNBody(string(body), src)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("cdn parse %s: %w", u, err)
|
||||||
|
}
|
||||||
|
etag := strings.TrimSpace(resp.Header.Get("ETag"))
|
||||||
|
patch := &store.CDNSourcePatch{}
|
||||||
|
if etag != "" && etag != strings.TrimSpace(src.Etag) {
|
||||||
|
e := etag
|
||||||
|
patch.Etag = &e
|
||||||
|
}
|
||||||
|
refreshedAt := now
|
||||||
|
patch.LastRefreshedAt = &refreshedAt
|
||||||
|
_, _ = st.UpdateCDNSource(tenantID, moduleID, src.ID, patch)
|
||||||
|
|
||||||
|
rows := cdnRowsFromParsed(mod, src, prefixStrs)
|
||||||
|
if err := mergeCDNSourceIntoModuleSnapshot(st, tenantID, mod, src.ID, rows); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
return rows, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,244 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"os"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func prefixRowsForSource(rows []store.PrefixRow, sourceKey string) []store.PrefixRow {
|
||||||
|
if len(rows) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
var out []store.PrefixRow
|
||||||
|
for _, row := range rows {
|
||||||
|
if row.Source == sourceKey {
|
||||||
|
out = append(out, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
func collectASPrefixRows(ctx context.Context, st store.Backend, hc *http.Client, tenantID string, mod *store.Module, list []*store.ASEntry) ([]store.PrefixRow, error) {
|
||||||
|
moduleID := mod.ID
|
||||||
|
legacy := strings.TrimSpace(os.Getenv("EVOBGP_ASN_RESOLVE")) == "0"
|
||||||
|
if legacy {
|
||||||
|
var rows []store.PrefixRow
|
||||||
|
for _, e := range list {
|
||||||
|
if !store.ValidASN(e.ASN) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
comm := e.CommunityID
|
||||||
|
if comm == nil && mod.DefaultCommunityID != nil {
|
||||||
|
c := *mod.DefaultCommunityID
|
||||||
|
comm = &c
|
||||||
|
}
|
||||||
|
rows = append(rows, store.PrefixRow{Prefix: MaterializedASPrefixKey(e.ASN), CommunityID: comm, Source: "as_entry"})
|
||||||
|
}
|
||||||
|
return rows, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type entryResult struct {
|
||||||
|
rows []store.PrefixRow
|
||||||
|
metaID string
|
||||||
|
asn int64
|
||||||
|
holder string
|
||||||
|
count int64
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
|
||||||
|
var valid []*store.ASEntry
|
||||||
|
for _, e := range list {
|
||||||
|
if e != nil && store.ValidASN(e.ASN) {
|
||||||
|
valid = append(valid, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
sem := make(chan struct{}, collectConcurrency())
|
||||||
|
results := make([]entryResult, len(valid))
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i, e := range valid {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(idx int, entry *store.ASEntry) {
|
||||||
|
defer wg.Done()
|
||||||
|
sem <- struct{}{}
|
||||||
|
defer func() { <-sem }()
|
||||||
|
|
||||||
|
comm := entry.CommunityID
|
||||||
|
if comm == nil && mod.DefaultCommunityID != nil {
|
||||||
|
c := *mod.DefaultCommunityID
|
||||||
|
comm = &c
|
||||||
|
}
|
||||||
|
pfxs, holder, err := resolveASNForEntry(ctx, st, hc, entry.ASN)
|
||||||
|
if err != nil {
|
||||||
|
results[idx] = entryResult{err: fmt.Errorf("resolve AS%d: %w", entry.ASN, err)}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
src := fmt.Sprintf("as:%d", entry.ASN)
|
||||||
|
var rows []store.PrefixRow
|
||||||
|
for _, pfx := range pfxs {
|
||||||
|
rows = append(rows, store.PrefixRow{Prefix: pfx.String(), CommunityID: comm, Source: src})
|
||||||
|
}
|
||||||
|
results[idx] = entryResult{
|
||||||
|
rows: rows,
|
||||||
|
metaID: entry.ID,
|
||||||
|
asn: entry.ASN,
|
||||||
|
holder: holder,
|
||||||
|
count: int64(len(pfxs)),
|
||||||
|
}
|
||||||
|
}(i, e)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
|
||||||
|
seenPfx := make(map[string]struct{})
|
||||||
|
var out []store.PrefixRow
|
||||||
|
now := time.Now().UTC()
|
||||||
|
for _, r := range results {
|
||||||
|
if r.err != nil {
|
||||||
|
return nil, r.err
|
||||||
|
}
|
||||||
|
if r.metaID != "" {
|
||||||
|
if err := st.UpdateASEntryResolveMeta(tenantID, moduleID, r.metaID, r.holder, r.count, now); err != nil {
|
||||||
|
return nil, fmt.Errorf("as entry meta AS%d: %w", r.asn, err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, row := range r.rows {
|
||||||
|
k := row.Prefix
|
||||||
|
if _, ok := seenPfx[k]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seenPfx[k] = struct{}{}
|
||||||
|
out = append(out, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func collectCDNPrefixRows(ctx context.Context, st store.Backend, hc *http.Client, tenantID string, mod *store.Module, sources []*store.CDNSource, priorSnapshot []store.PrefixRow) ([]store.PrefixRow, error) {
|
||||||
|
moduleID := mod.ID
|
||||||
|
now := time.Now().UTC()
|
||||||
|
|
||||||
|
var valid []*store.CDNSource
|
||||||
|
for _, s := range sources {
|
||||||
|
if s != nil {
|
||||||
|
valid = append(valid, s)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
type srcResult struct {
|
||||||
|
rows []store.PrefixRow
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
results := make([]srcResult, len(valid))
|
||||||
|
sem := make(chan struct{}, collectConcurrency())
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
|
||||||
|
for i, src := range valid {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(idx int, src *store.CDNSource) {
|
||||||
|
defer wg.Done()
|
||||||
|
sem <- struct{}{}
|
||||||
|
defer func() { <-sem }()
|
||||||
|
|
||||||
|
sourceKey := "cdn:" + src.ID
|
||||||
|
if shouldSkipCDNSourceFetch(src, now) {
|
||||||
|
if cached := prefixRowsForSource(priorSnapshot, sourceKey); len(cached) > 0 {
|
||||||
|
results[idx] = srcResult{rows: cached}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if cached := latestCDNRowsBySource(st, tenantID)[sourceKey]; len(cached) > 0 {
|
||||||
|
results[idx] = srcResult{rows: cached}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rows, err := applyCDNSourceHTTPResult(ctx, st, hc, tenantID, moduleID, mod, src, priorSnapshot, now)
|
||||||
|
if err != nil {
|
||||||
|
results[idx] = srcResult{err: err}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
results[idx] = srcResult{rows: rows}
|
||||||
|
}(i, src)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
|
||||||
|
var out []store.PrefixRow
|
||||||
|
for _, r := range results {
|
||||||
|
if r.err != nil {
|
||||||
|
return nil, r.err
|
||||||
|
}
|
||||||
|
out = append(out, r.rows...)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func collectDomainPrefixRows(ctx context.Context, hc *http.Client, mod *store.Module, profiles []*store.DohProfile, policy string, entries []*store.DomainEntry) ([]store.PrefixRow, error) {
|
||||||
|
var validDom []*store.DomainEntry
|
||||||
|
for _, e := range entries {
|
||||||
|
if e != nil {
|
||||||
|
validDom = append(validDom, e)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
type domResult struct {
|
||||||
|
rows []store.PrefixRow
|
||||||
|
err error
|
||||||
|
}
|
||||||
|
results := make([]domResult, len(validDom))
|
||||||
|
sem := make(chan struct{}, collectConcurrency())
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
|
||||||
|
for i, e := range validDom {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(idx int, entry *store.DomainEntry) {
|
||||||
|
defer wg.Done()
|
||||||
|
sem <- struct{}{}
|
||||||
|
defer func() { <-sem }()
|
||||||
|
|
||||||
|
comm := entry.CommunityID
|
||||||
|
if comm == nil && mod.DefaultCommunityID != nil {
|
||||||
|
c := *mod.DefaultCommunityID
|
||||||
|
comm = &c
|
||||||
|
}
|
||||||
|
addrs, err := resolveDomainIPsWithPolicy(ctx, hc, profiles, policy, entry.FQDN)
|
||||||
|
if err != nil {
|
||||||
|
results[idx] = domResult{err: fmt.Errorf("resolve domain %q: %w", entry.FQDN, err)}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
src := "domain:" + strings.TrimSpace(entry.FQDN)
|
||||||
|
var rows []store.PrefixRow
|
||||||
|
for _, ip := range addrs {
|
||||||
|
cidr := ipToHostPrefix(ip)
|
||||||
|
if cidr == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
rows = append(rows, store.PrefixRow{
|
||||||
|
Prefix: cidr,
|
||||||
|
CommunityID: comm,
|
||||||
|
Source: src,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
results[idx] = domResult{rows: rows}
|
||||||
|
}(i, e)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
|
||||||
|
seen := make(map[string]struct{})
|
||||||
|
var out []store.PrefixRow
|
||||||
|
for _, r := range results {
|
||||||
|
if r.err != nil {
|
||||||
|
return nil, r.err
|
||||||
|
}
|
||||||
|
for _, row := range r.rows {
|
||||||
|
key := row.Prefix + "|" + row.Source
|
||||||
|
if _, ok := seen[key]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[key] = struct{}{}
|
||||||
|
out = append(out, row)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,3 @@
|
|||||||
|
// Package pipeline ingests module inputs (CDN, ASN, domains, IP ranges), materializes
|
||||||
|
// prefixes, and creates BIRD preview revisions via store.
|
||||||
|
package pipeline
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
|
||||||
|
"github.com/miekg/dns"
|
||||||
|
)
|
||||||
|
|
||||||
|
func loadModuleDohProfiles(st store.Backend, tenantID string, mod *store.Module) ([]*store.DohProfile, string, error) {
|
||||||
|
if mod == nil {
|
||||||
|
return nil, store.DohPolicyPrimaryOnly, nil
|
||||||
|
}
|
||||||
|
policy := store.NormalizeDohResolverPolicy(mod.DohResolverPolicy)
|
||||||
|
var profiles []*store.DohProfile
|
||||||
|
for _, id := range mod.EffectiveDohProfileIDs() {
|
||||||
|
prof, err := st.GetDohProfile(tenantID, id)
|
||||||
|
if err != nil {
|
||||||
|
return nil, "", fmt.Errorf("get doh profile %s: %w", id, err)
|
||||||
|
}
|
||||||
|
profiles = append(profiles, prof)
|
||||||
|
}
|
||||||
|
return profiles, policy, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveDomainIPsWithPolicy(ctx context.Context, hc *http.Client, profiles []*store.DohProfile, policy, fqdn string) ([]netip.Addr, error) {
|
||||||
|
policy = store.NormalizeDohResolverPolicy(policy)
|
||||||
|
if len(profiles) == 0 {
|
||||||
|
return resolveDomainIPs(ctx, hc, nil, fqdn)
|
||||||
|
}
|
||||||
|
if len(profiles) == 1 {
|
||||||
|
return resolveDomainIPs(ctx, hc, profiles[0], fqdn)
|
||||||
|
}
|
||||||
|
|
||||||
|
switch policy {
|
||||||
|
case store.DohPolicyUnion:
|
||||||
|
return resolveDomainIPsUnion(ctx, hc, profiles, fqdn)
|
||||||
|
case store.DohPolicyFailover:
|
||||||
|
return resolveDomainIPsFailover(ctx, hc, profiles, fqdn)
|
||||||
|
default:
|
||||||
|
return resolveDomainIPs(ctx, hc, profiles[0], fqdn)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveDomainIPsUnion(ctx context.Context, hc *http.Client, profiles []*store.DohProfile, fqdn string) ([]netip.Addr, error) {
|
||||||
|
var merged []netip.Addr
|
||||||
|
var errs []error
|
||||||
|
for _, prof := range profiles {
|
||||||
|
if prof == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ips, err := resolveDomainIPsNoSystemFallback(ctx, hc, prof, fqdn)
|
||||||
|
if err != nil {
|
||||||
|
errs = append(errs, fmt.Errorf("%s: %w", strings.TrimSpace(prof.URL), err))
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
merged = append(merged, ips...)
|
||||||
|
}
|
||||||
|
merged = uniqAddrs(merged)
|
||||||
|
if len(merged) > 0 {
|
||||||
|
return merged, nil
|
||||||
|
}
|
||||||
|
if len(errs) > 0 {
|
||||||
|
return nil, fmt.Errorf("doh union failed: %v", errs)
|
||||||
|
}
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func resolveDomainIPsFailover(ctx context.Context, hc *http.Client, profiles []*store.DohProfile, fqdn string) ([]netip.Addr, error) {
|
||||||
|
var lastErr error
|
||||||
|
for _, prof := range profiles {
|
||||||
|
if prof == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
ips, err := resolveDomainIPsNoSystemFallback(ctx, hc, prof, fqdn)
|
||||||
|
if err != nil {
|
||||||
|
lastErr = err
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if len(ips) > 0 {
|
||||||
|
return ips, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if lastErr != nil {
|
||||||
|
return nil, lastErr
|
||||||
|
}
|
||||||
|
return resolveDomainIPs(ctx, hc, nil, fqdn)
|
||||||
|
}
|
||||||
|
|
||||||
|
// resolveDomainIPsNoSystemFallback queries one DoH profile without falling back to OS resolver.
|
||||||
|
func resolveDomainIPsNoSystemFallback(ctx context.Context, hc *http.Client, profile *store.DohProfile, fqdn string) ([]netip.Addr, error) {
|
||||||
|
host := strings.TrimSpace(strings.TrimSuffix(fqdn, "."))
|
||||||
|
if host == "" {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
if profile == nil || strings.TrimSpace(profile.URL) == "" {
|
||||||
|
return nil, fmt.Errorf("empty doh profile")
|
||||||
|
}
|
||||||
|
|
||||||
|
timeout := dohProfileTimeout(profile)
|
||||||
|
dctx, cancel := context.WithTimeout(ctx, timeout)
|
||||||
|
defer cancel()
|
||||||
|
|
||||||
|
baseURL := strings.TrimSpace(profile.URL)
|
||||||
|
v4, err4 := resolveDomainWithDOHMessage(dctx, hc, baseURL, host, dns.TypeA)
|
||||||
|
v6, err6 := resolveDomainWithDOHMessage(dctx, hc, baseURL, host, dns.TypeAAAA)
|
||||||
|
if err4 != nil {
|
||||||
|
v4, err4 = resolveDomainWithDOHJSON(dctx, hc, baseURL, host, "A")
|
||||||
|
}
|
||||||
|
if err6 != nil {
|
||||||
|
v6, err6 = resolveDomainWithDOHJSON(dctx, hc, baseURL, host, "AAAA")
|
||||||
|
}
|
||||||
|
if err4 != nil && err6 != nil {
|
||||||
|
return nil, fmt.Errorf("doh failed for A and AAAA: %v; %v", err4, err6)
|
||||||
|
}
|
||||||
|
return uniqAddrs(append(v4, v6...)), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func dohProfileTimeout(profile *store.DohProfile) time.Duration {
|
||||||
|
timeout := 10 * time.Second
|
||||||
|
if profile != nil && profile.TimeoutMs != nil && *profile.TimeoutMs > 0 {
|
||||||
|
timeout = time.Duration(*profile.TimeoutMs) * time.Millisecond
|
||||||
|
}
|
||||||
|
return timeout
|
||||||
|
}
|
||||||
@@ -0,0 +1,148 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestResolveDomainIPsWithPolicy_Union(t *testing.T) {
|
||||||
|
srvRU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.1"}]}`))
|
||||||
|
}))
|
||||||
|
defer srvRU.Close()
|
||||||
|
srvEU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"203.0.113.1"}]}`))
|
||||||
|
}))
|
||||||
|
defer srvEU.Close()
|
||||||
|
|
||||||
|
profiles := []*store.DohProfile{
|
||||||
|
{URL: srvRU.URL},
|
||||||
|
{URL: srvEU.URL},
|
||||||
|
}
|
||||||
|
ips, err := resolveDomainIPsWithPolicy(context.Background(), srvRU.Client(), profiles, store.DohPolicyUnion, "example.com")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(ips) != 2 {
|
||||||
|
t.Fatalf("want 2 ips, got %v", ips)
|
||||||
|
}
|
||||||
|
seen := map[string]bool{ips[0].String(): true, ips[1].String(): true}
|
||||||
|
if !seen["198.51.100.1"] || !seen["203.0.113.1"] {
|
||||||
|
t.Fatalf("unexpected ips: %v", ips)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveDomainIPsWithPolicy_Failover(t *testing.T) {
|
||||||
|
var calls int
|
||||||
|
srvBad := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
calls++
|
||||||
|
http.Error(w, "fail", http.StatusBadGateway)
|
||||||
|
}))
|
||||||
|
defer srvBad.Close()
|
||||||
|
srvOK := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
calls++
|
||||||
|
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.5"}]}`))
|
||||||
|
}))
|
||||||
|
defer srvOK.Close()
|
||||||
|
|
||||||
|
profiles := []*store.DohProfile{
|
||||||
|
{URL: srvBad.URL},
|
||||||
|
{URL: srvOK.URL},
|
||||||
|
}
|
||||||
|
ips, err := resolveDomainIPsWithPolicy(context.Background(), srvBad.Client(), profiles, store.DohPolicyFailover, "example.com")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(ips) != 1 || ips[0].String() != "198.51.100.5" {
|
||||||
|
t.Fatalf("unexpected ips: %v", ips)
|
||||||
|
}
|
||||||
|
if calls < 2 {
|
||||||
|
t.Fatalf("want at least 2 resolver calls, got %d", calls)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestResolveDomainIPsWithPolicy_PrimaryOnly(t *testing.T) {
|
||||||
|
var secondCalled bool
|
||||||
|
srv1 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.9"}]}`))
|
||||||
|
}))
|
||||||
|
defer srv1.Close()
|
||||||
|
srv2 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
secondCalled = true
|
||||||
|
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"203.0.113.9"}]}`))
|
||||||
|
}))
|
||||||
|
defer srv2.Close()
|
||||||
|
|
||||||
|
profiles := []*store.DohProfile{
|
||||||
|
{URL: srv1.URL},
|
||||||
|
{URL: srv2.URL},
|
||||||
|
}
|
||||||
|
ips, err := resolveDomainIPsWithPolicy(context.Background(), srv1.Client(), profiles, store.DohPolicyPrimaryOnly, "example.com")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(ips) != 1 || ips[0].String() != "198.51.100.9" {
|
||||||
|
t.Fatalf("unexpected ips: %v", ips)
|
||||||
|
}
|
||||||
|
if secondCalled {
|
||||||
|
t.Fatal("secondary resolver must not be queried in primary_only mode")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCollectModulePrefixRows_DohUnion(t *testing.T) {
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, _, _, _ := m.DemoIDs()
|
||||||
|
|
||||||
|
mod, err := m.CreateModule(tenant, &store.Module{
|
||||||
|
Type: "DOMAINS",
|
||||||
|
Name: "domains-union",
|
||||||
|
Enabled: true,
|
||||||
|
DohResolverPolicy: store.DohPolicyUnion,
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
srvRU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.2"}]}`))
|
||||||
|
}))
|
||||||
|
defer srvRU.Close()
|
||||||
|
srvEU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"203.0.113.2"}]}`))
|
||||||
|
}))
|
||||||
|
defer srvEU.Close()
|
||||||
|
|
||||||
|
ru, err := m.CreateDohProfile(tenant, &store.DohProfile{Name: "ru", URL: srvRU.URL})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
eu, err := m.CreateDohProfile(tenant, &store.DohProfile{Name: "eu", URL: srvEU.URL})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := m.UpdateModule(tenant, mod.ID, &store.ModulePatch{
|
||||||
|
DohProfileIDs: &[]string{ru.ID, eu.ID},
|
||||||
|
}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
mod, err = m.GetModule(tenant, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if _, err := m.CreateDomainEntry(tenant, mod.ID, &store.DomainEntry{FQDN: "svc.example.com"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rows, err := collectModulePrefixRows(context.Background(), m, srvRU.Client(), tenant, mod, nil)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(rows) != 2 {
|
||||||
|
t.Fatalf("want 2 prefix rows, got %+v", rows)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,49 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"evobgp/internal/birdfmt"
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestBuildPreviewFragments_SamePrefixDifferentCommunity(t *testing.T) {
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, moduleIP, _, _ := m.DemoIDs()
|
||||||
|
|
||||||
|
c1, err := m.CreateCommunity(tenant, &store.Community{Community: "65000:1", Title: "A", ValueJSON: "{}"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
c2, err := m.CreateCommunity(tenant, &store.Community{Community: "65000:2", Title: "B", ValueJSON: "{}"})
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "203.0.113.0/24", CommunityID: &c1.ID, Source: "ip_range"},
|
||||||
|
{Prefix: "203.0.113.0/24", CommunityID: &c2.ID, Source: "cdn:test"},
|
||||||
|
}
|
||||||
|
out, err := buildPreviewFragments(m, tenant, moduleIP, "test-revision", rows)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
staticV4 := out[birdfmt.FragmentIncludePath(birdfmt.FragmentPrefixesV4)]
|
||||||
|
if !strings.Contains(staticV4, "protocol static evobgp_prefixes_v4_"+communityProtocolSuffix(c1.ID)) {
|
||||||
|
t.Fatalf("expected dedicated static protocol for first community, got:\n%s", staticV4)
|
||||||
|
}
|
||||||
|
if !strings.Contains(staticV4, "protocol static evobgp_prefixes_v4_"+communityProtocolSuffix(c2.ID)) {
|
||||||
|
t.Fatalf("expected dedicated static protocol for second community, got:\n%s", staticV4)
|
||||||
|
}
|
||||||
|
out2, err := buildPreviewFragments(m, tenant, moduleIP, "test-revision", rows)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
staticV4Second := out2[birdfmt.FragmentIncludePath(birdfmt.FragmentPrefixesV4)]
|
||||||
|
if staticV4 != staticV4Second {
|
||||||
|
t.Fatalf("expected deterministic static preview text, got first:\n%s\nsecond:\n%s", staticV4, staticV4Second)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
@@ -0,0 +1,122 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"crypto/sha256"
|
||||||
|
"encoding/hex"
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strings"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// moduleIngestInputHash fingerprints module config and child entries so snapshots invalidate on CRUD.
|
||||||
|
func moduleIngestInputHash(st store.Backend, tenantID string, mod *store.Module) (string, error) {
|
||||||
|
if st == nil || mod == nil {
|
||||||
|
return "", fmt.Errorf("pipeline: module hash: missing store or module")
|
||||||
|
}
|
||||||
|
h := sha256.New()
|
||||||
|
_, _ = fmt.Fprintf(h, "type=%s\n", strings.TrimSpace(mod.Type))
|
||||||
|
_, _ = fmt.Fprintf(h, "enabled=%t\n", mod.Enabled)
|
||||||
|
if mod.DefaultCommunityID != nil {
|
||||||
|
_, _ = fmt.Fprintf(h, "default_community=%s\n", strings.TrimSpace(*mod.DefaultCommunityID))
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(h, "doh_policy=%s\n", store.NormalizeDohResolverPolicy(mod.DohResolverPolicy))
|
||||||
|
for _, pid := range mod.EffectiveDohProfileIDs() {
|
||||||
|
_, _ = fmt.Fprintf(h, "doh_profile=%s\n", pid)
|
||||||
|
if prof, err := st.GetDohProfile(tenantID, pid); err == nil && prof != nil {
|
||||||
|
_, _ = fmt.Fprintf(h, "doh_url=%s\n", strings.TrimSpace(prof.URL))
|
||||||
|
if prof.TimeoutMs != nil {
|
||||||
|
_, _ = fmt.Fprintf(h, "doh_timeout=%d\n", *prof.TimeoutMs)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
switch mod.Type {
|
||||||
|
case "IP_RANGES":
|
||||||
|
list, err := st.ListIPRangeEntries(tenantID, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
sort.Slice(list, func(i, j int) bool { return list[i].Prefix < list[j].Prefix })
|
||||||
|
for _, e := range list {
|
||||||
|
comm := ""
|
||||||
|
if e.CommunityID != nil {
|
||||||
|
comm = *e.CommunityID
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(h, "ip=%s|c=%s\n", e.Prefix, comm)
|
||||||
|
}
|
||||||
|
case "AS_PREFIXES":
|
||||||
|
list, err := st.ListASEntries(tenantID, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
sort.Slice(list, func(i, j int) bool { return list[i].ASN < list[j].ASN })
|
||||||
|
for _, e := range list {
|
||||||
|
comm := ""
|
||||||
|
if e.CommunityID != nil {
|
||||||
|
comm = *e.CommunityID
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(h, "as=%d|c=%s\n", e.ASN, comm)
|
||||||
|
}
|
||||||
|
case "CDN_CIDRS":
|
||||||
|
list, err := st.ListCDNSources(tenantID, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
sort.Slice(list, func(i, j int) bool { return list[i].ID < list[j].ID })
|
||||||
|
for _, s := range list {
|
||||||
|
comm := ""
|
||||||
|
if s.CommunityID != nil {
|
||||||
|
comm = *s.CommunityID
|
||||||
|
}
|
||||||
|
interval := 0
|
||||||
|
if s.RefreshIntervalSec != nil {
|
||||||
|
interval = *s.RefreshIntervalSec
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(h, "cdn=%s|url=%s|kind=%s|path=%s|c=%s|etag=%s|interval=%d\n",
|
||||||
|
s.ID, strings.TrimSpace(s.URL), s.SourceKind, strings.TrimSpace(s.PrefixPath), comm,
|
||||||
|
strings.TrimSpace(s.Etag), interval)
|
||||||
|
}
|
||||||
|
case "DOMAINS":
|
||||||
|
list, err := st.ListDomainEntries(tenantID, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
sort.Slice(list, func(i, j int) bool { return list[i].FQDN < list[j].FQDN })
|
||||||
|
for _, e := range list {
|
||||||
|
comm := ""
|
||||||
|
if e.CommunityID != nil {
|
||||||
|
comm = *e.CommunityID
|
||||||
|
}
|
||||||
|
_, _ = fmt.Fprintf(h, "dom=%s|c=%s\n", strings.TrimSpace(e.FQDN), comm)
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
_, _ = fmt.Fprintf(h, "unknown_type=%s\n", mod.Type)
|
||||||
|
}
|
||||||
|
return hex.EncodeToString(h.Sum(nil)), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func persistModuleSnapshot(st store.Backend, tenantID string, mod *store.Module, rows []store.PrefixRow) error {
|
||||||
|
if st == nil || mod == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
hash, err := moduleIngestInputHash(st, tenantID, mod)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
return st.SetModulePrefixSnapshot(tenantID, mod.ID, hash, rows)
|
||||||
|
}
|
||||||
|
|
||||||
|
func moduleRowsFromSnapshot(st store.Backend, tenantID string, mod *store.Module) ([]store.PrefixRow, bool, error) {
|
||||||
|
hash, err := moduleIngestInputHash(st, tenantID, mod)
|
||||||
|
if err != nil {
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
snap, ok, err := st.GetModulePrefixSnapshot(tenantID, mod.ID)
|
||||||
|
if err != nil || !ok || snap == nil || snap.InputHash != hash {
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
cp := append([]store.PrefixRow(nil), snap.Prefixes...)
|
||||||
|
return cp, true, nil
|
||||||
|
}
|
||||||
@@ -2,10 +2,18 @@ package pipeline
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
|
"encoding/json"
|
||||||
|
"fmt"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"strings"
|
"strings"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// Защита от pathological JSON: глубокая рекурсия при обходе и взрыв числа узлов по пути.
|
||||||
|
const (
|
||||||
|
maxJSONWalkDepth = 512
|
||||||
|
maxJSONPathBreadth = 50000
|
||||||
|
)
|
||||||
|
|
||||||
// ParseCIDRLines extracts unique IPv4/IPv6 CIDRs from plain text (one per line, # comments, empty lines skipped).
|
// ParseCIDRLines extracts unique IPv4/IPv6 CIDRs from plain text (one per line, # comments, empty lines skipped).
|
||||||
func ParseCIDRLines(body string) []netip.Prefix {
|
func ParseCIDRLines(body string) []netip.Prefix {
|
||||||
seen := make(map[string]struct{})
|
seen := make(map[string]struct{})
|
||||||
@@ -31,6 +39,126 @@ func ParseCIDRLines(body string) []netip.Prefix {
|
|||||||
return out
|
return out
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ExtractCIDRs parses CIDRs from either plaintext lines or JSON payload.
|
||||||
|
// For sourceKind="json", prefixPath supports dotted traversal, with [] for arrays:
|
||||||
|
// e.g. "prefixes[]", "data.items[].cidr".
|
||||||
|
func ExtractCIDRs(body, sourceKind, prefixPath string) ([]netip.Prefix, error) {
|
||||||
|
if strings.EqualFold(strings.TrimSpace(sourceKind), "json") {
|
||||||
|
return parseCIDRsFromJSON(body, prefixPath)
|
||||||
|
}
|
||||||
|
return ParseCIDRLines(body), nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func parseCIDRsFromJSON(body, prefixPath string) ([]netip.Prefix, error) {
|
||||||
|
var root any
|
||||||
|
if err := json.Unmarshal([]byte(body), &root); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
values, err := jsonValuesAtPath(root, prefixPath)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
seen := make(map[string]struct{})
|
||||||
|
var out []netip.Prefix
|
||||||
|
for _, raw := range values {
|
||||||
|
pfx := parseOneCIDR(raw)
|
||||||
|
if !pfx.IsValid() {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
m := pfx.Masked()
|
||||||
|
s := m.String()
|
||||||
|
if _, ok := seen[s]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[s] = struct{}{}
|
||||||
|
out = append(out, m)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func jsonValuesAtPath(root any, prefixPath string) ([]string, error) {
|
||||||
|
path := strings.TrimSpace(prefixPath)
|
||||||
|
if path == "" {
|
||||||
|
return flattenJSONStrings(root, 0)
|
||||||
|
}
|
||||||
|
parts := strings.Split(path, ".")
|
||||||
|
nodes := []any{root}
|
||||||
|
for _, p := range parts {
|
||||||
|
part := strings.TrimSpace(p)
|
||||||
|
if part == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
iter := strings.HasSuffix(part, "[]")
|
||||||
|
key := strings.TrimSuffix(part, "[]")
|
||||||
|
var next []any
|
||||||
|
for _, n := range nodes {
|
||||||
|
obj, ok := n.(map[string]any)
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
child, ok := obj[key]
|
||||||
|
if !ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if iter {
|
||||||
|
if arr, ok := child.([]any); ok {
|
||||||
|
next = append(next, arr...)
|
||||||
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
next = append(next, child)
|
||||||
|
}
|
||||||
|
if len(next) > maxJSONPathBreadth {
|
||||||
|
return nil, fmt.Errorf("json path: слишком много узлов на шаге (>%d)", maxJSONPathBreadth)
|
||||||
|
}
|
||||||
|
nodes = next
|
||||||
|
if len(nodes) == 0 {
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var out []string
|
||||||
|
for _, n := range nodes {
|
||||||
|
part, err := flattenJSONStrings(n, 0)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, part...)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func flattenJSONStrings(v any, depth int) ([]string, error) {
|
||||||
|
if depth > maxJSONWalkDepth {
|
||||||
|
return nil, fmt.Errorf("json: глубина вложенности превышает %d", maxJSONWalkDepth)
|
||||||
|
}
|
||||||
|
switch x := v.(type) {
|
||||||
|
case string:
|
||||||
|
return []string{strings.TrimSpace(x)}, nil
|
||||||
|
case []any:
|
||||||
|
var out []string
|
||||||
|
for _, item := range x {
|
||||||
|
part, err := flattenJSONStrings(item, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, part...)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
case map[string]any:
|
||||||
|
var out []string
|
||||||
|
for _, item := range x {
|
||||||
|
part, err := flattenJSONStrings(item, depth+1)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
out = append(out, part...)
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
default:
|
||||||
|
return nil, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func parseOneCIDR(s string) netip.Prefix {
|
func parseOneCIDR(s string) netip.Prefix {
|
||||||
if p, err := netip.ParsePrefix(s); err == nil {
|
if p, err := netip.ParsePrefix(s); err == nil {
|
||||||
return p
|
return p
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"testing"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestExtractCIDRs_JSONNested(t *testing.T) {
|
||||||
|
body := `{"a":{"b":{"c":"192.0.2.0/24"}}}`
|
||||||
|
pfx, err := ExtractCIDRs(body, "json", "a.b.c")
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if len(pfx) != 1 || pfx[0].String() != "192.0.2.0/24" {
|
||||||
|
t.Fatalf("got %#v", pfx)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractCIDRs_JSONDepthLimit(t *testing.T) {
|
||||||
|
var b strings.Builder
|
||||||
|
b.WriteString(`{"x":`)
|
||||||
|
for i := 0; i < maxJSONWalkDepth+4; i++ {
|
||||||
|
b.WriteString(`{"k":`)
|
||||||
|
}
|
||||||
|
b.WriteString(`"192.0.2.1"`)
|
||||||
|
for i := 0; i < maxJSONWalkDepth+4; i++ {
|
||||||
|
b.WriteByte('}')
|
||||||
|
}
|
||||||
|
b.WriteByte('}')
|
||||||
|
_, err := ExtractCIDRs(b.String(), "json", "")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected depth error")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestExtractCIDRs_JSONPathBreadth(t *testing.T) {
|
||||||
|
var b strings.Builder
|
||||||
|
b.WriteString(`{"items":{"x":[`)
|
||||||
|
for i := 0; i < maxJSONPathBreadth+1; i++ {
|
||||||
|
if i > 0 {
|
||||||
|
b.WriteByte(',')
|
||||||
|
}
|
||||||
|
b.WriteString(`"192.0.2.0/24"`)
|
||||||
|
}
|
||||||
|
b.WriteString(`]}}`)
|
||||||
|
_, err := ExtractCIDRs(b.String(), "json", "items.x[]")
|
||||||
|
if err == nil {
|
||||||
|
t.Fatal("expected breadth error")
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -5,11 +5,12 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"strings"
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
"evobgp/internal/store"
|
"evobgp/internal/store"
|
||||||
)
|
)
|
||||||
|
|
||||||
// PrefetchCDNSourceETags performs conditional GETs for CDN module sources and updates stored ETags when the origin responds 200.
|
// PrefetchCDNSourceETags performs conditional GETs for CDN sources; on 200 parses CIDRs into module_prefix_snapshot.
|
||||||
func PrefetchCDNSourceETags(ctx context.Context, st store.Backend, hc *http.Client) error {
|
func PrefetchCDNSourceETags(ctx context.Context, st store.Backend, hc *http.Client) error {
|
||||||
if hc == nil {
|
if hc == nil {
|
||||||
hc = http.DefaultClient
|
hc = http.DefaultClient
|
||||||
@@ -18,42 +19,70 @@ func PrefetchCDNSourceETags(ctx context.Context, st store.Backend, hc *http.Clie
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
now := time.Now().UTC()
|
||||||
for _, tid := range tenants {
|
for _, tid := range tenants {
|
||||||
for _, mod := range st.ListModules(tid) {
|
for _, mod := range st.ListModules(tid) {
|
||||||
if !mod.Enabled || mod.Type != "CDN_CIDRS" {
|
if mod == nil || !mod.Enabled || mod.Type != "CDN_CIDRS" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
omod, err := st.GetModule(tid, mod.ID)
|
||||||
|
if err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
sources, err := st.ListCDNSources(tid, mod.ID)
|
sources, err := st.ListCDNSources(tid, mod.ID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
var prior []store.PrefixRow
|
||||||
|
if snap, ok, _ := st.GetModulePrefixSnapshot(tid, mod.ID); ok && snap != nil {
|
||||||
|
prior = snap.Prefixes
|
||||||
|
}
|
||||||
for _, src := range sources {
|
for _, src := range sources {
|
||||||
u := strings.TrimSpace(src.URL)
|
if src == nil || strings.TrimSpace(src.URL) == "" {
|
||||||
if u == "" {
|
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, strings.TrimSpace(src.URL), nil)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if strings.TrimSpace(src.Etag) != "" {
|
if etag := strings.TrimSpace(src.Etag); etag != "" {
|
||||||
req.Header.Set("If-None-Match", strings.TrimSpace(src.Etag))
|
req.Header.Set("If-None-Match", etag)
|
||||||
}
|
}
|
||||||
resp, err := hc.Do(req)
|
resp, err := hc.Do(req)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
_, _ = io.Copy(io.Discard, resp.Body)
|
if resp.StatusCode == http.StatusNotModified {
|
||||||
_ = resp.Body.Close()
|
_ = resp.Body.Close()
|
||||||
|
continue
|
||||||
|
}
|
||||||
if resp.StatusCode != http.StatusOK {
|
if resp.StatusCode != http.StatusOK {
|
||||||
|
_, _ = io.Copy(io.Discard, resp.Body)
|
||||||
|
_ = resp.Body.Close()
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
etag := strings.TrimSpace(resp.Header.Get("ETag"))
|
body, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
|
||||||
if etag == "" || etag == strings.TrimSpace(src.Etag) {
|
_ = resp.Body.Close()
|
||||||
|
if err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
e := etag
|
prefixStrs, err := parseCDNBody(string(body), src)
|
||||||
_, _ = st.UpdateCDNSource(tid, mod.ID, src.ID, &store.CDNSourcePatch{Etag: &e})
|
if err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
newEtag := strings.TrimSpace(resp.Header.Get("ETag"))
|
||||||
|
patch := &store.CDNSourcePatch{LastRefreshedAt: &now}
|
||||||
|
if newEtag != "" && newEtag != strings.TrimSpace(src.Etag) {
|
||||||
|
e := newEtag
|
||||||
|
patch.Etag = &e
|
||||||
|
}
|
||||||
|
_, _ = st.UpdateCDNSource(tid, mod.ID, src.ID, patch)
|
||||||
|
rows := cdnRowsFromParsed(omod, src, prefixStrs)
|
||||||
|
_ = mergeCDNSourceIntoModuleSnapshot(st, tid, omod, src.ID, rows)
|
||||||
|
_ = prior // prior may be stale after merge; refresh for next source in loop
|
||||||
|
if snap, ok, _ := st.GetModulePrefixSnapshot(tid, mod.ID); ok && snap != nil {
|
||||||
|
prior = snap.Prefixes
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+745
-178
File diff suppressed because it is too large.
Load diff
@@ -8,10 +8,30 @@ import (
|
|||||||
"evobgp/internal/store"
|
"evobgp/internal/store"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
func prefixRowSet(rows []store.PrefixRow) map[string]struct{} {
|
||||||
|
got := make(map[string]struct{}, len(rows))
|
||||||
|
for _, r := range rows {
|
||||||
|
c := prefixRowCommunity(r)
|
||||||
|
got[r.Prefix+"|"+c+"|"+r.Source] = struct{}{}
|
||||||
|
}
|
||||||
|
return got
|
||||||
|
}
|
||||||
|
|
||||||
func TestRefreshModule_AggregatesAllEnabledModules(t *testing.T) {
|
func TestRefreshModule_AggregatesAllEnabledModules(t *testing.T) {
|
||||||
|
t.Setenv("EVOBGP_ASN_RESOLVE", "0")
|
||||||
|
|
||||||
m := store.NewMemory()
|
m := store.NewMemory()
|
||||||
m.SeedDemo()
|
m.SeedDemo()
|
||||||
tenant, _, modIP, _, _ := m.DemoIDs()
|
tenant, _, modIP, _, _ := m.DemoIDs()
|
||||||
|
for _, mod := range m.ListModules(tenant) {
|
||||||
|
if mod == nil || mod.ID == modIP {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
disabled := false
|
||||||
|
if _, err := m.UpdateModule(tenant, mod.ID, &store.ModulePatch{Enabled: &disabled}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
mod2, err := m.CreateModule(tenant, &store.Module{Type: "IP_RANGES", Name: "extra-ip", Enabled: true, Priority: 30})
|
mod2, err := m.CreateModule(tenant, &store.Module{Type: "IP_RANGES", Name: "extra-ip", Enabled: true, Priority: 30})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -25,27 +45,227 @@ func TestRefreshModule_AggregatesAllEnabledModules(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
if _, err := RefreshModule(ctx, m, http.DefaultClient, tenant, modIP); err != nil {
|
before, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
|
beforeN := len(before)
|
||||||
|
|
||||||
|
rev1, err := RefreshModule(ctx, m, http.DefaultClient, tenant, modIP)
|
||||||
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
revs, _, _ := m.ListRevisions(tenant, modIP, "", 1)
|
after1, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
if len(revs) == 0 {
|
if len(after1) != beforeN+1 {
|
||||||
t.Fatal("no revision")
|
t.Fatalf("first refresh: want one new tenant revision, got %d -> %d", beforeN, len(after1))
|
||||||
}
|
}
|
||||||
px, _, _ := m.ListRevisionPrefixes(tenant, revs[0].ID, "", 1000)
|
px, _, _ := m.ListRevisionPrefixes(tenant, rev1, "", 1000)
|
||||||
if len(px) != 2 {
|
if len(px) != 2 {
|
||||||
t.Fatalf("first refresh: want 2 aggregated prefixes, got %d: %+v", len(px), px)
|
t.Fatalf("first refresh: want 2 aggregated prefixes, got %d: %+v", len(px), px)
|
||||||
}
|
}
|
||||||
|
|
||||||
if _, err := RefreshModule(ctx, m, http.DefaultClient, tenant, mod2.ID); err != nil {
|
rev2, err := RefreshModule(ctx, m, http.DefaultClient, tenant, mod2.ID)
|
||||||
|
if err != nil {
|
||||||
t.Fatal(err)
|
t.Fatal(err)
|
||||||
}
|
}
|
||||||
revs2, _, _ := m.ListRevisions(tenant, mod2.ID, "", 1)
|
if rev2 == "" {
|
||||||
if len(revs2) == 0 {
|
t.Fatal("second refresh: expected non-empty revision id")
|
||||||
t.Fatal("no revision for mod2")
|
|
||||||
}
|
}
|
||||||
px2, _, _ := m.ListRevisionPrefixes(tenant, revs2[0].ID, "", 1000)
|
after2, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
if len(px2) != 2 {
|
if len(after2) < len(after1) {
|
||||||
t.Fatalf("second refresh: want 2 aggregated prefixes, got %d: %+v", len(px2), px2)
|
t.Fatalf("second refresh: revisions count should not decrease, had %d now %d", len(after1), len(after2))
|
||||||
|
}
|
||||||
|
|
||||||
|
if _, err := m.CreateIPRangeEntry(tenant, mod2.ID, &store.IPRangeEntry{Prefix: "10.0.1.0/24"}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
rev3, err := RefreshModule(ctx, m, http.DefaultClient, tenant, mod2.ID)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if rev3 == rev1 {
|
||||||
|
t.Fatal("after prefix change, expected a new revision")
|
||||||
|
}
|
||||||
|
after3, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
|
if len(after3) < len(after2) {
|
||||||
|
t.Fatalf("third refresh: revisions count should not decrease, had %d now %d", len(after2), len(after3))
|
||||||
|
}
|
||||||
|
px3, _, _ := m.ListRevisionPrefixes(tenant, rev3, "", 1000)
|
||||||
|
if len(px3) != 2 {
|
||||||
|
t.Fatalf("third refresh: smart aggregation should merge adjacent /24, want 2 prefixes, got %d", len(px3))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSmartAggregatePrefixRows_RespectsCommunityAndSource(t *testing.T) {
|
||||||
|
commA := "c-a"
|
||||||
|
commB := "c-b"
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "10.0.0.0/24", CommunityID: &commA, Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.1.0/24", CommunityID: &commA, Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.2.0/24", CommunityID: &commA, Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.3.0/24", CommunityID: &commA, Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.4.0/24", CommunityID: &commB, Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.5.0/24", CommunityID: &commB, Source: "cdn:x"},
|
||||||
|
}
|
||||||
|
out := smartAggregatePrefixRows(rows)
|
||||||
|
got := make(map[string]struct{}, len(out))
|
||||||
|
for _, r := range out {
|
||||||
|
c := ""
|
||||||
|
if r.CommunityID != nil {
|
||||||
|
c = *r.CommunityID
|
||||||
|
}
|
||||||
|
got[r.Prefix+"|"+c+"|"+r.Source] = struct{}{}
|
||||||
|
}
|
||||||
|
// First four /24 collapse into /22 because attributes are identical.
|
||||||
|
if _, ok := got["10.0.0.0/22|c-a|ip_range"]; !ok {
|
||||||
|
t.Fatalf("expected merged prefix for c-a/ip_range, got: %+v", out)
|
||||||
|
}
|
||||||
|
// Different community/source must stay separate.
|
||||||
|
if _, ok := got["10.0.4.0/24|c-b|ip_range"]; !ok {
|
||||||
|
t.Fatalf("expected distinct prefix for c-b/ip_range, got: %+v", out)
|
||||||
|
}
|
||||||
|
if _, ok := got["10.0.5.0/24|c-b|cdn:x"]; !ok {
|
||||||
|
t.Fatalf("expected distinct prefix for c-b/cdn:x, got: %+v", out)
|
||||||
|
}
|
||||||
|
if len(out) != 3 {
|
||||||
|
t.Fatalf("expected 3 resulting rows, got %d: %+v", len(out), out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSmartAggregatePrefixRows_PruneCoveredPrefixes(t *testing.T) {
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "10.0.0.0/16", Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.1.0/24", Source: "ip_range"},
|
||||||
|
}
|
||||||
|
out := smartAggregatePrefixRows(rows)
|
||||||
|
got := prefixRowSet(out)
|
||||||
|
if len(out) != 1 {
|
||||||
|
t.Fatalf("expected 1 row after covered prune, got %d: %+v", len(out), out)
|
||||||
|
}
|
||||||
|
if _, ok := got["10.0.0.0/16||ip_range"]; !ok {
|
||||||
|
t.Fatalf("expected /16 only, got: %+v", out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSmartAggregatePrefixRows_MergeChainFourSlash24(t *testing.T) {
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.1.0/24", Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.2.0/24", Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.3.0/24", Source: "ip_range"},
|
||||||
|
}
|
||||||
|
out := smartAggregatePrefixRows(rows)
|
||||||
|
got := prefixRowSet(out)
|
||||||
|
if _, ok := got["10.0.0.0/22||ip_range"]; !ok {
|
||||||
|
t.Fatalf("expected merged /22, got: %+v", out)
|
||||||
|
}
|
||||||
|
if len(out) != 1 {
|
||||||
|
t.Fatalf("expected 1 row, got %d: %+v", len(out), out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSmartAggregatePrefixRows_DoesNotMergeAdjacentSlash8(t *testing.T) {
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "10.0.0.0/8", Source: "ip_range"},
|
||||||
|
{Prefix: "11.0.0.0/8", Source: "ip_range"},
|
||||||
|
}
|
||||||
|
out := smartAggregatePrefixRows(rows)
|
||||||
|
if len(out) != 2 {
|
||||||
|
t.Fatalf("expected two /8 prefixes, got %d: %+v", len(out), out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSmartAggregatePrefixRows_MergesIPv6Slash64(t *testing.T) {
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "2001:db8:0:0::/64", Source: "ip_range"},
|
||||||
|
{Prefix: "2001:db8:0:1::/64", Source: "ip_range"},
|
||||||
|
}
|
||||||
|
out := smartAggregatePrefixRows(rows)
|
||||||
|
got := prefixRowSet(out)
|
||||||
|
if _, ok := got["2001:db8::/63||ip_range"]; !ok {
|
||||||
|
t.Fatalf("expected merged IPv6 /63, got: %+v", out)
|
||||||
|
}
|
||||||
|
if len(out) != 1 {
|
||||||
|
t.Fatalf("expected 1 row, got %d: %+v", len(out), out)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestSmartAggregatePrefixRows_StableOutputOrder(t *testing.T) {
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "192.168.0.0/24", Source: "cdn:b"},
|
||||||
|
{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.1.0/24", Source: "ip_range"},
|
||||||
|
}
|
||||||
|
out1 := smartAggregatePrefixRows(rows)
|
||||||
|
out2 := smartAggregatePrefixRows(rows)
|
||||||
|
if len(out1) != len(out2) {
|
||||||
|
t.Fatalf("length mismatch: %d vs %d", len(out1), len(out2))
|
||||||
|
}
|
||||||
|
for i := range out1 {
|
||||||
|
if out1[i].Prefix != out2[i].Prefix || out1[i].Source != out2[i].Source {
|
||||||
|
t.Fatalf("order not stable at %d: %+v vs %+v", i, out1[i], out2[i])
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestRenderTenantRevisionFromPrefixes_SkipsWhenRedundantRawRows(t *testing.T) {
|
||||||
|
t.Setenv("EVOBGP_ASN_RESOLVE", "0")
|
||||||
|
|
||||||
|
m := store.NewMemory()
|
||||||
|
m.SeedDemo()
|
||||||
|
tenant, _, modIP, _, _ := m.DemoIDs()
|
||||||
|
for _, mod := range m.ListModules(tenant) {
|
||||||
|
if mod == nil || mod.ID == modIP {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
disabled := false
|
||||||
|
if _, err := m.UpdateModule(tenant, mod.ID, &store.ModulePatch{Enabled: &disabled}); err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ctx := context.Background()
|
||||||
|
baseRows := []store.PrefixRow{{Prefix: "10.0.0.0/16", Source: "ip_range"}}
|
||||||
|
rev1, err := RenderTenantRevisionFromPrefixes(ctx, m, http.DefaultClient, tenant, modIP, baseRows)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
|
||||||
|
rev2, err := RenderTenantRevisionFromPrefixes(ctx, m, http.DefaultClient, tenant, modIP, append(baseRows,
|
||||||
|
store.PrefixRow{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if rev2 != rev1 {
|
||||||
|
t.Fatalf("expected hash skip for redundant covered prefix, got rev1=%s rev2=%s", rev1, rev2)
|
||||||
|
}
|
||||||
|
|
||||||
|
before, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
|
rev3, err := RenderTenantRevisionFromPrefixes(ctx, m, http.DefaultClient, tenant, modIP, append(baseRows,
|
||||||
|
store.PrefixRow{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
store.PrefixRow{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
))
|
||||||
|
if err != nil {
|
||||||
|
t.Fatal(err)
|
||||||
|
}
|
||||||
|
if rev3 != rev1 {
|
||||||
|
t.Fatalf("expected hash skip for duplicate raw rows, got rev1=%s rev3=%s", rev1, rev3)
|
||||||
|
}
|
||||||
|
after, _, _ := m.ListRevisions(tenant, "", "", 200)
|
||||||
|
if len(after) != len(before) {
|
||||||
|
t.Fatalf("duplicate raw rows should not create revision, before=%d after=%d", len(before), len(after))
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestHashAggregatedMaterialization_DedupesIdenticalRows(t *testing.T) {
|
||||||
|
rows := []store.PrefixRow{
|
||||||
|
{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
}
|
||||||
|
h1 := hashAggregatedMaterialization("tenant-a", rows)
|
||||||
|
h2 := hashAggregatedMaterialization("tenant-a", []store.PrefixRow{
|
||||||
|
{Prefix: "10.0.0.0/24", Source: "ip_range"},
|
||||||
|
})
|
||||||
|
if h1 != h2 {
|
||||||
|
t.Fatalf("expected deduped hash to match, got %s vs %s", h1, h2)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// SchedulerTickSec matches the scheduler ticker interval (internal/scheduler).
|
||||||
|
const SchedulerTickSec = 30
|
||||||
|
|
||||||
|
// ModuleDueForScheduler reports whether a module's refresh interval bucket rolled since the last scheduler tick.
|
||||||
|
func ModuleDueForScheduler(mod *store.Module, now time.Time) bool {
|
||||||
|
if mod == nil || !mod.Enabled || mod.Type == "IP_RANGES" || mod.RefreshIntervalSec <= 0 {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
win := int64(mod.RefreshIntervalSec)
|
||||||
|
if win < 60 {
|
||||||
|
win = 60
|
||||||
|
}
|
||||||
|
cur := now.Unix() / win
|
||||||
|
prev := (now.Unix() - SchedulerTickSec) / win
|
||||||
|
return cur != prev
|
||||||
|
}
|
||||||
@@ -0,0 +1,80 @@
|
|||||||
|
package pipeline
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"fmt"
|
||||||
|
"net/http"
|
||||||
|
"strings"
|
||||||
|
"sync"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RefreshTenantModules ingests all listed modules in parallel and updates per-module snapshots.
|
||||||
|
func RefreshTenantModules(ctx context.Context, st store.Backend, hc *http.Client, tenantID string, moduleIDs []string) error {
|
||||||
|
if hc == nil {
|
||||||
|
hc = http.DefaultClient
|
||||||
|
}
|
||||||
|
var ids []string
|
||||||
|
seen := make(map[string]struct{})
|
||||||
|
for _, id := range moduleIDs {
|
||||||
|
id = strings.TrimSpace(id)
|
||||||
|
if id == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := seen[id]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[id] = struct{}{}
|
||||||
|
ids = append(ids, id)
|
||||||
|
}
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if len(ids) == 1 {
|
||||||
|
return RefreshModuleIngest(ctx, st, hc, tenantID, ids[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
sem := make(chan struct{}, collectConcurrency())
|
||||||
|
errs := make([]error, len(ids))
|
||||||
|
var wg sync.WaitGroup
|
||||||
|
for i, mid := range ids {
|
||||||
|
wg.Add(1)
|
||||||
|
go func(idx int, moduleID string) {
|
||||||
|
defer wg.Done()
|
||||||
|
sem <- struct{}{}
|
||||||
|
defer func() { <-sem }()
|
||||||
|
errs[idx] = RefreshModuleIngest(ctx, st, hc, tenantID, moduleID)
|
||||||
|
}(i, mid)
|
||||||
|
}
|
||||||
|
wg.Wait()
|
||||||
|
for _, err := range errs {
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// PickTenantRefreshTriggerModule returns a module id for revision metadata (first enabled due module).
|
||||||
|
func PickTenantRefreshTriggerModule(st store.Backend, tenantID string, moduleIDs []string) (string, error) {
|
||||||
|
for _, id := range moduleIDs {
|
||||||
|
id = strings.TrimSpace(id)
|
||||||
|
if id == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
mod, err := st.GetModule(tenantID, id)
|
||||||
|
if err != nil {
|
||||||
|
return "", err
|
||||||
|
}
|
||||||
|
if mod.Enabled {
|
||||||
|
return mod.ID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for _, mod := range st.ListModules(tenantID) {
|
||||||
|
if mod != nil && mod.Enabled {
|
||||||
|
return mod.ID, nil
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return "", fmt.Errorf("pipeline: no enabled module for tenant refresh")
|
||||||
|
}
|
||||||
@@ -0,0 +1,151 @@
|
|||||||
|
package reports
|
||||||
|
|
||||||
|
import (
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RouterListsCatalog is the aggregated router-lists payload for GET /router-lists/catalog.
|
||||||
|
type RouterListsCatalog struct {
|
||||||
|
Modules []map[string]any
|
||||||
|
Domains []map[string]any
|
||||||
|
ASNs []map[string]any
|
||||||
|
IPRanges []map[string]any
|
||||||
|
Communities []map[string]any
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildRouterListsCatalog loads modules and entries for the tenant catalog endpoint.
|
||||||
|
func BuildRouterListsCatalog(st store.Backend, tenantID string) (*RouterListsCatalog, error) {
|
||||||
|
mods := st.ListModules(tenantID)
|
||||||
|
out := &RouterListsCatalog{
|
||||||
|
Modules: make([]map[string]any, 0),
|
||||||
|
Domains: make([]map[string]any, 0),
|
||||||
|
ASNs: make([]map[string]any, 0),
|
||||||
|
IPRanges: make([]map[string]any, 0),
|
||||||
|
Communities: make([]map[string]any, 0),
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, mod := range mods {
|
||||||
|
switch mod.Type {
|
||||||
|
case "DOMAINS", "AS_PREFIXES", "IP_RANGES":
|
||||||
|
out.Modules = append(out.Modules, moduleMap(mod))
|
||||||
|
default:
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
switch mod.Type {
|
||||||
|
case "DOMAINS":
|
||||||
|
list, err := st.ListDomainEntries(tenantID, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, x := range list {
|
||||||
|
out.Domains = append(out.Domains, map[string]any{
|
||||||
|
"module_id": mod.ID,
|
||||||
|
"entry": domainEntryMap(x),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
case "AS_PREFIXES":
|
||||||
|
list, err := st.ListASEntries(tenantID, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, x := range list {
|
||||||
|
out.ASNs = append(out.ASNs, map[string]any{
|
||||||
|
"module_id": mod.ID,
|
||||||
|
"entry": asEntryMap(x),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
case "IP_RANGES":
|
||||||
|
list, err := st.ListIPRangeEntries(tenantID, mod.ID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, x := range list {
|
||||||
|
out.IPRanges = append(out.IPRanges, map[string]any{
|
||||||
|
"module_id": mod.ID,
|
||||||
|
"entry": ipRangeMap(x),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
comms, err := st.ListCommunities(tenantID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
for _, c := range comms {
|
||||||
|
out.Communities = append(out.Communities, map[string]any{
|
||||||
|
"id": c.ID,
|
||||||
|
"community": c.Community,
|
||||||
|
"title": c.Title,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
return out, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func moduleMap(mod *store.Module) map[string]any {
|
||||||
|
m := map[string]any{
|
||||||
|
"id": mod.ID,
|
||||||
|
"type": mod.Type,
|
||||||
|
"name": mod.Name,
|
||||||
|
"enabled": mod.Enabled,
|
||||||
|
"priority": mod.Priority,
|
||||||
|
"refresh_interval_sec": mod.RefreshIntervalSec,
|
||||||
|
"cron_expr": mod.CronExpr,
|
||||||
|
}
|
||||||
|
if mod.LastRefreshedAt != nil {
|
||||||
|
m["last_refreshed_at"] = mod.LastRefreshedAt.UTC().Format(time.RFC3339Nano)
|
||||||
|
} else {
|
||||||
|
m["last_refreshed_at"] = nil
|
||||||
|
}
|
||||||
|
if mod.DefaultCommunityID != nil {
|
||||||
|
m["default_community_id"] = *mod.DefaultCommunityID
|
||||||
|
} else {
|
||||||
|
m["default_community_id"] = nil
|
||||||
|
}
|
||||||
|
ids := mod.EffectiveDohProfileIDs()
|
||||||
|
if len(ids) > 0 {
|
||||||
|
m["doh_profile_ids"] = ids
|
||||||
|
} else {
|
||||||
|
m["doh_profile_ids"] = []string{}
|
||||||
|
}
|
||||||
|
m["doh_resolver_policy"] = store.NormalizeDohResolverPolicy(mod.DohResolverPolicy)
|
||||||
|
if mod.DohProfileID != nil {
|
||||||
|
m["doh_profile_id"] = *mod.DohProfileID
|
||||||
|
} else {
|
||||||
|
m["doh_profile_id"] = nil
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
func domainEntryMap(x *store.DomainEntry) map[string]any {
|
||||||
|
m := map[string]any{"id": x.ID, "fqdn": x.FQDN}
|
||||||
|
if x.CommunityID != nil {
|
||||||
|
m["community_id"] = *x.CommunityID
|
||||||
|
} else {
|
||||||
|
m["community_id"] = nil
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
func asEntryMap(x *store.ASEntry) map[string]any {
|
||||||
|
m := map[string]any{"id": x.ID, "asn": x.ASN}
|
||||||
|
if x.CommunityID != nil {
|
||||||
|
m["community_id"] = *x.CommunityID
|
||||||
|
} else {
|
||||||
|
m["community_id"] = nil
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
|
|
||||||
|
func ipRangeMap(x *store.IPRangeEntry) map[string]any {
|
||||||
|
m := map[string]any{"id": x.ID, "prefix": x.Prefix}
|
||||||
|
if x.CommunityID != nil {
|
||||||
|
m["community_id"] = *x.CommunityID
|
||||||
|
} else {
|
||||||
|
m["community_id"] = nil
|
||||||
|
}
|
||||||
|
return m
|
||||||
|
}
|
||||||
@@ -0,0 +1,236 @@
|
|||||||
|
package reports
|
||||||
|
|
||||||
|
import (
|
||||||
|
"fmt"
|
||||||
|
"sort"
|
||||||
|
"strconv"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
// RevisionDiagnosticLog is plain-text diagnostic output for a revision.
|
||||||
|
type RevisionDiagnosticLog struct {
|
||||||
|
Body []byte
|
||||||
|
Filename string
|
||||||
|
}
|
||||||
|
|
||||||
|
// BuildRevisionDiagnosticLog aggregates revision prefixes by source and community.
|
||||||
|
func BuildRevisionDiagnosticLog(st store.Backend, tenantID string, rev *store.Revision) (*RevisionDiagnosticLog, error) {
|
||||||
|
moduleType := ""
|
||||||
|
moduleName := ""
|
||||||
|
if strings.TrimSpace(rev.ModuleID) != "" {
|
||||||
|
if mod, modErr := st.GetModule(tenantID, rev.ModuleID); modErr == nil && mod != nil {
|
||||||
|
moduleType = strings.TrimSpace(mod.Type)
|
||||||
|
moduleName = strings.TrimSpace(mod.Name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
communityLabels := map[string]string{}
|
||||||
|
if communities, listErr := st.ListCommunities(tenantID); listErr == nil {
|
||||||
|
for _, c := range communities {
|
||||||
|
if c == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
label := strings.TrimSpace(c.Title)
|
||||||
|
if label == "" {
|
||||||
|
label = strings.TrimSpace(c.Community)
|
||||||
|
}
|
||||||
|
if label == "" {
|
||||||
|
label = c.ID
|
||||||
|
}
|
||||||
|
communityLabels[c.ID] = label
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
type rawRow struct {
|
||||||
|
Prefix string
|
||||||
|
Source string
|
||||||
|
Kind string
|
||||||
|
SourceName string
|
||||||
|
SourceDetail string
|
||||||
|
CommunityID string
|
||||||
|
CommunityLabel string
|
||||||
|
}
|
||||||
|
type summary struct {
|
||||||
|
Kind string
|
||||||
|
Source string
|
||||||
|
SourceDetail string
|
||||||
|
CommunityID string
|
||||||
|
CommunityLabel string
|
||||||
|
Count int
|
||||||
|
Sample []string
|
||||||
|
}
|
||||||
|
|
||||||
|
cdnSourceURLByID := map[string]string{}
|
||||||
|
if moduleType == "CDN_CIDRS" && strings.TrimSpace(rev.ModuleID) != "" {
|
||||||
|
if sources, srcErr := st.ListCDNSources(tenantID, rev.ModuleID); srcErr == nil {
|
||||||
|
for _, src := range sources {
|
||||||
|
if src == nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
id := strings.TrimSpace(src.ID)
|
||||||
|
url := strings.TrimSpace(src.URL)
|
||||||
|
if id != "" && url != "" {
|
||||||
|
cdnSourceURLByID[id] = url
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
rows := make([]rawRow, 0, rev.MaterializedPrefixCount)
|
||||||
|
byGroup := map[string]*summary{}
|
||||||
|
cursor := ""
|
||||||
|
for {
|
||||||
|
page, next, more := st.ListRevisionPrefixes(tenantID, rev.ID, cursor, 2000)
|
||||||
|
for _, p := range page {
|
||||||
|
kind, sourceName := classifyRevisionSource(p.Source)
|
||||||
|
sourceDetail := sourceName
|
||||||
|
if kind == "cdn" {
|
||||||
|
if url, ok := cdnSourceURLByID[sourceName]; ok && strings.TrimSpace(url) != "" {
|
||||||
|
sourceDetail = url
|
||||||
|
}
|
||||||
|
}
|
||||||
|
communityID := "none"
|
||||||
|
if p.CommunityID != nil && strings.TrimSpace(*p.CommunityID) != "" {
|
||||||
|
communityID = strings.TrimSpace(*p.CommunityID)
|
||||||
|
}
|
||||||
|
communityLabel := "без community"
|
||||||
|
if communityID != "none" {
|
||||||
|
if lbl, ok := communityLabels[communityID]; ok && strings.TrimSpace(lbl) != "" {
|
||||||
|
communityLabel = lbl
|
||||||
|
} else {
|
||||||
|
communityLabel = communityID
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rows = append(rows, rawRow{
|
||||||
|
Prefix: p.Prefix,
|
||||||
|
Source: p.Source,
|
||||||
|
Kind: kind,
|
||||||
|
SourceName: sourceName,
|
||||||
|
SourceDetail: sourceDetail,
|
||||||
|
CommunityID: communityID,
|
||||||
|
CommunityLabel: communityLabel,
|
||||||
|
})
|
||||||
|
groupKey := kind + "|" + sourceName + "|" + communityID
|
||||||
|
g, ok := byGroup[groupKey]
|
||||||
|
if !ok {
|
||||||
|
g = &summary{
|
||||||
|
Kind: kind,
|
||||||
|
Source: sourceName,
|
||||||
|
SourceDetail: sourceDetail,
|
||||||
|
CommunityID: communityID,
|
||||||
|
CommunityLabel: communityLabel,
|
||||||
|
Sample: make([]string, 0, 5),
|
||||||
|
}
|
||||||
|
byGroup[groupKey] = g
|
||||||
|
}
|
||||||
|
g.Count++
|
||||||
|
if len(g.Sample) < 5 {
|
||||||
|
g.Sample = append(g.Sample, p.Prefix)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if !more || strings.TrimSpace(next) == "" {
|
||||||
|
break
|
||||||
|
}
|
||||||
|
cursor = next
|
||||||
|
}
|
||||||
|
|
||||||
|
summaryKeys := make([]string, 0, len(byGroup))
|
||||||
|
for k := range byGroup {
|
||||||
|
summaryKeys = append(summaryKeys, k)
|
||||||
|
}
|
||||||
|
sort.Strings(summaryKeys)
|
||||||
|
|
||||||
|
sort.Slice(rows, func(i, j int) bool {
|
||||||
|
if rows[i].Kind != rows[j].Kind {
|
||||||
|
return rows[i].Kind < rows[j].Kind
|
||||||
|
}
|
||||||
|
if rows[i].SourceName != rows[j].SourceName {
|
||||||
|
return rows[i].SourceName < rows[j].SourceName
|
||||||
|
}
|
||||||
|
if rows[i].CommunityID != rows[j].CommunityID {
|
||||||
|
return rows[i].CommunityID < rows[j].CommunityID
|
||||||
|
}
|
||||||
|
return rows[i].Prefix < rows[j].Prefix
|
||||||
|
})
|
||||||
|
|
||||||
|
var b strings.Builder
|
||||||
|
b.WriteString("EvoBGP revision diagnostic log\n")
|
||||||
|
b.WriteString("generated_at=" + time.Now().UTC().Format(time.RFC3339Nano) + "\n")
|
||||||
|
b.WriteString("tenant_id=" + tenantID + "\n")
|
||||||
|
b.WriteString("revision_id=" + rev.ID + "\n")
|
||||||
|
b.WriteString("revision_created_at=" + rev.CreatedAt.UTC().Format(time.RFC3339Nano) + "\n")
|
||||||
|
b.WriteString("content_hash=" + rev.ContentHash + "\n")
|
||||||
|
b.WriteString("materialized_prefix_count=" + strconv.Itoa(rev.MaterializedPrefixCount) + "\n")
|
||||||
|
if rev.ModuleID != "" {
|
||||||
|
b.WriteString("module_id=" + rev.ModuleID + "\n")
|
||||||
|
} else {
|
||||||
|
b.WriteString("module_id=\n")
|
||||||
|
}
|
||||||
|
b.WriteString("module_type=" + moduleType + "\n")
|
||||||
|
b.WriteString("module_name=" + moduleName + "\n")
|
||||||
|
b.WriteString("fetched_rows=" + strconv.Itoa(len(rows)) + "\n\n")
|
||||||
|
|
||||||
|
b.WriteString("## Aggregation by source and community\n")
|
||||||
|
for _, k := range summaryKeys {
|
||||||
|
g := byGroup[k]
|
||||||
|
b.WriteString("- kind=" + g.Kind +
|
||||||
|
" source=" + g.Source +
|
||||||
|
" source_detail=" + g.SourceDetail +
|
||||||
|
" community_id=" + g.CommunityID +
|
||||||
|
" community_label=" + g.CommunityLabel +
|
||||||
|
" count=" + strconv.Itoa(g.Count))
|
||||||
|
if len(g.Sample) > 0 {
|
||||||
|
b.WriteString(" sample=" + strings.Join(g.Sample, ","))
|
||||||
|
}
|
||||||
|
b.WriteByte('\n')
|
||||||
|
}
|
||||||
|
|
||||||
|
b.WriteString("\n## Raw rows\n")
|
||||||
|
b.WriteString("prefix\tkind\tsource\tsource_detail\tcommunity_id\tcommunity_label\n")
|
||||||
|
for _, row := range rows {
|
||||||
|
b.WriteString(row.Prefix)
|
||||||
|
b.WriteByte('\t')
|
||||||
|
b.WriteString(row.Kind)
|
||||||
|
b.WriteByte('\t')
|
||||||
|
b.WriteString(row.Source)
|
||||||
|
b.WriteByte('\t')
|
||||||
|
b.WriteString(strings.ReplaceAll(row.SourceDetail, "\t", " "))
|
||||||
|
b.WriteByte('\t')
|
||||||
|
b.WriteString(row.CommunityID)
|
||||||
|
b.WriteByte('\t')
|
||||||
|
b.WriteString(strings.ReplaceAll(row.CommunityLabel, "\t", " "))
|
||||||
|
b.WriteByte('\n')
|
||||||
|
}
|
||||||
|
|
||||||
|
filename := fmt.Sprintf("revision-%s-diagnostic.log", shortRevisionID(rev.ID))
|
||||||
|
return &RevisionDiagnosticLog{Body: []byte(b.String()), Filename: filename}, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func classifyRevisionSource(src string) (kind, sourceName string) {
|
||||||
|
switch {
|
||||||
|
case strings.HasPrefix(src, "as:"):
|
||||||
|
return "asn", strings.TrimPrefix(src, "as:")
|
||||||
|
case strings.HasPrefix(src, "domain:"):
|
||||||
|
return "domain", strings.TrimPrefix(src, "domain:")
|
||||||
|
case strings.HasPrefix(src, "cdn:"):
|
||||||
|
return "cdn", strings.TrimPrefix(src, "cdn:")
|
||||||
|
case src == "ip_range":
|
||||||
|
return "ip_range", "manual_ranges"
|
||||||
|
default:
|
||||||
|
if strings.TrimSpace(src) == "" {
|
||||||
|
return "unknown", "unknown"
|
||||||
|
}
|
||||||
|
return "source", strings.TrimSpace(src)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func shortRevisionID(id string) string {
|
||||||
|
s := strings.TrimSpace(id)
|
||||||
|
if len(s) <= 8 {
|
||||||
|
return s
|
||||||
|
}
|
||||||
|
return s[:8]
|
||||||
|
}
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
package repository
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (p *Postgres) GetASNPrefixCache(asn int64) (*store.ASNPrefixCacheEntry, bool, error) {
|
||||||
|
ctx := context.Background()
|
||||||
|
var holder string
|
||||||
|
var fetchedAt time.Time
|
||||||
|
var raw []byte
|
||||||
|
err := p.pool.QueryRow(ctx, `
|
||||||
|
SELECT holder, fetched_at, prefixes_json FROM asn_prefix_cache WHERE asn = $1`, asn).
|
||||||
|
Scan(&holder, &fetchedAt, &raw)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
return nil, false, nil
|
||||||
|
}
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
var prefixes []string
|
||||||
|
if len(raw) > 0 {
|
||||||
|
_ = json.Unmarshal(raw, &prefixes)
|
||||||
|
}
|
||||||
|
return &store.ASNPrefixCacheEntry{
|
||||||
|
ASN: asn,
|
||||||
|
Holder: holder,
|
||||||
|
Prefixes: prefixes,
|
||||||
|
FetchedAt: fetchedAt.UTC(),
|
||||||
|
}, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) SetASNPrefixCache(asn int64, holder string, prefixes []string) error {
|
||||||
|
raw, err := json.Marshal(prefixes)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ctx := context.Background()
|
||||||
|
_, err = p.pool.Exec(ctx, `
|
||||||
|
INSERT INTO asn_prefix_cache (asn, holder, prefixes_json, fetched_at)
|
||||||
|
VALUES ($1, $2, $3::jsonb, now())
|
||||||
|
ON CONFLICT (asn) DO UPDATE SET
|
||||||
|
holder = EXCLUDED.holder,
|
||||||
|
prefixes_json = EXCLUDED.prefixes_json,
|
||||||
|
fetched_at = EXCLUDED.fetched_at`,
|
||||||
|
asn, holder, string(raw))
|
||||||
|
return err
|
||||||
|
}
|
||||||
@@ -0,0 +1,70 @@
|
|||||||
|
package repository
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"encoding/json"
|
||||||
|
"errors"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
|
||||||
|
"github.com/jackc/pgx/v5"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (p *Postgres) GetModulePrefixSnapshot(tenantID, moduleID string) (*store.ModulePrefixSnapshot, bool, error) {
|
||||||
|
ctx := context.Background()
|
||||||
|
var inputHash string
|
||||||
|
var collectedAt time.Time
|
||||||
|
var raw []byte
|
||||||
|
err := p.pool.QueryRow(ctx, `
|
||||||
|
SELECT input_hash, collected_at, prefixes_json
|
||||||
|
FROM module_prefix_snapshot
|
||||||
|
WHERE tenant_id = $1 AND module_id = $2`,
|
||||||
|
tenantID, moduleID).Scan(&inputHash, &collectedAt, &raw)
|
||||||
|
if err != nil {
|
||||||
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
|
return nil, false, nil
|
||||||
|
}
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
var prefixes []store.PrefixRow
|
||||||
|
if len(raw) > 0 {
|
||||||
|
if err := json.Unmarshal(raw, &prefixes); err != nil {
|
||||||
|
return nil, false, err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return &store.ModulePrefixSnapshot{
|
||||||
|
InputHash: inputHash,
|
||||||
|
CollectedAt: collectedAt.UTC(),
|
||||||
|
Prefixes: prefixes,
|
||||||
|
}, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) SetModulePrefixSnapshot(tenantID, moduleID, inputHash string, prefixes []store.PrefixRow) error {
|
||||||
|
if strings.TrimSpace(tenantID) == "" || strings.TrimSpace(moduleID) == "" || strings.TrimSpace(inputHash) == "" {
|
||||||
|
return store.ErrInvalidInput
|
||||||
|
}
|
||||||
|
raw, err := json.Marshal(prefixes)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ctx := context.Background()
|
||||||
|
_, err = p.pool.Exec(ctx, `
|
||||||
|
INSERT INTO module_prefix_snapshot (tenant_id, module_id, input_hash, collected_at, prefixes_json)
|
||||||
|
VALUES ($1::uuid, $2::uuid, $3, now(), $4::jsonb)
|
||||||
|
ON CONFLICT (tenant_id, module_id) DO UPDATE SET
|
||||||
|
input_hash = EXCLUDED.input_hash,
|
||||||
|
collected_at = EXCLUDED.collected_at,
|
||||||
|
prefixes_json = EXCLUDED.prefixes_json`,
|
||||||
|
tenantID, moduleID, inputHash, string(raw))
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) DeleteModulePrefixSnapshot(tenantID, moduleID string) error {
|
||||||
|
ctx := context.Background()
|
||||||
|
_, err := p.pool.Exec(ctx, `
|
||||||
|
DELETE FROM module_prefix_snapshot WHERE tenant_id = $1 AND module_id = $2`,
|
||||||
|
tenantID, moduleID)
|
||||||
|
return err
|
||||||
|
}
|
||||||
+281
-148
@@ -6,7 +6,8 @@ import (
|
|||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"sort"
|
"os"
|
||||||
|
"runtime"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
@@ -18,6 +19,36 @@ import (
|
|||||||
"github.com/jackc/pgx/v5/pgxpool"
|
"github.com/jackc/pgx/v5/pgxpool"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// #region agent log
|
||||||
|
func agentDebugNDJSON3214(hypothesisID, location, message string, data map[string]any) {
|
||||||
|
if os.Getenv("EVOBGP_DEBUG_LOG") != "1" {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
f, err := os.OpenFile("debug-3214dc.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0644)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
defer f.Close()
|
||||||
|
var ms runtime.MemStats
|
||||||
|
runtime.ReadMemStats(&ms)
|
||||||
|
payload := map[string]any{
|
||||||
|
"sessionId": "3214dc",
|
||||||
|
"hypothesisId": hypothesisID,
|
||||||
|
"location": location,
|
||||||
|
"message": message,
|
||||||
|
"data": data,
|
||||||
|
"timestamp": time.Now().UnixMilli(),
|
||||||
|
"allocBytes": ms.Alloc,
|
||||||
|
}
|
||||||
|
b, err := json.Marshal(payload)
|
||||||
|
if err != nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
_, _ = f.Write(append(b, '\n'))
|
||||||
|
}
|
||||||
|
|
||||||
|
// #endregion
|
||||||
|
|
||||||
// Postgres implements store.Backend using pgxpool.
|
// Postgres implements store.Backend using pgxpool.
|
||||||
type Postgres struct {
|
type Postgres struct {
|
||||||
pool *pgxpool.Pool
|
pool *pgxpool.Pool
|
||||||
@@ -42,22 +73,15 @@ func (p *Postgres) DemoIDs() (tenant, moduleCDN, moduleIP, revision, speaker str
|
|||||||
|
|
||||||
func (p *Postgres) MaterializedPrefixStats() (max int, sum int) {
|
func (p *Postgres) MaterializedPrefixStats() (max int, sum int) {
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
rows, err := p.pool.Query(ctx, `
|
// Агрегация в БД — не тащим все строки config_revision в память.
|
||||||
SELECT COALESCE((meta_json->>'materialized_prefix_count')::int, 0) AS n
|
err := p.pool.QueryRow(ctx, `
|
||||||
FROM config_revision`)
|
SELECT
|
||||||
|
COALESCE(MAX((meta_json->>'materialized_prefix_count')::int), 0),
|
||||||
|
COALESCE(SUM((meta_json->>'materialized_prefix_count')::int), 0)
|
||||||
|
FROM config_revision`).Scan(&max, &sum)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, 0
|
return 0, 0
|
||||||
}
|
}
|
||||||
defer rows.Close()
|
|
||||||
for rows.Next() {
|
|
||||||
var n int
|
|
||||||
if rows.Scan(&n) == nil {
|
|
||||||
sum += n
|
|
||||||
if n > max {
|
|
||||||
max = n
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
return max, sum
|
return max, sum
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -89,7 +113,8 @@ func (p *Postgres) PeerSessionCountsByState() map[string]int {
|
|||||||
func (p *Postgres) ListModules(tenantID string) []*store.Module {
|
func (p *Postgres) ListModules(tenantID string) []*store.Module {
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
rows, err := p.pool.Query(ctx, `
|
rows, err := p.pool.Query(ctx, `
|
||||||
SELECT id, type, name, enabled, priority, doh_profile_id::text, refresh_interval_sec, cron_expr, default_community_id::text
|
SELECT id, type, name, enabled, priority, doh_profile_id::text, doh_resolver_policy,
|
||||||
|
refresh_interval_sec, cron_expr, default_community_id::text, last_refreshed_at
|
||||||
FROM module WHERE tenant_id = $1 AND deleted_at IS NULL ORDER BY priority, name`, tenantID)
|
FROM module WHERE tenant_id = $1 AND deleted_at IS NULL ORDER BY priority, name`, tenantID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil
|
return nil
|
||||||
@@ -101,9 +126,11 @@ func (p *Postgres) ListModules(tenantID string) []*store.Module {
|
|||||||
m.TenantID = tenantID
|
m.TenantID = tenantID
|
||||||
var doh, dc, cron *string
|
var doh, dc, cron *string
|
||||||
var refresh *int32
|
var refresh *int32
|
||||||
if err := rows.Scan(&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &refresh, &cron, &dc); err != nil {
|
var last *time.Time
|
||||||
|
if err := rows.Scan(&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &m.DohResolverPolicy, &refresh, &cron, &dc, &last); err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
m.DohResolverPolicy = store.NormalizeDohResolverPolicy(m.DohResolverPolicy)
|
||||||
if refresh != nil {
|
if refresh != nil {
|
||||||
m.RefreshIntervalSec = int(*refresh)
|
m.RefreshIntervalSec = int(*refresh)
|
||||||
}
|
}
|
||||||
@@ -116,6 +143,13 @@ func (p *Postgres) ListModules(tenantID string) []*store.Module {
|
|||||||
if dc != nil && *dc != "" {
|
if dc != nil && *dc != "" {
|
||||||
m.DefaultCommunityID = dc
|
m.DefaultCommunityID = dc
|
||||||
}
|
}
|
||||||
|
if last != nil {
|
||||||
|
t := last.UTC()
|
||||||
|
m.LastRefreshedAt = &t
|
||||||
|
}
|
||||||
|
if err := p.fillModuleDohFields(ctx, &m); err != nil {
|
||||||
|
continue
|
||||||
|
}
|
||||||
out = append(out, &m)
|
out = append(out, &m)
|
||||||
}
|
}
|
||||||
return out
|
return out
|
||||||
@@ -127,10 +161,12 @@ func (p *Postgres) GetModule(tenantID, moduleID string) (*store.Module, error) {
|
|||||||
m.TenantID = tenantID
|
m.TenantID = tenantID
|
||||||
var doh, dc, cron *string
|
var doh, dc, cron *string
|
||||||
var refresh *int32
|
var refresh *int32
|
||||||
|
var last *time.Time
|
||||||
err := p.pool.QueryRow(ctx, `
|
err := p.pool.QueryRow(ctx, `
|
||||||
SELECT id, type, name, enabled, priority, doh_profile_id::text, refresh_interval_sec, cron_expr, default_community_id::text
|
SELECT id, type, name, enabled, priority, doh_profile_id::text, doh_resolver_policy,
|
||||||
|
refresh_interval_sec, cron_expr, default_community_id::text, last_refreshed_at
|
||||||
FROM module WHERE id = $1 AND tenant_id = $2 AND deleted_at IS NULL`, moduleID, tenantID).Scan(
|
FROM module WHERE id = $1 AND tenant_id = $2 AND deleted_at IS NULL`, moduleID, tenantID).Scan(
|
||||||
&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &refresh, &cron, &dc)
|
&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &m.DohResolverPolicy, &refresh, &cron, &dc, &last)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, pgx.ErrNoRows) {
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
return nil, store.ErrNotFound
|
return nil, store.ErrNotFound
|
||||||
@@ -149,6 +185,14 @@ func (p *Postgres) GetModule(tenantID, moduleID string) (*store.Module, error) {
|
|||||||
if dc != nil && *dc != "" {
|
if dc != nil && *dc != "" {
|
||||||
m.DefaultCommunityID = dc
|
m.DefaultCommunityID = dc
|
||||||
}
|
}
|
||||||
|
if last != nil {
|
||||||
|
t := last.UTC()
|
||||||
|
m.LastRefreshedAt = &t
|
||||||
|
}
|
||||||
|
m.DohResolverPolicy = store.NormalizeDohResolverPolicy(m.DohResolverPolicy)
|
||||||
|
if err := p.fillModuleDohFields(ctx, &m); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
return &m, nil
|
return &m, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -156,6 +200,7 @@ func (p *Postgres) CreateModule(tenantID string, in *store.Module) (*store.Modul
|
|||||||
if in == nil {
|
if in == nil {
|
||||||
return nil, store.ErrInvalidInput
|
return nil, store.ErrInvalidInput
|
||||||
}
|
}
|
||||||
|
store.NormalizeModuleDoh(in)
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
id := uuid.NewString()
|
id := uuid.NewString()
|
||||||
var doh, dc any
|
var doh, dc any
|
||||||
@@ -173,13 +218,21 @@ func (p *Postgres) CreateModule(tenantID string, in *store.Module) (*store.Modul
|
|||||||
if strings.TrimSpace(in.CronExpr) != "" {
|
if strings.TrimSpace(in.CronExpr) != "" {
|
||||||
cronArg = strings.TrimSpace(in.CronExpr)
|
cronArg = strings.TrimSpace(in.CronExpr)
|
||||||
}
|
}
|
||||||
|
var lastArg any
|
||||||
|
if in.LastRefreshedAt != nil {
|
||||||
|
lastArg = in.LastRefreshedAt.UTC()
|
||||||
|
}
|
||||||
|
policy := store.NormalizeDohResolverPolicy(in.DohResolverPolicy)
|
||||||
_, err := p.pool.Exec(ctx, `
|
_, err := p.pool.Exec(ctx, `
|
||||||
INSERT INTO module (id, tenant_id, type, name, enabled, priority, doh_profile_id, refresh_interval_sec, cron_expr, default_community_id)
|
INSERT INTO module (id, tenant_id, type, name, enabled, priority, doh_profile_id, doh_resolver_policy, refresh_interval_sec, cron_expr, default_community_id, last_refreshed_at)
|
||||||
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10)`,
|
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12)`,
|
||||||
id, tenantID, in.Type, in.Name, in.Enabled, in.Priority, doh, ri, cronArg, dc)
|
id, tenantID, in.Type, in.Name, in.Enabled, in.Priority, doh, policy, ri, cronArg, dc, lastArg)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
if err := p.setModuleDohProfiles(ctx, id, in.DohProfileIDs); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
return p.GetModule(tenantID, id)
|
return p.GetModule(tenantID, id)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -192,68 +245,68 @@ func (p *Postgres) UpdateModule(tenantID, moduleID string, patch *store.ModulePa
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
name := base.Name
|
work := *base
|
||||||
en := base.Enabled
|
|
||||||
pr := base.Priority
|
|
||||||
ri := base.RefreshIntervalSec
|
|
||||||
cron := base.CronExpr
|
|
||||||
var dc, doh *string
|
|
||||||
dc = base.DefaultCommunityID
|
|
||||||
doh = base.DohProfileID
|
|
||||||
if patch.Name != nil {
|
if patch.Name != nil {
|
||||||
name = strings.TrimSpace(*patch.Name)
|
work.Name = strings.TrimSpace(*patch.Name)
|
||||||
}
|
}
|
||||||
if patch.Enabled != nil {
|
if patch.Enabled != nil {
|
||||||
en = *patch.Enabled
|
work.Enabled = *patch.Enabled
|
||||||
}
|
}
|
||||||
if patch.Priority != nil {
|
if patch.Priority != nil {
|
||||||
pr = *patch.Priority
|
work.Priority = *patch.Priority
|
||||||
}
|
}
|
||||||
if patch.RefreshIntervalSec != nil {
|
if patch.RefreshIntervalSec != nil {
|
||||||
ri = *patch.RefreshIntervalSec
|
work.RefreshIntervalSec = *patch.RefreshIntervalSec
|
||||||
}
|
}
|
||||||
if patch.CronExpr != nil {
|
if patch.CronExpr != nil {
|
||||||
cron = *patch.CronExpr
|
work.CronExpr = *patch.CronExpr
|
||||||
}
|
}
|
||||||
if patch.DefaultCommunityID != nil {
|
if patch.DefaultCommunityID != nil {
|
||||||
v := strings.TrimSpace(*patch.DefaultCommunityID)
|
v := strings.TrimSpace(*patch.DefaultCommunityID)
|
||||||
if v == "" {
|
if v == "" {
|
||||||
dc = nil
|
work.DefaultCommunityID = nil
|
||||||
} else {
|
} else {
|
||||||
dc = &v
|
work.DefaultCommunityID = &v
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if patch.DohProfileID != nil {
|
store.ApplyModuleDohPatch(&work, patch)
|
||||||
v := strings.TrimSpace(*patch.DohProfileID)
|
if patch.LastRefreshedAt != nil {
|
||||||
if v == "" {
|
t := patch.LastRefreshedAt.UTC()
|
||||||
doh = nil
|
work.LastRefreshedAt = &t
|
||||||
} else {
|
|
||||||
doh = &v
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
var dcArg, dohArg any
|
var dcArg, dohArg any
|
||||||
if dc != nil {
|
if work.DefaultCommunityID != nil {
|
||||||
dcArg = *dc
|
dcArg = *work.DefaultCommunityID
|
||||||
}
|
}
|
||||||
if doh != nil {
|
if work.DohProfileID != nil {
|
||||||
dohArg = *doh
|
dohArg = *work.DohProfileID
|
||||||
}
|
}
|
||||||
var riArg any
|
var riArg any
|
||||||
if ri != 0 {
|
if work.RefreshIntervalSec != 0 {
|
||||||
riArg = ri
|
riArg = work.RefreshIntervalSec
|
||||||
}
|
}
|
||||||
var cronArg any
|
var cronArg any
|
||||||
if strings.TrimSpace(cron) != "" {
|
if strings.TrimSpace(work.CronExpr) != "" {
|
||||||
cronArg = strings.TrimSpace(cron)
|
cronArg = strings.TrimSpace(work.CronExpr)
|
||||||
}
|
}
|
||||||
|
var lastArg any
|
||||||
|
if work.LastRefreshedAt != nil {
|
||||||
|
lastArg = work.LastRefreshedAt.UTC()
|
||||||
|
}
|
||||||
|
policy := store.NormalizeDohResolverPolicy(work.DohResolverPolicy)
|
||||||
_, err = p.pool.Exec(ctx, `
|
_, err = p.pool.Exec(ctx, `
|
||||||
UPDATE module SET name=$3, enabled=$4, priority=$5, refresh_interval_sec=$6, cron_expr=$7,
|
UPDATE module SET name=$3, enabled=$4, priority=$5, refresh_interval_sec=$6, cron_expr=$7,
|
||||||
default_community_id=$8, doh_profile_id=$9, updated_at=now()
|
default_community_id=$8, doh_profile_id=$9, doh_resolver_policy=$10, last_refreshed_at=$11, updated_at=now()
|
||||||
WHERE id=$1 AND tenant_id=$2 AND deleted_at IS NULL`,
|
WHERE id=$1 AND tenant_id=$2 AND deleted_at IS NULL`,
|
||||||
moduleID, tenantID, name, en, pr, riArg, cronArg, dcArg, dohArg)
|
moduleID, tenantID, work.Name, work.Enabled, work.Priority, riArg, cronArg, dcArg, dohArg, policy, lastArg)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
if patch.DohProfileIDs != nil || patch.DohProfileID != nil {
|
||||||
|
if err := p.setModuleDohProfiles(ctx, moduleID, work.DohProfileIDs); err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
}
|
||||||
return p.GetModule(tenantID, moduleID)
|
return p.GetModule(tenantID, moduleID)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -554,14 +607,29 @@ func (p *Postgres) ListRevisions(tenantID, moduleID string, cursor string, limit
|
|||||||
if limit <= 0 {
|
if limit <= 0 {
|
||||||
limit = 50
|
limit = 50
|
||||||
}
|
}
|
||||||
|
off := 0
|
||||||
|
if cursor != "" {
|
||||||
|
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
|
||||||
|
off = n
|
||||||
|
}
|
||||||
|
}
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
q := `SELECT id::text, module_id::text, content_hash, parent_revision_id::text, meta_json, created_at FROM config_revision WHERE tenant_id=$1`
|
// List endpoint only needs materialized_prefix_count from meta_json — not full preview_fragments blobs.
|
||||||
|
// LIMIT/OFFSET in SQL avoids loading every revision for the tenant into memory (was O(N) per request).
|
||||||
|
q := `SELECT id::text, module_id::text, content_hash, parent_revision_id::text,
|
||||||
|
COALESCE((meta_json->>'materialized_prefix_count')::int, 0), created_at
|
||||||
|
FROM config_revision WHERE tenant_id=$1`
|
||||||
args := []any{tenantID}
|
args := []any{tenantID}
|
||||||
|
n := 2
|
||||||
if moduleID != "" {
|
if moduleID != "" {
|
||||||
q += ` AND module_id = $2`
|
q += fmt.Sprintf(` AND module_id=$%d`, n)
|
||||||
args = append(args, moduleID)
|
args = append(args, moduleID)
|
||||||
|
n++
|
||||||
}
|
}
|
||||||
q += ` ORDER BY created_at DESC`
|
q += ` ORDER BY created_at DESC`
|
||||||
|
// Fetch limit+1 rows to compute has_more without COUNT(*).
|
||||||
|
q += fmt.Sprintf(` LIMIT $%d OFFSET $%d`, n, n+1)
|
||||||
|
args = append(args, limit+1, off)
|
||||||
rows, err := p.pool.Query(ctx, q, args...)
|
rows, err := p.pool.Query(ctx, q, args...)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, "", false
|
return nil, "", false
|
||||||
@@ -572,58 +640,53 @@ func (p *Postgres) ListRevisions(tenantID, moduleID string, cursor string, limit
|
|||||||
var r store.Revision
|
var r store.Revision
|
||||||
r.TenantID = tenantID
|
r.TenantID = tenantID
|
||||||
var mod, parent *string
|
var mod, parent *string
|
||||||
var meta []byte
|
var mpc int
|
||||||
if err := rows.Scan(&r.ID, &mod, &r.ContentHash, &parent, &meta, &r.CreatedAt); err != nil {
|
if err := rows.Scan(&r.ID, &mod, &r.ContentHash, &parent, &mpc, &r.CreatedAt); err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if mod != nil {
|
if mod != nil {
|
||||||
r.ModuleID = *mod
|
r.ModuleID = *mod
|
||||||
}
|
}
|
||||||
r.ParentRevisionID = strOrNil(parent)
|
r.ParentRevisionID = strOrNil(parent)
|
||||||
var mj struct {
|
r.MaterializedPrefixCount = mpc
|
||||||
PreviewFragments map[string]string `json:"preview_fragments"`
|
r.PreviewFragments = map[string]string{}
|
||||||
MaterializedPrefixCount int `json:"materialized_prefix_count"`
|
|
||||||
}
|
|
||||||
_ = json.Unmarshal(meta, &mj)
|
|
||||||
if mj.PreviewFragments == nil {
|
|
||||||
mj.PreviewFragments = map[string]string{}
|
|
||||||
}
|
|
||||||
r.PreviewFragments = mj.PreviewFragments
|
|
||||||
r.MaterializedPrefixCount = mj.MaterializedPrefixCount
|
|
||||||
all = append(all, &r)
|
all = append(all, &r)
|
||||||
}
|
}
|
||||||
|
agentDebugNDJSON3214("A", "repository/postgres.go:ListRevisions", "list_revisions_fetched", map[string]any{
|
||||||
|
"rows": len(all), "limit": limit, "offset": off,
|
||||||
|
})
|
||||||
|
hasMore := len(all) > limit
|
||||||
|
if hasMore {
|
||||||
|
all = all[:limit]
|
||||||
|
}
|
||||||
|
next := ""
|
||||||
|
if hasMore {
|
||||||
|
next = fmt.Sprintf("%d", off+limit)
|
||||||
|
}
|
||||||
|
if len(all) == 0 {
|
||||||
|
return nil, "", false
|
||||||
|
}
|
||||||
|
return all, next, hasMore
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) ListRevisionPrefixes(tenantID, revisionID string, cursor string, limit int) ([]store.PrefixRow, string, bool) {
|
||||||
|
if limit <= 0 {
|
||||||
|
limit = 50
|
||||||
|
}
|
||||||
off := 0
|
off := 0
|
||||||
if cursor != "" {
|
if cursor != "" {
|
||||||
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
|
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
|
||||||
off = n
|
off = n
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
end := off + limit
|
|
||||||
next := ""
|
|
||||||
hasMore := false
|
|
||||||
if end > len(all) {
|
|
||||||
end = len(all)
|
|
||||||
} else {
|
|
||||||
hasMore = true
|
|
||||||
next = fmt.Sprintf("%d", end)
|
|
||||||
}
|
|
||||||
if off >= len(all) {
|
|
||||||
return nil, "", false
|
|
||||||
}
|
|
||||||
return all[off:end], next, hasMore
|
|
||||||
}
|
|
||||||
|
|
||||||
func (p *Postgres) ListRevisionPrefixes(tenantID, revisionID string, cursor string, limit int) ([]store.PrefixRow, string, bool) {
|
|
||||||
if limit <= 0 {
|
|
||||||
limit = 50
|
|
||||||
}
|
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
if _, err := p.GetRevision(tenantID, revisionID); err != nil {
|
if _, err := p.GetRevision(tenantID, revisionID); err != nil {
|
||||||
return nil, "", false
|
return nil, "", false
|
||||||
}
|
}
|
||||||
rows, err := p.pool.Query(ctx, `
|
rows, err := p.pool.Query(ctx, `
|
||||||
SELECT prefix::text, community_id::text, source FROM revision_materialized_prefix
|
SELECT prefix::text, community_id::text, source FROM revision_materialized_prefix
|
||||||
WHERE revision_id=$1 ORDER BY id`, revisionID)
|
WHERE revision_id=$1 ORDER BY id
|
||||||
|
LIMIT $2 OFFSET $3`, revisionID, limit+1, off)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, "", false
|
return nil, "", false
|
||||||
}
|
}
|
||||||
@@ -638,25 +701,21 @@ func (p *Postgres) ListRevisionPrefixes(tenantID, revisionID string, cursor stri
|
|||||||
pr.CommunityID = comm
|
pr.CommunityID = comm
|
||||||
all = append(all, pr)
|
all = append(all, pr)
|
||||||
}
|
}
|
||||||
off := 0
|
agentDebugNDJSON3214("B", "repository/postgres.go:ListRevisionPrefixes", "list_prefixes_fetched", map[string]any{
|
||||||
if cursor != "" {
|
"rows": len(all), "limit": limit, "offset": off,
|
||||||
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
|
})
|
||||||
off = n
|
more := len(all) > limit
|
||||||
}
|
if more {
|
||||||
|
all = all[:limit]
|
||||||
}
|
}
|
||||||
end := off + limit
|
|
||||||
next := ""
|
next := ""
|
||||||
more := false
|
if more {
|
||||||
if end > len(all) {
|
next = fmt.Sprintf("%d", off+limit)
|
||||||
end = len(all)
|
|
||||||
} else {
|
|
||||||
more = true
|
|
||||||
next = fmt.Sprintf("%d", end)
|
|
||||||
}
|
}
|
||||||
if off >= len(all) {
|
if len(all) == 0 {
|
||||||
return nil, "", false
|
return nil, "", false
|
||||||
}
|
}
|
||||||
return all[off:end], next, more
|
return all, next, more
|
||||||
}
|
}
|
||||||
|
|
||||||
func (p *Postgres) CreateRollbackRevision(tenantID, sourceRevisionID string) (string, error) {
|
func (p *Postgres) CreateRollbackRevision(tenantID, sourceRevisionID string) (string, error) {
|
||||||
@@ -668,7 +727,7 @@ func (p *Postgres) CreateRollbackRevision(tenantID, sourceRevisionID string) (st
|
|||||||
newID := uuid.NewString()
|
newID := uuid.NewString()
|
||||||
parent := sourceRevisionID
|
parent := sourceRevisionID
|
||||||
meta, _ := json.Marshal(map[string]any{
|
meta, _ := json.Marshal(map[string]any{
|
||||||
"preview_fragments": src.PreviewFragments,
|
"preview_fragments": src.PreviewFragments,
|
||||||
"materialized_prefix_count": src.MaterializedPrefixCount,
|
"materialized_prefix_count": src.MaterializedPrefixCount,
|
||||||
})
|
})
|
||||||
var modArg any
|
var modArg any
|
||||||
@@ -697,32 +756,54 @@ func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, erro
|
|||||||
if _, err := p.GetRevision(tenantID, bID); err != nil {
|
if _, err := p.GetRevision(tenantID, bID); err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
pa, _, _ := p.ListRevisionPrefixes(tenantID, aID, "", 100000)
|
ctx := context.Background()
|
||||||
pb, _, _ := p.ListRevisionPrefixes(tenantID, bID, "", 100000)
|
var unchanged int
|
||||||
setA := make(map[string]struct{})
|
err := p.pool.QueryRow(ctx, `
|
||||||
setB := make(map[string]struct{})
|
SELECT COUNT(*)::int FROM (
|
||||||
for _, x := range pa {
|
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
|
||||||
setA[x.Prefix] = struct{}{}
|
INTERSECT
|
||||||
|
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
|
||||||
|
) t`, aID, bID).Scan(&unchanged)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
}
|
}
|
||||||
for _, x := range pb {
|
// added: в B, нет в A; removed: в A, нет в B — без загрузки полных снапшотов в память.
|
||||||
setB[x.Prefix] = struct{}{}
|
rowsAdded, err := p.pool.Query(ctx, `
|
||||||
|
SELECT prefix::text FROM (
|
||||||
|
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
|
||||||
|
EXCEPT
|
||||||
|
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
|
||||||
|
) s ORDER BY 1`, bID, aID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
}
|
}
|
||||||
var added, removed []string
|
defer rowsAdded.Close()
|
||||||
unchanged := 0
|
var added []string
|
||||||
for pfx := range setB {
|
for rowsAdded.Next() {
|
||||||
if _, ok := setA[pfx]; !ok {
|
var s string
|
||||||
added = append(added, pfx)
|
if err := rowsAdded.Scan(&s); err != nil {
|
||||||
} else {
|
continue
|
||||||
unchanged++
|
|
||||||
}
|
}
|
||||||
|
added = append(added, s)
|
||||||
}
|
}
|
||||||
for pfx := range setA {
|
rowsRem, err := p.pool.Query(ctx, `
|
||||||
if _, ok := setB[pfx]; !ok {
|
SELECT prefix::text FROM (
|
||||||
removed = append(removed, pfx)
|
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
|
||||||
|
EXCEPT
|
||||||
|
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
|
||||||
|
) s ORDER BY 1`, aID, bID)
|
||||||
|
if err != nil {
|
||||||
|
return nil, err
|
||||||
|
}
|
||||||
|
defer rowsRem.Close()
|
||||||
|
var removed []string
|
||||||
|
for rowsRem.Next() {
|
||||||
|
var s string
|
||||||
|
if err := rowsRem.Scan(&s); err != nil {
|
||||||
|
continue
|
||||||
}
|
}
|
||||||
|
removed = append(removed, s)
|
||||||
}
|
}
|
||||||
sort.Strings(added)
|
|
||||||
sort.Strings(removed)
|
|
||||||
return map[string]any{
|
return map[string]any{
|
||||||
"revision_a": aID,
|
"revision_a": aID,
|
||||||
"revision_b": bID,
|
"revision_b": bID,
|
||||||
@@ -732,6 +813,32 @@ func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, erro
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) PruneRevisionsBefore(tenantID string, cutoff time.Time) (int, error) {
|
||||||
|
ctx := context.Background()
|
||||||
|
cmd, err := p.pool.Exec(ctx, `
|
||||||
|
DELETE FROM config_revision AS cr
|
||||||
|
WHERE cr.tenant_id = $1
|
||||||
|
AND cr.created_at < $2
|
||||||
|
AND cr.id <> (
|
||||||
|
SELECT id
|
||||||
|
FROM config_revision
|
||||||
|
WHERE tenant_id = $1
|
||||||
|
ORDER BY created_at DESC
|
||||||
|
LIMIT 1
|
||||||
|
)
|
||||||
|
AND NOT EXISTS (
|
||||||
|
SELECT 1
|
||||||
|
FROM bgp_speaker AS sp
|
||||||
|
WHERE sp.tenant_id = $1
|
||||||
|
AND (sp.last_applied_revision_id = cr.id OR sp.published_revision_id = cr.id)
|
||||||
|
)`,
|
||||||
|
tenantID, cutoff.UTC())
|
||||||
|
if err != nil {
|
||||||
|
return 0, err
|
||||||
|
}
|
||||||
|
return int(cmd.RowsAffected()), nil
|
||||||
|
}
|
||||||
|
|
||||||
func (p *Postgres) SetLastAppliedRevision(tenantID, speakerID, revisionID string) error {
|
func (p *Postgres) SetLastAppliedRevision(tenantID, speakerID, revisionID string) error {
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
tag, err := p.pool.Exec(ctx, `
|
tag, err := p.pool.Exec(ctx, `
|
||||||
@@ -805,7 +912,7 @@ func (p *Postgres) CreateRenderRevision(revisionID, tenantID, moduleID string, p
|
|||||||
previewFragments = map[string]string{}
|
previewFragments = map[string]string{}
|
||||||
}
|
}
|
||||||
meta, err := json.Marshal(map[string]any{
|
meta, err := json.Marshal(map[string]any{
|
||||||
"preview_fragments": previewFragments,
|
"preview_fragments": previewFragments,
|
||||||
"materialized_prefix_count": len(prefixes),
|
"materialized_prefix_count": len(prefixes),
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -828,19 +935,22 @@ func (p *Postgres) CreateRenderRevision(revisionID, tenantID, moduleID string, p
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
for _, pr := range prefixes {
|
if len(prefixes) > 0 {
|
||||||
var comm any
|
_, err = tx.CopyFrom(ctx,
|
||||||
if pr.CommunityID != nil && strings.TrimSpace(*pr.CommunityID) != "" {
|
pgx.Identifier{"revision_materialized_prefix"},
|
||||||
comm = strings.TrimSpace(*pr.CommunityID)
|
[]string{"revision_id", "prefix", "community_id", "source"},
|
||||||
}
|
pgx.CopyFromSlice(len(prefixes), func(i int) ([]any, error) {
|
||||||
src := pr.Source
|
pr := prefixes[i]
|
||||||
if strings.TrimSpace(src) == "" {
|
var comm any
|
||||||
src = "render"
|
if pr.CommunityID != nil && strings.TrimSpace(*pr.CommunityID) != "" {
|
||||||
}
|
comm = strings.TrimSpace(*pr.CommunityID)
|
||||||
_, err = tx.Exec(ctx, `
|
}
|
||||||
INSERT INTO revision_materialized_prefix (revision_id, prefix, community_id, source)
|
src := pr.Source
|
||||||
VALUES ($1::uuid, $2, $3::uuid, $4)`,
|
if strings.TrimSpace(src) == "" {
|
||||||
strings.TrimSpace(revisionID), strings.TrimSpace(pr.Prefix), comm, src)
|
src = "render"
|
||||||
|
}
|
||||||
|
return []any{strings.TrimSpace(revisionID), strings.TrimSpace(pr.Prefix), comm, src}, nil
|
||||||
|
}))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
@@ -937,6 +1047,13 @@ func (p *Postgres) UpdateDohProfile(tenantID, id string, patch *store.DohProfile
|
|||||||
|
|
||||||
func (p *Postgres) DeleteDohProfile(tenantID, id string) error {
|
func (p *Postgres) DeleteDohProfile(tenantID, id string) error {
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
|
inUse, err := p.moduleDohProfileInUse(ctx, id)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
if inUse {
|
||||||
|
return store.ErrInvalidInput
|
||||||
|
}
|
||||||
var n int
|
var n int
|
||||||
_ = p.pool.QueryRow(ctx, `SELECT COUNT(*) FROM module WHERE doh_profile_id=$1::uuid AND deleted_at IS NULL`, id).Scan(&n)
|
_ = p.pool.QueryRow(ctx, `SELECT COUNT(*) FROM module WHERE doh_profile_id=$1::uuid AND deleted_at IS NULL`, id).Scan(&n)
|
||||||
if n > 0 {
|
if n > 0 {
|
||||||
@@ -954,7 +1071,7 @@ func (p *Postgres) DeleteDohProfile(tenantID, id string) error {
|
|||||||
|
|
||||||
func (p *Postgres) ListCommunities(tenantID string) ([]*store.Community, error) {
|
func (p *Postgres) ListCommunities(tenantID string) ([]*store.Community, error) {
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
rows, err := p.pool.Query(ctx, `SELECT id::text, name, kind, value_json::text FROM bgp_community WHERE tenant_id=$1`, tenantID)
|
rows, err := p.pool.Query(ctx, `SELECT id::text, community, title, value_json::text FROM bgp_community WHERE tenant_id=$1 ORDER BY COALESCE(NULLIF(trim(title), ''), community)`, tenantID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -963,7 +1080,7 @@ func (p *Postgres) ListCommunities(tenantID string) ([]*store.Community, error)
|
|||||||
for rows.Next() {
|
for rows.Next() {
|
||||||
var c store.Community
|
var c store.Community
|
||||||
c.TenantID = tenantID
|
c.TenantID = tenantID
|
||||||
if err := rows.Scan(&c.ID, &c.Name, &c.Kind, &c.ValueJSON); err != nil {
|
if err := rows.Scan(&c.ID, &c.Community, &c.Title, &c.ValueJSON); err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
out = append(out, &c)
|
out = append(out, &c)
|
||||||
@@ -975,8 +1092,8 @@ func (p *Postgres) GetCommunity(tenantID, id string) (*store.Community, error) {
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
var c store.Community
|
var c store.Community
|
||||||
c.TenantID = tenantID
|
c.TenantID = tenantID
|
||||||
err := p.pool.QueryRow(ctx, `SELECT id::text, name, kind, value_json::text FROM bgp_community WHERE id=$1 AND tenant_id=$2`, id, tenantID).Scan(
|
err := p.pool.QueryRow(ctx, `SELECT id::text, community, title, value_json::text FROM bgp_community WHERE id=$1 AND tenant_id=$2`, id, tenantID).Scan(
|
||||||
&c.ID, &c.Name, &c.Kind, &c.ValueJSON)
|
&c.ID, &c.Community, &c.Title, &c.ValueJSON)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
if errors.Is(err, pgx.ErrNoRows) {
|
if errors.Is(err, pgx.ErrNoRows) {
|
||||||
return nil, store.ErrNotFound
|
return nil, store.ErrNotFound
|
||||||
@@ -990,14 +1107,19 @@ func (p *Postgres) CreateCommunity(tenantID string, in *store.Community) (*store
|
|||||||
if in == nil {
|
if in == nil {
|
||||||
return nil, store.ErrInvalidInput
|
return nil, store.ErrInvalidInput
|
||||||
}
|
}
|
||||||
|
if strings.TrimSpace(in.Community) == "" {
|
||||||
|
return nil, store.ErrInvalidInput
|
||||||
|
}
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
id := uuid.NewString()
|
id := uuid.NewString()
|
||||||
vj := in.ValueJSON
|
vj := in.ValueJSON
|
||||||
if strings.TrimSpace(vj) == "" {
|
if strings.TrimSpace(vj) == "" {
|
||||||
vj = "{}"
|
vj = "{}"
|
||||||
}
|
}
|
||||||
_, err := p.pool.Exec(ctx, `INSERT INTO bgp_community (id, tenant_id, name, kind, value_json) VALUES ($1,$2,$3,$4,$5::jsonb)`,
|
comm := strings.TrimSpace(in.Community)
|
||||||
id, tenantID, in.Name, in.Kind, vj)
|
title := strings.TrimSpace(in.Title)
|
||||||
|
_, err := p.pool.Exec(ctx, `INSERT INTO bgp_community (id, tenant_id, community, title, value_json) VALUES ($1,$2,$3,$4,$5::jsonb)`,
|
||||||
|
id, tenantID, comm, title, vj)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -1009,18 +1131,21 @@ func (p *Postgres) UpdateCommunity(tenantID, id string, patch *store.CommunityPa
|
|||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
if patch.Name != nil {
|
if patch.Community != nil {
|
||||||
cur.Name = *patch.Name
|
cur.Community = strings.TrimSpace(*patch.Community)
|
||||||
}
|
}
|
||||||
if patch.Kind != nil {
|
if patch.Title != nil {
|
||||||
cur.Kind = *patch.Kind
|
cur.Title = strings.TrimSpace(*patch.Title)
|
||||||
}
|
}
|
||||||
if patch.ValueJSON != nil {
|
if patch.ValueJSON != nil {
|
||||||
cur.ValueJSON = *patch.ValueJSON
|
cur.ValueJSON = *patch.ValueJSON
|
||||||
}
|
}
|
||||||
|
if strings.TrimSpace(cur.Community) == "" {
|
||||||
|
return nil, store.ErrInvalidInput
|
||||||
|
}
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
_, err = p.pool.Exec(ctx, `UPDATE bgp_community SET name=$3, kind=$4, value_json=$5::jsonb, updated_at=now() WHERE id=$1 AND tenant_id=$2`,
|
_, err = p.pool.Exec(ctx, `UPDATE bgp_community SET community=$3, title=$4, value_json=$5::jsonb, updated_at=now() WHERE id=$1 AND tenant_id=$2`,
|
||||||
id, tenantID, cur.Name, cur.Kind, cur.ValueJSON)
|
id, tenantID, cur.Community, cur.Title, cur.ValueJSON)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -1092,6 +1217,14 @@ func nullInt32Ptr(i *int) *int32 {
|
|||||||
return &v
|
return &v
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func nullTimePtr(t *time.Time) *time.Time {
|
||||||
|
if t == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
v := t.UTC()
|
||||||
|
return &v
|
||||||
|
}
|
||||||
|
|
||||||
func nullJSON(s string) *string {
|
func nullJSON(s string) *string {
|
||||||
if strings.TrimSpace(s) == "" {
|
if strings.TrimSpace(s) == "" {
|
||||||
v := "{}"
|
v := "{}"
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ func (p *Postgres) ListCDNSources(tenantID, moduleID string) ([]*store.CDNSource
|
|||||||
}
|
}
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
rows, err := p.pool.Query(ctx, `
|
rows, err := p.pool.Query(ctx, `
|
||||||
SELECT id::text, source_kind, url, COALESCE(etag,''), refresh_interval_sec, community_id::text
|
SELECT id::text, source_kind, url, COALESCE(prefix_path,''), COALESCE(etag,''), refresh_interval_sec, community_id::text, last_refreshed_at
|
||||||
FROM module_cdn_source WHERE module_id=$1 ORDER BY url`, moduleID)
|
FROM module_cdn_source WHERE module_id=$1 ORDER BY url`, moduleID)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
@@ -35,7 +35,8 @@ func (p *Postgres) ListCDNSources(tenantID, moduleID string) ([]*store.CDNSource
|
|||||||
s.ModuleID = moduleID
|
s.ModuleID = moduleID
|
||||||
var ri *int32
|
var ri *int32
|
||||||
var comm *string
|
var comm *string
|
||||||
if err := rows.Scan(&s.ID, &s.SourceKind, &s.URL, &s.Etag, &ri, &comm); err != nil {
|
var last *time.Time
|
||||||
|
if err := rows.Scan(&s.ID, &s.SourceKind, &s.URL, &s.PrefixPath, &s.Etag, &ri, &comm, &last); err != nil {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
if ri != nil {
|
if ri != nil {
|
||||||
@@ -43,6 +44,10 @@ func (p *Postgres) ListCDNSources(tenantID, moduleID string) ([]*store.CDNSource
|
|||||||
s.RefreshIntervalSec = &v
|
s.RefreshIntervalSec = &v
|
||||||
}
|
}
|
||||||
s.CommunityID = strOrNil(comm)
|
s.CommunityID = strOrNil(comm)
|
||||||
|
if last != nil {
|
||||||
|
t := last.UTC()
|
||||||
|
s.LastRefreshedAt = &t
|
||||||
|
}
|
||||||
out = append(out, &s)
|
out = append(out, &s)
|
||||||
}
|
}
|
||||||
return out, nil
|
return out, nil
|
||||||
@@ -62,9 +67,9 @@ func (p *Postgres) CreateCDNSource(tenantID, moduleID string, in *store.CDNSourc
|
|||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
id := uuid.NewString()
|
id := uuid.NewString()
|
||||||
_, err = p.pool.Exec(ctx, `
|
_, err = p.pool.Exec(ctx, `
|
||||||
INSERT INTO module_cdn_source (id, module_id, source_kind, url, etag, refresh_interval_sec, community_id)
|
INSERT INTO module_cdn_source (id, module_id, source_kind, url, prefix_path, etag, refresh_interval_sec, community_id, last_refreshed_at)
|
||||||
VALUES ($1,$2,$3,$4,$5,$6, NULLIF($7::uuid, '00000000-0000-0000-0000-000000000000'::uuid))`,
|
VALUES ($1,$2,$3,$4,$5,$6,$7, NULLIF($8::uuid, '00000000-0000-0000-0000-000000000000'::uuid), $9)`,
|
||||||
id, moduleID, in.SourceKind, strings.TrimSpace(in.URL), in.Etag, nullInt32Ptr(in.RefreshIntervalSec), uuidOrNilPtr(in.CommunityID))
|
id, moduleID, in.SourceKind, strings.TrimSpace(in.URL), strings.TrimSpace(in.PrefixPath), in.Etag, nullInt32Ptr(in.RefreshIntervalSec), uuidOrNilPtr(in.CommunityID), nullTimePtr(in.LastRefreshedAt))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -76,9 +81,10 @@ func (p *Postgres) getCDNSource(ctx context.Context, moduleID, id string) (*stor
|
|||||||
s.ModuleID = moduleID
|
s.ModuleID = moduleID
|
||||||
var ri *int32
|
var ri *int32
|
||||||
var comm *string
|
var comm *string
|
||||||
|
var last *time.Time
|
||||||
err := p.pool.QueryRow(ctx, `
|
err := p.pool.QueryRow(ctx, `
|
||||||
SELECT id::text, source_kind, url, COALESCE(etag,''), refresh_interval_sec, community_id::text
|
SELECT id::text, source_kind, url, COALESCE(prefix_path,''), COALESCE(etag,''), refresh_interval_sec, community_id::text, last_refreshed_at
|
||||||
FROM module_cdn_source WHERE id=$1 AND module_id=$2`, id, moduleID).Scan(&s.ID, &s.SourceKind, &s.URL, &s.Etag, &ri, &comm)
|
FROM module_cdn_source WHERE id=$1 AND module_id=$2`, id, moduleID).Scan(&s.ID, &s.SourceKind, &s.URL, &s.PrefixPath, &s.Etag, &ri, &comm, &last)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -87,6 +93,10 @@ func (p *Postgres) getCDNSource(ctx context.Context, moduleID, id string) (*stor
|
|||||||
s.RefreshIntervalSec = &v
|
s.RefreshIntervalSec = &v
|
||||||
}
|
}
|
||||||
s.CommunityID = strOrNil(comm)
|
s.CommunityID = strOrNil(comm)
|
||||||
|
if last != nil {
|
||||||
|
t := last.UTC()
|
||||||
|
s.LastRefreshedAt = &t
|
||||||
|
}
|
||||||
return &s, nil
|
return &s, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -114,6 +124,9 @@ func (p *Postgres) UpdateCDNSource(tenantID, moduleID, sourceID string, patch *s
|
|||||||
if patch.URL != nil {
|
if patch.URL != nil {
|
||||||
cur.URL = strings.TrimSpace(*patch.URL)
|
cur.URL = strings.TrimSpace(*patch.URL)
|
||||||
}
|
}
|
||||||
|
if patch.PrefixPath != nil {
|
||||||
|
cur.PrefixPath = strings.TrimSpace(*patch.PrefixPath)
|
||||||
|
}
|
||||||
if patch.Etag != nil {
|
if patch.Etag != nil {
|
||||||
cur.Etag = *patch.Etag
|
cur.Etag = *patch.Etag
|
||||||
}
|
}
|
||||||
@@ -128,12 +141,16 @@ func (p *Postgres) UpdateCDNSource(tenantID, moduleID, sourceID string, patch *s
|
|||||||
cur.CommunityID = &v
|
cur.CommunityID = &v
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if patch.LastRefreshedAt != nil {
|
||||||
|
t := patch.LastRefreshedAt.UTC()
|
||||||
|
cur.LastRefreshedAt = &t
|
||||||
|
}
|
||||||
ctx := context.Background()
|
ctx := context.Background()
|
||||||
_, err = p.pool.Exec(ctx, `
|
_, err = p.pool.Exec(ctx, `
|
||||||
UPDATE module_cdn_source SET source_kind=$3, url=$4, etag=$5, refresh_interval_sec=$6,
|
UPDATE module_cdn_source SET source_kind=$3, url=$4, prefix_path=$5, etag=$6, refresh_interval_sec=$7,
|
||||||
community_id=NULLIF($7::uuid, '00000000-0000-0000-0000-000000000000'::uuid), updated_at=now()
|
community_id=NULLIF($8::uuid, '00000000-0000-0000-0000-000000000000'::uuid), last_refreshed_at=$9, updated_at=now()
|
||||||
WHERE id=$1 AND module_id=$2`,
|
WHERE id=$1 AND module_id=$2`,
|
||||||
sourceID, moduleID, cur.SourceKind, cur.URL, cur.Etag, nullInt32Ptr(cur.RefreshIntervalSec), uuidOrNilPtr(cur.CommunityID))
|
sourceID, moduleID, cur.SourceKind, cur.URL, cur.PrefixPath, cur.Etag, nullInt32Ptr(cur.RefreshIntervalSec), uuidOrNilPtr(cur.CommunityID), nullTimePtr(cur.LastRefreshedAt))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,67 @@
|
|||||||
|
package repository
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
|
||||||
|
"evobgp/internal/store"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (p *Postgres) fillModuleDohFields(ctx context.Context, m *store.Module) error {
|
||||||
|
if m == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
rows, err := p.pool.Query(ctx, `
|
||||||
|
SELECT doh_profile_id::text
|
||||||
|
FROM module_doh_profile
|
||||||
|
WHERE module_id = $1
|
||||||
|
ORDER BY sort_order, doh_profile_id`, m.ID)
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
defer rows.Close()
|
||||||
|
var ids []string
|
||||||
|
for rows.Next() {
|
||||||
|
var id string
|
||||||
|
if err := rows.Scan(&id); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
ids = append(ids, id)
|
||||||
|
}
|
||||||
|
if err := rows.Err(); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
m.DohProfileIDs = store.NormalizeDohProfileIDList(ids)
|
||||||
|
m.SyncLegacyDohProfileID()
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) setModuleDohProfiles(ctx context.Context, moduleID string, ids []string) error {
|
||||||
|
ids = store.NormalizeDohProfileIDList(ids)
|
||||||
|
if _, err := p.pool.Exec(ctx, `DELETE FROM module_doh_profile WHERE module_id = $1`, moduleID); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
for i, id := range ids {
|
||||||
|
if _, err := p.pool.Exec(ctx, `
|
||||||
|
INSERT INTO module_doh_profile (module_id, doh_profile_id, sort_order)
|
||||||
|
VALUES ($1, $2, $3)`, moduleID, id, i); err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
}
|
||||||
|
var dohArg any
|
||||||
|
if len(ids) > 0 {
|
||||||
|
dohArg = ids[0]
|
||||||
|
}
|
||||||
|
_, err := p.pool.Exec(ctx, `UPDATE module SET doh_profile_id = $2, updated_at = now() WHERE id = $1`, moduleID, dohArg)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
|
func (p *Postgres) moduleDohProfileInUse(ctx context.Context, dohProfileID string) (bool, error) {
|
||||||
|
var n int
|
||||||
|
if err := p.pool.QueryRow(ctx, `
|
||||||
|
SELECT COUNT(*) FROM module_doh_profile mdp
|
||||||
|
JOIN module m ON m.id = mdp.module_id
|
||||||
|
WHERE mdp.doh_profile_id = $1::uuid AND m.deleted_at IS NULL`, dohProfileID).Scan(&n); err != nil {
|
||||||
|
return false, err
|
||||||
|
}
|
||||||
|
return n > 0, nil
|
||||||
|
}
|
||||||
@@ -58,7 +58,7 @@ protocol direct {
|
|||||||
if _, err := tx.Exec(ctx, `INSERT INTO tenant (id, name, slug) VALUES ($1,'Demo','demo')`, tid); err != nil {
|
if _, err := tx.Exec(ctx, `INSERT INTO tenant (id, name, slug) VALUES ($1,'Demo','demo')`, tid); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if _, err := tx.Exec(ctx, `INSERT INTO bgp_community (id, tenant_id, name, kind, value_json) VALUES ($1,$2,'demo-comm','large','{}')`, cid, tid); err != nil {
|
if _, err := tx.Exec(ctx, `INSERT INTO bgp_community (id, tenant_id, community, title, value_json) VALUES ($1,$2,'demo-comm','Demo','{}')`, cid, tid); err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
if _, err := tx.Exec(ctx, `
|
if _, err := tx.Exec(ctx, `
|
||||||
|
|||||||
+39
-34
@@ -1,8 +1,10 @@
|
|||||||
// Package scheduler drives module refresh intervals and enqueues module_refresh jobs on the shared Registry.
|
// Package scheduler drives module refresh intervals and enqueues tenant_refresh jobs on the shared Registry.
|
||||||
package scheduler
|
package scheduler
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"bytes"
|
||||||
"context"
|
"context"
|
||||||
|
"encoding/json"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"log"
|
"log"
|
||||||
@@ -13,6 +15,7 @@ import (
|
|||||||
"evobgp/internal/broker"
|
"evobgp/internal/broker"
|
||||||
"evobgp/internal/config"
|
"evobgp/internal/config"
|
||||||
"evobgp/internal/jobs"
|
"evobgp/internal/jobs"
|
||||||
|
"evobgp/internal/pipeline"
|
||||||
"evobgp/internal/store"
|
"evobgp/internal/store"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -39,9 +42,9 @@ func Run(ctx context.Context, deps *Deps) {
|
|||||||
t := time.NewTicker(30 * time.Second)
|
t := time.NewTicker(30 * time.Second)
|
||||||
defer t.Stop()
|
defer t.Stop()
|
||||||
if deps.Jobs != nil {
|
if deps.Jobs != nil {
|
||||||
log.Printf("evobgp-scheduler: active (in-process enqueue module_refresh)")
|
log.Printf("evobgp-scheduler: active (in-process enqueue tenant_refresh)")
|
||||||
} else {
|
} else {
|
||||||
log.Printf("evobgp-scheduler: active (HTTP POST .../modules/{id}/refresh → %s)", strings.TrimSpace(deps.APIBase))
|
log.Printf("evobgp-scheduler: active (HTTP POST .../tenant/refresh → %s)", strings.TrimSpace(deps.APIBase))
|
||||||
}
|
}
|
||||||
for {
|
for {
|
||||||
select {
|
select {
|
||||||
@@ -60,51 +63,53 @@ func tick(ctx context.Context, deps *Deps) {
|
|||||||
log.Printf("evobgp-scheduler: list tenants: %v", err)
|
log.Printf("evobgp-scheduler: list tenants: %v", err)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
now := time.Now().UTC()
|
||||||
|
tickBucket := now.Unix() / pipeline.SchedulerTickSec
|
||||||
for _, tid := range tenants {
|
for _, tid := range tenants {
|
||||||
|
var due []string
|
||||||
for _, mod := range deps.Store.ListModules(tid) {
|
for _, mod := range deps.Store.ListModules(tid) {
|
||||||
if !mod.Enabled || mod.Type == "IP_RANGES" {
|
if pipeline.ModuleDueForScheduler(mod, now) {
|
||||||
|
due = append(due, mod.ID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if len(due) == 0 {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
key := fmt.Sprintf("sched-tenant-%s-%d", tid, tickBucket)
|
||||||
|
if deps.Jobs != nil {
|
||||||
|
ids := append([]string(nil), due...)
|
||||||
|
_, created, err := deps.Jobs.Enqueue(tid, jobs.KindTenantRefresh, &key, nil, map[string]any{
|
||||||
|
"module_ids": ids,
|
||||||
|
"trigger": "scheduler",
|
||||||
|
})
|
||||||
|
if err != nil {
|
||||||
|
log.Printf("evobgp-scheduler: enqueue tenant %s: %v", tid, err)
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
interval := mod.RefreshIntervalSec
|
if created {
|
||||||
if interval <= 0 {
|
log.Printf("evobgp-scheduler: queued tenant refresh for %d module(s) in tenant %s", len(due), tid)
|
||||||
continue
|
|
||||||
}
|
|
||||||
win := interval
|
|
||||||
if win < 60 {
|
|
||||||
win = 60
|
|
||||||
}
|
|
||||||
bucket := time.Now().Unix() / int64(win)
|
|
||||||
key := fmt.Sprintf("sched-%s-%d", mod.ID, bucket)
|
|
||||||
if deps.Jobs != nil {
|
|
||||||
mid := mod.ID
|
|
||||||
_, created, err := deps.Jobs.Enqueue(tid, jobs.KindModuleRefresh, &key, &mid, map[string]any{
|
|
||||||
"module_id": mod.ID,
|
|
||||||
"trigger": "scheduler",
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
log.Printf("evobgp-scheduler: enqueue module %s: %v", mod.ID, err)
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if created {
|
|
||||||
log.Printf("evobgp-scheduler: queued refresh for module %s (%s)", mod.ID, mod.Type)
|
|
||||||
}
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
if err := postModuleRefresh(ctx, deps, mod.ID, key); err != nil {
|
|
||||||
log.Printf("evobgp-scheduler: http refresh module %s: %v", mod.ID, err)
|
|
||||||
}
|
}
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if err := postTenantRefresh(ctx, deps, due, key); err != nil {
|
||||||
|
log.Printf("evobgp-scheduler: http tenant refresh %s: %v", tid, err)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func postModuleRefresh(ctx context.Context, deps *Deps, moduleID, idempotencyKey string) error {
|
func postTenantRefresh(ctx context.Context, deps *Deps, moduleIDs []string, idempotencyKey string) error {
|
||||||
base := strings.TrimRight(strings.TrimSpace(deps.APIBase), "/")
|
base := strings.TrimRight(strings.TrimSpace(deps.APIBase), "/")
|
||||||
u := base + "/v1/modules/" + moduleID + "/refresh"
|
u := base + "/v1/tenant/refresh"
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, nil)
|
body, err := json.Marshal(map[string]any{"module_ids": moduleIDs})
|
||||||
|
if err != nil {
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
req, err := http.NewRequestWithContext(ctx, http.MethodPost, u, bytes.NewReader(body))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
req.Header.Set("Authorization", "Bearer "+strings.TrimSpace(deps.APIToken))
|
req.Header.Set("Authorization", "Bearer "+strings.TrimSpace(deps.APIToken))
|
||||||
|
req.Header.Set("Content-Type", "application/json")
|
||||||
if idempotencyKey != "" {
|
if idempotencyKey != "" {
|
||||||
req.Header.Set("Idempotency-Key", idempotencyKey)
|
req.Header.Set("Idempotency-Key", idempotencyKey)
|
||||||
}
|
}
|
||||||
|
|||||||
+74
-40
@@ -13,6 +13,7 @@ type Backend interface {
|
|||||||
// ListTenantIDs returns distinct tenant identifiers (for background workers).
|
// ListTenantIDs returns distinct tenant identifiers (for background workers).
|
||||||
ListTenantIDs() ([]string, error)
|
ListTenantIDs() ([]string, error)
|
||||||
|
|
||||||
|
// ListModules returns all modules for a tenant (control plane may paginate in httpapi).
|
||||||
ListModules(tenantID string) []*Module
|
ListModules(tenantID string) []*Module
|
||||||
GetModule(tenantID, moduleID string) (*Module, error)
|
GetModule(tenantID, moduleID string) (*Module, error)
|
||||||
CreateModule(tenantID string, in *Module) (*Module, error)
|
CreateModule(tenantID string, in *Module) (*Module, error)
|
||||||
@@ -53,6 +54,7 @@ type Backend interface {
|
|||||||
UpdateCommunity(tenantID, id string, patch *CommunityPatch) (*Community, error)
|
UpdateCommunity(tenantID, id string, patch *CommunityPatch) (*Community, error)
|
||||||
DeleteCommunity(tenantID, id string) error
|
DeleteCommunity(tenantID, id string) error
|
||||||
|
|
||||||
|
// ListPeers returns all BGP peers for a tenant (control plane may paginate in httpapi).
|
||||||
ListPeers(tenantID string) []*BGPPeer
|
ListPeers(tenantID string) []*BGPPeer
|
||||||
GetPeer(tenantID, id string) (*BGPPeer, error)
|
GetPeer(tenantID, id string) (*BGPPeer, error)
|
||||||
CreatePeer(tenantID string, in *BGPPeer) (*BGPPeer, error)
|
CreatePeer(tenantID string, in *BGPPeer) (*BGPPeer, error)
|
||||||
@@ -72,42 +74,74 @@ type Backend interface {
|
|||||||
// CreateRenderRevision inserts a new config_revision (revID must be unique) with materialized prefixes and preview fragments.
|
// CreateRenderRevision inserts a new config_revision (revID must be unique) with materialized prefixes and preview fragments.
|
||||||
CreateRenderRevision(revisionID, tenantID, moduleID string, parentRevisionID *string, contentHash string, previewFragments map[string]string, prefixes []PrefixRow) error
|
CreateRenderRevision(revisionID, tenantID, moduleID string, parentRevisionID *string, contentHash string, previewFragments map[string]string, prefixes []PrefixRow) error
|
||||||
RevisionDiff(tenantID, aID, bID string) (map[string]any, error)
|
RevisionDiff(tenantID, aID, bID string) (map[string]any, error)
|
||||||
|
PruneRevisionsBefore(tenantID string, cutoff time.Time) (deleted int, err error)
|
||||||
SetLastAppliedRevision(tenantID, speakerID, revisionID string) error
|
SetLastAppliedRevision(tenantID, speakerID, revisionID string) error
|
||||||
PublishRevisionForSpeaker(speakerID, revisionID string) error
|
PublishRevisionForSpeaker(speakerID, revisionID string) error
|
||||||
LatestPublishedRevision(speakerID string) (revisionID string, publishedAt time.Time, err error)
|
LatestPublishedRevision(speakerID string) (revisionID string, publishedAt time.Time, err error)
|
||||||
|
|
||||||
ListGlobalSettings(tenantID string) (map[string]any, error)
|
ListGlobalSettings(tenantID string) (map[string]any, error)
|
||||||
PatchGlobalSettings(tenantID string, patch map[string]any) error
|
PatchGlobalSettings(tenantID string, patch map[string]any) error
|
||||||
|
|
||||||
|
// Module prefix snapshots cache last successful collect per module (pipeline ingest/render).
|
||||||
|
GetModulePrefixSnapshot(tenantID, moduleID string) (*ModulePrefixSnapshot, bool, error)
|
||||||
|
SetModulePrefixSnapshot(tenantID, moduleID, inputHash string, prefixes []PrefixRow) error
|
||||||
|
DeleteModulePrefixSnapshot(tenantID, moduleID string) error
|
||||||
|
|
||||||
|
// ASNPrefixCache stores RIPEstat announced-prefixes per ASN (global TTL cache).
|
||||||
|
GetASNPrefixCache(asn int64) (*ASNPrefixCacheEntry, bool, error)
|
||||||
|
SetASNPrefixCache(asn int64, holder string, prefixes []string) error
|
||||||
|
}
|
||||||
|
|
||||||
|
// ASNPrefixCacheEntry is a cached RIPEstat response for one ASN.
|
||||||
|
type ASNPrefixCacheEntry struct {
|
||||||
|
ASN int64
|
||||||
|
Holder string
|
||||||
|
Prefixes []string
|
||||||
|
FetchedAt time.Time
|
||||||
|
}
|
||||||
|
|
||||||
|
// ModulePrefixSnapshot is the cached materialization for one module between refreshes.
|
||||||
|
type ModulePrefixSnapshot struct {
|
||||||
|
InputHash string
|
||||||
|
CollectedAt time.Time
|
||||||
|
Prefixes []PrefixRow
|
||||||
}
|
}
|
||||||
|
|
||||||
// ModulePatch is a partial update for module.
|
// ModulePatch is a partial update for module.
|
||||||
type ModulePatch struct {
|
type ModulePatch struct {
|
||||||
Name *string `json:"name,omitempty"`
|
Name *string `json:"name,omitempty"`
|
||||||
Enabled *bool `json:"enabled,omitempty"`
|
Enabled *bool `json:"enabled,omitempty"`
|
||||||
Priority *int `json:"priority,omitempty"`
|
Priority *int `json:"priority,omitempty"`
|
||||||
RefreshIntervalSec *int `json:"refresh_interval_sec,omitempty"`
|
RefreshIntervalSec *int `json:"refresh_interval_sec,omitempty"`
|
||||||
CronExpr *string `json:"cron_expr,omitempty"`
|
CronExpr *string `json:"cron_expr,omitempty"`
|
||||||
DefaultCommunityID *string `json:"default_community_id,omitempty"`
|
DefaultCommunityID *string `json:"default_community_id,omitempty"`
|
||||||
DohProfileID *string `json:"doh_profile_id,omitempty"`
|
DohProfileID *string `json:"doh_profile_id,omitempty"`
|
||||||
|
DohProfileIDs *[]string `json:"doh_profile_ids,omitempty"`
|
||||||
|
DohResolverPolicy *string `json:"doh_resolver_policy,omitempty"`
|
||||||
|
LastRefreshedAt *time.Time `json:"-"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// CDNSource is a row under a CDN module.
|
// CDNSource is a row under a CDN module.
|
||||||
type CDNSource struct {
|
type CDNSource struct {
|
||||||
ID string `json:"id,omitempty"`
|
ID string `json:"id,omitempty"`
|
||||||
ModuleID string `json:"module_id,omitempty"`
|
ModuleID string `json:"module_id,omitempty"`
|
||||||
SourceKind string `json:"source_kind"`
|
SourceKind string `json:"source_kind"`
|
||||||
URL string `json:"url"`
|
URL string `json:"url"`
|
||||||
Etag string `json:"etag"`
|
PrefixPath string `json:"prefix_path,omitempty"`
|
||||||
RefreshIntervalSec *int `json:"refresh_interval_sec"`
|
Etag string `json:"etag"`
|
||||||
CommunityID *string `json:"community_id"`
|
RefreshIntervalSec *int `json:"refresh_interval_sec"`
|
||||||
|
CommunityID *string `json:"community_id"`
|
||||||
|
LastRefreshedAt *time.Time `json:"-"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type CDNSourcePatch struct {
|
type CDNSourcePatch struct {
|
||||||
SourceKind *string `json:"source_kind,omitempty"`
|
SourceKind *string `json:"source_kind,omitempty"`
|
||||||
URL *string `json:"url,omitempty"`
|
URL *string `json:"url,omitempty"`
|
||||||
Etag *string `json:"etag,omitempty"`
|
PrefixPath *string `json:"prefix_path,omitempty"`
|
||||||
RefreshIntervalSec *int `json:"refresh_interval_sec,omitempty"`
|
Etag *string `json:"etag,omitempty"`
|
||||||
CommunityID *string `json:"community_id,omitempty"`
|
RefreshIntervalSec *int `json:"refresh_interval_sec,omitempty"`
|
||||||
|
CommunityID *string `json:"community_id,omitempty"`
|
||||||
|
LastRefreshedAt *time.Time `json:"-"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type ASEntry struct {
|
type ASEntry struct {
|
||||||
@@ -131,10 +165,10 @@ func ValidASN(n int64) bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type DomainEntry struct {
|
type DomainEntry struct {
|
||||||
ID string `json:"id,omitempty"`
|
ID string `json:"id,omitempty"`
|
||||||
ModuleID string `json:"module_id,omitempty"`
|
ModuleID string `json:"module_id,omitempty"`
|
||||||
FQDN string `json:"fqdn"`
|
FQDN string `json:"fqdn"`
|
||||||
CommunityID *string `json:"community_id"`
|
CommunityID *string `json:"community_id"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type DomainEntryPatch struct {
|
type DomainEntryPatch struct {
|
||||||
@@ -143,10 +177,10 @@ type DomainEntryPatch struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type IPRangeEntry struct {
|
type IPRangeEntry struct {
|
||||||
ID string `json:"id,omitempty"`
|
ID string `json:"id,omitempty"`
|
||||||
ModuleID string `json:"module_id,omitempty"`
|
ModuleID string `json:"module_id,omitempty"`
|
||||||
Prefix string `json:"prefix"`
|
Prefix string `json:"prefix"`
|
||||||
CommunityID *string `json:"community_id"`
|
CommunityID *string `json:"community_id"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type IPRangePatch struct {
|
type IPRangePatch struct {
|
||||||
@@ -155,12 +189,12 @@ type IPRangePatch struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type DohProfile struct {
|
type DohProfile struct {
|
||||||
ID string `json:"id,omitempty"`
|
ID string `json:"id,omitempty"`
|
||||||
TenantID string `json:"tenant_id,omitempty"`
|
TenantID string `json:"tenant_id,omitempty"`
|
||||||
Name string `json:"name"`
|
Name string `json:"name"`
|
||||||
URL string `json:"url"`
|
URL string `json:"url"`
|
||||||
TimeoutMs *int `json:"timeout_ms"`
|
TimeoutMs *int `json:"timeout_ms"`
|
||||||
SecretRef *string `json:"vault_secret_ref"`
|
SecretRef *string `json:"vault_secret_ref"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type DohProfilePatch struct {
|
type DohProfilePatch struct {
|
||||||
@@ -173,14 +207,14 @@ type DohProfilePatch struct {
|
|||||||
type Community struct {
|
type Community struct {
|
||||||
ID string `json:"id,omitempty"`
|
ID string `json:"id,omitempty"`
|
||||||
TenantID string `json:"tenant_id,omitempty"`
|
TenantID string `json:"tenant_id,omitempty"`
|
||||||
Name string `json:"name"`
|
Community string `json:"community"`
|
||||||
Kind string `json:"kind"`
|
Title string `json:"title"`
|
||||||
ValueJSON string `json:"value_json"`
|
ValueJSON string `json:"value_json"`
|
||||||
}
|
}
|
||||||
|
|
||||||
type CommunityPatch struct {
|
type CommunityPatch struct {
|
||||||
Name *string `json:"name,omitempty"`
|
Community *string `json:"community,omitempty"`
|
||||||
Kind *string `json:"kind,omitempty"`
|
Title *string `json:"title,omitempty"`
|
||||||
ValueJSON *string `json:"value_json,omitempty"`
|
ValueJSON *string `json:"value_json,omitempty"`
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -202,7 +236,7 @@ type SpeakerPatch struct {
|
|||||||
|
|
||||||
// PrefixRow is one materialized prefix for GET /revisions/.../prefixes.
|
// PrefixRow is one materialized prefix for GET /revisions/.../prefixes.
|
||||||
type PrefixRow struct {
|
type PrefixRow struct {
|
||||||
Prefix string
|
Prefix string
|
||||||
CommunityID *string
|
CommunityID *string
|
||||||
Source string
|
Source string
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,110 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import "strings"
|
||||||
|
|
||||||
|
const (
|
||||||
|
DohPolicyPrimaryOnly = "primary_only"
|
||||||
|
DohPolicyFailover = "failover"
|
||||||
|
DohPolicyUnion = "union"
|
||||||
|
)
|
||||||
|
|
||||||
|
// NormalizeDohResolverPolicy returns a supported resolver policy name.
|
||||||
|
func NormalizeDohResolverPolicy(policy string) string {
|
||||||
|
switch strings.TrimSpace(policy) {
|
||||||
|
case DohPolicyFailover, DohPolicyUnion:
|
||||||
|
return strings.TrimSpace(policy)
|
||||||
|
default:
|
||||||
|
return DohPolicyPrimaryOnly
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// NormalizeDohProfileIDList deduplicates profile ids preserving order.
|
||||||
|
func NormalizeDohProfileIDList(ids []string) []string {
|
||||||
|
if len(ids) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
seen := make(map[string]struct{}, len(ids))
|
||||||
|
out := make([]string, 0, len(ids))
|
||||||
|
for _, id := range ids {
|
||||||
|
id = strings.TrimSpace(id)
|
||||||
|
if id == "" {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, ok := seen[id]; ok {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
seen[id] = struct{}{}
|
||||||
|
out = append(out, id)
|
||||||
|
}
|
||||||
|
if len(out) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
return out
|
||||||
|
}
|
||||||
|
|
||||||
|
// EffectiveDohProfileIDs returns ordered DoH profile ids for a module.
|
||||||
|
func (m *Module) EffectiveDohProfileIDs() []string {
|
||||||
|
if m == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
if ids := NormalizeDohProfileIDList(m.DohProfileIDs); len(ids) > 0 {
|
||||||
|
return ids
|
||||||
|
}
|
||||||
|
if m.DohProfileID != nil {
|
||||||
|
if id := strings.TrimSpace(*m.DohProfileID); id != "" {
|
||||||
|
return []string{id}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// SyncLegacyDohProfileID keeps deprecated doh_profile_id aligned with the first profile.
|
||||||
|
func (m *Module) SyncLegacyDohProfileID() {
|
||||||
|
if m == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ids := NormalizeDohProfileIDList(m.DohProfileIDs)
|
||||||
|
if len(ids) == 0 {
|
||||||
|
m.DohProfileID = nil
|
||||||
|
return
|
||||||
|
}
|
||||||
|
first := ids[0]
|
||||||
|
m.DohProfileID = &first
|
||||||
|
}
|
||||||
|
|
||||||
|
// NormalizeModuleDoh fills doh_profile_ids, policy and legacy id from module input.
|
||||||
|
func NormalizeModuleDoh(m *Module) {
|
||||||
|
if m == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
ids := NormalizeDohProfileIDList(m.DohProfileIDs)
|
||||||
|
if len(ids) == 0 && m.DohProfileID != nil {
|
||||||
|
if id := strings.TrimSpace(*m.DohProfileID); id != "" {
|
||||||
|
ids = []string{id}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
m.DohProfileIDs = ids
|
||||||
|
m.DohResolverPolicy = NormalizeDohResolverPolicy(m.DohResolverPolicy)
|
||||||
|
m.SyncLegacyDohProfileID()
|
||||||
|
}
|
||||||
|
|
||||||
|
// ApplyModuleDohPatch merges DoH-related fields from patch into mod.
|
||||||
|
func ApplyModuleDohPatch(mod *Module, patch *ModulePatch) {
|
||||||
|
if mod == nil || patch == nil {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
if patch.DohProfileIDs != nil {
|
||||||
|
mod.DohProfileIDs = NormalizeDohProfileIDList(*patch.DohProfileIDs)
|
||||||
|
} else if patch.DohProfileID != nil {
|
||||||
|
v := strings.TrimSpace(*patch.DohProfileID)
|
||||||
|
if v == "" {
|
||||||
|
mod.DohProfileIDs = nil
|
||||||
|
} else {
|
||||||
|
mod.DohProfileIDs = []string{v}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if patch.DohResolverPolicy != nil {
|
||||||
|
mod.DohResolverPolicy = NormalizeDohResolverPolicy(*patch.DohResolverPolicy)
|
||||||
|
}
|
||||||
|
mod.SyncLegacyDohProfileID()
|
||||||
|
}
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import "testing"
|
||||||
|
|
||||||
|
func TestNormalizeDohProfileIDList(t *testing.T) {
|
||||||
|
got := NormalizeDohProfileIDList([]string{" a ", "b", "a", "", "b"})
|
||||||
|
if len(got) != 2 || got[0] != "a" || got[1] != "b" {
|
||||||
|
t.Fatalf("unexpected: %v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestApplyModuleDohPatch(t *testing.T) {
|
||||||
|
mod := &Module{DohProfileIDs: []string{"one"}, DohResolverPolicy: DohPolicyPrimaryOnly}
|
||||||
|
ids := []string{"ru", "eu"}
|
||||||
|
policy := DohPolicyUnion
|
||||||
|
ApplyModuleDohPatch(mod, &ModulePatch{DohProfileIDs: &ids, DohResolverPolicy: &policy})
|
||||||
|
if len(mod.DohProfileIDs) != 2 || mod.DohResolverPolicy != DohPolicyUnion {
|
||||||
|
t.Fatalf("unexpected module doh fields: %+v", mod)
|
||||||
|
}
|
||||||
|
if mod.DohProfileID == nil || *mod.DohProfileID != "ru" {
|
||||||
|
t.Fatalf("legacy id not synced: %+v", mod.DohProfileID)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestEffectiveDohProfileIDs_LegacyField(t *testing.T) {
|
||||||
|
id := "legacy-id"
|
||||||
|
mod := &Module{DohProfileID: &id}
|
||||||
|
got := mod.EffectiveDohProfileIDs()
|
||||||
|
if len(got) != 1 || got[0] != "legacy-id" {
|
||||||
|
t.Fatalf("unexpected: %v", got)
|
||||||
|
}
|
||||||
|
}
|
||||||
+90
-15
@@ -38,8 +38,10 @@ type Memory struct {
|
|||||||
asEntries map[string]*ASEntry
|
asEntries map[string]*ASEntry
|
||||||
domainEnt map[string]*DomainEntry
|
domainEnt map[string]*DomainEntry
|
||||||
ipRanges map[string]*IPRangeEntry
|
ipRanges map[string]*IPRangeEntry
|
||||||
settings map[string]map[string]any // tenantID -> key -> JSON-compatible value
|
settings map[string]map[string]any // tenantID -> key -> JSON-compatible value
|
||||||
revPrefixes map[string][]PrefixRow
|
revPrefixes map[string][]PrefixRow
|
||||||
|
moduleSnapshots map[string]*moduleSnapshotRec
|
||||||
|
asnPrefixCache map[int64]*ASNPrefixCacheEntry
|
||||||
|
|
||||||
// DemoIDs valid after SeedDemo()
|
// DemoIDs valid after SeedDemo()
|
||||||
demoTenantID string
|
demoTenantID string
|
||||||
@@ -61,17 +63,20 @@ type Tenant struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
type Module struct {
|
type Module struct {
|
||||||
ID string
|
ID string
|
||||||
TenantID string
|
TenantID string
|
||||||
Type string // AS_PREFIXES, CDN_CIDRS, DOMAINS, IP_RANGES
|
Type string // AS_PREFIXES, CDN_CIDRS, DOMAINS, IP_RANGES
|
||||||
Name string
|
Name string
|
||||||
Enabled bool
|
Enabled bool
|
||||||
RefreshIntervalSec int // 0 = unset
|
RefreshIntervalSec int // 0 = unset
|
||||||
CronExpr string // optional cron for scheduler (display / future use)
|
CronExpr string // optional cron for scheduler (display / future use)
|
||||||
Priority int
|
Priority int
|
||||||
DefaultCommunityID *string
|
DefaultCommunityID *string
|
||||||
DohProfileID *string
|
DohProfileID *string // deprecated: first id in DohProfileIDs
|
||||||
DeletedAt *time.Time
|
DohProfileIDs []string
|
||||||
|
DohResolverPolicy string
|
||||||
|
LastRefreshedAt *time.Time
|
||||||
|
DeletedAt *time.Time
|
||||||
}
|
}
|
||||||
|
|
||||||
type Revision struct {
|
type Revision struct {
|
||||||
@@ -125,6 +130,8 @@ func NewMemory() *Memory {
|
|||||||
ipRanges: make(map[string]*IPRangeEntry),
|
ipRanges: make(map[string]*IPRangeEntry),
|
||||||
settings: make(map[string]map[string]any),
|
settings: make(map[string]map[string]any),
|
||||||
revPrefixes: make(map[string][]PrefixRow),
|
revPrefixes: make(map[string][]PrefixRow),
|
||||||
|
moduleSnapshots: make(map[string]*moduleSnapshotRec),
|
||||||
|
asnPrefixCache: make(map[int64]*ASNPrefixCacheEntry),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -235,7 +242,7 @@ protocol direct {
|
|||||||
|
|
||||||
// Demo materialized prefixes for /revisions/{id}/prefixes
|
// Demo materialized prefixes for /revisions/{id}/prefixes
|
||||||
cid := uuid.NewString()
|
cid := uuid.NewString()
|
||||||
m.communities[cid] = &Community{ID: cid, TenantID: tid, Name: "demo-comm", Kind: "large", ValueJSON: "{}"}
|
m.communities[cid] = &Community{ID: cid, TenantID: tid, Community: "demo-comm", Title: "Demo", ValueJSON: "{}"}
|
||||||
m.revPrefixes[rid] = []PrefixRow{
|
m.revPrefixes[rid] = []PrefixRow{
|
||||||
{Prefix: "203.0.113.0/24", CommunityID: &cid, Source: "demo"},
|
{Prefix: "203.0.113.0/24", CommunityID: &cid, Source: "demo"},
|
||||||
{Prefix: "2001:db8::/32", CommunityID: &cid, Source: "demo"},
|
{Prefix: "2001:db8::/32", CommunityID: &cid, Source: "demo"},
|
||||||
@@ -320,13 +327,22 @@ func (m *Memory) CreateRenderRevision(revisionID, tenantID, moduleID string, par
|
|||||||
frag[k] = v
|
frag[k] = v
|
||||||
}
|
}
|
||||||
parent := parentRevisionID
|
parent := parentRevisionID
|
||||||
|
createdAt := time.Now().UTC()
|
||||||
|
for _, r := range m.revisions {
|
||||||
|
if r == nil || r.TenantID != tenantID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !r.CreatedAt.Before(createdAt) {
|
||||||
|
createdAt = r.CreatedAt.Add(time.Microsecond)
|
||||||
|
}
|
||||||
|
}
|
||||||
m.revisions[revisionID] = &Revision{
|
m.revisions[revisionID] = &Revision{
|
||||||
ID: revisionID,
|
ID: revisionID,
|
||||||
TenantID: tenantID,
|
TenantID: tenantID,
|
||||||
ModuleID: moduleID,
|
ModuleID: moduleID,
|
||||||
ContentHash: contentHash,
|
ContentHash: contentHash,
|
||||||
ParentRevisionID: parent,
|
ParentRevisionID: parent,
|
||||||
CreatedAt: time.Now().UTC(),
|
CreatedAt: createdAt,
|
||||||
MaterializedPrefixCount: len(prefixes),
|
MaterializedPrefixCount: len(prefixes),
|
||||||
PreviewFragments: frag,
|
PreviewFragments: frag,
|
||||||
}
|
}
|
||||||
@@ -565,6 +581,62 @@ func (m *Memory) RevisionDiff(tenantID, aID, bID string) (map[string]any, error)
|
|||||||
}, nil
|
}, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func (m *Memory) PruneRevisionsBefore(tenantID string, cutoff time.Time) (int, error) {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
|
||||||
|
var newest *Revision
|
||||||
|
for _, rev := range m.revisions {
|
||||||
|
if rev.TenantID != tenantID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if newest == nil || rev.CreatedAt.After(newest.CreatedAt) {
|
||||||
|
newest = rev
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if newest == nil {
|
||||||
|
return 0, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
protected := map[string]struct{}{
|
||||||
|
newest.ID: {},
|
||||||
|
}
|
||||||
|
for _, sp := range m.speakers {
|
||||||
|
if sp == nil || sp.TenantID != tenantID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if sp.LastAppliedRevisionID != nil && strings.TrimSpace(*sp.LastAppliedRevisionID) != "" {
|
||||||
|
protected[strings.TrimSpace(*sp.LastAppliedRevisionID)] = struct{}{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for speakerID, info := range m.publishedRevision {
|
||||||
|
sp, ok := m.speakers[speakerID]
|
||||||
|
if !ok || sp == nil || sp.TenantID != tenantID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if strings.TrimSpace(info.RevisionID) != "" {
|
||||||
|
protected[strings.TrimSpace(info.RevisionID)] = struct{}{}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
deleted := 0
|
||||||
|
for id, rev := range m.revisions {
|
||||||
|
if rev == nil || rev.TenantID != tenantID {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if !rev.CreatedAt.Before(cutoff) {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
if _, keep := protected[id]; keep {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
delete(m.revisions, id)
|
||||||
|
delete(m.revPrefixes, id)
|
||||||
|
deleted++
|
||||||
|
}
|
||||||
|
return deleted, nil
|
||||||
|
}
|
||||||
|
|
||||||
func (m *Memory) getRevisionLocked(tenantID, revisionID string) (*Revision, error) {
|
func (m *Memory) getRevisionLocked(tenantID, revisionID string) (*Revision, error) {
|
||||||
rev, ok := m.revisions[revisionID]
|
rev, ok := m.revisions[revisionID]
|
||||||
if !ok {
|
if !ok {
|
||||||
@@ -594,6 +666,9 @@ func (m *Memory) ListRevisions(tenantID, moduleID string, cursor string, limit i
|
|||||||
all = append(all, r)
|
all = append(all, r)
|
||||||
}
|
}
|
||||||
sort.Slice(all, func(i, j int) bool {
|
sort.Slice(all, func(i, j int) bool {
|
||||||
|
if all[i].CreatedAt.Equal(all[j].CreatedAt) {
|
||||||
|
return all[i].ID > all[j].ID
|
||||||
|
}
|
||||||
return all[i].CreatedAt.After(all[j].CreatedAt)
|
return all[i].CreatedAt.After(all[j].CreatedAt)
|
||||||
})
|
})
|
||||||
off := 0
|
off := 0
|
||||||
|
|||||||
@@ -0,0 +1,40 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (m *Memory) GetASNPrefixCache(asn int64) (*ASNPrefixCacheEntry, bool, error) {
|
||||||
|
m.mu.RLock()
|
||||||
|
defer m.mu.RUnlock()
|
||||||
|
if m.asnPrefixCache == nil {
|
||||||
|
return nil, false, nil
|
||||||
|
}
|
||||||
|
e, ok := m.asnPrefixCache[asn]
|
||||||
|
if !ok || e == nil {
|
||||||
|
return nil, false, nil
|
||||||
|
}
|
||||||
|
pfx := append([]string(nil), e.Prefixes...)
|
||||||
|
return &ASNPrefixCacheEntry{
|
||||||
|
ASN: asn,
|
||||||
|
Holder: e.Holder,
|
||||||
|
Prefixes: pfx,
|
||||||
|
FetchedAt: e.FetchedAt,
|
||||||
|
}, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Memory) SetASNPrefixCache(asn int64, holder string, prefixes []string) error {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
if m.asnPrefixCache == nil {
|
||||||
|
m.asnPrefixCache = make(map[int64]*ASNPrefixCacheEntry)
|
||||||
|
}
|
||||||
|
cp := append([]string(nil), prefixes...)
|
||||||
|
m.asnPrefixCache[asn] = &ASNPrefixCacheEntry{
|
||||||
|
ASN: asn,
|
||||||
|
Holder: holder,
|
||||||
|
Prefixes: cp,
|
||||||
|
FetchedAt: time.Now().UTC(),
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -22,17 +22,20 @@ func (m *Memory) CreateModule(tenantID string, in *Module) (*Module, error) {
|
|||||||
}
|
}
|
||||||
id := uuid.NewString()
|
id := uuid.NewString()
|
||||||
mod := &Module{
|
mod := &Module{
|
||||||
ID: id,
|
ID: id,
|
||||||
TenantID: tenantID,
|
TenantID: tenantID,
|
||||||
Type: in.Type,
|
Type: in.Type,
|
||||||
Name: strings.TrimSpace(in.Name),
|
Name: strings.TrimSpace(in.Name),
|
||||||
Enabled: in.Enabled,
|
Enabled: in.Enabled,
|
||||||
Priority: in.Priority,
|
Priority: in.Priority,
|
||||||
RefreshIntervalSec: in.RefreshIntervalSec,
|
RefreshIntervalSec: in.RefreshIntervalSec,
|
||||||
CronExpr: in.CronExpr,
|
CronExpr: in.CronExpr,
|
||||||
DefaultCommunityID: in.DefaultCommunityID,
|
DefaultCommunityID: in.DefaultCommunityID,
|
||||||
DohProfileID: in.DohProfileID,
|
DohProfileIDs: append([]string(nil), in.DohProfileIDs...),
|
||||||
|
DohResolverPolicy: in.DohResolverPolicy,
|
||||||
|
LastRefreshedAt: in.LastRefreshedAt,
|
||||||
}
|
}
|
||||||
|
NormalizeModuleDoh(mod)
|
||||||
m.modules[id] = mod
|
m.modules[id] = mod
|
||||||
return mod, nil
|
return mod, nil
|
||||||
}
|
}
|
||||||
@@ -70,13 +73,10 @@ func (m *Memory) UpdateModule(tenantID, moduleID string, patch *ModulePatch) (*M
|
|||||||
mod.DefaultCommunityID = &v
|
mod.DefaultCommunityID = &v
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if patch.DohProfileID != nil {
|
ApplyModuleDohPatch(mod, patch)
|
||||||
v := strings.TrimSpace(*patch.DohProfileID)
|
if patch.LastRefreshedAt != nil {
|
||||||
if v == "" {
|
t := patch.LastRefreshedAt.UTC()
|
||||||
mod.DohProfileID = nil
|
mod.LastRefreshedAt = &t
|
||||||
} else {
|
|
||||||
mod.DohProfileID = &v
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
return mod, nil
|
return mod, nil
|
||||||
}
|
}
|
||||||
@@ -84,6 +84,9 @@ func (m *Memory) UpdateModule(tenantID, moduleID string, patch *ModulePatch) (*M
|
|||||||
func (m *Memory) SoftDeleteModule(tenantID, moduleID string) error {
|
func (m *Memory) SoftDeleteModule(tenantID, moduleID string) error {
|
||||||
m.mu.Lock()
|
m.mu.Lock()
|
||||||
defer m.mu.Unlock()
|
defer m.mu.Unlock()
|
||||||
|
if m.moduleSnapshots != nil {
|
||||||
|
delete(m.moduleSnapshots, moduleSnapshotKey(tenantID, moduleID))
|
||||||
|
}
|
||||||
mod, ok := m.modules[moduleID]
|
mod, ok := m.modules[moduleID]
|
||||||
if !ok || mod.TenantID != tenantID {
|
if !ok || mod.TenantID != tenantID {
|
||||||
return ErrNotFound
|
return ErrNotFound
|
||||||
@@ -139,9 +142,11 @@ func (m *Memory) CreateCDNSource(tenantID, moduleID string, in *CDNSource) (*CDN
|
|||||||
ModuleID: moduleID,
|
ModuleID: moduleID,
|
||||||
SourceKind: in.SourceKind,
|
SourceKind: in.SourceKind,
|
||||||
URL: strings.TrimSpace(in.URL),
|
URL: strings.TrimSpace(in.URL),
|
||||||
|
PrefixPath: strings.TrimSpace(in.PrefixPath),
|
||||||
Etag: in.Etag,
|
Etag: in.Etag,
|
||||||
RefreshIntervalSec: in.RefreshIntervalSec,
|
RefreshIntervalSec: in.RefreshIntervalSec,
|
||||||
CommunityID: in.CommunityID,
|
CommunityID: in.CommunityID,
|
||||||
|
LastRefreshedAt: in.LastRefreshedAt,
|
||||||
}
|
}
|
||||||
m.cdnSources[id] = s
|
m.cdnSources[id] = s
|
||||||
return s, nil
|
return s, nil
|
||||||
@@ -166,6 +171,9 @@ func (m *Memory) UpdateCDNSource(tenantID, moduleID, sourceID string, patch *CDN
|
|||||||
if patch.URL != nil {
|
if patch.URL != nil {
|
||||||
s.URL = strings.TrimSpace(*patch.URL)
|
s.URL = strings.TrimSpace(*patch.URL)
|
||||||
}
|
}
|
||||||
|
if patch.PrefixPath != nil {
|
||||||
|
s.PrefixPath = strings.TrimSpace(*patch.PrefixPath)
|
||||||
|
}
|
||||||
if patch.Etag != nil {
|
if patch.Etag != nil {
|
||||||
s.Etag = *patch.Etag
|
s.Etag = *patch.Etag
|
||||||
}
|
}
|
||||||
@@ -180,6 +188,10 @@ func (m *Memory) UpdateCDNSource(tenantID, moduleID, sourceID string, patch *CDN
|
|||||||
s.CommunityID = &v
|
s.CommunityID = &v
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
if patch.LastRefreshedAt != nil {
|
||||||
|
t := patch.LastRefreshedAt.UTC()
|
||||||
|
s.LastRefreshedAt = &t
|
||||||
|
}
|
||||||
return s, nil
|
return s, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -538,8 +550,13 @@ func (m *Memory) DeleteDohProfile(tenantID, id string) error {
|
|||||||
return ErrNotFound
|
return ErrNotFound
|
||||||
}
|
}
|
||||||
for _, mod := range m.modules {
|
for _, mod := range m.modules {
|
||||||
if mod.DohProfileID != nil && *mod.DohProfileID == id {
|
if mod == nil || mod.DeletedAt != nil {
|
||||||
return ErrInvalidInput
|
continue
|
||||||
|
}
|
||||||
|
for _, pid := range mod.EffectiveDohProfileIDs() {
|
||||||
|
if pid == id {
|
||||||
|
return ErrInvalidInput
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
delete(m.dohProfiles, id)
|
delete(m.dohProfiles, id)
|
||||||
@@ -569,7 +586,7 @@ func (m *Memory) GetCommunity(tenantID, id string) (*Community, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
func (m *Memory) CreateCommunity(tenantID string, in *Community) (*Community, error) {
|
func (m *Memory) CreateCommunity(tenantID string, in *Community) (*Community, error) {
|
||||||
if in == nil || strings.TrimSpace(in.Kind) == "" {
|
if in == nil || strings.TrimSpace(in.Community) == "" {
|
||||||
return nil, ErrInvalidInput
|
return nil, ErrInvalidInput
|
||||||
}
|
}
|
||||||
m.mu.Lock()
|
m.mu.Lock()
|
||||||
@@ -582,7 +599,7 @@ func (m *Memory) CreateCommunity(tenantID string, in *Community) (*Community, er
|
|||||||
if strings.TrimSpace(vj) == "" {
|
if strings.TrimSpace(vj) == "" {
|
||||||
vj = "{}"
|
vj = "{}"
|
||||||
}
|
}
|
||||||
c := &Community{ID: id, TenantID: tenantID, Name: in.Name, Kind: strings.TrimSpace(in.Kind), ValueJSON: vj}
|
c := &Community{ID: id, TenantID: tenantID, Community: strings.TrimSpace(in.Community), Title: strings.TrimSpace(in.Title), ValueJSON: vj}
|
||||||
m.communities[id] = c
|
m.communities[id] = c
|
||||||
return c, nil
|
return c, nil
|
||||||
}
|
}
|
||||||
@@ -597,11 +614,11 @@ func (m *Memory) UpdateCommunity(tenantID, id string, patch *CommunityPatch) (*C
|
|||||||
if !ok || c.TenantID != tenantID {
|
if !ok || c.TenantID != tenantID {
|
||||||
return nil, ErrNotFound
|
return nil, ErrNotFound
|
||||||
}
|
}
|
||||||
if patch.Name != nil {
|
if patch.Community != nil {
|
||||||
c.Name = *patch.Name
|
c.Community = strings.TrimSpace(*patch.Community)
|
||||||
}
|
}
|
||||||
if patch.Kind != nil {
|
if patch.Title != nil {
|
||||||
c.Kind = strings.TrimSpace(*patch.Kind)
|
c.Title = strings.TrimSpace(*patch.Title)
|
||||||
}
|
}
|
||||||
if patch.ValueJSON != nil {
|
if patch.ValueJSON != nil {
|
||||||
c.ValueJSON = *patch.ValueJSON
|
c.ValueJSON = *patch.ValueJSON
|
||||||
@@ -654,7 +671,7 @@ func (m *Memory) CreatePeer(tenantID string, in *BGPPeer) (*BGPPeer, error) {
|
|||||||
p := &BGPPeer{
|
p := &BGPPeer{
|
||||||
ID: id, TenantID: tenantID, SpeakerID: in.SpeakerID, Name: in.Name,
|
ID: id, TenantID: tenantID, SpeakerID: in.SpeakerID, Name: in.Name,
|
||||||
Neighbor: neighbor, RemoteASN: in.RemoteASN,
|
Neighbor: neighbor, RemoteASN: in.RemoteASN,
|
||||||
Enabled: EffectivePeerEnabledOnCreate(in.Enabled, in.SessionState),
|
Enabled: EffectivePeerEnabledOnCreate(in.Enabled, in.SessionState),
|
||||||
SessionState: in.SessionState, PoliciesJSON: in.PoliciesJSON,
|
SessionState: in.SessionState, PoliciesJSON: in.PoliciesJSON,
|
||||||
}
|
}
|
||||||
m.peers[id] = p
|
m.peers[id] = p
|
||||||
|
|||||||
@@ -0,0 +1,62 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import (
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
func (m *Memory) GetModulePrefixSnapshot(tenantID, moduleID string) (*ModulePrefixSnapshot, bool, error) {
|
||||||
|
m.mu.RLock()
|
||||||
|
defer m.mu.RUnlock()
|
||||||
|
key := moduleSnapshotKey(tenantID, moduleID)
|
||||||
|
snap, ok := m.moduleSnapshots[key]
|
||||||
|
if !ok || snap == nil {
|
||||||
|
return nil, false, nil
|
||||||
|
}
|
||||||
|
cp := make([]PrefixRow, len(snap.Prefixes))
|
||||||
|
copy(cp, snap.Prefixes)
|
||||||
|
return &ModulePrefixSnapshot{
|
||||||
|
InputHash: snap.InputHash,
|
||||||
|
CollectedAt: snap.CollectedAt,
|
||||||
|
Prefixes: cp,
|
||||||
|
}, true, nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Memory) SetModulePrefixSnapshot(tenantID, moduleID, inputHash string, prefixes []PrefixRow) error {
|
||||||
|
if strings.TrimSpace(tenantID) == "" || strings.TrimSpace(moduleID) == "" || strings.TrimSpace(inputHash) == "" {
|
||||||
|
return ErrInvalidInput
|
||||||
|
}
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
if m.moduleSnapshots == nil {
|
||||||
|
m.moduleSnapshots = make(map[string]*moduleSnapshotRec)
|
||||||
|
}
|
||||||
|
cp := make([]PrefixRow, len(prefixes))
|
||||||
|
copy(cp, prefixes)
|
||||||
|
key := moduleSnapshotKey(tenantID, moduleID)
|
||||||
|
m.moduleSnapshots[key] = &moduleSnapshotRec{
|
||||||
|
InputHash: inputHash,
|
||||||
|
CollectedAt: time.Now().UTC(),
|
||||||
|
Prefixes: cp,
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
func (m *Memory) DeleteModulePrefixSnapshot(tenantID, moduleID string) error {
|
||||||
|
m.mu.Lock()
|
||||||
|
defer m.mu.Unlock()
|
||||||
|
if m.moduleSnapshots != nil {
|
||||||
|
delete(m.moduleSnapshots, moduleSnapshotKey(tenantID, moduleID))
|
||||||
|
}
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
type moduleSnapshotRec struct {
|
||||||
|
InputHash string
|
||||||
|
CollectedAt time.Time
|
||||||
|
Prefixes []PrefixRow
|
||||||
|
}
|
||||||
|
|
||||||
|
func moduleSnapshotKey(tenantID, moduleID string) string {
|
||||||
|
return strings.TrimSpace(tenantID) + "\x00" + strings.TrimSpace(moduleID)
|
||||||
|
}
|
||||||
@@ -0,0 +1,29 @@
|
|||||||
|
package store
|
||||||
|
|
||||||
|
import "strconv"
|
||||||
|
|
||||||
|
// PaginateOffset returns a page from a pre-sorted slice using numeric string cursors (same scheme as ListRevisions).
|
||||||
|
func PaginateOffset[T any](all []T, cursor string, limit int) (page []T, nextCursor string, hasMore bool) {
|
||||||
|
if limit <= 0 {
|
||||||
|
limit = 50
|
||||||
|
}
|
||||||
|
off := 0
|
||||||
|
if cursor != "" {
|
||||||
|
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
|
||||||
|
off = n
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if off > len(all) {
|
||||||
|
off = len(all)
|
||||||
|
}
|
||||||
|
end := off + limit
|
||||||
|
if end > len(all) {
|
||||||
|
end = len(all)
|
||||||
|
}
|
||||||
|
page = all[off:end]
|
||||||
|
if end < len(all) {
|
||||||
|
nextCursor = strconv.Itoa(end)
|
||||||
|
hasMore = true
|
||||||
|
}
|
||||||
|
return page, nextCursor, hasMore
|
||||||
|
}
|
||||||
Loaded 100 of 397 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user