Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ec43591a99 | ||
|
|
0e5fb065e2 | ||
|
|
5188b2aff2 | ||
|
|
cd1fd2c9d3 |
@@ -38,8 +38,49 @@ jobs:
|
||||
node .ci/scripts/compute-release.mjs
|
||||
test -f .ci/release/release-manifest.json
|
||||
|
||||
backend-test:
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Node 22
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: 22
|
||||
|
||||
- name: Start PostgreSQL 18
|
||||
shell: bash
|
||||
run: |
|
||||
set -euo pipefail
|
||||
docker run -d --name mm-pg \
|
||||
-e POSTGRES_USER=mmapp \
|
||||
-e POSTGRES_PASSWORD=mmapp \
|
||||
-e POSTGRES_DB=mmapp \
|
||||
-p 5432:5432 \
|
||||
postgres:18-alpine
|
||||
for i in $(seq 1 40); do
|
||||
if docker exec mm-pg pg_isready -U mmapp -d mmapp; then
|
||||
exit 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
echo "PostgreSQL не поднялся"
|
||||
exit 1
|
||||
|
||||
- name: Install and test backend
|
||||
shell: bash
|
||||
env:
|
||||
DATABASE_URL: postgres://mmapp:mmapp@127.0.0.1:5432/mmapp
|
||||
run: |
|
||||
set -euo pipefail
|
||||
npm ci
|
||||
npm run test --prefix backend
|
||||
|
||||
backend-image:
|
||||
needs: prepare-release
|
||||
needs:
|
||||
- prepare-release
|
||||
- backend-test
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Checkout
|
||||
|
||||
@@ -19,7 +19,7 @@
|
||||
| Компонент | Путь | Стек | Порт (runtime) | Docker-образ |
|
||||
|-----------|------|------|----------------|--------------|
|
||||
| Frontend | корень (`app/`, `components/`, `lib/`, …) | Next.js 16.2.4 (App Router), React 19 | 3000 | `…-frontend` |
|
||||
| Backend | `backend/` | Fastify 5, TypeScript ESM, SQLite, Drizzle | 8000 | `…-backend` |
|
||||
| Backend | `backend/` | Fastify 5, TypeScript ESM, PostgreSQL 18, Drizzle | 8000 | `…-backend` |
|
||||
| Контракты | `packages/contracts/` | Zod 4, `@mmapp/contracts` | — | встраиваются в frontend/backend |
|
||||
| Updater | `deploy/updater/` | bash, `docker:27-cli`, curl, jq | — | `…-updater` |
|
||||
| Деплой | `deploy/docker-compose.yml` | Docker Compose | — | — |
|
||||
@@ -43,11 +43,11 @@ flowchart TB
|
||||
Browser["Браузер"]
|
||||
NextDev["Next.js :3000"]
|
||||
FastifyDev["Fastify :8000"]
|
||||
SqliteDev["SQLite файл"]
|
||||
PostgresDev["PostgreSQL 18"]
|
||||
Contracts["@mmapp/contracts"]
|
||||
Browser --> NextDev
|
||||
Browser -->|"fetch /api, CORS"| FastifyDev
|
||||
FastifyDev --> SqliteDev
|
||||
FastifyDev --> PostgresDev
|
||||
Contracts --> NextDev
|
||||
Contracts --> FastifyDev
|
||||
end
|
||||
@@ -60,7 +60,8 @@ flowchart TB
|
||||
subgraph prod [Прод-сервер]
|
||||
FE["mmapp-frontend :3000"]
|
||||
BE["mmapp-backend :8000"]
|
||||
DBVol["volume backend-data /app/data"]
|
||||
PG["mmapp-postgres"]
|
||||
DBVol["volume sqlite ETL /app/data"]
|
||||
UPD["mmapp-updater"]
|
||||
Sock["/var/run/docker.sock"]
|
||||
Reg --> FE
|
||||
@@ -69,6 +70,7 @@ flowchart TB
|
||||
UPD --> Sock
|
||||
UPD --> FE
|
||||
UPD --> BE
|
||||
BE --> PG
|
||||
BE --> DBVol
|
||||
BrowserProd["Клиент"] --> FE
|
||||
BrowserProd -->|"live + backendUrl"| BE
|
||||
@@ -117,7 +119,7 @@ sequenceDiagram
|
||||
```
|
||||
|
||||
- Триггер workflow: `push` в ветку `main`, ручной `workflow_dispatch`.
|
||||
- Параллельные jobs: `backend-image`, `frontend-image`, `updater-image`; опционально `notify-webhook` после backend и frontend, если задан секрет `DEPLOY_WEBHOOK_URL`.
|
||||
- Параллельные jobs: `backend-test` (PostgreSQL 18), `backend-image`, `frontend-image`, `updater-image`; опционально `notify-webhook` после backend и frontend, если задан секрет `DEPLOY_WEBHOOK_URL`.
|
||||
- Теги на каждый успешный push: **`:latest`** и **`:<commit-sha>`**; платформа **linux/amd64**.
|
||||
- На сервере образы подтягиваются вручную (`docker compose pull`) и/или через **updater** (сравнение digest у тега из `targets.json`). Webhook CI **не заменяет** updater.
|
||||
|
||||
@@ -125,7 +127,8 @@ sequenceDiagram
|
||||
|
||||
| Зависимость | Реализация |
|
||||
|-------------|------------|
|
||||
| SQLite | Не отдельный контейнер. Файл `mikrotik.db` в томе `backend-data` → `/app/data` (`DATABASE_PATH=/app/data/mikrotik.db` в образе backend). |
|
||||
| PostgreSQL | Контейнер `mmapp-postgres` (`postgres:18-alpine`). `DATABASE_URL=postgres://mmapp:…@postgres:5432/mmapp`. |
|
||||
| SQLite (ETL) | Файл `mikrotik.db` на томе `/app/data` (`DATABASE_PATH=/app/data/mikrotik.db`). При первом старте, если PG пустой, backend сам импортирует данные и ставит маркер. Повторный старт не копирует заново. |
|
||||
| Docker socket | Только у контейнера updater: `/var/run/docker.sock` — доступ к Docker API хоста (управление контейнерами, pull). |
|
||||
|
||||
## Локальная разработка
|
||||
@@ -134,7 +137,8 @@ sequenceDiagram
|
||||
|
||||
- **Node.js 22** (как в `Dockerfile.frontend` и `backend/Dockerfile`).
|
||||
- **npm** с workspaces; установка из корня: `npm ci` или `npm install`.
|
||||
- Для нативной сборки `better-sqlite3` на Linux может понадобиться toolchain (`python3`, `make`, `g++`); в Docker-образе backend они уже ставятся.
|
||||
- Для нативной сборки `better-sqlite3` на Linux может понадобиться toolchain (`python3`, `make`, `g++`); в Docker-образе backend они уже ставятся (нужен только для одноразового boot-ETL из `mikrotik.db`).
|
||||
- Локально нужен PostgreSQL 18 (`deploy/docker-compose.postgres.yml` или `DATABASE_URL`).
|
||||
- Backend Docker-образ ставит только workspaces `backend` + `contracts` (без корневых Next/React deps); в production логи — JSON без `pino-pretty`.
|
||||
|
||||
### Запуск
|
||||
@@ -161,7 +165,8 @@ Backend — скопировать `backend/.env.example` в `backend/.env`:
|
||||
|
||||
| Переменная | По умолчанию | Назначение |
|
||||
|------------|--------------|------------|
|
||||
| `DATABASE_PATH` | `./mikrotik.db` | путь к файлу SQLite |
|
||||
| `DATABASE_URL` | `postgres://mmapp:mmapp@127.0.0.1:5432/mmapp` | подключение к PostgreSQL |
|
||||
| `DATABASE_PATH` | `./mikrotik.db` | SQLite только для boot-ETL, если PG пустой |
|
||||
| `PORT` | `8000` | порт Fastify |
|
||||
| `CORS_ORIGIN` | `http://localhost:3000` | origin фронтенда для CORS |
|
||||
|
||||
@@ -179,12 +184,10 @@ Frontend — в репозитории нет корневого `.env.example`.
|
||||
npm run build -w @mmapp/contracts
|
||||
```
|
||||
|
||||
После изменения схем Drizzle:
|
||||
Схема PostgreSQL накатывается при старте backend из [`backend/drizzle/0000_postgresql.sql`](backend/drizzle/0000_postgresql.sql). Опциональный ETL:
|
||||
|
||||
```bash
|
||||
npm --prefix backend run db:generate
|
||||
npm --prefix backend run db:migrate
|
||||
npm --prefix backend run db:studio
|
||||
npm --prefix backend run db:migrate-from-sqlite
|
||||
```
|
||||
|
||||
## CI/CD (Gitea Actions)
|
||||
@@ -281,7 +284,8 @@ SSO auth-portal: [`docs/integrate-auth-portal.md`](docs/integrate-auth-portal.md
|
||||
|
||||
| Сервис | `container_name` | Образ (пример) | Порты host:container | Тома | `restart` |
|
||||
|--------|------------------|----------------|----------------------|------|-----------|
|
||||
| backend | `mmapp-backend` | `git.shx.one/denozord/mikrotikmanager-backend:latest` | `8000:8000` | `backend-data` → `/app/data` | `unless-stopped` |
|
||||
| postgres | `mmapp-postgres` | `postgres:18-alpine` | нет (внутренняя сеть) | `mmapp-pgdata` | `unless-stopped` |
|
||||
| backend | `mmapp-backend` | `git.shx.one/denozord/mikrotikmanager-backend:latest` | `8000:8000` | sqlite ETL → `/app/data` | `unless-stopped` |
|
||||
| frontend | `mmapp-frontend` | `git.shx.one/denozord/mikrotikmanager-frontend:latest` | `3000:3000` | — | `unless-stopped` |
|
||||
| updater | `mmapp-updater` | `git.shx.one/denozord/mikrotikmanager-updater:latest` | не публикуются | docker.sock, `updater-state` → `/state`, `targets.json` → `/etc/updater/targets.json:ro` | `unless-stopped` |
|
||||
|
||||
@@ -297,11 +301,14 @@ SSO auth-portal: [`docs/integrate-auth-portal.md`](docs/integrate-auth-portal.md
|
||||
|
||||
### Переменные окружения (прод)
|
||||
|
||||
**Backend** (в образе заданы `NODE_ENV=production`, `PORT=8000`, `DATABASE_PATH=/app/data/mikrotik.db`; в compose обычно переопределяют только CORS):
|
||||
**Backend** (в образе заданы `NODE_ENV=production`, `PORT=8000`; в compose задают `DATABASE_URL`, `DATABASE_PATH` для ETL и CORS):
|
||||
|
||||
| Переменная | Источник в compose | Назначение |
|
||||
|------------|-------------------|------------|
|
||||
| `CORS_ORIGIN` | `${CORS_ORIGIN:-http://localhost:3000}` | origin UI, с которого браузер вызывает API |
|
||||
| `DATABASE_URL` | `postgres://mmapp:${POSTGRES_PASSWORD}@postgres:5432/mmapp` | PostgreSQL |
|
||||
| `DATABASE_PATH` | `/app/data/mikrotik.db` | SQLite на томе для одноразового ETL |
|
||||
| `POSTGRES_PASSWORD` | обязателен | пароль пользователя `mmapp` |
|
||||
|
||||
**Frontend** (в образе: `PORT=3000`, `HOSTNAME=0.0.0.0`). URL API в образ **не** зашит — задаётся в браузере (режим live + URL бэкенда) вместе с `CORS_ORIGIN` на backend.
|
||||
|
||||
@@ -312,7 +319,7 @@ SSO auth-portal: [`docs/integrate-auth-portal.md`](docs/integrate-auth-portal.md
|
||||
| `TARGETS_FILE` | `/etc/updater/targets.json` | список целей |
|
||||
| `STATE_FILE` | `/state/updater-state.json` | digest и ошибки |
|
||||
| `POLL_INTERVAL_SECONDS` | `300` | пауза между циклами опроса |
|
||||
| `HEALTH_TIMEOUT_SECONDS` | `120` | ожидание HTTP health |
|
||||
| `HEALTH_TIMEOUT_SECONDS` | `600` | ожидание HTTP health (первый boot с ETL может занять минуты) |
|
||||
| `STOP_TIMEOUT_SECONDS` | `30` | `docker stop -t` |
|
||||
| `REGISTRY` | `git.shx.one` | registry для `docker login` |
|
||||
| `REGISTRY_USERNAME` | пусто | логин (если заданы оба с паролем) |
|
||||
@@ -516,18 +523,20 @@ docker compose up -d --force-recreate
|
||||
### Откат
|
||||
|
||||
- **Автоматически:** updater при failed health после обновления — контейнер на `previous_digest`.
|
||||
- **Вручную:** остановить контейнер, запустить образ с нужным тегом или digest из registry, сохранив те же volume и labels. Откат **схемы SQLite** updater не выполняет — нужен отдельный backup тома `backend-data` / файла `mikrotik.db`.
|
||||
- **Вручную:** остановить контейнер, запустить образ с нужным тегом или digest из registry, сохранив те же volume и labels. Откат **схемы PostgreSQL** updater не выполняет — нужен `pg_dump` / restore или том `mmapp-pgdata`.
|
||||
|
||||
### Резервное копирование SQLite
|
||||
### Резервное копирование PostgreSQL
|
||||
|
||||
Том `backend-data` (или `mmapp-backend-data` при явном `docker volume create`). Пример остановки backend для консистентной копии:
|
||||
Том `mmapp-pgdata` и (до cleanup) sqlite `mikrotik.db` на `/app/data`. Из UI: Настройки → «Скачать бэкап» (`pg_dump -Fc`). С хоста:
|
||||
|
||||
```bash
|
||||
docker stop mmapp-backend
|
||||
docker run --rm -v backend-data:/data -v $(pwd):/backup alpine tar czf /backup/mikrotik-db-backup.tar.gz -C /data .
|
||||
docker start mmapp-backend
|
||||
docker exec mmapp-postgres pg_dump -Fc -U mmapp mmapp > mikrotik-manager.dump
|
||||
```
|
||||
|
||||
Восстановление: UI «Восстановить из файла» или `pg_restore --clean --if-exists -d …`.
|
||||
|
||||
Том sqlite после успешного ETL и smoke **не удаляйте сразу** — оставьте как fallback, затем уберите `mikrotik.db`, когда маркер `data_migration.sqlite_imported_at` подтверждён.
|
||||
|
||||
### Отладка
|
||||
|
||||
| Действие | Команда |
|
||||
|
||||
@@ -2846,7 +2846,7 @@ export default function AlertsPage() {
|
||||
</CollapsibleTrigger>
|
||||
<CollapsibleContent className="px-3 pb-3 text-[11px] text-muted-foreground leading-relaxed space-y-2 border-t border-border/60 pt-2">
|
||||
<p>
|
||||
Движок читает последние сэмплы из SQLite (uptime ресурсы, REST-пинг, пробы, GRE/BGP). Условие
|
||||
Движок читает последние сэмплы из PostgreSQL (uptime ресурсы, REST-пинг, пробы, GRE/BGP). Условие
|
||||
срабатывает только если цепочка состояний совпадает с текстом правила.
|
||||
</p>
|
||||
<ul className="list-disc pl-4 space-y-1.5">
|
||||
|
||||
@@ -220,7 +220,7 @@ function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
|
||||
</Alert>
|
||||
) : null}
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Запись в SQLite на{" "}
|
||||
Запись в PostgreSQL на{" "}
|
||||
<span className="font-mono tabular-nums">{new Date(g.sampledAt).toLocaleString("ru-RU")}</span>
|
||||
{" "}— строки GRE и BGP для движка оповещений.
|
||||
</p>
|
||||
@@ -347,7 +347,7 @@ function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Снимок на{" "}
|
||||
<span className="font-mono tabular-nums">{new Date(a.sampledAt).toLocaleString("ru-RU")}</span>
|
||||
. Данные для правил берутся из SQLite после джоб сбора (трафик, uptime, REST, GRE+BGP и т.д.).
|
||||
. Данные для правил берутся из PostgreSQL после джоб сбора (трафик, uptime, REST, GRE+BGP и т.д.).
|
||||
</p>
|
||||
<dl className="grid grid-cols-2 sm:grid-cols-4 gap-3 text-xs">
|
||||
<div>
|
||||
@@ -1215,7 +1215,7 @@ export default function DataCollectionPage() {
|
||||
|
||||
<OpsPanel
|
||||
title="Журнал прогонов"
|
||||
description="SQLite `scheduler_runs` — до 80 записей; раскройте строку для полей и текста ошибки."
|
||||
description="Таблица `scheduler_runs` — до 80 записей; раскройте строку для полей и текста ошибки."
|
||||
headerRight={
|
||||
<div className="flex items-center gap-2 shrink-0">
|
||||
<label className="text-xs text-muted-foreground whitespace-nowrap" htmlFor="run-filter">
|
||||
|
||||
+317
-78
@@ -16,7 +16,10 @@ import {
|
||||
buildGreMapEdges,
|
||||
buildServerResourceMap,
|
||||
buildWanJhEdges,
|
||||
clipSegmentCircleToRect,
|
||||
computeNetworkMapLayout,
|
||||
MAP_SERVICE_NODE_H,
|
||||
MAP_SERVICE_NODE_W,
|
||||
NETWORK_MAP_H,
|
||||
NETWORK_MAP_LAYOUT_REVISION,
|
||||
NETWORK_MAP_PIPELINE_Y,
|
||||
@@ -52,7 +55,7 @@ import {
|
||||
matchNetflowForWan,
|
||||
type MatchedNetflowHop,
|
||||
} from "@/lib/map-netflow-hops"
|
||||
import type { FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge } from "@mmapp/contracts/traffic-flow"
|
||||
import type { FlowMapHop, FlowMapHopsDto, FlowMapService, FlowMapServiceEdge, FlowMapServicePath } from "@mmapp/contracts/traffic-flow"
|
||||
import { ServiceBrandIcon } from "@/components/network-map/service-brand-icon"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
@@ -289,10 +292,106 @@ const MOCK_MAP_SERVICE_EDGES: FlowMapServiceEdge[] = [
|
||||
{ fromId: "srv3", toId: "svc:aws", bytes: 9_000_000, bps: 3_600_000, bpsFwd: 2_700_000, bpsRev: 900_000, clientName: "Bob", clients: [{ id: "u2", name: "Bob" }] },
|
||||
]
|
||||
|
||||
const MOCK_MAP_SERVICE_PATHS: FlowMapServicePath[] = [
|
||||
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "svc:google", bytes: 14_000_000, bps: 5_600_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:google", bytes: 8_000_000, bps: 3_200_000 },
|
||||
{ clientId: "u1", clientName: "Alice", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv2", enName: "mt-spb-edge-01", serviceId: "svc:cloudflare", bytes: 9_000_000, bps: 3_600_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:cloudflare", bytes: 5_000_000, bps: 2_000_000 },
|
||||
{ clientId: "u2", clientName: "Bob", viaId: "srv1", viaName: "mt-msk-core-01", enId: "srv3", enName: "mt-fra-edge-01", serviceId: "svc:aws", bytes: 9_000_000, bps: 3_600_000 },
|
||||
]
|
||||
|
||||
function servicePathKey(p: Pick<FlowMapServicePath, "clientId" | "viaId" | "enId" | "serviceId">): string {
|
||||
return `${p.clientId}|${p.viaId}|${p.enId}|${p.serviceId}`
|
||||
}
|
||||
|
||||
function greMatchesPath(e: GreMapEdge, h: { viaId: string; enId: string }): boolean {
|
||||
const ids = new Set([e.fromServer.id, e.toServer.id])
|
||||
return ids.has(h.viaId) && ids.has(h.enId)
|
||||
}
|
||||
|
||||
function serviceSharePct(share: number): string {
|
||||
return `${Math.round(share * 100)}%`
|
||||
}
|
||||
|
||||
function exitNodeIdsFromGre(greEdges: GreMapEdge[]): string[] {
|
||||
const ids = new Set<string>()
|
||||
for (const g of greEdges) {
|
||||
if (g.fromServer.type === "exit-node") ids.add(g.fromServer.id)
|
||||
if (g.toServer.type === "exit-node") ids.add(g.toServer.id)
|
||||
}
|
||||
return [...ids]
|
||||
}
|
||||
|
||||
function remapServiceEdgeFromId(
|
||||
fromId: string,
|
||||
servers: Server[],
|
||||
greEdges: GreMapEdge[],
|
||||
soleEnId: string | null,
|
||||
): string | null {
|
||||
const srv = servers.find((s) => s.id === fromId)
|
||||
if (srv?.type === "exit-node") return fromId
|
||||
for (const g of greEdges) {
|
||||
if (g.fromServer.id === fromId && g.toServer.type === "exit-node") return g.toServer.id
|
||||
if (g.toServer.id === fromId && g.fromServer.type === "exit-node") return g.fromServer.id
|
||||
}
|
||||
return soleEnId
|
||||
}
|
||||
|
||||
/** API-рёбра на EN; если hop нет — синтез от единственного EN на карте (GRE / каталог). */
|
||||
function drawableServiceEdges(
|
||||
services: FlowMapService[],
|
||||
edges: FlowMapServiceEdge[],
|
||||
servers: Server[],
|
||||
greEdges: GreMapEdge[],
|
||||
nodePosById: Record<string, { x: number; y: number }>,
|
||||
): FlowMapServiceEdge[] {
|
||||
const keep = new Set(services.map((s) => s.id))
|
||||
const greEn = exitNodeIdsFromGre(greEdges)
|
||||
const catalogEn = servers.filter((s) => s.type === "exit-node").map((s) => s.id)
|
||||
const enPool = greEn.length > 0 ? greEn : catalogEn
|
||||
const soleEnId = enPool.length === 1 ? enPool[0]! : null
|
||||
const merged = new Map<string, FlowMapServiceEdge>()
|
||||
|
||||
function bump(e: FlowMapServiceEdge) {
|
||||
const key = `${e.fromId}|${e.toId}`
|
||||
const prev = merged.get(key)
|
||||
if (!prev) {
|
||||
merged.set(key, { ...e })
|
||||
return
|
||||
}
|
||||
merged.set(key, {
|
||||
...prev,
|
||||
bytes: prev.bytes + e.bytes,
|
||||
bps: prev.bps + e.bps,
|
||||
bpsFwd: prev.bpsFwd + e.bpsFwd,
|
||||
bpsRev: prev.bpsRev + e.bpsRev,
|
||||
})
|
||||
}
|
||||
|
||||
for (const e of edges) {
|
||||
if (!keep.has(e.toId)) continue
|
||||
const fromId = remapServiceEdgeFromId(e.fromId, servers, greEdges, soleEnId)
|
||||
if (!fromId || !nodePosById[fromId]) continue
|
||||
bump({ ...e, fromId })
|
||||
}
|
||||
|
||||
const covered = new Set([...merged.values()].map((e) => e.toId))
|
||||
if (soleEnId && nodePosById[soleEnId]) {
|
||||
for (const svc of services) {
|
||||
if (covered.has(svc.id)) continue
|
||||
bump({
|
||||
fromId: soleEnId,
|
||||
toId: svc.id,
|
||||
bytes: svc.bytes,
|
||||
bps: svc.bps,
|
||||
bpsFwd: svc.bps,
|
||||
bpsRev: 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
return [...merged.values()]
|
||||
}
|
||||
|
||||
// ─── Helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
function pingColor(ms: number | null) {
|
||||
@@ -651,8 +750,8 @@ function ServiceNode({
|
||||
onClick: () => void
|
||||
onMouseDown: (e: React.MouseEvent) => void
|
||||
}) {
|
||||
const bw = 86
|
||||
const bh = 58
|
||||
const bw = MAP_SERVICE_NODE_W
|
||||
const bh = MAP_SERVICE_NODE_H
|
||||
return (
|
||||
<g
|
||||
transform={`translate(${x},${y})`}
|
||||
@@ -698,6 +797,59 @@ function ServiceNode({
|
||||
)
|
||||
}
|
||||
|
||||
function ServicePathList({
|
||||
paths,
|
||||
servers,
|
||||
services,
|
||||
highlight,
|
||||
viaMode,
|
||||
onToggle,
|
||||
}: {
|
||||
paths: FlowMapServicePath[]
|
||||
servers: Server[]
|
||||
services: FlowMapService[]
|
||||
highlight: { viaId: string; enId: string; serviceId: string } | null
|
||||
viaMode: "via" | "service"
|
||||
onToggle: (p: FlowMapServicePath) => void
|
||||
}) {
|
||||
if (paths.length === 0) {
|
||||
return <p className="text-xs text-muted-foreground">—</p>
|
||||
}
|
||||
return (
|
||||
<div className="flex flex-col gap-1">
|
||||
{paths.map((p) => {
|
||||
const rowKey = servicePathKey(p)
|
||||
const via = servers.find((s) => s.id === p.viaId)
|
||||
const viaLabel = via?.site || p.viaName
|
||||
const svc = services.find((s) => s.id === p.serviceId)
|
||||
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId)
|
||||
const active = Boolean(
|
||||
highlight
|
||||
&& highlight.viaId === p.viaId
|
||||
&& highlight.enId === p.enId
|
||||
&& highlight.serviceId === p.serviceId,
|
||||
)
|
||||
return (
|
||||
<button
|
||||
key={rowKey}
|
||||
type="button"
|
||||
onClick={() => onToggle(p)}
|
||||
className={cn(
|
||||
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
|
||||
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
|
||||
)}
|
||||
>
|
||||
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums shrink-0">
|
||||
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
|
||||
</span>
|
||||
</button>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function WanSatNode({ x, y, wan, color, active, isSel, isDragged, onSelect, onMouseDown }: {
|
||||
x: number; y: number
|
||||
wan: { name: string; isp: string; maxDl: number; maxUl: number }
|
||||
@@ -938,6 +1090,7 @@ export default function NetworkMapPage() {
|
||||
const [mapHops, setMapHops] = useState<FlowMapHop[]>([])
|
||||
const [mapServices, setMapServices] = useState<FlowMapService[]>([])
|
||||
const [mapServiceEdges, setMapServiceEdges] = useState<FlowMapServiceEdge[]>([])
|
||||
const [mapServicePaths, setMapServicePaths] = useState<FlowMapServicePath[]>([])
|
||||
const [mapSharePct, setMapSharePct] = useState(5)
|
||||
/** FQDN из GRE outer → IPv4 (ответ POST /api/network/resolve-hosts), для матчинга с WAN. */
|
||||
const [greResolvedIpv4ByHost, setGreResolvedIpv4ByHost] = useState<Record<string, string>>({})
|
||||
@@ -1033,6 +1186,7 @@ export default function NetworkMapPage() {
|
||||
setMapHops([])
|
||||
setMapServices(MOCK_MAP_SERVICES)
|
||||
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
|
||||
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
|
||||
setMapSharePct(5)
|
||||
setDataError(null)
|
||||
})
|
||||
@@ -1062,6 +1216,7 @@ export default function NetworkMapPage() {
|
||||
// ── Interaction ─────────────────────────────────────────────────────────────
|
||||
const [selected, setSelected] = useState<Server | null>(null)
|
||||
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
|
||||
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
|
||||
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
|
||||
const [hoveredId, setHoveredId] = useState<string | null>(null)
|
||||
|
||||
@@ -1110,6 +1265,7 @@ export default function NetworkMapPage() {
|
||||
setMapHops([])
|
||||
setMapServices(MOCK_MAP_SERVICES)
|
||||
setMapServiceEdges(MOCK_MAP_SERVICE_EDGES)
|
||||
setMapServicePaths(MOCK_MAP_SERVICE_PATHS)
|
||||
setMapSharePct(5)
|
||||
})
|
||||
return
|
||||
@@ -1119,6 +1275,7 @@ export default function NetworkMapPage() {
|
||||
setMapHops([])
|
||||
setMapServices([])
|
||||
setMapServiceEdges([])
|
||||
setMapServicePaths([])
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -1133,6 +1290,7 @@ export default function NetworkMapPage() {
|
||||
setMapHops(res.hops ?? [])
|
||||
setMapServices(res.services ?? [])
|
||||
setMapServiceEdges(res.serviceEdges ?? [])
|
||||
setMapServicePaths(res.servicePaths ?? [])
|
||||
if (res.mapServiceMinSharePct != null) setMapSharePct(res.mapServiceMinSharePct)
|
||||
})
|
||||
.catch((err: unknown) => {
|
||||
@@ -1337,9 +1495,9 @@ export default function NetworkMapPage() {
|
||||
}, [homeRouters, wanJhEdges, mapHops, showNetflow])
|
||||
|
||||
const visibleMapServices = showServices ? mapServices : []
|
||||
const visibleServiceEdges = showServices ? mapServiceEdges.filter((e) =>
|
||||
visibleMapServices.some((s) => s.id === e.toId),
|
||||
) : []
|
||||
const visibleServiceEdges = showServices
|
||||
? drawableServiceEdges(visibleMapServices, mapServiceEdges, mapServers, greEdges, nodePosById)
|
||||
: []
|
||||
|
||||
const nodes = mapServers
|
||||
.map((s) => ({ ...s, ...nodePosById[s.id]! }))
|
||||
@@ -1440,7 +1598,13 @@ export default function NetworkMapPage() {
|
||||
useEffect(() => {
|
||||
function onKey(e: KeyboardEvent) {
|
||||
if (e.target instanceof HTMLInputElement || e.target instanceof HTMLTextAreaElement) return
|
||||
if (e.key === "Escape") { setSelected(null); setSelWanIdx(null); setSelectedGreEdge(null); setSelectedService(null) }
|
||||
if (e.key === "Escape") {
|
||||
setSelected(null)
|
||||
setSelWanIdx(null)
|
||||
setSelectedGreEdge(null)
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
}
|
||||
if (e.key === "=" || e.key === "+") applyZoomCenter(1.25)
|
||||
if (e.key === "-") applyZoomCenter(1 / 1.25)
|
||||
if (e.key === "0" || e.key.toLowerCase() === "f") fitView()
|
||||
@@ -1531,7 +1695,13 @@ export default function NetworkMapPage() {
|
||||
const moved = dragRef.current?.moved ?? false
|
||||
dragRef.current = null
|
||||
setIsDragging(false)
|
||||
if (!moved) { setSelected(null); setSelWanIdx(null); setSelectedGreEdge(null); setSelectedService(null) }
|
||||
if (!moved) {
|
||||
setSelected(null)
|
||||
setSelWanIdx(null)
|
||||
setSelectedGreEdge(null)
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
}
|
||||
}
|
||||
|
||||
// ── Node drag start ──────────────────────────────────────────────────────
|
||||
@@ -1568,6 +1738,7 @@ export default function NetworkMapPage() {
|
||||
function selectServer(s: Server) {
|
||||
setSelectedGreEdge(null)
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
setSelected(prev => prev?.id === s.id ? null : s)
|
||||
setSelWanIdx(null)
|
||||
setHoveredId(null)
|
||||
@@ -1576,15 +1747,28 @@ export default function NetworkMapPage() {
|
||||
setSelectedGreEdge(null)
|
||||
setSelected(null)
|
||||
setSelWanIdx(null)
|
||||
setHighlightedPath(null)
|
||||
setSelectedService((prev: FlowMapService | null) => prev?.id === svc.id ? null : svc)
|
||||
}
|
||||
function selectWan(s: Server, wanIdx: number) {
|
||||
setSelectedGreEdge(null)
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
setSelected(s)
|
||||
setSelWanIdx(prev => prev === wanIdx && selected?.id === s.id ? null : wanIdx)
|
||||
}
|
||||
|
||||
function togglePathHighlight(p: FlowMapServicePath) {
|
||||
setHighlightedPath((prev) => (
|
||||
prev
|
||||
&& prev.viaId === p.viaId
|
||||
&& prev.enId === p.enId
|
||||
&& prev.serviceId === p.serviceId
|
||||
? null
|
||||
: { viaId: p.viaId, enId: p.enId, serviceId: p.serviceId }
|
||||
))
|
||||
}
|
||||
|
||||
function openWanJhSpeedDetail(ev: React.MouseEvent<SVGElement>, edge: WanJhEdge) {
|
||||
ev.stopPropagation()
|
||||
const home = mapServers.find((s) => s.id === edge.homeId)
|
||||
@@ -1879,13 +2063,23 @@ export default function NetworkMapPage() {
|
||||
setSelectedGreEdge(e)
|
||||
setSelected(null)
|
||||
setSelectedService(null)
|
||||
setHighlightedPath(null)
|
||||
setSelWanIdx(null)
|
||||
}
|
||||
return (
|
||||
<g key={edgeId} opacity={dimmed ? 0.05 : 1} style={{ transition: "opacity 0.3s" }}>
|
||||
<g
|
||||
key={edgeId}
|
||||
opacity={dimmed ? 0.05 : (highlightedPath && !greMatchesPath(e, highlightedPath) ? 0.14 : 1)}
|
||||
style={{ transition: "opacity 0.3s" }}
|
||||
>
|
||||
<line
|
||||
x1={e.from.x} y1={e.from.y} x2={e.to.x} y2={e.to.y}
|
||||
stroke={ts.stroke} strokeWidth={hopHasRate(flowHop) ? 2.6 : 1.5}
|
||||
stroke={ts.stroke}
|
||||
strokeWidth={
|
||||
highlightedPath && greMatchesPath(e, highlightedPath)
|
||||
? 3.4
|
||||
: hopHasRate(flowHop) ? 2.6 : 1.5
|
||||
}
|
||||
strokeDasharray={e.tunnel.ipsec ? "7 4" : "none"}
|
||||
opacity={ts.opacity}
|
||||
/>
|
||||
@@ -2019,55 +2213,6 @@ export default function NetworkMapPage() {
|
||||
)
|
||||
})}
|
||||
|
||||
{/* ── EN → destination services ── */}
|
||||
{visibleServiceEdges.map((edge) => {
|
||||
const from = nodeById[edge.fromId] ?? nodePosById[edge.fromId]
|
||||
const to = servicePosById[edge.toId]
|
||||
if (!from || !to) return null
|
||||
const hop: MatchedNetflowHop = {
|
||||
bytes: edge.bytes,
|
||||
bps: edge.bps,
|
||||
bpsFwd: edge.bpsFwd,
|
||||
bpsRev: edge.bpsRev,
|
||||
}
|
||||
const { mx, my } = edgeBadgePosition(from.x, from.y, to.x, to.y, 0.55, 16)
|
||||
const hl = selectedService?.id === edge.toId || selected?.id === edge.fromId
|
||||
const svc = visibleMapServices.find((s) => s.id === edge.toId)
|
||||
const enName = mapServers.find((s) => s.id === edge.fromId)?.name ?? edge.fromId
|
||||
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
|
||||
const pathTitle = `${clientLabel} → ${enName} → ${svc?.label ?? edge.toId}`
|
||||
return (
|
||||
<g key={`${edge.fromId}|${edge.toId}`} opacity={hl ? 1 : 0.72} style={{ transition: "opacity 0.3s" }}>
|
||||
<title>{pathTitle}</title>
|
||||
<line
|
||||
x1={from.x} y1={from.y} x2={to.x} y2={to.y}
|
||||
stroke="#22d3ee"
|
||||
strokeWidth={hopHasRate(hop) ? 2.2 : 1.3}
|
||||
strokeDasharray="5 5"
|
||||
opacity="0.7"
|
||||
/>
|
||||
{showAnimDots && hopHasRate(hop) && (
|
||||
<circle r="3" fill="#67e8f9" opacity="0.85" pointerEvents="none">
|
||||
<animateMotion dur="2.6s" repeatCount="indefinite"
|
||||
path={`M ${from.x} ${from.y} L ${to.x} ${to.y}`} />
|
||||
</circle>
|
||||
)}
|
||||
{hopHasRate(hop) && (
|
||||
<NetflowRateBadge
|
||||
mx={mx}
|
||||
my={my}
|
||||
hop={hop}
|
||||
onOpen={(ev) => {
|
||||
ev.stopPropagation()
|
||||
const svc = visibleMapServices.find((s) => s.id === edge.toId)
|
||||
if (svc) selectService(svc)
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</g>
|
||||
)
|
||||
})}
|
||||
|
||||
{/* ── Server nodes ── */}
|
||||
{nodes.map(n => (
|
||||
<ServerNode
|
||||
@@ -2136,6 +2281,90 @@ export default function NetworkMapPage() {
|
||||
)
|
||||
})}
|
||||
|
||||
{/* ── EN → destination services (поверх узлов, чтобы пунктир не прятался) ── */}
|
||||
{visibleServiceEdges.map((edge) => {
|
||||
const from = nodeById[edge.fromId] ?? nodePosById[edge.fromId]
|
||||
const to = servicePosById[edge.toId]
|
||||
if (!from || !to) return null
|
||||
const hop: MatchedNetflowHop = {
|
||||
bytes: edge.bytes,
|
||||
bps: edge.bps,
|
||||
bpsFwd: edge.bpsFwd,
|
||||
bpsRev: edge.bpsRev,
|
||||
}
|
||||
const fromR = "type" in from && from.type
|
||||
? TYPE_STYLE[from.type].r
|
||||
: TYPE_STYLE["exit-node"].r
|
||||
const clipped = clipSegmentCircleToRect(
|
||||
from.x,
|
||||
from.y,
|
||||
fromR,
|
||||
to.x,
|
||||
to.y,
|
||||
MAP_SERVICE_NODE_W / 2,
|
||||
MAP_SERVICE_NODE_H / 2,
|
||||
)
|
||||
const { mx, my } = edgeBadgePosition(clipped.x1, clipped.y1, clipped.x2, clipped.y2, 0.55, 16)
|
||||
const pathHit = Boolean(
|
||||
highlightedPath
|
||||
&& highlightedPath.enId === edge.fromId
|
||||
&& highlightedPath.serviceId === edge.toId,
|
||||
)
|
||||
const pathDim = Boolean(highlightedPath) && !pathHit
|
||||
const hl = pathHit || (!highlightedPath && (selectedService?.id === edge.toId || selected?.id === edge.fromId))
|
||||
const svc = visibleMapServices.find((s) => s.id === edge.toId)
|
||||
const enName = mapServers.find((s) => s.id === edge.fromId)?.name ?? edge.fromId
|
||||
const clientLabel = (edge.clients?.map((c) => c.name).filter(Boolean).join(", ") || edge.clientName || "—")
|
||||
const pathTitle = `${clientLabel} → ${enName} → ${svc?.label ?? edge.toId}`
|
||||
return (
|
||||
<g
|
||||
key={`${edge.fromId}|${edge.toId}`}
|
||||
opacity={pathDim ? 0.12 : hl ? 1 : 0.72}
|
||||
style={{ transition: "opacity 0.3s" }}
|
||||
>
|
||||
<title>{pathTitle}</title>
|
||||
<line
|
||||
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
|
||||
stroke="#22d3ee"
|
||||
strokeWidth={pathHit ? 3.2 : hopHasRate(hop) ? 2.4 : 1.4}
|
||||
strokeDasharray="6 5"
|
||||
opacity="0.85"
|
||||
pointerEvents="none"
|
||||
/>
|
||||
<line
|
||||
x1={clipped.x1} y1={clipped.y1} x2={clipped.x2} y2={clipped.y2}
|
||||
stroke="#00000000"
|
||||
strokeWidth={14}
|
||||
strokeLinecap="round"
|
||||
style={{ cursor: "pointer" }}
|
||||
onPointerDown={(ev) => { ev.stopPropagation() }}
|
||||
onClick={(ev) => {
|
||||
ev.stopPropagation()
|
||||
if (svc) selectService(svc)
|
||||
}}
|
||||
/>
|
||||
{showAnimDots && hopHasRate(hop) && (
|
||||
<circle r="3" fill="#67e8f9" opacity="0.85" pointerEvents="none">
|
||||
<animateMotion dur="2.6s" repeatCount="indefinite"
|
||||
path={`M ${clipped.x1} ${clipped.y1} L ${clipped.x2} ${clipped.y2}`} />
|
||||
</circle>
|
||||
)}
|
||||
{hopHasRate(hop) && (
|
||||
<NetflowRateBadge
|
||||
mx={mx}
|
||||
my={my}
|
||||
hop={hop}
|
||||
onOpen={(ev) => {
|
||||
ev.stopPropagation()
|
||||
const hit = visibleMapServices.find((s) => s.id === edge.toId)
|
||||
if (hit) selectService(hit)
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</g>
|
||||
)
|
||||
})}
|
||||
|
||||
{/* ── Hover tooltip ── */}
|
||||
{hoveredNode && !isDragging && (
|
||||
<SvgTooltip n={hoveredNode} />
|
||||
@@ -2528,25 +2757,18 @@ export default function NetworkMapPage() {
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Клиенты</p>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{(() => {
|
||||
const names = new Map<string, string>()
|
||||
for (const e of visibleServiceEdges.filter((x) => x.toId === selectedService.id)) {
|
||||
if (e.clients?.length) {
|
||||
for (const c of e.clients) names.set(c.id, c.name)
|
||||
} else if (e.clientName) {
|
||||
names.set(e.clientId ?? e.clientName, e.clientName)
|
||||
}
|
||||
}
|
||||
if (names.size === 0) {
|
||||
return <p className="text-xs text-muted-foreground">—</p>
|
||||
}
|
||||
return [...names.values()].map((name) => (
|
||||
<p key={name} className="text-xs font-mono truncate">{name}</p>
|
||||
))
|
||||
})()}
|
||||
</div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.serviceId === selectedService.id)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
@@ -2775,6 +2997,23 @@ export default function NetworkMapPage() {
|
||||
</div>
|
||||
)}
|
||||
|
||||
{(selected.type === "jump-host" || selected.type === "exit-node") && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.viaId === selected.id || p.enId === selected.id)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="service"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Resources */}
|
||||
{(() => {
|
||||
const res = srvResMap[selected.id]
|
||||
|
||||
@@ -88,7 +88,7 @@ function DatabaseRestoreConfirm({
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Восстановить базу приложения?</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
Файл: {filename}. Текущие данные SQLite на бекенде будут полностью заменены.
|
||||
Файл: {filename}. Текущие данные PostgreSQL на бекенде будут полностью заменены.
|
||||
Рекомендуется сначала скачать актуальный бэкап.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
@@ -435,7 +435,7 @@ export default function SettingsPage() {
|
||||
|
||||
<OpsPanel
|
||||
title="База данных приложения"
|
||||
description="Резервная копия SQLite бекенда и сброс таблиц NetFlow. На время операции планировщик и коллектор IPFIX приостанавливаются. Preview: https://reui.io/preview/base/settings-16"
|
||||
description="Резервная копия PostgreSQL (pg_dump custom) и сброс таблиц NetFlow. На время операции планировщик и коллектор IPFIX приостанавливаются. Preview: https://reui.io/preview/base/settings-16"
|
||||
contentClassName="divide-y px-5"
|
||||
>
|
||||
{!systemDbAvailable && (
|
||||
@@ -446,7 +446,7 @@ export default function SettingsPage() {
|
||||
)}
|
||||
<SettingRow
|
||||
label="Скачать бэкап"
|
||||
description="Консистентная копия файла mikrotik.db"
|
||||
description="Консистентный дамп PostgreSQL (pg_dump -Fc)"
|
||||
>
|
||||
<Button
|
||||
variant="outline"
|
||||
@@ -461,7 +461,7 @@ export default function SettingsPage() {
|
||||
</SettingRow>
|
||||
<SettingRow
|
||||
label="Восстановить из файла"
|
||||
description="Полностью заменяет текущую базу SQLite"
|
||||
description="Полностью заменяет текущую базу PostgreSQL (pg_restore)"
|
||||
>
|
||||
<div className="flex flex-col items-end gap-2">
|
||||
<Button
|
||||
@@ -481,7 +481,7 @@ export default function SettingsPage() {
|
||||
</SettingRow>
|
||||
<SettingRow
|
||||
label="Сбросить данные NetFlow"
|
||||
description="Удалит сессии и агрегаты из SQLite, затем VACUUM. Ключи WG и пиры не трогает"
|
||||
description="Удалит сессии и агрегаты NetFlow в PostgreSQL. Ключи WG и пиры не трогает"
|
||||
>
|
||||
<Button
|
||||
variant="destructive"
|
||||
@@ -563,7 +563,7 @@ export default function SettingsPage() {
|
||||
>
|
||||
access.md
|
||||
</a>
|
||||
. URL и ключ хранятся в SQLite на сервере бекенда (
|
||||
. URL и ключ хранятся в PostgreSQL на сервере бекенда (
|
||||
<code className="font-mono bg-muted px-1 rounded">evobgp_settings</code>
|
||||
). Каталог —{" "}
|
||||
<code className="font-mono bg-muted px-1 rounded">GET /v1/router-lists/catalog</code> через прокси.
|
||||
@@ -937,8 +937,8 @@ export default function SettingsPage() {
|
||||
open={dbRestoreDialogOpen}
|
||||
onOpenChange={setDbRestoreDialogOpen}
|
||||
title="Восстановление базы данных"
|
||||
description="Выберите файл SQLite (.db) — текущая база будет полностью заменена"
|
||||
accept=".db,.sqlite,.sqlite3,application/octet-stream"
|
||||
description="Выберите файл pg_dump custom (.dump) — текущая база будет полностью заменена"
|
||||
accept=".dump,.backup,application/octet-stream"
|
||||
onImport={async (files) => {
|
||||
const file = files[0]
|
||||
if (!file) return
|
||||
|
||||
@@ -1,5 +1,9 @@
|
||||
# Path to SQLite database file
|
||||
# PostgreSQL connection string
|
||||
DATABASE_URL=postgres://mmapp:mmapp@127.0.0.1:5432/mmapp
|
||||
|
||||
# SQLite file used only for one-shot boot ETL (empty PG + existing mikrotik.db)
|
||||
DATABASE_PATH=./mikrotik.db
|
||||
# SQLITE_IMPORT_FULL_HISTORY=true
|
||||
|
||||
# Port for the Fastify server
|
||||
PORT=8000
|
||||
|
||||
+9
-1
@@ -47,8 +47,16 @@ COPY --from=build /app/node_modules ./node_modules
|
||||
COPY --from=build /app/packages/contracts ./packages/contracts
|
||||
COPY --from=build /app/backend/dist ./backend/dist
|
||||
COPY --from=build /app/backend/package.json ./backend/package.json
|
||||
COPY --from=build /app/backend/drizzle ./backend/drizzle
|
||||
# Nested install from lockfile (dotenv etc.) — ESM resolves from /app/backend/dist → ../node_modules
|
||||
COPY --from=build /app/backend/node_modules ./backend/node_modules
|
||||
RUN mkdir -p /app/data
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends gnupg wget ca-certificates \
|
||||
&& wget -qO- https://www.postgresql.org/media/keys/ACCC4CF8.asc | gpg --dearmor -o /usr/share/keyrings/pgdg.gpg \
|
||||
&& echo "deb [signed-by=/usr/share/keyrings/pgdg.gpg] http://apt.postgresql.org/pub/repos/apt bookworm-pgdg main" > /etc/apt/sources.list.d/pgdg.list \
|
||||
&& apt-get update \
|
||||
&& apt-get install -y --no-install-recommends postgresql-client-18 \
|
||||
&& rm -rf /var/lib/apt/lists/* \
|
||||
&& mkdir -p /app/data
|
||||
EXPOSE 8000
|
||||
CMD ["node", "backend/dist/index.js"]
|
||||
|
||||
@@ -6,9 +6,9 @@ config()
|
||||
export default defineConfig({
|
||||
schema: "./src/db/schema.ts",
|
||||
out: "./drizzle",
|
||||
dialect: "sqlite",
|
||||
dialect: "postgresql",
|
||||
dbCredentials: {
|
||||
url: process.env.DATABASE_PATH ?? "./mikrotik.db",
|
||||
url: process.env.DATABASE_URL ?? "postgres://mmapp:mmapp@127.0.0.1:5432/mmapp",
|
||||
},
|
||||
verbose: true,
|
||||
strict: true,
|
||||
|
||||
@@ -1,36 +0,0 @@
|
||||
CREATE TABLE `server_snapshots` (
|
||||
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
|
||||
`server_id` integer NOT NULL,
|
||||
`polled_at` text NOT NULL,
|
||||
`status` text NOT NULL,
|
||||
`latency_ms` real,
|
||||
`ros_version` text,
|
||||
`board_name` text,
|
||||
`uptime` text,
|
||||
`cpu_load` integer,
|
||||
`free_memory` integer,
|
||||
`total_memory` integer,
|
||||
`identity_name` text,
|
||||
`raw_interfaces` text,
|
||||
`raw_ip_addresses` text,
|
||||
FOREIGN KEY (`server_id`) REFERENCES `servers`(`id`) ON UPDATE no action ON DELETE cascade
|
||||
);
|
||||
--> statement-breakpoint
|
||||
CREATE TABLE `servers` (
|
||||
`id` integer PRIMARY KEY AUTOINCREMENT NOT NULL,
|
||||
`name` text DEFAULT '' NOT NULL,
|
||||
`host` text NOT NULL,
|
||||
`port` integer DEFAULT 443 NOT NULL,
|
||||
`username` text DEFAULT 'admin' NOT NULL,
|
||||
`password` text DEFAULT '' NOT NULL,
|
||||
`use_ssl` integer DEFAULT true NOT NULL,
|
||||
`verify_ssl` integer DEFAULT false NOT NULL,
|
||||
`type` text DEFAULT 'home-router' NOT NULL,
|
||||
`site` text DEFAULT '' NOT NULL,
|
||||
`country` text DEFAULT '' NOT NULL,
|
||||
`asn` text DEFAULT '' NOT NULL,
|
||||
`comment` text DEFAULT '' NOT NULL,
|
||||
`enabled` integer DEFAULT true NOT NULL,
|
||||
`created_at` text DEFAULT (datetime('now')) NOT NULL,
|
||||
`updated_at` text DEFAULT (datetime('now')) NOT NULL
|
||||
);
|
||||
@@ -0,0 +1,626 @@
|
||||
-- MikrotikManager PostgreSQL 18 schema (not 1:1 SQLite).
|
||||
-- Applied once at backend boot via src/db/migrate.ts
|
||||
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
id TEXT PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
host TEXT NOT NULL,
|
||||
port INTEGER NOT NULL DEFAULT 443,
|
||||
username TEXT NOT NULL DEFAULT 'admin',
|
||||
password TEXT NOT NULL DEFAULT '',
|
||||
use_ssl BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
verify_ssl BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
type TEXT NOT NULL DEFAULT 'home-router'
|
||||
CHECK (type IN ('jump-host', 'exit-node', 'home-router')),
|
||||
site TEXT NOT NULL DEFAULT '',
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
asn TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
lan_subnet TEXT NOT NULL DEFAULT '',
|
||||
wan_uplinks JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
mgmt_tunnel_ip TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 30,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers_api_ping_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 120,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_flow_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
collector_ip TEXT NOT NULL DEFAULT '10.255.254.1',
|
||||
flow_listen_port INTEGER NOT NULL DEFAULT 4739,
|
||||
wg_listen_port INTEGER NOT NULL DEFAULT 51821,
|
||||
prefix TEXT NOT NULL DEFAULT '10.255.254.0/24',
|
||||
public_endpoint TEXT NOT NULL DEFAULT '',
|
||||
host_public_key TEXT NOT NULL DEFAULT '',
|
||||
host_private_key TEXT NOT NULL DEFAULT '',
|
||||
hub_server_id BIGINT,
|
||||
retention_hours INTEGER NOT NULL DEFAULT 24,
|
||||
top_n INTEGER NOT NULL DEFAULT 200,
|
||||
map_service_min_share_pct DOUBLE PRECISION NOT NULL DEFAULT 5,
|
||||
last_datagram_at TIMESTAMPTZ,
|
||||
last_exporter_ip TEXT,
|
||||
last_error TEXT,
|
||||
packets_received BIGINT NOT NULL DEFAULT 0,
|
||||
peers_json JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
resources_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
ping_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
speed_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 15,
|
||||
probe_interval_sec INTEGER NOT NULL DEFAULT 15,
|
||||
speed_interval_sec INTEGER NOT NULL DEFAULT 60,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS evobgp_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
base_url TEXT NOT NULL DEFAULT '',
|
||||
api_key TEXT NOT NULL DEFAULT '',
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_telegram_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
bot_token TEXT NOT NULL DEFAULT '',
|
||||
chat_id TEXT NOT NULL DEFAULT '',
|
||||
message_thread_id INTEGER,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS acme_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
directory_url TEXT NOT NULL DEFAULT 'https://acme-v02.api.letsencrypt.org/directory',
|
||||
cloudflare_api_token TEXT NOT NULL DEFAULT '',
|
||||
default_zone_id TEXT NOT NULL DEFAULT '',
|
||||
account_private_key TEXT NOT NULL DEFAULT '',
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS certificate_renew_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 21600,
|
||||
renew_before_days INTEGER NOT NULL DEFAULT 30,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_schedule_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
frequency TEXT NOT NULL DEFAULT 'daily' CHECK (frequency IN ('daily', 'weekly', 'monthly')),
|
||||
hour INTEGER NOT NULL DEFAULT 3,
|
||||
minute INTEGER NOT NULL DEFAULT 0,
|
||||
week_day INTEGER NOT NULL DEFAULT 0,
|
||||
month_day INTEGER NOT NULL DEFAULT 1,
|
||||
keep_count INTEGER NOT NULL DEFAULT 7,
|
||||
format TEXT NOT NULL DEFAULT 'rsc' CHECK (format IN ('rsc', 'backup')),
|
||||
server_ids_json JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
last_run_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS internet_path_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 300,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_cursor (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
last_source_finished_at TIMESTAMPTZ,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS data_migration (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
sqlite_imported_at TIMESTAMPTZ,
|
||||
sqlite_path TEXT,
|
||||
sqlite_sha256 TEXT,
|
||||
report_json JSONB
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS filter_rules (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
community TEXT NOT NULL,
|
||||
community_name TEXT,
|
||||
action TEXT NOT NULL DEFAULT 'route' CHECK (action IN ('route', 'blackhole')),
|
||||
gateway TEXT NOT NULL DEFAULT '',
|
||||
gateway_tunnel_id TEXT NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_filter_rules_server_sort ON filter_rules(server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS recursive_routes (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
dst_address TEXT NOT NULL,
|
||||
gateway TEXT NOT NULL,
|
||||
distance INTEGER NOT NULL DEFAULT 1,
|
||||
scope INTEGER,
|
||||
target_scope INTEGER,
|
||||
routing_table TEXT NOT NULL DEFAULT 'main',
|
||||
check_gateway TEXT NOT NULL DEFAULT '',
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
disabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_recursive_routes_server_sort ON recursive_routes(server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS server_snapshots (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
polled_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('online', 'offline')),
|
||||
latency_ms DOUBLE PRECISION,
|
||||
ros_version TEXT,
|
||||
board_name TEXT,
|
||||
uptime TEXT,
|
||||
cpu_load INTEGER,
|
||||
free_memory BIGINT,
|
||||
total_memory BIGINT,
|
||||
identity_name TEXT,
|
||||
raw_interfaces JSONB,
|
||||
raw_ip_addresses JSONB,
|
||||
PRIMARY KEY (id, polled_at)
|
||||
) PARTITION BY RANGE (polled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_server_snapshots_server_time ON server_snapshots(server_id, polled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_server_snapshots_polled_brin ON server_snapshots USING BRIN (polled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_samples (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
interface_name TEXT NOT NULL,
|
||||
peer_public_key TEXT NOT NULL DEFAULT '',
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
rx_bytes BIGINT NOT NULL DEFAULT 0,
|
||||
tx_bytes BIGINT NOT NULL DEFAULT 0,
|
||||
rx_bps BIGINT NOT NULL DEFAULT 0,
|
||||
tx_bps BIGINT NOT NULL DEFAULT 0,
|
||||
running BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
disabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
PRIMARY KEY (id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_traffic_samples_server_time ON traffic_samples(server_id, sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_traffic_samples_server_iface_time ON traffic_samples(server_id, interface_name, sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_traffic_samples_sampled_brin ON traffic_samples USING BRIN (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers_rest_ping_samples (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
ok BOOLEAN NOT NULL,
|
||||
latency_ms INTEGER,
|
||||
error TEXT,
|
||||
PRIMARY KEY (id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_servers_rest_ping_samples_server_id ON servers_rest_ping_samples(server_id, sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_servers_rest_ping_samples_brin ON servers_rest_ping_samples USING BRIN (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_buckets (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
src TEXT NOT NULL,
|
||||
dst TEXT NOT NULL,
|
||||
proto INTEGER NOT NULL DEFAULT 0,
|
||||
src_port INTEGER NOT NULL DEFAULT 0,
|
||||
dst_port INTEGER NOT NULL DEFAULT 0,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
in_iface TEXT NOT NULL DEFAULT '',
|
||||
out_iface TEXT NOT NULL DEFAULT '',
|
||||
next_hop TEXT NOT NULL DEFAULT '',
|
||||
flow_start_ms BIGINT NOT NULL DEFAULT 0,
|
||||
flow_end_ms BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_buckets_server_time ON flow_buckets(server_id, bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_buckets_bucket_brin ON flow_buckets USING BRIN (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_minute_stats (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
unique_src INTEGER NOT NULL DEFAULT 0,
|
||||
unique_dst INTEGER NOT NULL DEFAULT 0,
|
||||
conversations INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_minute_stats_brin ON flow_minute_stats USING BRIN (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_minute_dims (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
dim TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, dim, key)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_minute_dims_time ON flow_minute_dims(bucket_at, dim);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_minute_dims_brin ON flow_minute_dims USING BRIN (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_daily_dims (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
day DATE NOT NULL,
|
||||
dim TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, day, dim, key)
|
||||
) PARTITION BY RANGE (day);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_daily_dims_day ON flow_daily_dims(day, dim);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_ip_meta (
|
||||
prefix TEXT PRIMARY KEY,
|
||||
asn INTEGER NOT NULL DEFAULT 0,
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
holder TEXT NOT NULL DEFAULT '',
|
||||
ok INTEGER NOT NULL DEFAULT 1,
|
||||
fetched_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_asn_meta (
|
||||
asn INTEGER PRIMARY KEY,
|
||||
holder TEXT NOT NULL DEFAULT '',
|
||||
fetched_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_probes (
|
||||
id TEXT PRIMARY KEY,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
name TEXT NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
probe_filter TEXT NOT NULL DEFAULT '—',
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 0,
|
||||
show_on_dashboard BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_probes_server_sort ON uptime_probes(src_server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_probe_samples (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
probe_id TEXT NOT NULL REFERENCES uptime_probes(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
rtt_ms INTEGER,
|
||||
loss_pct INTEGER NOT NULL DEFAULT 0,
|
||||
status TEXT NOT NULL DEFAULT 'down' CHECK (status IN ('up', 'warn', 'down')),
|
||||
PRIMARY KEY (id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_probe_samples_probe_time ON uptime_probe_samples(probe_id, sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_probe_samples_brin ON uptime_probe_samples USING BRIN (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_resource_samples (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'offline' CHECK (status IN ('online', 'offline')),
|
||||
cpu_load INTEGER NOT NULL DEFAULT 0,
|
||||
free_memory BIGINT NOT NULL DEFAULT 0,
|
||||
total_memory BIGINT NOT NULL DEFAULT 0,
|
||||
free_hdd_space BIGINT NOT NULL DEFAULT 0,
|
||||
total_hdd_space BIGINT NOT NULL DEFAULT 0,
|
||||
uptime_seconds BIGINT NOT NULL DEFAULT 0,
|
||||
board_name TEXT NOT NULL DEFAULT '',
|
||||
ros_version TEXT NOT NULL DEFAULT '',
|
||||
PRIMARY KEY (id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_resource_samples_server_time ON uptime_resource_samples(server_id, sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_resource_samples_brin ON uptime_resource_samples USING BRIN (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_speed_probes (
|
||||
id TEXT PRIMARY KEY,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
dst_interface TEXT NOT NULL DEFAULT '',
|
||||
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
|
||||
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
|
||||
duration_sec INTEGER NOT NULL DEFAULT 10,
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
last_run_at TIMESTAMPTZ,
|
||||
last_tx_avg_mbps DOUBLE PRECISION,
|
||||
last_rx_avg_mbps DOUBLE PRECISION,
|
||||
last_status TEXT CHECK (last_status IN ('done', 'error')),
|
||||
last_error TEXT,
|
||||
last_ping_rtt_ms INTEGER,
|
||||
last_ping_loss_pct INTEGER,
|
||||
last_ping_at TIMESTAMPTZ,
|
||||
last_ping_error TEXT,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_speed_probes_src_sort ON uptime_speed_probes(src_server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_speed_test_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
probe_id TEXT,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
dst_interface TEXT NOT NULL DEFAULT '',
|
||||
src_address TEXT,
|
||||
dst_address TEXT,
|
||||
src_interface_address TEXT,
|
||||
dst_interface_address TEXT,
|
||||
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
|
||||
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
|
||||
duration_sec INTEGER NOT NULL DEFAULT 10,
|
||||
tx_avg_mbps DOUBLE PRECISION,
|
||||
rx_avg_mbps DOUBLE PRECISION,
|
||||
ping_rtt_ms INTEGER,
|
||||
ping_loss_pct INTEGER,
|
||||
ping_error TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'done' CHECK (status IN ('done', 'error')),
|
||||
error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_speed_test_runs_created_at ON uptime_speed_test_runs(created_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS certificate_issue_jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'done', 'failed')),
|
||||
step TEXT NOT NULL DEFAULT 'queued',
|
||||
source TEXT NOT NULL DEFAULT 'manual' CHECK (source IN ('manual', 'scheduler')),
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
cert_name TEXT NOT NULL,
|
||||
domain_names JSONB NOT NULL,
|
||||
key_type TEXT NOT NULL DEFAULT 'rsa2048',
|
||||
trust_store TEXT NOT NULL DEFAULT 'www,api',
|
||||
requested_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
error TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_entries (
|
||||
id TEXT PRIMARY KEY,
|
||||
server_id BIGINT REFERENCES servers(id) ON DELETE SET NULL,
|
||||
server_name TEXT NOT NULL,
|
||||
filename TEXT NOT NULL,
|
||||
size_bytes BIGINT NOT NULL,
|
||||
kind TEXT NOT NULL DEFAULT 'manual' CHECK (kind IN ('manual', 'auto')),
|
||||
notes TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_backup_entries_filename ON backup_entries(filename);
|
||||
CREATE INDEX IF NOT EXISTS idx_backup_entries_server_created ON backup_entries(server_id, created_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_groups (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
combine_mode TEXT NOT NULL DEFAULT 'any' CHECK (combine_mode IN ('any', 'all')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
cooldown_override TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rules (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
condition TEXT NOT NULL,
|
||||
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
cooldown TEXT NOT NULL DEFAULT '5м',
|
||||
rule_chat_id TEXT NOT NULL DEFAULT '',
|
||||
confirm_stability_sec INTEGER,
|
||||
recovery_mode TEXT NOT NULL DEFAULT 'always' CHECK (recovery_mode IN ('always', 'never', 'conditional')),
|
||||
recovery_stability_sec INTEGER,
|
||||
group_id TEXT REFERENCES alert_groups(id) ON DELETE SET NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rule_targets (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
|
||||
target TEXT NOT NULL,
|
||||
sort_index INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_rule_targets_rule ON alert_rule_targets(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rule_conditions (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
|
||||
condition_line TEXT NOT NULL,
|
||||
sort_index INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_rule_conditions_rule ON alert_rule_conditions(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_state (
|
||||
scope_key TEXT PRIMARY KEY,
|
||||
last_fired_at TEXT NOT NULL DEFAULT '',
|
||||
last_payload_hash TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_prev_live (
|
||||
kind TEXT PRIMARY KEY CHECK (kind IN ('gre', 'bgp')),
|
||||
payload_json JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_confirm_pending (
|
||||
rule_id TEXT PRIMARY KEY,
|
||||
payload_hash TEXT NOT NULL,
|
||||
since_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_history (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT,
|
||||
group_id TEXT,
|
||||
rule_name TEXT NOT NULL,
|
||||
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
|
||||
message TEXT NOT NULL,
|
||||
sent_ok BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
fired_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_history_fired_at ON alert_history(fired_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_history_rule_id ON alert_history(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_outbox (
|
||||
id TEXT PRIMARY KEY,
|
||||
dedupe_key TEXT NOT NULL,
|
||||
channel TEXT NOT NULL DEFAULT 'telegram' CHECK (channel IN ('telegram')),
|
||||
status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'sent', 'failed')),
|
||||
retry_count INTEGER NOT NULL DEFAULT 0,
|
||||
max_retries INTEGER NOT NULL DEFAULT 3,
|
||||
next_attempt_at TIMESTAMPTZ NOT NULL,
|
||||
payload_json JSONB NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
sent_at TIMESTAMPTZ,
|
||||
last_error TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_alert_outbox_dedupe ON alert_outbox(dedupe_key);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_outbox_status_next_attempt ON alert_outbox(status, next_attempt_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_outbox_pending_next ON alert_outbox(next_attempt_at) WHERE status = 'pending';
|
||||
|
||||
CREATE TABLE IF NOT EXISTS scheduler_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
job_key TEXT NOT NULL,
|
||||
started_at TIMESTAMPTZ NOT NULL,
|
||||
finished_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('ok', 'error')),
|
||||
error TEXT,
|
||||
duration_ms INTEGER NOT NULL DEFAULT 0,
|
||||
result_json JSONB
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_scheduler_runs_job_time ON scheduler_runs(job_key, started_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events (
|
||||
id TEXT PRIMARY KEY,
|
||||
created_at TIMESTAMPTZ NOT NULL,
|
||||
level TEXT NOT NULL CHECK (level IN ('critical', 'warning', 'info')),
|
||||
event_type TEXT NOT NULL,
|
||||
source_module TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
message TEXT NOT NULL,
|
||||
entity_type TEXT,
|
||||
entity_id TEXT,
|
||||
payload_json JSONB
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_level_created_at ON events(level, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_source_created_at ON events(source_module, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_event_type_created_at ON events(event_type, created_at DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS app_users (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
login TEXT NOT NULL UNIQUE,
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
role TEXT NOT NULL DEFAULT 'viewer' CHECK (role IN ('admin', 'operator', 'viewer')),
|
||||
active BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
avatar TEXT NOT NULL DEFAULT '',
|
||||
last_seen TIMESTAMPTZ,
|
||||
sections_json JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
servers_json JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS user_interface_bindings (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES app_users(id) ON DELETE CASCADE,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
interface_name TEXT NOT NULL,
|
||||
interface_type TEXT NOT NULL DEFAULT 'other' CHECK (interface_type IN ('ether', 'gre', 'wg', 'other')),
|
||||
peer_public_key TEXT NOT NULL DEFAULT '',
|
||||
peer_name TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE (server_id, interface_name, peer_public_key)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_user_iface_bind_user ON user_interface_bindings(user_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS internet_path_snapshots (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
payload_json JSONB NOT NULL,
|
||||
PRIMARY KEY (id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_internet_path_snapshots_sampled ON internet_path_snapshots(sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_internet_path_snapshots_brin ON internet_path_snapshots USING BRIN (sampled_at);
|
||||
|
||||
INSERT INTO traffic_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO traffic_flow_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO uptime_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO evobgp_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO servers_api_ping_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO internet_path_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO alert_telegram_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO acme_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO certificate_renew_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO backup_schedule_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO alert_engine_cursor (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO data_migration (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"version": "7",
|
||||
"dialect": "sqlite",
|
||||
"dialect": "postgresql",
|
||||
"entries": [
|
||||
{
|
||||
"idx": 0,
|
||||
"version": "6",
|
||||
"version": "7",
|
||||
"when": 1777572014210,
|
||||
"tag": "0000_living_xorn",
|
||||
"tag": "0000_postgresql",
|
||||
"breakpoints": true
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "mikrotik-manager-backend",
|
||||
"version": "1.0.0",
|
||||
"description": "MikroTik Manager backend — Fastify + Drizzle + SQLite",
|
||||
"description": "MikroTik Manager backend — Fastify + Drizzle + PostgreSQL",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "tsx watch src/index.ts",
|
||||
@@ -11,11 +11,14 @@
|
||||
"db:generate": "drizzle-kit generate",
|
||||
"db:migrate": "drizzle-kit migrate",
|
||||
"db:studio": "drizzle-kit studio",
|
||||
"db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts",
|
||||
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
|
||||
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
|
||||
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-hardening.test.ts && tsx src/services/traffic-flow-purge.test.ts",
|
||||
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts"
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts",
|
||||
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/pg-schema.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg"
|
||||
},
|
||||
"dependencies": {
|
||||
"@fastify/cors": "^11.2.0",
|
||||
@@ -28,12 +31,14 @@
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"fastify": "^5.8.5",
|
||||
"fastify-plugin": "^5.1.0",
|
||||
"pg": "^8.23.0",
|
||||
"undici": "^8.1.0",
|
||||
"zod": "^4.4.1"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/better-sqlite3": "^7.6.13",
|
||||
"@types/node": "^22.15.3",
|
||||
"@types/pg": "^8.23.1",
|
||||
"drizzle-kit": "^0.31.10",
|
||||
"jose": "^6.2.11",
|
||||
"pino-pretty": "^13.1.3",
|
||||
|
||||
@@ -0,0 +1,39 @@
|
||||
import Database from "better-sqlite3"
|
||||
import fs from "node:fs"
|
||||
|
||||
const path = process.argv[2] ?? "./mikrotik.db"
|
||||
if (!fs.existsSync(path)) {
|
||||
console.error("NO_SQLITE", path)
|
||||
process.exit(2)
|
||||
}
|
||||
const st = fs.statSync(path)
|
||||
const db = new Database(path, { readonly: true, fileMustExist: true })
|
||||
const tables = db
|
||||
.prepare(`SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%' ORDER BY 1`)
|
||||
.all()
|
||||
console.log("FILE_BYTES", st.size)
|
||||
console.log("TABLES", tables.length)
|
||||
for (const t of tables) {
|
||||
const c = db.prepare(`SELECT COUNT(*) AS c FROM "${t.name}"`).get()
|
||||
console.log(`COUNT\t${t.name}\t${c.c}`)
|
||||
}
|
||||
try {
|
||||
const d = db.prepare(`SELECT COUNT(*) AS c FROM (
|
||||
SELECT dedupe_key FROM alert_outbox GROUP BY dedupe_key HAVING COUNT(*) > 1
|
||||
)`).get()
|
||||
console.log("DUP_DEDUPE_KEYS", d.c)
|
||||
} catch (e) {
|
||||
console.log("DUP_DEDUPE_ERR", e instanceof Error ? e.message : e)
|
||||
}
|
||||
for (const [label, sql] of [
|
||||
["ORPHAN_FLOW_MINUTE", `SELECT COUNT(*) c FROM flow_minute_stats s WHERE s.server_id NOT IN (SELECT id FROM servers)`],
|
||||
["ORPHAN_FLOW_BUCKETS", `SELECT COUNT(*) c FROM flow_buckets s WHERE s.server_id NOT IN (SELECT id FROM servers)`],
|
||||
["ORPHAN_BINDINGS", `SELECT COUNT(*) c FROM user_interface_bindings s WHERE s.server_id NOT IN (SELECT id FROM servers)`],
|
||||
]) {
|
||||
try {
|
||||
console.log(label, db.prepare(sql).get().c)
|
||||
} catch (e) {
|
||||
console.log(label + "_ERR", e instanceof Error ? e.message : e)
|
||||
}
|
||||
}
|
||||
db.close()
|
||||
@@ -11,7 +11,9 @@ function boolEnv(v: string | undefined, fallback: boolean): boolean {
|
||||
const isProd = process.env.NODE_ENV === "production"
|
||||
|
||||
const envSchema = z.object({
|
||||
DATABASE_URL: z.string().default("postgres://mmapp:mmapp@127.0.0.1:5432/mmapp"),
|
||||
DATABASE_PATH: z.string().default("./mikrotik.db"),
|
||||
SQLITE_IMPORT_FULL_HISTORY: z.boolean().default(false),
|
||||
PORT: z.coerce.number().int().positive().default(8000),
|
||||
CORS_ORIGIN: z.string().default("http://localhost:3000"),
|
||||
AUTH_REQUIRED: z.boolean().default(false),
|
||||
@@ -21,7 +23,9 @@ const envSchema = z.object({
|
||||
})
|
||||
|
||||
const raw = {
|
||||
DATABASE_URL: process.env.DATABASE_URL,
|
||||
DATABASE_PATH: process.env.DATABASE_PATH,
|
||||
SQLITE_IMPORT_FULL_HISTORY: boolEnv(process.env.SQLITE_IMPORT_FULL_HISTORY, false),
|
||||
PORT: process.env.PORT,
|
||||
CORS_ORIGIN: process.env.CORS_ORIGIN,
|
||||
AUTH_REQUIRED: boolEnv(process.env.AUTH_REQUIRED, false),
|
||||
@@ -53,7 +57,9 @@ if (parsed.data.AUTH_REQUIRED && parsed.data.AUTH_JWT_SECRET.length < 8) {
|
||||
}
|
||||
|
||||
export const env = {
|
||||
DATABASE_URL: parsed.data.DATABASE_URL,
|
||||
DATABASE_PATH: parsed.data.DATABASE_PATH,
|
||||
sqliteImportFullHistory: parsed.data.SQLITE_IMPORT_FULL_HISTORY,
|
||||
PORT: parsed.data.PORT,
|
||||
CORS_ORIGIN: parsed.data.CORS_ORIGIN,
|
||||
authRequired: parsed.data.AUTH_REQUIRED,
|
||||
|
||||
@@ -0,0 +1,33 @@
|
||||
import { pool } from "./index.js"
|
||||
import { applySqlMigrations } from "./migrate.js"
|
||||
import { dropExpiredPartitions, ensurePartitionsAround } from "./partitions.js"
|
||||
import { importSqliteToPostgres, shouldImportSqlite } from "./sqlite-import.js"
|
||||
import { env } from "../config.js"
|
||||
|
||||
const ETL_LOCK = 8723101
|
||||
|
||||
export async function initDatabase(): Promise<void> {
|
||||
await pool.query("SELECT 1")
|
||||
await applySqlMigrations(pool)
|
||||
await ensurePartitionsAround(pool)
|
||||
const client = await pool.connect()
|
||||
try {
|
||||
await client.query("SELECT pg_advisory_lock($1)", [ETL_LOCK])
|
||||
if (await shouldImportSqlite(pool, env.DATABASE_PATH)) {
|
||||
console.log(`SQLite → PostgreSQL: импорт ${env.DATABASE_PATH}`)
|
||||
const report = await importSqliteToPostgres(pool, env.DATABASE_PATH, { strict: true })
|
||||
console.log(
|
||||
`SQLite → PostgreSQL: готово за ${report.durationMs}ms, таблиц ${Object.keys(report.tables).length}`,
|
||||
)
|
||||
}
|
||||
} finally {
|
||||
try {
|
||||
await client.query("SELECT pg_advisory_unlock($1)", [ETL_LOCK])
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
client.release()
|
||||
}
|
||||
await dropExpiredPartitions(pool)
|
||||
await ensurePartitionsAround(pool)
|
||||
}
|
||||
+56
-983
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,43 @@
|
||||
/** JSONB from node-pg may already be an object; SQLite leftovers may be strings. */
|
||||
|
||||
export function parseJsonObject(value: unknown): Record<string, unknown> {
|
||||
if (value && typeof value === "object" && !Array.isArray(value)) {
|
||||
return value as Record<string, unknown>
|
||||
}
|
||||
if (typeof value === "string" && value.trim()) {
|
||||
try {
|
||||
const parsed = JSON.parse(value) as unknown
|
||||
if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
|
||||
return parsed as Record<string, unknown>
|
||||
}
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
return {}
|
||||
}
|
||||
|
||||
export function parseJsonArray(value: unknown): unknown[] {
|
||||
if (Array.isArray(value)) return value
|
||||
if (typeof value === "string" && value.trim()) {
|
||||
try {
|
||||
const parsed = JSON.parse(value) as unknown
|
||||
if (Array.isArray(parsed)) return parsed
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
return []
|
||||
}
|
||||
|
||||
export function toJsonb(value: unknown): unknown {
|
||||
if (value == null) return null
|
||||
if (typeof value === "string") {
|
||||
try {
|
||||
return JSON.parse(value) as unknown
|
||||
} catch {
|
||||
return value
|
||||
}
|
||||
}
|
||||
return value
|
||||
}
|
||||
@@ -0,0 +1,43 @@
|
||||
import { readFileSync } from "node:fs"
|
||||
import { dirname, join } from "node:path"
|
||||
import { fileURLToPath } from "node:url"
|
||||
import type { Pool } from "pg"
|
||||
|
||||
const MIGRATION_ID = "0000_postgresql"
|
||||
|
||||
function migrationsDir(): string {
|
||||
const here = dirname(fileURLToPath(import.meta.url))
|
||||
const candidates = [
|
||||
join(here, "..", "..", "drizzle"),
|
||||
join(process.cwd(), "drizzle"),
|
||||
join(process.cwd(), "backend", "drizzle"),
|
||||
]
|
||||
for (const dir of candidates) {
|
||||
try {
|
||||
readFileSync(join(dir, `${MIGRATION_ID}.sql`), "utf8")
|
||||
return dir
|
||||
} catch {
|
||||
/* try next */
|
||||
}
|
||||
}
|
||||
throw new Error("Не найден backend/drizzle/0000_postgresql.sql")
|
||||
}
|
||||
|
||||
export async function applySqlMigrations(pool: Pool): Promise<void> {
|
||||
await pool.query(`
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
id TEXT PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
)
|
||||
`)
|
||||
const { rows } = await pool.query<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = $1`,
|
||||
[MIGRATION_ID],
|
||||
)
|
||||
if (rows.length > 0) return
|
||||
const sql = readFileSync(join(migrationsDir(), `${MIGRATION_ID}.sql`), "utf8")
|
||||
await pool.query(sql)
|
||||
await pool.query(`INSERT INTO schema_migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`, [
|
||||
MIGRATION_ID,
|
||||
])
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
import type { Pool } from "pg"
|
||||
|
||||
export type PartitionKind = "day" | "week" | "month"
|
||||
|
||||
export interface PartitionSpec {
|
||||
parent: string
|
||||
kind: PartitionKind
|
||||
keepDays: number
|
||||
}
|
||||
|
||||
export const PARTITION_SPECS: PartitionSpec[] = [
|
||||
{ parent: "flow_buckets", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_stats", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_dims", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_daily_dims", kind: "month", keepDays: 420 },
|
||||
{ parent: "traffic_samples", kind: "week", keepDays: 21 },
|
||||
{ parent: "servers_rest_ping_samples", kind: "week", keepDays: 35 },
|
||||
{ parent: "uptime_probe_samples", kind: "week", keepDays: 21 },
|
||||
{ parent: "uptime_resource_samples", kind: "week", keepDays: 21 },
|
||||
{ parent: "server_snapshots", kind: "week", keepDays: 21 },
|
||||
{ parent: "internet_path_snapshots", kind: "week", keepDays: 21 },
|
||||
]
|
||||
|
||||
function utcDate(d: Date): Date {
|
||||
return new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()))
|
||||
}
|
||||
|
||||
function addUtcDays(d: Date, n: number): Date {
|
||||
const x = utcDate(d)
|
||||
x.setUTCDate(x.getUTCDate() + n)
|
||||
return x
|
||||
}
|
||||
|
||||
function startOfWeekUtc(d: Date): Date {
|
||||
const x = utcDate(d)
|
||||
const day = x.getUTCDay() || 7
|
||||
x.setUTCDate(x.getUTCDate() - (day - 1))
|
||||
return x
|
||||
}
|
||||
|
||||
function startOfMonthUtc(d: Date): Date {
|
||||
return new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), 1))
|
||||
}
|
||||
|
||||
function fmtDay(d: Date): string {
|
||||
return d.toISOString().slice(0, 10)
|
||||
}
|
||||
|
||||
function partName(parent: string, kind: PartitionKind, start: Date): string {
|
||||
if (kind === "day") return `${parent}_d_${fmtDay(start).replaceAll("-", "")}`
|
||||
if (kind === "week") return `${parent}_w_${fmtDay(start).replaceAll("-", "")}`
|
||||
return `${parent}_m_${start.getUTCFullYear()}${String(start.getUTCMonth() + 1).padStart(2, "0")}`
|
||||
}
|
||||
|
||||
function rangeFor(kind: PartitionKind, ts: Date): { start: Date; end: Date } {
|
||||
if (kind === "day") {
|
||||
const start = utcDate(ts)
|
||||
return { start, end: addUtcDays(start, 1) }
|
||||
}
|
||||
if (kind === "week") {
|
||||
const start = startOfWeekUtc(ts)
|
||||
return { start, end: addUtcDays(start, 7) }
|
||||
}
|
||||
const start = startOfMonthUtc(ts)
|
||||
const end = new Date(Date.UTC(start.getUTCFullYear(), start.getUTCMonth() + 1, 1))
|
||||
return { start, end }
|
||||
}
|
||||
|
||||
export async function ensurePartitionFor(
|
||||
pool: Pool,
|
||||
parent: string,
|
||||
kind: PartitionKind,
|
||||
ts: Date,
|
||||
): Promise<string> {
|
||||
const { start, end } = rangeFor(kind, ts)
|
||||
const name = partName(parent, kind, start)
|
||||
const from = fmtDay(start)
|
||||
const to = fmtDay(end)
|
||||
await pool.query(
|
||||
`CREATE TABLE IF NOT EXISTS ${name} PARTITION OF ${parent} FOR VALUES FROM ('${from}') TO ('${to}')`,
|
||||
)
|
||||
return name
|
||||
}
|
||||
|
||||
export async function ensurePartitionsAround(pool: Pool, around = new Date()): Promise<void> {
|
||||
for (const spec of PARTITION_SPECS) {
|
||||
const daysAhead = spec.kind === "month" ? 40 : spec.kind === "week" ? 21 : 8
|
||||
const start = addUtcDays(around, -spec.keepDays)
|
||||
const end = addUtcDays(around, daysAhead)
|
||||
for (let t = new Date(start.getTime()); t < end; ) {
|
||||
await ensurePartitionFor(pool, spec.parent, spec.kind, t)
|
||||
if (spec.kind === "day") t = addUtcDays(t, 1)
|
||||
else if (spec.kind === "week") t = addUtcDays(t, 7)
|
||||
else t = new Date(Date.UTC(t.getUTCFullYear(), t.getUTCMonth() + 1, 1))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export async function dropExpiredPartitions(pool: Pool, around = new Date()): Promise<void> {
|
||||
for (const spec of PARTITION_SPECS) {
|
||||
const cutoff = addUtcDays(around, -spec.keepDays)
|
||||
const { rows } = await pool.query<{ relname: string }>(
|
||||
`SELECT c.relname
|
||||
FROM pg_inherits i
|
||||
JOIN pg_class c ON c.oid = i.inhrelid
|
||||
JOIN pg_class p ON p.oid = i.inhparent
|
||||
WHERE p.relname = $1`,
|
||||
[spec.parent],
|
||||
)
|
||||
for (const row of rows) {
|
||||
const m = row.relname.match(/_([dwm])_(\d{8}|\d{6})$/)
|
||||
if (!m) continue
|
||||
const stamp = m[2]
|
||||
let start: Date
|
||||
if (stamp.length === 6) {
|
||||
start = new Date(Date.UTC(Number(stamp.slice(0, 4)), Number(stamp.slice(4, 6)) - 1, 1))
|
||||
} else {
|
||||
start = new Date(`${stamp.slice(0, 4)}-${stamp.slice(4, 6)}-${stamp.slice(6, 8)}T00:00:00Z`)
|
||||
}
|
||||
if (start < cutoff) {
|
||||
await pool.query(`DROP TABLE IF EXISTS ${row.relname}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function specForParent(parent: string): PartitionSpec | undefined {
|
||||
return PARTITION_SPECS.find((s) => s.parent === parent)
|
||||
}
|
||||
@@ -0,0 +1,56 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery, pool } from "./index.js"
|
||||
import { ensurePartitionFor } from "./partitions.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("pg-schema.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ n: string }>(`SELECT COUNT(*)::text AS n FROM servers`)
|
||||
assert.ok(rows[0])
|
||||
}
|
||||
|
||||
{
|
||||
await dbQuery(`
|
||||
INSERT INTO servers (name, host) VALUES ('pg-schema-test', '127.0.0.1')
|
||||
`)
|
||||
const { rows } = await dbQuery<{ id: number }>(`SELECT id FROM servers WHERE name = 'pg-schema-test' LIMIT 1`)
|
||||
const id = rows[0]?.id
|
||||
assert.ok(id)
|
||||
|
||||
await ensurePartitionFor(pool, "flow_buckets", "day", new Date())
|
||||
const bucketAt = new Date().toISOString()
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface
|
||||
) VALUES ($1, $2, '10.0.0.1', '8.8.8.8', 6, 1, 443, 10, 1, 'wg-flow')
|
||||
ON CONFLICT DO NOTHING
|
||||
`, [id, bucketAt])
|
||||
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [id])
|
||||
await dbQuery(`DELETE FROM servers WHERE id = $1`, [id])
|
||||
}
|
||||
|
||||
{
|
||||
await dbQuery(`
|
||||
INSERT INTO alert_outbox (id, dedupe_key, payload_json, next_attempt_at)
|
||||
VALUES ('pg-test-1', 'pg-dedupe-key', '{}'::jsonb, now())
|
||||
ON CONFLICT (dedupe_key) DO NOTHING
|
||||
`)
|
||||
let threw = false
|
||||
try {
|
||||
await dbQuery(`
|
||||
INSERT INTO alert_outbox (id, dedupe_key, payload_json, next_attempt_at)
|
||||
VALUES ('pg-test-2', 'pg-dedupe-key', '{}'::jsonb, now())
|
||||
`)
|
||||
} catch {
|
||||
threw = true
|
||||
}
|
||||
assert.equal(threw, true, "alert_outbox.dedupe_key UNIQUE")
|
||||
await dbQuery(`DELETE FROM alert_outbox WHERE dedupe_key = 'pg-dedupe-key'`)
|
||||
}
|
||||
|
||||
console.log("pg-schema.test.ts: ok")
|
||||
+348
-364
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,21 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { bindSql } from "./sql-bind.js"
|
||||
|
||||
{
|
||||
const q = bindSql("SELECT 1")
|
||||
assert.deepEqual(q, { text: "SELECT 1", values: [] })
|
||||
}
|
||||
|
||||
{
|
||||
const q = bindSql("SELECT * FROM t WHERE a = ? AND b = ?", [1, "x"])
|
||||
assert.equal(q.text, "SELECT * FROM t WHERE a = $1 AND b = $2")
|
||||
assert.deepEqual(q.values, [1, "x"])
|
||||
}
|
||||
|
||||
{
|
||||
const q = bindSql("INSERT INTO t (a, b) VALUES (@a, @b)", { a: 3, b: "y" })
|
||||
assert.equal(q.text, "INSERT INTO t (a, b) VALUES ($1, $2)")
|
||||
assert.deepEqual(q.values, [3, "y"])
|
||||
}
|
||||
|
||||
console.log("sql-bind.test.ts: ok")
|
||||
@@ -0,0 +1,22 @@
|
||||
/** Convert SQLite-style @name / ? placeholders to node-pg $n. */
|
||||
|
||||
export function bindSql(
|
||||
sql: string,
|
||||
params?: unknown[] | Record<string, unknown>,
|
||||
): { text: string; values: unknown[] } {
|
||||
if (params == null) return { text: sql, values: [] }
|
||||
if (Array.isArray(params)) {
|
||||
let i = 0
|
||||
const text = sql.replace(/\?/g, () => {
|
||||
i += 1
|
||||
return `$${i}`
|
||||
})
|
||||
return { text, values: params }
|
||||
}
|
||||
const values: unknown[] = []
|
||||
const text = sql.replace(/@([a-zA-Z_][a-zA-Z0-9_]*)/g, (_, name: string) => {
|
||||
values.push(params[name])
|
||||
return `$${values.length}`
|
||||
})
|
||||
return { text, values }
|
||||
}
|
||||
@@ -0,0 +1,479 @@
|
||||
import { createHash } from "node:crypto"
|
||||
import { existsSync, readFileSync } from "node:fs"
|
||||
import Database from "better-sqlite3"
|
||||
import type { Pool } from "pg"
|
||||
import { env } from "../config.js"
|
||||
import { ensurePartitionFor, specForParent } from "./partitions.js"
|
||||
|
||||
export interface ImportReport {
|
||||
sqlitePath: string
|
||||
sqliteSha256: string
|
||||
tables: Record<string, { sqlite: number; copied: number; skipped: number }>
|
||||
rejects: string[]
|
||||
durationMs: number
|
||||
}
|
||||
|
||||
const SNAPSHOT_RETENTION_DAYS = 14
|
||||
|
||||
type ColKind = "ts" | "date" | "bool" | "json" | "json-null" | "bigint-id" | "int" | "text" | "num"
|
||||
|
||||
interface TableCopy {
|
||||
table: string
|
||||
columns: Array<[string, ColKind]>
|
||||
timeCol?: string
|
||||
retentionDays?: number
|
||||
identity?: boolean
|
||||
upsert?: boolean
|
||||
}
|
||||
|
||||
const TABLES: TableCopy[] = [
|
||||
{ table: "servers", identity: true, columns: [
|
||||
["id", "int"], ["name", "text"], ["host", "text"], ["port", "int"], ["username", "text"],
|
||||
["password", "text"], ["use_ssl", "bool"], ["verify_ssl", "bool"], ["type", "text"],
|
||||
["site", "text"], ["country", "text"], ["asn", "text"], ["comment", "text"], ["enabled", "bool"],
|
||||
["lan_subnet", "text"], ["wan_uplinks", "json"], ["mgmt_tunnel_ip", "text"],
|
||||
["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "traffic_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"], ["retention_days", "int"],
|
||||
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"],
|
||||
["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "servers_api_ping_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"],
|
||||
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"],
|
||||
["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "traffic_flow_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["collector_ip", "text"], ["flow_listen_port", "int"],
|
||||
["wg_listen_port", "int"], ["prefix", "text"], ["public_endpoint", "text"],
|
||||
["host_public_key", "text"], ["host_private_key", "text"], ["hub_server_id", "int"],
|
||||
["retention_hours", "int"], ["top_n", "int"], ["map_service_min_share_pct", "num"],
|
||||
["last_datagram_at", "ts"], ["last_exporter_ip", "text"], ["last_error", "text"],
|
||||
["packets_received", "int"], ["peers_json", "json"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "uptime_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["resources_enabled", "bool"], ["ping_enabled", "bool"],
|
||||
["speed_enabled", "bool"], ["interval_sec", "int"], ["probe_interval_sec", "int"],
|
||||
["speed_interval_sec", "int"], ["retention_days", "int"], ["last_collected_at", "ts"],
|
||||
["last_duration_ms", "int"], ["last_error", "text"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "evobgp_settings", upsert: true, columns: [
|
||||
["id", "int"], ["base_url", "text"], ["api_key", "text"], ["enabled", "bool"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_telegram_settings", upsert: true, columns: [
|
||||
["id", "int"], ["bot_token", "text"], ["chat_id", "text"], ["message_thread_id", "int"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "acme_settings", upsert: true, columns: [
|
||||
["id", "int"], ["directory_url", "text"], ["cloudflare_api_token", "text"],
|
||||
["default_zone_id", "text"], ["account_private_key", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "certificate_renew_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"], ["renew_before_days", "int"],
|
||||
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "backup_schedule_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["frequency", "text"], ["hour", "int"], ["minute", "int"],
|
||||
["week_day", "int"], ["month_day", "int"], ["keep_count", "int"], ["format", "text"],
|
||||
["server_ids_json", "json"], ["last_run_at", "ts"], ["last_duration_ms", "int"],
|
||||
["last_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "internet_path_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"], ["retention_days", "int"],
|
||||
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"],
|
||||
["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_engine_cursor", upsert: true, columns: [
|
||||
["id", "int"], ["last_source_finished_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "filter_rules", identity: true, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sort_order", "int"], ["community", "text"],
|
||||
["community_name", "text"], ["action", "text"], ["gateway", "text"],
|
||||
["gateway_tunnel_id", "text"], ["description", "text"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "recursive_routes", identity: true, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sort_order", "int"], ["dst_address", "text"],
|
||||
["gateway", "text"], ["distance", "int"], ["scope", "int"], ["target_scope", "int"],
|
||||
["routing_table", "text"], ["check_gateway", "text"], ["country", "text"], ["comment", "text"],
|
||||
["disabled", "bool"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "server_snapshots", identity: true, timeCol: "polled_at", retentionDays: SNAPSHOT_RETENTION_DAYS, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["polled_at", "ts"], ["status", "text"], ["latency_ms", "num"],
|
||||
["ros_version", "text"], ["board_name", "text"], ["uptime", "text"], ["cpu_load", "int"],
|
||||
["free_memory", "int"], ["total_memory", "int"], ["identity_name", "text"],
|
||||
["raw_interfaces", "json-null"], ["raw_ip_addresses", "json-null"],
|
||||
]},
|
||||
{ table: "traffic_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["interface_name", "text"], ["peer_public_key", "text"],
|
||||
["sampled_at", "ts"], ["rx_bytes", "int"], ["tx_bytes", "int"], ["rx_bps", "int"], ["tx_bps", "int"],
|
||||
["running", "bool"], ["disabled", "bool"],
|
||||
]},
|
||||
{ table: "servers_rest_ping_samples", identity: true, timeCol: "sampled_at", retentionDays: 30, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["ok", "bool"], ["latency_ms", "int"], ["error", "text"],
|
||||
]},
|
||||
{ table: "flow_buckets", timeCol: "bucket_at", retentionDays: 2, columns: [
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["src", "text"], ["dst", "text"], ["proto", "int"],
|
||||
["src_port", "int"], ["dst_port", "int"], ["bytes", "int"], ["packets", "int"],
|
||||
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "text"],
|
||||
["flow_start_ms", "int"], ["flow_end_ms", "int"],
|
||||
]},
|
||||
{ table: "flow_minute_stats", timeCol: "bucket_at", retentionDays: 3, columns: [
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["bytes", "int"], ["packets", "int"],
|
||||
["unique_src", "int"], ["unique_dst", "int"], ["conversations", "int"],
|
||||
]},
|
||||
{ table: "flow_minute_dims", timeCol: "bucket_at", retentionDays: 3, columns: [
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["dim", "text"], ["key", "text"], ["bytes", "int"], ["packets", "int"],
|
||||
]},
|
||||
{ table: "flow_daily_dims", timeCol: "day", retentionDays: 396, columns: [
|
||||
["server_id", "int"], ["day", "date"], ["dim", "text"], ["key", "text"], ["bytes", "int"], ["packets", "int"],
|
||||
]},
|
||||
{ table: "flow_ip_meta", columns: [
|
||||
["prefix", "text"], ["asn", "int"], ["country", "text"], ["lat", "num"], ["lng", "num"],
|
||||
["holder", "text"], ["ok", "int"], ["fetched_at", "ts"],
|
||||
]},
|
||||
{ table: "flow_asn_meta", columns: [
|
||||
["asn", "int"], ["holder", "text"], ["fetched_at", "ts"],
|
||||
]},
|
||||
{ table: "uptime_probes", columns: [
|
||||
["id", "text"], ["src_server_id", "int"], ["src_interface", "text"], ["name", "text"],
|
||||
["target", "text"], ["probe_filter", "text"], ["enabled", "bool"], ["interval_sec", "int"],
|
||||
["show_on_dashboard", "bool"], ["sort_order", "int"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "uptime_probe_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["probe_id", "text"], ["sampled_at", "ts"], ["rtt_ms", "int"], ["loss_pct", "int"], ["status", "text"],
|
||||
]},
|
||||
{ table: "uptime_resource_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["status", "text"], ["cpu_load", "int"],
|
||||
["free_memory", "int"], ["total_memory", "int"], ["free_hdd_space", "int"], ["total_hdd_space", "int"],
|
||||
["uptime_seconds", "int"], ["board_name", "text"], ["ros_version", "text"],
|
||||
]},
|
||||
{ table: "uptime_speed_probes", columns: [
|
||||
["id", "text"], ["src_server_id", "int"], ["dst_server_id", "int"], ["src_interface", "text"],
|
||||
["dst_interface", "text"], ["protocol", "text"], ["direction", "text"], ["duration_sec", "int"],
|
||||
["enabled", "bool"], ["last_run_at", "ts"], ["last_tx_avg_mbps", "num"], ["last_rx_avg_mbps", "num"],
|
||||
["last_status", "text"], ["last_error", "text"], ["last_ping_rtt_ms", "int"],
|
||||
["last_ping_loss_pct", "int"], ["last_ping_at", "ts"], ["last_ping_error", "text"],
|
||||
["sort_order", "int"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "uptime_speed_test_runs", columns: [
|
||||
["id", "text"], ["probe_id", "text"], ["src_server_id", "int"], ["dst_server_id", "int"],
|
||||
["src_interface", "text"], ["dst_interface", "text"], ["src_address", "text"], ["dst_address", "text"],
|
||||
["src_interface_address", "text"], ["dst_interface_address", "text"], ["protocol", "text"],
|
||||
["direction", "text"], ["duration_sec", "int"], ["tx_avg_mbps", "num"], ["rx_avg_mbps", "num"],
|
||||
["ping_rtt_ms", "int"], ["ping_loss_pct", "int"], ["ping_error", "text"], ["status", "text"],
|
||||
["error", "text"], ["created_at", "ts"],
|
||||
]},
|
||||
{ table: "app_users", columns: [
|
||||
["id", "text"], ["name", "text"], ["login", "text"], ["email", "text"], ["role", "text"],
|
||||
["active", "bool"], ["avatar", "text"], ["last_seen", "ts"], ["sections_json", "json"],
|
||||
["servers_json", "json"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "user_interface_bindings", columns: [
|
||||
["id", "text"], ["user_id", "text"], ["server_id", "int"], ["interface_name", "text"],
|
||||
["interface_type", "text"], ["peer_public_key", "text"], ["peer_name", "text"],
|
||||
["comment", "text"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_groups", columns: [
|
||||
["id", "text"], ["name", "text"], ["combine_mode", "text"], ["enabled", "bool"],
|
||||
["cooldown_override", "text"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_rules", columns: [
|
||||
["id", "text"], ["name", "text"], ["type", "text"], ["target", "text"], ["condition", "text"],
|
||||
["severity", "text"], ["enabled", "bool"], ["cooldown", "text"], ["rule_chat_id", "text"],
|
||||
["confirm_stability_sec", "int"], ["recovery_mode", "text"], ["recovery_stability_sec", "int"],
|
||||
["group_id", "text"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_rule_targets", columns: [
|
||||
["id", "text"], ["rule_id", "text"], ["target", "text"], ["sort_index", "int"],
|
||||
]},
|
||||
{ table: "alert_rule_conditions", columns: [
|
||||
["id", "text"], ["rule_id", "text"], ["condition_line", "text"], ["sort_index", "int"],
|
||||
]},
|
||||
{ table: "alert_engine_state", columns: [
|
||||
["scope_key", "text"], ["last_fired_at", "text"], ["last_payload_hash", "text"],
|
||||
]},
|
||||
{ table: "alert_engine_prev_live", columns: [
|
||||
["kind", "text"], ["payload_json", "json"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_engine_confirm_pending", columns: [
|
||||
["rule_id", "text"], ["payload_hash", "text"], ["since_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_history", columns: [
|
||||
["id", "text"], ["rule_id", "text"], ["group_id", "text"], ["rule_name", "text"],
|
||||
["severity", "text"], ["message", "text"], ["sent_ok", "bool"], ["fired_at", "ts"],
|
||||
]},
|
||||
{ table: "alert_outbox", columns: [
|
||||
["id", "text"], ["dedupe_key", "text"], ["channel", "text"], ["status", "text"],
|
||||
["retry_count", "int"], ["max_retries", "int"], ["next_attempt_at", "ts"],
|
||||
["payload_json", "json"], ["created_at", "ts"], ["sent_at", "ts"], ["last_error", "text"],
|
||||
]},
|
||||
{ table: "certificate_issue_jobs", columns: [
|
||||
["id", "text"], ["status", "text"], ["step", "text"], ["source", "text"],
|
||||
["server_id", "bigint-id"], ["cert_name", "text"], ["domain_names", "json"],
|
||||
["key_type", "text"], ["trust_store", "text"], ["requested_at", "ts"],
|
||||
["started_at", "ts"], ["finished_at", "ts"], ["error", "text"],
|
||||
]},
|
||||
{ table: "backup_entries", columns: [
|
||||
["id", "text"], ["server_id", "bigint-id"], ["server_name", "text"], ["filename", "text"],
|
||||
["size_bytes", "int"], ["kind", "text"], ["notes", "text"], ["created_at", "ts"],
|
||||
]},
|
||||
{ table: "scheduler_runs", timeCol: "started_at", retentionDays: 30, columns: [
|
||||
["id", "text"], ["job_key", "text"], ["started_at", "ts"], ["finished_at", "ts"],
|
||||
["status", "text"], ["error", "text"], ["duration_ms", "int"], ["result_json", "json-null"],
|
||||
]},
|
||||
{ table: "events", timeCol: "created_at", retentionDays: 30, columns: [
|
||||
["id", "text"], ["created_at", "ts"], ["level", "text"], ["event_type", "text"],
|
||||
["source_module", "text"], ["title", "text"], ["message", "text"],
|
||||
["entity_type", "text"], ["entity_id", "text"], ["payload_json", "json-null"],
|
||||
]},
|
||||
{ table: "internet_path_snapshots", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["sampled_at", "ts"], ["payload_json", "json"],
|
||||
]},
|
||||
]
|
||||
|
||||
function parseTs(value: unknown, strict: boolean, rejects: string[], ctx: string): string | null {
|
||||
if (value == null || value === "") return null
|
||||
const s = String(value)
|
||||
try {
|
||||
if (/^\d{4}-\d{2}-\d{2}T/.test(s)) return new Date(s).toISOString()
|
||||
if (/^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/.test(s)) {
|
||||
return new Date(`${s.replace(" ", "T")}Z`).toISOString()
|
||||
}
|
||||
const d = new Date(s)
|
||||
if (!Number.isNaN(d.getTime())) return d.toISOString()
|
||||
} catch {
|
||||
/* fallthrough */
|
||||
}
|
||||
const msg = `${ctx}: неразобранный timestamp ${s}`
|
||||
if (strict) throw new Error(msg)
|
||||
rejects.push(msg)
|
||||
return null
|
||||
}
|
||||
|
||||
function parseDate(value: unknown): string | null {
|
||||
if (value == null || value === "") return null
|
||||
return String(value).slice(0, 10)
|
||||
}
|
||||
|
||||
function parseBool(value: unknown): boolean {
|
||||
return value === true || value === 1 || value === "1" || value === "true"
|
||||
}
|
||||
|
||||
function parseJson(value: unknown, fallback: unknown): unknown {
|
||||
if (value == null || value === "") return fallback
|
||||
if (typeof value !== "string") return value
|
||||
try {
|
||||
return JSON.parse(value) as unknown
|
||||
} catch {
|
||||
return fallback
|
||||
}
|
||||
}
|
||||
|
||||
function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[], ctx: string): unknown {
|
||||
switch (kind) {
|
||||
case "ts":
|
||||
return parseTs(value, strict, rejects, ctx)
|
||||
case "date":
|
||||
return parseDate(value)
|
||||
case "bool":
|
||||
return parseBool(value)
|
||||
case "json":
|
||||
return parseJson(value, [])
|
||||
case "json-null":
|
||||
return value == null || value === "" ? null : parseJson(value, null)
|
||||
case "bigint-id": {
|
||||
const t = String(value ?? "").trim()
|
||||
if (!t) return null
|
||||
const n = Number(t)
|
||||
if (!Number.isFinite(n)) {
|
||||
const msg = `${ctx}: server_id ${t}`
|
||||
if (strict) throw new Error(msg)
|
||||
rejects.push(msg)
|
||||
return null
|
||||
}
|
||||
return n
|
||||
}
|
||||
case "int":
|
||||
if (value == null || value === "") return null
|
||||
return Number(value)
|
||||
case "num":
|
||||
if (value == null || value === "") return null
|
||||
return Number(value)
|
||||
case "text":
|
||||
return value == null ? "" : String(value)
|
||||
default:
|
||||
return value == null ? null : String(value)
|
||||
}
|
||||
}
|
||||
|
||||
function sqliteTableExists(sqlite: Database.Database, name: string): boolean {
|
||||
const row = sqlite.prepare(
|
||||
`SELECT 1 AS ok FROM sqlite_master WHERE type = 'table' AND name = ?`,
|
||||
).get(name) as { ok?: number } | undefined
|
||||
return Boolean(row)
|
||||
}
|
||||
|
||||
async function setval(pool: Pool, table: string): Promise<void> {
|
||||
await pool.query(
|
||||
`SELECT setval(
|
||||
pg_get_serial_sequence($1, 'id'),
|
||||
GREATEST(COALESCE((SELECT MAX(id) FROM ${table}), 1), 1),
|
||||
true
|
||||
)`,
|
||||
[table],
|
||||
)
|
||||
}
|
||||
|
||||
async function copyTable(
|
||||
sqlite: Database.Database,
|
||||
pool: Pool,
|
||||
spec: TableCopy,
|
||||
opts: { strict: boolean; fullHistory: boolean; rejects: string[] },
|
||||
): Promise<{ sqlite: number; copied: number; skipped: number }> {
|
||||
if (!sqliteTableExists(sqlite, spec.table)) {
|
||||
return { sqlite: 0, copied: 0, skipped: 0 }
|
||||
}
|
||||
let where = ""
|
||||
if (spec.timeCol && spec.retentionDays && !opts.fullHistory) {
|
||||
const cutoff = new Date(Date.now() - spec.retentionDays * 86400_000).toISOString()
|
||||
where = ` WHERE ${spec.timeCol} >= '${cutoff.replace("T", " ").slice(0, 19)}' OR ${spec.timeCol} >= '${cutoff}'`
|
||||
}
|
||||
const total = (sqlite.prepare(`SELECT COUNT(*) AS c FROM ${spec.table}${where}`).get() as { c: number }).c
|
||||
const part = specForParent(spec.table)
|
||||
const cols = spec.columns.map(([c]) => c)
|
||||
const placeholders = cols.map((_, i) => `$${i + 1}`).join(", ")
|
||||
const conflictSql = spec.upsert
|
||||
? `ON CONFLICT (id) DO UPDATE SET ${cols.filter((c) => c !== "id").map((c) => `${c} = EXCLUDED.${c}`).join(", ")}`
|
||||
: `ON CONFLICT DO NOTHING`
|
||||
const insertSql = `INSERT INTO ${spec.table} (${cols.join(", ")}) VALUES (${placeholders}) ${conflictSql}`
|
||||
let copied = 0
|
||||
let skipped = 0
|
||||
const stmt = sqlite.prepare(`SELECT * FROM ${spec.table}${where}`)
|
||||
const batch: unknown[][] = []
|
||||
const flush = async () => {
|
||||
if (batch.length === 0) return
|
||||
const client = await pool.connect()
|
||||
try {
|
||||
await client.query("BEGIN")
|
||||
for (const values of batch) {
|
||||
await client.query(insertSql, values)
|
||||
copied += 1
|
||||
}
|
||||
await client.query("COMMIT")
|
||||
} catch (err) {
|
||||
await client.query("ROLLBACK")
|
||||
throw err
|
||||
} finally {
|
||||
client.release()
|
||||
batch.length = 0
|
||||
}
|
||||
}
|
||||
for (const row of stmt.iterate() as Iterable<Record<string, unknown>>) {
|
||||
try {
|
||||
if (part && spec.timeCol) {
|
||||
const raw = row[spec.timeCol]
|
||||
const ts = spec.columns.find((c) => c[0] === spec.timeCol)?.[1] === "date"
|
||||
? `${String(raw).slice(0, 10)}T00:00:00Z`
|
||||
: parseTs(raw, false, opts.rejects, spec.table)
|
||||
if (ts) await ensurePartitionFor(pool, spec.table, part.kind, new Date(ts))
|
||||
}
|
||||
const values = spec.columns.map(([col, kind]) =>
|
||||
coerce(kind, row[col], opts.strict, opts.rejects, `${spec.table}.${col}`),
|
||||
)
|
||||
if (spec.table === "certificate_issue_jobs" && values[4] == null) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
batch.push(values)
|
||||
if (batch.length >= 200) await flush()
|
||||
} catch (err) {
|
||||
skipped += 1
|
||||
const msg = `${spec.table}: ${err instanceof Error ? err.message : String(err)}`
|
||||
opts.rejects.push(msg)
|
||||
if (opts.strict) throw err
|
||||
}
|
||||
}
|
||||
await flush()
|
||||
if (spec.identity) {
|
||||
try {
|
||||
await setval(pool, spec.table)
|
||||
} catch {
|
||||
/* partitioned identity sequence name may differ */
|
||||
}
|
||||
}
|
||||
return { sqlite: total, copied, skipped }
|
||||
}
|
||||
|
||||
export function sqliteFileLooksPresent(path: string): boolean {
|
||||
if (!existsSync(path) || path === ":memory:") return false
|
||||
try {
|
||||
const buf = readFileSync(path)
|
||||
return buf.subarray(0, 16).toString("utf8").startsWith("SQLite format 3")
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
export async function shouldImportSqlite(pool: Pool, sqlitePath: string): Promise<boolean> {
|
||||
if (!sqliteFileLooksPresent(sqlitePath)) return false
|
||||
const marker = await pool.query<{ sqlite_imported_at: string | null }>(
|
||||
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
|
||||
)
|
||||
if (marker.rows[0]?.sqlite_imported_at) return false
|
||||
const servers = await pool.query<{ c: string }>(`SELECT COUNT(*)::text AS c FROM servers`)
|
||||
if (Number(servers.rows[0]?.c ?? 0) > 0) return false
|
||||
return true
|
||||
}
|
||||
|
||||
export async function importSqliteToPostgres(
|
||||
pool: Pool,
|
||||
sqlitePath: string,
|
||||
opts?: { dryRun?: boolean; strict?: boolean; fullHistory?: boolean },
|
||||
): Promise<ImportReport> {
|
||||
const started = Date.now()
|
||||
const rejects: string[] = []
|
||||
const sqlite = new Database(sqlitePath, { readonly: true, fileMustExist: true })
|
||||
try {
|
||||
sqlite.pragma("journal_mode = WAL")
|
||||
} catch {
|
||||
/* readonly */
|
||||
}
|
||||
const sha = createHash("sha256").update(readFileSync(sqlitePath)).digest("hex")
|
||||
const report: ImportReport = {
|
||||
sqlitePath,
|
||||
sqliteSha256: sha,
|
||||
tables: {},
|
||||
rejects,
|
||||
durationMs: 0,
|
||||
}
|
||||
const fullHistory = opts?.fullHistory ?? env.sqliteImportFullHistory
|
||||
const strict = opts?.strict ?? true
|
||||
if (opts?.dryRun) {
|
||||
for (const spec of TABLES) {
|
||||
if (!sqliteTableExists(sqlite, spec.table)) {
|
||||
report.tables[spec.table] = { sqlite: 0, copied: 0, skipped: 0 }
|
||||
continue
|
||||
}
|
||||
const c = (sqlite.prepare(`SELECT COUNT(*) AS c FROM ${spec.table}`).get() as { c: number }).c
|
||||
report.tables[spec.table] = { sqlite: c, copied: 0, skipped: 0 }
|
||||
}
|
||||
sqlite.close()
|
||||
report.durationMs = Date.now() - started
|
||||
return report
|
||||
}
|
||||
for (const spec of TABLES) {
|
||||
report.tables[spec.table] = await copyTable(sqlite, pool, spec, { strict, fullHistory, rejects })
|
||||
}
|
||||
sqlite.close()
|
||||
await pool.query(
|
||||
`UPDATE data_migration
|
||||
SET sqlite_imported_at = now(), sqlite_path = $1, sqlite_sha256 = $2, report_json = $3::jsonb
|
||||
WHERE id = 1`,
|
||||
[sqlitePath, sha, JSON.stringify(report.tables)],
|
||||
)
|
||||
report.durationMs = Date.now() - started
|
||||
if (strict && rejects.length > 0) {
|
||||
throw new Error(`SQLite import strict: ${rejects.length} rejects\n${rejects.slice(0, 20).join("\n")}`)
|
||||
}
|
||||
return report
|
||||
}
|
||||
@@ -3,6 +3,8 @@ import cors from "@fastify/cors"
|
||||
import { serializerCompiler, validatorCompiler } from "@fastify/type-provider-zod"
|
||||
import { monitorEventLoopDelay } from "node:perf_hooks"
|
||||
import { env } from "./config.js"
|
||||
import { initDatabase } from "./db/bootstrap.js"
|
||||
import { closePool } from "./db/index.js"
|
||||
import authPlugin, { requireAuth } from "./plugins/auth.js"
|
||||
import serversRoutes from "./routes/servers.js"
|
||||
import bgpRoutes from "./routes/bgp.js"
|
||||
@@ -132,8 +134,8 @@ export async function buildApp(opts?: {
|
||||
await app.register(usersRoutes, { prefix: "/api" })
|
||||
|
||||
if (opts?.startScheduler !== false) {
|
||||
refreshScheduler()
|
||||
startTrafficFlowListener()
|
||||
await refreshScheduler()
|
||||
await startTrafficFlowListener()
|
||||
app.addHook("onClose", async () => {
|
||||
stopScheduler()
|
||||
stopTrafficFlowListener()
|
||||
@@ -149,6 +151,7 @@ const isMain =
|
||||
|
||||
if (isMain) {
|
||||
try {
|
||||
await initDatabase()
|
||||
const app = await buildApp()
|
||||
let shuttingDown = false
|
||||
const shutdown = async (code: number) => {
|
||||
@@ -157,6 +160,7 @@ if (isMain) {
|
||||
try {
|
||||
stopTrafficFlowListener()
|
||||
await app.close()
|
||||
await closePool()
|
||||
} catch (err) {
|
||||
console.error(err)
|
||||
} finally {
|
||||
|
||||
@@ -24,18 +24,7 @@ export type ListEventsParams = {
|
||||
to?: string
|
||||
}
|
||||
|
||||
function parsePayload(payloadJson: string | null): Record<string, unknown> {
|
||||
if (!payloadJson) return {}
|
||||
try {
|
||||
const parsed = JSON.parse(payloadJson) as unknown
|
||||
if (parsed && typeof parsed === "object" && !Array.isArray(parsed)) {
|
||||
return parsed as Record<string, unknown>
|
||||
}
|
||||
} catch {
|
||||
// keep backwards compatibility with malformed legacy payloads
|
||||
}
|
||||
return {}
|
||||
}
|
||||
import { parseJsonObject } from "../../../db/json.js"
|
||||
|
||||
function mapEventRow(row: typeof events.$inferSelect): EventItem {
|
||||
return {
|
||||
@@ -48,13 +37,13 @@ function mapEventRow(row: typeof events.$inferSelect): EventItem {
|
||||
message: row.message,
|
||||
entityType: row.entityType ?? null,
|
||||
entityId: row.entityId ?? null,
|
||||
payload: parsePayload(row.payloadJson ?? null),
|
||||
payload: parseJsonObject(row.payloadJson),
|
||||
}
|
||||
}
|
||||
|
||||
export function insertEventsBatch(items: EventInsertInput[]) {
|
||||
export async function insertEventsBatch(items: EventInsertInput[]) {
|
||||
if (items.length === 0) return
|
||||
db.insert(events)
|
||||
await db.insert(events)
|
||||
.values(
|
||||
items.map((item) => ({
|
||||
id: item.id,
|
||||
@@ -66,25 +55,23 @@ export function insertEventsBatch(items: EventInsertInput[]) {
|
||||
message: item.message,
|
||||
entityType: item.entityType ?? null,
|
||||
entityId: item.entityId ?? null,
|
||||
payloadJson: item.payload ? JSON.stringify(item.payload) : null,
|
||||
payloadJson: item.payload ?? null,
|
||||
})),
|
||||
)
|
||||
.run()
|
||||
}
|
||||
|
||||
export function listEvents(params: ListEventsParams): EventItem[] {
|
||||
export async function listEvents(params: ListEventsParams): Promise<EventItem[]> {
|
||||
const where = and(
|
||||
params.level ? eq(events.level, params.level) : undefined,
|
||||
params.sourceModule ? eq(events.sourceModule, params.sourceModule) : undefined,
|
||||
params.from ? gte(events.createdAt, params.from) : undefined,
|
||||
params.to ? lte(events.createdAt, params.to) : undefined,
|
||||
)
|
||||
const rows = db
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(events)
|
||||
.where(where)
|
||||
.orderBy(desc(events.createdAt))
|
||||
.limit(params.limit)
|
||||
.all()
|
||||
return rows.map(mapEventRow)
|
||||
}
|
||||
|
||||
@@ -23,17 +23,17 @@ function normalizeEventInput(input: AppendEventBody): EventInsertInput {
|
||||
}
|
||||
}
|
||||
|
||||
export function appendEvent(input: AppendEventBody) {
|
||||
export async function appendEvent(input: AppendEventBody) {
|
||||
const parsed = appendEventSchema.parse(input)
|
||||
insertEventsBatch([normalizeEventInput(parsed)])
|
||||
await insertEventsBatch([normalizeEventInput(parsed)])
|
||||
}
|
||||
|
||||
export function appendEvents(inputs: AppendEventBody[]) {
|
||||
export async function appendEvents(inputs: AppendEventBody[]) {
|
||||
const rows = inputs.map((entry) => normalizeEventInput(appendEventSchema.parse(entry)))
|
||||
insertEventsBatch(rows)
|
||||
await insertEventsBatch(rows)
|
||||
}
|
||||
|
||||
export function readEvents(query: Partial<ListEventsQuery>): EventItem[] {
|
||||
export async function readEvents(query: Partial<ListEventsQuery>): Promise<EventItem[]> {
|
||||
const parsed = listEventsQuerySchema.parse(query)
|
||||
return listEvents(parsed)
|
||||
return await listEvents(parsed)
|
||||
}
|
||||
|
||||
@@ -1,29 +1,24 @@
|
||||
import { parseJsonArray } from "../../../db/json.js"
|
||||
import type { ServerRead, WanUplinkRead } from "../../../types/server.js"
|
||||
import type { ServerRow, SnapshotRow } from "../repository/servers-repository.js"
|
||||
|
||||
function parseWanUplinksJson(raw: string | null | undefined): WanUplinkRead[] {
|
||||
if (raw == null || raw === "") return []
|
||||
try {
|
||||
const data = JSON.parse(raw) as unknown
|
||||
if (!Array.isArray(data)) return []
|
||||
const out: WanUplinkRead[] = []
|
||||
for (const row of data) {
|
||||
if (typeof row !== "object" || row === null) continue
|
||||
const r = row as Record<string, unknown>
|
||||
out.push({
|
||||
id: typeof r.id === "string" ? r.id : "",
|
||||
name: typeof r.name === "string" ? r.name : "",
|
||||
isp: typeof r.isp === "string" ? r.isp : "",
|
||||
iface: typeof r.iface === "string" ? r.iface : "",
|
||||
ip: typeof r.ip === "string" ? r.ip : "",
|
||||
maxDl: typeof r.maxDl === "number" && Number.isFinite(r.maxDl) ? r.maxDl : 0,
|
||||
maxUl: typeof r.maxUl === "number" && Number.isFinite(r.maxUl) ? r.maxUl : 0,
|
||||
})
|
||||
}
|
||||
return out
|
||||
} catch {
|
||||
return []
|
||||
function parseWanUplinksJson(raw: unknown): WanUplinkRead[] {
|
||||
const data = parseJsonArray(raw)
|
||||
const out: WanUplinkRead[] = []
|
||||
for (const row of data) {
|
||||
if (typeof row !== "object" || row === null) continue
|
||||
const r = row as Record<string, unknown>
|
||||
out.push({
|
||||
id: typeof r.id === "string" ? r.id : "",
|
||||
name: typeof r.name === "string" ? r.name : "",
|
||||
isp: typeof r.isp === "string" ? r.isp : "",
|
||||
iface: typeof r.iface === "string" ? r.iface : "",
|
||||
ip: typeof r.ip === "string" ? r.ip : "",
|
||||
maxDl: typeof r.maxDl === "number" && Number.isFinite(r.maxDl) ? r.maxDl : 0,
|
||||
maxUl: typeof r.maxUl === "number" && Number.isFinite(r.maxUl) ? r.maxUl : 0,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function toServerRead(server: ServerRow, snap: SnapshotRow | undefined): ServerRead {
|
||||
|
||||
@@ -1,53 +1,57 @@
|
||||
import { desc, eq } from "drizzle-orm"
|
||||
import { db } from "../../../db/index.js"
|
||||
import { serverSnapshots, servers } from "../../../db/schema.js"
|
||||
import { invalidateFlowCatalogCache } from "../../../services/traffic-flow-topology.js"
|
||||
|
||||
export type ServerRow = typeof servers.$inferSelect
|
||||
export type SnapshotRow = typeof serverSnapshots.$inferSelect
|
||||
|
||||
export function listServerRows(): ServerRow[] {
|
||||
return db.select().from(servers).all()
|
||||
export async function listServerRows(): Promise<ServerRow[]> {
|
||||
return await db.select().from(servers)
|
||||
}
|
||||
|
||||
export function getServerRowById(id: number): ServerRow | undefined {
|
||||
return db.select().from(servers).where(eq(servers.id, id)).limit(1).all()[0]
|
||||
export async function getServerRowById(id: number): Promise<ServerRow | undefined> {
|
||||
const rows = await db.select().from(servers).where(eq(servers.id, id)).limit(1)
|
||||
return rows[0]
|
||||
}
|
||||
|
||||
export function createServerRow(
|
||||
export async function createServerRow(
|
||||
values: Omit<typeof servers.$inferInsert, "id">,
|
||||
): ServerRow {
|
||||
const [inserted] = db.insert(servers).values(values).returning().all()
|
||||
): Promise<ServerRow> {
|
||||
const [inserted] = await db.insert(servers).values(values).returning()
|
||||
invalidateFlowCatalogCache()
|
||||
return inserted
|
||||
}
|
||||
|
||||
export function updateServerRowById(
|
||||
export async function updateServerRowById(
|
||||
id: number,
|
||||
values: Partial<ServerRow>,
|
||||
): ServerRow {
|
||||
const [updated] = db.update(servers).set(values).where(eq(servers.id, id)).returning().all()
|
||||
): Promise<ServerRow> {
|
||||
const [updated] = await db.update(servers).set(values).where(eq(servers.id, id)).returning()
|
||||
invalidateFlowCatalogCache()
|
||||
return updated
|
||||
}
|
||||
|
||||
export function deleteServerRowById(id: number): void {
|
||||
db.delete(servers).where(eq(servers.id, id)).run()
|
||||
export async function deleteServerRowById(id: number): Promise<void> {
|
||||
await db.delete(servers).where(eq(servers.id, id))
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
export function listSnapshotsByServerId(serverId: number, limit: number): SnapshotRow[] {
|
||||
export async function listSnapshotsByServerId(serverId: number, limit: number): Promise<SnapshotRow[]> {
|
||||
return db
|
||||
.select()
|
||||
.from(serverSnapshots)
|
||||
.where(eq(serverSnapshots.serverId, serverId))
|
||||
.orderBy(desc(serverSnapshots.polledAt))
|
||||
.limit(limit)
|
||||
.all()
|
||||
}
|
||||
|
||||
export function getLatestSnapshot(serverId: number): SnapshotRow | undefined {
|
||||
return db
|
||||
export async function getLatestSnapshot(serverId: number): Promise<SnapshotRow | undefined> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(serverSnapshots)
|
||||
.where(eq(serverSnapshots.serverId, serverId))
|
||||
.orderBy(desc(serverSnapshots.polledAt))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
return rows[0]
|
||||
}
|
||||
|
||||
@@ -11,45 +11,46 @@ import {
|
||||
import { toServerRead } from "../mapper/servers-mapper.js"
|
||||
import type { ServerCreate, ServerRead, ServerUpdate, SnapshotRead } from "../../../types/server.js"
|
||||
|
||||
export function listServersRead(): ServerRead[] {
|
||||
return listServerRows().map((server) => toServerRead(server, getLatestSnapshot(server.id)))
|
||||
export async function listServersRead(): Promise<ServerRead[]> {
|
||||
const rows = await listServerRows()
|
||||
return Promise.all(rows.map(async (server) => toServerRead(server, await getLatestSnapshot(server.id))))
|
||||
}
|
||||
|
||||
export function getServerReadById(id: number): ServerRead | undefined {
|
||||
const row = getServerRowById(id)
|
||||
export async function getServerReadById(id: number): Promise<ServerRead | undefined> {
|
||||
const row = await getServerRowById(id)
|
||||
if (!row) return undefined
|
||||
return toServerRead(row, getLatestSnapshot(row.id))
|
||||
return toServerRead(row, await getLatestSnapshot(row.id))
|
||||
}
|
||||
|
||||
export function createServer(input: ServerCreate): ServerRead {
|
||||
export async function createServer(input: ServerCreate): Promise<ServerRead> {
|
||||
const now = new Date().toISOString()
|
||||
const { wanUplinks, ...rest } = input
|
||||
const inserted = createServerRow({
|
||||
const inserted = await createServerRow({
|
||||
...rest,
|
||||
wanUplinks: JSON.stringify(wanUplinks ?? []),
|
||||
wanUplinks: wanUplinks ?? [],
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
})
|
||||
return toServerRead(inserted, undefined)
|
||||
}
|
||||
|
||||
export function updateServer(id: number, input: ServerUpdate): ServerRead {
|
||||
export async function updateServer(id: number, input: ServerUpdate): Promise<ServerRead> {
|
||||
const { wanUplinks, ...rest } = input
|
||||
const setPayload: Record<string, unknown> = { updatedAt: new Date().toISOString() }
|
||||
for (const [k, v] of Object.entries(rest)) {
|
||||
if (v !== undefined) setPayload[k] = v
|
||||
}
|
||||
if (wanUplinks !== undefined) {
|
||||
setPayload.wanUplinks = JSON.stringify(wanUplinks)
|
||||
setPayload.wanUplinks = wanUplinks
|
||||
}
|
||||
const updated = updateServerRowById(id, setPayload as never)
|
||||
return toServerRead(updated, getLatestSnapshot(updated.id))
|
||||
const updated = await updateServerRowById(id, setPayload as never)
|
||||
return toServerRead(updated, await getLatestSnapshot(updated.id))
|
||||
}
|
||||
|
||||
export function deleteServer(id: number): void {
|
||||
deleteServerRowById(id)
|
||||
export async function deleteServer(id: number): Promise<void> {
|
||||
await deleteServerRowById(id)
|
||||
}
|
||||
|
||||
export function listServerSnapshots(id: number, limit: number): SnapshotRead[] {
|
||||
return listSnapshotsByServerId(id, limit).map(toSnapshotRead)
|
||||
export async function listServerSnapshots(id: number, limit: number): Promise<SnapshotRead[]> {
|
||||
return (await listSnapshotsByServerId(id, limit)).map(toSnapshotRead)
|
||||
}
|
||||
|
||||
@@ -1,79 +1,7 @@
|
||||
import assert from "node:assert/strict"
|
||||
import Database from "better-sqlite3"
|
||||
import { normalizeBindingPeer, PeerBindError } from "./peer-bind.js"
|
||||
|
||||
const sqlite = new Database(":memory:")
|
||||
sqlite.pragma("foreign_keys = ON")
|
||||
sqlite.exec(`
|
||||
CREATE TABLE servers (
|
||||
id INTEGER PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
host TEXT NOT NULL DEFAULT '127.0.0.1'
|
||||
);
|
||||
CREATE TABLE app_users (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
login TEXT NOT NULL UNIQUE,
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
role TEXT NOT NULL DEFAULT 'viewer',
|
||||
active INTEGER NOT NULL DEFAULT 1,
|
||||
avatar TEXT NOT NULL DEFAULT '',
|
||||
last_seen TEXT,
|
||||
sections_json TEXT NOT NULL DEFAULT '[]',
|
||||
servers_json TEXT NOT NULL DEFAULT '[]',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now'))
|
||||
);
|
||||
CREATE TABLE user_interface_bindings (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL,
|
||||
server_id INTEGER NOT NULL,
|
||||
interface_name TEXT NOT NULL,
|
||||
interface_type TEXT NOT NULL DEFAULT 'other',
|
||||
peer_public_key TEXT NOT NULL DEFAULT '',
|
||||
peer_name TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
created_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
updated_at TEXT NOT NULL DEFAULT (datetime('now')),
|
||||
FOREIGN KEY (user_id) REFERENCES app_users(id) ON DELETE CASCADE,
|
||||
FOREIGN KEY (server_id) REFERENCES servers(id) ON DELETE CASCADE,
|
||||
UNIQUE (server_id, interface_name, peer_public_key)
|
||||
);
|
||||
`)
|
||||
|
||||
sqlite.prepare("INSERT INTO servers (id, name, host) VALUES (1, 'jh', '10.0.0.1')").run()
|
||||
sqlite.prepare("INSERT INTO app_users (id, name, login) VALUES ('u1', 'A', 'a.user')").run()
|
||||
sqlite.prepare("INSERT INTO app_users (id, name, login) VALUES ('u2', 'B', 'b.user')").run()
|
||||
sqlite.prepare(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('b1', 'u1', 1, 'gre-office', 'gre')
|
||||
`).run()
|
||||
|
||||
assert.throws(
|
||||
() => sqlite.prepare(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('b2', 'u2', 1, 'gre-office', 'gre')
|
||||
`).run(),
|
||||
/UNIQUE/i,
|
||||
"один интерфейс на сервере — один пользователь",
|
||||
)
|
||||
|
||||
sqlite.prepare(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type, peer_public_key, peer_name)
|
||||
VALUES ('wg1', 'u1', 1, 'wg-server', 'wg', 'peer-key-aaa', 'phone')
|
||||
`).run()
|
||||
sqlite.prepare(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type, peer_public_key, peer_name)
|
||||
VALUES ('wg2', 'u2', 1, 'wg-server', 'wg', 'peer-key-bbb', 'laptop')
|
||||
`).run()
|
||||
assert.throws(
|
||||
() => sqlite.prepare(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type, peer_public_key)
|
||||
VALUES ('wg3', 'u2', 1, 'wg-server', 'wg', 'peer-key-aaa')
|
||||
`).run(),
|
||||
/UNIQUE/i,
|
||||
"один пир — один пользователь",
|
||||
)
|
||||
import { withPgOrSkip } from "../../test/pg.js"
|
||||
import { dbQuery } from "../../db/index.js"
|
||||
|
||||
assert.throws(
|
||||
() => normalizeBindingPeer("wg", ""),
|
||||
@@ -83,8 +11,64 @@ assert.throws(
|
||||
assert.equal(normalizeBindingPeer("ether", "ignored"), "")
|
||||
assert.equal(normalizeBindingPeer("wg", " abc "), "abc")
|
||||
|
||||
sqlite.prepare("DELETE FROM app_users WHERE id = 'u1'").run()
|
||||
const leftover = sqlite.prepare("SELECT COUNT(*) AS n FROM user_interface_bindings").get() as { n: number }
|
||||
assert.equal(leftover.n, 1, "каскад: привязки u1 удаляются, пир u2 остаётся")
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("users bindings unique+cascade tests skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO servers (id, name, host) VALUES (91001, 'jh-bind-test', '10.0.0.1')
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login)
|
||||
VALUES ('u-bind-1', 'A', 'a.bind.test'), ('u-bind-2', 'B', 'b.bind.test')
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = 91001`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('b-bind-1', 'u-bind-1', 91001, 'gre-office', 'gre')
|
||||
`)
|
||||
|
||||
let uniqueIface = false
|
||||
try {
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('b-bind-2', 'u-bind-2', 91001, 'gre-office', 'gre')
|
||||
`)
|
||||
} catch {
|
||||
uniqueIface = true
|
||||
}
|
||||
assert.equal(uniqueIface, true, "один интерфейс на сервере — один пользователь")
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type, peer_public_key, peer_name)
|
||||
VALUES ('wg-bind-1', 'u-bind-1', 91001, 'wg-server', 'wg', 'peer-key-aaa', 'phone')
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type, peer_public_key, peer_name)
|
||||
VALUES ('wg-bind-2', 'u-bind-2', 91001, 'wg-server', 'wg', 'peer-key-bbb', 'laptop')
|
||||
`)
|
||||
|
||||
let uniquePeer = false
|
||||
try {
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type, peer_public_key)
|
||||
VALUES ('wg-bind-3', 'u-bind-2', 91001, 'wg-server', 'wg', 'peer-key-aaa')
|
||||
`)
|
||||
} catch {
|
||||
uniquePeer = true
|
||||
}
|
||||
assert.equal(uniquePeer, true, "один пир — один пользователь")
|
||||
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-bind-1'`)
|
||||
const leftover = await dbQuery<{ n: number }>(
|
||||
`SELECT COUNT(*)::int AS n FROM user_interface_bindings WHERE server_id = 91001`,
|
||||
)
|
||||
assert.equal(leftover.rows[0]?.n, 1, "каскад: привязки u1 удаляются, пир u2 остаётся")
|
||||
|
||||
await dbQuery(`DELETE FROM app_users WHERE id IN ('u-bind-1', 'u-bind-2')`)
|
||||
await dbQuery(`DELETE FROM servers WHERE id = 91001`)
|
||||
|
||||
console.log("users bindings unique+cascade tests ok")
|
||||
|
||||
@@ -27,10 +27,10 @@ function asBool(raw: unknown): boolean {
|
||||
return s === "true" || s === "yes" || s === "1"
|
||||
}
|
||||
|
||||
export function parseRawInterfaces(json: string | null | undefined): ParsedRosIface[] {
|
||||
if (!json) return []
|
||||
export function parseRawInterfaces(json: unknown): ParsedRosIface[] {
|
||||
if (json == null || json === "") return []
|
||||
try {
|
||||
const parsed = JSON.parse(json) as unknown
|
||||
const parsed = typeof json === "string" ? JSON.parse(json) as unknown : json
|
||||
const arr = Array.isArray(parsed) ? parsed : []
|
||||
const out: ParsedRosIface[] = []
|
||||
for (const item of arr) {
|
||||
@@ -56,5 +56,5 @@ export function isUniqueConstraintError(err: unknown): boolean {
|
||||
const rec = err as { code?: unknown; message?: unknown }
|
||||
const code = String(rec.code ?? "")
|
||||
const msg = String(rec.message ?? "")
|
||||
return code.includes("SQLITE_CONSTRAINT") || /unique constraint/i.test(msg)
|
||||
return code === "23505" || code.includes("SQLITE_CONSTRAINT") || /unique constraint/i.test(msg)
|
||||
}
|
||||
|
||||
@@ -1,57 +1,64 @@
|
||||
import { and, eq } from "drizzle-orm"
|
||||
import { and, count, eq } from "drizzle-orm"
|
||||
import { db } from "../../../db/index.js"
|
||||
import { appUsers, userInterfaceBindings } from "../../../db/schema.js"
|
||||
import { invalidateFlowCatalogCache } from "../../../services/traffic-flow-topology.js"
|
||||
|
||||
export type AppUserRow = typeof appUsers.$inferSelect
|
||||
export type BindingRow = typeof userInterfaceBindings.$inferSelect
|
||||
|
||||
export function listUserRows(): AppUserRow[] {
|
||||
return db.select().from(appUsers).all()
|
||||
export async function listUserRows(): Promise<AppUserRow[]> {
|
||||
return await db.select().from(appUsers)
|
||||
}
|
||||
|
||||
export function getUserRowById(id: string): AppUserRow | undefined {
|
||||
return db.select().from(appUsers).where(eq(appUsers.id, id)).limit(1).all()[0]
|
||||
export async function getUserRowById(id: string): Promise<AppUserRow | undefined> {
|
||||
const rows = await db.select().from(appUsers).where(eq(appUsers.id, id)).limit(1)
|
||||
return rows[0]
|
||||
}
|
||||
|
||||
export function getUserRowByLogin(login: string): AppUserRow | undefined {
|
||||
return db.select().from(appUsers).where(eq(appUsers.login, login)).limit(1).all()[0]
|
||||
export async function getUserRowByLogin(login: string): Promise<AppUserRow | undefined> {
|
||||
const rows = await db.select().from(appUsers).where(eq(appUsers.login, login)).limit(1)
|
||||
return rows[0]
|
||||
}
|
||||
|
||||
export function createUserRow(values: typeof appUsers.$inferInsert): AppUserRow {
|
||||
const [inserted] = db.insert(appUsers).values(values).returning().all()
|
||||
export async function createUserRow(values: typeof appUsers.$inferInsert): Promise<AppUserRow> {
|
||||
const [inserted] = await db.insert(appUsers).values(values).returning()
|
||||
invalidateFlowCatalogCache()
|
||||
return inserted
|
||||
}
|
||||
|
||||
export function updateUserRowById(
|
||||
export async function updateUserRowById(
|
||||
id: string,
|
||||
values: Partial<AppUserRow>,
|
||||
): AppUserRow {
|
||||
const [updated] = db.update(appUsers).set(values).where(eq(appUsers.id, id)).returning().all()
|
||||
): Promise<AppUserRow> {
|
||||
const [updated] = await db.update(appUsers).set(values).where(eq(appUsers.id, id)).returning()
|
||||
invalidateFlowCatalogCache()
|
||||
return updated
|
||||
}
|
||||
|
||||
export function deleteUserRowById(id: string): void {
|
||||
db.delete(appUsers).where(eq(appUsers.id, id)).run()
|
||||
export async function deleteUserRowById(id: string): Promise<void> {
|
||||
await db.delete(appUsers).where(eq(appUsers.id, id))
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
export function listBindingRows(): BindingRow[] {
|
||||
return db.select().from(userInterfaceBindings).all()
|
||||
export async function listBindingRows(): Promise<BindingRow[]> {
|
||||
return await db.select().from(userInterfaceBindings)
|
||||
}
|
||||
|
||||
export function listBindingRowsByUser(userId: string): BindingRow[] {
|
||||
return db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId)).all()
|
||||
export async function listBindingRowsByUser(userId: string): Promise<BindingRow[]> {
|
||||
return await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
|
||||
}
|
||||
|
||||
export function getBindingRowById(id: string): BindingRow | undefined {
|
||||
return db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.id, id)).limit(1).all()[0]
|
||||
export async function getBindingRowById(id: string): Promise<BindingRow | undefined> {
|
||||
const rows = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.id, id)).limit(1)
|
||||
return rows[0]
|
||||
}
|
||||
|
||||
export function getBindingByServerIfacePeer(
|
||||
export async function getBindingByServerIfacePeer(
|
||||
serverId: number,
|
||||
interfaceName: string,
|
||||
peerPublicKey = "",
|
||||
): BindingRow | undefined {
|
||||
return db
|
||||
): Promise<BindingRow | undefined> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(userInterfaceBindings)
|
||||
.where(and(
|
||||
@@ -60,18 +67,21 @@ export function getBindingByServerIfacePeer(
|
||||
eq(userInterfaceBindings.peerPublicKey, peerPublicKey),
|
||||
))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
return rows[0]
|
||||
}
|
||||
|
||||
export function createBindingRow(values: typeof userInterfaceBindings.$inferInsert): BindingRow {
|
||||
const [inserted] = db.insert(userInterfaceBindings).values(values).returning().all()
|
||||
export async function createBindingRow(values: typeof userInterfaceBindings.$inferInsert): Promise<BindingRow> {
|
||||
const [inserted] = await db.insert(userInterfaceBindings).values(values).returning()
|
||||
invalidateFlowCatalogCache()
|
||||
return inserted
|
||||
}
|
||||
|
||||
export function deleteBindingRowById(id: string): void {
|
||||
db.delete(userInterfaceBindings).where(eq(userInterfaceBindings.id, id)).run()
|
||||
export async function deleteBindingRowById(id: string): Promise<void> {
|
||||
await db.delete(userInterfaceBindings).where(eq(userInterfaceBindings.id, id))
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
export function countUserRows(): number {
|
||||
return db.select().from(appUsers).all().length
|
||||
export async function countUserRows(): Promise<number> {
|
||||
const rows = await db.select({ n: count() }).from(appUsers)
|
||||
return rows[0]?.n ?? 0
|
||||
}
|
||||
|
||||
@@ -12,6 +12,7 @@ import type {
|
||||
UserBindingCreate,
|
||||
} from "@mmapp/contracts/users"
|
||||
import { db } from "../../../db/index.js"
|
||||
import { parseJsonArray } from "../../../db/json.js"
|
||||
import { servers, trafficSamples } from "../../../db/schema.js"
|
||||
import {
|
||||
createBindingRow,
|
||||
@@ -52,13 +53,9 @@ export class UsersServiceError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
function parseJsonArray<T>(raw: string, fallback: T[]): T[] {
|
||||
try {
|
||||
const parsed = JSON.parse(raw) as unknown
|
||||
return Array.isArray(parsed) ? (parsed as T[]) : fallback
|
||||
} catch {
|
||||
return fallback
|
||||
}
|
||||
function asPermArray<T>(raw: unknown, fallback: T[]): T[] {
|
||||
const arr = parseJsonArray(raw)
|
||||
return arr.length ? arr as T[] : fallback
|
||||
}
|
||||
|
||||
function initials(name: string): string {
|
||||
@@ -66,8 +63,9 @@ function initials(name: string): string {
|
||||
return parts.map((p) => p[0] ?? "").slice(0, 2).join("").toUpperCase() || "??"
|
||||
}
|
||||
|
||||
function serverMeta(serverId: number): { name: string; site: string; country: string } {
|
||||
const row = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
async function serverMeta(serverId: number): Promise<{ name: string; site: string; country: string }> {
|
||||
const rows = await db.select().from(servers).where(eq(servers.id, serverId)).limit(1)
|
||||
const row = rows[0]
|
||||
return {
|
||||
name: row?.name || row?.host || String(serverId),
|
||||
site: row?.site || "—",
|
||||
@@ -75,8 +73,8 @@ function serverMeta(serverId: number): { name: string; site: string; country: st
|
||||
}
|
||||
}
|
||||
|
||||
function toBindingDto(row: BindingRow): UserBinding {
|
||||
const meta = serverMeta(row.serverId)
|
||||
async function toBindingDto(row: BindingRow): Promise<UserBinding> {
|
||||
const meta = await serverMeta(row.serverId)
|
||||
return {
|
||||
id: row.id,
|
||||
userId: row.userId,
|
||||
@@ -94,7 +92,7 @@ function toBindingDto(row: BindingRow): UserBinding {
|
||||
}
|
||||
}
|
||||
|
||||
function toUserDto(row: AppUserRow, bindings: BindingRow[]): AppUserRead {
|
||||
async function toUserDto(row: AppUserRow, bindings: BindingRow[]): Promise<AppUserRead> {
|
||||
return {
|
||||
id: row.id,
|
||||
name: row.name,
|
||||
@@ -104,39 +102,39 @@ function toUserDto(row: AppUserRow, bindings: BindingRow[]): AppUserRead {
|
||||
active: Boolean(row.active),
|
||||
avatar: row.avatar,
|
||||
lastSeen: row.lastSeen ?? null,
|
||||
sections: parseJsonArray<SectionPerm>(row.sectionsJson, []),
|
||||
servers: parseJsonArray<ServerPerm>(row.serversJson, []),
|
||||
bindings: bindings.map(toBindingDto),
|
||||
sections: asPermArray<SectionPerm>(row.sectionsJson, []),
|
||||
servers: asPermArray<ServerPerm>(row.serversJson, []),
|
||||
bindings: await Promise.all(bindings.map((b) => toBindingDto(b))),
|
||||
createdAt: row.createdAt,
|
||||
updatedAt: row.updatedAt,
|
||||
}
|
||||
}
|
||||
|
||||
export function listUsers(): AppUserRead[] {
|
||||
const users = listUserRows()
|
||||
const allBindings = listBindingRows()
|
||||
export async function listUsers(): Promise<AppUserRead[]> {
|
||||
const users = await listUserRows()
|
||||
const allBindings = await listBindingRows()
|
||||
const byUser = new Map<string, BindingRow[]>()
|
||||
for (const b of allBindings) {
|
||||
const arr = byUser.get(b.userId) ?? []
|
||||
arr.push(b)
|
||||
byUser.set(b.userId, arr)
|
||||
}
|
||||
return users.map((u) => toUserDto(u, byUser.get(u.id) ?? []))
|
||||
return Promise.all(users.map((u) => toUserDto(u, byUser.get(u.id) ?? [])))
|
||||
}
|
||||
|
||||
export function getUserById(id: string): AppUserRead | undefined {
|
||||
const row = getUserRowById(id)
|
||||
export async function getUserById(id: string): Promise<AppUserRead | undefined> {
|
||||
const row = await getUserRowById(id)
|
||||
if (!row) return undefined
|
||||
return toUserDto(row, listBindingRowsByUser(id))
|
||||
return await toUserDto(row, await listBindingRowsByUser(id))
|
||||
}
|
||||
|
||||
export function createUser(input: AppUserCreate): AppUserRead {
|
||||
export async function createUser(input: AppUserCreate): Promise<AppUserRead> {
|
||||
const login = input.login.trim()
|
||||
if (getUserRowByLogin(login)) {
|
||||
if (await getUserRowByLogin(login)) {
|
||||
throw new UsersServiceError("Логин уже занят", 409)
|
||||
}
|
||||
const now = new Date().toISOString()
|
||||
const row = createUserRow({
|
||||
const row = await createUserRow({
|
||||
id: randomUUID(),
|
||||
name: input.name.trim(),
|
||||
login,
|
||||
@@ -145,19 +143,19 @@ export function createUser(input: AppUserCreate): AppUserRead {
|
||||
active: input.active ?? true,
|
||||
avatar: (input.avatar ?? "").trim() || initials(input.name),
|
||||
lastSeen: input.lastSeen ?? null,
|
||||
sectionsJson: JSON.stringify(input.sections ?? []),
|
||||
serversJson: JSON.stringify(input.servers ?? []),
|
||||
sectionsJson: input.sections ?? [],
|
||||
serversJson: input.servers ?? [],
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
})
|
||||
return toUserDto(row, [])
|
||||
return await toUserDto(row, [])
|
||||
}
|
||||
|
||||
export function updateUser(id: string, input: AppUserUpdate): AppUserRead {
|
||||
const existing = getUserRowById(id)
|
||||
export async function updateUser(id: string, input: AppUserUpdate): Promise<AppUserRead> {
|
||||
const existing = await getUserRowById(id)
|
||||
if (!existing) throw new UsersServiceError("Пользователь не найден", 404)
|
||||
if (input.login != null) {
|
||||
const other = getUserRowByLogin(input.login.trim())
|
||||
const other = await getUserRowByLogin(input.login.trim())
|
||||
if (other && other.id !== id) throw new UsersServiceError("Логин уже занят", 409)
|
||||
}
|
||||
const patch: Partial<AppUserRow> = { updatedAt: new Date().toISOString() }
|
||||
@@ -168,26 +166,26 @@ export function updateUser(id: string, input: AppUserUpdate): AppUserRead {
|
||||
if (input.active != null) patch.active = input.active
|
||||
if (input.avatar != null) patch.avatar = input.avatar.trim() || existing.avatar
|
||||
if (input.lastSeen !== undefined) patch.lastSeen = input.lastSeen
|
||||
if (input.sections != null) patch.sectionsJson = JSON.stringify(input.sections)
|
||||
if (input.servers != null) patch.serversJson = JSON.stringify(input.servers)
|
||||
const updated = updateUserRowById(id, patch)
|
||||
return toUserDto(updated, listBindingRowsByUser(id))
|
||||
if (input.sections != null) patch.sectionsJson = input.sections
|
||||
if (input.servers != null) patch.serversJson = input.servers
|
||||
const updated = await updateUserRowById(id, patch)
|
||||
return await toUserDto(updated, await listBindingRowsByUser(id))
|
||||
}
|
||||
|
||||
export function deleteUser(id: string): void {
|
||||
const existing = getUserRowById(id)
|
||||
export async function deleteUser(id: string): Promise<void> {
|
||||
const existing = await getUserRowById(id)
|
||||
if (!existing) throw new UsersServiceError("Пользователь не найден", 404)
|
||||
deleteUserRowById(id)
|
||||
await deleteUserRowById(id)
|
||||
}
|
||||
|
||||
export function addBinding(userId: string, input: UserBindingCreate): UserBinding {
|
||||
const user = getUserRowById(userId)
|
||||
export async function addBinding(userId: string, input: UserBindingCreate): Promise<UserBinding> {
|
||||
const user = await getUserRowById(userId)
|
||||
if (!user) throw new UsersServiceError("Пользователь не найден", 404)
|
||||
const server = db.select().from(servers).where(eq(servers.id, input.serverId)).limit(1).all()[0]
|
||||
if (!server) throw new UsersServiceError("Сервер не найден", 404)
|
||||
const serverRows = await db.select().from(servers).where(eq(servers.id, input.serverId)).limit(1)
|
||||
if (!serverRows[0]) throw new UsersServiceError("Сервер не найден", 404)
|
||||
const ifaceName = input.interfaceName.trim()
|
||||
if (!ifaceName) throw new UsersServiceError("Имя интерфейса обязательно", 400)
|
||||
const type: InterfaceType = input.interfaceType ?? inferIfaceType(input.serverId, ifaceName)
|
||||
const type: InterfaceType = input.interfaceType ?? await inferIfaceType(input.serverId, ifaceName)
|
||||
let peerPublicKey = ""
|
||||
try {
|
||||
peerPublicKey = normalizeBindingPeer(type, input.peerPublicKey)
|
||||
@@ -201,7 +199,7 @@ export function addBinding(userId: string, input: UserBindingCreate): UserBindin
|
||||
name: input.peerName,
|
||||
})
|
||||
: ""
|
||||
const taken = getBindingByServerIfacePeer(input.serverId, ifaceName, peerPublicKey)
|
||||
const taken = await getBindingByServerIfacePeer(input.serverId, ifaceName, peerPublicKey)
|
||||
if (taken) {
|
||||
throw new UsersServiceError(
|
||||
type === "wg"
|
||||
@@ -212,7 +210,7 @@ export function addBinding(userId: string, input: UserBindingCreate): UserBindin
|
||||
}
|
||||
const now = new Date().toISOString()
|
||||
try {
|
||||
const row = createBindingRow({
|
||||
const row = await createBindingRow({
|
||||
id: randomUUID(),
|
||||
userId,
|
||||
serverId: input.serverId,
|
||||
@@ -224,7 +222,7 @@ export function addBinding(userId: string, input: UserBindingCreate): UserBindin
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
})
|
||||
return toBindingDto(row)
|
||||
return await toBindingDto(row)
|
||||
} catch (err) {
|
||||
if (isUniqueConstraintError(err)) {
|
||||
throw new UsersServiceError(
|
||||
@@ -238,37 +236,37 @@ export function addBinding(userId: string, input: UserBindingCreate): UserBindin
|
||||
}
|
||||
}
|
||||
|
||||
export function removeBinding(userId: string, bindingId: string): void {
|
||||
const row = getBindingRowById(bindingId)
|
||||
export async function removeBinding(userId: string, bindingId: string): Promise<void> {
|
||||
const row = await getBindingRowById(bindingId)
|
||||
if (!row || row.userId !== userId) {
|
||||
throw new UsersServiceError("Привязка не найдена", 404)
|
||||
}
|
||||
deleteBindingRowById(bindingId)
|
||||
await deleteBindingRowById(bindingId)
|
||||
}
|
||||
|
||||
function inferIfaceType(serverId: number, ifaceName: string): InterfaceType {
|
||||
const snap = getLatestSnapshot(serverId)
|
||||
async function inferIfaceType(serverId: number, ifaceName: string): Promise<InterfaceType> {
|
||||
const snap = await getLatestSnapshot(serverId)
|
||||
const parsed = parseRawInterfaces(snap?.rawInterfaces)
|
||||
const found = parsed.find((i) => i.name === ifaceName)
|
||||
return found?.type ?? "other"
|
||||
}
|
||||
|
||||
export async function listInterfaceCatalog(serverId: number): Promise<CatalogInterface[]> {
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
if (!server) throw new UsersServiceError("Сервер не найден", 404)
|
||||
const serverRows = await db.select().from(servers).where(eq(servers.id, serverId)).limit(1)
|
||||
if (!serverRows[0]) throw new UsersServiceError("Сервер не найден", 404)
|
||||
|
||||
const snap = getLatestSnapshot(serverId)
|
||||
const snap = await getLatestSnapshot(serverId)
|
||||
let ifaces = parseRawInterfaces(snap?.rawInterfaces)
|
||||
if (ifaces.length === 0) {
|
||||
const last = db
|
||||
const lastRows = await db
|
||||
.select({ sampledAt: trafficSamples.sampledAt })
|
||||
.from(trafficSamples)
|
||||
.where(eq(trafficSamples.serverId, serverId))
|
||||
.orderBy(desc(trafficSamples.sampledAt))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
const last = lastRows[0]
|
||||
if (last) {
|
||||
const rows = db
|
||||
const rows = (await db
|
||||
.select({
|
||||
interfaceName: trafficSamples.interfaceName,
|
||||
peerPublicKey: trafficSamples.peerPublicKey,
|
||||
@@ -276,8 +274,7 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
|
||||
disabled: trafficSamples.disabled,
|
||||
})
|
||||
.from(trafficSamples)
|
||||
.where(eq(trafficSamples.serverId, serverId))
|
||||
.all()
|
||||
.where(eq(trafficSamples.serverId, serverId)))
|
||||
.filter((r) => r.interfaceName && !/^(lo|loopback)/i.test(r.interfaceName) && !(r.peerPublicKey ?? ""))
|
||||
const seen = new Set<string>()
|
||||
ifaces = []
|
||||
@@ -294,8 +291,8 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
|
||||
}
|
||||
}
|
||||
|
||||
const bindings = listBindingRows().filter((b) => b.serverId === serverId)
|
||||
const usersById = new Map(listUserRows().map((u) => [u.id, u]))
|
||||
const bindings = (await listBindingRows()).filter((b) => b.serverId === serverId)
|
||||
const usersById = new Map((await listUserRows()).map((u) => [u.id, u]))
|
||||
const hasWg = ifaces.some((i) => i.type === "wg")
|
||||
const wgLive = hasWg
|
||||
? await listWireGuardPeersForCatalog(serverId)
|
||||
|
||||
@@ -10,7 +10,6 @@ process.env.AUTH_JWT_SECRET = "test-secret-at-least-8"
|
||||
process.env.AUTH_ISSUER = "https://auth.test.local"
|
||||
process.env.AUTH_PORTAL_URL = "http://localhost:5175"
|
||||
process.env.CORS_ORIGIN = "http://localhost:3000"
|
||||
process.env.DATABASE_PATH = ":memory:"
|
||||
process.env.NODE_ENV = "test"
|
||||
|
||||
// Dynamic import after env is set
|
||||
|
||||
@@ -20,14 +20,14 @@ import {
|
||||
|
||||
const alertsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/alerts", async (_req, reply) => {
|
||||
const tg = getTelegramPublic()
|
||||
const tg = await getTelegramPublic()
|
||||
const connected = tg.tokenConfigured && Boolean(tg.chatId?.trim())
|
||||
return reply.send({
|
||||
telegram: { ...tg, connected },
|
||||
groups: listAlertGroups(),
|
||||
rules: listAlertRules(),
|
||||
history: listMergedHistory(),
|
||||
meta: getAlertsMeta(),
|
||||
groups: await listAlertGroups(),
|
||||
rules: await listAlertRules(),
|
||||
history: await listMergedHistory(),
|
||||
meta: await getAlertsMeta(),
|
||||
})
|
||||
})
|
||||
|
||||
@@ -38,7 +38,7 @@ const alertsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
const groups =
|
||||
parsed.data.groups ??
|
||||
listAlertGroups().map((g) => ({
|
||||
(await listAlertGroups()).map((g) => ({
|
||||
id: g.id,
|
||||
name: g.name,
|
||||
combineMode: g.combineMode,
|
||||
@@ -67,8 +67,8 @@ const alertsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
recoveryStabilitySec: r.recoveryStabilitySec ?? null,
|
||||
chatId: r.chatId,
|
||||
}))
|
||||
replaceAlertsConfig({ groups, rules })
|
||||
return reply.send({ ok: true, groups: listAlertGroups(), rules: listAlertRules() })
|
||||
await replaceAlertsConfig({ groups, rules })
|
||||
return reply.send({ ok: true, groups: await listAlertGroups(), rules: await listAlertRules() })
|
||||
})
|
||||
|
||||
app.put("/alerts/telegram", async (req, reply) => {
|
||||
@@ -76,7 +76,7 @@ const alertsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const next = updateTelegramSettings({
|
||||
const next = await updateTelegramSettings({
|
||||
token: parsed.data.token,
|
||||
chatId: parsed.data.chatId,
|
||||
messageThreadId: parsed.data.messageThreadId,
|
||||
@@ -90,9 +90,9 @@ const alertsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const fromDb = getTelegramBotToken()
|
||||
const fromDb = await getTelegramBotToken()
|
||||
const token = (parsed.data.token?.trim() || fromDb).trim()
|
||||
const pub = getTelegramPublic()
|
||||
const pub = await getTelegramPublic()
|
||||
const chatId = (parsed.data.chatId?.trim() || pub.chatId || "").trim()
|
||||
if (!token) {
|
||||
return reply.status(400).send({ error: "Не задан Bot Token (сохраните в БД или передайте в запросе)" })
|
||||
@@ -103,7 +103,7 @@ const alertsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const threadId =
|
||||
parsed.data.messageThreadId != null && parsed.data.messageThreadId >= 1
|
||||
? parsed.data.messageThreadId
|
||||
: getTelegramMessageThreadIdForApi()
|
||||
: await getTelegramMessageThreadIdForApi()
|
||||
const text = parsed.data.rulePreview
|
||||
? formatAlertRuleTestTelegramText(parsed.data.rulePreview)
|
||||
: "MikroTik Manager — тест оповещений (Telegram)."
|
||||
|
||||
@@ -61,7 +61,7 @@ async function processBackupJob(job: BackupJob, ids: string[], notes?: string) {
|
||||
}
|
||||
job.status = "done"
|
||||
job.finishedAt = new Date().toISOString()
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: job.failures.length > 0 ? "warning" : "info",
|
||||
eventType: "backups.job.done",
|
||||
sourceModule: "backups",
|
||||
@@ -80,11 +80,11 @@ async function processBackupJob(job: BackupJob, ids: string[], notes?: string) {
|
||||
|
||||
const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/backups", async (_req, reply) => {
|
||||
return reply.send(listBackups())
|
||||
return reply.send(await listBackups())
|
||||
})
|
||||
|
||||
app.get("/backups/schedule", async (_req, reply) => {
|
||||
return reply.send(getBackupScheduleSettings())
|
||||
return reply.send(await getBackupScheduleSettings())
|
||||
})
|
||||
|
||||
app.put("/backups/schedule", async (req, reply) => {
|
||||
@@ -92,15 +92,15 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const result = updateBackupScheduleSettings(parsed.data)
|
||||
refreshScheduler()
|
||||
const result = await updateBackupScheduleSettings(parsed.data)
|
||||
await refreshScheduler()
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
app.post("/backups/create", { schema: { body: CreateBackupBodySchema } }, async (req, reply) => {
|
||||
const inputIds = req.body.serverIds.map((x) => String(x))
|
||||
const notes = req.body.notes?.trim() || undefined
|
||||
const existingServers = new Set(listServersRead().map((s) => String(s.id)))
|
||||
const existingServers = new Set((await listServersRead()).map((s) => String(s.id)))
|
||||
const ids = [...new Set(inputIds)].filter((id) => existingServers.has(id))
|
||||
if (ids.length === 0) return reply.status(400).send({ error: "Не выбраны валидные серверы" })
|
||||
|
||||
@@ -115,7 +115,7 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
failures: [],
|
||||
}
|
||||
backupJobs.set(jobId, job)
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "backups.job.started",
|
||||
sourceModule: "backups",
|
||||
@@ -128,8 +128,8 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
notes: notes ?? null,
|
||||
},
|
||||
})
|
||||
queueMicrotask(() => {
|
||||
void processBackupJob(job, ids, notes).catch((err) => {
|
||||
queueMicrotask(async () => {
|
||||
void processBackupJob(job, ids, notes).catch(async (err) => {
|
||||
job.status = "failed"
|
||||
job.finishedAt = new Date().toISOString()
|
||||
const failure = {
|
||||
@@ -137,7 +137,7 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
error: err instanceof Error ? err.message : String(err),
|
||||
}
|
||||
job.failures.push(failure)
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "critical",
|
||||
eventType: "backups.job.failed",
|
||||
sourceModule: "backups",
|
||||
@@ -167,7 +167,7 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
app.get("/backups/:id/download", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
const hit = getBackupById(req.params.id)
|
||||
const hit = await getBackupById(req.params.id)
|
||||
if (!hit) return reply.status(404).send({ error: "Бэкап не найден" })
|
||||
const filePath = path.join(getBackupsDir(), hit.filename)
|
||||
const content = await readFile(filePath, "utf8").catch(() => null)
|
||||
|
||||
@@ -12,7 +12,7 @@ const bgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
// GET /api/bgp/sessions/raw/:id — raw RouterOS response for a single server (debug)
|
||||
app.get("/bgp/sessions/raw/:id", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = db.select().from(servers).where(eq(servers.id, params.id)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, params.id)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
@@ -26,7 +26,7 @@ const bgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// GET /api/bgp/sessions — aggregate from ALL enabled servers
|
||||
app.get("/bgp/sessions", async (_req, reply) => {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
|
||||
const results: BgpSessionRead[][] = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
@@ -46,10 +46,10 @@ const bgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
// GET /api/servers/:id/bgp/sessions — single server
|
||||
app.get("/servers/:id/bgp/sessions", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = db
|
||||
const server = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, params.id))
|
||||
.limit(1).all()[0]
|
||||
.limit(1))[0]
|
||||
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
|
||||
@@ -32,7 +32,7 @@ const certificatesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
app.get("/certificates/acme-settings", async (_req, reply) => {
|
||||
return reply.send(getAcmeSettingsPublic())
|
||||
return reply.send(await getAcmeSettingsPublic())
|
||||
})
|
||||
|
||||
app.put("/certificates/acme-settings", async (req, reply) => {
|
||||
@@ -40,11 +40,11 @@ const certificatesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(updateAcmeSettings(parsed.data))
|
||||
return reply.send(await updateAcmeSettings(parsed.data))
|
||||
})
|
||||
|
||||
app.get("/certificates/renew-settings", async (_req, reply) => {
|
||||
return reply.send(getCertificateRenewSettings())
|
||||
return reply.send(await getCertificateRenewSettings())
|
||||
})
|
||||
|
||||
app.put("/certificates/renew-settings", async (req, reply) => {
|
||||
@@ -52,8 +52,8 @@ const certificatesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const result = updateCertificateRenewSettings(parsed.data)
|
||||
refreshScheduler()
|
||||
const result = await updateCertificateRenewSettings(parsed.data)
|
||||
await refreshScheduler()
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
@@ -62,7 +62,7 @@ const certificatesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const effective = parsed.data.cloudflareApiToken?.trim() || getAcmeCloudflareToken()
|
||||
const effective = parsed.data.cloudflareApiToken?.trim() || await getAcmeCloudflareToken()
|
||||
if (!effective) {
|
||||
return reply.status(400).send({ error: "Не задан Cloudflare API token" })
|
||||
}
|
||||
@@ -83,12 +83,12 @@ const certificatesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const serverId = String(parsed.data.serverId)
|
||||
const server = getServerRowByIdString(serverId)
|
||||
const server = await getServerRowByIdString(serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
|
||||
const jobId = randomUUID()
|
||||
const trustStore = (parsed.data.trustStore ?? ["www", "api"]).join(",")
|
||||
createIssueJobRecord({
|
||||
await createIssueJobRecord({
|
||||
id: jobId,
|
||||
serverId,
|
||||
certName: parsed.data.certName.trim(),
|
||||
@@ -103,9 +103,9 @@ const certificatesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
app.get("/certificates/issue/:jobId", async (req, reply) => {
|
||||
const jobId = String((req.params as { jobId: string }).jobId)
|
||||
const row = getIssueJobRecord(jobId)
|
||||
const row = await getIssueJobRecord(jobId)
|
||||
if (!row) return reply.status(404).send({ error: "Задача не найдена" })
|
||||
return reply.send(toIssueJobDto(row))
|
||||
return reply.send(await toIssueJobDto(row))
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -8,7 +8,7 @@ const eventsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректные параметры запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send({ events: readEvents(parsed.data) })
|
||||
return reply.send({ events: await readEvents(parsed.data) })
|
||||
})
|
||||
|
||||
app.post("/events/batch", async (req, reply) => {
|
||||
@@ -16,7 +16,7 @@ const eventsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
appendEvents(parsed.data.events)
|
||||
await appendEvents(parsed.data.events)
|
||||
return reply.status(201).send({ ok: true, inserted: parsed.data.events.length })
|
||||
})
|
||||
}
|
||||
|
||||
@@ -17,11 +17,11 @@ const TestEvobgpSchema = z.object({
|
||||
apiKey: z.string().optional(),
|
||||
})
|
||||
|
||||
function ensureEvobgpRow() {
|
||||
let row = db.select().from(evobgpSettings).where(eq(evobgpSettings.id, 1)).limit(1).all()[0]
|
||||
async function ensureEvobgpRow() {
|
||||
let row = (await db.select().from(evobgpSettings).where(eq(evobgpSettings.id, 1)).limit(1))[0]
|
||||
if (!row) {
|
||||
db.insert(evobgpSettings).values({ id: 1 }).run()
|
||||
row = db.select().from(evobgpSettings).where(eq(evobgpSettings.id, 1)).limit(1).all()[0]
|
||||
await db.insert(evobgpSettings).values({ id: 1 })
|
||||
row = (await db.select().from(evobgpSettings).where(eq(evobgpSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
return row!
|
||||
}
|
||||
@@ -108,11 +108,10 @@ function aggregateCommunityRouteCounts(catalog: EvoCatalogRaw): Map<string, numb
|
||||
* Уникальные серверы из локальной БД, у которых в BGP-фильтре указан community (строка AS:NNN).
|
||||
* Ключ — значение `filter_rules.community` после trim (как в EvoBGP `community`).
|
||||
*/
|
||||
function distinctServersByFilterCommunityValue(): Map<string, number> {
|
||||
const rows = db
|
||||
async function distinctServersByFilterCommunityValue(): Promise<Map<string, number>> {
|
||||
const rows = await db
|
||||
.select({ serverId: filterRules.serverId, community: filterRules.community })
|
||||
.from(filterRules)
|
||||
.all()
|
||||
const byVal = new Map<string, Set<number>>()
|
||||
for (const r of rows) {
|
||||
const v = (r.community ?? "").trim()
|
||||
@@ -161,8 +160,8 @@ async function fetchEvoJson<T>(root: string, path: string, token: string): Promi
|
||||
}
|
||||
}
|
||||
|
||||
function credentialsFromDb(): { root: string; apiKey: string } | null {
|
||||
const row = ensureEvobgpRow()
|
||||
async function credentialsFromDb(): Promise<{ root: string; apiKey: string } | null> {
|
||||
const row = await ensureEvobgpRow()
|
||||
const root = normalizeBaseUrl(row.baseUrl)
|
||||
const apiKey = normalizeApiKey(row.apiKey)
|
||||
if (!root || !apiKey) return null
|
||||
@@ -171,7 +170,7 @@ function credentialsFromDb(): { root: string; apiKey: string } | null {
|
||||
|
||||
const evobgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/evobgp/settings", async (_req, reply) => {
|
||||
const row = ensureEvobgpRow()
|
||||
const row = await ensureEvobgpRow()
|
||||
return reply.send({
|
||||
baseUrl: row.baseUrl ?? "",
|
||||
enabled: Boolean(row.enabled),
|
||||
@@ -184,7 +183,7 @@ const evobgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const cur = ensureEvobgpRow()
|
||||
const cur = await ensureEvobgpRow()
|
||||
let nextBase = cur.baseUrl
|
||||
let nextEnabled = cur.enabled
|
||||
let nextKey = cur.apiKey
|
||||
@@ -200,17 +199,16 @@ const evobgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
: normalizeApiKey(parsed.data.apiKey)
|
||||
}
|
||||
|
||||
db.update(evobgpSettings)
|
||||
await db.update(evobgpSettings)
|
||||
.set({
|
||||
baseUrl: nextBase,
|
||||
enabled: nextEnabled,
|
||||
apiKey: nextKey,
|
||||
updatedAt: sql`(datetime('now'))`,
|
||||
updatedAt: sql`now()`,
|
||||
})
|
||||
.where(eq(evobgpSettings.id, 1))
|
||||
.run()
|
||||
|
||||
const row = ensureEvobgpRow()
|
||||
const row = await ensureEvobgpRow()
|
||||
return reply.send({
|
||||
baseUrl: row.baseUrl ?? "",
|
||||
enabled: Boolean(row.enabled),
|
||||
@@ -228,7 +226,7 @@ const evobgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const row = ensureEvobgpRow()
|
||||
const row = await ensureEvobgpRow()
|
||||
const d = parsed.data
|
||||
const urlRaw = d.baseUrl !== undefined ? d.baseUrl : row.baseUrl
|
||||
const keyRaw =
|
||||
@@ -253,11 +251,11 @@ const evobgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
* Прокси к GET /v1/router-lists/catalog — учётные данные только из БД.
|
||||
*/
|
||||
app.post("/evobgp/catalog", async (_req, reply) => {
|
||||
const row = ensureEvobgpRow()
|
||||
const row = await ensureEvobgpRow()
|
||||
if (!row.enabled) {
|
||||
return reply.status(400).send({ error: "Интеграция EvoBGP выключена в настройках" })
|
||||
}
|
||||
const cred = credentialsFromDb()
|
||||
const cred = await credentialsFromDb()
|
||||
if (!cred) {
|
||||
return reply.status(400).send({ error: "Не заданы базовый URL или API-ключ в БД" })
|
||||
}
|
||||
@@ -333,7 +331,7 @@ const evobgpRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
const routeCountsByCommUuid = aggregateCommunityRouteCounts(catalog)
|
||||
const serversByCommunityStr = distinctServersByFilterCommunityValue()
|
||||
const serversByCommunityStr = await distinctServersByFilterCommunityValue()
|
||||
|
||||
const communities = commItems.map((c) => {
|
||||
const valueStr = (c.community ?? "").trim()
|
||||
|
||||
@@ -314,10 +314,10 @@ const execRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
{ schema: { params: ServerIdParamSchema, body: ExecBodySchema } },
|
||||
async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = db
|
||||
const server = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, params.id))
|
||||
.limit(1).all()[0]
|
||||
.limit(1))[0]
|
||||
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
|
||||
@@ -186,12 +186,11 @@ function gatewayFromRecursiveDst(dstAddress: string): string {
|
||||
* Импорт с роутера даёт только `set gateway` без `set out-interface` — в таком виде не отличить от «поломанного» GRE.
|
||||
* Сопоставляем hop с локальной таблицей recursive_routes и восстанавливаем gatewayTunnelId = rec:<id>.
|
||||
*/
|
||||
function enrichRulesWithRecursiveGateway(serverId: number, rules: ApiFilterRule[]): ApiFilterRule[] {
|
||||
const rows = db
|
||||
async function enrichRulesWithRecursiveGateway(serverId: number, rules: ApiFilterRule[]): Promise<ApiFilterRule[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(recursiveRoutes)
|
||||
.where(and(eq(recursiveRoutes.serverId, serverId), eq(recursiveRoutes.disabled, false)))
|
||||
.all()
|
||||
|
||||
return rules.map(rule => {
|
||||
if (rule.action === "blackhole") return rule
|
||||
@@ -249,7 +248,7 @@ async function fetchServerFilters(server: ServerRow) {
|
||||
.filter(r => (r.chain ?? "").trim().toLowerCase() === "bgp-in")
|
||||
.flatMap(parseFilterRule)
|
||||
|
||||
const rules = enrichRulesWithRecursiveGateway(server.id, rulesRaw)
|
||||
const rules = await enrichRulesWithRecursiveGateway(server.id, rulesRaw)
|
||||
|
||||
return {
|
||||
serverId: String(server.id),
|
||||
@@ -258,8 +257,8 @@ async function fetchServerFilters(server: ServerRow) {
|
||||
}
|
||||
}
|
||||
|
||||
function toApiRulesets(serverRows: ServerRow[]) {
|
||||
const dbRules = db.select().from(filterRules).orderBy(asc(filterRules.serverId), asc(filterRules.sortOrder)).all()
|
||||
async function toApiRulesets(serverRows: ServerRow[]) {
|
||||
const dbRules = await db.select().from(filterRules).orderBy(asc(filterRules.serverId), asc(filterRules.sortOrder))
|
||||
return serverRows.map(s => ({
|
||||
serverId: String(s.id),
|
||||
rules: dbRules
|
||||
@@ -277,7 +276,7 @@ function toApiRulesets(serverRows: ServerRow[]) {
|
||||
}
|
||||
|
||||
/** GRE: gatewayTunnelId = имя интерфейса; рекурсивный: rec:<id строки recursive_routes */
|
||||
function resolveRouteTargets(serverId: number, rule: ApiFilterRule): { gateway: string; outIface: string } {
|
||||
async function resolveRouteTargets(serverId: number, rule: ApiFilterRule): Promise<{ gateway: string; outIface: string }> {
|
||||
if (rule.action === "blackhole") return { gateway: "", outIface: "" }
|
||||
const tid = (rule.gatewayTunnelId ?? "").trim()
|
||||
if (tid.startsWith("rec:")) {
|
||||
@@ -285,12 +284,11 @@ function resolveRouteTargets(serverId: number, rule: ApiFilterRule): { gateway:
|
||||
if (!Number.isFinite(rid)) {
|
||||
return { gateway: rule.gateway, outIface: "" }
|
||||
}
|
||||
const row = db
|
||||
const row = (await db
|
||||
.select()
|
||||
.from(recursiveRoutes)
|
||||
.where(and(eq(recursiveRoutes.serverId, serverId), eq(recursiveRoutes.id, rid)))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
if (!row) return { gateway: rule.gateway, outIface: "" }
|
||||
const gw = gatewayFromRecursiveDst(row.dstAddress)
|
||||
return { gateway: gw || rule.gateway, outIface: "" }
|
||||
@@ -303,30 +301,30 @@ function normalizeCommunity(c: string): string {
|
||||
}
|
||||
|
||||
/** Одинаковый эффект на роутере при одинаковой community (blackhole vs gateway + out-interface) */
|
||||
function ruleEffectSignature(serverId: number, r: ApiFilterRule): string {
|
||||
async function ruleEffectSignature(serverId: number, r: ApiFilterRule): Promise<string> {
|
||||
if (r.action === "blackhole") return `bh:${normalizeCommunity(r.community)}`
|
||||
const { gateway, outIface } = resolveRouteTargets(serverId, r)
|
||||
const { gateway, outIface } = await resolveRouteTargets(serverId, r)
|
||||
return `rt:${normalizeCommunity(r.community)}:${gateway}:${outIface}`
|
||||
}
|
||||
|
||||
export type FilterRouterCompareStatus = "synced" | "drift" | "missing"
|
||||
|
||||
function compareDbRulesWithRouter(
|
||||
async function compareDbRulesWithRouter(
|
||||
serverId: number,
|
||||
dbRules: ApiFilterRule[],
|
||||
remoteRules: ApiFilterRule[],
|
||||
): Record<string, FilterRouterCompareStatus> {
|
||||
): Promise<Record<string, FilterRouterCompareStatus>> {
|
||||
const remoteSigByComm = new Map<string, string>()
|
||||
for (const rr of remoteRules) {
|
||||
const c = normalizeCommunity(rr.community)
|
||||
if (!remoteSigByComm.has(c)) {
|
||||
remoteSigByComm.set(c, ruleEffectSignature(serverId, rr))
|
||||
remoteSigByComm.set(c, await ruleEffectSignature(serverId, rr))
|
||||
}
|
||||
}
|
||||
const out: Record<string, FilterRouterCompareStatus> = {}
|
||||
for (const dr of dbRules) {
|
||||
const c = normalizeCommunity(dr.community)
|
||||
const sigD = ruleEffectSignature(serverId, dr)
|
||||
const sigD = await ruleEffectSignature(serverId, dr)
|
||||
const sigR = remoteSigByComm.get(c)
|
||||
if (sigR === undefined) {
|
||||
out[c] = "missing"
|
||||
@@ -339,7 +337,7 @@ function compareDbRulesWithRouter(
|
||||
return out
|
||||
}
|
||||
|
||||
function toRouterRuleBody(serverId: number, rules: ApiFilterRule[]): string {
|
||||
async function toRouterRuleBody(serverId: number, rules: ApiFilterRule[]): Promise<string> {
|
||||
if (rules.length === 0) return ""
|
||||
// Группируем по эффекту (action + gateway + out-interface). Communities с одним и тем же
|
||||
// `set gw` объединяются через `||` в один if-блок — компактнее и ближе к привычному
|
||||
@@ -361,7 +359,7 @@ function toRouterRuleBody(serverId: number, rules: ApiFilterRule[]): string {
|
||||
const isBlackhole = rule.action === "blackhole"
|
||||
const { gateway, outIface } = isBlackhole
|
||||
? { gateway: "", outIface: "" }
|
||||
: resolveRouteTargets(serverId, rule)
|
||||
: await resolveRouteTargets(serverId, rule)
|
||||
const key = isBlackhole ? "bh" : `rt:${gateway}:${outIface}`
|
||||
let idx = indexByKey.get(key)
|
||||
if (idx === undefined) {
|
||||
@@ -404,10 +402,10 @@ function toRouterRuleBody(serverId: number, rules: ApiFilterRule[]): string {
|
||||
}
|
||||
|
||||
async function replaceDbRules(serverId: number, rules: ApiFilterRule[]) {
|
||||
db.delete(filterRules).where(eq(filterRules.serverId, serverId)).run()
|
||||
await db.delete(filterRules).where(eq(filterRules.serverId, serverId))
|
||||
if (rules.length === 0) return
|
||||
const now = new Date().toISOString()
|
||||
db.insert(filterRules).values(
|
||||
await db.insert(filterRules).values(
|
||||
rules.map((r, i) => ({
|
||||
serverId,
|
||||
sortOrder: i,
|
||||
@@ -420,7 +418,7 @@ async function replaceDbRules(serverId: number, rules: ApiFilterRule[]) {
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
})),
|
||||
).run()
|
||||
)
|
||||
}
|
||||
|
||||
const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
@@ -432,17 +430,16 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "serverId is required" })
|
||||
}
|
||||
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
const remote = await fetchServerFilters(server)
|
||||
const rows = db
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(filterRules)
|
||||
.where(eq(filterRules.serverId, serverId))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
.all()
|
||||
|
||||
const dbRules: ApiFilterRule[] = rows.map(r => ({
|
||||
id: String(r.id),
|
||||
@@ -454,7 +451,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
description: r.description,
|
||||
}))
|
||||
|
||||
const byCommunity = compareDbRulesWithRouter(serverId, dbRules, remote.rules)
|
||||
const byCommunity = await compareDbRulesWithRouter(serverId, dbRules, remote.rules)
|
||||
return reply.send({ byCommunity })
|
||||
} catch (err) {
|
||||
app.log.error({ serverId, err: String(err) }, "filters router-compare failed")
|
||||
@@ -466,7 +463,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/filters/gre-tunnels", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const sid = parseDbServerId(q.serverId)
|
||||
const enabledServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
|
||||
if (sid !== null) {
|
||||
const server = enabledServers.find(s => s.id === sid)
|
||||
@@ -492,8 +489,8 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
/** Только правила фильтров из БД (без опроса MikroTik за GRE) */
|
||||
app.get("/filters/rules", async (_req, reply) => {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const dbRulesets = toApiRulesets(allServers)
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const dbRulesets = await toApiRulesets(allServers)
|
||||
|
||||
return reply.send({
|
||||
rulesets: dbRulesets,
|
||||
@@ -506,7 +503,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const payload = body.rulesets ?? []
|
||||
const serverIds = payload.map(r => Number.parseInt(r.serverId, 10)).filter(Number.isFinite)
|
||||
if (serverIds.length > 0) {
|
||||
db.delete(filterRules).where(inArray(filterRules.serverId, serverIds)).run()
|
||||
await db.delete(filterRules).where(inArray(filterRules.serverId, serverIds))
|
||||
}
|
||||
for (const rs of payload) {
|
||||
const sid = Number.parseInt(rs.serverId, 10)
|
||||
@@ -524,12 +521,12 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "serverId is required" })
|
||||
}
|
||||
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
app.log.info({ serverId, host: server.host }, "Filters sync from router started")
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.from_router.started",
|
||||
sourceModule: "filters",
|
||||
@@ -541,7 +538,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const remote = await fetchServerFilters(server)
|
||||
await replaceDbRules(server.id, remote.rules)
|
||||
app.log.info({ serverId, totalRules: remote.rules.length }, "Filters sync from router completed")
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.from_router.done",
|
||||
sourceModule: "filters",
|
||||
@@ -553,7 +550,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send({ ok: true, updatedServers: 1, totalRules: remote.rules.length, serverId })
|
||||
} catch (err) {
|
||||
app.log.error({ serverId, err: String(err) }, "Filters sync from router failed")
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "critical",
|
||||
eventType: "filters.sync.from_router.failed",
|
||||
sourceModule: "filters",
|
||||
@@ -570,7 +567,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const requestedServerId = parseDbServerId(body?.serverId)
|
||||
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const targetServers = requestedServerId !== null
|
||||
? allServers.filter(s => s.id === requestedServerId)
|
||||
: allServers
|
||||
@@ -582,7 +579,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
let updatedServers = 0
|
||||
let pushedRules = 0
|
||||
const errors: Array<{ serverId: number; error: string }> = []
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.to_router.started",
|
||||
sourceModule: "filters",
|
||||
@@ -618,10 +615,9 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
.map(r => r[".id"])
|
||||
.filter((id): id is string => Boolean(id))
|
||||
|
||||
const rows = db.select().from(filterRules)
|
||||
const rows = await db.select().from(filterRules)
|
||||
.where(and(eq(filterRules.serverId, server.id)))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
.all()
|
||||
|
||||
const rules: ApiFilterRule[] = rows.map(r => ({
|
||||
id: String(r.id),
|
||||
@@ -639,7 +635,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
// трактует как «вызов команды» и отдаёт 400 «no such command».
|
||||
// См. https://help.mikrotik.com/docs/spaces/ROS/pages/47579162/REST+API
|
||||
if (rules.length > 0) {
|
||||
const ruleBody = toRouterRuleBody(server.id, rules)
|
||||
const ruleBody = await toRouterRuleBody(server.id, rules)
|
||||
if (managedRule && managedRule[".id"]) {
|
||||
await client.patch(
|
||||
`/routing/filter/rule/${encodeURIComponent(managedRule[".id"])}`,
|
||||
@@ -688,7 +684,7 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
}
|
||||
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: errors.length === 0 ? "info" : "warning",
|
||||
eventType: errors.length === 0 ? "filters.sync.to_router.done" : "filters.sync.to_router.partial",
|
||||
sourceModule: "filters",
|
||||
|
||||
@@ -116,8 +116,8 @@ function rosErr(e: unknown): string {
|
||||
return String(e)
|
||||
}
|
||||
|
||||
function requireServer(serverId: string) {
|
||||
return getEnabledServerById(serverId)
|
||||
async function requireServer(serverId: string) {
|
||||
return await getEnabledServerById(serverId)
|
||||
}
|
||||
|
||||
const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
@@ -132,7 +132,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)
|
||||
@@ -150,7 +150,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = `${firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)}/${encodeRosId(body.rosId)}`
|
||||
@@ -168,7 +168,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
|
||||
@@ -186,7 +186,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
|
||||
@@ -204,7 +204,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = `${firewallRestPath(body.family, body.table)}/move`
|
||||
@@ -225,7 +225,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
@@ -242,7 +242,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
@@ -260,7 +260,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
@@ -278,7 +278,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = requireServer(body.serverId)
|
||||
const server = await requireServer(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
|
||||
@@ -8,7 +8,7 @@ import {
|
||||
|
||||
const internetPathRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/internet-path/settings", async (_req, reply) => {
|
||||
const s = getInternetPathSettings()
|
||||
const s = await getInternetPathSettings()
|
||||
return reply.send({
|
||||
enabled: s.enabled,
|
||||
intervalSec: s.intervalSec,
|
||||
@@ -25,12 +25,12 @@ const internetPathRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
intervalSec?: number | string
|
||||
retentionDays?: number | string
|
||||
}
|
||||
const updated = updateInternetPathSettings({
|
||||
const updated = await updateInternetPathSettings({
|
||||
enabled: body.enabled,
|
||||
intervalSec: body.intervalSec == null ? undefined : Math.max(30, Number.parseInt(String(body.intervalSec), 10) || 300),
|
||||
retentionDays: body.retentionDays == null ? undefined : Math.max(1, Number.parseInt(String(body.retentionDays), 10) || 14),
|
||||
})
|
||||
refreshScheduler()
|
||||
await refreshScheduler()
|
||||
return reply.send({
|
||||
ok: true,
|
||||
settings: {
|
||||
@@ -45,13 +45,15 @@ const internetPathRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
app.get("/internet-path/latest", async (_req, reply) => {
|
||||
const row = getLatestInternetPathSnapshot()
|
||||
const row = await getLatestInternetPathSnapshot()
|
||||
if (!row) return reply.send({ snapshot: null })
|
||||
let payload: unknown = null
|
||||
try {
|
||||
payload = JSON.parse(row.payloadJson)
|
||||
} catch {
|
||||
payload = null
|
||||
let payload: unknown = row.payloadJson
|
||||
if (typeof row.payloadJson === "string") {
|
||||
try {
|
||||
payload = JSON.parse(row.payloadJson)
|
||||
} catch {
|
||||
payload = null
|
||||
}
|
||||
}
|
||||
return reply.send({
|
||||
snapshot: payload,
|
||||
|
||||
+23
-28
@@ -207,23 +207,21 @@ function calcRouteScore(pingMs: number, dlMbps: number, ulMbps: number, pingWeig
|
||||
return Math.round(w * pingScore + (1 - w) * speedScore)
|
||||
}
|
||||
|
||||
function getLatestSnapshotLatencyMs(serverId: number): number {
|
||||
const latest = db
|
||||
async function getLatestSnapshotLatencyMs(serverId: number): Promise<number> {
|
||||
const latest = await db
|
||||
.select()
|
||||
.from(serverSnapshots)
|
||||
.where(eq(serverSnapshots.serverId, serverId))
|
||||
.orderBy(desc(serverSnapshots.polledAt))
|
||||
.limit(1)
|
||||
.all()
|
||||
return latest.length > 0 ? Math.max(1, Math.round(latest[0].latencyMs ?? 100)) : 100
|
||||
}
|
||||
|
||||
function latestTrafficByInterface(serverId: number): Map<string, TrafficSampleRow> {
|
||||
const rows = db
|
||||
async function latestTrafficByInterface(serverId: number): Promise<Map<string, { id: number; serverId: number; disabled: boolean; sampledAt: string; interfaceName: string; peerPublicKey: string; rxBytes: number; txBytes: number; rxBps: number; txBps: number; running: boolean; }>> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(trafficSamples)
|
||||
.where(eq(trafficSamples.serverId, serverId))
|
||||
.all()
|
||||
const map = new Map<string, TrafficSampleRow>()
|
||||
for (const row of rows) {
|
||||
const prev = map.get(row.interfaceName)
|
||||
@@ -302,14 +300,14 @@ async function buildOspfOptimizationPlan(server: ServerRow, pingWeight: number):
|
||||
const areaMap = buildAreaMap(areas)
|
||||
const parsed = parseInterfaces(server, ifaceTemplates, areas, instances, areaMap)
|
||||
const editable = parsed.filter((i) => !i.disabled && !isRefInterfaceName(i.interface))
|
||||
const localLatencyMs = getLatestSnapshotLatencyMs(server.id)
|
||||
const latestTraffic = latestTrafficByInterface(server.id)
|
||||
const localLatencyMs = await getLatestSnapshotLatencyMs(server.id)
|
||||
const latestTraffic = await latestTrafficByInterface(server.id)
|
||||
|
||||
// 1) Сопоставляем remote OSPF router-id -> сервер из каталога (чтобы взять унифицированный ping как в карте /route-optimizer).
|
||||
const neededRouterIds = new Set(neighbors.map((n) => String(n["router-id"] ?? "")).filter((x) => x.length > 0))
|
||||
|
||||
async function buildRouterIdToServerMap(needed: Set<string>): Promise<Map<string, ServerRow["id"]>> {
|
||||
const enabledServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const out = new Map<string, ServerRow["id"]>()
|
||||
const neededLeft = new Set(needed)
|
||||
|
||||
@@ -356,11 +354,10 @@ async function buildOspfOptimizationPlan(server: ServerRow, pingWeight: number):
|
||||
}
|
||||
|
||||
// 3) Предгружаем uptimeSpeedProbes: оттуда берём и ping, и скорость (единый источник как у карты).
|
||||
const allSourceProbes = db
|
||||
const allSourceProbes = (await db
|
||||
.select()
|
||||
.from(uptimeSpeedProbes)
|
||||
.where(eq(uptimeSpeedProbes.srcServerId, server.id))
|
||||
.all()
|
||||
.where(eq(uptimeSpeedProbes.srcServerId, server.id)))
|
||||
.filter((r: SpeedProbeRow) => r.enabled !== false)
|
||||
const speedProbesByDest = new Map<number, SpeedProbeRow[]>()
|
||||
for (const p of allSourceProbes) {
|
||||
@@ -482,8 +479,7 @@ async function buildOspfOptimizationPlan(server: ServerRow, pingWeight: number):
|
||||
for (const [iface, probe] of assigned.entries()) assignedProbeByIface.set(iface, probe)
|
||||
}
|
||||
|
||||
return editable
|
||||
.map((iface) => {
|
||||
return (await Promise.all(editable.map(async (iface) => {
|
||||
const t = latestTraffic.get(iface.interface)
|
||||
const speed = normalizeSpeedMbps(t)
|
||||
|
||||
@@ -494,7 +490,7 @@ async function buildOspfOptimizationPlan(server: ServerRow, pingWeight: number):
|
||||
bestProbe?.lastPingRttMs != null
|
||||
? Math.max(1, Math.round(bestProbe.lastPingRttMs))
|
||||
: dstServerId != null
|
||||
? Math.min(995, Math.round(localLatencyMs + getLatestSnapshotLatencyMs(dstServerId)))
|
||||
? Math.min(995, Math.round(localLatencyMs + await getLatestSnapshotLatencyMs(dstServerId)))
|
||||
: localLatencyMs
|
||||
|
||||
const dlMbps =
|
||||
@@ -517,8 +513,7 @@ async function buildOspfOptimizationPlan(server: ServerRow, pingWeight: number):
|
||||
score: calcRouteScore(pingMs, dlMbps, ulMbps, pingWeight),
|
||||
optimalCost: 0,
|
||||
}
|
||||
})
|
||||
.sort((a, b) => b.score - a.score || a.interface.localeCompare(b.interface))
|
||||
}))).sort((a, b) => b.score - a.score || a.interface.localeCompare(b.interface))
|
||||
.map((row, idx) => ({ ...row, optimalCost: (idx + 1) * 10 }))
|
||||
}
|
||||
|
||||
@@ -528,7 +523,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// GET /api/ospf/neighbors — aggregate OSPF neighbors from ALL enabled servers
|
||||
app.get("/ospf/neighbors", async (_req, reply) => {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
|
||||
const results = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
@@ -547,7 +542,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// GET /api/ospf/interfaces — aggregate OSPF interface templates from ALL enabled servers
|
||||
app.get("/ospf/interfaces", async (_req, reply) => {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
|
||||
const results = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
@@ -566,7 +561,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// GET /api/ospf/instances — aggregate OSPF instances from ALL enabled servers
|
||||
app.get("/ospf/instances", async (_req, reply) => {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
|
||||
const results = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
@@ -584,7 +579,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// GET /api/ospf/all — single round-trip: neighbors + interfaces + instances + BFD
|
||||
app.get("/ospf/all", async (_req, reply) => {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
|
||||
const perServer = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
@@ -613,7 +608,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// GET /api/bfd/sessions — BFD sessions only (for direct access)
|
||||
app.get("/bfd/sessions", async (_req, reply) => {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const results = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
try {
|
||||
@@ -631,10 +626,10 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
// GET /api/servers/:id/ospf — single server OSPF + BFD data
|
||||
app.get("/servers/:id/ospf", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = db
|
||||
const server = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, params.id))
|
||||
.limit(1).all()[0]
|
||||
.limit(1))[0]
|
||||
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
@@ -659,10 +654,10 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
{ schema: { params: ServerIdParamSchema, body: OspfOptimizeBodySchema } },
|
||||
async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = db
|
||||
const server = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, params.id))
|
||||
.limit(1).all()[0]
|
||||
.limit(1))[0]
|
||||
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
@@ -691,10 +686,10 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
{ schema: { params: ServerIdParamSchema, body: OspfOptimizeBodySchema } },
|
||||
async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = db
|
||||
const server = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, params.id))
|
||||
.limit(1).all()[0]
|
||||
.limit(1))[0]
|
||||
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
|
||||
@@ -256,14 +256,14 @@ async function mtuDiscover(client: MikrotikClient, address: string, srcIpv4: str
|
||||
return lines.join("\n")
|
||||
}
|
||||
|
||||
function resolveBtestPeer(remoteHost: string, explicitDstId?: number) {
|
||||
async function resolveBtestPeer(remoteHost: string, explicitDstId?: number) {
|
||||
if (explicitDstId != null && Number.isFinite(explicitDstId)) {
|
||||
const s = db.select().from(servers).where(eq(servers.id, explicitDstId)).limit(1).all()[0]
|
||||
const s = (await db.select().from(servers).where(eq(servers.id, explicitDstId)).limit(1))[0]
|
||||
if (s) return s
|
||||
}
|
||||
const norm = remoteHost.trim().toLowerCase()
|
||||
return db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
.find((x) => x.host.trim().toLowerCase() === norm)
|
||||
const enabled = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
return enabled.find((x) => x.host.trim().toLowerCase() === norm)
|
||||
}
|
||||
|
||||
const probesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
@@ -271,7 +271,7 @@ const probesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const sid = parseServerId(String((req.params as { id?: string }).id ?? ""))
|
||||
if (sid === null) return reply.status(400).send({ error: "Invalid server id" })
|
||||
|
||||
const server = db.select().from(servers).where(eq(servers.id, sid)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, sid)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const body = req.body as {
|
||||
@@ -369,7 +369,7 @@ const probesRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
case "bandwidth": {
|
||||
const remote = String(body.bwRemoteAddress ?? "").trim()
|
||||
if (!remote) return reply.status(400).send({ error: "bwRemoteAddress required" })
|
||||
const dst = resolveBtestPeer(remote, body.dstServerId)
|
||||
const dst = await resolveBtestPeer(remote, body.dstServerId)
|
||||
if (!dst) {
|
||||
return reply.status(400).send({
|
||||
error: "Не найден сервер назначения для bandwidth-test: добавьте узел с host = GRE remote или укажите dstServerId",
|
||||
|
||||
@@ -80,13 +80,12 @@ function splitGateway(raw: string): { ip: string; name: string } | null {
|
||||
return { ip, name: name || ip }
|
||||
}
|
||||
|
||||
function mapDbRoutes(serverId: number): RecursiveRouteDto[] {
|
||||
const rows = db
|
||||
async function mapDbRoutes(serverId: number): Promise<RecursiveRouteDto[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(recursiveRoutes)
|
||||
.where(eq(recursiveRoutes.serverId, serverId))
|
||||
.orderBy(asc(recursiveRoutes.sortOrder))
|
||||
.all()
|
||||
|
||||
return rows.map((r) => ({
|
||||
id: String(r.id),
|
||||
@@ -118,10 +117,10 @@ function toRouterPayload(route: RecursiveRouteDto): Record<string, string> {
|
||||
}
|
||||
|
||||
async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
|
||||
db.delete(recursiveRoutes).where(eq(recursiveRoutes.serverId, serverId)).run()
|
||||
await db.delete(recursiveRoutes).where(eq(recursiveRoutes.serverId, serverId))
|
||||
if (routes.length === 0) return
|
||||
const now = new Date().toISOString()
|
||||
db.insert(recursiveRoutes).values(
|
||||
await db.insert(recursiveRoutes).values(
|
||||
routes.map((r, i) => ({
|
||||
serverId,
|
||||
sortOrder: i,
|
||||
@@ -138,7 +137,7 @@ async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
})),
|
||||
).run()
|
||||
)
|
||||
}
|
||||
|
||||
const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
@@ -146,7 +145,7 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
@@ -176,16 +175,16 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
return reply.send({ routes: mapDbRoutes(serverId) })
|
||||
return reply.send({ routes: await mapDbRoutes(serverId) })
|
||||
})
|
||||
|
||||
app.put("/recursive-routes", async (req, reply) => {
|
||||
const body = req.body as { serverId?: string | number; routes?: RecursiveRouteDto[] }
|
||||
const serverId = parseDbServerId(body.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
await replaceDbRoutes(serverId, body.routes ?? [])
|
||||
return reply.send({ ok: true })
|
||||
@@ -195,10 +194,10 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const serverId = parseDbServerId(body?.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server: ServerRow | undefined = db
|
||||
const server: ServerRow | undefined = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, serverId))
|
||||
.limit(1).all()[0]
|
||||
.limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
@@ -231,7 +230,7 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const serverId = parseDbServerId(body?.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
@@ -243,7 +242,7 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
await client.delete(`/ip/route/${encodeURIComponent(r[".id"])}`)
|
||||
}
|
||||
|
||||
const dbRows = mapDbRoutes(serverId)
|
||||
const dbRows = await mapDbRoutes(serverId)
|
||||
for (const route of dbRows) {
|
||||
await client.post("/ip/route", toRouterPayload(route))
|
||||
}
|
||||
|
||||
@@ -14,7 +14,7 @@ function parseJobKey(raw: string | undefined): typeof JOB_KEYS[number] | null {
|
||||
|
||||
const schedulerRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/scheduler/status", async (_req, reply) => {
|
||||
return reply.send(getSchedulerStatus())
|
||||
return reply.send(await getSchedulerStatus())
|
||||
})
|
||||
|
||||
app.get("/scheduler/runs", async (req, reply) => {
|
||||
@@ -25,12 +25,12 @@ const schedulerRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
const limit = Math.min(200, Math.max(1, Number.parseInt(String(q.limit ?? "50"), 10) || 50))
|
||||
const offset = Math.max(0, Number.parseInt(String(q.offset ?? "0"), 10) || 0)
|
||||
return reply.send(listSchedulerRuns({ jobKey: jobKey ?? undefined, limit, offset }))
|
||||
return reply.send(await listSchedulerRuns({ jobKey: jobKey ?? undefined, limit, offset }))
|
||||
})
|
||||
|
||||
app.post("/scheduler/refresh", async (_req, reply) => {
|
||||
refreshScheduler()
|
||||
return reply.send({ ok: true, status: getSchedulerStatus() })
|
||||
await refreshScheduler()
|
||||
return reply.send({ ok: true, status: await getSchedulerStatus() })
|
||||
})
|
||||
|
||||
app.post("/scheduler/jobs/:jobKey/run-now", async (req, reply) => {
|
||||
@@ -38,7 +38,7 @@ const schedulerRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!jobKey) return reply.status(400).send({ error: "Некорректный jobKey" })
|
||||
try {
|
||||
await runSchedulerJobNow(jobKey)
|
||||
return reply.send({ ok: true, status: getSchedulerStatus() })
|
||||
return reply.send({ ok: true, status: await getSchedulerStatus() })
|
||||
} catch (e) {
|
||||
const statusCode = (e as Error & { statusCode?: number }).statusCode
|
||||
if (statusCode === 409) {
|
||||
|
||||
@@ -10,7 +10,7 @@ import {
|
||||
|
||||
const serversApiPingRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/servers-api-ping/settings", async (_req, reply) => {
|
||||
const settings = getServersApiPingSettings()
|
||||
const settings = await getServersApiPingSettings()
|
||||
const state = getServersRestPingCollectorState()
|
||||
return reply.send({
|
||||
enabled: settings.enabled,
|
||||
@@ -29,11 +29,11 @@ const serversApiPingRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
const intervalSec =
|
||||
body.intervalSec == null ? undefined : Math.max(10, Number.parseInt(String(body.intervalSec), 10) || 120)
|
||||
const updated = updateServersApiPingSettings({
|
||||
const updated = await updateServersApiPingSettings({
|
||||
enabled: body.enabled,
|
||||
intervalSec,
|
||||
})
|
||||
refreshScheduler()
|
||||
await refreshScheduler()
|
||||
return reply.send({
|
||||
ok: true,
|
||||
settings: {
|
||||
@@ -49,7 +49,7 @@ const serversApiPingRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.post("/servers-api-ping/collect-now", async (_req, reply) => {
|
||||
await collectServersRestPingOnce()
|
||||
scheduleAlertEngineAfterDataCollectors()
|
||||
const row = getServersApiPingSettings()
|
||||
const row = await getServersApiPingSettings()
|
||||
return reply.send({
|
||||
ok: true,
|
||||
lastCollectedAt: row.lastCollectedAt ?? null,
|
||||
|
||||
@@ -77,13 +77,13 @@ const serversRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// GET /api/servers
|
||||
app.get("/", async (_req, reply) => {
|
||||
return reply.send(listServersRead())
|
||||
return reply.send(await listServersRead())
|
||||
})
|
||||
|
||||
// GET /api/servers/:id/ros-src-address — IPv4 для src-address в RouterOS (не FQDN)
|
||||
app.get("/:id/ros-src-address", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = getServerRowById(params.id)
|
||||
const server = await getServerRowById(params.id)
|
||||
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
const ipv4 = await resolveRosSrcIpv4(server.host)
|
||||
@@ -93,7 +93,7 @@ const serversRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
// GET /api/servers/:id/wan-runtime — DHCP lease + active default route for HomeRouter WAN uplinks
|
||||
app.get("/:id/wan-runtime", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = getServerReadById(params.id)
|
||||
const server = await getServerReadById(params.id)
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const toIp = (raw: string | null | undefined): string | null => {
|
||||
@@ -141,7 +141,9 @@ const serversRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(getServerRowById(params.id)!)
|
||||
const row = await getServerRowById(params.id)
|
||||
if (!row) return reply.status(404).send({ error: "Server not found" })
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const isTrue = (v: unknown) => {
|
||||
const s = String(v ?? "").trim().toLowerCase()
|
||||
return s === "true" || s === "yes"
|
||||
@@ -228,13 +230,13 @@ const serversRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
// POST /api/servers
|
||||
app.post("/", { schema: { body: ServerCreateSchema } }, async (req, reply) => {
|
||||
return reply.status(201).send(createServer(req.body as ServerCreateRequest))
|
||||
return reply.status(201).send(await createServer(req.body as ServerCreateRequest))
|
||||
})
|
||||
|
||||
// GET /api/servers/:id
|
||||
app.get("/:id", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = getServerReadById(params.id)
|
||||
const server = await getServerReadById(params.id)
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
return reply.send(server)
|
||||
})
|
||||
@@ -245,25 +247,25 @@ const serversRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
{ schema: { params: ServerIdParamSchema, body: ServerUpdateSchema } },
|
||||
async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const existing = getServerReadById(params.id)
|
||||
const existing = await getServerReadById(params.id)
|
||||
if (!existing) return reply.status(404).send({ error: "Server not found" })
|
||||
return reply.send(updateServer(params.id, req.body as ServerUpdateRequest))
|
||||
return reply.send(await updateServer(params.id, req.body as ServerUpdateRequest))
|
||||
},
|
||||
)
|
||||
|
||||
// DELETE /api/servers/:id
|
||||
app.delete("/:id", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const existing = getServerReadById(params.id)
|
||||
const existing = await getServerReadById(params.id)
|
||||
if (!existing) return reply.status(404).send({ error: "Server not found" })
|
||||
deleteServer(params.id)
|
||||
await deleteServer(params.id)
|
||||
return reply.status(204).send()
|
||||
})
|
||||
|
||||
// POST /api/servers/:id/poll
|
||||
app.post("/:id/poll", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const existing = getServerReadById(params.id)
|
||||
const existing = await getServerReadById(params.id)
|
||||
if (!existing) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
@@ -281,9 +283,9 @@ const serversRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const query = req.query as SnapshotsQuery
|
||||
const existing = getServerReadById(params.id)
|
||||
const existing = await getServerReadById(params.id)
|
||||
if (!existing) return reply.status(404).send({ error: "Server not found" })
|
||||
return reply.send(listServerSnapshots(params.id, query.limit))
|
||||
return reply.send(await listServerSnapshots(params.id, query.limit))
|
||||
},
|
||||
)
|
||||
}
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { count } from "drizzle-orm"
|
||||
import { listCertificatesFromServers } from "../services/certificates-service.js"
|
||||
import { countWireGuardInterfaces } from "../services/wireguard-live.js"
|
||||
import { db } from "../db/index.js"
|
||||
@@ -11,27 +12,24 @@ import {
|
||||
} from "../db/schema.js"
|
||||
import { listUsers } from "../modules/users/service/users-service.js"
|
||||
|
||||
async function tableCount(table: typeof servers | typeof filterRules | typeof uptimeProbes | typeof uptimeSpeedProbes | typeof recursiveRoutes): Promise<number> {
|
||||
const rows = await db.select({ n: count() }).from(table)
|
||||
return rows[0]?.n ?? 0
|
||||
}
|
||||
|
||||
const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/sidebar-counts", async (_req, reply) => {
|
||||
const [
|
||||
serversTotal,
|
||||
filterRulesTotal,
|
||||
uptimeProbesTotal,
|
||||
uptimeSpeedProbesTotal,
|
||||
recursiveRoutesTotal,
|
||||
certificatesTotal,
|
||||
wireguardTotal,
|
||||
usersTotal,
|
||||
] = await Promise.all([
|
||||
Promise.resolve(db.select().from(servers).all().length),
|
||||
Promise.resolve(db.select().from(filterRules).all().length),
|
||||
Promise.resolve(db.select().from(uptimeProbes).all().length),
|
||||
Promise.resolve(db.select().from(uptimeSpeedProbes).all().length),
|
||||
Promise.resolve(db.select().from(recursiveRoutes).all().length),
|
||||
listCertificatesFromServers().then((res) => res.certificates.length),
|
||||
const serversTotal = await tableCount(servers)
|
||||
const filterRulesTotal = await tableCount(filterRules)
|
||||
const uptimeProbesTotal = await tableCount(uptimeProbes)
|
||||
const uptimeSpeedProbesTotal = await tableCount(uptimeSpeedProbes)
|
||||
const recursiveRoutesTotal = await tableCount(recursiveRoutes)
|
||||
const [certRes, wireguardTotal] = await Promise.all([
|
||||
listCertificatesFromServers(),
|
||||
countWireGuardInterfaces().catch(() => 0),
|
||||
Promise.resolve(listUsers().length),
|
||||
])
|
||||
const certificatesTotal = certRes.certificates.length
|
||||
const usersTotal = (await listUsers()).length
|
||||
|
||||
return reply.send({
|
||||
servers: serversTotal,
|
||||
|
||||
@@ -18,7 +18,7 @@ const systemDatabaseRoutes: FastifyPluginAsync = async (app) => {
|
||||
app.get("/system/database/backup", async (_req, reply) => {
|
||||
try {
|
||||
const { filename, buffer } = await exportSystemDatabaseBackup()
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "system.database.backup",
|
||||
sourceModule: "system",
|
||||
@@ -44,16 +44,16 @@ const systemDatabaseRoutes: FastifyPluginAsync = async (app) => {
|
||||
app.post("/system/database/restore", async (req, reply) => {
|
||||
const body = req.body
|
||||
if (!Buffer.isBuffer(body) || body.length === 0) {
|
||||
return reply.status(400).send({ error: "Ожидается тело запроса с файлом SQLite" })
|
||||
return reply.status(400).send({ error: "Ожидается тело запроса с файлом pg_dump (custom, PGDMP)" })
|
||||
}
|
||||
try {
|
||||
await restoreSystemDatabaseBackup(body)
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "warning",
|
||||
eventType: "system.database.restore",
|
||||
sourceModule: "system",
|
||||
title: "Восстановлена база приложения",
|
||||
message: "Данные SQLite заменены из загруженного файла",
|
||||
message: "Данные PostgreSQL заменены из загруженного дампа",
|
||||
entityType: "system_database",
|
||||
entityId: "restore",
|
||||
payload: {
|
||||
|
||||
@@ -94,7 +94,7 @@ function analyticsQuery(req: FastifyRequest) {
|
||||
|
||||
async function sendFlowTalkers(req: FastifyRequest, reply: FastifyReply) {
|
||||
const q = req.query as { range?: string }
|
||||
return reply.send(listFlowTalkers(rangeToMinutes(q.range)))
|
||||
return reply.send(await listFlowTalkers(rangeToMinutes(q.range)))
|
||||
}
|
||||
|
||||
function requestPublicHost(req: FastifyRequest): string {
|
||||
@@ -145,7 +145,7 @@ function sleep(ms: number, signal: AbortSignal): Promise<void> {
|
||||
|
||||
const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/traffic/flow/settings", async (_req, reply) => {
|
||||
return reply.send(toTrafficFlowSettingsDto(getFlowListenerState()))
|
||||
return reply.send(await toTrafficFlowSettingsDto(getFlowListenerState()))
|
||||
})
|
||||
|
||||
app.put("/traffic/flow/settings", async (req, reply) => {
|
||||
@@ -153,31 +153,31 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
updateTrafficFlowSettings(parsed.data)
|
||||
startTrafficFlowListener()
|
||||
return reply.send({ ok: true, settings: toTrafficFlowSettingsDto(getFlowListenerState()) })
|
||||
await updateTrafficFlowSettings(parsed.data)
|
||||
await startTrafficFlowListener()
|
||||
return reply.send({ ok: true, settings: await toTrafficFlowSettingsDto(getFlowListenerState()) })
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/settings/generate-keys", async (_req, reply) => {
|
||||
const result = ensureHostKeys()
|
||||
const result = await ensureHostKeys()
|
||||
return reply.send({
|
||||
ok: true,
|
||||
created: result.created,
|
||||
publicKey: result.publicKey,
|
||||
settings: toTrafficFlowSettingsDto(getFlowListenerState()),
|
||||
settings: await toTrafficFlowSettingsDto(getFlowListenerState()),
|
||||
})
|
||||
})
|
||||
|
||||
app.get("/traffic/flow/host-files", async (_req, reply) => {
|
||||
const row = getTrafficFlowSettingsRow()
|
||||
if (!row.hostPrivateKey) ensureHostKeys()
|
||||
return reply.send({ files: listTrafficFlowHostFiles() })
|
||||
const row = await getTrafficFlowSettingsRow()
|
||||
if (!row.hostPrivateKey) await ensureHostKeys()
|
||||
return reply.send({ files: await listTrafficFlowHostFiles() })
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/purge", async (_req, reply) => {
|
||||
try {
|
||||
const result = await purgeTrafficFlowStore()
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "warning",
|
||||
eventType: "traffic.flow.purge",
|
||||
sourceModule: "traffic",
|
||||
@@ -211,20 +211,20 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
app.get("/traffic/flow/exporters", async (req, reply) => {
|
||||
const q = req.query as { range?: string }
|
||||
return reply.send(listFlowExporters(rangeToMinutes(q.range)))
|
||||
return reply.send(await listFlowExporters(rangeToMinutes(q.range)))
|
||||
})
|
||||
|
||||
app.get("/traffic/flow/clients", async (req, reply) => {
|
||||
const q = req.query as { range?: string }
|
||||
return reply.send(listFlowClients(rangeToMinutes(q.range)))
|
||||
return reply.send(await listFlowClients(rangeToMinutes(q.range)))
|
||||
})
|
||||
|
||||
app.get("/traffic/flow/analytics", async (req, reply) => {
|
||||
return reply.send(buildFlowAnalytics(analyticsQuery(req)))
|
||||
return reply.send(await buildFlowAnalytics(analyticsQuery(req)))
|
||||
})
|
||||
|
||||
app.get("/traffic/flow/map-hops", async (req, reply) => {
|
||||
return reply.send(buildFlowMapHops(analyticsQuery(req)))
|
||||
return reply.send(await buildFlowMapHops(analyticsQuery(req)))
|
||||
})
|
||||
|
||||
app.get("/traffic/flow/monthly", async (req, reply) => {
|
||||
@@ -233,7 +233,7 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const month = /^\d{4}-\d{2}$/.test(q.month ?? "")
|
||||
? (q.month as string)
|
||||
: `${now.getUTCFullYear()}-${String(now.getUTCMonth() + 1).padStart(2, "0")}`
|
||||
return reply.send(getFlowMonthly(month, parseId(q.serverId)))
|
||||
return reply.send(await getFlowMonthly(month, parseId(q.serverId)))
|
||||
})
|
||||
|
||||
app.get("/traffic/flow/live", async (req, reply) => {
|
||||
|
||||
@@ -57,14 +57,13 @@ interface TrafficInterfaceDto {
|
||||
const LIVE_TICK_MS = 1500
|
||||
const LIVE_ROS_TIMEOUT_MS = 4000
|
||||
|
||||
function latestSnapshot(serverId: number): SnapshotRow | undefined {
|
||||
return db
|
||||
async function latestSnapshot(serverId: number) {
|
||||
return (await db
|
||||
.select()
|
||||
.from(serverSnapshots)
|
||||
.where(eq(serverSnapshots.serverId, serverId))
|
||||
.orderBy(desc(serverSnapshots.polledAt))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
}
|
||||
|
||||
function rangeToMinutes(range: string | undefined): number {
|
||||
@@ -114,8 +113,8 @@ function buildServerTraffic(
|
||||
}
|
||||
}
|
||||
|
||||
function snapshotStatus(serverId: number): TrafficServerDto["status"] {
|
||||
const snap = latestSnapshot(serverId)
|
||||
async function snapshotStatus(serverId: number): Promise<TrafficServerDto["status"]> {
|
||||
const snap = await latestSnapshot(serverId)
|
||||
return snap?.status === "offline" ? "offline" : (snap?.status === "online" ? "online" : "degraded")
|
||||
}
|
||||
|
||||
@@ -222,8 +221,8 @@ async function monitorTrafficOnce(
|
||||
|
||||
const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/traffic/settings", async (_req, reply) => {
|
||||
const settings = getTrafficSettings()
|
||||
const state = getTrafficCollectorState()
|
||||
const settings = await getTrafficSettings()
|
||||
const state = await getTrafficCollectorState()
|
||||
return reply.send({
|
||||
enabled: settings.enabled,
|
||||
intervalSec: settings.intervalSec,
|
||||
@@ -243,12 +242,12 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
const intervalSec = body.intervalSec == null ? undefined : Math.max(5, Number.parseInt(String(body.intervalSec), 10) || 30)
|
||||
const retentionDays = body.retentionDays == null ? undefined : Math.max(1, Number.parseInt(String(body.retentionDays), 10) || 14)
|
||||
const updated = updateTrafficSettings({
|
||||
const updated = await updateTrafficSettings({
|
||||
enabled: body.enabled,
|
||||
intervalSec,
|
||||
retentionDays,
|
||||
})
|
||||
refreshScheduler()
|
||||
await refreshScheduler()
|
||||
return reply.send({
|
||||
ok: true,
|
||||
settings: {
|
||||
@@ -266,8 +265,8 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
try {
|
||||
await collectTrafficOnce()
|
||||
scheduleAlertEngineAfterDataCollectors()
|
||||
const updated = getTrafficSettings()
|
||||
appendEvent({
|
||||
const updated = await getTrafficSettings()
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "traffic.collect.manual.ok",
|
||||
sourceModule: "traffic",
|
||||
@@ -285,7 +284,7 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
} catch (error) {
|
||||
const msg = error instanceof Error ? error.message : String(error)
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "critical",
|
||||
eventType: "traffic.collect.manual.failed",
|
||||
sourceModule: "traffic",
|
||||
@@ -302,11 +301,11 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const rangeEndMs = Date.now()
|
||||
const rangeStartMs = rangeEndMs - minutes * 60_000
|
||||
const sinceIso = new Date(rangeStartMs).toISOString()
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const data = allServers.map((s): TrafficServerDto => {
|
||||
const rows = readServerSamplesInRange(s.id, sinceIso)
|
||||
return buildServerTraffic(s, snapshotStatus(s.id), rows, rangeStartMs, rangeEndMs)
|
||||
})
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const data = await Promise.all(allServers.map(async (s): Promise<TrafficServerDto> => {
|
||||
const rows = await readServerSamplesInRange(s.id, sinceIso)
|
||||
return buildServerTraffic(s, await snapshotStatus(s.id), rows, rangeStartMs, rangeEndMs)
|
||||
}))
|
||||
|
||||
return reply.send({ servers: data })
|
||||
})
|
||||
@@ -314,7 +313,7 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/traffic/servers/:id/live", async (req, reply) => {
|
||||
const p = req.params as { id?: string | number }
|
||||
const q = req.query as { iface?: string }
|
||||
const server = getEnabledServerById(p.id ?? "")
|
||||
const server = await getEnabledServerById(p.id ?? "")
|
||||
if (!server || !server.enabled) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const onlyInterface = q.iface && q.iface !== "__all__" ? q.iface : undefined
|
||||
@@ -375,12 +374,12 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const serverId = Number.parseInt(String(p.id ?? ""), 10)
|
||||
if (!Number.isFinite(serverId)) return reply.status(400).send({ error: "id is required" })
|
||||
|
||||
const allRows = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()
|
||||
const allRows = await db.select().from(servers).where(eq(servers.id, serverId)).limit(1)
|
||||
const server = allRows[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const sinceIso = new Date(Date.now() - 24 * 60 * 60 * 1000).toISOString()
|
||||
const rows = readServerSamplesInRange(serverId, sinceIso)
|
||||
const rows = await readServerSamplesInRange(serverId, sinceIso)
|
||||
const byIface = new Map<string, typeof rows>()
|
||||
for (const r of rows) {
|
||||
if (isLoopbackName(r.interfaceName)) continue
|
||||
@@ -413,17 +412,17 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const q = req.query as { range?: string; iface?: string }
|
||||
const serverId = Number.parseInt(String(p.id ?? ""), 10)
|
||||
if (!Number.isFinite(serverId)) return reply.status(400).send({ error: "id is required" })
|
||||
const server = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const minutes = rangeToMinutes(q.range)
|
||||
const rangeEndMs = Date.now()
|
||||
const rangeStartMs = rangeEndMs - minutes * 60_000
|
||||
const sinceIso = new Date(rangeStartMs).toISOString()
|
||||
const rows = readServerSamplesInRange(server.id, sinceIso)
|
||||
const rows = await readServerSamplesInRange(server.id, sinceIso)
|
||||
const data = buildServerTraffic(
|
||||
server,
|
||||
snapshotStatus(server.id),
|
||||
await snapshotStatus(server.id),
|
||||
rows,
|
||||
rangeStartMs,
|
||||
rangeEndMs,
|
||||
@@ -437,7 +436,7 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const minutes = rangeToMinutes(q.range)
|
||||
const rangeEndMs = Date.now()
|
||||
const rangeStartMs = rangeEndMs - minutes * 60_000
|
||||
return reply.send({ users: buildUserTrafficList(rangeStartMs, rangeEndMs) })
|
||||
return reply.send({ users: await buildUserTrafficList(rangeStartMs, rangeEndMs) })
|
||||
})
|
||||
|
||||
app.get("/traffic/users/:id", async (req, reply) => {
|
||||
@@ -448,7 +447,7 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const minutes = rangeToMinutes(q.range)
|
||||
const rangeEndMs = Date.now()
|
||||
const rangeStartMs = rangeEndMs - minutes * 60_000
|
||||
const users = buildUserTrafficList(rangeStartMs, rangeEndMs)
|
||||
const users = await buildUserTrafficList(rangeStartMs, rangeEndMs)
|
||||
const user = users.find((u) => u.id === id)
|
||||
if (!user) return reply.status(404).send({ error: "Пользователь не найден" })
|
||||
return reply.send({ user })
|
||||
@@ -459,7 +458,7 @@ const trafficRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const minutes = rangeToMinutes(q.range)
|
||||
const rangeEndMs = Date.now()
|
||||
const rangeStartMs = rangeEndMs - minutes * 60_000
|
||||
return reply.send({ interfaces: buildBoundInterfaceTraffic(rangeStartMs, rangeEndMs) })
|
||||
return reply.send({ interfaces: await buildBoundInterfaceTraffic(rangeStartMs, rangeEndMs) })
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -36,8 +36,8 @@ function toSeries(values: number[], target = 40): number[] {
|
||||
return [...Array(target - values.length).fill(values[0] ?? 0), ...values]
|
||||
}
|
||||
|
||||
/** Сэмпл ресурсов из SQLite (readResourceSamplesSince). */
|
||||
type ResourceSampleRow = ReturnType<typeof readResourceSamplesSince>[number]
|
||||
/** Сэмпл ресурсов из PostgreSQL (readResourceSamplesSince). */
|
||||
type ResourceSampleRow = Awaited<ReturnType<typeof readResourceSamplesSince>>[number]
|
||||
|
||||
/**
|
||||
* Последний сэмпл по времени может быть offline после сбоя опроса — тогда UI «пустой».
|
||||
@@ -67,7 +67,7 @@ function pickResourceDisplayRow(rows: ResourceSampleRow[], maxGapMs: number): {
|
||||
|
||||
const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/uptime/settings", async (_req, reply) => {
|
||||
const s = readUptimeSettings()
|
||||
const s = await readUptimeSettings()
|
||||
return reply.send({
|
||||
enabled: s.enabled,
|
||||
resourcesEnabled: s.resourcesEnabled ?? s.enabled,
|
||||
@@ -82,7 +82,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
lastCollectedAt: s.lastCollectedAt ?? null,
|
||||
lastDurationMs: s.lastDurationMs ?? null,
|
||||
lastError: s.lastError || null,
|
||||
scheduler: getSchedulerStatus(),
|
||||
scheduler: await getSchedulerStatus(),
|
||||
})
|
||||
})
|
||||
|
||||
@@ -97,7 +97,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
speedIntervalSec?: number | string
|
||||
retentionDays?: number | string
|
||||
}
|
||||
const updated = updateUptimeSettings({
|
||||
const updated = await updateUptimeSettings({
|
||||
enabled: body.enabled,
|
||||
resourcesEnabled: body.resourcesEnabled,
|
||||
pingEnabled: body.pingEnabled,
|
||||
@@ -107,7 +107,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
speedIntervalSec: body.speedIntervalSec == null ? undefined : Math.max(10, Number.parseInt(String(body.speedIntervalSec), 10) || 60),
|
||||
retentionDays: body.retentionDays == null ? undefined : Math.max(1, Number.parseInt(String(body.retentionDays), 10) || 14),
|
||||
})
|
||||
refreshScheduler()
|
||||
await refreshScheduler()
|
||||
return reply.send({
|
||||
ok: true,
|
||||
settings: {
|
||||
@@ -122,7 +122,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
lastCollectedAt: updated.lastCollectedAt ?? null,
|
||||
lastDurationMs: updated.lastDurationMs ?? null,
|
||||
lastError: updated.lastError || null,
|
||||
scheduler: getSchedulerStatus(),
|
||||
scheduler: await getSchedulerStatus(),
|
||||
},
|
||||
})
|
||||
})
|
||||
@@ -130,7 +130,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.post("/uptime/collect-now", async (_req, reply) => {
|
||||
await collectUptimeOnce()
|
||||
scheduleAlertEngineAfterDataCollectors()
|
||||
const s = readUptimeSettings()
|
||||
const s = await readUptimeSettings()
|
||||
return reply.send({
|
||||
ok: true,
|
||||
lastCollectedAt: s.lastCollectedAt ?? null,
|
||||
@@ -158,7 +158,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
app.get("/uptime/probes", async (_req, reply) => {
|
||||
const rows = readProbeRows()
|
||||
const rows = await readProbeRows()
|
||||
return reply.send({
|
||||
probes: rows.map((p) => ({
|
||||
id: p.id,
|
||||
@@ -178,7 +178,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
app.get("/uptime/speed-probes", async (_req, reply) => {
|
||||
const rows = db.select().from(uptimeSpeedProbes).orderBy(asc(uptimeSpeedProbes.sortOrder)).all()
|
||||
const rows = await db.select().from(uptimeSpeedProbes).orderBy(asc(uptimeSpeedProbes.sortOrder))
|
||||
return reply.send({
|
||||
probes: rows.map((r) => ({
|
||||
id: r.id,
|
||||
@@ -204,7 +204,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
app.get("/uptime/speed-test/runs", async (_req, reply) => {
|
||||
const rows = db.select().from(uptimeSpeedTestRuns).orderBy(desc(uptimeSpeedTestRuns.createdAt)).limit(200).all()
|
||||
const rows = await db.select().from(uptimeSpeedTestRuns).orderBy(desc(uptimeSpeedTestRuns.createdAt)).limit(200)
|
||||
return reply.send({
|
||||
runs: rows.map((r) => ({
|
||||
id: r.id,
|
||||
@@ -263,12 +263,12 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
.filter((p) => Number.isFinite(p.srcServerId) && Number.isFinite(p.dstServerId) && p.srcServerId !== p.dstServerId)
|
||||
|
||||
const now = new Date().toISOString()
|
||||
const existing = db.select({ id: uptimeSpeedProbes.id }).from(uptimeSpeedProbes).all()
|
||||
const existing = await db.select({ id: uptimeSpeedProbes.id }).from(uptimeSpeedProbes)
|
||||
const nextIds = new Set(normalized.map((p) => p.id))
|
||||
|
||||
for (const row of existing) {
|
||||
if (nextIds.has(row.id)) continue
|
||||
db.delete(uptimeSpeedProbes).where(eq(uptimeSpeedProbes.id, row.id)).run()
|
||||
await db.delete(uptimeSpeedProbes).where(eq(uptimeSpeedProbes.id, row.id))
|
||||
}
|
||||
|
||||
for (let i = 0; i < normalized.length; i += 1) {
|
||||
@@ -285,15 +285,15 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
sortOrder: i,
|
||||
updatedAt: now,
|
||||
}
|
||||
const updated = db.update(uptimeSpeedProbes).set(patch).where(eq(uptimeSpeedProbes.id, p.id)).run()
|
||||
if ((updated.changes ?? 0) > 0) continue
|
||||
db.insert(uptimeSpeedProbes).values({
|
||||
const updated = await db.update(uptimeSpeedProbes).set(patch).where(eq(uptimeSpeedProbes.id, p.id)).returning({ id: uptimeSpeedProbes.id })
|
||||
if (updated.length > 0) continue
|
||||
await db.insert(uptimeSpeedProbes).values({
|
||||
id: p.id,
|
||||
...patch,
|
||||
createdAt: now,
|
||||
}).run()
|
||||
})
|
||||
}
|
||||
refreshScheduler()
|
||||
await refreshScheduler()
|
||||
return reply.send({ ok: true })
|
||||
})
|
||||
|
||||
@@ -326,8 +326,8 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
: Number.parseInt(String(p.intervalSec), 10),
|
||||
}))
|
||||
.filter((p) => Number.isFinite(p.srcServerId) && p.name && p.target)
|
||||
replaceProbes(normalized)
|
||||
refreshScheduler()
|
||||
await replaceProbes(normalized)
|
||||
await refreshScheduler()
|
||||
return reply.send({ ok: true })
|
||||
})
|
||||
|
||||
@@ -339,7 +339,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (typeof body.showOnDashboard !== "boolean") {
|
||||
return reply.status(400).send({ error: "showOnDashboard boolean required" })
|
||||
}
|
||||
const ok = updateProbeShowOnDashboard(probeId, body.showOnDashboard)
|
||||
const ok = await updateProbeShowOnDashboard(probeId, body.showOnDashboard)
|
||||
if (!ok) return reply.status(404).send({ error: "Probe not found" })
|
||||
return reply.send({ ok: true })
|
||||
})
|
||||
@@ -348,7 +348,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const params = req.params as { id?: string | number }
|
||||
const serverId = Number.parseInt(String(params.id ?? ""), 10)
|
||||
if (!Number.isFinite(serverId)) return reply.status(400).send({ error: "Invalid server id" })
|
||||
const srv = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const srv = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!srv) return reply.status(404).send({ error: "Server not found" })
|
||||
try {
|
||||
const rows = await MikrotikClient.fromServer(srv).getInterfaces()
|
||||
@@ -386,8 +386,8 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Invalid src/dst server id" })
|
||||
}
|
||||
if (srcId === dstId) return reply.status(400).send({ error: "Source and destination must be different" })
|
||||
const src = db.select().from(servers).where(eq(servers.id, srcId)).limit(1).all()[0]
|
||||
const dst = db.select().from(servers).where(eq(servers.id, dstId)).limit(1).all()[0]
|
||||
const src = (await db.select().from(servers).where(eq(servers.id, srcId)).limit(1))[0]
|
||||
const dst = (await db.select().from(servers).where(eq(servers.id, dstId)).limit(1))[0]
|
||||
if (!src || !dst) return reply.status(404).send({ error: "Server not found" })
|
||||
try {
|
||||
const result = await runBandwidthSpeedTest({
|
||||
@@ -404,7 +404,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send({ ok: true, result })
|
||||
} catch (e) {
|
||||
const raw = e instanceof Error ? e.message : "Bandwidth test failed"
|
||||
recordSpeedTestFailure({
|
||||
await recordSpeedTestFailure({
|
||||
probeId: String(body.probeId ?? "").trim() || undefined,
|
||||
runId: String(body.runId ?? "").trim() || undefined,
|
||||
srcServerId: srcId,
|
||||
@@ -430,7 +430,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const params = req.params as { id?: string | number }
|
||||
const serverId = Number.parseInt(String(params.id ?? ""), 10)
|
||||
if (!Number.isFinite(serverId)) return reply.status(400).send({ error: "Invalid server id" })
|
||||
const srv = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const srv = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!srv) return reply.status(404).send({ error: "Server not found" })
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(srv)
|
||||
@@ -473,10 +473,10 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const minutes = rangeToMinutes(q.range)
|
||||
const sinceIso = new Date(Date.now() - minutes * 60_000).toISOString()
|
||||
|
||||
const allServers = db.select().from(servers).all()
|
||||
const probeRows = readProbeRows()
|
||||
const probeSamples = readProbeSamplesSince(sinceIso)
|
||||
const uptimeCfg = readUptimeSettings()
|
||||
const allServers = await db.select().from(servers)
|
||||
const probeRows = await readProbeRows()
|
||||
const probeSamples = await readProbeSamplesSince(sinceIso)
|
||||
const uptimeCfg = await readUptimeSettings()
|
||||
const intervalSec = Math.max(15, uptimeCfg.intervalSec ?? 300)
|
||||
/** Допустимый разрыв между последней попыткой и последним «хорошим» сэмплом (не показывать данные часовой давности). */
|
||||
const resourceFallbackMaxGapMs = Math.min(2 * 3600 * 1000, Math.max(10 * 60 * 1000, intervalSec * 6 * 1000))
|
||||
@@ -501,8 +501,8 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
})
|
||||
|
||||
const resources = allServers.map((s) => {
|
||||
const rows = readResourceSamplesSince(sinceIso, s.id)
|
||||
const resources = await Promise.all(allServers.map(async (s) => {
|
||||
const rows = await readResourceSamplesSince(sinceIso, s.id)
|
||||
const { row: pick, hasData } = pickResourceDisplayRow(rows, resourceFallbackMaxGapMs)
|
||||
const cpuHistory = toSeries(rows.map((r) => r.cpuLoad), 40)
|
||||
return {
|
||||
@@ -518,7 +518,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
boardName: pick?.boardName || "RouterBOARD",
|
||||
temp: undefined as number | undefined,
|
||||
}
|
||||
})
|
||||
}))
|
||||
|
||||
return reply.send({ probes, resources })
|
||||
})
|
||||
|
||||
@@ -31,7 +31,7 @@ function sendServiceError(reply: { status: (c: number) => { send: (b: unknown) =
|
||||
|
||||
const usersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/users", async (_req, reply) => {
|
||||
return reply.send({ users: listUsers() })
|
||||
return reply.send({ users: await listUsers() })
|
||||
})
|
||||
|
||||
app.get("/users/interface-catalog", {
|
||||
@@ -49,7 +49,7 @@ const usersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
schema: { params: appUserIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params as { id: string }
|
||||
const user = getUserById(id)
|
||||
const user = await getUserById(id)
|
||||
if (!user) return reply.status(404).send({ error: "Пользователь не найден" })
|
||||
return reply.send({ user })
|
||||
})
|
||||
@@ -58,7 +58,7 @@ const usersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
schema: { body: appUserCreateSchema },
|
||||
}, async (req, reply) => {
|
||||
try {
|
||||
const user = createUser(req.body as AppUserCreate)
|
||||
const user = await createUser(req.body as AppUserCreate)
|
||||
return reply.status(201).send({ user })
|
||||
} catch (err) {
|
||||
return sendServiceError(reply, err)
|
||||
@@ -70,7 +70,7 @@ const usersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params as { id: string }
|
||||
try {
|
||||
const user = updateUser(id, req.body as AppUserUpdate)
|
||||
const user = await updateUser(id, req.body as AppUserUpdate)
|
||||
return reply.send({ user })
|
||||
} catch (err) {
|
||||
return sendServiceError(reply, err)
|
||||
@@ -82,7 +82,7 @@ const usersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params as { id: string }
|
||||
try {
|
||||
deleteUser(id)
|
||||
await deleteUser(id)
|
||||
return reply.status(204).send()
|
||||
} catch (err) {
|
||||
return sendServiceError(reply, err)
|
||||
@@ -94,7 +94,7 @@ const usersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params as { id: string }
|
||||
try {
|
||||
const binding = addBinding(id, req.body as UserBindingCreate)
|
||||
const binding = await addBinding(id, req.body as UserBindingCreate)
|
||||
return reply.status(201).send({ binding })
|
||||
} catch (err) {
|
||||
return sendServiceError(reply, err)
|
||||
@@ -106,7 +106,7 @@ const usersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}, async (req, reply) => {
|
||||
const { id, bindingId } = req.params as { id: string; bindingId: string }
|
||||
try {
|
||||
removeBinding(id, bindingId)
|
||||
await removeBinding(id, bindingId)
|
||||
return reply.status(204).send()
|
||||
} catch (err) {
|
||||
return sendServiceError(reply, err)
|
||||
|
||||
@@ -154,7 +154,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = getEnabledServerById(body.serverId)
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
@@ -194,7 +194,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const server = getEnabledServerById(serverIdParam(serverId))
|
||||
const server = await getEnabledServerById(serverIdParam(serverId))
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const d = parsed.data
|
||||
@@ -219,7 +219,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
app.delete("/wireguard/interfaces/:serverId/:rosId", async (req, reply) => {
|
||||
const { serverId, rosId } = req.params as { serverId: string; rosId: string }
|
||||
const server = getEnabledServerById(serverIdParam(serverId))
|
||||
const server = await getEnabledServerById(serverIdParam(serverId))
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
@@ -237,7 +237,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = getEnabledServerById(body.serverId)
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
@@ -255,7 +255,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const server = getEnabledServerById(serverIdParam(serverId))
|
||||
const server = await getEnabledServerById(serverIdParam(serverId))
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const d = parsed.data
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
@@ -286,7 +286,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
app.delete("/wireguard/peers/:serverId/:rosId", async (req, reply) => {
|
||||
const { serverId, rosId } = req.params as { serverId: string; rosId: string }
|
||||
const server = getEnabledServerById(serverIdParam(serverId))
|
||||
const server = await getEnabledServerById(serverIdParam(serverId))
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
@@ -315,7 +315,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send({ dryRun: true, preview })
|
||||
}
|
||||
|
||||
const server = getEnabledServerById(body.serverId)
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
@@ -333,14 +333,14 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = getEnabledServerById(body.serverId)
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
|
||||
const list = await listWireGuardInterfaces({
|
||||
serverId: String(server.id),
|
||||
includePrivateKey: body.includePrivateKey === true,
|
||||
})
|
||||
const iface = findIface(list.interfaces, String(server.id), body.interfaceName)
|
||||
const iface = await findIface(list.interfaces, String(server.id), body.interfaceName)
|
||||
if (!iface) return reply.status(404).send({ error: "Интерфейс не найден" })
|
||||
|
||||
if (body.format === "rsc") {
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
import { env } from "../config.js"
|
||||
import { closePool, pool } from "../db/index.js"
|
||||
import { applySqlMigrations } from "../db/migrate.js"
|
||||
import { ensurePartitionsAround } from "../db/partitions.js"
|
||||
import { importSqliteToPostgres } from "../db/sqlite-import.js"
|
||||
|
||||
const sqlitePath = process.argv.find((a) => a.startsWith("--sqlite="))?.slice(9) ?? env.DATABASE_PATH
|
||||
const dryRun = process.argv.includes("--dry-run")
|
||||
const fullHistory = process.argv.includes("--full-history")
|
||||
const strict = process.argv.includes("--strict") || !dryRun
|
||||
|
||||
await applySqlMigrations(pool)
|
||||
await ensurePartitionsAround(pool)
|
||||
const report = await importSqliteToPostgres(pool, sqlitePath, { dryRun, strict, fullHistory })
|
||||
console.log(JSON.stringify(report, null, 2))
|
||||
if (!dryRun && report.rejects.length > 0 && strict) {
|
||||
await closePool()
|
||||
process.exit(1)
|
||||
}
|
||||
await closePool()
|
||||
@@ -85,15 +85,11 @@ async function deleteTxtRecord(token: string, zoneId: string, recordId: string):
|
||||
await cloudflareRequest(token, `/zones/${zoneId}/dns_records/${recordId}`, { method: "DELETE" })
|
||||
}
|
||||
|
||||
function parseWanUplinks(raw: string | null | undefined): WanUplinkRow[] {
|
||||
if (!raw?.trim()) return []
|
||||
try {
|
||||
const data = JSON.parse(raw) as unknown
|
||||
if (!Array.isArray(data)) return []
|
||||
return data.filter((row): row is WanUplinkRow => row != null && typeof row === "object")
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
function parseWanUplinks(raw: unknown): WanUplinkRow[] {
|
||||
const data = Array.isArray(raw) ? raw : typeof raw === "string" && raw.trim()
|
||||
? (() => { try { const p = JSON.parse(raw) as unknown; return Array.isArray(p) ? p : [] } catch { return [] } })()
|
||||
: []
|
||||
return data.filter((row): row is WanUplinkRow => row != null && typeof row === "object")
|
||||
}
|
||||
|
||||
function normalizeIpv4(raw: string | undefined): string | null {
|
||||
@@ -217,10 +213,10 @@ async function sleep(ms: number) {
|
||||
}
|
||||
|
||||
async function getOrCreateAccountKey(): Promise<Buffer> {
|
||||
const existing = getAcmeAccountPrivateKey()
|
||||
const existing = await getAcmeAccountPrivateKey()
|
||||
if (existing) return Buffer.from(existing)
|
||||
const key = await acme.crypto.createPrivateKey()
|
||||
saveAcmeAccountPrivateKey(key.toString("utf8"))
|
||||
await saveAcmeAccountPrivateKey(key.toString("utf8"))
|
||||
return key
|
||||
}
|
||||
|
||||
@@ -247,8 +243,8 @@ export async function issueCertificateWithCloudflareDns(params: {
|
||||
trustStore: string[]
|
||||
onStep?: (step: string) => void
|
||||
}): Promise<void> {
|
||||
const settings = getAcmeSettingsPublic()
|
||||
const token = getAcmeCloudflareToken()
|
||||
const settings = await getAcmeSettingsPublic()
|
||||
const token = await getAcmeCloudflareToken()
|
||||
if (!token) throw new Error("Не настроен Cloudflare API token")
|
||||
|
||||
const domains = [...new Set(params.domainNames.map((d) => d.trim().toLowerCase()).filter(Boolean))]
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
/**
|
||||
* Внутренние хуки: после записи сигналов в SQLite (коллекторы) — отложенный прогон `alert_engine`.
|
||||
* Внутренние хуки: после записи сигналов в PostgreSQL (коллекторы) — отложенный прогон `alert_engine`.
|
||||
* Не импортирует `scheduler.ts` (избегаем цикла); runner задаётся через `wireAlertEngineRunner`.
|
||||
*
|
||||
* Любой код, пишущий таблицы для `buildSignalSnapshot`, после успешного коммита должен вызывать
|
||||
@@ -15,7 +15,7 @@ let alertAfterCollectChain: Promise<void> = Promise.resolve()
|
||||
|
||||
let alertEngineRunner: (() => Promise<void>) | null = null
|
||||
|
||||
/** Задать runner один раз при старте приложения (обычно `() => executeSchedulerJob("alert_engine")`). */
|
||||
/** Задать runner один раз при старте приложения (обычно `() => await executeSchedulerJob("alert_engine")`). */
|
||||
export function wireAlertEngineRunner(run: () => Promise<void>): void {
|
||||
alertEngineRunner = run
|
||||
}
|
||||
@@ -39,7 +39,7 @@ async function runAlertEngineWhenIdle(): Promise<void> {
|
||||
/** После успешной записи сигналов в БД коллекторами (не из `alert_engine`). */
|
||||
export function scheduleAlertEngineAfterDataCollectors(): void {
|
||||
if (alertAfterCollectTimer != null) clearTimeout(alertAfterCollectTimer)
|
||||
alertAfterCollectTimer = setTimeout(() => {
|
||||
alertAfterCollectTimer = setTimeout(async () => {
|
||||
alertAfterCollectTimer = null
|
||||
alertAfterCollectChain = alertAfterCollectChain
|
||||
.then(() => runAlertEngineWhenIdle())
|
||||
|
||||
@@ -4,11 +4,10 @@ import { db } from "../../db/index.js"
|
||||
import { alertEngineConfirmPending } from "../../db/schema.js"
|
||||
import type { ApiAlertRule } from "../alerts-service.js"
|
||||
import { computeStabilityReadyMap, deleteConfirmPending } from "./confirm-stability.js"
|
||||
import { withPgOrSkip } from "../../test/pg.js"
|
||||
|
||||
const ruleId = "test-confirm-stability-recovery-bypass"
|
||||
|
||||
deleteConfirmPending(ruleId)
|
||||
|
||||
const baseRule: ApiAlertRule = {
|
||||
id: ruleId,
|
||||
name: "Server transition",
|
||||
@@ -46,25 +45,31 @@ const recoveryHit = {
|
||||
payloadHash: "recovery-1",
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("alert-engine confirm-stability tests skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
await deleteConfirmPending(ruleId)
|
||||
|
||||
{
|
||||
const m = computeStabilityReadyMap([baseRule], new Map([[ruleId, problemHit]]))
|
||||
const m = await computeStabilityReadyMap([baseRule], new Map([[ruleId, problemHit]]))
|
||||
assert.equal(m.get(ruleId), false, "проблемный переход должен ждать confirmStabilitySec")
|
||||
}
|
||||
|
||||
{
|
||||
const m = computeStabilityReadyMap([baseRule], new Map([[ruleId, recoveryHit]]))
|
||||
const m = await computeStabilityReadyMap([baseRule], new Map([[ruleId, recoveryHit]]))
|
||||
assert.equal(m.get(ruleId), true, "восстановление должно отправляться без задержки")
|
||||
}
|
||||
|
||||
{
|
||||
const m = computeStabilityReadyMap([baseRule], new Map([[ruleId, null]]))
|
||||
const m = await computeStabilityReadyMap([baseRule], new Map([[ruleId, null]]))
|
||||
assert.equal(m.get(ruleId), false, "без hit отправка не готова")
|
||||
const left = db
|
||||
const left = (await db
|
||||
.select()
|
||||
.from(alertEngineConfirmPending)
|
||||
.where(eq(alertEngineConfirmPending.ruleId, ruleId))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
assert.equal(left, undefined, "pending должен очищаться после отсутствия hit")
|
||||
}
|
||||
|
||||
|
||||
@@ -4,46 +4,44 @@ import { alertEngineConfirmPending } from "../../db/schema.js"
|
||||
import type { ApiAlertRule } from "../alerts-service.js"
|
||||
import type { RuleEvalHit } from "./types.js"
|
||||
|
||||
export function deleteConfirmPending(ruleId: string) {
|
||||
db.delete(alertEngineConfirmPending).where(eq(alertEngineConfirmPending.ruleId, ruleId)).run()
|
||||
export async function deleteConfirmPending(ruleId: string) {
|
||||
await db.delete(alertEngineConfirmPending).where(eq(alertEngineConfirmPending.ruleId, ruleId))
|
||||
}
|
||||
|
||||
/**
|
||||
* Обновляет состояние ожидания стабильности и возвращает, можно ли отправлять уведомление в этом тике.
|
||||
* При отсутствии срабатывания (`hit == null`) ожидание сбрасывается.
|
||||
*/
|
||||
export function stabilityReadyForSend(
|
||||
export async function stabilityReadyForSend(
|
||||
ruleId: string,
|
||||
hit: RuleEvalHit | null,
|
||||
delaySec: number | null | undefined,
|
||||
): boolean {
|
||||
): Promise<boolean> {
|
||||
if (!hit) {
|
||||
deleteConfirmPending(ruleId)
|
||||
await deleteConfirmPending(ruleId)
|
||||
return false
|
||||
}
|
||||
const sec = delaySec != null && delaySec > 0 ? Math.floor(delaySec) : 0
|
||||
if (sec <= 0) {
|
||||
deleteConfirmPending(ruleId)
|
||||
await deleteConfirmPending(ruleId)
|
||||
return true
|
||||
}
|
||||
|
||||
const now = Date.now()
|
||||
const nowIso = new Date().toISOString()
|
||||
const existing = db
|
||||
const existing = (await db
|
||||
.select()
|
||||
.from(alertEngineConfirmPending)
|
||||
.where(eq(alertEngineConfirmPending.ruleId, ruleId))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
|
||||
if (!existing || existing.payloadHash !== hit.payloadHash) {
|
||||
if (existing) {
|
||||
db.update(alertEngineConfirmPending)
|
||||
await db.update(alertEngineConfirmPending)
|
||||
.set({ payloadHash: hit.payloadHash, sinceAt: nowIso })
|
||||
.where(eq(alertEngineConfirmPending.ruleId, ruleId))
|
||||
.run()
|
||||
} else {
|
||||
db.insert(alertEngineConfirmPending).values({ ruleId, payloadHash: hit.payloadHash, sinceAt: nowIso }).run()
|
||||
await db.insert(alertEngineConfirmPending).values({ ruleId, payloadHash: hit.payloadHash, sinceAt: nowIso })
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -54,18 +52,18 @@ export function stabilityReadyForSend(
|
||||
}
|
||||
|
||||
/** После успешной отправки — сбросить таймер стабильности для следующего цикла. */
|
||||
export function clearConfirmPendingAfterSend(ruleIds: string[]) {
|
||||
for (const id of ruleIds) deleteConfirmPending(id)
|
||||
export async function clearConfirmPendingAfterSend(ruleIds: string[]) {
|
||||
for (const id of ruleIds) await deleteConfirmPending(id)
|
||||
}
|
||||
|
||||
export function computeStabilityReadyMap(
|
||||
export async function computeStabilityReadyMap(
|
||||
rules: ApiAlertRule[],
|
||||
hitByRule: Map<string, RuleEvalHit | null>,
|
||||
): Map<string, boolean> {
|
||||
): Promise<Map<string, boolean>> {
|
||||
const m = new Map<string, boolean>()
|
||||
for (const rule of rules) {
|
||||
if (!rule.enabled) {
|
||||
deleteConfirmPending(rule.id)
|
||||
await deleteConfirmPending(rule.id)
|
||||
m.set(rule.id, false)
|
||||
continue
|
||||
}
|
||||
@@ -76,7 +74,7 @@ export function computeStabilityReadyMap(
|
||||
? rule.recoveryStabilitySec
|
||||
: 0
|
||||
: rule.confirmStabilitySec
|
||||
m.set(rule.id, stabilityReadyForSend(rule.id, hit, delaySec))
|
||||
m.set(rule.id, await stabilityReadyForSend(rule.id, hit, delaySec))
|
||||
}
|
||||
return m
|
||||
}
|
||||
|
||||
@@ -73,20 +73,20 @@ export function buildRuleDiagnostics(
|
||||
return out
|
||||
}
|
||||
|
||||
export function computeDecisions(args: {
|
||||
export async function computeDecisions(args: {
|
||||
rules: ApiAlertRule[]
|
||||
groups: ApiAlertGroup[]
|
||||
hitByRule: Map<string, RuleEvalHit | null>
|
||||
state: EngineStateMap
|
||||
canSend: boolean
|
||||
}): {
|
||||
}): Promise<{
|
||||
stabilityReady: Map<string, boolean>
|
||||
standalone: StandaloneDecision[]
|
||||
grouped: GroupDecision[]
|
||||
ruleDiag: AlertEngineRuleDiag[]
|
||||
} {
|
||||
}> {
|
||||
const { rules, groups, hitByRule, state, canSend } = args
|
||||
const stabilityReady = computeStabilityReadyMap(rules, hitByRule)
|
||||
const stabilityReady = await computeStabilityReadyMap(rules, hitByRule)
|
||||
const ruleDiag = buildRuleDiagnostics(rules, hitByRule, stabilityReady, state, canSend)
|
||||
const standalone: StandaloneDecision[] = []
|
||||
const grouped: GroupDecision[] = []
|
||||
|
||||
@@ -17,7 +17,9 @@ export type AlertOutboxPayload = {
|
||||
}
|
||||
}
|
||||
|
||||
function safeParsePayload(raw: string): AlertOutboxPayload | null {
|
||||
function safeParsePayload(raw: unknown): AlertOutboxPayload | null {
|
||||
if (raw && typeof raw === "object") return raw as AlertOutboxPayload
|
||||
if (typeof raw !== "string") return null
|
||||
try {
|
||||
return JSON.parse(raw) as AlertOutboxPayload
|
||||
} catch {
|
||||
@@ -25,21 +27,20 @@ function safeParsePayload(raw: string): AlertOutboxPayload | null {
|
||||
}
|
||||
}
|
||||
|
||||
export function enqueueTelegramOutbox(input: {
|
||||
export async function enqueueTelegramOutbox(input: {
|
||||
id: string
|
||||
dedupeKey: string
|
||||
payload: AlertOutboxPayload
|
||||
maxRetries?: number
|
||||
}): boolean {
|
||||
const existing = db
|
||||
}): Promise<boolean> {
|
||||
const existing = (await db
|
||||
.select()
|
||||
.from(alertOutbox)
|
||||
.where(eq(alertOutbox.dedupeKey, input.dedupeKey))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
if (existing && (existing.status === "pending" || existing.status === "sent")) return false
|
||||
const nowIso = new Date().toISOString()
|
||||
db.insert(alertOutbox)
|
||||
await db.insert(alertOutbox)
|
||||
.values({
|
||||
id: input.id,
|
||||
dedupeKey: input.dedupeKey,
|
||||
@@ -48,22 +49,20 @@ export function enqueueTelegramOutbox(input: {
|
||||
retryCount: 0,
|
||||
maxRetries: Math.max(1, Math.floor(input.maxRetries ?? 3)),
|
||||
nextAttemptAt: nowIso,
|
||||
payloadJson: JSON.stringify(input.payload),
|
||||
payloadJson: input.payload,
|
||||
})
|
||||
.run()
|
||||
return true
|
||||
}
|
||||
|
||||
export function dispatchPendingOutbox(limit = 25): Promise<{ sent: number; failed: number; errors: string[] }> {
|
||||
return (async () => {
|
||||
const nowIso = new Date().toISOString()
|
||||
const rows = db
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(alertOutbox)
|
||||
.where(and(eq(alertOutbox.status, "pending"), lte(alertOutbox.nextAttemptAt, nowIso)))
|
||||
.orderBy(asc(alertOutbox.createdAt))
|
||||
.limit(Math.max(1, limit))
|
||||
.all()
|
||||
|
||||
let sent = 0
|
||||
let failed = 0
|
||||
@@ -72,27 +71,25 @@ export function dispatchPendingOutbox(limit = 25): Promise<{ sent: number; faile
|
||||
for (const row of rows) {
|
||||
const payload = safeParsePayload(row.payloadJson)
|
||||
if (!payload) {
|
||||
db.update(alertOutbox)
|
||||
await db.update(alertOutbox)
|
||||
.set({
|
||||
status: "failed",
|
||||
lastError: "invalid payload_json",
|
||||
})
|
||||
.where(eq(alertOutbox.id, row.id))
|
||||
.run()
|
||||
failed += 1
|
||||
continue
|
||||
}
|
||||
const send = await sendTelegramAlertMessage({ text: payload.text, chatId: payload.chatId })
|
||||
if (send.ok) {
|
||||
db.update(alertOutbox)
|
||||
await db.update(alertOutbox)
|
||||
.set({
|
||||
status: "sent",
|
||||
sentAt: new Date().toISOString(),
|
||||
lastError: null,
|
||||
})
|
||||
.where(eq(alertOutbox.id, row.id))
|
||||
.run()
|
||||
appendAlertHistory({
|
||||
await appendAlertHistory({
|
||||
id: payload.history.id,
|
||||
ruleId: payload.history.ruleId ?? null,
|
||||
groupId: payload.history.groupId ?? null,
|
||||
@@ -107,7 +104,7 @@ export function dispatchPendingOutbox(limit = 25): Promise<{ sent: number; faile
|
||||
}
|
||||
const nextRetry = row.retryCount + 1
|
||||
const exhausted = nextRetry >= row.maxRetries
|
||||
db.update(alertOutbox)
|
||||
await db.update(alertOutbox)
|
||||
.set({
|
||||
retryCount: nextRetry,
|
||||
status: exhausted ? "failed" : "pending",
|
||||
@@ -115,9 +112,8 @@ export function dispatchPendingOutbox(limit = 25): Promise<{ sent: number; faile
|
||||
lastError: send.error,
|
||||
})
|
||||
.where(eq(alertOutbox.id, row.id))
|
||||
.run()
|
||||
if (exhausted) {
|
||||
appendAlertHistory({
|
||||
await appendAlertHistory({
|
||||
id: payload.history.id,
|
||||
ruleId: payload.history.ruleId ?? null,
|
||||
groupId: payload.history.groupId ?? null,
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../../db/index.js"
|
||||
import { parseJsonObject } from "../../db/json.js"
|
||||
import { alertEnginePrevLive } from "../../db/schema.js"
|
||||
|
||||
export type PrevLiveKind = "gre" | "bgp"
|
||||
@@ -7,41 +8,32 @@ export type PrevLiveKind = "gre" | "bgp"
|
||||
/** Предыдущие строковые состояния (GRE: up|down|degraded; BGP: state как с роутера). */
|
||||
export type PrevLiveStringMap = Record<string, string>
|
||||
|
||||
function safeParseMap(json: string | null | undefined): PrevLiveStringMap {
|
||||
if (!json || json.trim() === "") return {}
|
||||
try {
|
||||
const v = JSON.parse(json) as unknown
|
||||
if (v == null || typeof v !== "object" || Array.isArray(v)) return {}
|
||||
return v as PrevLiveStringMap
|
||||
} catch {
|
||||
return {}
|
||||
}
|
||||
function safeParseMap(json: unknown): PrevLiveStringMap {
|
||||
return parseJsonObject(json) as PrevLiveStringMap
|
||||
}
|
||||
|
||||
export function loadPrevLiveMap(kind: PrevLiveKind): PrevLiveStringMap {
|
||||
const row = db
|
||||
export async function loadPrevLiveMap(kind: PrevLiveKind): Promise<PrevLiveStringMap> {
|
||||
const row = (await db
|
||||
.select()
|
||||
.from(alertEnginePrevLive)
|
||||
.where(eq(alertEnginePrevLive.kind, kind))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
return safeParseMap(row?.payloadJson)
|
||||
}
|
||||
|
||||
export function savePrevLiveMap(kind: PrevLiveKind, map: PrevLiveStringMap) {
|
||||
const payloadJson = JSON.stringify(map)
|
||||
const existing = db
|
||||
export async function savePrevLiveMap(kind: PrevLiveKind, map: PrevLiveStringMap) {
|
||||
const payloadJson = map
|
||||
const existing = (await db
|
||||
.select()
|
||||
.from(alertEnginePrevLive)
|
||||
.where(eq(alertEnginePrevLive.kind, kind))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
if (existing) {
|
||||
db.update(alertEnginePrevLive)
|
||||
if (existing.payloadJson === payloadJson) return
|
||||
await db.update(alertEnginePrevLive)
|
||||
.set({ payloadJson, updatedAt: new Date().toISOString() })
|
||||
.where(eq(alertEnginePrevLive.kind, kind))
|
||||
.run()
|
||||
} else {
|
||||
db.insert(alertEnginePrevLive).values({ kind, payloadJson, updatedAt: new Date().toISOString() }).run()
|
||||
return
|
||||
}
|
||||
await db.insert(alertEnginePrevLive).values({ kind, payloadJson, updatedAt: new Date().toISOString() })
|
||||
}
|
||||
|
||||
@@ -26,7 +26,7 @@ const SNAPSHOT_SOURCE_WAIT_MS = 30_000
|
||||
const SNAPSHOT_SOURCE_POLL_MS = 20
|
||||
|
||||
/**
|
||||
* Не строить снимок, пока коллекторы пишут в SQLite — иначе гонка с `alert_engine` по таймеру
|
||||
* Не строить снимок, пока коллекторы пишут в PostgreSQL — иначе гонка с `alert_engine` по таймеру
|
||||
* (в т.ч. слот планировщика занят до первого `await` в коллекторе, когда внутренний `collecting` ещё false).
|
||||
*/
|
||||
async function awaitSnapshotSourcesIdle(): Promise<void> {
|
||||
@@ -52,8 +52,8 @@ function newOutboxId(): string {
|
||||
return `ao-${Date.now()}-${Math.random().toString(36).slice(2, 10)}`
|
||||
}
|
||||
|
||||
function loadEngineStateMap(): Map<string, { lastFiredAt: string; lastPayloadHash: string | null }> {
|
||||
const rows = db.select().from(alertEngineState).all()
|
||||
async function loadEngineStateMap(): Promise<Map<string, { lastFiredAt: string; lastPayloadHash: string | null }>> {
|
||||
const rows = await db.select().from(alertEngineState)
|
||||
const m = new Map<string, { lastFiredAt: string; lastPayloadHash: string | null }>()
|
||||
for (const r of rows) {
|
||||
m.set(r.scopeKey, { lastFiredAt: r.lastFiredAt, lastPayloadHash: r.lastPayloadHash ?? null })
|
||||
@@ -61,15 +61,14 @@ function loadEngineStateMap(): Map<string, { lastFiredAt: string; lastPayloadHas
|
||||
return m
|
||||
}
|
||||
|
||||
function upsertEngineState(scopeKey: string, lastFiredAt: string, lastPayloadHash: string | null) {
|
||||
const existing = db.select().from(alertEngineState).where(eq(alertEngineState.scopeKey, scopeKey)).limit(1).all()[0]
|
||||
async function upsertEngineState(scopeKey: string, lastFiredAt: string, lastPayloadHash: string | null) {
|
||||
const existing = (await db.select().from(alertEngineState).where(eq(alertEngineState.scopeKey, scopeKey)).limit(1))[0]
|
||||
if (existing) {
|
||||
db.update(alertEngineState)
|
||||
await db.update(alertEngineState)
|
||||
.set({ lastFiredAt, lastPayloadHash })
|
||||
.where(eq(alertEngineState.scopeKey, scopeKey))
|
||||
.run()
|
||||
} else {
|
||||
db.insert(alertEngineState).values({ scopeKey, lastFiredAt, lastPayloadHash }).run()
|
||||
await db.insert(alertEngineState).values({ scopeKey, lastFiredAt, lastPayloadHash })
|
||||
}
|
||||
}
|
||||
|
||||
@@ -90,16 +89,16 @@ function ruleScopeKey(ruleId: string, transition?: RuleEvalHit["transition"]): s
|
||||
export async function runAlertEngineOnce(): Promise<AlertEngineRunResult> {
|
||||
const errors: string[] = []
|
||||
const sampledAt = new Date().toISOString()
|
||||
const latestSourceFinishedAt = getLatestSourceFinishedAt()
|
||||
const prevSourceFinishedAt = getSourceWatermark()
|
||||
const latestSourceFinishedAt = await getLatestSourceFinishedAt()
|
||||
const prevSourceFinishedAt = await getSourceWatermark()
|
||||
const hasNewSources =
|
||||
latestSourceFinishedAt != null &&
|
||||
(prevSourceFinishedAt == null || latestSourceFinishedAt > prevSourceFinishedAt)
|
||||
|
||||
const rules = listAlertRules()
|
||||
const groups = listAlertGroups()
|
||||
const state = loadEngineStateMap()
|
||||
const pub = getTelegramPublic()
|
||||
const rules = await listAlertRules()
|
||||
const groups = await listAlertGroups()
|
||||
const state = await loadEngineStateMap()
|
||||
const pub = await getTelegramPublic()
|
||||
const canSend = pub.tokenConfigured && Boolean(pub.chatId?.trim())
|
||||
let standaloneFires = 0
|
||||
let groupFires = 0
|
||||
@@ -107,10 +106,10 @@ export async function runAlertEngineOnce(): Promise<AlertEngineRunResult> {
|
||||
let ruleDiag: AlertEngineRunResult["ruleDiag"] = []
|
||||
if (hasNewSources) {
|
||||
await awaitSnapshotSourcesIdle()
|
||||
const snap = ingestAlertSignals()
|
||||
const snap = await ingestAlertSignals()
|
||||
const matched = matchRules(rules, snap)
|
||||
errors.push(...matched.errors)
|
||||
const { standalone, grouped, ruleDiag: diag } = computeDecisions({
|
||||
const { standalone, grouped, ruleDiag: diag } = await computeDecisions({
|
||||
rules,
|
||||
groups,
|
||||
hitByRule: matched.hitByRule,
|
||||
@@ -123,7 +122,7 @@ export async function runAlertEngineOnce(): Promise<AlertEngineRunResult> {
|
||||
if (!canSend) break
|
||||
const firedAt = new Date().toISOString()
|
||||
const key = ruleScopeKey(d.rule.id, d.hit.transition)
|
||||
const queued = enqueueTelegramOutbox({
|
||||
const queued = await enqueueTelegramOutbox({
|
||||
id: newOutboxId(),
|
||||
dedupeKey: `${key}:${d.hit.payloadHash}`,
|
||||
payload: {
|
||||
@@ -142,8 +141,8 @@ export async function runAlertEngineOnce(): Promise<AlertEngineRunResult> {
|
||||
})
|
||||
if (!queued) continue
|
||||
standaloneFires += 1
|
||||
clearConfirmPendingAfterSend([d.rule.id])
|
||||
upsertEngineState(key, firedAt, d.hit.payloadHash)
|
||||
await clearConfirmPendingAfterSend([d.rule.id])
|
||||
await upsertEngineState(key, firedAt, d.hit.payloadHash)
|
||||
state.set(key, { lastFiredAt: firedAt, lastPayloadHash: d.hit.payloadHash })
|
||||
}
|
||||
|
||||
@@ -155,7 +154,7 @@ export async function runAlertEngineOnce(): Promise<AlertEngineRunResult> {
|
||||
const body = `Группа «${d.group.name}» (${d.group.combineMode === "any" ? "ANY" : "ALL"})\n\n${lines.join("\n")}`
|
||||
const hash = d.hits.map((h) => h.payloadHash).sort().join("|")
|
||||
const key = `group:${d.group.id}`
|
||||
const queued = enqueueTelegramOutbox({
|
||||
const queued = await enqueueTelegramOutbox({
|
||||
id: newOutboxId(),
|
||||
dedupeKey: `${key}:${hash}`,
|
||||
payload: {
|
||||
@@ -173,18 +172,18 @@ export async function runAlertEngineOnce(): Promise<AlertEngineRunResult> {
|
||||
})
|
||||
if (!queued) continue
|
||||
groupFires += 1
|
||||
upsertEngineState(key, firedAt, hash)
|
||||
await upsertEngineState(key, firedAt, hash)
|
||||
state.set(key, { lastFiredAt: firedAt, lastPayloadHash: hash })
|
||||
clearConfirmPendingAfterSend(d.members.map((r) => r.id))
|
||||
await clearConfirmPendingAfterSend(d.members.map((r) => r.id))
|
||||
}
|
||||
updateSourceWatermark(latestSourceFinishedAt)
|
||||
await updateSourceWatermark(latestSourceFinishedAt)
|
||||
}
|
||||
|
||||
const outbox = await dispatchPendingOutbox()
|
||||
errors.push(...outbox.errors)
|
||||
|
||||
if (standaloneFires > 0 || groupFires > 0) {
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "alerts.engine.fired",
|
||||
sourceModule: "alerts",
|
||||
@@ -197,7 +196,7 @@ export async function runAlertEngineOnce(): Promise<AlertEngineRunResult> {
|
||||
})
|
||||
}
|
||||
if (errors.length > 0) {
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "warning",
|
||||
eventType: "alerts.engine.errors",
|
||||
sourceModule: "alerts",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { buildSignalSnapshotFromCollectors } from "./signals-from-collectors.js"
|
||||
|
||||
/** Единая точка входа для подготовки сигналов к rule matching. */
|
||||
export function ingestAlertSignals() {
|
||||
return buildSignalSnapshotFromCollectors()
|
||||
export async function ingestAlertSignals() {
|
||||
return await buildSignalSnapshotFromCollectors()
|
||||
}
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { sqliteDatabase } from "../../db/index.js"
|
||||
import { dbAll } from "../../db/index.js"
|
||||
import type {
|
||||
GreBgpSnapshotRunSnapshot,
|
||||
PingRunSnapshot,
|
||||
@@ -20,25 +20,23 @@ function normalizeNameKey(v: string): string {
|
||||
return v.trim().toLowerCase()
|
||||
}
|
||||
|
||||
function safeParseRunSnapshot(raw: string | null): SchedulerRunSnapshot | null {
|
||||
if (!raw) return null
|
||||
try {
|
||||
return JSON.parse(raw) as SchedulerRunSnapshot
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
function safeParseRunSnapshot(raw: unknown): SchedulerRunSnapshot | null {
|
||||
const obj = typeof raw === "string" ? (() => {
|
||||
try { return JSON.parse(raw) as unknown } catch { return null }
|
||||
})() : raw
|
||||
if (!obj || typeof obj !== "object") return null
|
||||
return obj as SchedulerRunSnapshot
|
||||
}
|
||||
|
||||
function readLatestOkRunSnapshots(jobKey: string, limit = 32): SchedulerRunSnapshot[] {
|
||||
const rows = sqliteDatabase
|
||||
.prepare(
|
||||
`SELECT result_json AS resultJson
|
||||
FROM scheduler_runs
|
||||
WHERE job_key = ? AND status = 'ok' AND result_json IS NOT NULL
|
||||
ORDER BY finished_at DESC
|
||||
LIMIT ?`,
|
||||
)
|
||||
.all(jobKey, limit) as { resultJson: string | null }[]
|
||||
async function readLatestOkRunSnapshots(jobKey: string, limit = 32): Promise<SchedulerRunSnapshot[]> {
|
||||
const rows = await dbAll<{ resultJson: unknown }>(
|
||||
`SELECT result_json AS "resultJson"
|
||||
FROM scheduler_runs
|
||||
WHERE job_key = ? AND status = 'ok' AND result_json IS NOT NULL
|
||||
ORDER BY finished_at DESC
|
||||
LIMIT ?`,
|
||||
[jobKey, limit],
|
||||
)
|
||||
const out: SchedulerRunSnapshot[] = []
|
||||
for (const row of rows) {
|
||||
const parsed = safeParseRunSnapshot(row.resultJson)
|
||||
@@ -227,13 +225,13 @@ function collectBgpSignalsFromRuns(runs: GreBgpSnapshotRunSnapshot[]): BgpPeerSi
|
||||
}
|
||||
|
||||
/** Собирает сигналы alert_engine напрямую из результатов collector jobs (`scheduler_runs.result_json`). */
|
||||
export function buildSignalSnapshotFromCollectors(): SignalSnapshot {
|
||||
export async function buildSignalSnapshotFromCollectors(): Promise<SignalSnapshot> {
|
||||
const sampledAt = new Date().toISOString()
|
||||
const resourceRuns = readLatestOkRunSnapshots("uptime_resources", 16) as ResourcesRunSnapshot[]
|
||||
const restRuns = readLatestOkRunSnapshots("servers_rest_ping", 16) as ServersRestPingRunSnapshot[]
|
||||
const pingRuns = readLatestOkRunSnapshots("uptime_ping", 24) as PingRunSnapshot[]
|
||||
const trafficRuns = readLatestOkRunSnapshots("traffic", 8) as TrafficRunSnapshot[]
|
||||
const greBgpRuns = readLatestOkRunSnapshots("gre_bgp", 8) as GreBgpSnapshotRunSnapshot[]
|
||||
const resourceRuns = await readLatestOkRunSnapshots("uptime_resources", 16) as ResourcesRunSnapshot[]
|
||||
const restRuns = await readLatestOkRunSnapshots("servers_rest_ping", 16) as ServersRestPingRunSnapshot[]
|
||||
const pingRuns = await readLatestOkRunSnapshots("uptime_ping", 24) as PingRunSnapshot[]
|
||||
const trafficRuns = await readLatestOkRunSnapshots("traffic", 8) as TrafficRunSnapshot[]
|
||||
const greBgpRuns = await readLatestOkRunSnapshots("gre_bgp", 8) as GreBgpSnapshotRunSnapshot[]
|
||||
|
||||
const resourceByKey = collectServerSignalsFromResourceRuns(resourceRuns)
|
||||
const restByKey = collectServerSignalsFromRestRuns(restRuns)
|
||||
|
||||
@@ -2,6 +2,6 @@ import { buildSignalSnapshotFromCollectors } from "./signals-from-collectors.js"
|
||||
import type { SignalSnapshot } from "./types.js"
|
||||
|
||||
/** Legacy-совместимость: единый источник сигналов — collector snapshots из scheduler_runs. */
|
||||
export function buildSignalSnapshot(): SignalSnapshot {
|
||||
return buildSignalSnapshotFromCollectors()
|
||||
export async function buildSignalSnapshot(): Promise<SignalSnapshot> {
|
||||
return await buildSignalSnapshotFromCollectors()
|
||||
}
|
||||
|
||||
@@ -11,44 +11,40 @@ const SOURCE_JOBS = [
|
||||
"gre_bgp",
|
||||
] as const
|
||||
|
||||
export function getLatestSourceFinishedAt(): string | null {
|
||||
const row = db
|
||||
export async function getLatestSourceFinishedAt(): Promise<string | null> {
|
||||
const row = (await db
|
||||
.select({ finishedAt: schedulerRuns.finishedAt })
|
||||
.from(schedulerRuns)
|
||||
.where(inArray(schedulerRuns.jobKey, [...SOURCE_JOBS]))
|
||||
.orderBy(desc(schedulerRuns.finishedAt))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
return row?.finishedAt ?? null
|
||||
}
|
||||
|
||||
export function getSourceWatermark(): string | null {
|
||||
const row = db.select().from(alertEngineCursor).where(eq(alertEngineCursor.id, 1)).limit(1).all()[0]
|
||||
export async function getSourceWatermark(): Promise<string | null> {
|
||||
const row = (await db.select().from(alertEngineCursor).where(eq(alertEngineCursor.id, 1)).limit(1))[0]
|
||||
return row?.lastSourceFinishedAt ?? null
|
||||
}
|
||||
|
||||
export function updateSourceWatermark(lastSourceFinishedAt: string | null): void {
|
||||
const existing = db
|
||||
export async function updateSourceWatermark(lastSourceFinishedAt: string | null): Promise<void> {
|
||||
const existing = (await db
|
||||
.select()
|
||||
.from(alertEngineCursor)
|
||||
.where(eq(alertEngineCursor.id, 1))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
if (existing) {
|
||||
db.update(alertEngineCursor)
|
||||
await db.update(alertEngineCursor)
|
||||
.set({
|
||||
lastSourceFinishedAt,
|
||||
updatedAt: new Date().toISOString(),
|
||||
})
|
||||
.where(eq(alertEngineCursor.id, 1))
|
||||
.run()
|
||||
return
|
||||
}
|
||||
db.insert(alertEngineCursor)
|
||||
await db.insert(alertEngineCursor)
|
||||
.values({
|
||||
id: 1,
|
||||
lastSourceFinishedAt,
|
||||
updatedAt: new Date().toISOString(),
|
||||
})
|
||||
.run()
|
||||
}
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { asc, desc, eq, sql } from "drizzle-orm"
|
||||
import { db, sqliteDatabase } from "../db/index.js"
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import {
|
||||
alertGroups,
|
||||
alertHistory,
|
||||
@@ -55,24 +55,22 @@ export type ApiAlertHistoryEntry = {
|
||||
source: "db" | "live"
|
||||
}
|
||||
|
||||
function ensureTelegramRow() {
|
||||
let row = db.select().from(alertTelegramSettings).where(eq(alertTelegramSettings.id, 1)).limit(1).all()[0]
|
||||
async function ensureTelegramRow() {
|
||||
let row = (await db.select().from(alertTelegramSettings).where(eq(alertTelegramSettings.id, 1)).limit(1))[0]
|
||||
if (!row) {
|
||||
db.insert(alertTelegramSettings).values({ id: 1 }).run()
|
||||
row = db.select().from(alertTelegramSettings).where(eq(alertTelegramSettings.id, 1)).limit(1).all()[0]
|
||||
await db.insert(alertTelegramSettings).values({ id: 1 })
|
||||
row = (await db.select().from(alertTelegramSettings).where(eq(alertTelegramSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
return row!
|
||||
}
|
||||
|
||||
function lastFiredByRuleId(): Map<string, string> {
|
||||
const rows = sqliteDatabase
|
||||
.prepare(
|
||||
`SELECT rule_id AS ruleId, MAX(fired_at) AS mx
|
||||
FROM alert_history
|
||||
WHERE rule_id IS NOT NULL AND rule_id != ''
|
||||
GROUP BY rule_id`,
|
||||
)
|
||||
.all() as { ruleId: string; mx: string }[]
|
||||
async function lastFiredByRuleId(): Promise<Map<string, string>> {
|
||||
const rows = await dbAll<{ ruleId: string; mx: string }>(
|
||||
`SELECT rule_id AS "ruleId", MAX(fired_at) AS mx
|
||||
FROM alert_history
|
||||
WHERE rule_id IS NOT NULL AND rule_id != ''
|
||||
GROUP BY rule_id`,
|
||||
)
|
||||
const m = new Map<string, string>()
|
||||
for (const r of rows) {
|
||||
if (r.ruleId && r.mx) m.set(r.ruleId, r.mx)
|
||||
@@ -80,12 +78,11 @@ function lastFiredByRuleId(): Map<string, string> {
|
||||
return m
|
||||
}
|
||||
|
||||
function targetsByRuleId(): Map<string, string[]> {
|
||||
const rows = db
|
||||
async function targetsByRuleId(): Promise<Map<string, string[]>> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(alertRuleTargets)
|
||||
.orderBy(asc(alertRuleTargets.sortIndex), asc(alertRuleTargets.id))
|
||||
.all()
|
||||
const m = new Map<string, string[]>()
|
||||
for (const t of rows) {
|
||||
const rid = t.ruleId
|
||||
@@ -96,12 +93,11 @@ function targetsByRuleId(): Map<string, string[]> {
|
||||
return m
|
||||
}
|
||||
|
||||
function conditionsByRuleId(): Map<string, string[]> {
|
||||
const rows = db
|
||||
async function conditionsByRuleId(): Promise<Map<string, string[]>> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(alertRuleConditions)
|
||||
.orderBy(asc(alertRuleConditions.sortIndex), asc(alertRuleConditions.id))
|
||||
.all()
|
||||
const m = new Map<string, string[]>()
|
||||
for (const t of rows) {
|
||||
const rid = t.ruleId
|
||||
@@ -112,26 +108,25 @@ function conditionsByRuleId(): Map<string, string[]> {
|
||||
return m
|
||||
}
|
||||
|
||||
export function listAlertGroups(): ApiAlertGroup[] {
|
||||
return db
|
||||
export async function listAlertGroups(): Promise<ApiAlertGroup[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(alertGroups)
|
||||
.orderBy(asc(alertGroups.name))
|
||||
.all()
|
||||
.map((g) => ({
|
||||
id: g.id,
|
||||
name: g.name,
|
||||
combineMode: g.combineMode,
|
||||
enabled: g.enabled,
|
||||
cooldownOverride: g.cooldownOverride ?? null,
|
||||
}))
|
||||
return rows.map((g) => ({
|
||||
id: g.id,
|
||||
name: g.name,
|
||||
combineMode: g.combineMode,
|
||||
enabled: g.enabled,
|
||||
cooldownOverride: g.cooldownOverride ?? null,
|
||||
}))
|
||||
}
|
||||
|
||||
export function listAlertRules(): ApiAlertRule[] {
|
||||
const lf = lastFiredByRuleId()
|
||||
const tgtMap = targetsByRuleId()
|
||||
const condMap = conditionsByRuleId()
|
||||
const rows = db.select().from(alertRules).all()
|
||||
export async function listAlertRules(): Promise<ApiAlertRule[]> {
|
||||
const lf = await lastFiredByRuleId()
|
||||
const tgtMap = await targetsByRuleId()
|
||||
const condMap = await conditionsByRuleId()
|
||||
const rows = await db.select().from(alertRules)
|
||||
return rows.map((r) => {
|
||||
const targets = tgtMap.get(r.id)
|
||||
const ts = targets?.length ? targets : [r.target]
|
||||
@@ -207,23 +202,23 @@ export type ReplaceAlertGroupInput = {
|
||||
}
|
||||
|
||||
/** Атомарно заменяет группы, правила и строки targets. */
|
||||
export function replaceAlertsConfig(payload: { groups: ReplaceAlertGroupInput[]; rules: ReplaceAlertRuleInput[] }) {
|
||||
const now = sql`(datetime('now'))`
|
||||
export async function replaceAlertsConfig(payload: { groups: ReplaceAlertGroupInput[]; rules: ReplaceAlertRuleInput[] }) {
|
||||
const now = sql`now()`
|
||||
const { groups, rules } = payload
|
||||
db.transaction((tx) => {
|
||||
tx.delete(alertRuleConditions).run()
|
||||
tx.delete(alertRuleTargets).run()
|
||||
tx.delete(alertRules).run()
|
||||
tx.delete(alertGroups).run()
|
||||
await db.transaction(async (tx) => {
|
||||
await tx.delete(alertRuleConditions)
|
||||
await tx.delete(alertRuleTargets)
|
||||
await tx.delete(alertRules)
|
||||
await tx.delete(alertGroups)
|
||||
for (const g of groups) {
|
||||
tx.insert(alertGroups).values({
|
||||
await tx.insert(alertGroups).values({
|
||||
id: g.id,
|
||||
name: g.name.trim() || g.id,
|
||||
combineMode: g.combineMode,
|
||||
enabled: g.enabled,
|
||||
cooldownOverride: g.cooldownOverride ?? null,
|
||||
updatedAt: now,
|
||||
}).run()
|
||||
})
|
||||
}
|
||||
for (const r of rules) {
|
||||
const rawTargets =
|
||||
@@ -250,7 +245,7 @@ export function replaceAlertsConfig(payload: { groups: ReplaceAlertGroupInput[];
|
||||
r.recoveryStabilitySec > 0
|
||||
? Math.min(86400, Math.max(1, Math.floor(r.recoveryStabilitySec)))
|
||||
: null
|
||||
tx.insert(alertRules).values({
|
||||
await tx.insert(alertRules).values({
|
||||
id: r.id,
|
||||
name: r.name,
|
||||
type: r.type,
|
||||
@@ -265,39 +260,39 @@ export function replaceAlertsConfig(payload: { groups: ReplaceAlertGroupInput[];
|
||||
chatId: r.chatId ?? "",
|
||||
groupId: r.groupId && r.groupId.trim() ? r.groupId.trim() : null,
|
||||
updatedAt: now,
|
||||
}).run()
|
||||
rawTargets.forEach((t, i) => {
|
||||
tx.insert(alertRuleTargets).values({
|
||||
})
|
||||
for (const [i, t] of rawTargets.entries()) {
|
||||
await tx.insert(alertRuleTargets).values({
|
||||
id: `rt-${r.id}-${i}`,
|
||||
ruleId: r.id,
|
||||
target: t,
|
||||
sortIndex: i,
|
||||
}).run()
|
||||
})
|
||||
rawConds.forEach((c, i) => {
|
||||
tx.insert(alertRuleConditions).values({
|
||||
})
|
||||
}
|
||||
for (const [i, c] of rawConds.entries()) {
|
||||
await tx.insert(alertRuleConditions).values({
|
||||
id: `rc-${r.id}-${i}`,
|
||||
ruleId: r.id,
|
||||
conditionLine: c,
|
||||
sortIndex: i,
|
||||
}).run()
|
||||
})
|
||||
})
|
||||
}
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
/** Совместимость: только правила, группы не трогаем. */
|
||||
export function replaceAlertRules(rules: ReplaceAlertRuleInput[]) {
|
||||
const curGroups = listAlertGroups()
|
||||
replaceAlertsConfig({ groups: curGroups.map((g) => ({ ...g })), rules })
|
||||
export async function replaceAlertRules(rules: ReplaceAlertRuleInput[]) {
|
||||
const curGroups = await listAlertGroups()
|
||||
await replaceAlertsConfig({ groups: curGroups.map((g) => ({ ...g })), rules })
|
||||
}
|
||||
|
||||
export function getTelegramSettingsRow() {
|
||||
return ensureTelegramRow()
|
||||
export async function getTelegramSettingsRow() {
|
||||
return await ensureTelegramRow()
|
||||
}
|
||||
|
||||
export function getTelegramPublic() {
|
||||
const row = ensureTelegramRow()
|
||||
export async function getTelegramPublic() {
|
||||
const row = await ensureTelegramRow()
|
||||
const tid = row.messageThreadId
|
||||
const messageThreadId =
|
||||
typeof tid === "number" && Number.isFinite(tid) && Math.floor(tid) >= 1 ? Math.floor(tid) : null
|
||||
@@ -308,13 +303,13 @@ export function getTelegramPublic() {
|
||||
}
|
||||
}
|
||||
|
||||
export function updateTelegramSettings(patch: {
|
||||
export async function updateTelegramSettings(patch: {
|
||||
token?: string | null
|
||||
chatId?: string | undefined
|
||||
/** undefined — не менять; null — сбросить тему */
|
||||
messageThreadId?: number | null
|
||||
}) {
|
||||
const cur = ensureTelegramRow()
|
||||
const cur = await ensureTelegramRow()
|
||||
let nextToken = cur.botToken
|
||||
let nextChat = cur.chatId ?? ""
|
||||
let nextThread: number | null | undefined = undefined
|
||||
@@ -335,27 +330,26 @@ export function updateTelegramSettings(patch: {
|
||||
const base = {
|
||||
botToken: nextToken,
|
||||
chatId: nextChat,
|
||||
updatedAt: sql`(datetime('now'))`,
|
||||
updatedAt: sql`now()`,
|
||||
}
|
||||
if (nextThread !== undefined) {
|
||||
db.update(alertTelegramSettings)
|
||||
await db.update(alertTelegramSettings)
|
||||
.set({ ...base, messageThreadId: nextThread })
|
||||
.where(eq(alertTelegramSettings.id, 1))
|
||||
.run()
|
||||
} else {
|
||||
db.update(alertTelegramSettings).set(base).where(eq(alertTelegramSettings.id, 1)).run()
|
||||
await db.update(alertTelegramSettings).set(base).where(eq(alertTelegramSettings.id, 1))
|
||||
}
|
||||
return getTelegramPublic()
|
||||
return await getTelegramPublic()
|
||||
}
|
||||
|
||||
/** Токен только для внутреннего вызова Telegram API (не отдаётся клиенту). */
|
||||
export function getTelegramBotToken(): string {
|
||||
return ensureTelegramRow().botToken?.trim() ?? ""
|
||||
export async function getTelegramBotToken(): Promise<string> {
|
||||
return (await ensureTelegramRow()).botToken?.trim() ?? ""
|
||||
}
|
||||
|
||||
/** Для `sendMessage`: только если в БД задана валидная тема. */
|
||||
export function getTelegramMessageThreadIdForApi(): number | undefined {
|
||||
const { messageThreadId } = getTelegramPublic()
|
||||
export async function getTelegramMessageThreadIdForApi(): Promise<number | undefined> {
|
||||
const { messageThreadId } = await getTelegramPublic()
|
||||
return messageThreadId ?? undefined
|
||||
}
|
||||
|
||||
@@ -390,9 +384,9 @@ export async function sendTelegramAlertMessage(opts: {
|
||||
/** Переопределение темы; undefined — как в настройках */
|
||||
messageThreadId?: number | null
|
||||
}): Promise<{ ok: true } | { ok: false; error: string }> {
|
||||
const token = (opts.token?.trim() || getTelegramBotToken()).trim()
|
||||
const token = (opts.token?.trim() || await getTelegramBotToken()).trim()
|
||||
if (!token) return { ok: false, error: "Не задан Bot Token" }
|
||||
const pub = getTelegramPublic()
|
||||
const pub = await getTelegramPublic()
|
||||
const chatId = (opts.chatId?.trim() || pub.chatId || "").trim()
|
||||
if (!chatId) return { ok: false, error: "Не задан Chat ID" }
|
||||
const threadId =
|
||||
@@ -400,7 +394,7 @@ export async function sendTelegramAlertMessage(opts: {
|
||||
? opts.messageThreadId >= 1
|
||||
? Math.floor(opts.messageThreadId)
|
||||
: undefined
|
||||
: getTelegramMessageThreadIdForApi()
|
||||
: await getTelegramMessageThreadIdForApi()
|
||||
const url = `https://api.telegram.org/bot${encodeURIComponent(token)}/sendMessage`
|
||||
try {
|
||||
const payload: { chat_id: string; text: string; message_thread_id?: number } = {
|
||||
@@ -423,7 +417,7 @@ export async function sendTelegramAlertMessage(opts: {
|
||||
}
|
||||
}
|
||||
|
||||
export function appendAlertHistory(entry: {
|
||||
export async function appendAlertHistory(entry: {
|
||||
id: string
|
||||
ruleId?: string | null
|
||||
groupId?: string | null
|
||||
@@ -434,7 +428,7 @@ export function appendAlertHistory(entry: {
|
||||
firedAt?: string
|
||||
}) {
|
||||
const firedAt = entry.firedAt ?? new Date().toISOString()
|
||||
db.insert(alertHistory)
|
||||
await db.insert(alertHistory)
|
||||
.values({
|
||||
id: entry.id,
|
||||
ruleId: entry.ruleId ?? null,
|
||||
@@ -445,7 +439,6 @@ export function appendAlertHistory(entry: {
|
||||
sentOk: entry.sentOk,
|
||||
firedAt,
|
||||
})
|
||||
.run()
|
||||
}
|
||||
|
||||
interface LiveResRow {
|
||||
@@ -462,38 +455,38 @@ interface LiveProbeRow {
|
||||
lossPct: number | null
|
||||
}
|
||||
|
||||
function loadLiveResourceIssues(): LiveResRow[] {
|
||||
return sqliteDatabase
|
||||
.prepare(
|
||||
`SELECT s.name AS serverName, urs.sampled_at AS sampledAt
|
||||
FROM uptime_resource_samples urs
|
||||
INNER JOIN (
|
||||
SELECT server_id, MAX(id) AS mid FROM uptime_resource_samples GROUP BY server_id
|
||||
) latest ON latest.mid = urs.id
|
||||
INNER JOIN servers s ON s.id = urs.server_id
|
||||
WHERE urs.status = 'offline'`,
|
||||
)
|
||||
.all() as LiveResRow[]
|
||||
async function loadLiveResourceIssues(): Promise<LiveResRow[]> {
|
||||
return dbAll<LiveResRow>(
|
||||
`SELECT s.name AS "serverName", urs.sampled_at AS "sampledAt"
|
||||
FROM uptime_resource_samples urs
|
||||
INNER JOIN (
|
||||
SELECT DISTINCT ON (server_id) server_id, sampled_at, status
|
||||
FROM uptime_resource_samples
|
||||
ORDER BY server_id, sampled_at DESC
|
||||
) latest ON latest.server_id = urs.server_id AND latest.sampled_at = urs.sampled_at
|
||||
INNER JOIN servers s ON s.id = urs.server_id
|
||||
WHERE urs.status = 'offline'`,
|
||||
)
|
||||
}
|
||||
|
||||
function loadLiveProbeIssues(): LiveProbeRow[] {
|
||||
return sqliteDatabase
|
||||
.prepare(
|
||||
`SELECT p.name AS probeName, p.target AS target, ups.sampled_at AS sampledAt,
|
||||
ups.status AS status, ups.rtt_ms AS rttMs, ups.loss_pct AS lossPct
|
||||
FROM uptime_probe_samples ups
|
||||
INNER JOIN (
|
||||
SELECT probe_id, MAX(id) AS mid FROM uptime_probe_samples GROUP BY probe_id
|
||||
) latest ON latest.mid = ups.id
|
||||
INNER JOIN uptime_probes p ON p.id = ups.probe_id
|
||||
WHERE ups.status IN ('down', 'warn')`,
|
||||
)
|
||||
.all() as LiveProbeRow[]
|
||||
async function loadLiveProbeIssues(): Promise<LiveProbeRow[]> {
|
||||
return dbAll<LiveProbeRow>(
|
||||
`SELECT p.name AS "probeName", p.target AS target, ups.sampled_at AS "sampledAt",
|
||||
ups.status AS status, ups.rtt_ms AS "rttMs", ups.loss_pct AS "lossPct"
|
||||
FROM uptime_probe_samples ups
|
||||
INNER JOIN (
|
||||
SELECT DISTINCT ON (probe_id) probe_id, sampled_at, status, rtt_ms, loss_pct
|
||||
FROM uptime_probe_samples
|
||||
ORDER BY probe_id, sampled_at DESC
|
||||
) latest ON latest.probe_id = ups.probe_id AND latest.sampled_at = ups.sampled_at
|
||||
INNER JOIN uptime_probes p ON p.id = ups.probe_id
|
||||
WHERE ups.status IN ('down', 'warn')`,
|
||||
)
|
||||
}
|
||||
|
||||
function toLiveHistoryEntries(): ApiAlertHistoryEntry[] {
|
||||
async function toLiveHistoryEntries(): Promise<ApiAlertHistoryEntry[]> {
|
||||
const out: ApiAlertHistoryEntry[] = []
|
||||
for (const r of loadLiveResourceIssues()) {
|
||||
for (const r of await loadLiveResourceIssues()) {
|
||||
const safeId = `live-res-${r.serverName}-${r.sampledAt}`.replace(/[^a-zA-Z0-9._-]+/g, "_")
|
||||
out.push({
|
||||
id: safeId,
|
||||
@@ -505,7 +498,7 @@ function toLiveHistoryEntries(): ApiAlertHistoryEntry[] {
|
||||
source: "live",
|
||||
})
|
||||
}
|
||||
for (const p of loadLiveProbeIssues()) {
|
||||
for (const p of await loadLiveProbeIssues()) {
|
||||
const sev = p.status === "down" ? "critical" : "warning"
|
||||
const loss = p.lossPct != null ? `${p.lossPct}%` : "—"
|
||||
const rtt = p.rttMs != null ? `${p.rttMs} мс` : "—"
|
||||
@@ -523,13 +516,12 @@ function toLiveHistoryEntries(): ApiAlertHistoryEntry[] {
|
||||
return out
|
||||
}
|
||||
|
||||
export function listPersistedHistory(limit: number): ApiAlertHistoryEntry[] {
|
||||
const rows = db
|
||||
export async function listPersistedHistory(limit: number): Promise<ApiAlertHistoryEntry[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(alertHistory)
|
||||
.orderBy(desc(alertHistory.firedAt))
|
||||
.limit(limit)
|
||||
.all()
|
||||
return rows.map((h) => ({
|
||||
id: h.id,
|
||||
ruleName: h.ruleName,
|
||||
@@ -541,9 +533,9 @@ export function listPersistedHistory(limit: number): ApiAlertHistoryEntry[] {
|
||||
}))
|
||||
}
|
||||
|
||||
export function listMergedHistory(persistLimit = 40, liveCap = 25): ApiAlertHistoryEntry[] {
|
||||
const persisted = listPersistedHistory(persistLimit)
|
||||
const live = toLiveHistoryEntries().slice(0, liveCap)
|
||||
export async function listMergedHistory(persistLimit = 40, liveCap = 25): Promise<ApiAlertHistoryEntry[]> {
|
||||
const persisted = await listPersistedHistory(persistLimit)
|
||||
const live = (await toLiveHistoryEntries()).slice(0, liveCap)
|
||||
const merged = [...persisted, ...live].sort((a, b) => (a.firedAt < b.firedAt ? 1 : a.firedAt > b.firedAt ? -1 : 0))
|
||||
const seen = new Set<string>()
|
||||
const dedup: ApiAlertHistoryEntry[] = []
|
||||
@@ -556,8 +548,8 @@ export function listMergedHistory(persistLimit = 40, liveCap = 25): ApiAlertHist
|
||||
return dedup
|
||||
}
|
||||
|
||||
export function getAlertsMeta() {
|
||||
const srvRows = db
|
||||
export async function getAlertsMeta() {
|
||||
const srvRows = await db
|
||||
.select({
|
||||
name: servers.name,
|
||||
host: servers.host,
|
||||
@@ -566,7 +558,6 @@ export function getAlertsMeta() {
|
||||
type: servers.type,
|
||||
})
|
||||
.from(servers)
|
||||
.all()
|
||||
const labels = srvRows.map((s) => (s.name?.trim() ? s.name.trim() : s.host))
|
||||
const serversDetail = srvRows.map((r) => ({
|
||||
name: r.name?.trim() || "",
|
||||
@@ -575,7 +566,7 @@ export function getAlertsMeta() {
|
||||
country: r.country?.trim() || "",
|
||||
type: (r.type ?? "home-router") as string,
|
||||
}))
|
||||
const probes = db.select({ name: uptimeProbes.name, target: uptimeProbes.target }).from(uptimeProbes).all()
|
||||
const probes = await db.select({ name: uptimeProbes.name, target: uptimeProbes.target }).from(uptimeProbes)
|
||||
const probeTargets = probes.map((p) => `${p.name} → ${p.target}`)
|
||||
const defaults = ["8.8.8.8", "1.1.1.1"]
|
||||
const rttLossTargets: string[] = []
|
||||
|
||||
@@ -18,7 +18,7 @@ export function getBackupSchedulerCollectorState(): { running: boolean } {
|
||||
|
||||
export async function collectScheduledBackupsOnce(): Promise<BackupsRunSnapshot> {
|
||||
const sampledAt = new Date().toISOString()
|
||||
const settings = getBackupScheduleSettings()
|
||||
const settings = await getBackupScheduleSettings()
|
||||
const snapshot: BackupsRunSnapshot = {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "backups",
|
||||
@@ -49,7 +49,7 @@ export async function collectScheduledBackupsOnce(): Promise<BackupsRunSnapshot>
|
||||
if (settings.format !== "rsc") {
|
||||
snapshot.skipped = true
|
||||
snapshot.errors = ["Формат backup пока не поддерживается, используйте rsc"]
|
||||
touchBackupScheduleRunMeta({
|
||||
await touchBackupScheduleRunMeta({
|
||||
lastRunAt: sampledAt,
|
||||
lastDurationMs: 0,
|
||||
lastError: snapshot.errors[0],
|
||||
@@ -59,9 +59,9 @@ export async function collectScheduledBackupsOnce(): Promise<BackupsRunSnapshot>
|
||||
|
||||
collecting = true
|
||||
const started = Date.now()
|
||||
const serverIds = resolveBackupServerIds(settings)
|
||||
const serverIds = await resolveBackupServerIds(settings)
|
||||
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "backups.job.started",
|
||||
sourceModule: "backups",
|
||||
@@ -88,13 +88,13 @@ export async function collectScheduledBackupsOnce(): Promise<BackupsRunSnapshot>
|
||||
snapshot.pruned += await pruneBackupsForServer(id, settings.keepCount)
|
||||
}
|
||||
|
||||
touchBackupScheduleRunMeta({
|
||||
await touchBackupScheduleRunMeta({
|
||||
lastRunAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
lastError: snapshot.errors?.length ? snapshot.errors.join("; ") : null,
|
||||
})
|
||||
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: snapshot.failures > 0 ? "warning" : "info",
|
||||
eventType: "backups.job.done",
|
||||
sourceModule: "backups",
|
||||
@@ -115,12 +115,12 @@ export async function collectScheduledBackupsOnce(): Promise<BackupsRunSnapshot>
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.fatalError = message
|
||||
snapshot.errors?.push(message)
|
||||
touchBackupScheduleRunMeta({
|
||||
await touchBackupScheduleRunMeta({
|
||||
lastRunAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
lastError: message,
|
||||
})
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "critical",
|
||||
eventType: "backups.job.failed",
|
||||
sourceModule: "backups",
|
||||
|
||||
@@ -4,6 +4,7 @@ import path from "node:path"
|
||||
import { desc, eq } from "drizzle-orm"
|
||||
import type { BackupScheduleSettingsDto } from "@mmapp/contracts/backups"
|
||||
import { db } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { backupEntries, backupScheduleSettings } from "../db/schema.js"
|
||||
import { getServerRowById } from "../modules/servers/repository/servers-repository.js"
|
||||
import { listServersRead } from "../modules/servers/service/servers-service.js"
|
||||
@@ -14,7 +15,7 @@ const BACKUPS_DIR = path.resolve(process.cwd(), "storage", "backups")
|
||||
|
||||
export type BackupMeta = {
|
||||
id: string
|
||||
serverId: string
|
||||
serverId: number | null
|
||||
serverName: string
|
||||
filename: string
|
||||
sizeBytes: number
|
||||
@@ -40,17 +41,18 @@ export async function ensureBackupStorage(): Promise<void> {
|
||||
await mkdir(BACKUPS_DIR, { recursive: true })
|
||||
}
|
||||
|
||||
export function listBackups(): BackupMeta[] {
|
||||
return db.select().from(backupEntries).orderBy(desc(backupEntries.createdAt)).all().map(rowToMeta)
|
||||
export async function listBackups(): Promise<BackupMeta[]> {
|
||||
const rows = await db.select().from(backupEntries).orderBy(desc(backupEntries.createdAt))
|
||||
return rows.map(rowToMeta)
|
||||
}
|
||||
|
||||
export function getBackupById(id: string): BackupMeta | null {
|
||||
const row = db.select().from(backupEntries).where(eq(backupEntries.id, id)).limit(1).all()[0]
|
||||
export async function getBackupById(id: string): Promise<BackupMeta | null> {
|
||||
const row = (await db.select().from(backupEntries).where(eq(backupEntries.id, id)).limit(1))[0]
|
||||
return row ? rowToMeta(row) : null
|
||||
}
|
||||
|
||||
export function insertBackup(meta: BackupMeta): void {
|
||||
db.insert(backupEntries).values({
|
||||
export async function insertBackup(meta: BackupMeta): Promise<void> {
|
||||
await db.insert(backupEntries).values({
|
||||
id: meta.id,
|
||||
serverId: meta.serverId,
|
||||
serverName: meta.serverName,
|
||||
@@ -59,13 +61,13 @@ export function insertBackup(meta: BackupMeta): void {
|
||||
kind: meta.kind,
|
||||
notes: meta.notes ?? null,
|
||||
createdAt: meta.createdAt,
|
||||
}).run()
|
||||
})
|
||||
}
|
||||
|
||||
export async function deleteBackupRecord(id: string): Promise<BackupMeta | null> {
|
||||
const hit = getBackupById(id)
|
||||
const hit = await getBackupById(id)
|
||||
if (!hit) return null
|
||||
db.delete(backupEntries).where(eq(backupEntries.id, id)).run()
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
return hit
|
||||
}
|
||||
@@ -75,18 +77,12 @@ function fmtTs(d = new Date()): string {
|
||||
return `${d.getFullYear()}-${p(d.getMonth() + 1)}-${p(d.getDate())}_${p(d.getHours())}-${p(d.getMinutes())}-${p(d.getSeconds())}`
|
||||
}
|
||||
|
||||
function parseServerIds(raw: string): string[] {
|
||||
try {
|
||||
const parsed = JSON.parse(raw) as unknown
|
||||
if (!Array.isArray(parsed)) return []
|
||||
return parsed.map(String).filter(Boolean)
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
function parseServerIds(raw: unknown): string[] {
|
||||
return parseJsonArray(raw).map(String).filter(Boolean)
|
||||
}
|
||||
|
||||
function getBackupScheduleSettingsRow() {
|
||||
return db.select().from(backupScheduleSettings).where(eq(backupScheduleSettings.id, SETTINGS_ID)).limit(1).all()[0]
|
||||
async function getBackupScheduleSettingsRow() {
|
||||
return (await db.select().from(backupScheduleSettings).where(eq(backupScheduleSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
?? {
|
||||
id: SETTINGS_ID,
|
||||
enabled: true,
|
||||
@@ -97,7 +93,7 @@ function getBackupScheduleSettingsRow() {
|
||||
monthDay: 1,
|
||||
keepCount: 7,
|
||||
format: "rsc" as const,
|
||||
serverIdsJson: "[]",
|
||||
serverIdsJson: [],
|
||||
lastRunAt: null,
|
||||
lastDurationMs: null,
|
||||
lastError: null,
|
||||
@@ -105,8 +101,8 @@ function getBackupScheduleSettingsRow() {
|
||||
}
|
||||
}
|
||||
|
||||
export function getBackupScheduleSettings(): BackupScheduleSettingsDto {
|
||||
const row = getBackupScheduleSettingsRow()
|
||||
export async function getBackupScheduleSettings(): Promise<BackupScheduleSettingsDto> {
|
||||
const row = await getBackupScheduleSettingsRow()
|
||||
return {
|
||||
enabled: row.enabled,
|
||||
frequency: row.frequency,
|
||||
@@ -124,7 +120,7 @@ export function getBackupScheduleSettings(): BackupScheduleSettingsDto {
|
||||
}
|
||||
}
|
||||
|
||||
export function updateBackupScheduleSettings(patch: Partial<{
|
||||
export async function updateBackupScheduleSettings(patch: Partial<{
|
||||
enabled: boolean
|
||||
frequency: "daily" | "weekly" | "monthly"
|
||||
hour: number
|
||||
@@ -135,7 +131,7 @@ export function updateBackupScheduleSettings(patch: Partial<{
|
||||
format: "rsc" | "backup"
|
||||
serverIds: string[]
|
||||
}>) {
|
||||
const prev = getBackupScheduleSettingsRow()
|
||||
const prev = await getBackupScheduleSettingsRow()
|
||||
const now = new Date().toISOString()
|
||||
const next = {
|
||||
enabled: patch.enabled ?? prev.enabled,
|
||||
@@ -146,33 +142,33 @@ export function updateBackupScheduleSettings(patch: Partial<{
|
||||
monthDay: patch.monthDay ?? prev.monthDay,
|
||||
keepCount: patch.keepCount ?? prev.keepCount,
|
||||
format: patch.format ?? prev.format,
|
||||
serverIdsJson: patch.serverIds ? JSON.stringify(patch.serverIds) : prev.serverIdsJson,
|
||||
serverIdsJson: patch.serverIds ?? prev.serverIdsJson,
|
||||
updatedAt: now,
|
||||
}
|
||||
if (db.select().from(backupScheduleSettings).where(eq(backupScheduleSettings.id, SETTINGS_ID)).limit(1).all()[0]) {
|
||||
db.update(backupScheduleSettings).set(next).where(eq(backupScheduleSettings.id, SETTINGS_ID)).run()
|
||||
if ((await db.select().from(backupScheduleSettings).where(eq(backupScheduleSettings.id, SETTINGS_ID)).limit(1))[0]) {
|
||||
await db.update(backupScheduleSettings).set(next).where(eq(backupScheduleSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
db.insert(backupScheduleSettings).values({ id: SETTINGS_ID, ...next }).run()
|
||||
await db.insert(backupScheduleSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return getBackupScheduleSettings()
|
||||
return await getBackupScheduleSettings()
|
||||
}
|
||||
|
||||
export function touchBackupScheduleRunMeta(patch: {
|
||||
export async function touchBackupScheduleRunMeta(patch: {
|
||||
lastRunAt?: string
|
||||
lastDurationMs?: number
|
||||
lastError?: string | null
|
||||
}) {
|
||||
const prev = getBackupScheduleSettingsRow()
|
||||
db.update(backupScheduleSettings).set({
|
||||
const prev = await getBackupScheduleSettingsRow()
|
||||
await db.update(backupScheduleSettings).set({
|
||||
lastRunAt: patch.lastRunAt ?? prev.lastRunAt,
|
||||
lastDurationMs: patch.lastDurationMs ?? prev.lastDurationMs,
|
||||
lastError: patch.lastError === undefined ? prev.lastError : patch.lastError,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}).where(eq(backupScheduleSettings.id, SETTINGS_ID)).run()
|
||||
}).where(eq(backupScheduleSettings.id, SETTINGS_ID))
|
||||
}
|
||||
|
||||
export function resolveBackupServerIds(settings: BackupScheduleSettingsDto): string[] {
|
||||
const enabled = new Set(listServersRead().map((s) => String(s.id)))
|
||||
export async function resolveBackupServerIds(settings: BackupScheduleSettingsDto): Promise<string[]> {
|
||||
const enabled = new Set((await listServersRead()).map((s) => String(s.id)))
|
||||
const requested = settings.serverIds.length > 0 ? settings.serverIds : [...enabled]
|
||||
return [...new Set(requested)].filter((id) => enabled.has(id))
|
||||
}
|
||||
@@ -227,7 +223,7 @@ export async function runBackupForServer(
|
||||
if (!Number.isFinite(serverIdNum)) {
|
||||
throw new Error("Невалидный id сервера")
|
||||
}
|
||||
const row = getServerRowById(serverIdNum)
|
||||
const row = await getServerRowById(serverIdNum)
|
||||
if (!row) {
|
||||
throw new Error("Сервер не найден")
|
||||
}
|
||||
@@ -242,7 +238,7 @@ export async function runBackupForServer(
|
||||
const st = await stat(filePath)
|
||||
const meta: BackupMeta = {
|
||||
id: randomUUID(),
|
||||
serverId: String(row.id),
|
||||
serverId: row.id,
|
||||
serverName: row.name,
|
||||
filename,
|
||||
sizeBytes: st.size,
|
||||
@@ -250,19 +246,18 @@ export async function runBackupForServer(
|
||||
kind,
|
||||
notes,
|
||||
}
|
||||
insertBackup(meta)
|
||||
await insertBackup(meta)
|
||||
return meta
|
||||
}
|
||||
|
||||
export async function pruneBackupsForServer(serverId: string, keepCount: number): Promise<number> {
|
||||
const rows = db.select().from(backupEntries)
|
||||
.where(eq(backupEntries.serverId, serverId))
|
||||
const rows = await db.select().from(backupEntries)
|
||||
.where(eq(backupEntries.serverId, Number(serverId)))
|
||||
.orderBy(desc(backupEntries.createdAt))
|
||||
.all()
|
||||
if (rows.length <= keepCount) return 0
|
||||
const toDelete = rows.slice(keepCount)
|
||||
for (const hit of toDelete) {
|
||||
db.delete(backupEntries).where(eq(backupEntries.id, hit.id)).run()
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, hit.id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
}
|
||||
return toDelete.length
|
||||
|
||||
@@ -14,7 +14,7 @@ export function bgpPeerAlertKey(s: BgpSessionRead): string {
|
||||
|
||||
/** Агрегат BGP-сессий с включённых серверов — как `/api/bgp/sessions`. */
|
||||
export async function fetchBgpSessionsForAlerts(): Promise<BgpSessionRead[]> {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const results = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
try {
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
import { issueCertificateWithCloudflareDns } from "./acme-cloudflare.js"
|
||||
import {
|
||||
@@ -11,15 +12,15 @@ const runningJobs = new Set<string>()
|
||||
export async function runCertificateIssueJob(jobId: string): Promise<void> {
|
||||
if (runningJobs.has(jobId)) return
|
||||
runningJobs.add(jobId)
|
||||
const row = getIssueJobRecord(jobId)
|
||||
const row = await getIssueJobRecord(jobId)
|
||||
if (!row) {
|
||||
runningJobs.delete(jobId)
|
||||
return
|
||||
}
|
||||
|
||||
const server = getServerRowByIdString(row.serverId)
|
||||
const server = await getServerRowByIdString(row.serverId)
|
||||
if (!server) {
|
||||
updateIssueJobRecord(jobId, {
|
||||
await updateIssueJobRecord(jobId, {
|
||||
status: "failed",
|
||||
step: "failed",
|
||||
finishedAt: new Date().toISOString(),
|
||||
@@ -29,20 +30,15 @@ export async function runCertificateIssueJob(jobId: string): Promise<void> {
|
||||
return
|
||||
}
|
||||
|
||||
let domainNames: string[] = []
|
||||
try {
|
||||
domainNames = JSON.parse(row.domainNames) as string[]
|
||||
} catch {
|
||||
domainNames = []
|
||||
}
|
||||
const domainNames = parseJsonArray(row.domainNames).map(String)
|
||||
|
||||
const trustStore = row.trustStore.split(",").map((s) => s.trim()).filter(Boolean)
|
||||
const startedAt = new Date().toISOString()
|
||||
const isScheduler = row.source === "scheduler"
|
||||
updateIssueJobRecord(jobId, { status: "running", step: "acme_order", startedAt, error: null })
|
||||
await updateIssueJobRecord(jobId, { status: "running", step: "acme_order", startedAt, error: null })
|
||||
|
||||
if (isScheduler) {
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "certificates.renew.started",
|
||||
sourceModule: "certificates",
|
||||
@@ -60,15 +56,15 @@ export async function runCertificateIssueJob(jobId: string): Promise<void> {
|
||||
domainNames,
|
||||
keyType: row.keyType === "ec256" ? "ec256" : "rsa2048",
|
||||
trustStore: trustStore.length > 0 ? trustStore : ["www", "api"],
|
||||
onStep: (step) => updateIssueJobRecord(jobId, { step }),
|
||||
onStep: (step) => { void updateIssueJobRecord(jobId, { step }) },
|
||||
})
|
||||
updateIssueJobRecord(jobId, {
|
||||
await updateIssueJobRecord(jobId, {
|
||||
status: "done",
|
||||
step: "done",
|
||||
finishedAt: new Date().toISOString(),
|
||||
error: null,
|
||||
})
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: isScheduler ? "certificates.renew.done" : "certificates.issue.done",
|
||||
sourceModule: "certificates",
|
||||
@@ -79,13 +75,13 @@ export async function runCertificateIssueJob(jobId: string): Promise<void> {
|
||||
})
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
updateIssueJobRecord(jobId, {
|
||||
await updateIssueJobRecord(jobId, {
|
||||
status: "failed",
|
||||
step: "failed",
|
||||
finishedAt: new Date().toISOString(),
|
||||
error: message,
|
||||
})
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "warning",
|
||||
eventType: isScheduler ? "certificates.renew.failed" : "certificates.issue.failed",
|
||||
sourceModule: "certificates",
|
||||
|
||||
@@ -22,7 +22,7 @@ export function getCertificateRenewCollectorState(): { running: boolean } {
|
||||
|
||||
export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewRunSnapshot> {
|
||||
const sampledAt = new Date().toISOString()
|
||||
const settings = getCertificateRenewSettings()
|
||||
const settings = await getCertificateRenewSettings()
|
||||
if (collecting) {
|
||||
return {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
@@ -55,7 +55,7 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
return snapshot
|
||||
}
|
||||
|
||||
const targets = listManagedCertificateTargets()
|
||||
const targets = await listManagedCertificateTargets()
|
||||
for (const target of targets) {
|
||||
snapshot.checked += 1
|
||||
const item = {
|
||||
@@ -66,12 +66,12 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
message: "",
|
||||
}
|
||||
|
||||
if (hasActiveIssueJob(target.serverId, target.certName)) {
|
||||
if (await hasActiveIssueJob(target.serverId, target.certName)) {
|
||||
item.action = "skipped"
|
||||
item.message = "Уже выполняется выпуск"
|
||||
snapshot.skippedTargets += 1
|
||||
snapshot.targets?.push(item)
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "certificates.renew.skipped",
|
||||
sourceModule: "certificates",
|
||||
@@ -83,7 +83,7 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
continue
|
||||
}
|
||||
|
||||
const server = getServerRowByIdString(target.serverId)
|
||||
const server = await getServerRowByIdString(target.serverId)
|
||||
if (!server) {
|
||||
item.action = "error"
|
||||
item.message = "Сервер не найден"
|
||||
@@ -114,7 +114,7 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
}
|
||||
|
||||
const jobId = randomUUID()
|
||||
createIssueJobRecord({
|
||||
await createIssueJobRecord({
|
||||
id: jobId,
|
||||
serverId: target.serverId,
|
||||
certName: target.certName,
|
||||
@@ -137,7 +137,7 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
}
|
||||
}
|
||||
|
||||
touchCertificateRenewRunMeta({
|
||||
await touchCertificateRenewRunMeta({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
lastError: snapshot.errors.length > 0 ? snapshot.errors.join("; ") : null,
|
||||
@@ -146,7 +146,7 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.fatalError = message
|
||||
touchCertificateRenewRunMeta({
|
||||
await touchCertificateRenewRunMeta({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
lastError: message,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { desc, eq, inArray } from "drizzle-orm"
|
||||
import type { CertificateDto, CertificateRenewSettingsDto } from "@mmapp/contracts/certificates"
|
||||
import { db } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import {
|
||||
acmeSettings,
|
||||
certificateIssueJobs,
|
||||
@@ -25,7 +26,7 @@ export async function listCertificatesFromServers(): Promise<{
|
||||
certificates: CertificateDto[]
|
||||
failures: Array<{ serverId: string; serverName?: string; error: string }>
|
||||
}> {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const certificates: CertificateDto[] = []
|
||||
const failures: Array<{ serverId: string; serverName?: string; error: string }> = []
|
||||
|
||||
@@ -53,8 +54,8 @@ export async function listCertificatesFromServers(): Promise<{
|
||||
return { certificates, failures }
|
||||
}
|
||||
|
||||
export function getAcmeSettingsPublic() {
|
||||
const row = db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1).all()[0]
|
||||
export async function getAcmeSettingsPublic() {
|
||||
const row = (await db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
if (!row) {
|
||||
return {
|
||||
directoryUrl: "https://acme-v02.api.letsencrypt.org/directory",
|
||||
@@ -71,30 +72,29 @@ export function getAcmeSettingsPublic() {
|
||||
}
|
||||
}
|
||||
|
||||
export function getAcmeCloudflareToken(): string {
|
||||
const row = db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1).all()[0]
|
||||
export async function getAcmeCloudflareToken(): Promise<string> {
|
||||
const row = (await db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
return row?.cloudflareApiToken?.trim() ?? ""
|
||||
}
|
||||
|
||||
export function getAcmeAccountPrivateKey(): string {
|
||||
const row = db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1).all()[0]
|
||||
export async function getAcmeAccountPrivateKey(): Promise<string> {
|
||||
const row = (await db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
return row?.accountPrivateKey?.trim() ?? ""
|
||||
}
|
||||
|
||||
export function saveAcmeAccountPrivateKey(pem: string) {
|
||||
export async function saveAcmeAccountPrivateKey(pem: string) {
|
||||
const now = new Date().toISOString()
|
||||
db.update(acmeSettings)
|
||||
await db.update(acmeSettings)
|
||||
.set({ accountPrivateKey: pem, updatedAt: now })
|
||||
.where(eq(acmeSettings.id, SETTINGS_ID))
|
||||
.run()
|
||||
}
|
||||
|
||||
export function updateAcmeSettings(input: {
|
||||
export async function updateAcmeSettings(input: {
|
||||
directoryUrl?: string
|
||||
defaultZoneId?: string | null
|
||||
cloudflareApiToken?: string | null
|
||||
}) {
|
||||
const row = db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1).all()[0]
|
||||
const row = (await db.select().from(acmeSettings).where(eq(acmeSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
const now = new Date().toISOString()
|
||||
const next = {
|
||||
directoryUrl: input.directoryUrl?.trim() || row?.directoryUrl || "https://acme-v02.api.letsencrypt.org/directory",
|
||||
@@ -109,14 +109,14 @@ export function updateAcmeSettings(input: {
|
||||
updatedAt: now,
|
||||
}
|
||||
if (row) {
|
||||
db.update(acmeSettings).set(next).where(eq(acmeSettings.id, SETTINGS_ID)).run()
|
||||
await db.update(acmeSettings).set(next).where(eq(acmeSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
db.insert(acmeSettings).values({ id: SETTINGS_ID, accountPrivateKey: "", ...next }).run()
|
||||
await db.insert(acmeSettings).values({ id: SETTINGS_ID, accountPrivateKey: "", ...next })
|
||||
}
|
||||
return getAcmeSettingsPublic()
|
||||
return await getAcmeSettingsPublic()
|
||||
}
|
||||
|
||||
export function createIssueJobRecord(input: {
|
||||
export async function createIssueJobRecord(input: {
|
||||
id: string
|
||||
serverId: string
|
||||
certName: string
|
||||
@@ -126,21 +126,21 @@ export function createIssueJobRecord(input: {
|
||||
source?: "manual" | "scheduler"
|
||||
}) {
|
||||
const now = new Date().toISOString()
|
||||
db.insert(certificateIssueJobs).values({
|
||||
await db.insert(certificateIssueJobs).values({
|
||||
id: input.id,
|
||||
status: "queued",
|
||||
step: "queued",
|
||||
source: input.source ?? "manual",
|
||||
serverId: input.serverId,
|
||||
serverId: Number(input.serverId),
|
||||
certName: input.certName,
|
||||
domainNames: JSON.stringify(input.domainNames),
|
||||
domainNames: input.domainNames,
|
||||
keyType: input.keyType,
|
||||
trustStore: input.trustStore,
|
||||
requestedAt: now,
|
||||
}).run()
|
||||
})
|
||||
}
|
||||
|
||||
export function updateIssueJobRecord(
|
||||
export async function updateIssueJobRecord(
|
||||
id: string,
|
||||
patch: Partial<{
|
||||
status: "queued" | "running" | "done" | "failed"
|
||||
@@ -150,21 +150,15 @@ export function updateIssueJobRecord(
|
||||
error: string | null
|
||||
}>,
|
||||
) {
|
||||
db.update(certificateIssueJobs).set(patch).where(eq(certificateIssueJobs.id, id)).run()
|
||||
await db.update(certificateIssueJobs).set(patch).where(eq(certificateIssueJobs.id, id))
|
||||
}
|
||||
|
||||
export function getIssueJobRecord(id: string) {
|
||||
return db.select().from(certificateIssueJobs).where(eq(certificateIssueJobs.id, id)).limit(1).all()[0] ?? null
|
||||
export async function getIssueJobRecord(id: string) {
|
||||
return (await db.select().from(certificateIssueJobs).where(eq(certificateIssueJobs.id, id)).limit(1))[0] ?? null
|
||||
}
|
||||
|
||||
export function toIssueJobDto(row: NonNullable<ReturnType<typeof getIssueJobRecord>>) {
|
||||
let domainNames: string[] = []
|
||||
try {
|
||||
const parsed = JSON.parse(row.domainNames) as unknown
|
||||
if (Array.isArray(parsed)) domainNames = parsed.map(String)
|
||||
} catch {
|
||||
domainNames = []
|
||||
}
|
||||
export function toIssueJobDto(row: NonNullable<Awaited<ReturnType<typeof getIssueJobRecord>>>) {
|
||||
const domainNames = parseJsonArray(row.domainNames).map(String)
|
||||
return {
|
||||
id: row.id,
|
||||
status: row.status,
|
||||
@@ -179,14 +173,14 @@ export function toIssueJobDto(row: NonNullable<ReturnType<typeof getIssueJobReco
|
||||
}
|
||||
}
|
||||
|
||||
export function getServerRowByIdString(serverId: string) {
|
||||
const id = Number.parseInt(serverId, 10)
|
||||
export async function getServerRowByIdString(serverId: string | number) {
|
||||
const id = typeof serverId === "number" ? serverId : Number.parseInt(serverId, 10)
|
||||
if (!Number.isFinite(id)) return null
|
||||
return db.select().from(servers).where(eq(servers.id, id)).limit(1).all()[0] ?? null
|
||||
return (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0] ?? null
|
||||
}
|
||||
|
||||
function getCertificateRenewSettingsRow() {
|
||||
return db.select().from(certificateRenewSettings).where(eq(certificateRenewSettings.id, SETTINGS_ID)).limit(1).all()[0]
|
||||
async function getCertificateRenewSettingsRow() {
|
||||
return (await db.select().from(certificateRenewSettings).where(eq(certificateRenewSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
?? {
|
||||
id: SETTINGS_ID,
|
||||
enabled: true,
|
||||
@@ -199,8 +193,8 @@ function getCertificateRenewSettingsRow() {
|
||||
}
|
||||
}
|
||||
|
||||
export function getCertificateRenewSettings(): CertificateRenewSettingsDto {
|
||||
const row = getCertificateRenewSettingsRow()
|
||||
export async function getCertificateRenewSettings(): Promise<CertificateRenewSettingsDto> {
|
||||
const row = await getCertificateRenewSettingsRow()
|
||||
return {
|
||||
enabled: row.enabled,
|
||||
intervalSec: row.intervalSec,
|
||||
@@ -212,12 +206,12 @@ export function getCertificateRenewSettings(): CertificateRenewSettingsDto {
|
||||
}
|
||||
}
|
||||
|
||||
export function updateCertificateRenewSettings(patch: {
|
||||
export async function updateCertificateRenewSettings(patch: {
|
||||
enabled?: boolean
|
||||
intervalSec?: number
|
||||
renewBeforeDays?: number
|
||||
}) {
|
||||
const prev = getCertificateRenewSettingsRow()
|
||||
const prev = await getCertificateRenewSettingsRow()
|
||||
const now = new Date().toISOString()
|
||||
const next = {
|
||||
enabled: patch.enabled ?? prev.enabled,
|
||||
@@ -225,47 +219,40 @@ export function updateCertificateRenewSettings(patch: {
|
||||
renewBeforeDays: patch.renewBeforeDays ?? prev.renewBeforeDays,
|
||||
updatedAt: now,
|
||||
}
|
||||
if (db.select().from(certificateRenewSettings).where(eq(certificateRenewSettings.id, SETTINGS_ID)).limit(1).all()[0]) {
|
||||
db.update(certificateRenewSettings).set(next).where(eq(certificateRenewSettings.id, SETTINGS_ID)).run()
|
||||
if ((await db.select().from(certificateRenewSettings).where(eq(certificateRenewSettings.id, SETTINGS_ID)).limit(1))[0]) {
|
||||
await db.update(certificateRenewSettings).set(next).where(eq(certificateRenewSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
db.insert(certificateRenewSettings).values({ id: SETTINGS_ID, ...next }).run()
|
||||
await db.insert(certificateRenewSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return getCertificateRenewSettings()
|
||||
return await getCertificateRenewSettings()
|
||||
}
|
||||
|
||||
export function touchCertificateRenewRunMeta(patch: {
|
||||
export async function touchCertificateRenewRunMeta(patch: {
|
||||
lastCollectedAt?: string
|
||||
lastDurationMs?: number
|
||||
lastError?: string | null
|
||||
}) {
|
||||
const prev = getCertificateRenewSettingsRow()
|
||||
db.update(certificateRenewSettings).set({
|
||||
const prev = await getCertificateRenewSettingsRow()
|
||||
await db.update(certificateRenewSettings).set({
|
||||
lastCollectedAt: patch.lastCollectedAt ?? prev.lastCollectedAt,
|
||||
lastDurationMs: patch.lastDurationMs ?? prev.lastDurationMs,
|
||||
lastError: patch.lastError === undefined ? prev.lastError : patch.lastError,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}).where(eq(certificateRenewSettings.id, SETTINGS_ID)).run()
|
||||
}).where(eq(certificateRenewSettings.id, SETTINGS_ID))
|
||||
}
|
||||
|
||||
export function listManagedCertificateTargets(): ManagedCertificateTarget[] {
|
||||
const rows = db.select().from(certificateIssueJobs)
|
||||
export async function listManagedCertificateTargets(): Promise<ManagedCertificateTarget[]> {
|
||||
const rows = await db.select().from(certificateIssueJobs)
|
||||
.where(eq(certificateIssueJobs.status, "done"))
|
||||
.orderBy(desc(certificateIssueJobs.finishedAt), desc(certificateIssueJobs.requestedAt))
|
||||
.all()
|
||||
|
||||
const byKey = new Map<string, ManagedCertificateTarget>()
|
||||
for (const row of rows) {
|
||||
const key = `${row.serverId}::${row.certName}`
|
||||
if (byKey.has(key)) continue
|
||||
let domainNames: string[] = []
|
||||
try {
|
||||
const parsed = JSON.parse(row.domainNames) as unknown
|
||||
if (Array.isArray(parsed)) domainNames = parsed.map(String)
|
||||
} catch {
|
||||
domainNames = []
|
||||
}
|
||||
const domainNames = parseJsonArray(row.domainNames).map(String)
|
||||
byKey.set(key, {
|
||||
serverId: row.serverId,
|
||||
serverId: row.serverId == null ? "" : String(row.serverId),
|
||||
certName: row.certName,
|
||||
domainNames,
|
||||
keyType: row.keyType,
|
||||
@@ -276,11 +263,10 @@ export function listManagedCertificateTargets(): ManagedCertificateTarget[] {
|
||||
return [...byKey.values()]
|
||||
}
|
||||
|
||||
export function hasActiveIssueJob(serverId: string, certName: string): boolean {
|
||||
const row = db.select().from(certificateIssueJobs)
|
||||
export async function hasActiveIssueJob(serverId: string, certName: string): Promise<boolean> {
|
||||
const rows = await db.select().from(certificateIssueJobs)
|
||||
.where(inArray(certificateIssueJobs.status, ["queued", "running"]))
|
||||
.orderBy(desc(certificateIssueJobs.requestedAt))
|
||||
.all()
|
||||
.find((r) => r.serverId === serverId && r.certName === certName)
|
||||
return Boolean(row)
|
||||
const sid = Number(serverId)
|
||||
return rows.some((r) => r.serverId === sid && r.certName === certName)
|
||||
}
|
||||
|
||||
@@ -179,7 +179,7 @@ export async function listFirewallAll(): Promise<{
|
||||
rules: FirewallRuleDto[]
|
||||
addressLists: FirewallAddressListDto[]
|
||||
}> {
|
||||
const allServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const perServer = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
try {
|
||||
|
||||
@@ -1,5 +1,3 @@
|
||||
import { db, sqliteDatabase } from "../db/index.js"
|
||||
import { alertBgpPeerSamples, alertGreTunnelSamples } from "../db/schema.js"
|
||||
import { bgpPeerAlertKey, fetchBgpSessionsForAlerts } from "./bgp-peers-live.js"
|
||||
import { fetchGreTunnelLiveRows } from "./gre-tunnels-live.js"
|
||||
import type { GreBgpSnapshotRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
@@ -12,8 +10,8 @@ export function getGreBgpSnapshotCollectorState(): { running: boolean } {
|
||||
}
|
||||
|
||||
/**
|
||||
* Один опрос GRE + BGP по включённым серверам и запись строк в SQLite для `buildSignalSnapshot`.
|
||||
* Движок оповещений больше не дублирует эти REST-запросы.
|
||||
* Один опрос GRE + BGP по включённым серверам.
|
||||
* Снимок для алертов живёт в `scheduler_runs.result_json` (`greTunnels` / `bgpPeers`).
|
||||
*/
|
||||
export async function collectGreBgpSnapshotOnce(): Promise<GreBgpSnapshotRunSnapshot> {
|
||||
const sampledAt = new Date().toISOString()
|
||||
@@ -43,8 +41,8 @@ export async function collectGreBgpSnapshotOnce(): Promise<GreBgpSnapshotRunSnap
|
||||
|
||||
try {
|
||||
const [greSettled, bgpSettled] = await Promise.allSettled([
|
||||
fetchGreTunnelLiveRows(),
|
||||
fetchBgpSessionsForAlerts(),
|
||||
await fetchGreTunnelLiveRows(),
|
||||
await fetchBgpSessionsForAlerts(),
|
||||
])
|
||||
|
||||
if (greSettled.status === "rejected") {
|
||||
@@ -62,32 +60,8 @@ export async function collectGreBgpSnapshotOnce(): Promise<GreBgpSnapshotRunSnap
|
||||
const bgpRows = bgpSettled.status === "fulfilled" ? bgpSettled.value : []
|
||||
snapshot.greTunnels = greRows.map((r) => ({ targetLabel: r.targetLabel, status: r.status }))
|
||||
snapshot.bgpPeers = bgpRows.map((s) => ({ key: bgpPeerAlertKey(s), state: s.state }))
|
||||
|
||||
db.transaction((tx) => {
|
||||
for (const r of greRows) {
|
||||
tx.insert(alertGreTunnelSamples).values({
|
||||
sampledAt,
|
||||
targetLabel: r.targetLabel,
|
||||
status: r.status,
|
||||
}).run()
|
||||
snapshot.greWritten += 1
|
||||
}
|
||||
for (const s of bgpRows) {
|
||||
tx.insert(alertBgpPeerSamples).values({
|
||||
sampledAt,
|
||||
peerKey: bgpPeerAlertKey(s),
|
||||
state: s.state,
|
||||
}).run()
|
||||
snapshot.bgpWritten += 1
|
||||
}
|
||||
})
|
||||
|
||||
sqliteDatabase
|
||||
.prepare(`DELETE FROM alert_gre_tunnel_samples WHERE sampled_at < datetime('now', '-30 days')`)
|
||||
.run()
|
||||
sqliteDatabase
|
||||
.prepare(`DELETE FROM alert_bgp_peer_samples WHERE sampled_at < datetime('now', '-30 days')`)
|
||||
.run()
|
||||
snapshot.greWritten = greRows.length
|
||||
snapshot.bgpWritten = bgpRows.length
|
||||
|
||||
if (errors.length) snapshot.errors = errors
|
||||
} catch (e) {
|
||||
|
||||
@@ -46,7 +46,7 @@ async function fetchGreForServer(server: ServerRow): Promise<GreTunnelLiveRow[]>
|
||||
|
||||
/** GRE с включённых серверов — тот же смысл, что `/api/filters/gre-tunnels` (без правил фильтров). */
|
||||
export async function fetchGreTunnelLiveRows(): Promise<GreTunnelLiveRow[]> {
|
||||
const enabledServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const results = await Promise.all(
|
||||
enabledServers.map(async (server) => {
|
||||
try {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { and, asc, eq, lt } from "drizzle-orm"
|
||||
import { and, asc, desc, eq, lt } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import {
|
||||
filterRules,
|
||||
@@ -30,29 +30,29 @@ function norm(v: string | null | undefined): string {
|
||||
return String(v ?? "").trim().toLowerCase()
|
||||
}
|
||||
|
||||
function getSettingsRow() {
|
||||
const row = db.select().from(internetPathSettings).where(eq(internetPathSettings.id, 1)).limit(1).all()[0]
|
||||
async function getSettingsRow() {
|
||||
const row = (await db.select().from(internetPathSettings).where(eq(internetPathSettings.id, 1)).limit(1))[0]
|
||||
if (row) return row
|
||||
const now = new Date().toISOString()
|
||||
db.insert(internetPathSettings).values({
|
||||
await db.insert(internetPathSettings).values({
|
||||
id: 1,
|
||||
enabled: true,
|
||||
intervalSec: 300,
|
||||
retentionDays: 14,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
}).run()
|
||||
return db.select().from(internetPathSettings).where(eq(internetPathSettings.id, 1)).limit(1).all()[0]
|
||||
})
|
||||
return (await db.select().from(internetPathSettings).where(eq(internetPathSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
function cleanupSnapshots(retentionDays: number) {
|
||||
async function cleanupSnapshots(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
db.delete(internetPathSnapshots).where(lt(internetPathSnapshots.sampledAt, cutoff)).run()
|
||||
await db.delete(internetPathSnapshots).where(lt(internetPathSnapshots.sampledAt, cutoff))
|
||||
}
|
||||
|
||||
function buildRulesets() {
|
||||
const enabled = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const rules = db.select().from(filterRules).orderBy(asc(filterRules.serverId), asc(filterRules.sortOrder)).all()
|
||||
async function buildRulesets() {
|
||||
const enabled = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const rules = await db.select().from(filterRules).orderBy(asc(filterRules.serverId), asc(filterRules.sortOrder))
|
||||
return enabled.map((s) => ({
|
||||
serverId: String(s.id),
|
||||
rules: rules
|
||||
@@ -70,7 +70,7 @@ function buildRulesets() {
|
||||
}
|
||||
|
||||
async function readRouteLookup(serverId: number): Promise<{ gateway: string | null; routingMark: string | null }> {
|
||||
const row = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const row = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!row) return { gateway: null, routingMark: null }
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const routes = await client.get<Array<Record<string, string>>>("/ip/route").catch(() => [])
|
||||
@@ -91,8 +91,8 @@ async function readRouteLookup(serverId: number): Promise<{ gateway: string | nu
|
||||
}
|
||||
|
||||
async function readWanRuntime(serverId: number) {
|
||||
const server = listServersRead().find((s) => Number(s.id) === serverId)
|
||||
const row = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const server = (await listServersRead()).find((s) => Number(s.id) === serverId)
|
||||
const row = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server || !row) return null
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const [dhcpRaw, ipAddrs, routes] = await Promise.all([
|
||||
@@ -144,8 +144,8 @@ async function readWanRuntime(serverId: number) {
|
||||
}
|
||||
}
|
||||
|
||||
function mapSpeedProbes() {
|
||||
const rows = db.select().from(uptimeSpeedProbes).orderBy(asc(uptimeSpeedProbes.sortOrder)).all()
|
||||
async function mapSpeedProbes() {
|
||||
const rows = await db.select().from(uptimeSpeedProbes).orderBy(asc(uptimeSpeedProbes.sortOrder))
|
||||
return rows.map((r) => ({
|
||||
id: r.id,
|
||||
srcServerId: String(r.srcServerId),
|
||||
@@ -175,7 +175,7 @@ function parseInnerIps(comment: string): { localInnerIp: string; remoteInnerIp:
|
||||
}
|
||||
|
||||
async function collectGreTunnels() {
|
||||
const enabled = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const enabled = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const all = await Promise.all(enabled.map(async (srv) => {
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(srv)
|
||||
@@ -214,23 +214,24 @@ async function collectGreTunnels() {
|
||||
return all.flat()
|
||||
}
|
||||
|
||||
export function getInternetPathSettings() {
|
||||
return getSettingsRow()
|
||||
export async function getInternetPathSettings() {
|
||||
return await getSettingsRow()
|
||||
}
|
||||
|
||||
export function updateInternetPathSettings(patch: { enabled?: boolean; intervalSec?: number; retentionDays?: number }) {
|
||||
const prev = getSettingsRow()
|
||||
db.update(internetPathSettings).set({
|
||||
export async function updateInternetPathSettings(patch: { enabled?: boolean; intervalSec?: number; retentionDays?: number }) {
|
||||
const prev = await getSettingsRow()
|
||||
await db.update(internetPathSettings).set({
|
||||
enabled: patch.enabled ?? prev.enabled,
|
||||
intervalSec: patch.intervalSec ?? prev.intervalSec,
|
||||
retentionDays: patch.retentionDays ?? prev.retentionDays,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}).where(eq(internetPathSettings.id, 1)).run()
|
||||
return getSettingsRow()
|
||||
}).where(eq(internetPathSettings.id, 1))
|
||||
return await getSettingsRow()
|
||||
}
|
||||
|
||||
export function getLatestInternetPathSnapshot() {
|
||||
return db.select().from(internetPathSnapshots).orderBy(asc(internetPathSnapshots.id)).all().at(-1) ?? null
|
||||
export async function getLatestInternetPathSnapshot() {
|
||||
const rows = await db.select().from(internetPathSnapshots).orderBy(desc(internetPathSnapshots.id)).limit(1)
|
||||
return rows[0] ?? null
|
||||
}
|
||||
|
||||
export async function collectInternetPathSnapshotOnce(): Promise<InternetPathRunSnapshot> {
|
||||
@@ -246,12 +247,12 @@ export async function collectInternetPathSnapshotOnce(): Promise<InternetPathRun
|
||||
}
|
||||
collecting = true
|
||||
const started = Date.now()
|
||||
const settings = getSettingsRow()
|
||||
const settings = await getSettingsRow()
|
||||
try {
|
||||
const serversRead = listServersRead()
|
||||
const serversRead = await listServersRead()
|
||||
const homes = serversRead.filter((s) => s.type === "home-router")
|
||||
const [greTunnels, rulesets] = await Promise.all([collectGreTunnels(), Promise.resolve(buildRulesets())])
|
||||
const speedProbes = mapSpeedProbes()
|
||||
const [greTunnels, rulesets] = await Promise.all([await collectGreTunnels(), Promise.resolve(await buildRulesets())])
|
||||
const speedProbes = await mapSpeedProbes()
|
||||
const routeLookupByServerId: Record<string, { gateway: string | null; routingMark: string | null }> = {}
|
||||
const wanRuntimeByHomeId: Record<string, unknown> = {}
|
||||
for (const h of homes) {
|
||||
@@ -268,17 +269,17 @@ export async function collectInternetPathSnapshotOnce(): Promise<InternetPathRun
|
||||
routeLookupByServerId,
|
||||
wanRuntimeByHomeId,
|
||||
}
|
||||
db.insert(internetPathSnapshots).values({
|
||||
await db.insert(internetPathSnapshots).values({
|
||||
sampledAt,
|
||||
payloadJson: JSON.stringify(payload),
|
||||
}).run()
|
||||
cleanupSnapshots(Math.max(1, settings.retentionDays))
|
||||
db.update(internetPathSettings).set({
|
||||
payloadJson: payload,
|
||||
})
|
||||
await cleanupSnapshots(Math.max(1, settings.retentionDays))
|
||||
await db.update(internetPathSettings).set({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
lastError: "",
|
||||
updatedAt: sampledAt,
|
||||
}).where(eq(internetPathSettings.id, 1)).run()
|
||||
}).where(eq(internetPathSettings.id, 1))
|
||||
return {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "internet_path",
|
||||
@@ -288,12 +289,12 @@ export async function collectInternetPathSnapshotOnce(): Promise<InternetPathRun
|
||||
}
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
db.update(internetPathSettings).set({
|
||||
await db.update(internetPathSettings).set({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
lastError: msg,
|
||||
updatedAt: sampledAt,
|
||||
}).where(eq(internetPathSettings.id, 1)).run()
|
||||
}).where(eq(internetPathSettings.id, 1))
|
||||
return {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "internet_path",
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { eq, lt } from "drizzle-orm"
|
||||
import { db, pool } from "../db/index.js"
|
||||
import { dropExpiredPartitions, ensurePartitionsAround } from "../db/partitions.js"
|
||||
import { servers, serverSnapshots } from "../db/schema.js"
|
||||
import type { SnapshotInsert } from "../db/schema.js"
|
||||
import type { SnapshotRead } from "../types/server.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import { parseRosCpuLoadPercent, parseRosDataSizeBytes } from "./ros-metric-parse.js"
|
||||
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
|
||||
|
||||
// ── pollServer ─────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -13,12 +15,11 @@ import { parseRosCpuLoadPercent, parseRosDataSizeBytes } from "./ros-metric-pars
|
||||
* and sync the server's display name from system/identity.
|
||||
*/
|
||||
export async function pollServer(serverId: number): Promise<SnapshotRead> {
|
||||
const server = db
|
||||
const server = (await db
|
||||
.select()
|
||||
.from(servers)
|
||||
.where(eq(servers.id, serverId))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
|
||||
if (!server) {
|
||||
throw new Error(`Server with id=${serverId} not found`)
|
||||
@@ -60,26 +61,30 @@ export async function pollServer(serverId: number): Promise<SnapshotRead> {
|
||||
cpuLoad,
|
||||
freeMemory: freeMem,
|
||||
totalMemory: totalMem,
|
||||
rawInterfaces: JSON.stringify(ifaces),
|
||||
rawIpAddresses: JSON.stringify(addresses),
|
||||
rawInterfaces: ifaces,
|
||||
rawIpAddresses: addresses,
|
||||
} satisfies Partial<SnapshotInsert>)
|
||||
|
||||
// Keep server.name in sync with RouterOS identity
|
||||
db.update(servers)
|
||||
.set({ name: identity.name, updatedAt: now })
|
||||
.where(eq(servers.id, serverId))
|
||||
.run()
|
||||
if ((identity.name || "") !== (server.name || "")) {
|
||||
await db.update(servers)
|
||||
.set({ name: identity.name, updatedAt: now })
|
||||
.where(eq(servers.id, serverId))
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
} catch (err) {
|
||||
// Log but don't throw — we still persist the offline snapshot
|
||||
console.warn(`[poller] server id=${serverId} unreachable:`, (err as Error).message)
|
||||
}
|
||||
|
||||
const [inserted] = db
|
||||
const [inserted] = await db
|
||||
.insert(serverSnapshots)
|
||||
.values(partialSnap as SnapshotInsert)
|
||||
.returning()
|
||||
.all()
|
||||
|
||||
const cutoffIso = new Date(Date.now() - 14 * 24 * 3600_000).toISOString()
|
||||
await db.delete(serverSnapshots).where(lt(serverSnapshots.polledAt, cutoffIso))
|
||||
void dropExpiredPartitions(pool).then(() => ensurePartitionsAround(pool))
|
||||
|
||||
return toSnapshotRead(inserted)
|
||||
}
|
||||
|
||||
@@ -6,8 +6,8 @@
|
||||
* 2. `traffic`, `uptime_resources`, `servers_rest_ping` — средний (REST `/system/identity` по каталогу).
|
||||
* 3. `uptime_speed` — ниже (BW-test, тяжёлый); локи узлов — `withBtestNodeLocks` в speed-сервисе.
|
||||
*
|
||||
* **Оповещения (`alert_engine`):** читают SQLite после джоб сбора (см. `buildSignalSnapshot`), в т.ч.
|
||||
* `gre_bgp` → `alert_gre_tunnel_samples` / `alert_bgp_peer_samples`. После успешного завершения джоб
|
||||
* **Оповещения (`alert_engine`):** читают PostgreSQL после джоб сбора (см. `buildSignalSnapshot`), в т.ч.
|
||||
* `gre_bgp` → snapshot в `scheduler_runs.result_json` (`greTunnels` / `bgpPeers`). После успешного завершения джоб
|
||||
* `traffic`, `uptime_*`, `servers_rest_ping`, `gre_bgp` планируется **дополнительный** прогон движка
|
||||
* (debounce), см. [`alert-collector-hooks.ts`](./alert-collector-hooks.ts); любой другой писатель сэмплов
|
||||
* для снимка оповещений тоже должен вызывать `scheduleAlertEngineAfterDataCollectors()` после коммита.
|
||||
@@ -17,7 +17,7 @@
|
||||
*/
|
||||
import { desc, eq, lt } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { schedulerRuns } from "../db/schema.js"
|
||||
import { events, schedulerRuns } from "../db/schema.js"
|
||||
import type { SchedulerRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
|
||||
import {
|
||||
@@ -71,11 +71,25 @@ const timers = new Map<string, ReturnType<typeof setInterval>>()
|
||||
|
||||
const RUN_LOG_RETENTION_MS = 30 * 24 * 60 * 60 * 1000
|
||||
|
||||
const QUIET_SCHEDULER_OK_JOBS = new Set<SchedulerJobKey>([
|
||||
"traffic",
|
||||
"servers_rest_ping",
|
||||
"uptime_resources",
|
||||
"uptime_ping",
|
||||
"uptime_speed",
|
||||
"gre_bgp",
|
||||
"alert_engine",
|
||||
])
|
||||
|
||||
export function shouldAppendSchedulerOkEvent(jobKey: SchedulerJobKey): boolean {
|
||||
return !QUIET_SCHEDULER_OK_JOBS.has(jobKey)
|
||||
}
|
||||
|
||||
function newRunId(): string {
|
||||
return `sch-${Date.now()}-${Math.random().toString(36).slice(2, 10)}`
|
||||
}
|
||||
|
||||
function appendSchedulerRun(row: {
|
||||
async function appendSchedulerRun(row: {
|
||||
jobKey: string
|
||||
startedAt: string
|
||||
finishedAt: string
|
||||
@@ -84,18 +98,21 @@ function appendSchedulerRun(row: {
|
||||
durationMs: number
|
||||
result?: SchedulerRunSnapshot | null
|
||||
}) {
|
||||
db.insert(schedulerRuns).values({
|
||||
id: newRunId(),
|
||||
jobKey: row.jobKey,
|
||||
startedAt: row.startedAt,
|
||||
finishedAt: row.finishedAt,
|
||||
status: row.status,
|
||||
error: row.error,
|
||||
durationMs: row.durationMs,
|
||||
resultJson: row.result ? JSON.stringify(row.result) : null,
|
||||
}).run()
|
||||
const cutoff = new Date(Date.now() - RUN_LOG_RETENTION_MS).toISOString()
|
||||
db.delete(schedulerRuns).where(lt(schedulerRuns.finishedAt, cutoff)).run()
|
||||
await db.transaction(async (tx) => {
|
||||
await tx.insert(schedulerRuns).values({
|
||||
id: newRunId(),
|
||||
jobKey: row.jobKey,
|
||||
startedAt: row.startedAt,
|
||||
finishedAt: row.finishedAt,
|
||||
status: row.status,
|
||||
error: row.error,
|
||||
durationMs: row.durationMs,
|
||||
resultJson: row.result ?? null,
|
||||
})
|
||||
await tx.delete(schedulerRuns).where(lt(schedulerRuns.finishedAt, cutoff))
|
||||
await tx.delete(events).where(lt(events.createdAt, cutoff))
|
||||
})
|
||||
}
|
||||
|
||||
async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
@@ -150,7 +167,7 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
throw new Error(`Unknown job: ${jobKey}`)
|
||||
}
|
||||
const finishedIso = new Date().toISOString()
|
||||
appendSchedulerRun({
|
||||
await appendSchedulerRun({
|
||||
jobKey,
|
||||
startedAt: startedIso,
|
||||
finishedAt: finishedIso,
|
||||
@@ -159,19 +176,21 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
durationMs: Date.now() - startedAt,
|
||||
result: snapshot ?? null,
|
||||
})
|
||||
appendEvent({
|
||||
level: "info",
|
||||
eventType: "scheduler.job.ok",
|
||||
sourceModule: "scheduler",
|
||||
title: "Задача планировщика завершена",
|
||||
message: `${jobKey}: выполнено за ${Date.now() - startedAt} мс`,
|
||||
entityType: "job",
|
||||
entityId: jobKey,
|
||||
payload: {
|
||||
startedAt: startedIso,
|
||||
finishedAt: finishedIso,
|
||||
},
|
||||
})
|
||||
if (shouldAppendSchedulerOkEvent(jobKey)) {
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "scheduler.job.ok",
|
||||
sourceModule: "scheduler",
|
||||
title: "Задача планировщика завершена",
|
||||
message: `${jobKey}: выполнено за ${Date.now() - startedAt} мс`,
|
||||
entityType: "job",
|
||||
entityId: jobKey,
|
||||
payload: {
|
||||
startedAt: startedIso,
|
||||
finishedAt: finishedIso,
|
||||
},
|
||||
})
|
||||
}
|
||||
if (
|
||||
jobKey === "traffic" ||
|
||||
jobKey === "servers_rest_ping" ||
|
||||
@@ -186,7 +205,7 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
const finishedIso = new Date().toISOString()
|
||||
appendSchedulerRun({
|
||||
await appendSchedulerRun({
|
||||
jobKey,
|
||||
startedAt: startedIso,
|
||||
finishedAt: finishedIso,
|
||||
@@ -195,7 +214,7 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
durationMs: Date.now() - startedAt,
|
||||
result: snapshot ?? null,
|
||||
})
|
||||
appendEvent({
|
||||
await appendEvent({
|
||||
level: "critical",
|
||||
eventType: "scheduler.job.failed",
|
||||
sourceModule: "scheduler",
|
||||
@@ -248,10 +267,10 @@ function clearAllTimers() {
|
||||
}
|
||||
|
||||
/** Пересоздать интервалы после смены настроек. */
|
||||
export function refreshScheduler(): void {
|
||||
export async function refreshScheduler(): Promise<void> {
|
||||
clearAllTimers()
|
||||
|
||||
const traffic = getTrafficSettings()
|
||||
const traffic = await getTrafficSettings()
|
||||
if (traffic.enabled) {
|
||||
const ms = Math.max(5_000, traffic.intervalSec * 1000)
|
||||
void executeSchedulerJob("traffic").catch(() => {})
|
||||
@@ -263,8 +282,8 @@ export function refreshScheduler(): void {
|
||||
)
|
||||
}
|
||||
|
||||
const apiPing = getServersApiPingSettings()
|
||||
const internetPath = getInternetPathSettings()
|
||||
const apiPing = await getServersApiPingSettings()
|
||||
const internetPath = await getInternetPathSettings()
|
||||
if (apiPing.enabled) {
|
||||
const apiMs = Math.max(10_000, apiPing.intervalSec * 1000)
|
||||
void executeSchedulerJob("servers_rest_ping").catch(() => {})
|
||||
@@ -276,7 +295,7 @@ export function refreshScheduler(): void {
|
||||
)
|
||||
}
|
||||
|
||||
const uptime = getUptimeSettings()
|
||||
const uptime = await getUptimeSettings()
|
||||
const resOn = uptime.resourcesEnabled ?? uptime.enabled
|
||||
const pingOn = uptime.pingEnabled ?? uptime.enabled
|
||||
const spdOn = uptime.speedEnabled ?? uptime.enabled
|
||||
@@ -334,7 +353,7 @@ export function refreshScheduler(): void {
|
||||
}, greBgpMs),
|
||||
)
|
||||
|
||||
const certRenew = getCertificateRenewSettings()
|
||||
const certRenew = await getCertificateRenewSettings()
|
||||
if (certRenew.enabled) {
|
||||
const certRenewMs = Math.max(300_000, certRenew.intervalSec * 1000)
|
||||
void executeSchedulerJob("certificates_renew").catch(() => {})
|
||||
@@ -346,7 +365,7 @@ export function refreshScheduler(): void {
|
||||
)
|
||||
}
|
||||
|
||||
const backupSchedule = getBackupScheduleSettings()
|
||||
const backupSchedule = await getBackupScheduleSettings()
|
||||
if (backupSchedule.enabled) {
|
||||
const backupMs = 60_000
|
||||
void executeSchedulerJob("backups").catch(() => {})
|
||||
@@ -376,21 +395,20 @@ export function isJobRunning(jobKey: string): boolean {
|
||||
return isSchedulerJobRunning(jobKey)
|
||||
}
|
||||
|
||||
function lastRunForJob(jobKey: string) {
|
||||
return db.select().from(schedulerRuns)
|
||||
async function lastRunForJob(jobKey: string) {
|
||||
return (await db.select().from(schedulerRuns)
|
||||
.where(eq(schedulerRuns.jobKey, jobKey))
|
||||
.orderBy(desc(schedulerRuns.finishedAt))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
}
|
||||
|
||||
export function getSchedulerStatus() {
|
||||
const traffic = getTrafficSettings()
|
||||
const uptime = getUptimeSettings()
|
||||
const apiPing = getServersApiPingSettings()
|
||||
const internetPath = getInternetPathSettings()
|
||||
const certRenew = getCertificateRenewSettings()
|
||||
const backupSchedule = getBackupScheduleSettings()
|
||||
export async function getSchedulerStatus() {
|
||||
const traffic = await getTrafficSettings()
|
||||
const uptime = await getUptimeSettings()
|
||||
const apiPing = await getServersApiPingSettings()
|
||||
const internetPath = await getInternetPathSettings()
|
||||
const certRenew = await getCertificateRenewSettings()
|
||||
const backupSchedule = await getBackupScheduleSettings()
|
||||
|
||||
const resOn = uptime.resourcesEnabled ?? uptime.enabled
|
||||
const pingOn = uptime.pingEnabled ?? uptime.enabled
|
||||
@@ -410,8 +428,8 @@ export function getSchedulerStatus() {
|
||||
}
|
||||
|
||||
return {
|
||||
jobs: JOB_KEYS.map((jobKey) => {
|
||||
const last = lastRunForJob(jobKey)
|
||||
jobs: await Promise.all(JOB_KEYS.map(async (jobKey) => {
|
||||
const last = await lastRunForJob(jobKey)
|
||||
const m = jobMeta[jobKey]
|
||||
return {
|
||||
jobKey,
|
||||
@@ -423,29 +441,27 @@ export function getSchedulerStatus() {
|
||||
lastDurationMs: last?.durationMs ?? null,
|
||||
lastError: last?.error ?? null,
|
||||
}
|
||||
}),
|
||||
})),
|
||||
}
|
||||
}
|
||||
|
||||
export function listSchedulerRuns(opts: { jobKey?: string; limit: number; offset: number }) {
|
||||
export async function listSchedulerRuns(opts: { jobKey?: string; limit: number; offset: number }) {
|
||||
const limit = Math.min(200, Math.max(1, opts.limit))
|
||||
const offset = Math.max(0, opts.offset)
|
||||
if (opts.jobKey) {
|
||||
return {
|
||||
runs: db.select().from(schedulerRuns)
|
||||
runs: await db.select().from(schedulerRuns)
|
||||
.where(eq(schedulerRuns.jobKey, opts.jobKey))
|
||||
.orderBy(desc(schedulerRuns.finishedAt))
|
||||
.limit(limit)
|
||||
.offset(offset)
|
||||
.all(),
|
||||
.offset(offset),
|
||||
}
|
||||
}
|
||||
return {
|
||||
runs: db.select().from(schedulerRuns)
|
||||
runs: await db.select().from(schedulerRuns)
|
||||
.orderBy(desc(schedulerRuns.finishedAt))
|
||||
.limit(limit)
|
||||
.offset(offset)
|
||||
.all(),
|
||||
.offset(offset),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { eq, or } from "drizzle-orm"
|
||||
import { db, sqliteDatabase } from "../db/index.js"
|
||||
import { db, dbAll, dbQuery } from "../db/index.js"
|
||||
import { servers, serversApiPingSettings, serversRestPingSamples } from "../db/schema.js"
|
||||
import type { ServersRestPingRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
|
||||
@@ -7,75 +7,67 @@ import { MikrotikClient } from "./mikrotik.js"
|
||||
|
||||
let collecting = false
|
||||
|
||||
function getSettingsRow() {
|
||||
const row = db.select().from(serversApiPingSettings).where(eq(serversApiPingSettings.id, 1)).limit(1).all()[0]
|
||||
async function getSettingsRow() {
|
||||
const row = (await db.select().from(serversApiPingSettings).where(eq(serversApiPingSettings.id, 1)).limit(1))[0]
|
||||
if (row) return row
|
||||
const now = new Date().toISOString()
|
||||
db.insert(serversApiPingSettings).values({
|
||||
await db.insert(serversApiPingSettings).values({
|
||||
id: 1,
|
||||
enabled: false,
|
||||
intervalSec: 120,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
}).run()
|
||||
return db.select().from(serversApiPingSettings).where(eq(serversApiPingSettings.id, 1)).limit(1).all()[0]
|
||||
})
|
||||
return (await db.select().from(serversApiPingSettings).where(eq(serversApiPingSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
/** Как uptime: включённые в каталоге или Home Router (часто «выкл.» при доступе только из LAN). */
|
||||
function serversForRestProbe() {
|
||||
return db
|
||||
async function serversForRestProbe() {
|
||||
return await db
|
||||
.select()
|
||||
.from(servers)
|
||||
.where(or(eq(servers.enabled, true), eq(servers.type, "home-router")))
|
||||
.all()
|
||||
}
|
||||
|
||||
export function getServersApiPingSettings() {
|
||||
return getSettingsRow()
|
||||
export async function getServersApiPingSettings() {
|
||||
return await getSettingsRow()
|
||||
}
|
||||
|
||||
export function updateServersApiPingSettings(patch: { enabled?: boolean; intervalSec?: number }) {
|
||||
const prev = getSettingsRow()
|
||||
export async function updateServersApiPingSettings(patch: { enabled?: boolean; intervalSec?: number }) {
|
||||
const prev = await getSettingsRow()
|
||||
const next = {
|
||||
enabled: patch.enabled ?? prev.enabled,
|
||||
intervalSec: patch.intervalSec ?? prev.intervalSec,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
db.update(serversApiPingSettings).set(next).where(eq(serversApiPingSettings.id, 1)).run()
|
||||
return getSettingsRow()
|
||||
await db.update(serversApiPingSettings).set(next).where(eq(serversApiPingSettings.id, 1))
|
||||
return await getSettingsRow()
|
||||
}
|
||||
|
||||
export function getServersRestPingCollectorState(): { running: boolean } {
|
||||
return { running: collecting }
|
||||
}
|
||||
|
||||
function loadLastRestSampleByServer(): Map<number, { ok: boolean }> {
|
||||
const rows = sqliteDatabase
|
||||
.prepare(
|
||||
`SELECT s.server_id AS serverId, s.ok
|
||||
FROM servers_rest_ping_samples s
|
||||
INNER JOIN (
|
||||
SELECT server_id, MAX(id) AS mid FROM servers_rest_ping_samples GROUP BY server_id
|
||||
) t ON t.mid = s.id`,
|
||||
)
|
||||
.all() as { serverId: number; ok: number }[]
|
||||
async function loadLastRestSampleByServer(): Promise<Map<number, { ok: boolean }>> {
|
||||
const rows = await dbAll<{ serverId: number; ok: boolean | number }>(
|
||||
`SELECT DISTINCT ON (server_id) server_id AS "serverId", ok
|
||||
FROM servers_rest_ping_samples
|
||||
ORDER BY server_id, sampled_at DESC`,
|
||||
)
|
||||
const m = new Map<number, { ok: boolean }>()
|
||||
for (const r of rows) m.set(r.serverId, { ok: Boolean(r.ok) })
|
||||
return m
|
||||
}
|
||||
|
||||
/** Последний сэмпл ресурсов — для согласования «восстановился» с REST без второго тика uptime. */
|
||||
function loadLatestResourceDownServerIds(): Set<number> {
|
||||
const rows = sqliteDatabase
|
||||
.prepare(
|
||||
`WITH r AS (
|
||||
SELECT urs.server_id AS serverId, urs.status,
|
||||
ROW_NUMBER() OVER (PARTITION BY urs.server_id ORDER BY urs.id DESC) AS rn
|
||||
FROM uptime_resource_samples urs
|
||||
)
|
||||
SELECT serverId FROM r WHERE rn = 1 AND status IN ('offline', 'degraded')`,
|
||||
)
|
||||
.all() as { serverId: number }[]
|
||||
async function loadLatestResourceDownServerIds(): Promise<Set<number>> {
|
||||
const rows = await dbAll<{ serverId: number }>(
|
||||
`SELECT server_id AS "serverId" FROM (
|
||||
SELECT DISTINCT ON (server_id) server_id, status
|
||||
FROM uptime_resource_samples
|
||||
ORDER BY server_id, sampled_at DESC
|
||||
) latest WHERE status IN ('offline', 'degraded')`,
|
||||
)
|
||||
return new Set(rows.map((x) => x.serverId))
|
||||
}
|
||||
|
||||
@@ -104,7 +96,7 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
|
||||
}
|
||||
|
||||
try {
|
||||
const targets = serversForRestProbe()
|
||||
const targets = await serversForRestProbe()
|
||||
const rows = await Promise.all(
|
||||
targets.map(async (srv) => {
|
||||
const name = srv.name || srv.host
|
||||
@@ -134,23 +126,23 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
|
||||
)
|
||||
snapshot.servers = rows.sort((a, b) => a.serverId - b.serverId)
|
||||
|
||||
const lastByServer = loadLastRestSampleByServer()
|
||||
const resourceDown = loadLatestResourceDownServerIds()
|
||||
const lastByServer = await loadLastRestSampleByServer()
|
||||
const resourceDown = await loadLatestResourceDownServerIds()
|
||||
|
||||
db.transaction((tx) => {
|
||||
await db.transaction(async (tx) => {
|
||||
for (const row of snapshot.servers) {
|
||||
const sid = row.serverId
|
||||
const last = lastByServer.get(sid)
|
||||
const errText = row.ok ? null : (row.error ?? null)
|
||||
|
||||
const insertOne = (ok: boolean, latencyMs: number | null, error: string | null) => {
|
||||
tx.insert(serversRestPingSamples).values({
|
||||
const insertOne = async (ok: boolean, latencyMs: number | null, error: string | null) => {
|
||||
await tx.insert(serversRestPingSamples).values({
|
||||
serverId: sid,
|
||||
sampledAt,
|
||||
ok,
|
||||
latencyMs,
|
||||
error,
|
||||
}).run()
|
||||
})
|
||||
}
|
||||
|
||||
const lat = row.latencyMs ?? null
|
||||
@@ -158,23 +150,21 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
|
||||
if (row.ok) {
|
||||
const downRes = resourceDown.has(sid)
|
||||
if (last === undefined && downRes) {
|
||||
insertOne(false, null, null)
|
||||
await insertOne(false, null, null)
|
||||
}
|
||||
insertOne(true, lat, null)
|
||||
await insertOne(true, lat, null)
|
||||
if (last?.ok === false || (last === undefined && downRes)) {
|
||||
insertOne(true, lat, null)
|
||||
await insertOne(true, lat, null)
|
||||
}
|
||||
} else {
|
||||
insertOne(false, null, errText)
|
||||
await insertOne(false, null, errText)
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
sqliteDatabase
|
||||
.prepare(`DELETE FROM servers_rest_ping_samples WHERE sampled_at < datetime('now', '-30 days')`)
|
||||
.run()
|
||||
await dbQuery(`DELETE FROM servers_rest_ping_samples WHERE sampled_at < now() - interval '30 days'`)
|
||||
|
||||
db.update(serversApiPingSettings)
|
||||
await db.update(serversApiPingSettings)
|
||||
.set({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - startedAt,
|
||||
@@ -182,11 +172,10 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
|
||||
updatedAt: sampledAt,
|
||||
})
|
||||
.where(eq(serversApiPingSettings.id, 1))
|
||||
.run()
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
snapshot.fatalError = msg
|
||||
db.update(serversApiPingSettings)
|
||||
await db.update(serversApiPingSettings)
|
||||
.set({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - startedAt,
|
||||
@@ -194,7 +183,6 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
|
||||
updatedAt: sampledAt,
|
||||
})
|
||||
.where(eq(serversApiPingSettings.id, 1))
|
||||
.run()
|
||||
} finally {
|
||||
collecting = false
|
||||
}
|
||||
|
||||
@@ -1,68 +1,78 @@
|
||||
import { randomUUID } from "node:crypto"
|
||||
import { spawn } from "node:child_process"
|
||||
import { mkdir, readFile, rm, writeFile } from "node:fs/promises"
|
||||
import os from "node:os"
|
||||
import path from "node:path"
|
||||
import Database from "better-sqlite3"
|
||||
import { env } from "../config.js"
|
||||
import { beginSqliteExclusiveOp, endSqliteExclusiveOp, reopenSqlite, sqliteDatabase } from "../db/index.js"
|
||||
import { withAdvisoryLock } from "../db/index.js"
|
||||
import { refreshScheduler, stopScheduler } from "./scheduler.js"
|
||||
import {
|
||||
reattachFlowSqlite,
|
||||
startTrafficFlowListener,
|
||||
stopTrafficFlowListener,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
|
||||
const SQLITE_MAGIC = Buffer.from("SQLite format 3\0")
|
||||
const PG_DUMP_MAGIC = Buffer.from("PGDMP")
|
||||
const MAX_RESTORE_BYTES = 512 * 1024 * 1024
|
||||
|
||||
type SqliteHandle = InstanceType<typeof Database>
|
||||
const BACKUP_LOCK_KEY = 8_723_102
|
||||
|
||||
function fmtTimestamp(date = new Date()): string {
|
||||
const pad = (n: number) => String(n).padStart(2, "0")
|
||||
return `${date.getFullYear()}-${pad(date.getMonth() + 1)}-${pad(date.getDate())}_${pad(date.getHours())}-${pad(date.getMinutes())}-${pad(date.getSeconds())}`
|
||||
}
|
||||
|
||||
function resolveDatabasePath(): string {
|
||||
return path.resolve(process.cwd(), env.DATABASE_PATH)
|
||||
}
|
||||
|
||||
function assertSqliteFile(buffer: Buffer): void {
|
||||
if (buffer.length < SQLITE_MAGIC.length) {
|
||||
throw new Error("Файл слишком маленький для SQLite")
|
||||
function assertPgDumpFile(buffer: Buffer): void {
|
||||
if (buffer.length < PG_DUMP_MAGIC.length) {
|
||||
throw new Error("Файл слишком маленький для дампа PostgreSQL")
|
||||
}
|
||||
if (!buffer.subarray(0, SQLITE_MAGIC.length).equals(SQLITE_MAGIC)) {
|
||||
throw new Error("Файл не похож на резервную копию SQLite")
|
||||
if (!buffer.subarray(0, PG_DUMP_MAGIC.length).equals(PG_DUMP_MAGIC)) {
|
||||
throw new Error("Файл не похож на резервную копию PostgreSQL (ожидается формат custom, PGDMP)")
|
||||
}
|
||||
}
|
||||
|
||||
async function withDatabaseOperation<T>(fn: () => Promise<T> | T): Promise<T> {
|
||||
beginSqliteExclusiveOp()
|
||||
stopTrafficFlowListener()
|
||||
stopScheduler()
|
||||
try {
|
||||
return await fn()
|
||||
} finally {
|
||||
startTrafficFlowListener()
|
||||
refreshScheduler()
|
||||
endSqliteExclusiveOp()
|
||||
}
|
||||
function spawnCollected(command: string, args: string[], opts?: { stdin?: Buffer }): Promise<Buffer> {
|
||||
return new Promise((resolve, reject) => {
|
||||
const child = spawn(command, args, {
|
||||
env: { ...process.env, DATABASE_URL: env.DATABASE_URL },
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
})
|
||||
const chunks: Buffer[] = []
|
||||
let err = ""
|
||||
child.stdout.on("data", (chunk: Buffer) => chunks.push(chunk))
|
||||
child.stderr.on("data", (chunk: Buffer) => {
|
||||
err += chunk.toString("utf8")
|
||||
})
|
||||
child.on("error", reject)
|
||||
child.on("close", (code) => {
|
||||
if (code !== 0) {
|
||||
reject(new Error(err.trim() || `${command} завершился с кодом ${code}`))
|
||||
return
|
||||
}
|
||||
resolve(Buffer.concat(chunks))
|
||||
})
|
||||
if (opts?.stdin) child.stdin.end(opts.stdin)
|
||||
else child.stdin.end()
|
||||
})
|
||||
}
|
||||
|
||||
async function withDatabaseOperation<T>(fn: () => Promise<T>): Promise<T> {
|
||||
return await withAdvisoryLock(BACKUP_LOCK_KEY, async () => {
|
||||
stopTrafficFlowListener()
|
||||
stopScheduler()
|
||||
try {
|
||||
return await fn()
|
||||
} finally {
|
||||
await startTrafficFlowListener()
|
||||
await refreshScheduler()
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export async function exportSystemDatabaseBackup(): Promise<{ filename: string; buffer: Buffer }> {
|
||||
return withDatabaseOperation(async () => {
|
||||
sqliteDatabase.pragma("wal_checkpoint(TRUNCATE)")
|
||||
const tempDir = path.join(os.tmpdir(), "mmapp-db-backup")
|
||||
await mkdir(tempDir, { recursive: true })
|
||||
const tempPath = path.join(tempDir, `manager-${randomUUID()}.db`)
|
||||
try {
|
||||
await sqliteDatabase.backup(tempPath)
|
||||
const buffer = await readFile(tempPath)
|
||||
return {
|
||||
filename: `mikrotik-manager_${fmtTimestamp()}.db`,
|
||||
buffer,
|
||||
}
|
||||
} finally {
|
||||
await rm(tempPath, { force: true })
|
||||
return await withDatabaseOperation(async () => {
|
||||
const buffer = await spawnCollected("pg_dump", ["-Fc", env.DATABASE_URL])
|
||||
return {
|
||||
filename: `mikrotik-manager_${fmtTimestamp()}.dump`,
|
||||
buffer,
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -71,27 +81,21 @@ export async function restoreSystemDatabaseBackup(buffer: Buffer): Promise<void>
|
||||
if (buffer.length > MAX_RESTORE_BYTES) {
|
||||
throw new Error("Файл бэкапа слишком большой")
|
||||
}
|
||||
assertSqliteFile(buffer)
|
||||
assertPgDumpFile(buffer)
|
||||
|
||||
await withDatabaseOperation(async () => {
|
||||
const tempDir = path.join(os.tmpdir(), "mmapp-db-restore")
|
||||
await mkdir(tempDir, { recursive: true })
|
||||
const tempPath = path.join(tempDir, `restore-${randomUUID()}.db`)
|
||||
let source: SqliteHandle | null = null
|
||||
const tempPath = path.join(tempDir, `restore-${randomUUID()}.dump`)
|
||||
try {
|
||||
await writeFile(tempPath, buffer)
|
||||
source = new Database(tempPath, { readonly: true, fileMustExist: true })
|
||||
await source.backup(resolveDatabasePath())
|
||||
reopenSqlite()
|
||||
reattachFlowSqlite()
|
||||
sqliteDatabase.pragma("wal_checkpoint(TRUNCATE)")
|
||||
await spawnCollected("pg_restore", ["--clean", "--if-exists", "-d", env.DATABASE_URL, tempPath])
|
||||
} finally {
|
||||
source?.close()
|
||||
await rm(tempPath, { force: true })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export function getSystemDatabasePath(): string {
|
||||
return resolveDatabasePath()
|
||||
return env.DATABASE_URL
|
||||
}
|
||||
|
||||
@@ -64,38 +64,36 @@ function toNum(raw: unknown): number {
|
||||
return Number.isFinite(n) ? Math.max(0, Math.round(n)) : 0
|
||||
}
|
||||
|
||||
function getSettingsRow() {
|
||||
const row = db.select().from(trafficSettings).where(eq(trafficSettings.id, 1)).limit(1).all()[0]
|
||||
async function getSettingsRow() {
|
||||
const row = (await db.select().from(trafficSettings).where(eq(trafficSettings.id, 1)).limit(1))[0]
|
||||
if (row) return row
|
||||
const now = new Date().toISOString()
|
||||
db.insert(trafficSettings).values({
|
||||
await db.insert(trafficSettings).values({
|
||||
id: 1,
|
||||
enabled: true,
|
||||
intervalSec: 30,
|
||||
retentionDays: 14,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
}).run()
|
||||
return db.select().from(trafficSettings).where(eq(trafficSettings.id, 1)).limit(1).all()[0]
|
||||
})
|
||||
return (await db.select().from(trafficSettings).where(eq(trafficSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
function cleanupOldSamples(retentionDays: number) {
|
||||
async function cleanupOldSamples(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
db.delete(trafficSamples)
|
||||
await db.delete(trafficSamples)
|
||||
.where(lt(trafficSamples.sampledAt, cutoff))
|
||||
.run()
|
||||
}
|
||||
|
||||
function readPreviousWave(serverId: number): Map<string, { rxBytes: number; txBytes: number; sampledAt: string }> {
|
||||
const last = db
|
||||
async function readPreviousWave(serverId: number): Promise<Map<string, { rxBytes: number; txBytes: number; sampledAt: string }>> {
|
||||
const last = (await db
|
||||
.select({ sampledAt: trafficSamples.sampledAt })
|
||||
.from(trafficSamples)
|
||||
.where(eq(trafficSamples.serverId, serverId))
|
||||
.orderBy(desc(trafficSamples.sampledAt))
|
||||
.limit(1)
|
||||
.all()[0]
|
||||
.limit(1))[0]
|
||||
if (!last) return new Map()
|
||||
const rows = db
|
||||
const rows = await db
|
||||
.select({
|
||||
interfaceName: trafficSamples.interfaceName,
|
||||
peerPublicKey: trafficSamples.peerPublicKey,
|
||||
@@ -108,7 +106,6 @@ function readPreviousWave(serverId: number): Map<string, { rxBytes: number; txBy
|
||||
eq(trafficSamples.serverId, serverId),
|
||||
eq(trafficSamples.sampledAt, last.sampledAt),
|
||||
))
|
||||
.all()
|
||||
return new Map(rows.map((r) => [`${r.interfaceName}\0${r.peerPublicKey ?? ""}`, r]))
|
||||
}
|
||||
|
||||
@@ -126,7 +123,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
collecting = true
|
||||
const startedAt = Date.now()
|
||||
const now = sampledAt
|
||||
const settings = getSettingsRow()
|
||||
const settings = await getSettingsRow()
|
||||
const snapshot: TrafficRunSnapshot = {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "traffic",
|
||||
@@ -135,7 +132,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
}
|
||||
|
||||
try {
|
||||
const enabledServers = db.select().from(servers).where(eq(servers.enabled, true)).all()
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
|
||||
for (const srv of enabledServers) {
|
||||
const name = srv.name || srv.host
|
||||
@@ -143,7 +140,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
const client = MikrotikClient.fromServer(srv)
|
||||
const ifaces = await client.get<RosIfaceTraffic[]>("/interface")
|
||||
rememberServerIfaces(srv.id, ifaces)
|
||||
const prevWave = readPreviousWave(srv.id)
|
||||
const prevWave = await readPreviousWave(srv.id)
|
||||
const nowMs = Date.parse(now)
|
||||
let sumRxMbps = 0
|
||||
let sumTxMbps = 0
|
||||
@@ -205,7 +202,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
/* WG peers optional — iface samples already recorded */
|
||||
}
|
||||
if (rows.length > 0) {
|
||||
db.insert(trafficSamples).values(rows).run()
|
||||
await db.insert(trafficSamples).values(rows)
|
||||
}
|
||||
snapshot.servers.push({
|
||||
serverId: srv.id,
|
||||
@@ -227,22 +224,22 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
}
|
||||
}
|
||||
|
||||
cleanupOldSamples(Math.max(1, settings.retentionDays))
|
||||
db.update(trafficSettings).set({
|
||||
await cleanupOldSamples(Math.max(1, settings.retentionDays))
|
||||
await db.update(trafficSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - startedAt,
|
||||
lastError: "",
|
||||
updatedAt: now,
|
||||
}).where(eq(trafficSettings.id, 1)).run()
|
||||
}).where(eq(trafficSettings.id, 1))
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
snapshot.fatalError = msg
|
||||
db.update(trafficSettings).set({
|
||||
await db.update(trafficSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - startedAt,
|
||||
lastError: msg,
|
||||
updatedAt: now,
|
||||
}).where(eq(trafficSettings.id, 1)).run()
|
||||
}).where(eq(trafficSettings.id, 1))
|
||||
} finally {
|
||||
collecting = false
|
||||
}
|
||||
@@ -254,8 +251,8 @@ export function stopTrafficCollector() {}
|
||||
|
||||
export function restartTrafficCollector() {}
|
||||
|
||||
export function getTrafficCollectorState(): TrafficCollectorState {
|
||||
const s = getSettingsRow()
|
||||
export async function getTrafficCollectorState(): Promise<TrafficCollectorState> {
|
||||
const s = await getSettingsRow()
|
||||
return {
|
||||
running: Boolean(s.enabled),
|
||||
lastRunAt: s.lastCollectedAt ?? null,
|
||||
@@ -263,38 +260,37 @@ export function getTrafficCollectorState(): TrafficCollectorState {
|
||||
}
|
||||
}
|
||||
|
||||
/** Идёт активный прогон `collectTrafficOnce` (запись в SQLite ещё не завершена). */
|
||||
/** Идёт активный прогон `collectTrafficOnce` (запись в PostgreSQL ещё не завершена). */
|
||||
export function isTrafficCollecting(): boolean {
|
||||
return collecting
|
||||
}
|
||||
|
||||
export function getTrafficSettings() {
|
||||
return getSettingsRow()
|
||||
export async function getTrafficSettings() {
|
||||
return await getSettingsRow()
|
||||
}
|
||||
|
||||
export function updateTrafficSettings(patch: {
|
||||
export async function updateTrafficSettings(patch: {
|
||||
enabled?: boolean
|
||||
intervalSec?: number
|
||||
retentionDays?: number
|
||||
}) {
|
||||
const prev = getSettingsRow()
|
||||
const prev = await getSettingsRow()
|
||||
const next = {
|
||||
enabled: patch.enabled ?? prev.enabled,
|
||||
intervalSec: patch.intervalSec ?? prev.intervalSec,
|
||||
retentionDays: patch.retentionDays ?? prev.retentionDays,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
db.update(trafficSettings).set(next).where(eq(trafficSettings.id, 1)).run()
|
||||
return getSettingsRow()
|
||||
await db.update(trafficSettings).set(next).where(eq(trafficSettings.id, 1))
|
||||
return await getSettingsRow()
|
||||
}
|
||||
|
||||
export function readServerSamplesInRange(serverId: number, sinceIso: string) {
|
||||
return db.select()
|
||||
export async function readServerSamplesInRange(serverId: number, sinceIso: string) {
|
||||
return await db.select()
|
||||
.from(trafficSamples)
|
||||
.where(and(
|
||||
eq(trafficSamples.serverId, serverId),
|
||||
gte(trafficSamples.sampledAt, sinceIso),
|
||||
))
|
||||
.orderBy(asc(trafficSamples.sampledAt))
|
||||
.all()
|
||||
}
|
||||
|
||||
@@ -5,7 +5,8 @@ import {
|
||||
resetFlowRingsForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { buildFlowAnalytics, formatLiveSseFromBuilder, getFlowMonthly, listFlowClients, listFlowExporters } from "./traffic-flow-analytics.js"
|
||||
import { sqliteDatabase } from "../db/index.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests, seedFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
@@ -15,6 +16,11 @@ import {
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-analytics.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
@@ -63,7 +69,7 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
])
|
||||
|
||||
try {
|
||||
const all = buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
const all = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
assert.equal(all.applications[0]?.label, "HTTPS")
|
||||
assert.ok(all.protocols.some((p) => p.label === "TCP"))
|
||||
assert.equal(all.ifaces[0]?.name, "ether1")
|
||||
@@ -75,14 +81,14 @@ try {
|
||||
assert.equal(conv.application, "HTTPS")
|
||||
assert.ok(!/^\d+$/.test(conv.inIface))
|
||||
|
||||
const filtered = buildFlowAnalytics({ minutes: 5, serverId: 7, iface: "ether1" })
|
||||
const filtered = await buildFlowAnalytics({ minutes: 5, serverId: 7, iface: "ether1" })
|
||||
assert.ok(filtered.bytes >= 12_000)
|
||||
assert.equal(filtered.ifaces[0]?.name, "ether1")
|
||||
|
||||
const miss = buildFlowAnalytics({ minutes: 5, serverId: 7, iface: "wg-flow" })
|
||||
const miss = await buildFlowAnalytics({ minutes: 5, serverId: 7, iface: "wg-flow" })
|
||||
assert.equal(miss.conversations, 0)
|
||||
|
||||
const other = buildFlowAnalytics({ minutes: 5, serverId: 99 })
|
||||
const other = await buildFlowAnalytics({ minutes: 5, serverId: 99 })
|
||||
assert.equal(other.conversations, 0)
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
@@ -121,10 +127,10 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
},
|
||||
])
|
||||
try {
|
||||
const summed = buildFlowAnalytics({ minutes: 5, serverId: 7, dedup: false })
|
||||
const summed = await buildFlowAnalytics({ minutes: 5, serverId: 7, dedup: false })
|
||||
assert.equal(summed.bytes, 21_000)
|
||||
assert.equal(summed.conversations, 2)
|
||||
const deduped = buildFlowAnalytics({ minutes: 5, serverId: 7, dedup: true })
|
||||
const deduped = await buildFlowAnalytics({ minutes: 5, serverId: 7, dedup: true })
|
||||
assert.equal(deduped.bytes, 12_000)
|
||||
assert.equal(deduped.conversations, 1)
|
||||
assert.equal(deduped.dedupApplied, true)
|
||||
@@ -165,7 +171,7 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
},
|
||||
])
|
||||
try {
|
||||
const geo = buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
const geo = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
assert.equal(geo.categories?.[0]?.label, "Игры")
|
||||
assert.ok(geo.asns?.some((r) => r.label.includes("AS15169")))
|
||||
assert.equal(geo.countries?.[0]?.id, "US")
|
||||
@@ -210,7 +216,7 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
},
|
||||
])
|
||||
try {
|
||||
const cf = buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
const cf = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
assert.equal(cf.countries?.[0]?.id, "US")
|
||||
assert.ok(cf.mapEdges?.every((e) => e.toCountry !== "?"))
|
||||
assert.equal(cf.services?.[0]?.label, "Cloudflare")
|
||||
@@ -238,7 +244,7 @@ try {
|
||||
outIface: "",
|
||||
},
|
||||
])
|
||||
const degraded = buildFlowAnalytics({ minutes: 5, serverId: 7, skipHeavy: true })
|
||||
const degraded = await buildFlowAnalytics({ minutes: 5, serverId: 7, skipHeavy: true })
|
||||
assert.equal(degraded.degraded, true)
|
||||
assert.equal(degraded.conversationsList.length, 0)
|
||||
assert.ok((degraded.bytes ?? 0) >= 12_000)
|
||||
@@ -249,30 +255,35 @@ try {
|
||||
assert.equal((liveErr.data as { error: string }).error, "SQLITE_BUSY")
|
||||
const liveOk = formatLiveSseFromBuilder(() => ({ ok: true }))
|
||||
assert.equal(liveOk.event, "sample")
|
||||
const exporters = listFlowExporters(5)
|
||||
const clients = listFlowClients(5)
|
||||
const exporters = await listFlowExporters(5)
|
||||
const clients = await listFlowClients(5)
|
||||
assert.ok(Array.isArray(exporters.exporters))
|
||||
assert.ok(Array.isArray(clients.clients))
|
||||
resetFlowRingsForTests()
|
||||
}
|
||||
|
||||
{
|
||||
sqliteDatabase.prepare(`DELETE FROM flow_daily_dims WHERE server_id = 7 AND day LIKE '2026-09-%'`).run()
|
||||
sqliteDatabase.exec(`
|
||||
INSERT INTO flow_daily_dims (server_id, day, dim, key, bytes, packets)
|
||||
VALUES
|
||||
(7, '2026-09-01', 'country', 'US', 1000, 10),
|
||||
(7, '2026-09-02', 'country', 'US', 500, 5),
|
||||
(7, '2026-09-01', 'service', 'steam', 800, 8),
|
||||
(7, '2026-09-01', 'asn', '15169', 900, 9),
|
||||
(7, '2026-09-01', 'asn', 'other', 100, 1)
|
||||
`)
|
||||
const monthly = getFlowMonthly("2026-09", 7)
|
||||
assert.equal(monthly.bytes, 1500)
|
||||
assert.equal(monthly.countries[0]?.id, "US")
|
||||
assert.equal(monthly.countries[0]?.bytes, 1500)
|
||||
assert.ok(monthly.asns.some((row) => row.id === "other"))
|
||||
sqliteDatabase.prepare(`DELETE FROM flow_daily_dims WHERE server_id = 7 AND day LIKE '2026-09-%'`).run()
|
||||
if (await withPgOrSkip()) {
|
||||
const { pool } = await import("../db/index.js")
|
||||
const { ensurePartitionFor } = await import("../db/partitions.js")
|
||||
await ensurePartitionFor(pool, "flow_daily_dims", "month", new Date("2026-09-01T00:00:00Z"))
|
||||
await dbQuery(`DELETE FROM flow_daily_dims WHERE server_id = 7 AND day >= '2026-09-01' AND day < '2026-10-01'`)
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_dims (server_id, day, dim, key, bytes, packets)
|
||||
VALUES
|
||||
(7, '2026-09-01', 'country', 'US', 1000, 10),
|
||||
(7, '2026-09-02', 'country', 'US', 500, 5),
|
||||
(7, '2026-09-01', 'service', 'steam', 800, 8),
|
||||
(7, '2026-09-01', 'asn', '15169', 900, 9),
|
||||
(7, '2026-09-01', 'asn', 'other', 100, 1)
|
||||
`)
|
||||
const monthly = await getFlowMonthly("2026-09", 7)
|
||||
assert.equal(monthly.bytes, 1500)
|
||||
assert.equal(monthly.countries[0]?.id, "US")
|
||||
assert.equal(monthly.countries[0]?.bytes, 1500)
|
||||
assert.ok(monthly.asns.some((row) => row.id === "other"))
|
||||
await dbQuery(`DELETE FROM flow_daily_dims WHERE server_id = 7 AND day >= '2026-09-01' AND day < '2026-10-01'`)
|
||||
}
|
||||
}
|
||||
|
||||
{
|
||||
@@ -353,7 +364,7 @@ try {
|
||||
},
|
||||
])
|
||||
try {
|
||||
const def = buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
const def = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
assert.equal(def.bytes, 12_000)
|
||||
assert.equal(def.bytesPayload, 12_000)
|
||||
assert.equal(def.bytesOverlay, 5_000_000)
|
||||
@@ -371,7 +382,7 @@ try {
|
||||
assert.equal(path.clientName, "Alice")
|
||||
assert.equal(path.enName, "NSK-SERVHOST-RTK")
|
||||
assert.equal(path.dst, "173.194.160.163")
|
||||
const withAll = buildFlowAnalytics({ minutes: 5, serverId: 7, excludeOverlay: false, excludeMesh: false })
|
||||
const withAll = await buildFlowAnalytics({ minutes: 5, serverId: 7, excludeOverlay: false, excludeMesh: false })
|
||||
assert.equal(withAll.bytes, 12_000 + 5_000_000 + 8000)
|
||||
assert.ok(withAll.conversationsList.some((r) => r.plane === "overlay"))
|
||||
assert.ok(withAll.conversationsList.some((r) => r.plane === "client_mesh"))
|
||||
@@ -415,7 +426,7 @@ try {
|
||||
},
|
||||
])
|
||||
try {
|
||||
const rev = buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
const rev = await buildFlowAnalytics({ minutes: 5, serverId: 7 })
|
||||
const google = rev.conversationsList.find((r) => r.src === "173.194.151.65")
|
||||
const cf = rev.conversationsList.find((r) => r.src === "104.18.35.51")
|
||||
assert.equal(google?.service, "Google")
|
||||
@@ -430,37 +441,42 @@ try {
|
||||
}
|
||||
|
||||
{
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
const sidRow = sqliteDatabase.prepare(`SELECT id FROM servers LIMIT 1`).get() as { id?: number } | undefined
|
||||
if (sidRow?.id) {
|
||||
const sid = sidRow.id
|
||||
rememberServerIfaces(sid, [{ ".id": "*4", name: "gre-en-nsk" }])
|
||||
ingestParsedFlowsForServerForTests(sid, [{
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 1,
|
||||
dstPort: 443,
|
||||
bytes: 100,
|
||||
packets: 1,
|
||||
inIface: "4",
|
||||
outIface: "4",
|
||||
}])
|
||||
sqliteDatabase.prepare(`
|
||||
INSERT INTO traffic_samples (server_id, interface_name, sampled_at, rx_bytes, tx_bytes, rx_bps, tx_bps)
|
||||
VALUES (?, 'gre-en-nsk', datetime('now'), 9000000, 1000000, 40000000, 2000000)
|
||||
`).run(sid)
|
||||
try {
|
||||
const wire = buildFlowAnalytics({ minutes: 5, serverId: sid })
|
||||
assert.ok((wire.bpsWire ?? 0) >= 40_000_000)
|
||||
assert.notEqual(wire.bpsWire, (wire.bytes * 8) / 300)
|
||||
} finally {
|
||||
sqliteDatabase.prepare(`DELETE FROM traffic_samples WHERE server_id = ? AND interface_name = 'gre-en-nsk'`).run(sid)
|
||||
if (await withPgOrSkip()) {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
const sidRow = await dbQuery<{ id: number }>(`SELECT id FROM servers LIMIT 1`)
|
||||
const sid = sidRow.rows[0]?.id
|
||||
if (sid) {
|
||||
const { pool } = await import("../db/index.js")
|
||||
const { ensurePartitionFor } = await import("../db/partitions.js")
|
||||
await ensurePartitionFor(pool, "traffic_samples", "week", new Date())
|
||||
rememberServerIfaces(sid, [{ ".id": "*4", name: "gre-en-nsk" }])
|
||||
ingestParsedFlowsForServerForTests(sid, [{
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 1,
|
||||
dstPort: 443,
|
||||
bytes: 100,
|
||||
packets: 1,
|
||||
inIface: "4",
|
||||
outIface: "4",
|
||||
}])
|
||||
await dbQuery(`
|
||||
INSERT INTO traffic_samples (server_id, interface_name, sampled_at, rx_bytes, tx_bytes, rx_bps, tx_bps)
|
||||
VALUES ($1, 'gre-en-nsk', now(), 9000000, 1000000, 40000000, 2000000)
|
||||
`, [sid])
|
||||
try {
|
||||
const wire = await buildFlowAnalytics({ minutes: 5, serverId: sid })
|
||||
assert.ok((wire.bpsWire ?? 0) >= 40_000_000)
|
||||
assert.notEqual(wire.bpsWire, (wire.bytes * 8) / 300)
|
||||
} finally {
|
||||
await dbQuery(`DELETE FROM traffic_samples WHERE server_id = $1 AND interface_name = 'gre-en-nsk'`, [sid])
|
||||
}
|
||||
}
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
}
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-analytics.test.ts: ok")
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db, sqliteDatabase } from "../db/index.js"
|
||||
import { appUsers, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { appUsers, userInterfaceBindings } from "../db/schema.js"
|
||||
import type {
|
||||
FlowAnalyticsDto,
|
||||
FlowBreakdownRow,
|
||||
@@ -21,7 +21,7 @@ import {
|
||||
listFlowRowsForWindow,
|
||||
type PendingFlowRow,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { MAX_PENDING, RING_OVERLAY } from "./traffic-flow-engine.js"
|
||||
import { flowDataEpoch, MAX_PENDING, RING_OVERLAY } from "./traffic-flow-engine.js"
|
||||
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-settings.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
@@ -33,6 +33,7 @@ import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-plan
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import {
|
||||
enGreIfaceNames,
|
||||
getServerCatalog,
|
||||
latestWireBps,
|
||||
loadFlowTopology,
|
||||
resolveClient,
|
||||
@@ -93,9 +94,9 @@ function bump(
|
||||
map.set(id, prev)
|
||||
}
|
||||
|
||||
function userIfaceAllow(userId: string): Map<number, Set<string>> | null {
|
||||
async function userIfaceAllow(userId: string): Promise<Map<number, Set<string>> | null> {
|
||||
if (!userId) return null
|
||||
const binds = db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId)).all()
|
||||
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
|
||||
const allow = new Map<number, Set<string>>()
|
||||
for (const b of binds) {
|
||||
const set = allow.get(b.serverId) ?? new Set<string>()
|
||||
@@ -141,20 +142,52 @@ function topLabel(map: Map<string, { bytes: number; packets: number; label?: str
|
||||
return best
|
||||
}
|
||||
|
||||
export function buildFlowAnalytics(q: FlowAnalyticsQuery): FlowAnalyticsDto {
|
||||
const settings = getTrafficFlowSettingsRow()
|
||||
export async function buildFlowAnalytics(q: FlowAnalyticsQuery): Promise<FlowAnalyticsDto> {
|
||||
const key = analyticsQueryKey(q)
|
||||
const now = Date.now()
|
||||
if (analyticsCache && analyticsCache.key === key && now - analyticsCache.at < ANALYTICS_CACHE_TTL_MS) {
|
||||
return analyticsCache.dto
|
||||
}
|
||||
const dto = await buildFlowAnalyticsUncached(q)
|
||||
analyticsCache = { key, at: now, dto }
|
||||
return dto
|
||||
}
|
||||
|
||||
export function resetFlowAnalyticsCacheForTests(): void {
|
||||
analyticsCache = null
|
||||
}
|
||||
|
||||
function analyticsQueryKey(q: FlowAnalyticsQuery): string {
|
||||
return JSON.stringify({
|
||||
epoch: flowDataEpoch(),
|
||||
minutes: q.minutes,
|
||||
serverId: q.serverId ?? null,
|
||||
userId: q.userId ?? null,
|
||||
iface: q.iface ?? null,
|
||||
dedup: q.dedup !== false,
|
||||
excludeMesh: q.excludeMesh !== false,
|
||||
excludeOverlay: q.excludeOverlay !== false,
|
||||
skipHeavy: Boolean(q.skipHeavy),
|
||||
})
|
||||
}
|
||||
|
||||
const ANALYTICS_CACHE_TTL_MS = 2000
|
||||
let analyticsCache: { key: string; at: number; dto: FlowAnalyticsDto } | null = null
|
||||
|
||||
async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAnalyticsDto> {
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
const top = Math.min(50, Math.max(10, settings.topN))
|
||||
const windowSec = Math.max(60, q.minutes * 60)
|
||||
const raw = listFlowRowsForWindow(q.minutes)
|
||||
const allow = q.userId ? userIfaceAllow(q.userId) : null
|
||||
const serverRows = db.select().from(servers).all()
|
||||
const nameById = new Map(serverRows.map((s) => [s.id, s.name || s.host]))
|
||||
const countryById = new Map(serverRows.map((s) => [s.id, (s.country || "").toUpperCase() || "UN"]))
|
||||
const raw = await listFlowRowsForWindow(q.minutes)
|
||||
const allow = q.userId ? await userIfaceAllow(q.userId) : null
|
||||
const catalog = await getServerCatalog()
|
||||
const nameById = new Map([...catalog.byId].map(([id, s]) => [id, s.name]))
|
||||
const countryById = new Map([...catalog.byId].map(([id, s]) => [id, s.country]))
|
||||
const ifaceFilter = q.iface && q.iface !== "__all__" ? q.iface : ""
|
||||
const wantDedup = q.dedup !== false && !ifaceFilter
|
||||
const excludeMesh = q.excludeMesh !== false
|
||||
const excludeOverlay = q.excludeOverlay !== false
|
||||
const topo = loadFlowTopology()
|
||||
const topo = await loadFlowTopology()
|
||||
|
||||
refreshFlowCatalogInBackground()
|
||||
|
||||
@@ -414,7 +447,7 @@ export function buildFlowAnalytics(q: FlowAnalyticsQuery): FlowAnalyticsDto {
|
||||
? getRingMbps(ringServer, RING_OVERLAY)
|
||||
: { rxNow: 0, txNow: 0 }
|
||||
const greNames = ringServer ? enGreIfaceNames(topo, ringServer, [...ifacesForWire]) : []
|
||||
const wire = ringServer ? latestWireBps(ringServer, greNames) : { bps: 0, bytes: 0 }
|
||||
const wire = ringServer ? await latestWireBps(ringServer, greNames) : { bps: 0, bytes: 0 }
|
||||
|
||||
return {
|
||||
bpsNow: (ring.rxNow + ring.txNow) * 1_000_000 || (totalBytes * 8) / windowSec,
|
||||
@@ -472,25 +505,25 @@ function summarizeByServer(rows: PendingFlowRow[]) {
|
||||
return { bytes, sessions }
|
||||
}
|
||||
|
||||
export function listFlowExporters(minutes: number): FlowExportersDto {
|
||||
const runtime = getFlowRuntimeCounters()
|
||||
const rows = listFlowRowsForWindow(minutes)
|
||||
export async function listFlowExporters(minutes: number): Promise<FlowExportersDto> {
|
||||
const runtime = await getFlowRuntimeCounters()
|
||||
const rows = await listFlowRowsForWindow(minutes)
|
||||
const { bytes, sessions } = summarizeByServer(rows)
|
||||
const ids = new Set<number>([...bytes.keys()])
|
||||
for (const p of listHostPeers()) ids.add(p.serverId)
|
||||
const serverRows = db.select().from(servers).all()
|
||||
for (const p of await listHostPeers()) ids.add(p.serverId)
|
||||
const catalog = await getServerCatalog()
|
||||
const emptySeries = Array(60).fill(0) as number[]
|
||||
const exporters = serverRows
|
||||
const exporters = catalog.list
|
||||
.filter((s) => ids.has(s.id))
|
||||
.map((s) => {
|
||||
const ring = getRingMbps(s.id, "__all__")
|
||||
const total = bytes.get(s.id) ?? 0
|
||||
return {
|
||||
id: String(s.id),
|
||||
name: s.name || s.host,
|
||||
name: s.name,
|
||||
subtitle: s.host,
|
||||
site: s.site || "—",
|
||||
country: s.country || "UN",
|
||||
site: s.site,
|
||||
country: s.country,
|
||||
status: snapshotStatus(s.id),
|
||||
rxNow: ring.rxNow || (total * 8) / Math.max(60, minutes * 60) / 1_000_000,
|
||||
txNow: ring.txNow,
|
||||
@@ -513,16 +546,16 @@ export function listFlowExporters(minutes: number): FlowExportersDto {
|
||||
}
|
||||
}
|
||||
|
||||
export function listFlowClients(minutes: number): FlowClientsDto {
|
||||
const users = db.select().from(appUsers).all()
|
||||
const binds = db.select().from(userInterfaceBindings).all()
|
||||
export async function listFlowClients(minutes: number): Promise<{ clients: { id: string; name: string; subtitle: string; site: string; country: string; status: "online" | "offline" | "degraded"; rxNow: number; txNow: number; sessions: number; rxSeries: number[]; txSeries: number[]; bytes: number; }[]; }> {
|
||||
const users = await db.select().from(appUsers)
|
||||
const binds = await db.select().from(userInterfaceBindings)
|
||||
const byUser = new Map<string, typeof binds>()
|
||||
for (const b of binds) {
|
||||
const list = byUser.get(b.userId) ?? []
|
||||
list.push(b)
|
||||
byUser.set(b.userId, list)
|
||||
}
|
||||
const rows = listFlowRowsForWindow(minutes)
|
||||
const rows = await listFlowRowsForWindow(minutes)
|
||||
const emptySeries = Array(60).fill(0) as number[]
|
||||
const windowSec = Math.max(60, minutes * 60)
|
||||
const clients: FlowEntityCard[] = []
|
||||
@@ -584,9 +617,9 @@ export function safeBuildLiveFlowSample(q: Omit<FlowAnalyticsQuery, "minutes" |
|
||||
event: "sample" | "error"
|
||||
data: unknown
|
||||
} {
|
||||
return formatLiveSseFromBuilder(() => {
|
||||
return formatLiveSseFromBuilder(async () => {
|
||||
const skipHeavy = isFlowAnalyticsDegraded()
|
||||
return buildFlowAnalytics({ ...q, minutes: LIVE_ANALYTICS_MINUTES, skipHeavy })
|
||||
return await buildFlowAnalytics({ ...q, minutes: LIVE_ANALYTICS_MINUTES, skipHeavy })
|
||||
})
|
||||
}
|
||||
|
||||
@@ -620,7 +653,7 @@ function toBreakdown(
|
||||
}))
|
||||
}
|
||||
|
||||
export function getFlowMonthly(month: string, serverId?: number): FlowMonthlyDto {
|
||||
export async function getFlowMonthly(month: string, serverId?: number): Promise<FlowMonthlyDto> {
|
||||
const bounds = monthBounds(month)
|
||||
if (!bounds) {
|
||||
return { month, bytes: 0, countries: [], services: [], asns: [] }
|
||||
@@ -631,12 +664,12 @@ export function getFlowMonthly(month: string, serverId?: number): FlowMonthlyDto
|
||||
where += " AND server_id = ?"
|
||||
params.push(serverId)
|
||||
}
|
||||
const rows = sqliteDatabase.prepare(`
|
||||
const rows = await dbAll<{ dim: string; key: string; bytes: number; packets: number }>(`
|
||||
SELECT dim AS dim, key AS key, SUM(bytes) AS bytes, SUM(packets) AS packets
|
||||
FROM flow_daily_dims
|
||||
WHERE ${where}
|
||||
GROUP BY dim, key
|
||||
`).all(...params) as Array<{ dim: string; key: string; bytes: number; packets: number }>
|
||||
`, params)
|
||||
|
||||
const countries: Array<{ key: string; bytes: number; packets: number }> = []
|
||||
const services: Array<{ key: string; bytes: number; packets: number }> = []
|
||||
|
||||
@@ -97,7 +97,7 @@ async function fetchCatalog(): Promise<void> {
|
||||
if (inflight) return inflight
|
||||
inflight = (async () => {
|
||||
try {
|
||||
const row = db.select().from(evobgpSettings).limit(1).all()[0]
|
||||
const row = (await db.select().from(evobgpSettings).limit(1))[0]
|
||||
if (!row?.enabled) return
|
||||
const root = String(row.baseUrl ?? "").replace(/\/+$/, "")
|
||||
const token = String(row.apiKey ?? "").replace(/^Bearer\s+/i, "").trim()
|
||||
|
||||
@@ -8,7 +8,6 @@ export interface ExporterMapPayload {
|
||||
}
|
||||
|
||||
export interface CollectorStartPayload {
|
||||
dbPath: string
|
||||
listenHost: string
|
||||
listenPort: number
|
||||
topN: number
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import { createSocket, type Socket } from "node:dgram"
|
||||
import { parentPort } from "node:worker_threads"
|
||||
import { sqliteDatabase } from "../db/index.js"
|
||||
import type {
|
||||
CollectorStartPayload,
|
||||
ExporterMapPayload,
|
||||
@@ -9,7 +8,6 @@ import type {
|
||||
} from "./traffic-flow-collector-ipc.js"
|
||||
import {
|
||||
TICK_MS,
|
||||
attachEngineSqlite,
|
||||
configureEngine,
|
||||
flushPending,
|
||||
getEngineStats,
|
||||
@@ -23,7 +21,6 @@ let socket: Socket | null = null
|
||||
let flushTimer: ReturnType<typeof setInterval> | null = null
|
||||
let bound = false
|
||||
let address: string | null = null
|
||||
let attached = false
|
||||
|
||||
function send(msg: WorkerToMain): void {
|
||||
parentPort?.postMessage(msg)
|
||||
@@ -58,22 +55,14 @@ function applyExporterMap(payload: ExporterMapPayload): void {
|
||||
})
|
||||
}
|
||||
|
||||
function ensureSqlite(): void {
|
||||
if (attached) return
|
||||
attachEngineSqlite(sqliteDatabase)
|
||||
attached = true
|
||||
}
|
||||
|
||||
function stopListener(): void {
|
||||
if (flushTimer) {
|
||||
clearInterval(flushTimer)
|
||||
flushTimer = null
|
||||
}
|
||||
try {
|
||||
flushPending()
|
||||
} catch (e) {
|
||||
void flushPending().catch((e) => {
|
||||
setEngineError(e instanceof Error ? e.message : String(e))
|
||||
}
|
||||
})
|
||||
if (socket) {
|
||||
try { socket.close() } catch { /* ignore */ }
|
||||
socket = null
|
||||
@@ -84,7 +73,6 @@ function stopListener(): void {
|
||||
|
||||
function startListener(payload: CollectorStartPayload): void {
|
||||
stopListener()
|
||||
ensureSqlite()
|
||||
configureEngine({ topN: payload.topN, retentionHours: payload.retentionHours })
|
||||
applyExporterMap(payload.exporterMap)
|
||||
|
||||
@@ -116,12 +104,9 @@ function startListener(payload: CollectorStartPayload): void {
|
||||
})
|
||||
socket = sock
|
||||
flushTimer = setInterval(() => {
|
||||
try {
|
||||
flushPending()
|
||||
} catch (e) {
|
||||
void flushPending().catch((e) => {
|
||||
setEngineError(e instanceof Error ? e.message : String(e))
|
||||
}
|
||||
heartbeat()
|
||||
}).finally(() => heartbeat())
|
||||
}, TICK_MS)
|
||||
}
|
||||
|
||||
|
||||
@@ -1,16 +1,16 @@
|
||||
import type Database from "better-sqlite3"
|
||||
import { dbAll, dbQuery, pool } from "../db/index.js"
|
||||
import { ensurePartitionFor, specForParent } from "../db/partitions.js"
|
||||
import { normalizeParsedFlow, parseFlowPacket, protoName, type ParsedFlow, type ParsedFlowInput } from "./traffic-flow-parse.js"
|
||||
import { classifyFlowPlaneLite } from "./traffic-flow-planes.js"
|
||||
import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js"
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached } from "./traffic-flow-ripe.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
|
||||
type SqliteHandle = InstanceType<typeof Database>
|
||||
|
||||
export const TICK_MS = 2_000
|
||||
export const RING_LEN = 60
|
||||
export const MAX_PENDING = 50_000
|
||||
@@ -19,6 +19,17 @@ export const DAILY_RETENTION_DAYS = 396
|
||||
export const MINUTE_RETENTION_HOURS = 48
|
||||
|
||||
let pendingCap = MAX_PENDING
|
||||
const ensuredParts = new Set<string>()
|
||||
|
||||
async function ensureParentPartition(parent: string, ts: string): Promise<void> {
|
||||
const spec = specForParent(parent)
|
||||
if (!spec) return
|
||||
const iso = ts.length === 10 ? `${ts}T00:00:00Z` : ts
|
||||
const key = `${parent}:${iso.slice(0, 10)}`
|
||||
if (ensuredParts.has(key)) return
|
||||
await ensurePartitionFor(pool, parent, spec.kind, new Date(iso))
|
||||
ensuredParts.add(key)
|
||||
}
|
||||
|
||||
export interface PendingFlowRow {
|
||||
serverId: number
|
||||
@@ -75,7 +86,6 @@ export interface ExporterResolveCtx {
|
||||
hostIps: Map<string, number>
|
||||
}
|
||||
|
||||
let sqliteRef: SqliteHandle | null = null
|
||||
let topN = 200
|
||||
let retentionHours = 24
|
||||
|
||||
@@ -94,11 +104,27 @@ let dropped = 0
|
||||
let rowsStored = 0
|
||||
let lastFlushUsedTransaction = false
|
||||
let lastPruneAt = 0
|
||||
let lastPassiveCheckpointAt = 0
|
||||
let dataEpoch = 0
|
||||
let lastPersistedStats: {
|
||||
packetsReceived: number
|
||||
lastDatagramAt: string | null
|
||||
lastExporterIp: string | null
|
||||
lastError: string
|
||||
} | null = null
|
||||
let exporterCtx: ExporterResolveCtx | null = null
|
||||
|
||||
const PRUNE_MS = 5 * 60_000
|
||||
const LIVE_WINDOW_MS = 15 * 60_000
|
||||
|
||||
function bumpDataEpoch(): void {
|
||||
dataEpoch += 1
|
||||
}
|
||||
|
||||
export function flowDataEpoch(): number {
|
||||
return dataEpoch
|
||||
}
|
||||
|
||||
function nowIso(): string {
|
||||
return new Date().toISOString()
|
||||
}
|
||||
@@ -183,8 +209,8 @@ function bumpRollup(serverId: number, bucketAt: string, flow: ParsedFlow, bytes:
|
||||
acc.conversations += 1
|
||||
}
|
||||
|
||||
export function attachEngineSqlite(handle: SqliteHandle): void {
|
||||
sqliteRef = handle
|
||||
export function attachEngineSqlite(_handle?: unknown): void {
|
||||
/* engine uses pg pool via dbQuery/dbAll */
|
||||
}
|
||||
|
||||
export function setPendingCapForTests(n: number | null): void {
|
||||
@@ -234,6 +260,7 @@ export function getEngineStats(): EngineStats {
|
||||
}
|
||||
|
||||
export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): void {
|
||||
if (flows.length) bumpDataEpoch()
|
||||
const bucketAt = minuteBucketIso()
|
||||
const ripeMisses: string[] = []
|
||||
for (const raw of flows) {
|
||||
@@ -422,8 +449,17 @@ export function applyRingSnapshot(rows: Array<{ key: string; inBps: number[]; ou
|
||||
}
|
||||
}
|
||||
|
||||
function persistListenerStats(handle: SqliteHandle): void {
|
||||
handle.prepare(`
|
||||
async function persistListenerStats(): Promise<boolean> {
|
||||
if (
|
||||
lastPersistedStats
|
||||
&& lastPersistedStats.packetsReceived === packetsReceived
|
||||
&& lastPersistedStats.lastDatagramAt === lastDatagramAt
|
||||
&& lastPersistedStats.lastExporterIp === lastExporterIp
|
||||
&& lastPersistedStats.lastError === lastError
|
||||
) {
|
||||
return false
|
||||
}
|
||||
await dbQuery(`
|
||||
UPDATE traffic_flow_settings
|
||||
SET packets_received = @packetsReceived,
|
||||
last_datagram_at = @lastDatagramAt,
|
||||
@@ -431,108 +467,102 @@ function persistListenerStats(handle: SqliteHandle): void {
|
||||
last_error = @lastError,
|
||||
updated_at = @updatedAt
|
||||
WHERE id = 1
|
||||
`).run({
|
||||
`, {
|
||||
packetsReceived,
|
||||
lastDatagramAt,
|
||||
lastExporterIp,
|
||||
lastError,
|
||||
updatedAt: nowIso(),
|
||||
})
|
||||
lastPersistedStats = {
|
||||
packetsReceived,
|
||||
lastDatagramAt,
|
||||
lastExporterIp,
|
||||
lastError,
|
||||
}
|
||||
invalidateTrafficFlowSettingsCache()
|
||||
return true
|
||||
}
|
||||
|
||||
function upsertMinuteAndDaily(handle: SqliteHandle): void {
|
||||
const upsertMinute = handle.prepare(`
|
||||
INSERT INTO flow_minute_stats (
|
||||
server_id, bucket_at, bytes, packets, unique_src, unique_dst, conversations
|
||||
) VALUES (
|
||||
@serverId, @bucketAt, @bytes, @packets, @uniqueSrc, @uniqueDst, @conversations
|
||||
)
|
||||
ON CONFLICT(server_id, bucket_at) DO UPDATE SET
|
||||
bytes = bytes + excluded.bytes,
|
||||
packets = packets + excluded.packets,
|
||||
unique_src = MAX(unique_src, excluded.unique_src),
|
||||
unique_dst = MAX(unique_dst, excluded.unique_dst),
|
||||
conversations = conversations + excluded.conversations
|
||||
`)
|
||||
const upsertDim = handle.prepare(`
|
||||
INSERT INTO flow_minute_dims (server_id, bucket_at, dim, key, bytes, packets)
|
||||
VALUES (@serverId, @bucketAt, @dim, @key, @bytes, @packets)
|
||||
ON CONFLICT(server_id, bucket_at, dim, key) DO UPDATE SET
|
||||
bytes = bytes + excluded.bytes,
|
||||
packets = packets + excluded.packets
|
||||
`)
|
||||
const upsertDaily = handle.prepare(`
|
||||
INSERT INTO flow_daily_dims (server_id, day, dim, key, bytes, packets)
|
||||
VALUES (@serverId, @day, @dim, @key, @bytes, @packets)
|
||||
ON CONFLICT(server_id, day, dim, key) DO UPDATE SET
|
||||
bytes = bytes + excluded.bytes,
|
||||
packets = packets + excluded.packets
|
||||
`)
|
||||
|
||||
const tx = handle.transaction(() => {
|
||||
for (const [k, acc] of minuteRollup) {
|
||||
const [serverIdRaw, bucketAt] = k.split("\0")
|
||||
upsertMinute.run({
|
||||
async function upsertMinuteAndDaily(): Promise<void> {
|
||||
for (const [k, acc] of minuteRollup) {
|
||||
const [serverIdRaw, bucketAt] = k.split("\0")
|
||||
await ensureParentPartition("flow_minute_stats", bucketAt ?? "")
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_minute_stats (
|
||||
server_id, bucket_at, bytes, packets, unique_src, unique_dst, conversations
|
||||
) VALUES (
|
||||
@serverId, @bucketAt, @bytes, @packets, @uniqueSrc, @uniqueDst, @conversations
|
||||
)
|
||||
ON CONFLICT(server_id, bucket_at) DO UPDATE SET
|
||||
bytes = flow_minute_stats.bytes + excluded.bytes,
|
||||
packets = flow_minute_stats.packets + excluded.packets,
|
||||
unique_src = GREATEST(flow_minute_stats.unique_src, excluded.unique_src),
|
||||
unique_dst = GREATEST(flow_minute_stats.unique_dst, excluded.unique_dst),
|
||||
conversations = flow_minute_stats.conversations + excluded.conversations
|
||||
`, {
|
||||
serverId: Number(serverIdRaw),
|
||||
bucketAt,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
uniqueSrc: acc.srcs.size,
|
||||
uniqueDst: acc.dsts.size,
|
||||
conversations: acc.conversations,
|
||||
})
|
||||
}
|
||||
for (const [k, acc] of minuteDims) {
|
||||
const [serverIdRaw, bucketAt, dim, key] = k.split("\0")
|
||||
await ensureParentPartition("flow_minute_dims", bucketAt ?? "")
|
||||
await ensureParentPartition("flow_daily_dims", dayKey(bucketAt ?? ""))
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_minute_dims (server_id, bucket_at, dim, key, bytes, packets)
|
||||
VALUES (@serverId, @bucketAt, @dim, @key, @bytes, @packets)
|
||||
ON CONFLICT(server_id, bucket_at, dim, key) DO UPDATE SET
|
||||
bytes = flow_minute_dims.bytes + excluded.bytes,
|
||||
packets = flow_minute_dims.packets + excluded.packets
|
||||
`, {
|
||||
serverId: Number(serverIdRaw),
|
||||
bucketAt,
|
||||
dim,
|
||||
key,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
})
|
||||
if (dim === "country" || dim === "service" || dim === "asn") {
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_dims (server_id, day, dim, key, bytes, packets)
|
||||
VALUES (@serverId, @day, @dim, @key, @bytes, @packets)
|
||||
ON CONFLICT(server_id, day, dim, key) DO UPDATE SET
|
||||
bytes = flow_daily_dims.bytes + excluded.bytes,
|
||||
packets = flow_daily_dims.packets + excluded.packets
|
||||
`, {
|
||||
serverId: Number(serverIdRaw),
|
||||
bucketAt,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
uniqueSrc: acc.srcs.size,
|
||||
uniqueDst: acc.dsts.size,
|
||||
conversations: acc.conversations,
|
||||
})
|
||||
}
|
||||
for (const [k, acc] of minuteDims) {
|
||||
const [serverIdRaw, bucketAt, dim, key] = k.split("\0")
|
||||
upsertDim.run({
|
||||
serverId: Number(serverIdRaw),
|
||||
bucketAt,
|
||||
day: dayKey(bucketAt ?? ""),
|
||||
dim,
|
||||
key,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
})
|
||||
if (dim === "country" || dim === "service" || dim === "asn") {
|
||||
upsertDaily.run({
|
||||
serverId: Number(serverIdRaw),
|
||||
day: dayKey(bucketAt ?? ""),
|
||||
dim,
|
||||
key,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
})
|
||||
}
|
||||
}
|
||||
})
|
||||
tx()
|
||||
}
|
||||
minuteRollup.clear()
|
||||
minuteDims.clear()
|
||||
}
|
||||
|
||||
function capDailyAsn(handle: SqliteHandle): void {
|
||||
async function capDailyAsn(): Promise<void> {
|
||||
const today = nowIso().slice(0, 10)
|
||||
const rows = handle.prepare(`
|
||||
SELECT server_id AS serverId, key, bytes, packets
|
||||
const rows = await dbAll<{ serverId: number; key: string; bytes: number; packets: number }>(`
|
||||
SELECT server_id AS "serverId", key, bytes, packets
|
||||
FROM flow_daily_dims
|
||||
WHERE day = ? AND dim = 'asn'
|
||||
ORDER BY server_id, bytes DESC
|
||||
`).all(today) as Array<{ serverId: number; key: string; bytes: number; packets: number }>
|
||||
`, [today])
|
||||
const byServer = new Map<number, typeof rows>()
|
||||
for (const row of rows) {
|
||||
const list = byServer.get(row.serverId) ?? []
|
||||
list.push(row)
|
||||
byServer.set(row.serverId, list)
|
||||
}
|
||||
const del = handle.prepare(`
|
||||
DELETE FROM flow_daily_dims WHERE server_id = ? AND day = ? AND dim = 'asn' AND key = ?
|
||||
`)
|
||||
const upsertOther = handle.prepare(`
|
||||
INSERT INTO flow_daily_dims (server_id, day, dim, key, bytes, packets)
|
||||
VALUES (?, ?, 'asn', 'other', ?, ?)
|
||||
ON CONFLICT(server_id, day, dim, key) DO UPDATE SET
|
||||
bytes = bytes + excluded.bytes,
|
||||
packets = packets + excluded.packets
|
||||
`)
|
||||
for (const [serverId, list] of byServer) {
|
||||
if (list.length <= DAILY_ASN_TOP) continue
|
||||
let otherBytes = 0
|
||||
@@ -541,51 +571,60 @@ function capDailyAsn(handle: SqliteHandle): void {
|
||||
if (row.key === "other") continue
|
||||
otherBytes += row.bytes
|
||||
otherPackets += row.packets
|
||||
del.run(serverId, today, row.key)
|
||||
await dbQuery(
|
||||
`DELETE FROM flow_daily_dims WHERE server_id = ? AND day = ? AND dim = 'asn' AND key = ?`,
|
||||
[serverId, today, row.key],
|
||||
)
|
||||
}
|
||||
if (otherBytes > 0) {
|
||||
await ensureParentPartition("flow_daily_dims", today)
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_dims (server_id, day, dim, key, bytes, packets)
|
||||
VALUES (?, ?, 'asn', 'other', ?, ?)
|
||||
ON CONFLICT(server_id, day, dim, key) DO UPDATE SET
|
||||
bytes = flow_daily_dims.bytes + excluded.bytes,
|
||||
packets = flow_daily_dims.packets + excluded.packets
|
||||
`, [serverId, today, otherBytes, otherPackets])
|
||||
}
|
||||
if (otherBytes > 0) upsertOther.run(serverId, today, otherBytes, otherPackets)
|
||||
}
|
||||
}
|
||||
|
||||
function pruneStored(handle: SqliteHandle): void {
|
||||
async function pruneStored(): Promise<void> {
|
||||
const now = Date.now()
|
||||
if (now - lastPruneAt < PRUNE_MS) return
|
||||
lastPruneAt = now
|
||||
const flowCutoff = new Date(now - retentionHours * 3600_000).toISOString()
|
||||
const minuteCutoff = new Date(now - MINUTE_RETENTION_HOURS * 3600_000).toISOString()
|
||||
const dailyCutoff = new Date(now - DAILY_RETENTION_DAYS * 86400_000).toISOString().slice(0, 10)
|
||||
handle.prepare(`DELETE FROM flow_buckets WHERE bucket_at < ?`).run(flowCutoff)
|
||||
handle.prepare(`DELETE FROM flow_minute_stats WHERE bucket_at < ?`).run(minuteCutoff)
|
||||
handle.prepare(`DELETE FROM flow_minute_dims WHERE bucket_at < ?`).run(minuteCutoff)
|
||||
handle.prepare(`DELETE FROM flow_daily_dims WHERE day < ?`).run(dailyCutoff)
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE bucket_at < ?`, [flowCutoff])
|
||||
await dbQuery(`DELETE FROM flow_minute_stats WHERE bucket_at < ?`, [minuteCutoff])
|
||||
await dbQuery(`DELETE FROM flow_minute_dims WHERE bucket_at < ?`, [minuteCutoff])
|
||||
await dbQuery(`DELETE FROM flow_daily_dims WHERE day < ?`, [dailyCutoff])
|
||||
|
||||
const keep = Math.max(20, topN)
|
||||
try {
|
||||
handle.prepare(`
|
||||
DELETE FROM flow_buckets WHERE id IN (
|
||||
SELECT id FROM (
|
||||
SELECT id, ROW_NUMBER() OVER (
|
||||
PARTITION BY server_id, bucket_at ORDER BY bytes DESC
|
||||
) AS rn
|
||||
FROM flow_buckets
|
||||
) ranked WHERE rn > ?
|
||||
)
|
||||
`).run(keep)
|
||||
} catch {
|
||||
const buckets = handle.prepare(`
|
||||
SELECT DISTINCT server_id AS serverId, bucket_at AS bucketAt FROM flow_buckets
|
||||
`).all() as Array<{ serverId: number; bucketAt: string }>
|
||||
for (const b of buckets) {
|
||||
const rows = handle.prepare(`
|
||||
SELECT id, bytes FROM flow_buckets
|
||||
WHERE server_id = ? AND bucket_at = ?
|
||||
ORDER BY bytes DESC
|
||||
`).all(b.serverId, b.bucketAt) as Array<{ id: number; bytes: number }>
|
||||
for (const extra of rows.slice(keep)) {
|
||||
handle.prepare(`DELETE FROM flow_buckets WHERE id = ?`).run(extra.id)
|
||||
}
|
||||
}
|
||||
}
|
||||
await dbQuery(`
|
||||
DELETE FROM flow_buckets fb
|
||||
USING (
|
||||
SELECT server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface
|
||||
FROM (
|
||||
SELECT server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface,
|
||||
ROW_NUMBER() OVER (
|
||||
PARTITION BY server_id, bucket_at ORDER BY bytes DESC
|
||||
) AS rn
|
||||
FROM flow_buckets
|
||||
) ranked
|
||||
WHERE rn > ?
|
||||
) drop_rows
|
||||
WHERE fb.server_id = drop_rows.server_id
|
||||
AND fb.bucket_at = drop_rows.bucket_at
|
||||
AND fb.src = drop_rows.src
|
||||
AND fb.dst = drop_rows.dst
|
||||
AND fb.proto = drop_rows.proto
|
||||
AND fb.src_port = drop_rows.src_port
|
||||
AND fb.dst_port = drop_rows.dst_port
|
||||
AND fb.in_iface = drop_rows.in_iface
|
||||
`, [keep])
|
||||
await pruneRipeSqlite(now)
|
||||
}
|
||||
|
||||
function topNPending(rows: PendingFlowRow[]): PendingFlowRow[] {
|
||||
@@ -605,17 +644,12 @@ function topNPending(rows: PendingFlowRow[]): PendingFlowRow[] {
|
||||
return out
|
||||
}
|
||||
|
||||
export function flushPending(): void {
|
||||
export async function flushPending(): Promise<void> {
|
||||
pruneRecent()
|
||||
rollFlowRings()
|
||||
const handle = sqliteRef
|
||||
if (!handle) {
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
persistListenerStats(handle)
|
||||
await persistListenerStats()
|
||||
if (pending.size === 0 && minuteRollup.size === 0 && minuteDims.size === 0) {
|
||||
pruneStored(handle)
|
||||
await pruneStored()
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
@@ -623,7 +657,7 @@ export function flushPending(): void {
|
||||
pending.clear()
|
||||
for (const row of rows) mergeInto(recent, row)
|
||||
|
||||
const upsertFlow = handle.prepare(`
|
||||
const upsertSql = `
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms
|
||||
) VALUES (
|
||||
@@ -631,44 +665,44 @@ export function flushPending(): void {
|
||||
)
|
||||
ON CONFLICT(server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
DO UPDATE SET
|
||||
bytes = bytes + excluded.bytes,
|
||||
packets = packets + excluded.packets,
|
||||
out_iface = CASE WHEN excluded.out_iface != '' THEN excluded.out_iface ELSE out_iface END,
|
||||
next_hop = CASE WHEN excluded.next_hop != '' THEN excluded.next_hop ELSE next_hop END,
|
||||
bytes = flow_buckets.bytes + excluded.bytes,
|
||||
packets = flow_buckets.packets + excluded.packets,
|
||||
out_iface = CASE WHEN excluded.out_iface != '' THEN excluded.out_iface ELSE flow_buckets.out_iface END,
|
||||
next_hop = CASE WHEN excluded.next_hop != '' THEN excluded.next_hop ELSE flow_buckets.next_hop END,
|
||||
flow_start_ms = CASE
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_start_ms = 0 OR excluded.flow_start_ms < flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_start_ms END,
|
||||
flow_end_ms = MAX(flow_end_ms, excluded.flow_end_ms)
|
||||
`)
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_buckets.flow_start_ms = 0 OR excluded.flow_start_ms < flow_buckets.flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_buckets.flow_start_ms END,
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms)
|
||||
`
|
||||
lastFlushUsedTransaction = false
|
||||
try {
|
||||
const tx = handle.transaction((batch: PendingFlowRow[]) => {
|
||||
for (const r of batch) {
|
||||
upsertFlow.run({
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: r.nextHop,
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
})
|
||||
}
|
||||
})
|
||||
tx(rows)
|
||||
for (const r of rows) {
|
||||
await ensureParentPartition("flow_buckets", r.bucketAt)
|
||||
await dbQuery(upsertSql, {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: r.nextHop,
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
})
|
||||
}
|
||||
lastFlushUsedTransaction = true
|
||||
rowsStored += rows.length
|
||||
bumpDataEpoch()
|
||||
} catch {
|
||||
for (const r of rows) {
|
||||
try {
|
||||
upsertFlow.run({
|
||||
await ensureParentPartition("flow_buckets", r.bucketAt)
|
||||
await dbQuery(upsertSql, {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
@@ -691,29 +725,24 @@ export function flushPending(): void {
|
||||
}
|
||||
}
|
||||
try {
|
||||
upsertMinuteAndDaily(handle)
|
||||
capDailyAsn(handle)
|
||||
await upsertMinuteAndDaily()
|
||||
await capDailyAsn()
|
||||
} catch {
|
||||
/* rollup best-effort */
|
||||
}
|
||||
pruneStored(handle)
|
||||
try {
|
||||
handle.pragma("wal_checkpoint(TRUNCATE)")
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
await pruneStored()
|
||||
}
|
||||
|
||||
export function lastFlushUsedTransactionForTests(): boolean {
|
||||
return lastFlushUsedTransaction
|
||||
}
|
||||
|
||||
export function flushPendingForTests(): void {
|
||||
flushPending()
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending()
|
||||
}
|
||||
|
||||
export function onEngineTick(): void {
|
||||
flushPending()
|
||||
void flushPending()
|
||||
}
|
||||
|
||||
export function ingestParsedFlowsForServerForTests(serverId: number, flows: ParsedFlowInput[]): void {
|
||||
@@ -736,6 +765,9 @@ export function resetEngineForTests(): void {
|
||||
rowsStored = 0
|
||||
lastFlushUsedTransaction = false
|
||||
lastPruneAt = 0
|
||||
lastPassiveCheckpointAt = Date.now()
|
||||
lastPersistedStats = null
|
||||
bumpDataEpoch()
|
||||
pendingCap = MAX_PENDING
|
||||
}
|
||||
|
||||
|
||||
@@ -1,23 +0,0 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { SQLITE_BUSY_TIMEOUT_MS, sqliteDatabase } from "../db/index.js"
|
||||
import {
|
||||
MAX_FLOW_LIVE_SUBSCRIBERS,
|
||||
resetFlowLiveSlotsForTests,
|
||||
tryAcquireFlowLiveSlot,
|
||||
releaseFlowLiveSlot,
|
||||
} from "../routes/traffic-flow.js"
|
||||
|
||||
const busy = sqliteDatabase.pragma("busy_timeout") as Array<{ busy_timeout: number }>
|
||||
const busyValue = Array.isArray(busy) ? Number(Object.values(busy[0] ?? {})[0]) : Number(busy)
|
||||
assert.equal(busyValue, SQLITE_BUSY_TIMEOUT_MS)
|
||||
|
||||
resetFlowLiveSlotsForTests()
|
||||
for (let i = 0; i < MAX_FLOW_LIVE_SUBSCRIBERS; i++) {
|
||||
assert.equal(tryAcquireFlowLiveSlot(), true)
|
||||
}
|
||||
assert.equal(tryAcquireFlowLiveSlot(), false)
|
||||
releaseFlowLiveSlot()
|
||||
assert.equal(tryAcquireFlowLiveSlot(), true)
|
||||
resetFlowLiveSlotsForTests()
|
||||
|
||||
console.log("traffic-flow-hardening.test.ts: ok")
|
||||
@@ -4,9 +4,9 @@ import type { TrafficFlowHostFile } from "@mmapp/contracts/traffic-flow"
|
||||
|
||||
const COMPOSE_DIR = "/opt/cdn-mm"
|
||||
|
||||
export function buildHostWgQuickConf(): string {
|
||||
const row = getTrafficFlowSettingsRow()
|
||||
const peers = listHostPeers()
|
||||
export async function buildHostWgQuickConf(): Promise<string> {
|
||||
const row = await getTrafficFlowSettingsRow()
|
||||
const peers = await listHostPeers()
|
||||
return generateNativeConf({
|
||||
name: "wg-flow",
|
||||
mtu: 1420,
|
||||
@@ -23,8 +23,8 @@ export function buildHostWgQuickConf(): string {
|
||||
})
|
||||
}
|
||||
|
||||
export function buildHostComposeOverride(): string {
|
||||
const row = getTrafficFlowSettingsRow()
|
||||
export async function buildHostComposeOverride(): Promise<string> {
|
||||
const row = await getTrafficFlowSettingsRow()
|
||||
return [
|
||||
"# Docker Compose merge для /opt/cdn-mm",
|
||||
"# Не править docker-compose.yml. Traefik не трогать.",
|
||||
@@ -42,10 +42,10 @@ export function buildHostComposeOverride(): string {
|
||||
].join("\n")
|
||||
}
|
||||
|
||||
export function buildHostLinuxInstallSh(): string {
|
||||
const row = getTrafficFlowSettingsRow()
|
||||
const conf = buildHostWgQuickConf().replace(/\s+$/, "") + "\n"
|
||||
const override = buildHostComposeOverride()
|
||||
export async function buildHostLinuxInstallSh(): Promise<string> {
|
||||
const row = await getTrafficFlowSettingsRow()
|
||||
const conf = (await buildHostWgQuickConf()).replace(/\s+$/, "") + "\n"
|
||||
const override = await buildHostComposeOverride()
|
||||
const collector = row.collectorIp
|
||||
const flowPort = row.flowListenPort
|
||||
|
||||
@@ -104,10 +104,10 @@ echo "Готово. Traefik не трогали. UDP \${FLOW_PORT} только
|
||||
`
|
||||
}
|
||||
|
||||
export function listTrafficFlowHostFiles(): TrafficFlowHostFile[] {
|
||||
export async function listTrafficFlowHostFiles(): Promise<TrafficFlowHostFile[]> {
|
||||
return [
|
||||
{ id: "linux", label: "Linux", filename: "install-wg-flow.sh", code: buildHostLinuxInstallSh() },
|
||||
{ id: "wg-quick", label: "wg-flow.conf", filename: "wg-flow.conf", code: buildHostWgQuickConf() },
|
||||
{ id: "compose", label: "compose", filename: "docker-compose.override.yml", code: buildHostComposeOverride() },
|
||||
{ id: "linux", label: "Linux", filename: "install-wg-flow.sh", code: await buildHostLinuxInstallSh() },
|
||||
{ id: "wg-quick", label: "wg-flow.conf", filename: "wg-flow.conf", code: await buildHostWgQuickConf() },
|
||||
{ id: "compose", label: "compose", filename: "docker-compose.override.yml", code: await buildHostComposeOverride() },
|
||||
]
|
||||
}
|
||||
|
||||
@@ -28,7 +28,7 @@ async function refreshServerIfacesInner(serverId: number, force = false): Promis
|
||||
if (!force && !shouldRefreshIfaces(serverId)) return
|
||||
inflight.add(serverId)
|
||||
try {
|
||||
const row = db.select().from(servers).where(eq(servers.id, serverId)).limit(1).all()[0]
|
||||
const row = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!row) return
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const ifaces = await client.get<RosIfaceIndexRow[]>("/interface")
|
||||
|
||||
@@ -22,7 +22,8 @@ import {
|
||||
simulateWorkerExitForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { configureEngine, droppedForTests, pendingSizeForTests } from "./traffic-flow-engine.js"
|
||||
import { sqliteDatabase } from "../db/index.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
resetFlowRingsForTests()
|
||||
@@ -81,16 +82,23 @@ const talkers = Array.from({ length: 25 }, (_, i) => ({
|
||||
inIface: "2",
|
||||
outIface: "",
|
||||
}))
|
||||
ingestParsedFlowsForServerForTests(9, talkers)
|
||||
flushPendingForTests()
|
||||
const stored = sqliteDatabase.prepare(`
|
||||
SELECT COUNT(*) AS n FROM flow_buckets WHERE server_id = 9
|
||||
`).get() as { n: number }
|
||||
assert.ok(stored.n <= 20, `expected topN cap, got ${stored.n}`)
|
||||
sqliteDatabase.prepare(`DELETE FROM flow_buckets WHERE server_id = 9`).run()
|
||||
sqliteDatabase.prepare(`DELETE FROM flow_minute_stats WHERE server_id = 9`).run()
|
||||
sqliteDatabase.prepare(`DELETE FROM flow_minute_dims WHERE server_id = 9`).run()
|
||||
sqliteDatabase.prepare(`DELETE FROM flow_daily_dims WHERE server_id = 9`).run()
|
||||
if (await withPgOrSkip()) {
|
||||
await dbQuery(`
|
||||
INSERT INTO servers (id, name, host) VALUES (91009, 'flow-ingest-test', '127.0.0.1')
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
ingestParsedFlowsForServerForTests(91009, talkers)
|
||||
await flushPendingForTests()
|
||||
const stored = await dbQuery<{ n: number }>(
|
||||
`SELECT COUNT(*)::int AS n FROM flow_buckets WHERE server_id = 91009`,
|
||||
)
|
||||
assert.ok((stored.rows[0]?.n ?? 99) <= 20, `expected topN cap, got ${stored.rows[0]?.n}`)
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = 91009`)
|
||||
await dbQuery(`DELETE FROM flow_minute_stats WHERE server_id = 91009`)
|
||||
await dbQuery(`DELETE FROM flow_minute_dims WHERE server_id = 91009`)
|
||||
await dbQuery(`DELETE FROM flow_daily_dims WHERE server_id = 91009`)
|
||||
await dbQuery(`DELETE FROM servers WHERE id = 91009`)
|
||||
}
|
||||
|
||||
applyHeartbeatForTests({
|
||||
bound: true,
|
||||
@@ -106,7 +114,7 @@ applyHeartbeatForTests({
|
||||
rings: [],
|
||||
})
|
||||
assert.equal(getFlowListenerState().bound, true)
|
||||
assert.equal(getFlowRuntimeCounters().packetsReceived, 42)
|
||||
assert.equal((await getFlowRuntimeCounters()).packetsReceived, 42)
|
||||
assert.equal(getFlowWorkerHealth().alive, false)
|
||||
setWantListenForTests(true)
|
||||
assert.equal(simulateWorkerExitForTests(), 1)
|
||||
|
||||
@@ -1,15 +1,12 @@
|
||||
import { Worker } from "node:worker_threads"
|
||||
import { existsSync, statSync } from "node:fs"
|
||||
import path from "node:path"
|
||||
import { gte, sql } from "drizzle-orm"
|
||||
import { beginSqliteExclusiveOp, db, endSqliteExclusiveOp, sqliteDatabase } from "../db/index.js"
|
||||
import { env } from "../config.js"
|
||||
import { db, dbGet, dbQuery, pool, withAdvisoryLock } from "../db/index.js"
|
||||
import { dropExpiredPartitions } from "../db/partitions.js"
|
||||
import { flowBuckets, servers } from "../db/schema.js"
|
||||
import type { FlowPurgeDto, FlowStatsDto, FlowTalkerDto } from "@mmapp/contracts/traffic-flow"
|
||||
import { protoName, type ParsedFlowInput } from "./traffic-flow-parse.js"
|
||||
import type { CollectorHeartbeat, ExporterMapPayload, MainToWorker, WorkerToMain } from "./traffic-flow-collector-ipc.js"
|
||||
import {
|
||||
attachEngineSqlite,
|
||||
applyRingSnapshot,
|
||||
configureEngine,
|
||||
flushPending,
|
||||
@@ -33,6 +30,7 @@ import {
|
||||
} from "./traffic-flow-settings.js"
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { getServerCatalog } from "./traffic-flow-topology.js"
|
||||
|
||||
export type { PendingFlowRow }
|
||||
|
||||
@@ -56,8 +54,6 @@ let lastHeartbeat: CollectorHeartbeat | null = null
|
||||
let state: FlowListenerState = { bound: false, address: null }
|
||||
let wantListen = false
|
||||
|
||||
attachEngineSqlite(sqliteDatabase)
|
||||
|
||||
function workerFileUrl(): URL {
|
||||
const ts = import.meta.url.includes(".ts")
|
||||
return new URL(
|
||||
@@ -66,13 +62,13 @@ function workerFileUrl(): URL {
|
||||
)
|
||||
}
|
||||
|
||||
export function buildExporterMapPayload(): ExporterMapPayload {
|
||||
const settings = getTrafficFlowSettingsRow()
|
||||
const rows = db.select({
|
||||
export async function buildExporterMapPayload(): Promise<ExporterMapPayload> {
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
const rows = await db.select({
|
||||
id: servers.id,
|
||||
host: servers.host,
|
||||
mgmtTunnelIp: servers.mgmtTunnelIp,
|
||||
}).from(servers).all()
|
||||
}).from(servers)
|
||||
const byTunnelIp: Array<[string, number]> = []
|
||||
const hostIps: Array<[string, number]> = []
|
||||
for (const row of rows) {
|
||||
@@ -82,13 +78,13 @@ export function buildExporterMapPayload(): ExporterMapPayload {
|
||||
return {
|
||||
overlayPrefix: settings.prefix,
|
||||
byTunnelIp,
|
||||
peers: listHostPeers(),
|
||||
peers: await listHostPeers(),
|
||||
hostIps,
|
||||
}
|
||||
}
|
||||
|
||||
function applyExporterCtxFromDb(): void {
|
||||
const payload = buildExporterMapPayload()
|
||||
async function applyExporterCtxFromDb(): Promise<void> {
|
||||
const payload = await buildExporterMapPayload()
|
||||
setExporterResolveCtx({
|
||||
overlayPrefix: payload.overlayPrefix,
|
||||
byTunnelIp: new Map(payload.byTunnelIp),
|
||||
@@ -116,11 +112,11 @@ function handleWorkerMessage(msg: WorkerToMain): void {
|
||||
}
|
||||
}
|
||||
|
||||
function spawnWorker(): void {
|
||||
async function spawnWorker(): Promise<void> {
|
||||
stopWorkerProcess()
|
||||
const settings = getTrafficFlowSettingsRow()
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
configureEngine({ topN: settings.topN, retentionHours: settings.retentionHours })
|
||||
applyExporterCtxFromDb()
|
||||
await applyExporterCtxFromDb()
|
||||
const w = new Worker(workerFileUrl(), { execArgv: process.execArgv })
|
||||
w.on("message", (msg: WorkerToMain) => handleWorkerMessage(msg))
|
||||
w.on("error", (err) => {
|
||||
@@ -148,7 +144,7 @@ function spawnWorker(): void {
|
||||
const delay = Math.min(30_000, 1000 * 2 ** restartAttempts)
|
||||
restartAttempts += 1
|
||||
restartTimer = setTimeout(() => {
|
||||
if (wantListen) spawnWorker()
|
||||
if (wantListen) void spawnWorker()
|
||||
}, delay)
|
||||
void code
|
||||
})
|
||||
@@ -157,12 +153,11 @@ function spawnWorker(): void {
|
||||
postToWorker({
|
||||
type: "start",
|
||||
payload: {
|
||||
dbPath: env.DATABASE_PATH,
|
||||
listenHost: host,
|
||||
listenPort: settings.flowListenPort,
|
||||
topN: settings.topN,
|
||||
retentionHours: settings.retentionHours,
|
||||
exporterMap: buildExporterMapPayload(),
|
||||
exporterMap: await buildExporterMapPayload(),
|
||||
},
|
||||
})
|
||||
}
|
||||
@@ -183,8 +178,24 @@ function stopWorkerProcess(): void {
|
||||
}
|
||||
}
|
||||
|
||||
async function stopWorkerProcessAsync(): Promise<void> {
|
||||
if (restartTimer) {
|
||||
clearTimeout(restartTimer)
|
||||
restartTimer = null
|
||||
}
|
||||
if (!worker) return
|
||||
const current = worker
|
||||
worker = null
|
||||
try {
|
||||
current.postMessage({ type: "stop" })
|
||||
await current.terminate()
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
|
||||
export function reattachFlowSqlite(): void {
|
||||
attachEngineSqlite(sqliteDatabase)
|
||||
/* no-op: engine uses PostgreSQL pool */
|
||||
}
|
||||
|
||||
export function applyHeartbeatForTests(payload: CollectorHeartbeat): void {
|
||||
@@ -220,44 +231,49 @@ export function getFlowWorkerHealth(): FlowWorkerHealth {
|
||||
}
|
||||
}
|
||||
|
||||
export function getFlowRuntimeCounters() {
|
||||
const settings = getTrafficFlowSettingsRow()
|
||||
export async function getFlowRuntimeCounters() {
|
||||
const hb = lastHeartbeat
|
||||
if (hb) {
|
||||
return {
|
||||
packetsReceived: hb.packetsReceived,
|
||||
lastExporterIp: hb.lastExporterIp ?? null,
|
||||
lastError: hb.lastError || null,
|
||||
lastDatagramAt: hb.lastDatagramAt ?? null,
|
||||
dropped: hb.dropped ?? 0,
|
||||
}
|
||||
}
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
return {
|
||||
packetsReceived: hb?.packetsReceived ?? settings.packetsReceived,
|
||||
lastExporterIp: hb?.lastExporterIp ?? settings.lastExporterIp ?? null,
|
||||
lastError: (hb?.lastError ?? settings.lastError) || null,
|
||||
lastDatagramAt: hb?.lastDatagramAt ?? settings.lastDatagramAt ?? null,
|
||||
dropped: hb?.dropped ?? 0,
|
||||
packetsReceived: settings.packetsReceived,
|
||||
lastExporterIp: settings.lastExporterIp ?? null,
|
||||
lastError: settings.lastError || null,
|
||||
lastDatagramAt: settings.lastDatagramAt ?? null,
|
||||
dropped: 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function startTrafficFlowListener() {
|
||||
export async function startTrafficFlowListener() {
|
||||
stopTrafficFlowListener()
|
||||
const settings = getTrafficFlowSettingsRow()
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
if (!settings.enabled) {
|
||||
wantListen = false
|
||||
state = { bound: false, address: null }
|
||||
return
|
||||
}
|
||||
wantListen = true
|
||||
spawnWorker()
|
||||
await spawnWorker()
|
||||
}
|
||||
|
||||
export function stopTrafficFlowListener() {
|
||||
wantListen = false
|
||||
stopWorkerProcess()
|
||||
try {
|
||||
flushPending()
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
void flushPending().catch(() => { /* ignore */ })
|
||||
state = { bound: false, address: null }
|
||||
}
|
||||
|
||||
export function refreshFlowExporterMap(): void {
|
||||
applyExporterCtxFromDb()
|
||||
postToWorker({ type: "updateExporterMap", payload: buildExporterMapPayload() })
|
||||
export async function refreshFlowExporterMap(): Promise<void> {
|
||||
await applyExporterCtxFromDb()
|
||||
postToWorker({ type: "updateExporterMap", payload: await buildExporterMapPayload() })
|
||||
}
|
||||
|
||||
export function getRingMbps(serverId: number, iface = "__all__") {
|
||||
@@ -279,21 +295,20 @@ function mergeInto(map: Map<string, PendingFlowRow>, row: PendingFlowRow): void
|
||||
map.set(key, { ...row })
|
||||
}
|
||||
|
||||
export function listLiveFlowRows(sinceIso: string): PendingFlowRow[] {
|
||||
export async function listLiveFlowRows(sinceIso: string): Promise<PendingFlowRow[]> {
|
||||
if (worker && lastHeartbeat?.workerAlive) {
|
||||
return listStoredFlowRows(sinceIso)
|
||||
return await listStoredFlowRows(sinceIso)
|
||||
}
|
||||
return engineListLive(sinceIso)
|
||||
}
|
||||
|
||||
export function listStoredFlowRows(sinceIso: string): PendingFlowRow[] {
|
||||
const settings = getTrafficFlowSettingsRow()
|
||||
export async function listStoredFlowRows(sinceIso: string): Promise<PendingFlowRow[]> {
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
const cap = Math.max(20, settings.topN) * 60
|
||||
const stored = db.select().from(flowBuckets)
|
||||
const stored = await db.select().from(flowBuckets)
|
||||
.where(gte(flowBuckets.bucketAt, sinceIso))
|
||||
.orderBy(sql`${flowBuckets.bytes} DESC`)
|
||||
.limit(cap)
|
||||
.all()
|
||||
const merged = new Map<string, PendingFlowRow>()
|
||||
for (const r of stored) {
|
||||
mergeInto(merged, {
|
||||
@@ -322,18 +337,18 @@ export function listStoredFlowRows(sinceIso: string): PendingFlowRow[] {
|
||||
return [...merged.values()]
|
||||
}
|
||||
|
||||
export function listFlowRowsForWindow(minutes: number): PendingFlowRow[] {
|
||||
export async function listFlowRowsForWindow(minutes: number): Promise<PendingFlowRow[]> {
|
||||
const sinceIso = new Date(Date.now() - minutes * 60_000).toISOString()
|
||||
if (minutes <= 15 && !worker) return listLiveFlowRows(sinceIso)
|
||||
return listStoredFlowRows(sinceIso)
|
||||
if (minutes <= 15 && !worker) return await listLiveFlowRows(sinceIso)
|
||||
return await listStoredFlowRows(sinceIso)
|
||||
}
|
||||
|
||||
export function listFlowTalkers(minutes = 5): FlowStatsDto {
|
||||
const settings = getTrafficFlowSettingsRow()
|
||||
const runtime = getFlowRuntimeCounters()
|
||||
const rows = listFlowRowsForWindow(minutes)
|
||||
const serverRows = db.select().from(servers).all()
|
||||
const nameById = new Map(serverRows.map((s) => [s.id, s.name || s.host]))
|
||||
export async function listFlowTalkers(minutes = 5): Promise<FlowStatsDto> {
|
||||
const settings = await getTrafficFlowSettingsRow()
|
||||
const runtime = await getFlowRuntimeCounters()
|
||||
const rows = await listFlowRowsForWindow(minutes)
|
||||
const catalog = await getServerCatalog()
|
||||
const nameById = new Map([...catalog.byId].map(([id, s]) => [id, s.name]))
|
||||
const agg = new Map<string, FlowTalkerDto & { rawBytes: number }>()
|
||||
const protoBytes = new Map<number, number>()
|
||||
const srcs = new Set<string>()
|
||||
@@ -404,13 +419,13 @@ export function listFlowTalkers(minutes = 5): FlowStatsDto {
|
||||
}
|
||||
}
|
||||
|
||||
export function ingestParsedFlowsForTests(exporterIp: string, flows: ParsedFlowInput[]) {
|
||||
applyExporterCtxFromDb()
|
||||
export async function ingestParsedFlowsForTests(exporterIp: string, flows: ParsedFlowInput[]) {
|
||||
await applyExporterCtxFromDb()
|
||||
const serverId = resolveServerId(exporterIp)
|
||||
if (serverId == null) return
|
||||
queueParsedFlows(serverId, flows)
|
||||
rollFlowRings()
|
||||
flushPending()
|
||||
await flushPending()
|
||||
}
|
||||
|
||||
export function ingestParsedFlowsForServerForTests(serverId: number, flows: ParsedFlowInput[]) {
|
||||
@@ -419,7 +434,6 @@ export function ingestParsedFlowsForServerForTests(serverId: number, flows: Pars
|
||||
|
||||
export function resetFlowRingsForTests() {
|
||||
resetEngineForTests()
|
||||
attachEngineSqlite(sqliteDatabase)
|
||||
lastHeartbeat = null
|
||||
wantListen = false
|
||||
restartAttempts = 0
|
||||
@@ -429,88 +443,50 @@ export function lastFlushUsedTransactionForTests(): boolean {
|
||||
return engineLastFlushTx()
|
||||
}
|
||||
|
||||
export function flushPendingForTests(): void {
|
||||
flushPending()
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending()
|
||||
}
|
||||
|
||||
function tableCount(name: string): number {
|
||||
const row = sqliteDatabase.prepare(`SELECT COUNT(*) AS n FROM ${name}`).get() as { n: number }
|
||||
async function tableCount(name: string): Promise<number> {
|
||||
const row = await dbGet<{ n: number }>(`SELECT COUNT(*) AS n FROM ${name}`)
|
||||
return Number(row?.n) || 0
|
||||
}
|
||||
|
||||
function dbFileBytes(): number {
|
||||
const resolved = path.resolve(process.cwd(), env.DATABASE_PATH)
|
||||
if (!existsSync(resolved)) return 0
|
||||
return statSync(resolved).size
|
||||
}
|
||||
const PURGE_LOCK_KEY = 8_723_103
|
||||
|
||||
async function stopWorkerProcessAsync(): Promise<void> {
|
||||
if (restartTimer) {
|
||||
clearTimeout(restartTimer)
|
||||
restartTimer = null
|
||||
}
|
||||
if (!worker) return
|
||||
const current = worker
|
||||
worker = null
|
||||
try {
|
||||
current.postMessage({ type: "stop" })
|
||||
await current.terminate()
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
|
||||
/** Удаляет сессии, minute/daily rollup и сжимает SQLite. Ключи WG и пиры JH не трогает. */
|
||||
/** Удаляет сессии и minute/daily rollup. Ключи WG и пиры JH не трогает. */
|
||||
export async function purgeTrafficFlowStore(): Promise<FlowPurgeDto> {
|
||||
beginSqliteExclusiveOp()
|
||||
try {
|
||||
return await withAdvisoryLock(PURGE_LOCK_KEY, async () => {
|
||||
wantListen = false
|
||||
await stopWorkerProcessAsync()
|
||||
resetEngineForTests()
|
||||
attachEngineSqlite(sqliteDatabase)
|
||||
lastHeartbeat = null
|
||||
state = { bound: false, address: null }
|
||||
const fileBytesBefore = dbFileBytes()
|
||||
const deleted = {
|
||||
buckets: tableCount("flow_buckets"),
|
||||
minuteStats: tableCount("flow_minute_stats"),
|
||||
minuteDims: tableCount("flow_minute_dims"),
|
||||
dailyDims: tableCount("flow_daily_dims"),
|
||||
buckets: await tableCount("flow_buckets"),
|
||||
minuteStats: await tableCount("flow_minute_stats"),
|
||||
minuteDims: await tableCount("flow_minute_dims"),
|
||||
dailyDims: await tableCount("flow_daily_dims"),
|
||||
}
|
||||
sqliteDatabase.exec(`
|
||||
DELETE FROM flow_buckets;
|
||||
DELETE FROM flow_minute_stats;
|
||||
DELETE FROM flow_minute_dims;
|
||||
DELETE FROM flow_daily_dims;
|
||||
`)
|
||||
resetFlowIngestCounters()
|
||||
await dbQuery(`DELETE FROM flow_buckets`)
|
||||
await dbQuery(`DELETE FROM flow_minute_stats`)
|
||||
await dbQuery(`DELETE FROM flow_minute_dims`)
|
||||
await dbQuery(`DELETE FROM flow_daily_dims`)
|
||||
await resetFlowIngestCounters()
|
||||
await dropExpiredPartitions(pool)
|
||||
try {
|
||||
sqliteDatabase.pragma("wal_checkpoint(TRUNCATE)")
|
||||
await startTrafficFlowListener()
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
let vacuumed = false
|
||||
try {
|
||||
sqliteDatabase.exec("VACUUM")
|
||||
vacuumed = true
|
||||
} catch {
|
||||
vacuumed = false
|
||||
/* ingest мог остаться выключенным */
|
||||
}
|
||||
return {
|
||||
ok: true,
|
||||
deleted,
|
||||
fileBytesBefore,
|
||||
fileBytesAfter: dbFileBytes(),
|
||||
vacuumed,
|
||||
fileBytesBefore: 0,
|
||||
fileBytesAfter: 0,
|
||||
vacuumed: false,
|
||||
}
|
||||
} finally {
|
||||
try {
|
||||
startTrafficFlowListener()
|
||||
} catch {
|
||||
/* ingest мог остаться выключенным */
|
||||
}
|
||||
endSqliteExclusiveOp()
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export { peekPendingFlows }
|
||||
|
||||
@@ -5,6 +5,7 @@ import {
|
||||
resetFlowRingsForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { buildFlowMapHops, resetFlowMapHopsCacheForTests } from "./traffic-flow-map-hops.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
@@ -14,6 +15,11 @@ import {
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-map-hops.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
@@ -115,7 +121,7 @@ ingestParsedFlowsForServerForTests(3, [
|
||||
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const def = buildFlowMapHops({ minutes: 5 })
|
||||
const def = await buildFlowMapHops({ minutes: 5 })
|
||||
assert.equal(def.excludeOverlayApplied, true)
|
||||
assert.equal(def.excludeMeshApplied, true)
|
||||
assert.equal(def.dedupApplied, true)
|
||||
@@ -144,7 +150,7 @@ try {
|
||||
assert.equal(wan.bytes, 3000)
|
||||
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const withAll = buildFlowMapHops({ minutes: 5, excludeOverlay: false, excludeMesh: false })
|
||||
const withAll = await buildFlowMapHops({ minutes: 5, excludeOverlay: false, excludeMesh: false })
|
||||
const overlayIface = withAll.hops.find((h) => h.iface === "gre-jh-en" && h.fromId === "7")
|
||||
assert.ok(overlayIface && overlayIface.bytes >= 5_000_000)
|
||||
const meshIface = withAll.hops.find((h) => h.iface === "gre-client" && h.fromId === "7" && h.kind === "iface")
|
||||
@@ -203,7 +209,7 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const six = buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const six = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.equal(six.totalBytes, 10_000)
|
||||
const google = six.services?.find((s) => s.id === "svc:google")
|
||||
assert.ok(google, "Google ≥ 5%")
|
||||
@@ -233,11 +239,11 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const four = buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const four = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.equal(four.totalBytes, 10_000)
|
||||
assert.ok(!(four.services ?? []).some((s) => s.id === "svc:google"), "Google < 5% hidden")
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const off = buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const off = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google 4%")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
@@ -270,7 +276,7 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const greOnly = buildFlowMapHops({ minutes: 5, excludeOverlay: false, minSharePct: 0 })
|
||||
const greOnly = await buildFlowMapHops({ minutes: 5, excludeOverlay: false, minSharePct: 0 })
|
||||
assert.ok(!(greOnly.services ?? []).some((s) => s.label === "GRE"), "GRE is not a destination service")
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
@@ -317,7 +323,7 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const rev = buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const rev = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
assert.ok(rev.services?.some((s) => s.id === "svc:google"), "реверс Google:443 → 10.x")
|
||||
assert.ok(rev.services?.some((s) => s.id === "svc:cloudflare"), "реверс Cloudflare:443 → 10.x")
|
||||
assert.ok(rev.serviceEdges?.some((e) => e.toId === "svc:google" && e.fromId === "9"))
|
||||
@@ -356,7 +362,7 @@ ingestParsedFlowsForServerForTests(7, [
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const wan = buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const wan = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const googleEdge = wan.serviceEdges?.find((e) => e.toId === "svc:google")
|
||||
assert.ok(googleEdge, "Google с WAN JH")
|
||||
assert.equal(googleEdge.fromId, "9", "якорь на EN, не на JH")
|
||||
@@ -371,4 +377,47 @@ try {
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*1", name: "SWE-VEESP" },
|
||||
])
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
{
|
||||
src: "173.194.151.65",
|
||||
dst: "10.200.100.53",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 57182,
|
||||
bytes: 9_000,
|
||||
packets: 80,
|
||||
inIface: "1",
|
||||
outIface: "1",
|
||||
nextHop: "",
|
||||
},
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const wanOnly = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const googleEdge = wanOnly.serviceEdges?.find((e) => e.toId === "svc:google")
|
||||
assert.ok(googleEdge, "Google WAN без GRE payload")
|
||||
assert.equal(googleEdge.fromId, "9", "единственный EN, даже без nextHop")
|
||||
assert.ok(googleEdge.bps > 0, "скорость на hop EN→сервис")
|
||||
assert.ok(!(wanOnly.serviceEdges ?? []).some((e) => e.fromId === "7"), "нет пунктира с JH")
|
||||
const googlePath = wanOnly.servicePaths?.find((p) => p.serviceId === "svc:google")
|
||||
assert.ok(googlePath, "путь WAN Google")
|
||||
assert.equal(googlePath.viaId, "7", "via = JH exporter")
|
||||
assert.equal(googlePath.enId, "9", "якорь EN")
|
||||
assert.ok(googlePath.bps > 0, "скорость на пути клиента")
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-map-hops.test.ts: ok")
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user