Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5f774ce26e | ||
|
|
25b82997b6 | ||
|
|
b4a3c3a925 | ||
|
|
9c0ee7940e | ||
|
|
5750590b68 | ||
|
|
3c42c114f5 | ||
|
|
5aef419582 | ||
|
|
0c0dfa1df7 | ||
|
|
c162a41bc0 | ||
|
|
97e43b2335 | ||
|
|
db21e1217c | ||
|
|
5bb9066be8 | ||
|
|
b1fd259f10 | ||
|
|
3687bb8fa2 | ||
|
|
a80caf5676 | ||
|
|
d39e3454aa | ||
|
|
a8f2055c77 | ||
|
|
cdeb97d841 | ||
|
|
36c5305db7 | ||
|
|
5f2b4e2d40 | ||
|
|
cff26813b9 |
@@ -0,0 +1,63 @@
|
||||
# Локальные GeoLite2-базы (Country + ASN) для потоков по странам и ASN
|
||||
|
||||
## Контекст
|
||||
|
||||
Сейчас страна и ASN для netflow-потоков резолвятся через внешний RIPEstat API (`backend/src/services/traffic-flow-ripe.ts`): лимит 30 новых префиксов/мин, очередь на 90, кэш в PG `flow_ip_meta`. Новые IP «дозревают» с задержкой, IPv6 не покрывается (кэш индексируется только по IPv4). Локальные mmdb-базы дают мгновенный синхронный lookup всех IP без внешних вызовов.
|
||||
|
||||
Решения (подтверждены):
|
||||
- Источник — **MaxMind GeoLite2 через P3TERX-зеркало**: `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb` и `.../GeoLite2-ASN.mmdb`. Без регистрации, ключей и tar-распаковки. Точность по стране у GeoLite2 и IPinfo паритетная (<1% ошибок у обоих, arXiv 2026); выбран P3TERX за надёжность зеркала (5.2k звёзд) и преемственность: текущий RIPE-путь и так читает GeoLite (`maxmind-geo-lite`), история в кэше остаётся консистентной.
|
||||
- **RIPEstat остаётся fallback** (до первой загрузки баз / если lookup не дал результата).
|
||||
- City-базу не качаем (lat/lng фронтенд не использует).
|
||||
|
||||
## Изменения
|
||||
|
||||
### 1. Зависимость
|
||||
- `npm install -w mikrotik-manager-backend maxmind` — sync-чтение mmdb, встроенные TS-типы, без транзитивных зависимостей, Node 22 ок.
|
||||
|
||||
### 2. Новый сервис `backend/src/services/traffic-flow-geoip.ts`
|
||||
(по конвенциям окружения traffic-flow-*: контракт → маршрут → сервис, без БД-логики в маршрутах)
|
||||
- Каталог: `backend/storage/geoip/` (конвенция `storage/backups`), файлы `GeoLite2-Country.mmdb`, `GeoLite2-ASN.mmdb`.
|
||||
- `initGeoip()` — открыть ридеры best-effort при старте (из `index.ts` рядом с `startTrafficFlowListener`), независимо от настроек автообновления: файлы есть — работают.
|
||||
- `lookupGeoip(ip): FlowIpMeta | null` — синхронно: `country.iso_code` (fallback `registered_country.iso_code`) с валидацией `isIsoCountry`, ASN = `autonomous_system_number`, holder = `autonomous_system_organization`; приватные IP → negative-запись как в RIPE (`isNonPublicIp`); в PG не пишем (lookup и так быстрый). IPv6 поддержан ридером.
|
||||
- `resolveFlowIp(ip)` — фасад: `lookupGeoip(ip) ?? lookupRipeCached(ip)`; главный экспорт для потребителей.
|
||||
- `geoipStatus()` — loaded, даты сборки баз (метаданные mmdb). Тест-хук `setGeoipReadersForTests`. Смена ридеров после обновления — атомарная замена ссылок.
|
||||
|
||||
### 3. Коллектор `backend/src/services/geoip-update-collector.ts`
|
||||
`collectGeoipUpdateOnce()` по образцу `certificate-renew-collector.ts`:
|
||||
1. Conditional GET с ETag/If-None-Match из настроек → 304 = skip (фолбэк-сравнение: размер/содержимое).
|
||||
2. Скачивание в `*.tmp` через глобальный `fetch` + AbortController с таймаутом (внешний HTTP из service-слоя — по правилу fastify-backend-drizzle).
|
||||
3. Валидация: открыть ридер из tmp-файла, пробой 8.8.8.8 (страна US, ASN 15169).
|
||||
4. `fs.rename` атомарная подмена, старый файл → `*.prev` (откат, если новый ридер не открылся).
|
||||
5. Перезагрузка ридеров, статус в настройках; snapshot для `scheduler_runs` (checked/downloaded/skipped/bytes/error).
|
||||
|
||||
### 4. Планировщик (`backend/src/services/scheduler.ts`)
|
||||
- `JOB_KEYS` += `geoip_update`; case в `runSchedulerJobBody`; блок в `refreshScheduler()` по образцу `certificates_renew`: интервал `Math.max(6ч, updateIntervalSec*1000)`, по умолчанию 7 дней (upstream обновляется еженедельно) + немедленный первый запуск при включённой настройке.
|
||||
|
||||
### 5. Схема и миграция
|
||||
- `backend/src/db/schema.ts`: singleton `geoip_settings` — `enabled` (default true), `updateIntervalSec` (default 604800), `lastCheckAt`, `lastSuccessAt`, `lastError`, `countryBuildAt`, `asnBuildAt`, `etagsJson` (jsonb), `createdAt/updatedAt`.
|
||||
- Миграция: `npm run db:generate` → файл в `backend/drizzle/`.
|
||||
|
||||
### 6. API + контракты
|
||||
- `packages/contracts/src/geoip.ts`: zod-схемы настроек/статуса (все входы — Zod, по правилам проекта).
|
||||
- Новый `backend/src/routes/geoip.ts`, регистрация в `index.ts` с prefix `/api`:
|
||||
- `GET /api/geoip` — настройки + статус (ready, даты сборки, последняя проверка/ошибка);
|
||||
- `PUT /api/geoip` — сохранить настройки, затем `refreshScheduler()`;
|
||||
- `POST /api/geoip/update` — запустить загрузку сейчас (409, если уже идёт; флаг-гард как в коллекторах).
|
||||
|
||||
### 7. Интеграция в пайплайн (geoip-first, RIPE-fallback)
|
||||
- `traffic-flow-engine.ts` (`queueParsedFlows`, ~строка 336): `lookupRipeCached` → `resolveFlowIp`. Логика misses не меняется: при готовом mmdb публичные IP (v4+v6) резолвятся сразу, очередь RIPE пустеет; до скачивания баз — прежнее поведение.
|
||||
- Остальные вызовы `lookupRipeCached` → `resolveFlowIp` (grep: как минимум `traffic-flow-analytics.ts` ~258–271).
|
||||
- `classifyFlowDst`/бренды не трогаем: holder из mmdb (org name) встаёт в существующие `HOLDER_BRANDS`-регулярки как есть.
|
||||
|
||||
### 8. Frontend (по next-shadcn-production / ui-guardian: только переиспользование)
|
||||
- Секция «GeoIP-базы (GeoLite2)» внутри существующей `components/traffic/netflow-settings-panel.tsx`: статус (готово/не скачано, даты сборки Country/ASN, последняя проверка, ошибка), тумблер автообновления, интервал, кнопка «Обновить сейчас» с индикатором. Только уже используемые в панели примитивы (Switch/Button/поля) — никаких новых визуальных паттернов и Card-shell. API-клиент через существующие http-хелперы.
|
||||
|
||||
### 9. Хаускипинг, тесты, проверки
|
||||
- `backend/.gitignore`: `storage/geoip/`.
|
||||
- Тесты `backend/src/services/traffic-flow-geoip.test.ts` + скрипт `test:geoip` (по образцу `test:traffic-flow`): приоритет фасада (geoip hit → RIPE не зовётся; miss → fallback), negative на приватных IP, фильтрация EU/ZZ через `isIsoCountry`, коллектор с мокнутым fetch (304-skip, битый файл → подмены нет, `.prev` сохранён), dims по стране/ASN с засеянным ридером.
|
||||
- Проверки после реализации (обязательно по правилам): типы/сборка бэка (`npm run build -w mikrotik-manager-backend`), типы фронта при правке UI (`npx tsc --noEmit`), `npm run test:geoip` и `test:traffic-flow`; предупреждения не игнорировать.
|
||||
- Коммит: `feat(netflow): <subject по-русски>` — новая пользовательская фича (мгновенные страна/ASN в потоках), по commit-messages-ru.
|
||||
- README: короткий раздел о GeoIP; примечание, что в Docker `storage/geoip` ephemeral без тома — базы перекачаются после пересоздания контейнера (~17 МБ); при желании смонтировать volume.
|
||||
|
||||
## Что это даёт
|
||||
Страна и ASN появляются у потока мгновенно при ingest (включая IPv6), без ограничения скорости RIPE; dims `country`/`asn` в `flow_daily_dims`, аналитика (карта, топы, monthly) становятся полными сразу. Внешняя зависимость от stat.ripe.net остаётся только как fallback до первой загрузки баз.
|
||||
@@ -190,6 +190,14 @@ npm run build -w @mmapp/contracts
|
||||
npm --prefix backend run db:migrate-from-sqlite
|
||||
```
|
||||
|
||||
### GeoIP-базы GeoLite2 (страны и ASN для NetFlow)
|
||||
|
||||
Backend держит локальные mmdb-базы MaxMind GeoLite2 (Country + ASN) в `backend/storage/geoip/` и скачивает их с зеркала [P3TERX/GeoLite.mmdb](https://github.com/P3TERX/GeoLite.mmdb) — без регистрации и ключей. Lookup страны/ASN потока при ingest становится мгновенным (включая IPv6) и не упирается в лимиты RIPEstat; пока базы не скачаны или lookup промахнулся, работает прежний RIPE-fallback.
|
||||
|
||||
Управление — секция «GeoIP-базы (GeoLite2)» в настройках NetFlow (страница «Сбор данных»): автообновление (по умолчанию проверка раз в 7 дней, upstream обновляется еженедельно), статус сборки баз и кнопка «Обновить сейчас». Джоба планировщика — `geoip_update`. Атрибуция: данные MaxMind GeoLite2, CC BY-SA 4.0.
|
||||
|
||||
Примечание для Docker: каталог `storage/geoip` внутри контейнера ephemeral — без смонтированного volume базы (~17 МБ) перекачаются после пересоздания контейнера. Каталог переопределяется переменной `GEOIP_DIR`.
|
||||
|
||||
## CI/CD (Gitea Actions)
|
||||
|
||||
Файл: `.gitea/workflows/docker.yml` (имя workflow: **Docker images**).
|
||||
|
||||
+309
-583
File diff suppressed because it is too large
Load Diff
+130
-42
@@ -1,6 +1,6 @@
|
||||
"use client"
|
||||
|
||||
import { Fragment, useState, useMemo, useEffect } from "react"
|
||||
import { useState, useMemo, useEffect } from "react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { BgpSessionsDataGrid } from "@/components/data-grids/bgp-sessions-data-grid"
|
||||
@@ -25,6 +25,9 @@ import {
|
||||
} from "lucide-react"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { servers as mockServers, type Server } from "@/lib/data"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ─── types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -233,6 +236,37 @@ interface BackendBgpSession {
|
||||
capabilities: string[]; lastError: string | null
|
||||
}
|
||||
|
||||
interface BackendServer {
|
||||
id: number
|
||||
name: string
|
||||
host: string
|
||||
type?: Server["type"]
|
||||
site?: string
|
||||
country: string
|
||||
asn?: string
|
||||
enabled: boolean
|
||||
status?: Server["status"]
|
||||
latency?: number | null
|
||||
}
|
||||
|
||||
function mapBackendServer(s: BackendServer): Server {
|
||||
return {
|
||||
id: String(s.id),
|
||||
name: s.name || s.host,
|
||||
host: s.host,
|
||||
model: "—",
|
||||
os: "—",
|
||||
site: s.site ?? "",
|
||||
country: s.country || "UN",
|
||||
asn: s.asn ?? "",
|
||||
type: s.type ?? "exit-node",
|
||||
enabled: s.enabled,
|
||||
status: s.status ?? "online",
|
||||
latency: s.latency ?? null,
|
||||
sessions: 0,
|
||||
}
|
||||
}
|
||||
|
||||
function backendToFrontend(b: BackendBgpSession): BgpSession {
|
||||
return {
|
||||
id: `${b.serverId}-${b.id}`,
|
||||
@@ -623,27 +657,40 @@ const TABS: Array<{ id: BgpTab; label: string; icon: React.ReactNode }> = [
|
||||
|
||||
export default function BgpPage() {
|
||||
const [activeTab, setActiveTab] = useState<BgpTab>("sessions")
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const { mode, backendUrl } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const [liveSessions, setLiveSessions] = useState<BgpSession[]>([])
|
||||
const [liveServers, setLiveServers] = useState<Server[]>([])
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [fetchedAt, setFetchedAt] = useState<Date | null>(null)
|
||||
const [liveError, setLiveError] = useState<string | null>(null)
|
||||
const [fetchTick, setFetchTick] = useState(0)
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) return
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
setLiveSessions([])
|
||||
setLiveServers([])
|
||||
setLiveError(null)
|
||||
})
|
||||
return
|
||||
}
|
||||
let cancelled = false
|
||||
queueMicrotask(() => {
|
||||
if (cancelled) return
|
||||
setLoading(true)
|
||||
setLiveError(null)
|
||||
void requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions")
|
||||
.then(data => {
|
||||
void Promise.all([
|
||||
requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions"),
|
||||
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
|
||||
])
|
||||
.then(([data, servers]) => {
|
||||
if (cancelled) return
|
||||
setLiveSessions(data.map(backendToFrontend))
|
||||
setLiveServers(servers.filter((s) => s.enabled).map(mapBackendServer))
|
||||
setFetchedAt(new Date())
|
||||
setLoading(false)
|
||||
})
|
||||
@@ -656,50 +703,87 @@ export default function BgpPage() {
|
||||
return () => { cancelled = true }
|
||||
}, [isLive, backendUrl, fetchTick])
|
||||
|
||||
// Use live or mock data for all tabs and KPI
|
||||
const sessions = isLive ? liveSessions : SESSIONS
|
||||
const allSessions = isLive ? liveSessions : SESSIONS
|
||||
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
|
||||
|
||||
const effectiveServerId =
|
||||
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const sessions = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return allSessions
|
||||
return allSessions.filter((s) => s.serverId === effectiveServerId)
|
||||
}, [allSessions, effectiveServerId])
|
||||
|
||||
const railItems = useMemo<ServerTileItem[]>(() => {
|
||||
const counts = new Map<string, number>()
|
||||
for (const s of allSessions) {
|
||||
counts.set(s.serverId, (counts.get(s.serverId) ?? 0) + 1)
|
||||
}
|
||||
return displayServers.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.name,
|
||||
host: s.host,
|
||||
site: s.site,
|
||||
country: s.country,
|
||||
status: s.status,
|
||||
type: s.type,
|
||||
count: counts.get(s.id) ?? 0,
|
||||
enabled: s.enabled,
|
||||
title: [s.name, s.host, s.asn].filter(Boolean).join(" · "),
|
||||
}))
|
||||
}, [displayServers, allSessions])
|
||||
|
||||
const established = sessions.filter(s => s.state === "Established").length
|
||||
const notEstab = sessions.length - established
|
||||
const totalRx = sessions.reduce((a, s) => a + s.prefixesRx, 0)
|
||||
const serverCount = useMemo(
|
||||
() => new Set(liveSessions.map(s => s.serverId)).size,
|
||||
[liveSessions],
|
||||
() => new Set(sessions.map(s => s.serverId)).size,
|
||||
[sessions],
|
||||
)
|
||||
|
||||
return (
|
||||
<div className="flex flex-col h-full">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "BGP" }]}
|
||||
actions={
|
||||
<>
|
||||
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
|
||||
{/* tab bar */}
|
||||
<div className="border-b bg-background shrink-0">
|
||||
<div className="flex items-center px-6">
|
||||
{TABS.map(t => (
|
||||
<button key={t.id} onClick={() => setActiveTab(t.id)}
|
||||
className={cn(
|
||||
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px",
|
||||
activeTab === t.id
|
||||
? "border-primary text-foreground"
|
||||
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border",
|
||||
)}>
|
||||
{t.icon}{t.label}
|
||||
</button>
|
||||
))}
|
||||
<ServerRailLayout
|
||||
items={railItems}
|
||||
selectedId={effectiveServerId}
|
||||
onSelect={setSelectedServerId}
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && loading && displayServers.length === 0}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "BGP" }]}
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
}
|
||||
banner={
|
||||
<div className="border-b bg-background shrink-0">
|
||||
<div className="flex items-center px-6">
|
||||
{TABS.map(t => (
|
||||
<button key={t.id} onClick={() => setActiveTab(t.id)}
|
||||
className={cn(
|
||||
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px",
|
||||
activeTab === t.id
|
||||
? "border-primary text-foreground"
|
||||
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border",
|
||||
)}>
|
||||
{t.icon}{t.label}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-6">
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-5">
|
||||
|
||||
{/* data source banner */}
|
||||
@@ -729,11 +813,16 @@ export default function BgpPage() {
|
||||
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
{isLive && !loading && liveSessions.length === 0 && !liveError && fetchedAt && (
|
||||
{isLive && !loading && allSessions.length === 0 && !liveError && fetchedAt && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
BGP не настроен ни на одном сервере
|
||||
</div>
|
||||
)}
|
||||
{isLive && !loading && allSessions.length > 0 && sessions.length === 0 && !liveError && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
На выбранном сервере нет BGP-сессий
|
||||
</div>
|
||||
)}
|
||||
{mode === "mock" && (
|
||||
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
|
||||
Моковые данные
|
||||
@@ -794,7 +883,6 @@ export default function BgpPage() {
|
||||
{activeTab === "analytics" && <AnalyticsTab sessions={sessions} />}
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</ServerRailLayout>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -11,6 +11,7 @@ import { OpsPanel } from "@/components/ops-panel"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { CertificatesDataGrid } from "@/components/data-grids/certificates-data-grid"
|
||||
import { CertificateRenewSettingsPanel } from "@/components/certificates/certificate-renew-settings"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
@@ -243,7 +244,7 @@ function CertPartReference() {
|
||||
return (
|
||||
<OpsPanel
|
||||
title="RouterOS 7 · /certificate — справка CLI"
|
||||
description="RouterOS 7.22+ · публичные LE для Cloudflare через backend DNS-01, не через /certificate add-acme на устройстве."
|
||||
description="RouterOS 7 умеет обновлять Let's Encrypt сам. Этот CLI — справка; автообновление MM включается панелью выше."
|
||||
contentClassName="px-5 py-4"
|
||||
>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
|
||||
@@ -715,6 +716,8 @@ export default function CertificatesPage() {
|
||||
|
||||
<CertPartKpi displayCerts={scopedCerts} expiring={expiring} expired={expired} />
|
||||
|
||||
<CertificateRenewSettingsPanel backendUrl={backendUrl} liveReady={liveReady} />
|
||||
|
||||
{liveReady && (
|
||||
<CertPartAcmeSettings
|
||||
acmeDirectoryUrl={acmeDirectoryUrl}
|
||||
|
||||
+253
-70
@@ -1,15 +1,17 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo, useState } from "react"
|
||||
import { useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { routerContainers, servers } from "@/lib/data"
|
||||
import type { RouterContainer } from "@/lib/data"
|
||||
import { routerContainers as mockContainers, servers as mockServers } from "@/lib/data"
|
||||
import type { RouterContainer, Server } from "@/lib/data"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { Frame, FramePanel } from "@/components/reui/frame"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Alert, AlertDescription } from "@/components/ui/alert"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { toast } from "sonner"
|
||||
import {
|
||||
DropdownMenu, DropdownMenuTrigger, DropdownMenuContent,
|
||||
DropdownMenuItem, DropdownMenuSeparator,
|
||||
@@ -18,18 +20,47 @@ import {
|
||||
BoxIcon, PlayIcon, StopCircleIcon, SearchIcon,
|
||||
MoreHorizontalIcon, Trash2Icon, PencilIcon, PowerIcon,
|
||||
CodeXmlIcon, ActivityIcon, ServerIcon,
|
||||
TerminalIcon, AlertCircleIcon,
|
||||
TerminalIcon, AlertCircleIcon, RefreshCwIcon,
|
||||
} from "lucide-react"
|
||||
import {
|
||||
Sheet, SheetContent, SheetHeader, SheetTitle,
|
||||
SheetDescription, SheetFooter, SheetClose,
|
||||
} from "@/components/ui/sheet"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ─── helpers ──────────────────────────────────────────────────────────────────
|
||||
interface BackendServer {
|
||||
id: number
|
||||
name: string
|
||||
host: string
|
||||
type?: Server["type"]
|
||||
site?: string
|
||||
country: string
|
||||
asn?: string
|
||||
enabled: boolean
|
||||
status?: Server["status"]
|
||||
latency?: number | null
|
||||
}
|
||||
|
||||
function serverFor(id: string) {
|
||||
return servers.find((s) => s.id === id)
|
||||
interface ContainersApiResponse {
|
||||
containers: RouterContainer[]
|
||||
}
|
||||
|
||||
function mapBackendServer(s: BackendServer): Server {
|
||||
return {
|
||||
id: String(s.id),
|
||||
name: s.name || s.host,
|
||||
host: s.host,
|
||||
model: "—",
|
||||
os: "—",
|
||||
site: s.site ?? "",
|
||||
country: s.country || "UN",
|
||||
asn: s.asn ?? "",
|
||||
type: s.type ?? "exit-node",
|
||||
enabled: s.enabled,
|
||||
status: s.status ?? "online",
|
||||
latency: s.latency ?? null,
|
||||
sessions: 0,
|
||||
}
|
||||
}
|
||||
|
||||
function statusConfig(status: RouterContainer["status"]) {
|
||||
@@ -52,10 +83,8 @@ function statusConfig(status: RouterContainer["status"]) {
|
||||
}[status]
|
||||
}
|
||||
|
||||
// ─── RSC generator ────────────────────────────────────────────────────────────
|
||||
|
||||
function generateContainerRsc(c: RouterContainer): string {
|
||||
const srv = serverFor(c.serverId)
|
||||
function generateContainerRsc(c: RouterContainer, serverById: Record<string, Server>): string {
|
||||
const srv = serverById[c.serverId]
|
||||
const lines: string[] = []
|
||||
lines.push(`# RouterOS Container — ${c.name}`)
|
||||
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
|
||||
@@ -63,13 +92,11 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
lines.push(`# RouterOS 7.4+ · /container`)
|
||||
lines.push(``)
|
||||
|
||||
// interface
|
||||
for (const iface of c.interfaces) {
|
||||
lines.push(`/interface/veth/add name=${iface} address=172.17.0.2/24 gateway=172.17.0.1`)
|
||||
}
|
||||
lines.push(``)
|
||||
|
||||
// envs
|
||||
if (c.envs.length > 0) {
|
||||
lines.push(`/container/envs/add name=${c.name}-envs \\`)
|
||||
for (const { key, value } of c.envs) {
|
||||
@@ -78,7 +105,6 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
lines.push(``)
|
||||
}
|
||||
|
||||
// mounts
|
||||
for (const m of c.mounts) {
|
||||
lines.push(`/container/mounts/add name=${c.name}-mount-${m.dst.replace(/\//g, "-").slice(1)} \\`)
|
||||
if (m.src) lines.push(` src=${m.src} \\`)
|
||||
@@ -86,7 +112,6 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
lines.push(``)
|
||||
}
|
||||
|
||||
// container
|
||||
lines.push(`/container/add \\`)
|
||||
lines.push(` remote-image=${c.image}:${c.tag} \\`)
|
||||
lines.push(` interface=${c.interfaces[0] ?? "veth-container"} \\`)
|
||||
@@ -100,12 +125,18 @@ function generateContainerRsc(c: RouterContainer): string {
|
||||
return lines.join("\n")
|
||||
}
|
||||
|
||||
// ─── Export Sheet ─────────────────────────────────────────────────────────────
|
||||
|
||||
function ExportSheet({ open, container, onClose }: {
|
||||
open: boolean; container: RouterContainer | null; onClose: () => void
|
||||
function ExportSheet({
|
||||
open, container, onClose, serverById,
|
||||
}: {
|
||||
open: boolean
|
||||
container: RouterContainer | null
|
||||
onClose: () => void
|
||||
serverById: Record<string, Server>
|
||||
}) {
|
||||
const code = useMemo(() => container ? generateContainerRsc(container) : "", [container])
|
||||
const code = useMemo(
|
||||
() => (container ? generateContainerRsc(container, serverById) : ""),
|
||||
[container, serverById],
|
||||
)
|
||||
|
||||
return (
|
||||
<CodeExportSheet
|
||||
@@ -125,17 +156,29 @@ function ExportSheet({ open, container, onClose }: {
|
||||
)
|
||||
}
|
||||
|
||||
// ─── Container card ───────────────────────────────────────────────────────────
|
||||
|
||||
function ContainerCard({
|
||||
container,
|
||||
server,
|
||||
live,
|
||||
busy,
|
||||
onExport,
|
||||
onStart,
|
||||
onStop,
|
||||
onRestart,
|
||||
onRemove,
|
||||
}: {
|
||||
container: RouterContainer
|
||||
server?: Server
|
||||
live: boolean
|
||||
busy: boolean
|
||||
onExport: () => void
|
||||
onStart: () => void
|
||||
onStop: () => void
|
||||
onRestart: () => void
|
||||
onRemove: () => void
|
||||
}) {
|
||||
const srv = serverFor(container.serverId)
|
||||
const cfg = statusConfig(container.status)
|
||||
const canMutate = live && Boolean(container.rosId)
|
||||
|
||||
return (
|
||||
<Frame dense className="w-full overflow-hidden">
|
||||
@@ -150,29 +193,36 @@ function ContainerCard({
|
||||
</div>
|
||||
<DropdownMenu>
|
||||
<DropdownMenuTrigger render={
|
||||
<Button variant="ghost" size="icon" className="size-7 shrink-0">
|
||||
<Button variant="ghost" size="icon" className="size-7 shrink-0" disabled={busy}>
|
||||
<MoreHorizontalIcon className="size-4" />
|
||||
</Button>
|
||||
} />
|
||||
<DropdownMenuContent side="bottom" align="end">
|
||||
{container.status === "running" ? (
|
||||
<DropdownMenuItem><StopCircleIcon className="size-4 text-amber-500" />Остановить</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={!canMutate} onClick={onStop}>
|
||||
<StopCircleIcon className="size-4 text-amber-500" />Остановить
|
||||
</DropdownMenuItem>
|
||||
) : (
|
||||
<DropdownMenuItem><PlayIcon className="size-4 text-emerald-500" />Запустить</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={!canMutate} onClick={onStart}>
|
||||
<PlayIcon className="size-4 text-emerald-500" />Запустить
|
||||
</DropdownMenuItem>
|
||||
)}
|
||||
<DropdownMenuItem><TerminalIcon className="size-4" />Логи</DropdownMenuItem>
|
||||
<DropdownMenuItem><PencilIcon className="size-4" />Редактировать</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled><TerminalIcon className="size-4" />Логи</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled><PencilIcon className="size-4" />Редактировать</DropdownMenuItem>
|
||||
<DropdownMenuItem onClick={onExport}><CodeXmlIcon className="size-4" />Экспорт .rsc</DropdownMenuItem>
|
||||
<DropdownMenuSeparator />
|
||||
<DropdownMenuItem><PowerIcon className="size-4" />Перезапустить</DropdownMenuItem>
|
||||
<DropdownMenuItem disabled={!canMutate} onClick={onRestart}>
|
||||
<PowerIcon className="size-4" />Перезапустить
|
||||
</DropdownMenuItem>
|
||||
<DropdownMenuSeparator />
|
||||
<DropdownMenuItem variant="destructive"><Trash2Icon className="size-4" />Удалить</DropdownMenuItem>
|
||||
<DropdownMenuItem variant="destructive" disabled={!canMutate} onClick={onRemove}>
|
||||
<Trash2Icon className="size-4" />Удалить
|
||||
</DropdownMenuItem>
|
||||
</DropdownMenuContent>
|
||||
</DropdownMenu>
|
||||
</div>
|
||||
|
||||
<div className="px-4 py-3 flex flex-col gap-3">
|
||||
{/* image */}
|
||||
<div className="flex items-center gap-2">
|
||||
<BoxIcon className="size-3.5 text-muted-foreground shrink-0" />
|
||||
<span className="font-mono text-xs text-foreground/80">
|
||||
@@ -180,17 +230,15 @@ function ContainerCard({
|
||||
</span>
|
||||
</div>
|
||||
|
||||
{/* server */}
|
||||
{srv && (
|
||||
{server && (
|
||||
<div className="flex items-center gap-2 text-xs text-muted-foreground">
|
||||
<ServerIcon className="size-3.5 shrink-0" />
|
||||
<Flag code={srv.country} size={12} />
|
||||
<span className="font-mono">{srv.name}</span>
|
||||
<Flag code={server.country} size={12} />
|
||||
<span className="font-mono">{server.name}</span>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* uptime + stats */}
|
||||
{container.status === "running" && (
|
||||
{container.status === "running" && (container.uptime || container.cpu !== undefined || container.memMb !== undefined) && (
|
||||
<div className="flex items-center gap-4 text-xs text-muted-foreground border-t pt-2.5">
|
||||
{container.uptime && (
|
||||
<div className="flex items-center gap-1">
|
||||
@@ -216,7 +264,6 @@ function ContainerCard({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* interfaces */}
|
||||
{container.interfaces.length > 0 && (
|
||||
<div className="flex flex-wrap gap-1">
|
||||
{container.interfaces.map((i) => (
|
||||
@@ -227,7 +274,6 @@ function ContainerCard({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* mounts */}
|
||||
{container.mounts.length > 0 && (
|
||||
<div className="flex flex-col gap-1">
|
||||
{container.mounts.map((m, idx) => (
|
||||
@@ -249,50 +295,183 @@ function ContainerCard({
|
||||
)
|
||||
}
|
||||
|
||||
// ════════════════════════════════════════════════════════════════════════════
|
||||
export default function ContainersPage() {
|
||||
const [search, setSearch] = useState("")
|
||||
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all")
|
||||
const { mode, backendUrl } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const [search, setSearch] = useState("")
|
||||
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all")
|
||||
const [exportContainer, setExportContainer] = useState<RouterContainer | null>(null)
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const [liveContainers, setLiveContainers] = useState<RouterContainer[]>([])
|
||||
const [liveServers, setLiveServers] = useState<Server[]>([])
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [busyId, setBusyId] = useState<string | null>(null)
|
||||
const [liveError, setLiveError] = useState<string | null>(null)
|
||||
|
||||
const loadLive = useCallback(async () => {
|
||||
if (!isLive) return
|
||||
setLoading(true)
|
||||
setLiveError(null)
|
||||
try {
|
||||
const [cRes, sRes] = await Promise.all([
|
||||
requestJson<ContainersApiResponse>(backendUrl, "/api/containers"),
|
||||
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
|
||||
])
|
||||
setLiveContainers(cRes.containers ?? [])
|
||||
setLiveServers(sRes.filter((s) => s.enabled).map(mapBackendServer))
|
||||
} catch (e) {
|
||||
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
|
||||
setLiveContainers([])
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}, [isLive, backendUrl])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
setLiveContainers([])
|
||||
setLiveServers([])
|
||||
setLiveError(null)
|
||||
})
|
||||
return
|
||||
}
|
||||
queueMicrotask(() => {
|
||||
void loadLive()
|
||||
})
|
||||
}, [isLive, loadLive])
|
||||
|
||||
const displayContainers = isLive ? liveContainers : mockContainers
|
||||
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
|
||||
|
||||
const effectiveServerId =
|
||||
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const scoped = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return displayContainers
|
||||
return displayContainers.filter((c) => c.serverId === effectiveServerId)
|
||||
}, [displayContainers, effectiveServerId])
|
||||
|
||||
const serverById = useMemo(
|
||||
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
|
||||
[displayServers],
|
||||
)
|
||||
|
||||
const railItems = useMemo<ServerTileItem[]>(() => (
|
||||
displayServers.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.name,
|
||||
host: s.host,
|
||||
site: s.site,
|
||||
country: s.country,
|
||||
status: s.status,
|
||||
type: s.type,
|
||||
enabled: s.enabled,
|
||||
meta: String(displayContainers.filter((c) => c.serverId === s.id).length),
|
||||
}))
|
||||
), [displayServers, displayContainers])
|
||||
|
||||
const filtered = useMemo(() => {
|
||||
return routerContainers.filter((c) => {
|
||||
return scoped.filter((c) => {
|
||||
if (statusFilter !== "all" && c.status !== statusFilter) return false
|
||||
if (!search) return true
|
||||
const q = search.toLowerCase()
|
||||
return (
|
||||
c.name.toLowerCase().includes(q) ||
|
||||
c.image.toLowerCase().includes(q) ||
|
||||
(serverFor(c.serverId)?.name.toLowerCase().includes(q) ?? false)
|
||||
(serverById[c.serverId]?.name.toLowerCase().includes(q) ?? false)
|
||||
)
|
||||
})
|
||||
}, [search, statusFilter])
|
||||
}, [search, statusFilter, scoped, serverById])
|
||||
|
||||
const running = routerContainers.filter((c) => c.status === "running").length
|
||||
const stopped = routerContainers.filter((c) => c.status === "stopped").length
|
||||
const errors = routerContainers.filter((c) => c.status === "error").length
|
||||
const running = scoped.filter((c) => c.status === "running").length
|
||||
const stopped = scoped.filter((c) => c.status === "stopped").length
|
||||
const errors = scoped.filter((c) => c.status === "error").length
|
||||
|
||||
async function mutate(c: RouterContainer, action: "start" | "stop" | "restart" | "remove") {
|
||||
if (!isLive || !c.rosId) {
|
||||
toast.info("Действие доступно только в live-режиме")
|
||||
return
|
||||
}
|
||||
if (action === "remove" && !window.confirm(`Удалить контейнер ${c.name}?`)) return
|
||||
setBusyId(c.id)
|
||||
try {
|
||||
await requestJson(backendUrl, `/api/servers/${c.serverId}/containers/${action}`, {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ rosId: c.rosId }),
|
||||
})
|
||||
const labels = { start: "запущен", stop: "остановлен", restart: "перезапущен", remove: "удалён" }
|
||||
toast.success(`${c.name}: ${labels[action]}`)
|
||||
await loadLive()
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Ошибка RouterOS")
|
||||
} finally {
|
||||
setBusyId(null)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="flex flex-col h-full">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]}
|
||||
actions={
|
||||
<Button size="sm">
|
||||
<BoxIcon className="size-4" />Новый контейнер
|
||||
</Button>
|
||||
}
|
||||
/>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-6">
|
||||
<>
|
||||
<ServerRailLayout
|
||||
items={railItems}
|
||||
selectedId={effectiveServerId}
|
||||
onSelect={setSelectedServerId}
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && loading && displayServers.length === 0}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]}
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => { void loadLive() }}
|
||||
disabled={!isLive || loading}
|
||||
>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button size="sm">
|
||||
<BoxIcon className="size-4" />Новый контейнер
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-5">
|
||||
|
||||
{isLive && liveError && (
|
||||
<Alert variant="warning" className="py-2">
|
||||
<AlertCircleIcon />
|
||||
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
{isLive && !loading && displayContainers.length === 0 && !liveError && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
Контейнеры не найдены. Нужен пакет container (RouterOS 7.4+).
|
||||
</div>
|
||||
)}
|
||||
{mode === "mock" && (
|
||||
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
|
||||
Моковые данные
|
||||
</span>
|
||||
)}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка контейнеров"
|
||||
items={[
|
||||
{
|
||||
id: "all",
|
||||
label: "Всего",
|
||||
value: routerContainers.length,
|
||||
value: scoped.length,
|
||||
icon: <BoxIcon className="size-4" />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
@@ -321,7 +500,6 @@ export default function ContainersPage() {
|
||||
]}
|
||||
/>
|
||||
|
||||
{/* Info banner */}
|
||||
<div className="flex items-start gap-3 rounded-lg bg-violet-500/5 border border-violet-500/20 px-4 py-3 text-sm">
|
||||
<BoxIcon className="size-5 text-violet-500 shrink-0 mt-0.5" />
|
||||
<div>
|
||||
@@ -333,7 +511,6 @@ export default function ContainersPage() {
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Toolbar */}
|
||||
<div className="flex items-center gap-3 flex-wrap">
|
||||
<div className="flex items-center gap-2 h-8 px-3 border border-input rounded-md bg-background min-w-[240px]">
|
||||
<SearchIcon className="size-3.5 text-muted-foreground shrink-0" />
|
||||
@@ -363,7 +540,6 @@ export default function ContainersPage() {
|
||||
<span className="text-sm text-muted-foreground ml-auto">{filtered.length} контейнеров</span>
|
||||
</div>
|
||||
|
||||
{/* Grid */}
|
||||
{filtered.length === 0 ? (
|
||||
<div className="flex flex-col items-center justify-center py-16 text-center text-muted-foreground">
|
||||
<BoxIcon className="size-10 mb-3 opacity-20" />
|
||||
@@ -376,13 +552,19 @@ export default function ContainersPage() {
|
||||
<ContainerCard
|
||||
key={c.id}
|
||||
container={c}
|
||||
server={serverById[c.serverId]}
|
||||
live={isLive}
|
||||
busy={busyId === c.id}
|
||||
onExport={() => setExportContainer(c)}
|
||||
onStart={() => { void mutate(c, "start") }}
|
||||
onStop={() => { void mutate(c, "stop") }}
|
||||
onRestart={() => { void mutate(c, "restart") }}
|
||||
onRemove={() => { void mutate(c, "remove") }}
|
||||
/>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* RouterOS reference */}
|
||||
<OpsPanel title="RouterOS 7.4+ · /container — быстрые команды" contentClassName="px-5 py-4">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
|
||||
{[
|
||||
@@ -442,13 +624,14 @@ export default function ContainersPage() {
|
||||
</OpsPanel>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</ServerRailLayout>
|
||||
|
||||
<ExportSheet
|
||||
open={!!exportContainer}
|
||||
container={exportContainer}
|
||||
onClose={() => setExportContainer(null)}
|
||||
serverById={serverById}
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
+260
-851
File diff suppressed because it is too large
Load Diff
@@ -34,6 +34,7 @@ import {
|
||||
type InternetPathRunSnapshot,
|
||||
type CertificatesRenewRunSnapshot,
|
||||
type BackupsRunSnapshot,
|
||||
type GeoipUpdateRunSnapshot,
|
||||
type PingRunSnapshot,
|
||||
type ResourcesRunSnapshot,
|
||||
type SchedulerRunSnapshot,
|
||||
@@ -340,6 +341,41 @@ function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (snap.job === "geoip_update") {
|
||||
const g = snap as GeoipUpdateRunSnapshot
|
||||
return (
|
||||
<div className="flex flex-col gap-3">
|
||||
{g.skipped ? (
|
||||
<p className="text-xs text-amber-600 dark:text-amber-400">Прогон пропущен: обновление уже выполнялось или задача отключена.</p>
|
||||
) : null}
|
||||
<dl className="grid grid-cols-2 gap-3 text-xs sm:grid-cols-4">
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Баз проверено</dt>
|
||||
<dd className="font-mono font-medium">{g.checked}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Скачано</dt>
|
||||
<dd className="font-mono font-medium">{g.downloaded}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Без изменений</dt>
|
||||
<dd className="font-mono font-medium">{g.skippedUnchanged}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Объём, МБ</dt>
|
||||
<dd className="font-mono font-medium">{(g.bytes / 1024 / 1024).toFixed(1)}</dd>
|
||||
</div>
|
||||
</dl>
|
||||
{g.errors.length ? (
|
||||
<div className="rounded-md border border-amber-500/40 bg-amber-500/10 px-3 py-2 text-xs text-amber-800 dark:text-amber-200 flex flex-col gap-1">
|
||||
{g.errors.map((e, i) => (
|
||||
<p key={i} className="break-words">{e}</p>
|
||||
))}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (snap.job === "alert_engine") {
|
||||
const a = snap as AlertEngineRunSnapshot
|
||||
return (
|
||||
@@ -1127,7 +1163,11 @@ export default function DataCollectionPage() {
|
||||
onChange={(e) => setRenewBeforeDaysDraft(e.target.value)}
|
||||
className="h-8 text-sm"
|
||||
inputMode="numeric"
|
||||
disabled={!draftCertRenewEnabled}
|
||||
/>
|
||||
<p className="text-[11px] text-muted-foreground mt-1">
|
||||
Вкл/выкл автообновления MM — также на странице «Сертификаты». Не включайте вместе со встроенным ACME RouterOS.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-[11px] text-muted-foreground leading-snug">
|
||||
|
||||
+172
-138
@@ -27,7 +27,7 @@ import {
|
||||
StarIcon, ArrowUpDownIcon, ArrowUpIcon, ArrowDownIcon,
|
||||
FileCodeIcon, CopyIcon, NetworkIcon, TagIcon,
|
||||
ArrowRightIcon, AlertTriangleIcon, LoaderCircleIcon, RouteIcon,
|
||||
CheckCircle2Icon, XCircleIcon, CircleDashedIcon, RefreshCwIcon,
|
||||
RefreshCwIcon, HistoryIcon,
|
||||
} from "lucide-react"
|
||||
import {
|
||||
Sheet, SheetContent, SheetHeader, SheetTitle, SheetDescription, SheetFooter,
|
||||
@@ -36,13 +36,13 @@ import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
|
||||
import { Tooltip, TooltipContent, TooltipTrigger } from "@/components/ui/tooltip"
|
||||
import { toast } from "sonner"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ── helpers ────────────────────────────────────────────────────────────────────
|
||||
|
||||
type FilterRouterSyncStatus = "synced" | "drift" | "missing"
|
||||
|
||||
function newId() { return `r${Date.now()}-${Math.random().toString(36).slice(2, 6)}` }
|
||||
function innerIpToGateway(ip: string) { return ip.split("/")[0] }
|
||||
|
||||
@@ -1239,6 +1239,9 @@ interface BackendServer {
|
||||
interface LiveFiltersResponse {
|
||||
rulesets: ServerFilterRuleset[]
|
||||
greTunnels: GreTunnel[]
|
||||
live?: boolean
|
||||
stale?: boolean
|
||||
error?: string
|
||||
}
|
||||
|
||||
function buildRulesets(serverList: Server[], sourceRulesets: ServerFilterRuleset[]): ServerFilterRuleset[] {
|
||||
@@ -1325,18 +1328,14 @@ export default function FiltersPage() {
|
||||
const [sheetMode, setSheetMode] = useState<"create" | "edit">("create")
|
||||
const [sheetInitial, setSheetInitial]= useState<RuleForm>(emptyForm())
|
||||
const [editingId, setEditingId] = useState<string | null>(null)
|
||||
const [previewOpen, setPreviewOpen] = useState(false)
|
||||
const [copyOpen, setCopyOpen] = useState(false)
|
||||
const [syncBusy, setSyncBusy] = useState<"from" | "to" | null>(null)
|
||||
const [routerCompare, setRouterCompare] = useState<{
|
||||
serverId: string
|
||||
byCommunity: Record<string, FilterRouterSyncStatus>
|
||||
} | null>(null)
|
||||
const [routerCompareLoading, setRouterCompareLoading] = useState(false)
|
||||
|
||||
useEffect(() => {
|
||||
setRouterCompare(null)
|
||||
}, [selectedServerId])
|
||||
const [previewOpen, setPreviewOpen] = useState(false)
|
||||
const [copyOpen, setCopyOpen] = useState(false)
|
||||
const [applyBusy, setApplyBusy] = useState(false)
|
||||
const [liveStale, setLiveStale] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
@@ -1347,6 +1346,7 @@ export default function FiltersPage() {
|
||||
setRulesets(buildRulesets(servers, serverFilterRulesets))
|
||||
setSelectedServerId(servers[0]?.id ?? "")
|
||||
setLiveLoadState("idle")
|
||||
setLiveStale(false)
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -1390,18 +1390,43 @@ export default function FiltersPage() {
|
||||
})
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
const loadLiveRules = useCallback(async (serverId: string) => {
|
||||
if (!isLive || !serverId) return
|
||||
try {
|
||||
const fresh = await apiFetch<LiveFiltersResponse>(
|
||||
`/api/filters/rules?serverId=${encodeURIComponent(serverId)}`,
|
||||
)
|
||||
const liveRules = fresh.rulesets.find((r) => r.serverId === serverId)?.rules ?? fresh.rulesets[0]?.rules ?? []
|
||||
setRulesets((prev) => {
|
||||
const has = prev.some((rs) => rs.serverId === serverId)
|
||||
if (!has) return [...prev, { serverId, rules: liveRules }]
|
||||
return prev.map((rs) => rs.serverId === serverId ? { ...rs, rules: liveRules } : rs)
|
||||
})
|
||||
setLiveStale(Boolean(fresh.stale))
|
||||
if (fresh.greTunnels?.length) {
|
||||
setGreByServer((prev) => ({ ...prev, [serverId]: fresh.greTunnels }))
|
||||
}
|
||||
} catch (err) {
|
||||
setLiveStale(true)
|
||||
toast.error("Не удалось прочитать правила с роутера", { description: String(err) })
|
||||
}
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
if (!isLive || !selectedServerId || liveLoadState !== "idle") return
|
||||
if (!liveServers.some((s) => s.id === selectedServerId)) return
|
||||
void Promise.all([
|
||||
loadLiveRules(selectedServerId),
|
||||
ensureGreTunnels(selectedServerId),
|
||||
ensureRecursiveRoutes(selectedServerId),
|
||||
])
|
||||
}, [isLive, selectedServerId, ensureGreTunnels, ensureRecursiveRoutes])
|
||||
}, [isLive, selectedServerId, liveLoadState, liveServers, ensureGreTunnels, ensureRecursiveRoutes, loadLiveRules])
|
||||
|
||||
const allServers = isLive ? liveServers : servers
|
||||
const allTunnels = isLive ? (greByServer[selectedServerId] ?? []) : greTunnels
|
||||
const allServers = !isLive || liveLoadState === "error" ? servers : liveServers
|
||||
const allTunnels = !isLive || liveLoadState === "error" ? greTunnels : (greByServer[selectedServerId] ?? [])
|
||||
const selectedServer = allServers.find(s => s.id === selectedServerId) ?? allServers[0]
|
||||
const totalRules = rulesets.reduce((s, r) => s + r.rules.length, 0)
|
||||
const mutationsLocked = isLive && (applyBusy || liveStale || liveLoadState === "error")
|
||||
|
||||
const filterRailItems = useMemo<ServerTileItem[]>(() => (
|
||||
allServers.map((s) => ({
|
||||
@@ -1427,21 +1452,6 @@ export default function FiltersPage() {
|
||||
[rulesets, selectedServerId],
|
||||
)
|
||||
|
||||
const fetchRouterCompare = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setRouterCompareLoading(true)
|
||||
try {
|
||||
const d = await apiFetch<{ byCommunity: Record<string, FilterRouterSyncStatus> }>(
|
||||
`/api/filters/router-compare?serverId=${encodeURIComponent(selectedServerId)}`,
|
||||
)
|
||||
setRouterCompare({ serverId: selectedServerId, byCommunity: d.byCommunity })
|
||||
} catch {
|
||||
setRouterCompare(null)
|
||||
} finally {
|
||||
setRouterCompareLoading(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch])
|
||||
|
||||
const filteredRules = useMemo(() => {
|
||||
const q = search.toLowerCase()
|
||||
if (!q) return currentRules
|
||||
@@ -1453,68 +1463,94 @@ export default function FiltersPage() {
|
||||
)
|
||||
}, [currentRules, search, communityNameMap])
|
||||
|
||||
const updateRules = useCallback((serverId: string, updater: (rules: FilterRule[]) => FilterRule[]) => {
|
||||
setRouterCompare(rc => (rc && rc.serverId === serverId ? null : rc))
|
||||
setRulesets(prev => {
|
||||
const next = prev.map(rs =>
|
||||
rs.serverId === serverId ? { ...rs, rules: updater(rs.rules) } : rs
|
||||
)
|
||||
if (isLive) {
|
||||
void apiFetch<{ ok: boolean }>("/api/filters/rules", {
|
||||
method: "PUT",
|
||||
body: JSON.stringify({ rulesets: next }),
|
||||
}).catch(() => {})
|
||||
}
|
||||
return next
|
||||
})
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
const syncFromRouter = useCallback(async () => {
|
||||
if (!isLive || syncBusy) return
|
||||
setSyncBusy("from")
|
||||
const applyRules = useCallback(async (
|
||||
serverId: string,
|
||||
nextRules: FilterRule[],
|
||||
source: "apply" | "copy" = "apply",
|
||||
) => {
|
||||
const prev = rulesets
|
||||
setRulesets((p) => p.map((rs) => rs.serverId === serverId ? { ...rs, rules: nextRules } : rs))
|
||||
if (!isLive) return
|
||||
if (liveStale) {
|
||||
setRulesets(prev)
|
||||
toast.error("Роутер недоступен — изменения заблокированы")
|
||||
return
|
||||
}
|
||||
setApplyBusy(true)
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/filters/sync/from-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
const res = await apiFetch<{ ok: boolean; rules?: FilterRule[] }>("/api/filters/rules", {
|
||||
method: "PUT",
|
||||
body: JSON.stringify({ serverId, rules: nextRules, source }),
|
||||
})
|
||||
const fresh = await apiFetch<LiveFiltersResponse>("/api/filters/rules")
|
||||
setRulesets(buildRulesets(allServers, fresh.rulesets))
|
||||
if (res.rules) {
|
||||
setRulesets((p) => p.map((rs) => rs.serverId === serverId ? { ...rs, rules: res.rules ?? nextRules } : rs))
|
||||
}
|
||||
toast.success("Правила применены на роутер")
|
||||
} catch (err) {
|
||||
setRulesets(prev)
|
||||
toast.error("Не удалось применить правила на роутер", { description: String(err) })
|
||||
} finally {
|
||||
setApplyBusy(false)
|
||||
}
|
||||
}, [isLive, apiFetch, rulesets, liveStale])
|
||||
|
||||
const refreshFromRouter = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId || applyBusy) return
|
||||
setApplyBusy(true)
|
||||
try {
|
||||
await loadLiveRules(selectedServerId)
|
||||
await Promise.all([
|
||||
ensureGreTunnels(selectedServerId),
|
||||
ensureRecursiveRoutes(selectedServerId),
|
||||
])
|
||||
await fetchRouterCompare()
|
||||
} finally {
|
||||
setSyncBusy(null)
|
||||
setApplyBusy(false)
|
||||
}
|
||||
}, [isLive, syncBusy, apiFetch, allServers, selectedServerId, ensureGreTunnels, ensureRecursiveRoutes, fetchRouterCompare])
|
||||
}, [isLive, selectedServerId, applyBusy, loadLiveRules, ensureGreTunnels, ensureRecursiveRoutes])
|
||||
|
||||
const syncToRouter = useCallback(async () => {
|
||||
if (!isLive || syncBusy || !selectedServerId) return
|
||||
setSyncBusy("to")
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await apiFetch<{
|
||||
ok: boolean
|
||||
updatedServers: number
|
||||
pushedRules: number
|
||||
errors?: Array<{ serverId: number; error: string }>
|
||||
}>("/api/filters/sync/to-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
})
|
||||
if (res.ok) {
|
||||
toast.success(`Загружено правил на роутер: ${res.pushedRules}`)
|
||||
} else {
|
||||
const detail = res.errors?.[0]?.error ?? "неизвестная ошибка"
|
||||
toast.error("Не удалось загрузить правила на роутер", { description: detail })
|
||||
}
|
||||
await fetchRouterCompare()
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/filters/revisions?serverId=${encodeURIComponent(selectedServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить правила на роутер", { description: String(err) })
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setSyncBusy(null)
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, syncBusy, selectedServerId, apiFetch, fetchRouterCompare])
|
||||
}, [isLive, selectedServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
const res = await apiFetch<{ ok: boolean; rules?: FilterRule[] }>(
|
||||
`/api/filters/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: selectedServerId }) },
|
||||
)
|
||||
if (res.rules) {
|
||||
setRulesets((p) => p.map((rs) => rs.serverId === selectedServerId ? { ...rs, rules: res.rules ?? [] } : rs))
|
||||
} else {
|
||||
await loadLiveRules(selectedServerId)
|
||||
}
|
||||
setLiveStale(false)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch, loadLiveRules, loadRevisions])
|
||||
|
||||
const updateRules = useCallback((serverId: string, updater: (rules: FilterRule[]) => FilterRule[]) => {
|
||||
const current = rulesets.find((rs) => rs.serverId === serverId)?.rules ?? []
|
||||
void applyRules(serverId, updater(current))
|
||||
}, [rulesets, applyRules])
|
||||
|
||||
const openCreate = () => {
|
||||
setSheetInitial(emptyForm()); setSheetMode("create"); setEditingId(null); setSheetOpen(true)
|
||||
@@ -1532,6 +1568,7 @@ export default function FiltersPage() {
|
||||
}
|
||||
|
||||
const handleSave = (form: RuleForm) => {
|
||||
if (mutationsLocked) return
|
||||
const { gatewayKind: _gk, ...payload } = form
|
||||
if (sheetMode === "create") {
|
||||
updateRules(selectedServerId, rules => [
|
||||
@@ -1549,34 +1586,30 @@ export default function FiltersPage() {
|
||||
setSheetOpen(false)
|
||||
}
|
||||
|
||||
const handleDelete = (id: string) => updateRules(selectedServerId, rules => rules.filter(r => r.id !== id))
|
||||
const handleDelete = (id: string) => {
|
||||
if (mutationsLocked) return
|
||||
updateRules(selectedServerId, rules => rules.filter(r => r.id !== id))
|
||||
}
|
||||
|
||||
const handleCopyRules = useCallback((targetServerId: string, rules: FilterRule[], mode: CopyMode) => {
|
||||
updateRules(targetServerId, existing =>
|
||||
mode === "replace" ? rules : [...existing, ...rules]
|
||||
)
|
||||
}, [updateRules])
|
||||
const existing = rulesets.find((rs) => rs.serverId === targetServerId)?.rules ?? []
|
||||
const next = mode === "replace" ? rules : [...existing, ...rules]
|
||||
void applyRules(targetServerId, next, "copy")
|
||||
}, [rulesets, applyRules])
|
||||
const handleMoveUp = (index: number) => {
|
||||
if (index === 0) return
|
||||
if (mutationsLocked || index === 0) return
|
||||
updateRules(selectedServerId, rules => {
|
||||
const n = [...rules]; [n[index - 1], n[index]] = [n[index], n[index - 1]]; return n
|
||||
})
|
||||
}
|
||||
const handleMoveDown = (index: number) => {
|
||||
if (mutationsLocked) return
|
||||
updateRules(selectedServerId, rules => {
|
||||
if (index >= rules.length - 1) return rules
|
||||
const n = [...rules]; [n[index], n[index + 1]] = [n[index + 1], n[index]]; return n
|
||||
})
|
||||
}
|
||||
|
||||
if (!selectedServer) {
|
||||
return (
|
||||
<div className="flex h-full items-center justify-center text-sm text-muted-foreground">
|
||||
Нет доступных серверов
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
if (isLive && liveLoadState === "loading") {
|
||||
return (
|
||||
<div className="flex h-full flex-col items-center justify-center gap-3 text-sm text-muted-foreground">
|
||||
@@ -1586,6 +1619,14 @@ export default function FiltersPage() {
|
||||
)
|
||||
}
|
||||
|
||||
if (!selectedServer) {
|
||||
return (
|
||||
<div className="flex h-full items-center justify-center text-sm text-muted-foreground">
|
||||
Нет доступных серверов
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<>
|
||||
<ServerRailLayout
|
||||
@@ -1604,35 +1645,25 @@ export default function FiltersPage() {
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={syncFromRouter}
|
||||
disabled={syncBusy !== null}
|
||||
title="Синхронизация Router → БД"
|
||||
onClick={() => void refreshFromRouter()}
|
||||
disabled={applyBusy}
|
||||
title="Прочитать актуальные правила с роутера"
|
||||
>
|
||||
{syncBusy === "from" ? "Синк Router → DB…" : "Router → DB"}
|
||||
<RefreshCwIcon className={cn("size-4", applyBusy && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={syncToRouter}
|
||||
disabled={syncBusy !== null}
|
||||
title="Синхронизация БД → Router"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={applyBusy}
|
||||
title="История версий и откат на CHR"
|
||||
>
|
||||
{syncBusy === "to" ? "Синк DB → Router…" : "DB → Router"}
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => void fetchRouterCompare()}
|
||||
disabled={syncBusy !== null || routerCompareLoading}
|
||||
title="Сравнить правила в БД с цепочкой bgp-in на MikroTik"
|
||||
className="gap-1.5"
|
||||
>
|
||||
{routerCompareLoading ? (
|
||||
<LoaderCircleIcon className="size-4 animate-spin" />
|
||||
) : (
|
||||
<RefreshCwIcon className="size-4" />
|
||||
)}
|
||||
Сверить
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
@@ -1642,12 +1673,12 @@ export default function FiltersPage() {
|
||||
<Button
|
||||
variant="outline" size="sm"
|
||||
onClick={() => setCopyOpen(true)}
|
||||
disabled={currentRules.length === 0}
|
||||
disabled={currentRules.length === 0 || mutationsLocked}
|
||||
title="Копировать правила на другой сервер"
|
||||
>
|
||||
<CopyIcon className="size-4" />Копировать
|
||||
</Button>
|
||||
<Button size="sm" onClick={openCreate}>
|
||||
<Button size="sm" onClick={openCreate} disabled={mutationsLocked}>
|
||||
<PlusIcon className="size-4" />Новое правило
|
||||
</Button>
|
||||
</>
|
||||
@@ -1662,6 +1693,12 @@ export default function FiltersPage() {
|
||||
Бекенд недоступен — показаны демо-данные из lib/data. Проверьте URL бекенда в настройках.
|
||||
</div>
|
||||
)}
|
||||
{isLive && liveStale && liveLoadState !== "error" && (
|
||||
<div className="shrink-0 border-b border-amber-500/30 bg-amber-500/10 px-6 py-2.5 text-xs text-amber-700 dark:text-amber-400 flex items-center gap-2">
|
||||
<AlertTriangleIcon className="size-3.5 shrink-0" />
|
||||
Роутер недоступен — показан кэш. Изменения заблокированы, пока не удастся прочитать CHR.
|
||||
</div>
|
||||
)}
|
||||
<div className="border-b px-4 py-3 flex items-center gap-3 flex-wrap shrink-0 md:px-6">
|
||||
<div className="relative min-w-[200px] max-w-xs flex-1">
|
||||
<SearchIcon className="absolute left-2.5 top-1/2 -translate-y-1/2 size-3.5 text-muted-foreground pointer-events-none" />
|
||||
@@ -1745,15 +1782,7 @@ export default function FiltersPage() {
|
||||
)}>{selectedServer.latency}мс</span>
|
||||
)}
|
||||
<div className="ml-auto flex items-center gap-2 text-xs text-muted-foreground flex-wrap justify-end">
|
||||
{isLive && routerCompare?.serverId === selectedServerId && currentRules.length > 0 && (
|
||||
<span className="font-mono tabular-nums">
|
||||
роутер:{" "}
|
||||
<span className="text-emerald-600 dark:text-emerald-500">
|
||||
{Object.values(routerCompare.byCommunity).filter(s => s === "synced").length}
|
||||
</span>
|
||||
/{currentRules.length} совпало
|
||||
</span>
|
||||
)}
|
||||
{applyBusy && <span>Применение на роутер…</span>}
|
||||
<span>{currentRules.length} правил</span>
|
||||
</div>
|
||||
</div>
|
||||
@@ -1790,12 +1819,6 @@ export default function FiltersPage() {
|
||||
serversList={allServers}
|
||||
communityNameMap={communityNameMap}
|
||||
recursiveRoutes={recRoutesByServer[selectedServerId] ?? []}
|
||||
routerSyncByCommunity={
|
||||
!isLive || !routerCompare || routerCompare.serverId !== selectedServerId
|
||||
? null
|
||||
: routerCompare.byCommunity
|
||||
}
|
||||
isLive={isLive}
|
||||
enableSorting={!!search}
|
||||
onEdit={openEdit}
|
||||
onDelete={handleDelete}
|
||||
@@ -1805,7 +1828,7 @@ export default function FiltersPage() {
|
||||
)}
|
||||
|
||||
{/* add rule shortcut */}
|
||||
<button onClick={openCreate}
|
||||
<button onClick={openCreate} disabled={mutationsLocked}
|
||||
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t">
|
||||
<PlusIcon className="size-3.5" />
|
||||
Добавить правило для {selectedServer.name}
|
||||
@@ -1854,6 +1877,17 @@ export default function FiltersPage() {
|
||||
recRoutesByServer={recRoutesByServer}
|
||||
ensureRecursiveFor={ensureRecursiveRoutes}
|
||||
/>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История фильтров"
|
||||
itemLabel="правил"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -49,12 +49,14 @@ import {
|
||||
PowerIcon, CheckCircleIcon,
|
||||
PlayIcon, SquareIcon, RotateCcwIcon, ZapIcon,
|
||||
CheckCircle2Icon, XCircleIcon, MinusCircleIcon, SkipForwardIcon,
|
||||
SlidersHorizontalIcon, RefreshCwIcon,
|
||||
SlidersHorizontalIcon, RefreshCwIcon, HistoryIcon,
|
||||
} from "lucide-react"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
import { toast } from "sonner"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
|
||||
// ─── Types ────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1947,6 +1949,10 @@ function FirewallPageInner() {
|
||||
const [exportOpen, setExportOpen] = useState(false)
|
||||
const [editingAddr, setEditingAddr] = useState<Partial<AddressListEntry> | null>(null)
|
||||
const [addrSheetOpen, setAddrSheetOpen] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
|
||||
const loadLive = useCallback(async () => {
|
||||
if (!isLive) return
|
||||
@@ -1970,6 +1976,42 @@ function FirewallPageInner() {
|
||||
}
|
||||
}, [isLive, apiFetch])
|
||||
|
||||
const historyServerId = selectedServerId === ALL_SERVERS_ID ? null : selectedServerId
|
||||
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/firewall/revisions?serverId=${encodeURIComponent(historyServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
await apiFetch(
|
||||
`/api/firewall/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
|
||||
)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadLive()
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch, loadLive, loadRevisions])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
@@ -2373,6 +2415,19 @@ function FirewallPageInner() {
|
||||
<RefreshCwIcon className={cn("size-4", dataLoading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={!isLive || !historyServerId || dataLoading}
|
||||
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
|
||||
>
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" onClick={() => setExportOpen(true)}>
|
||||
<CodeXmlIcon className="size-4" />Экспорт .rsc
|
||||
</Button>
|
||||
@@ -2586,6 +2641,17 @@ function FirewallPageInner() {
|
||||
onClose={() => setExportOpen(false)}
|
||||
rules={familyRules}
|
||||
/>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История Firewall"
|
||||
itemLabel="объектов"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
+289
-58
@@ -13,11 +13,23 @@ import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { toast } from "sonner"
|
||||
import { Frame, FramePanel } from "@/components/reui/frame"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import {
|
||||
AlertDialog,
|
||||
AlertDialogAction,
|
||||
AlertDialogCancel,
|
||||
AlertDialogContent,
|
||||
AlertDialogDescription,
|
||||
AlertDialogFooter,
|
||||
AlertDialogHeader,
|
||||
AlertDialogMedia,
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog"
|
||||
import {
|
||||
Sheet, SheetContent, SheetHeader, SheetTitle,
|
||||
SheetDescription, SheetFooter, SheetClose,
|
||||
@@ -31,7 +43,7 @@ import {
|
||||
LockIcon, LockOpenIcon, ShieldCheckIcon, NetworkIcon,
|
||||
EyeIcon, EyeOffIcon, ChevronDownIcon, ChevronRightIcon,
|
||||
CodeXmlIcon, PencilIcon, PowerIcon, Trash2Icon,
|
||||
DatabaseIcon,
|
||||
DatabaseIcon, HistoryIcon, TriangleAlertIcon,
|
||||
} from "lucide-react"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
@@ -56,6 +68,10 @@ const STATUS_MAP: Record<GreStatus, { label: string; dot: string }> = {
|
||||
down: { label: "Down", dot: "bg-red-500" },
|
||||
}
|
||||
|
||||
function greStatusMeta(status: GreStatus | undefined) {
|
||||
return STATUS_MAP[status ?? "degraded"] ?? STATUS_MAP.degraded
|
||||
}
|
||||
|
||||
// ─── RouterOS code generator ─────────────────────────────────────────────────
|
||||
|
||||
function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>): string {
|
||||
@@ -91,10 +107,10 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
|
||||
lines.push(` address=${t.localInnerIp} \\`)
|
||||
lines.push(` interface=${t.name}`)
|
||||
|
||||
// IPsec manual equivalent
|
||||
if (t.ipsec) {
|
||||
const ikeMode = t.ipsec.ikeVersion === "ikev2" ? "ike2" : "ike1"
|
||||
const pfsGroup = t.ipsec.pfs ? t.ipsec.dhGroup : "none"
|
||||
// IPsec: live CHR имеет только ipsec-secret; proposal — у моков/формы
|
||||
if (t.ipsec?.encAlg && t.ipsec.authAlg) {
|
||||
const ikeMode = t.ipsec.ikeVersion === "ikev1" ? "ike1" : "ike2"
|
||||
const pfsGroup = t.ipsec.pfs ? (t.ipsec.dhGroup ?? "none") : "none"
|
||||
|
||||
lines.push("")
|
||||
lines.push("# ── IPsec (авто через ipsec-secret; ручной эквивалент) ───────")
|
||||
@@ -111,13 +127,16 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
|
||||
lines.push(` enc-algorithms=${ENC_ROS[t.ipsec.encAlg]} \\`)
|
||||
lines.push(` auth-algorithms=${AUTH_ROS[t.ipsec.authAlg]} \\`)
|
||||
lines.push(` pfs-group=${pfsGroup} \\`)
|
||||
lines.push(` lifetime=${t.ipsec.lifetime}`)
|
||||
lines.push(` lifetime=${t.ipsec.lifetime ?? "1d"}`)
|
||||
lines.push("")
|
||||
lines.push(`/ip ipsec policy add \\`)
|
||||
lines.push(` src-address=${t.localAddress !== "0.0.0.0" ? t.localAddress + "/32" : "0.0.0.0/0"} \\`)
|
||||
lines.push(` dst-address=${t.remoteAddress}/32 \\`)
|
||||
lines.push(` proposal=${t.name} \\`)
|
||||
lines.push(` tunnel=yes`)
|
||||
} else if (t.ipsec) {
|
||||
lines.push("")
|
||||
lines.push("# IPsec: peer/policy создаёт RouterOS по ipsec-secret")
|
||||
}
|
||||
|
||||
return lines.join("\n")
|
||||
@@ -126,7 +145,7 @@ function generateRosCommands(t: GreTunnel, serverById: Record<string, Server>):
|
||||
// ─── small ui helpers ────────────────────────────────────────────────────────
|
||||
|
||||
function TunnelStatus({ status }: { status: GreStatus }) {
|
||||
const s = STATUS_MAP[status]
|
||||
const s = greStatusMeta(status)
|
||||
return (
|
||||
<span className="inline-flex items-center gap-1.5 text-sm">
|
||||
<span className={`size-1.5 rounded-full ${s.dot}`} />
|
||||
@@ -164,6 +183,7 @@ interface BackendServer {
|
||||
|
||||
interface GreTunnelsApiResponse {
|
||||
tunnels: GreTunnel[]
|
||||
failures?: Array<{ serverId: string; serverName?: string; error: string }>
|
||||
}
|
||||
|
||||
function makeApiFetch(backendUrl: string) {
|
||||
@@ -237,7 +257,6 @@ export default function GrePage() {
|
||||
const [liveTunnels, setLiveTunnels] = useState<GreTunnel[]>([])
|
||||
const [dataLoading, setDataLoading] = useState(false)
|
||||
const [dataError, setDataError] = useState<string | null>(null)
|
||||
const [syncJhBusy, setSyncJhBusy] = useState(false)
|
||||
|
||||
const [pageTab, setPageTab] = useState<PageTab>("tunnels")
|
||||
const [tabFilter, setTabFilter] = useState<TabFilter>("all")
|
||||
@@ -245,6 +264,15 @@ export default function GrePage() {
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const [tunnelOpen, setTunnelOpen] = useState(false)
|
||||
const [tunnelMode, setTunnelMode] = useState<"create" | "edit">("create")
|
||||
const [editingTunnel, setEditingTunnel] = useState<GreTunnel | null>(null)
|
||||
const [pendingDelete, setPendingDelete] = useState<GreTunnel | null>(null)
|
||||
const [mutateBusy, setMutateBusy] = useState(false)
|
||||
const [liveStale, setLiveStale] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [poolOpen, setPoolOpen] = useState(false)
|
||||
const [codePreviewTunnel, setCodePreviewTunnel] = useState<GreTunnel | null>(null)
|
||||
|
||||
@@ -261,14 +289,19 @@ export default function GrePage() {
|
||||
try {
|
||||
const [backendServers, greRes] = await Promise.all([
|
||||
apiFetch<BackendServer[]>("/api/servers"),
|
||||
apiFetch<GreTunnelsApiResponse>("/api/filters/gre-tunnels"),
|
||||
apiFetch<GreTunnelsApiResponse>("/api/gre/tunnels"),
|
||||
])
|
||||
setLiveServers(backendServers.map(mapBackendToServer))
|
||||
setLiveTunnels(greRes.tunnels)
|
||||
setLiveStale(false)
|
||||
if (greRes.failures?.length) {
|
||||
toast.warning(
|
||||
`Не удалось опросить: ${greRes.failures.map((f) => f.serverName ?? f.serverId).join(", ")}`,
|
||||
)
|
||||
}
|
||||
} catch (e) {
|
||||
setDataError(e instanceof Error ? e.message : "Ошибка загрузки")
|
||||
setLiveServers([])
|
||||
setLiveTunnels([])
|
||||
setLiveStale(true)
|
||||
} finally {
|
||||
setDataLoading(false)
|
||||
}
|
||||
@@ -280,6 +313,7 @@ export default function GrePage() {
|
||||
setLiveServers([])
|
||||
setLiveTunnels([])
|
||||
setDataError(null)
|
||||
setLiveStale(false)
|
||||
})
|
||||
return
|
||||
}
|
||||
@@ -323,39 +357,186 @@ export default function GrePage() {
|
||||
[displayPools],
|
||||
)
|
||||
|
||||
const syncJhToDb = useCallback(async () => {
|
||||
if (!isLive || syncJhBusy) return
|
||||
const jh = displayServers.filter((s) => s.type === "jump-host" && s.enabled)
|
||||
if (jh.length === 0) {
|
||||
toast.info("Нет включённых Jump Host в списке серверов")
|
||||
const historyServerId = selectedServerId === ALL_SERVERS_ID ? null : selectedServerId
|
||||
const mutationsLocked = isLive && (mutateBusy || liveStale || historyRestoring)
|
||||
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/gre/revisions?serverId=${encodeURIComponent(historyServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
await apiFetch(
|
||||
`/api/gre/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
|
||||
)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadLive()
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, historyServerId, apiFetch, loadLive, loadRevisions])
|
||||
|
||||
function tunnelWriteBody(form: typeof defaultTunnelForm) {
|
||||
return {
|
||||
serverId: form.serverId,
|
||||
name: form.name.trim(),
|
||||
localAddress: form.localAddress.trim() || undefined,
|
||||
remoteAddress: form.remoteAddress.trim(),
|
||||
localInnerIp: form.localInnerIp.trim() || undefined,
|
||||
remoteInnerIp: form.remoteInnerIp.trim() || undefined,
|
||||
comment: form.comment || undefined,
|
||||
enabled: form.enabled,
|
||||
mtu: form.mtu,
|
||||
keepaliveInterval: form.keepaliveInterval,
|
||||
keepaliveRetries: form.keepaliveRetries,
|
||||
dscp: form.dscp,
|
||||
clampTcpMss: form.clampTcpMss,
|
||||
allowFastPath: form.allowFastPath,
|
||||
ipsecSecret: form.ipsecEnabled ? form.ipsecSecret : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
async function submitTunnel() {
|
||||
if (!isLive) {
|
||||
toast.info("Создание на роутер доступно только в live-режиме")
|
||||
return
|
||||
}
|
||||
setSyncJhBusy(true)
|
||||
const errors: string[] = []
|
||||
try {
|
||||
for (const s of jh) {
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/filters/sync/from-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: s.id }),
|
||||
})
|
||||
} catch (e) {
|
||||
errors.push(`${s.name}: ${e instanceof Error ? e.message : "ошибка"}`)
|
||||
}
|
||||
}
|
||||
const fresh = await apiFetch<GreTunnelsApiResponse>("/api/filters/gre-tunnels")
|
||||
setLiveTunnels(fresh.tunnels)
|
||||
if (errors.length) {
|
||||
toast.warning(`Синхронизировано JH: ${jh.length - errors.length}/${jh.length}. Ошибки: ${errors.join("; ")}`)
|
||||
} else {
|
||||
toast.success(`Правила с ${jh.length} JH записаны в БД, список GRE обновлён.`)
|
||||
}
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Ошибка после синхронизации")
|
||||
} finally {
|
||||
setSyncJhBusy(false)
|
||||
if (liveStale) {
|
||||
toast.error("Роутер недоступен — изменения заблокированы")
|
||||
return
|
||||
}
|
||||
}, [isLive, syncJhBusy, apiFetch, displayServers])
|
||||
if (!tForm.name.trim() || !tForm.serverId || !tForm.remoteAddress.trim()) {
|
||||
toast.error("Заполните имя, сервер и удалённый адрес")
|
||||
return
|
||||
}
|
||||
if (tForm.ipsecEnabled && tForm.ipsecSecret.trim().length < 8) {
|
||||
toast.error("Для IPsec нужен PSK не короче 8 символов")
|
||||
return
|
||||
}
|
||||
setMutateBusy(true)
|
||||
try {
|
||||
if (tunnelMode === "edit" && editingTunnel) {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "PATCH",
|
||||
body: JSON.stringify({
|
||||
...tunnelWriteBody(tForm),
|
||||
rosId: editingTunnel.id,
|
||||
name: editingTunnel.name,
|
||||
}),
|
||||
})
|
||||
toast.success(`Туннель ${tForm.name} обновлён`)
|
||||
} else {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "POST",
|
||||
body: JSON.stringify(tunnelWriteBody(tForm)),
|
||||
})
|
||||
toast.success(`Туннель ${tForm.name} создан`)
|
||||
}
|
||||
setTunnelOpen(false)
|
||||
setEditingTunnel(null)
|
||||
await loadLive()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось сохранить туннель", { description: String(err) })
|
||||
} finally {
|
||||
setMutateBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function toggleTunnel(t: GreTunnel) {
|
||||
if (!isLive || mutationsLocked) return
|
||||
setMutateBusy(true)
|
||||
try {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "PATCH",
|
||||
body: JSON.stringify({
|
||||
serverId: t.serverId,
|
||||
rosId: t.id,
|
||||
name: t.name,
|
||||
enabled: !t.enabled,
|
||||
remoteAddress: t.remoteAddress,
|
||||
}),
|
||||
})
|
||||
toast.success(t.enabled ? `Выключен ${t.name}` : `Включён ${t.name}`)
|
||||
await loadLive()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось изменить туннель", { description: String(err) })
|
||||
} finally {
|
||||
setMutateBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function confirmDeleteTunnel() {
|
||||
const t = pendingDelete
|
||||
if (!t || !isLive) return
|
||||
setMutateBusy(true)
|
||||
try {
|
||||
await apiFetch("/api/gre/tunnels", {
|
||||
method: "DELETE",
|
||||
body: JSON.stringify({ serverId: t.serverId, rosId: t.id, name: t.name }),
|
||||
})
|
||||
toast.success(`Удалён ${t.name}`)
|
||||
setPendingDelete(null)
|
||||
await loadLive()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось удалить туннель", { description: String(err) })
|
||||
} finally {
|
||||
setMutateBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
function openCreateTunnel() {
|
||||
setTunnelMode("create")
|
||||
setEditingTunnel(null)
|
||||
setTForm({
|
||||
...defaultTunnelForm,
|
||||
serverId: selectedServerId === ALL_SERVERS_ID ? "" : selectedServerId,
|
||||
})
|
||||
setTunnelOpen(true)
|
||||
}
|
||||
|
||||
function openEditTunnel(t: GreTunnel) {
|
||||
setTunnelMode("edit")
|
||||
setEditingTunnel(t)
|
||||
setTForm({
|
||||
...defaultTunnelForm,
|
||||
name: t.name,
|
||||
serverId: t.serverId,
|
||||
localAddress: t.localAddress === "0.0.0.0" ? "" : t.localAddress,
|
||||
remoteAddress: t.remoteAddress,
|
||||
poolId: t.poolId === "live" ? "" : t.poolId,
|
||||
localInnerIp: t.localInnerIp,
|
||||
remoteInnerIp: t.remoteInnerIp,
|
||||
comment: t.comment,
|
||||
enabled: t.enabled,
|
||||
ipsecEnabled: !!t.ipsec,
|
||||
ipsecSecret: t.ipsec?.secret ?? "",
|
||||
mtu: t.mtu,
|
||||
keepaliveInterval: t.keepaliveInterval,
|
||||
keepaliveRetries: t.keepaliveRetries,
|
||||
dscp: String(t.dscp),
|
||||
clampTcpMss: t.clampTcpMss,
|
||||
allowFastPath: t.allowFastPath,
|
||||
})
|
||||
setTunnelOpen(true)
|
||||
}
|
||||
|
||||
useEffect(() => {
|
||||
if (dataError) toast.error(dataError)
|
||||
@@ -403,6 +584,14 @@ export default function GrePage() {
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && dataLoading && displayServers.length === 0}
|
||||
banner={
|
||||
isLive && liveStale ? (
|
||||
<div className="shrink-0 border-b border-amber-500/30 bg-amber-500/10 px-6 py-2.5 text-xs text-amber-700 dark:text-amber-400 flex items-center gap-2">
|
||||
<TriangleAlertIcon className="size-3.5 shrink-0" />
|
||||
Роутер недоступен — показан кэш. Изменения заблокированы, пока не удастся прочитать CHR.
|
||||
</div>
|
||||
) : null
|
||||
}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "GRE-туннели" }]}
|
||||
@@ -422,14 +611,17 @@ export default function GrePage() {
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => { void syncJhToDb() }}
|
||||
disabled={!isLive || syncJhBusy || dataLoading}
|
||||
title="Загрузить правила фильтрации с каждого Jump Host в БД и обновить опрос GRE"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={!isLive || !historyServerId || dataLoading}
|
||||
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
|
||||
>
|
||||
<DatabaseIcon className={cn("size-4", syncJhBusy && "animate-pulse")} />
|
||||
JH → БД
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
<Button size="sm" onClick={() => { setTForm(defaultTunnelForm); setTunnelOpen(true) }}>
|
||||
<Button size="sm" onClick={openCreateTunnel} disabled={mutateBusy}>
|
||||
<PlusIcon className="size-4" />Добавить туннель
|
||||
</Button>
|
||||
</>
|
||||
@@ -521,6 +713,10 @@ export default function GrePage() {
|
||||
servers={displayServers}
|
||||
pools={displayPools}
|
||||
onCodePreview={setCodePreviewTunnel}
|
||||
onEdit={openEditTunnel}
|
||||
onToggle={(t) => { void toggleTunnel(t) }}
|
||||
onDelete={setPendingDelete}
|
||||
mutationsLocked={mutationsLocked}
|
||||
/>
|
||||
</DataPageCard>
|
||||
)}
|
||||
@@ -548,7 +744,7 @@ export default function GrePage() {
|
||||
<div className="flex flex-wrap gap-2">
|
||||
{poolTunnels.map((t, tunnelIndex) => (
|
||||
<div key={`${t.id}:${t.serverId}:${t.name}:${tunnelIndex}`} className="flex items-center gap-2 border border-border rounded-md px-3 py-1.5 bg-muted/30 text-xs">
|
||||
<span className={`size-1.5 rounded-full ${STATUS_MAP[t.status].dot}`} />
|
||||
<span className={`size-1.5 rounded-full ${greStatusMeta(t.status).dot}`} />
|
||||
<span className="font-mono font-medium">{t.name}</span>
|
||||
<span className="text-muted-foreground">{t.localInnerIp} ↔ {t.remoteInnerIp}</span>
|
||||
{t.ipsec && <LockIcon className="size-3 text-emerald-400" />}
|
||||
@@ -609,8 +805,8 @@ export default function GrePage() {
|
||||
codePreviewTunnel ? (
|
||||
<div className="flex flex-wrap gap-3 text-xs shrink-0">
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className={`size-1.5 rounded-full ${STATUS_MAP[codePreviewTunnel.status].dot}`} />
|
||||
{STATUS_MAP[codePreviewTunnel.status].label}
|
||||
<span className={`size-1.5 rounded-full ${greStatusMeta(codePreviewTunnel.status).dot}`} />
|
||||
{greStatusMeta(codePreviewTunnel.status).label}
|
||||
</span>
|
||||
<span className="text-muted-foreground">·</span>
|
||||
<span>{serverById[codePreviewTunnel.serverId]?.name}</span>
|
||||
@@ -625,7 +821,7 @@ export default function GrePage() {
|
||||
<span className="text-muted-foreground">·</span>
|
||||
<span className="flex items-center gap-1 text-success">
|
||||
<LockIcon className="size-3" />
|
||||
IPsec {IKE_LABELS[codePreviewTunnel.ipsec.ikeVersion]}
|
||||
IPsec {codePreviewTunnel.ipsec.ikeVersion ? IKE_LABELS[codePreviewTunnel.ipsec.ikeVersion] : "PSK"}
|
||||
</span>
|
||||
</>
|
||||
) : null}
|
||||
@@ -638,18 +834,18 @@ export default function GrePage() {
|
||||
<Sheet open={tunnelOpen} onOpenChange={setTunnelOpen}>
|
||||
<SheetContent side="right" className="w-full sm:max-w-lg flex flex-col gap-0 p-0">
|
||||
<SheetHeader className="px-6 pt-6 pb-4 border-b shrink-0">
|
||||
<SheetTitle>Новый GRE-туннель</SheetTitle>
|
||||
<SheetDescription>RouterOS 7.20+ · /interface gre add</SheetDescription>
|
||||
<SheetTitle>{tunnelMode === "edit" ? "Редактировать GRE-туннель" : "Новый GRE-туннель"}</SheetTitle>
|
||||
<SheetDescription>RouterOS 7.20+ · /interface gre {tunnelMode === "edit" ? "set" : "add"}</SheetDescription>
|
||||
</SheetHeader>
|
||||
|
||||
<div className="flex-1 overflow-y-auto px-6 py-5 flex flex-col gap-5">
|
||||
<div className="flex flex-col gap-4">
|
||||
<SectionTitle>Основные</SectionTitle>
|
||||
<FormField label="Имя интерфейса" required hint="Только латиница, цифры и дефис, например gre-msk-spb">
|
||||
<Input className="font-mono" placeholder="gre-msk-spb" value={tForm.name} onChange={(e) => setT("name", e.target.value)} />
|
||||
<Input className="font-mono" placeholder="gre-msk-spb" value={tForm.name} disabled={tunnelMode === "edit"} onChange={(e) => setT("name", e.target.value)} />
|
||||
</FormField>
|
||||
<FormField label="Сервер (MikroTik)" required>
|
||||
<select value={tForm.serverId} onChange={(e) => setT("serverId", e.target.value)}
|
||||
<select value={tForm.serverId} onChange={(e) => setT("serverId", e.target.value)} disabled={tunnelMode === "edit"}
|
||||
className="h-8 w-full rounded-lg border border-input bg-background px-2.5 text-sm text-foreground outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50">
|
||||
<option value="" disabled>Выбрать сервер…</option>
|
||||
{displayServers.map((s) => <option key={s.id} value={s.id}>{s.name} ({s.site})</option>)}
|
||||
@@ -676,7 +872,7 @@ export default function GrePage() {
|
||||
|
||||
<div className="flex flex-col gap-4">
|
||||
<SectionTitle>Внутренний IP</SectionTitle>
|
||||
<FormField label="IP-пул" required hint="Из какого пула выделяется /30-блок">
|
||||
<FormField label="IP-пул" hint="Необязательно — внутренний IP можно указать вручную">
|
||||
<select value={tForm.poolId} onChange={(e) => setT("poolId", e.target.value)}
|
||||
className="h-8 w-full rounded-lg border border-input bg-background px-2.5 text-sm text-foreground outline-none focus-visible:border-ring focus-visible:ring-3 focus-visible:ring-ring/50">
|
||||
<option value="" disabled>Выбрать пул…</option>
|
||||
@@ -798,7 +994,9 @@ export default function GrePage() {
|
||||
|
||||
<SheetFooter className="px-6 py-4 border-t shrink-0 flex-row gap-2">
|
||||
<SheetClose render={<Button variant="outline" className="flex-1" />}>Отмена</SheetClose>
|
||||
<Button className="flex-1" onClick={() => setTunnelOpen(false)}>Создать туннель</Button>
|
||||
<Button className="flex-1" onClick={() => void submitTunnel()} disabled={mutateBusy}>
|
||||
{tunnelMode === "edit" ? "Сохранить" : "Создать туннель"}
|
||||
</Button>
|
||||
</SheetFooter>
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
@@ -845,6 +1043,39 @@ export default function GrePage() {
|
||||
</SheetFooter>
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История GRE"
|
||||
itemLabel="туннелей"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
|
||||
<AlertDialog open={!!pendingDelete} onOpenChange={(v) => { if (!v) setPendingDelete(null) }}>
|
||||
<AlertDialogContent size="default">
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogMedia className="bg-destructive/10 text-destructive">
|
||||
<Trash2Icon />
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Удалить GRE-туннель?</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
{pendingDelete
|
||||
? `${pendingDelete.name} на сервере ${serverById[pendingDelete.serverId]?.name ?? pendingDelete.serverId}. Будут удалены интерфейс и связанный /ip/address.`
|
||||
: null}
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel onClick={() => setPendingDelete(null)}>Отмена</AlertDialogCancel>
|
||||
<AlertDialogAction variant="destructive" onClick={() => void confirmDeleteTunnel()}>
|
||||
Удалить
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -67,6 +67,7 @@ import {
|
||||
CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon,
|
||||
} from "lucide-react"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
|
||||
import Link from "next/link"
|
||||
import { Flag } from "@/components/flag"
|
||||
|
||||
@@ -820,9 +821,15 @@ function ServicePathList({
|
||||
{paths.map((p) => {
|
||||
const rowKey = servicePathKey(p)
|
||||
const via = servers.find((s) => s.id === p.viaId)
|
||||
const viaLabel = via?.site || p.viaName
|
||||
const en = servers.find((s) => s.id === p.enId)
|
||||
const svc = services.find((s) => s.id === p.serviceId)
|
||||
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId)
|
||||
const label = formatServicePathLabel(p, viaMode, {
|
||||
viaName: via?.name,
|
||||
viaSite: via?.site,
|
||||
enName: en?.name,
|
||||
serviceLabel: svc?.label,
|
||||
})
|
||||
const title = formatServicePathTitle(label, svc?.label ?? p.serviceId)
|
||||
const active = Boolean(
|
||||
highlight
|
||||
&& highlight.viaId === p.viaId
|
||||
@@ -833,13 +840,14 @@ function ServicePathList({
|
||||
<button
|
||||
key={rowKey}
|
||||
type="button"
|
||||
title={title}
|
||||
onClick={() => onToggle(p)}
|
||||
className={cn(
|
||||
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
|
||||
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
|
||||
)}
|
||||
>
|
||||
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span>
|
||||
<span className="font-mono truncate min-w-0">{label}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums shrink-0">
|
||||
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
|
||||
</span>
|
||||
@@ -1216,6 +1224,9 @@ export default function NetworkMapPage() {
|
||||
// ── Interaction ─────────────────────────────────────────────────────────────
|
||||
const [selected, setSelected] = useState<Server | null>(null)
|
||||
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
|
||||
const liveSelectedService = selectedService
|
||||
? (mapServices.find((s) => s.id === selectedService.id) ?? selectedService)
|
||||
: null
|
||||
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
|
||||
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
|
||||
const [hoveredId, setHoveredId] = useState<string | null>(null)
|
||||
@@ -2702,16 +2713,16 @@ export default function NetworkMapPage() {
|
||||
})()}
|
||||
</div>
|
||||
</>
|
||||
) : selectedService ? (
|
||||
) : liveSelectedService ? (
|
||||
<>
|
||||
<div className="flex items-start gap-2 px-4 py-3 border-b">
|
||||
<div className="mt-0.5">
|
||||
<ServiceBrandIcon label={selectedService.label} size={22} />
|
||||
<ServiceBrandIcon label={liveSelectedService.label} size={22} />
|
||||
</div>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-mono font-semibold text-sm truncate">{selectedService.label}</p>
|
||||
<p className="font-mono font-semibold text-sm truncate">{liveSelectedService.label}</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">
|
||||
Конечный сервис · {selectedService.category}
|
||||
Конечный сервис · {liveSelectedService.category}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
@@ -2726,15 +2737,15 @@ export default function NetworkMapPage() {
|
||||
<div className="flex flex-col gap-0">
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Доля окна</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedService.share)}</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(liveSelectedService.share)}</span>
|
||||
</div>
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Скорость</span>
|
||||
<span className="text-xs font-mono font-medium">
|
||||
{formatNetflowRate({
|
||||
bytes: selectedService.bytes,
|
||||
bps: selectedService.bps,
|
||||
bpsFwd: selectedService.bps,
|
||||
bytes: liveSelectedService.bytes,
|
||||
bps: liveSelectedService.bps,
|
||||
bpsFwd: liveSelectedService.bps,
|
||||
bpsRev: 0,
|
||||
})}
|
||||
</span>
|
||||
@@ -2742,34 +2753,34 @@ export default function NetworkMapPage() {
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{visibleServiceEdges.filter((e) => e.toId === selectedService.id).map((e) => {
|
||||
<div className="flex flex-col gap-3">
|
||||
{visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
|
||||
const src = mapServers.find((s) => s.id === e.fromId)
|
||||
const enPaths = mapServicePaths
|
||||
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)
|
||||
return (
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex flex-col gap-1.5">
|
||||
<div className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
</div>
|
||||
<ServicePathList
|
||||
paths={enPaths}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.serviceId === selectedService.id)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
) : selected ? (
|
||||
@@ -3002,7 +3013,11 @@ export default function NetworkMapPage() {
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.viaId === selected.id || p.enId === selected.id)
|
||||
.filter((p) => (
|
||||
selected.type === "exit-node"
|
||||
? p.enId === selected.id
|
||||
: p.viaId === selected.id
|
||||
))
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
|
||||
@@ -122,11 +122,17 @@ interface BackendBfdSession {
|
||||
packetsRx: number; packetsTx: number; stateChanges: number
|
||||
}
|
||||
|
||||
interface BackendOspfRoute {
|
||||
id: string; serverId: number; serverName: string; serverSite: string
|
||||
destination: string; type: OspfRoute["type"]; cost: number; nextHop: string; via: string; area: string
|
||||
}
|
||||
|
||||
interface BackendOspfAll {
|
||||
neighbors: BackendNeighbor[]
|
||||
interfaces: BackendInterface[]
|
||||
instances: BackendInstance[]
|
||||
bfdSessions: BackendBfdSession[]
|
||||
routes?: BackendOspfRoute[]
|
||||
}
|
||||
|
||||
function isRefInterfaceName(name: string): boolean {
|
||||
@@ -213,6 +219,22 @@ function backendToBfdSession(b: BackendBfdSession): BfdSession {
|
||||
}
|
||||
}
|
||||
|
||||
function backendToRoute(b: BackendOspfRoute): OspfRoute {
|
||||
const allowed: OspfRoute["type"][] = ["O", "O IA", "O E1", "O E2"]
|
||||
const type = allowed.includes(b.type) ? b.type : "O"
|
||||
return {
|
||||
id: `${b.serverId}-${b.id}`,
|
||||
destination: b.destination,
|
||||
type,
|
||||
cost: b.cost,
|
||||
nextHop: b.nextHop,
|
||||
via: b.via,
|
||||
serverId: String(b.serverId),
|
||||
serverLabel: b.serverName,
|
||||
area: b.area || "—",
|
||||
}
|
||||
}
|
||||
|
||||
// ─── mock data ────────────────────────────────────────────────────────────────
|
||||
|
||||
const COST_STEP = 10
|
||||
@@ -1172,7 +1194,7 @@ export default function OspfPage() {
|
||||
}, [isLive, backendUrl, fetchTick])
|
||||
|
||||
// Derive frontend types from backend data or use mocks
|
||||
const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions } = useMemo(() => {
|
||||
const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes } = useMemo(() => {
|
||||
if (isLive && liveData) {
|
||||
// Build interface→cost map for neighbor cost lookup
|
||||
const ifaceMap = new Map<string, number>()
|
||||
@@ -1185,6 +1207,7 @@ export default function OspfPage() {
|
||||
.filter((item) => !isRefInterfaceName(item.interfaceName))
|
||||
const neighbors = liveData.neighbors.map(b => backendToNeighbor(b, ifaceMap))
|
||||
const bfdSessions = (liveData.bfdSessions ?? []).map(backendToBfdSession)
|
||||
const routes = (liveData.routes ?? []).map(backendToRoute)
|
||||
|
||||
// Build routerIds from instances
|
||||
const routerIds: Record<string, string> = {}
|
||||
@@ -1195,7 +1218,7 @@ export default function OspfPage() {
|
||||
}
|
||||
|
||||
const { nodes: graphNodes, edges: graphEdges } = buildLiveGraph(neighbors)
|
||||
return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions }
|
||||
return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes }
|
||||
}
|
||||
if (isLive) {
|
||||
return {
|
||||
@@ -1205,6 +1228,7 @@ export default function OspfPage() {
|
||||
graphEdges: [],
|
||||
routerIds: {},
|
||||
bfdSessions: [],
|
||||
routes: [],
|
||||
}
|
||||
}
|
||||
return {
|
||||
@@ -1214,6 +1238,7 @@ export default function OspfPage() {
|
||||
graphEdges: MOCK_GRAPH_EDGES,
|
||||
routerIds: MOCK_ROUTER_IDS,
|
||||
bfdSessions: MOCK_BFD,
|
||||
routes: MOCK_ROUTES,
|
||||
}
|
||||
}, [isLive, liveData])
|
||||
|
||||
@@ -1257,6 +1282,7 @@ export default function OspfPage() {
|
||||
const displayItems = filterServerId === ALL_SERVERS_ID ? items : items.filter(i => i.routerKey === filterServerId)
|
||||
const displayNeighbors = filterServerId === ALL_SERVERS_ID ? neighbors : neighbors.filter(n => n.localRouter === filterServerId)
|
||||
const displayBfdSessions = filterServerId === ALL_SERVERS_ID ? bfdSessions : bfdSessions.filter(b => b.serverId === filterServerId)
|
||||
const displayRoutes = filterServerId === ALL_SERVERS_ID ? routes : routes.filter(r => r.serverId === filterServerId)
|
||||
|
||||
const ospfRailItems = useMemo<ServerTileItem[]>(() => (
|
||||
ospfServers.map((s) => {
|
||||
@@ -1398,7 +1424,7 @@ export default function OspfPage() {
|
||||
routerIds={routerIds}
|
||||
/>
|
||||
)}
|
||||
{activeTab === "routes" && <RoutesTab routes={isLive ? [] : MOCK_ROUTES} />}
|
||||
{activeTab === "routes" && <RoutesTab routes={displayRoutes} />}
|
||||
{activeTab === "bfd" && <BfdTab sessions={displayBfdSessions} />}
|
||||
|
||||
</div>
|
||||
|
||||
@@ -18,9 +18,12 @@ import { Flag } from "@/components/flag"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { servers as mockServers, type Server } from "@/lib/data"
|
||||
import { PlusIcon, SaveIcon, TrashIcon, SearchIcon, XIcon, PencilIcon, CheckIcon, AlertCircleIcon } from "lucide-react"
|
||||
import { PlusIcon, TrashIcon, SearchIcon, XIcon, PencilIcon, CheckIcon, AlertCircleIcon, RefreshCwIcon, HistoryIcon, AlertTriangleIcon } from "lucide-react"
|
||||
import { toast } from "sonner"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import { type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
interface BackendServer {
|
||||
@@ -364,7 +367,7 @@ export default function RecursiveRoutesPage() {
|
||||
const [servers, setServers] = useState<Server[]>([])
|
||||
const [selectedServerId, setSelectedServerId] = useState<string>("")
|
||||
const [rows, setRows] = useState<RecursiveRouteRow[]>([])
|
||||
const [busy, setBusy] = useState<"load" | "save" | "from" | "to" | null>(null)
|
||||
const [busy, setBusy] = useState<"load" | "apply" | null>(null)
|
||||
const [search, setSearch] = useState("")
|
||||
const [sheetOpen, setSheetOpen] = useState(false)
|
||||
const [sheetMode, setSheetMode] = useState<"create" | "edit">("create")
|
||||
@@ -373,6 +376,11 @@ export default function RecursiveRoutesPage() {
|
||||
const [gatewayOptions, setGatewayOptions] = useState<GatewayOption[]>([])
|
||||
const [expandedGroupKey, setExpandedGroupKey] = useState<string | null>(null)
|
||||
const [opError, setOpError] = useState<string | null>(null)
|
||||
const [liveStale, setLiveStale] = useState(false)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
/** В live не дергаем API с id мока (srv1…) пока не подтянули /api/servers */
|
||||
const [liveServerListReady, setLiveServerListReady] = useState(false)
|
||||
|
||||
@@ -407,10 +415,13 @@ export default function RecursiveRoutesPage() {
|
||||
setOpError(null)
|
||||
setBusy("load")
|
||||
try {
|
||||
const res = await apiFetch<{ routes: RecursiveRouteRow[] }>(`/api/recursive-routes?serverId=${selectedServerId}`)
|
||||
const res = await apiFetch<{ routes: RecursiveRouteRow[]; stale?: boolean }>(
|
||||
`/api/recursive-routes?serverId=${selectedServerId}`,
|
||||
)
|
||||
setRows(res.routes)
|
||||
setLiveStale(Boolean(res.stale))
|
||||
} catch (e) {
|
||||
setRows([])
|
||||
setLiveStale(true)
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось загрузить маршруты")
|
||||
} finally {
|
||||
setBusy(null)
|
||||
@@ -437,56 +448,65 @@ export default function RecursiveRoutesPage() {
|
||||
void loadGateways()
|
||||
}, [loadGateways])
|
||||
|
||||
const saveToDb = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
const applyRoutes = useCallback(async (next: RecursiveRouteRow[]) => {
|
||||
if (!isLive || !selectedServerId || liveStale) return
|
||||
const prev = rows
|
||||
setRows(next)
|
||||
setOpError(null)
|
||||
setBusy("save")
|
||||
setBusy("apply")
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/recursive-routes", {
|
||||
const res = await apiFetch<{ ok: boolean; routes?: RecursiveRouteRow[] }>("/api/recursive-routes", {
|
||||
method: "PUT",
|
||||
body: JSON.stringify({ serverId: selectedServerId, routes: rows }),
|
||||
body: JSON.stringify({ serverId: selectedServerId, routes: next }),
|
||||
})
|
||||
await loadRoutes()
|
||||
setRows(res.routes ?? next)
|
||||
setLiveStale(false)
|
||||
toast.success("Маршруты применены на роутер")
|
||||
} catch (e) {
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось сохранить маршруты в БД")
|
||||
setRows(prev)
|
||||
const msg = e instanceof Error ? e.message : "Не удалось применить маршруты на роутер"
|
||||
setOpError(msg)
|
||||
toast.error(msg)
|
||||
} finally {
|
||||
setBusy(null)
|
||||
}
|
||||
}, [isLive, selectedServerId, rows, apiFetch, loadRoutes])
|
||||
}, [isLive, selectedServerId, liveStale, rows, apiFetch])
|
||||
|
||||
const syncFromRouter = useCallback(async () => {
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setOpError(null)
|
||||
setBusy("from")
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/recursive-routes/sync/from-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
})
|
||||
await loadRoutes()
|
||||
const res = await apiFetch<{ revisions: ConfigRevisionDto[] }>(
|
||||
`/api/recursive-routes/revisions?serverId=${encodeURIComponent(selectedServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (e) {
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось синхронизировать маршруты с роутера")
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось загрузить историю")
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setBusy(null)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch, loadRoutes])
|
||||
|
||||
const syncToRouter = useCallback(async () => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setOpError(null)
|
||||
setBusy("to")
|
||||
try {
|
||||
await apiFetch<{ ok: boolean }>("/api/recursive-routes/sync/to-router", {
|
||||
method: "POST",
|
||||
body: JSON.stringify({ serverId: selectedServerId }),
|
||||
})
|
||||
} catch (e) {
|
||||
setOpError(e instanceof Error ? e.message : "Не удалось применить маршруты на роутер")
|
||||
} finally {
|
||||
setBusy(null)
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !selectedServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
const res = await apiFetch<{ ok: boolean; routes?: RecursiveRouteRow[] }>(
|
||||
`/api/recursive-routes/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: selectedServerId }) },
|
||||
)
|
||||
setRows(res.routes ?? [])
|
||||
setLiveStale(false)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadRevisions()
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось откатить")
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, selectedServerId, apiFetch, loadRevisions])
|
||||
|
||||
function groupKeyOf(row: RecursiveRouteRow): string {
|
||||
return row.dstAddress.trim().toLowerCase()
|
||||
}
|
||||
@@ -529,22 +549,26 @@ export default function RecursiveRoutesPage() {
|
||||
disabled: false,
|
||||
country: ep.country || inferCountry(ep.gateway) || "",
|
||||
})
|
||||
let next: RecursiveRouteRow[]
|
||||
if (sheetMode === "create") {
|
||||
const base = `new-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
|
||||
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
|
||||
setRows(prev => [...prev, ...expanded])
|
||||
next = [...rows, ...expanded]
|
||||
} else if (editingGroupKey) {
|
||||
setRows(prev => {
|
||||
const kept = prev.filter(r => groupKeyOf(r) !== editingGroupKey)
|
||||
const base = `edit-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
|
||||
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
|
||||
return [...kept, ...expanded]
|
||||
})
|
||||
const kept = rows.filter(r => groupKeyOf(r) !== editingGroupKey)
|
||||
const base = `edit-${Date.now()}-${Math.random().toString(36).slice(2, 6)}`
|
||||
const expanded = v.endpoints.map((ep, i) => toRow(ep, `${base}-${i}`))
|
||||
next = [...kept, ...expanded]
|
||||
} else {
|
||||
setSheetOpen(false)
|
||||
return
|
||||
}
|
||||
setSheetOpen(false)
|
||||
void applyRoutes(next)
|
||||
}
|
||||
|
||||
const currentServer = servers.find(s => s.id === selectedServerId)
|
||||
const mutationsLocked = !isLive || busy !== null || liveStale
|
||||
const rrRailItems = useMemo<ServerTileItem[]>(() => (
|
||||
servers.map((s) => ({
|
||||
id: s.id,
|
||||
@@ -602,16 +626,33 @@ export default function RecursiveRoutesPage() {
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button variant="outline" size="sm" onClick={syncFromRouter} disabled={!isLive || busy !== null}>
|
||||
{busy === "from" ? "Синхронизация..." : "Router => DB"}
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" onClick={syncToRouter} disabled={!isLive || busy !== null}>
|
||||
{busy === "to" ? "Применение..." : "DB => Router"}
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" onClick={saveToDb} disabled={!isLive || busy !== null}>
|
||||
<SaveIcon className="size-4" />Сохранить в БД
|
||||
</Button>
|
||||
<Button size="sm" onClick={openCreate} disabled={!isLive || busy !== null}>
|
||||
{isLive && (
|
||||
<>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => void loadRoutes()}
|
||||
disabled={busy !== null}
|
||||
title="Прочитать маршруты с роутера"
|
||||
>
|
||||
<RefreshCwIcon className={cn("size-4", busy === "load" && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
disabled={busy !== null}
|
||||
>
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
<Button size="sm" onClick={openCreate} disabled={mutationsLocked}>
|
||||
<PlusIcon className="size-4" />Добавить
|
||||
</Button>
|
||||
</>
|
||||
@@ -643,6 +684,12 @@ export default function RecursiveRoutesPage() {
|
||||
{opError}
|
||||
</div>
|
||||
)}
|
||||
{liveStale && (
|
||||
<div className="w-full text-xs text-amber-700 dark:text-amber-400 bg-amber-500/10 border border-amber-500/20 rounded-md px-3 py-2 flex items-center gap-2">
|
||||
<AlertTriangleIcon className="size-3.5 shrink-0" />
|
||||
Роутер недоступен — показан кэш. Изменения заблокированы.
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
}
|
||||
>
|
||||
@@ -679,10 +726,13 @@ export default function RecursiveRoutesPage() {
|
||||
expandedKey={expandedGroupKey}
|
||||
onExpandedChange={setExpandedGroupKey}
|
||||
onEdit={openEdit}
|
||||
onDelete={(g) => setRows((prev) => prev.filter((r) => groupKeyOf(r) !== g.key))}
|
||||
onDelete={(g) => {
|
||||
if (mutationsLocked) return
|
||||
void applyRoutes(rows.filter((r) => groupKeyOf(r) !== g.key))
|
||||
}}
|
||||
/>
|
||||
<button onClick={openCreate}
|
||||
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t">
|
||||
<button onClick={openCreate} disabled={mutationsLocked}
|
||||
className="w-full flex items-center gap-2 px-5 py-2 text-xs text-muted-foreground hover:text-foreground hover:bg-muted/20 transition-colors border-t disabled:opacity-50">
|
||||
<PlusIcon className="size-3.5" />
|
||||
Добавить маршрут
|
||||
</button>
|
||||
@@ -698,6 +748,17 @@ export default function RecursiveRoutesPage() {
|
||||
onClose={() => setSheetOpen(false)}
|
||||
gateways={gatewayOptions}
|
||||
/>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История маршрутов"
|
||||
itemLabel="маршрутов"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,558 @@
|
||||
"use client"
|
||||
|
||||
import { Suspense, useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { useSearchParams } from "next/navigation"
|
||||
import {
|
||||
ActivityIcon,
|
||||
DatabaseIcon,
|
||||
GaugeIcon,
|
||||
ServerIcon,
|
||||
UsersIcon,
|
||||
} from "lucide-react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { SegmentedControl } from "@/components/form-kit"
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector"
|
||||
import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart"
|
||||
import { DimensionSelect, PivotDimSelect } from "@/components/statistics/dimension-select"
|
||||
import { SliceChips } from "@/components/statistics/slice-chips"
|
||||
import { BreakdownDashboard } from "@/components/statistics/breakdown-dashboard"
|
||||
import { StatisticsPivotGrid } from "@/components/statistics/statistics-pivot-grid"
|
||||
import {
|
||||
StatisticsBreakdownDataGrid,
|
||||
type StatisticsSliceKind,
|
||||
} from "@/components/data-grids/statistics-breakdown-data-grid"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import type { Filter } from "@/components/reui/filters"
|
||||
import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields"
|
||||
import {
|
||||
isStatisticsPivotDim,
|
||||
isStatisticsSliceKind,
|
||||
STATISTICS_DIMS,
|
||||
} from "@/lib/statistics-dims"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
|
||||
import {
|
||||
getStatistics,
|
||||
getStatisticsPivot,
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsQuery,
|
||||
} from "@/shared/api/statistics"
|
||||
import type { StatisticsBreakdownRow, StatisticsPivotDim } from "@mmapp/contracts/statistics"
|
||||
|
||||
/**
|
||||
* BI-куб трафика: критерий → остальные разрезы + pivot.
|
||||
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
|
||||
* · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/solution-analytics-8
|
||||
* · https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/data-grid
|
||||
*/
|
||||
|
||||
const EMPTY: StatisticsDto = {
|
||||
from: "",
|
||||
to: "",
|
||||
grain: "day",
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "",
|
||||
topService: "",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
|
||||
const EMPTY_PIVOT: StatisticsPivotDto = {
|
||||
rowDim: "country",
|
||||
colDim: "service",
|
||||
metric: "bytes",
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
|
||||
interface CubeSlices {
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: string
|
||||
serverId?: string
|
||||
userId?: string
|
||||
iface?: string
|
||||
}
|
||||
|
||||
const SLICE_KEYS = ["country", "service", "asn", "serverId", "userId", "iface"] as const
|
||||
|
||||
function readRange(sp: URLSearchParams): DateRangeYmd {
|
||||
const from = sp.get("from")
|
||||
const to = sp.get("to")
|
||||
if (from && to && from <= to) return { from, to }
|
||||
return rangeForPreset("7d")
|
||||
}
|
||||
|
||||
function readDim(sp: URLSearchParams): StatisticsSliceKind {
|
||||
const t = sp.get("dim") ?? sp.get("tab")
|
||||
return t && isStatisticsSliceKind(t) ? t : "users"
|
||||
}
|
||||
|
||||
function readView(sp: URLSearchParams): "explore" | "pivot" {
|
||||
return sp.get("view") === "pivot" ? "pivot" : "explore"
|
||||
}
|
||||
|
||||
function readPlanes(sp: URLSearchParams): "unique" | "all" {
|
||||
return sp.get("planes") === "all" ? "all" : "unique"
|
||||
}
|
||||
|
||||
function readPivotDim(sp: URLSearchParams, key: string, fallback: StatisticsPivotDim): StatisticsPivotDim {
|
||||
const v = sp.get(key)
|
||||
return v && isStatisticsPivotDim(v) ? v : fallback
|
||||
}
|
||||
|
||||
function readSlices(sp: URLSearchParams): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const key of SLICE_KEYS) {
|
||||
const v = sp.get(key)?.trim()
|
||||
if (v) next[key] = v
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function slicesToFilters(slices: CubeSlices): Filter[] {
|
||||
return SLICE_KEYS.flatMap((key) => {
|
||||
const val = slices[key]
|
||||
if (!val) return []
|
||||
return [{ id: key, field: key, operator: "is", values: [val] }]
|
||||
})
|
||||
}
|
||||
|
||||
function filtersToSlices(filters: Filter[]): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const f of filters) {
|
||||
const raw = String(f.values[0] ?? "").trim()
|
||||
if (!raw) continue
|
||||
if (f.field === "country") next.country = raw.toUpperCase().slice(0, 2)
|
||||
else if (f.field === "service") next.service = raw
|
||||
else if (f.field === "asn") next.asn = raw.replace(/[^\d]/g, "")
|
||||
else if (f.field === "serverId") next.serverId = raw
|
||||
else if (f.field === "userId") next.userId = raw
|
||||
else if (f.field === "iface") next.iface = raw
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function toQuery(range: DateRangeYmd, slices: CubeSlices, planes: "unique" | "all"): StatisticsQuery {
|
||||
const serverId = slices.serverId ? Number(slices.serverId) : undefined
|
||||
const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined
|
||||
return {
|
||||
from: range.from,
|
||||
to: range.to,
|
||||
serverId: Number.isFinite(serverId) && (serverId ?? 0) > 0 ? serverId : undefined,
|
||||
userId: slices.userId,
|
||||
iface: slices.iface,
|
||||
country: slices.country && slices.country.length === 2 ? slices.country : undefined,
|
||||
service: slices.service,
|
||||
asn: Number.isFinite(asn) ? asn : undefined,
|
||||
planes,
|
||||
}
|
||||
}
|
||||
|
||||
function selectedIdForKind(kind: StatisticsSliceKind, slices: CubeSlices): string | undefined {
|
||||
if (kind === "users") return slices.userId
|
||||
if (kind === "servers") return slices.serverId
|
||||
if (kind === "countries") return slices.country
|
||||
if (kind === "services") return slices.service
|
||||
if (kind === "asns") return slices.asn
|
||||
if (kind === "interfaces" && slices.serverId && slices.iface) {
|
||||
return `${slices.serverId}:${slices.iface}`
|
||||
}
|
||||
if (kind === "interfaces") return slices.iface
|
||||
return undefined
|
||||
}
|
||||
|
||||
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind) {
|
||||
if (kind === "users") return data.users
|
||||
if (kind === "servers") return data.servers
|
||||
if (kind === "interfaces") return data.interfaces
|
||||
if (kind === "countries") return data.countries
|
||||
if (kind === "services") return data.services
|
||||
return data.asns
|
||||
}
|
||||
|
||||
function hasAnySlice(slices: CubeSlices): boolean {
|
||||
return SLICE_KEYS.some((k) => Boolean(slices[k]))
|
||||
}
|
||||
|
||||
function hiddenKinds(slices: CubeSlices): Set<StatisticsSliceKind> {
|
||||
const hidden = new Set<StatisticsSliceKind>()
|
||||
if (slices.userId) hidden.add("users")
|
||||
if (slices.serverId) hidden.add("servers")
|
||||
if (slices.iface) hidden.add("interfaces")
|
||||
if (slices.country) hidden.add("countries")
|
||||
if (slices.service) hidden.add("services")
|
||||
if (slices.asn) hidden.add("asns")
|
||||
return hidden
|
||||
}
|
||||
|
||||
function applyDimValue(slices: CubeSlices, kind: StatisticsSliceKind, rowId: string): CubeSlices {
|
||||
const next: CubeSlices = { ...slices }
|
||||
if (kind === "users") {
|
||||
if (rowId === STATISTICS_UNBOUND_USER_ID) return next
|
||||
if (next.userId === rowId) delete next.userId
|
||||
else next.userId = rowId
|
||||
} else if (kind === "servers") {
|
||||
if (next.serverId === rowId) delete next.serverId
|
||||
else next.serverId = rowId
|
||||
} else if (kind === "countries") {
|
||||
if (next.country === rowId) delete next.country
|
||||
else next.country = rowId
|
||||
} else if (kind === "services") {
|
||||
if (next.service === rowId) delete next.service
|
||||
else next.service = rowId
|
||||
} else if (kind === "asns") {
|
||||
if (next.asn === rowId) delete next.asn
|
||||
else next.asn = rowId
|
||||
} else {
|
||||
const colon = rowId.indexOf(":")
|
||||
const sid = colon >= 0 ? rowId.slice(0, colon) : undefined
|
||||
const iface = colon >= 0 ? rowId.slice(colon + 1) : rowId
|
||||
if (next.iface === iface && next.serverId === sid) {
|
||||
delete next.iface
|
||||
delete next.serverId
|
||||
} else {
|
||||
next.iface = iface
|
||||
if (sid) next.serverId = sid
|
||||
}
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function applyPivotDim(slices: CubeSlices, dim: StatisticsPivotDim, id: string): CubeSlices {
|
||||
const kind = STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
|
||||
return applyDimValue(slices, kind, id)
|
||||
}
|
||||
|
||||
function chipList(slices: CubeSlices): { key: string; label: string }[] {
|
||||
const chips: { key: string; label: string }[] = []
|
||||
if (slices.country) chips.push({ key: "country", label: `страна ${slices.country}` })
|
||||
if (slices.service) chips.push({ key: "service", label: `сервис ${slices.service}` })
|
||||
if (slices.asn) chips.push({ key: "asn", label: `ASN ${slices.asn}` })
|
||||
if (slices.serverId) chips.push({ key: "serverId", label: `сервер ${slices.serverId}` })
|
||||
if (slices.userId) chips.push({ key: "userId", label: `пользователь ${slices.userId}` })
|
||||
if (slices.iface) chips.push({ key: "iface", label: `iface ${slices.iface}` })
|
||||
return chips
|
||||
}
|
||||
|
||||
function StatisticsPageInner() {
|
||||
const searchParams = useSearchParams()
|
||||
const { mode, backendUrl, prefsHydrated } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const range = useMemo(() => readRange(searchParams), [searchParams])
|
||||
const slices = useMemo(() => readSlices(searchParams), [searchParams])
|
||||
const filters = useMemo(() => slicesToFilters(slices), [slices])
|
||||
const dim = useMemo(() => readDim(searchParams), [searchParams])
|
||||
const view = useMemo(() => readView(searchParams), [searchParams])
|
||||
const planes = useMemo(() => readPlanes(searchParams), [searchParams])
|
||||
const pivotRow = useMemo(() => readPivotDim(searchParams, "pivotRow", "country"), [searchParams])
|
||||
const pivotCol = useMemo(() => readPivotDim(searchParams, "pivotCol", "service"), [searchParams])
|
||||
|
||||
const [data, setData] = useState<StatisticsDto>(EMPTY)
|
||||
const [pivot, setPivot] = useState<StatisticsPivotDto>(EMPTY_PIVOT)
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const replaceParams = useCallback(
|
||||
(patch: Record<string, string | undefined>) => {
|
||||
const sp = new URLSearchParams(searchParams.toString())
|
||||
for (const [k, v] of Object.entries(patch)) {
|
||||
if (v) sp.set(k, v)
|
||||
else sp.delete(k)
|
||||
}
|
||||
const qs = sp.toString()
|
||||
if (qs === searchParams.toString()) return
|
||||
window.history.replaceState(null, "", qs ? `/statistics?${qs}` : "/statistics")
|
||||
},
|
||||
[searchParams],
|
||||
)
|
||||
|
||||
const setRange = useCallback(
|
||||
(next: DateRangeYmd) => {
|
||||
replaceParams({ from: next.from, to: next.to })
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
const setSlices = useCallback(
|
||||
(next: CubeSlices) => {
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
})
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
useEffect(() => {
|
||||
if (!prefsHydrated || !isLive) return
|
||||
let cancelled = false
|
||||
void (async () => {
|
||||
setLoading(true)
|
||||
setError(null)
|
||||
try {
|
||||
const query = toQuery(range, slices, planes)
|
||||
const dto = await getStatistics(backendUrl, query)
|
||||
if (!cancelled) setData(dto)
|
||||
if (view === "pivot" && pivotRow !== pivotCol) {
|
||||
const matrix = await getStatisticsPivot(backendUrl, {
|
||||
...query,
|
||||
row: pivotRow,
|
||||
col: pivotCol,
|
||||
metric: "bytes",
|
||||
})
|
||||
if (!cancelled) setPivot(matrix)
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
if (!cancelled) {
|
||||
setData(EMPTY)
|
||||
setPivot(EMPTY_PIVOT)
|
||||
setError(e instanceof Error ? e.message : "Не удалось загрузить статистику")
|
||||
}
|
||||
} finally {
|
||||
if (!cancelled) setLoading(false)
|
||||
}
|
||||
})()
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [backendUrl, isLive, prefsHydrated, range, slices, view, pivotRow, pivotCol, planes])
|
||||
|
||||
const viewData = isLive ? data : EMPTY
|
||||
const sliced = hasAnySlice(slices)
|
||||
const emptyCube = !isLive || (!loading && viewData.kpis.bytes === 0 && viewData.interfaces.length === 0)
|
||||
|
||||
function handleRowClick(kind: StatisticsSliceKind, row: StatisticsBreakdownRow) {
|
||||
if (kind === "users" && row.id === STATISTICS_UNBOUND_USER_ID) return
|
||||
if (kind === "interfaces" && row.label.includes("· дубль")) return
|
||||
setSlices(applyDimValue(slices, kind, row.id))
|
||||
}
|
||||
|
||||
function handlePivotCell(rowId: string, colId: string) {
|
||||
if (rowId === "__other__" || colId === "__other__") return
|
||||
let next = applyPivotDim(slices, pivotRow, rowId)
|
||||
next = applyPivotDim(next, pivotCol, colId)
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
view: "explore",
|
||||
})
|
||||
}
|
||||
|
||||
const kpis = viewData.kpis
|
||||
const chips = chipList(slices)
|
||||
const countLabel =
|
||||
view === "pivot"
|
||||
? `${pivot.rows.length} × ${pivot.columns.length}`
|
||||
: sliced
|
||||
? `${STATISTICS_DIMS.filter((d) => !hiddenKinds(slices).has(d.id)).length} разрезов`
|
||||
: `${rowsForKind(viewData, dim).length} строк`
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Обзор", href: "/dashboard" }, { label: "Статистика" }]}
|
||||
actions={<PeriodSelector range={range} onChange={setRange} />}
|
||||
/>
|
||||
|
||||
<div className="flex flex-1 flex-col gap-4 overflow-y-auto px-4 py-4 md:gap-6 md:px-6 md:py-5">
|
||||
{!isLive ? (
|
||||
<Alert>
|
||||
<AlertTitle>Живые данные выключены</AlertTitle>
|
||||
<AlertDescription>
|
||||
Куб статистики строится из IPFIX. Переключитесь на живой источник, чтобы увидеть отчёт.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{error ? (
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>Ошибка загрузки</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{!slices.serverId && isLive && !emptyCube ? (
|
||||
<Alert>
|
||||
<AlertTitle>Уникальный объём</AlertTitle>
|
||||
<AlertDescription>
|
||||
Объём — трафик клиентов на GRE/WG, без повторного учёта JH↔EN и WAN.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка трафика"
|
||||
isLoading={loading}
|
||||
skeletonCount={5}
|
||||
items={[
|
||||
{
|
||||
id: "bytes",
|
||||
label: "Объём",
|
||||
value: formatBytes(kpis.bytes),
|
||||
hint: "GRE/WG клиентов, без hops",
|
||||
icon: <DatabaseIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "packets",
|
||||
label: "Пакеты",
|
||||
value: kpis.packets.toLocaleString("ru-RU"),
|
||||
hint: kpis.topService ? `топ: ${kpis.topService}` : undefined,
|
||||
icon: <ActivityIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "bps",
|
||||
label: "Средний bitrate",
|
||||
value: fmtBps(kpis.avgBps),
|
||||
icon: <GaugeIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "users",
|
||||
label: "Пользователи",
|
||||
value: String(kpis.users),
|
||||
icon: <UsersIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "servers",
|
||||
label: "Серверы",
|
||||
value: String(kpis.servers),
|
||||
hint: slices.serverId
|
||||
? (kpis.ifaces ? `${kpis.ifaces} iface` : undefined)
|
||||
: planes === "all"
|
||||
? "WAN и дубли в списке"
|
||||
: "без WAN и overlay",
|
||||
icon: <ServerIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
<StatisticsVolumeChart series={viewData.series} grain={viewData.grain} />
|
||||
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
leading={
|
||||
<div className="flex flex-wrap items-center gap-3">
|
||||
<SegmentedControl
|
||||
value={view}
|
||||
onChange={(next) => replaceParams({ view: next === "pivot" ? "pivot" : "explore" })}
|
||||
options={[
|
||||
{ value: "explore", label: "Разрез" },
|
||||
{ value: "pivot", label: "Сводка" },
|
||||
]}
|
||||
/>
|
||||
<SegmentedControl
|
||||
value={planes}
|
||||
onChange={(next) => replaceParams({ planes: next === "all" ? "all" : undefined })}
|
||||
options={[
|
||||
{ value: "unique", label: "Уникальный" },
|
||||
{ value: "all", label: "Все плоскости" },
|
||||
]}
|
||||
/>
|
||||
{view === "explore" && !sliced ? (
|
||||
<DimensionSelect
|
||||
label="Критерий"
|
||||
value={dim}
|
||||
onChange={(next) => replaceParams({ dim: next })}
|
||||
/>
|
||||
) : null}
|
||||
{view === "pivot" ? (
|
||||
<>
|
||||
<PivotDimSelect
|
||||
label="Строки"
|
||||
value={pivotRow}
|
||||
exclude={pivotCol}
|
||||
onChange={(next) => replaceParams({ pivotRow: next })}
|
||||
/>
|
||||
<PivotDimSelect
|
||||
label="Колонки"
|
||||
value={pivotCol}
|
||||
exclude={pivotRow}
|
||||
onChange={(next) => replaceParams({ pivotCol: next })}
|
||||
/>
|
||||
</>
|
||||
) : null}
|
||||
</div>
|
||||
}
|
||||
filters={filters}
|
||||
onFiltersChange={(next) => setSlices(filtersToSlices(next))}
|
||||
filterFields={STATISTICS_FILTER_FIELDS}
|
||||
countLabel={countLabel}
|
||||
/>
|
||||
<SliceChips
|
||||
chips={chips}
|
||||
onRemove={(key) => {
|
||||
const next = { ...slices }
|
||||
delete next[key as keyof CubeSlices]
|
||||
setSlices(next)
|
||||
}}
|
||||
/>
|
||||
{emptyCube ? (
|
||||
<EmptyState
|
||||
title="Нет данных куба"
|
||||
description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
|
||||
/>
|
||||
) : view === "pivot" ? (
|
||||
<StatisticsPivotGrid data={isLive ? pivot : EMPTY_PIVOT} onCellClick={handlePivotCell} isLoading={loading} />
|
||||
) : sliced ? (
|
||||
<BreakdownDashboard
|
||||
data={viewData}
|
||||
hidden={hiddenKinds(slices)}
|
||||
selectedIdFor={(kind) => selectedIdForKind(kind, slices)}
|
||||
onRowClick={handleRowClick}
|
||||
isLoading={loading}
|
||||
/>
|
||||
) : (
|
||||
<StatisticsBreakdownDataGrid
|
||||
rows={rowsForKind(viewData, dim)}
|
||||
kind={dim}
|
||||
selectedId={selectedIdForKind(dim, slices)}
|
||||
onRowClick={(row) => handleRowClick(dim, row)}
|
||||
isLoading={loading}
|
||||
/>
|
||||
)}
|
||||
</DataPageCard>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export default function StatisticsPage() {
|
||||
return (
|
||||
<Suspense fallback={null}>
|
||||
<StatisticsPageInner />
|
||||
</Suspense>
|
||||
)
|
||||
}
|
||||
+190
-45
@@ -1,30 +1,63 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo, useState } from "react"
|
||||
import { useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { vxlanTunnels, servers } from "@/lib/data"
|
||||
import type { VxlanTunnel } from "@/lib/data"
|
||||
import { vxlanTunnels as mockVxlanTunnels, servers as mockServers } from "@/lib/data"
|
||||
import type { Server, VxlanTunnel } from "@/lib/data"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { VxlanDataGrid } from "@/components/data-grids/vxlan-data-grid"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Alert, AlertDescription } from "@/components/ui/alert"
|
||||
import {
|
||||
NetworkIcon, PlusIcon, CodeXmlIcon, LayersIcon,
|
||||
NetworkIcon, PlusIcon, LayersIcon, RefreshCwIcon, AlertCircleIcon,
|
||||
} from "lucide-react"
|
||||
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { requestJson } from "@/shared/api/http-client"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
|
||||
// ─── helpers ──────────────────────────────────────────────────────────────────
|
||||
|
||||
function serverFor(id: string) {
|
||||
return servers.find((s) => s.id === id)
|
||||
interface BackendServer {
|
||||
id: number
|
||||
name: string
|
||||
host: string
|
||||
type?: Server["type"]
|
||||
site?: string
|
||||
country: string
|
||||
asn?: string
|
||||
enabled: boolean
|
||||
status?: Server["status"]
|
||||
latency?: number | null
|
||||
}
|
||||
|
||||
// ─── RSC generator ───────────────────────────────────────────────────────────
|
||||
interface VxlanApiResponse {
|
||||
tunnels: VxlanTunnel[]
|
||||
}
|
||||
|
||||
function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
const srv = serverFor(t.serverId)
|
||||
function mapBackendServer(s: BackendServer): Server {
|
||||
return {
|
||||
id: String(s.id),
|
||||
name: s.name || s.host,
|
||||
host: s.host,
|
||||
model: "—",
|
||||
os: "—",
|
||||
site: s.site ?? "",
|
||||
country: s.country || "UN",
|
||||
asn: s.asn ?? "",
|
||||
type: s.type ?? "exit-node",
|
||||
enabled: s.enabled,
|
||||
status: s.status ?? "online",
|
||||
latency: s.latency ?? null,
|
||||
sessions: 0,
|
||||
}
|
||||
}
|
||||
|
||||
function generateVxlanRsc(t: VxlanTunnel, serverById: Record<string, Server>): string {
|
||||
const srv = serverById[t.serverId]
|
||||
const lines: string[] = []
|
||||
lines.push(`# VXLAN — ${t.name} · VNI ${t.vni}`)
|
||||
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
|
||||
@@ -42,7 +75,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
if (!t.enabled) lines.push(` disabled=yes \\`)
|
||||
lines.push(``)
|
||||
|
||||
// FDB entries for remote VTEPs
|
||||
for (const vtep of t.remoteVteps) {
|
||||
lines.push(`/interface/vxlan/vteps/add \\`)
|
||||
lines.push(` interface=${t.name} \\`)
|
||||
@@ -50,7 +82,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
lines.push(``)
|
||||
}
|
||||
|
||||
// Bridge
|
||||
lines.push(`# Добавить в bridge:`)
|
||||
lines.push(`/interface/bridge/port/add \\`)
|
||||
lines.push(` bridge=bridge-overlay \\`)
|
||||
@@ -59,12 +90,18 @@ function generateVxlanRsc(t: VxlanTunnel): string {
|
||||
return lines.join("\n")
|
||||
}
|
||||
|
||||
// ─── Export Sheet ─────────────────────────────────────────────────────────────
|
||||
|
||||
function ExportSheet({ open, tunnel, onClose }: {
|
||||
open: boolean; tunnel: VxlanTunnel | null; onClose: () => void
|
||||
function ExportSheet({
|
||||
open, tunnel, onClose, serverById,
|
||||
}: {
|
||||
open: boolean
|
||||
tunnel: VxlanTunnel | null
|
||||
onClose: () => void
|
||||
serverById: Record<string, Server>
|
||||
}) {
|
||||
const code = useMemo(() => tunnel ? generateVxlanRsc(tunnel) : "", [tunnel])
|
||||
const code = useMemo(
|
||||
() => (tunnel ? generateVxlanRsc(tunnel, serverById) : ""),
|
||||
[tunnel, serverById],
|
||||
)
|
||||
|
||||
return (
|
||||
<CodeExportSheet
|
||||
@@ -84,46 +121,156 @@ function ExportSheet({ open, tunnel, onClose }: {
|
||||
)
|
||||
}
|
||||
|
||||
// ─── Export Sheet ─────────────────────────────────────────────────────────────
|
||||
export default function VxlanPage() {
|
||||
const [search, setSearch] = useState("")
|
||||
const { mode, backendUrl } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const [search, setSearch] = useState("")
|
||||
const [exportTunnel, setExportTunnel] = useState<VxlanTunnel | null>(null)
|
||||
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
|
||||
|
||||
const [liveTunnels, setLiveTunnels] = useState<VxlanTunnel[]>([])
|
||||
const [liveServers, setLiveServers] = useState<Server[]>([])
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [liveError, setLiveError] = useState<string | null>(null)
|
||||
|
||||
const loadLive = useCallback(async () => {
|
||||
if (!isLive) return
|
||||
setLoading(true)
|
||||
setLiveError(null)
|
||||
try {
|
||||
const [tunnelsRes, serversRes] = await Promise.all([
|
||||
requestJson<VxlanApiResponse>(backendUrl, "/api/vxlan"),
|
||||
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
|
||||
])
|
||||
setLiveTunnels(tunnelsRes.tunnels ?? [])
|
||||
setLiveServers(serversRes.filter((s) => s.enabled).map(mapBackendServer))
|
||||
} catch (e) {
|
||||
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
|
||||
setLiveTunnels([])
|
||||
} finally {
|
||||
setLoading(false)
|
||||
}
|
||||
}, [isLive, backendUrl])
|
||||
|
||||
useEffect(() => {
|
||||
if (!isLive) {
|
||||
queueMicrotask(() => {
|
||||
setLiveTunnels([])
|
||||
setLiveServers([])
|
||||
setLiveError(null)
|
||||
})
|
||||
return
|
||||
}
|
||||
queueMicrotask(() => {
|
||||
void loadLive()
|
||||
})
|
||||
}, [isLive, loadLive])
|
||||
|
||||
const displayTunnels = isLive ? liveTunnels : mockVxlanTunnels
|
||||
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
|
||||
|
||||
const effectiveServerId =
|
||||
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const scopedTunnels = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return displayTunnels
|
||||
return displayTunnels.filter((t) => t.serverId === effectiveServerId)
|
||||
}, [displayTunnels, effectiveServerId])
|
||||
|
||||
const serverById = useMemo(
|
||||
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
|
||||
[displayServers],
|
||||
)
|
||||
|
||||
const railItems = useMemo<ServerTileItem[]>(() => (
|
||||
displayServers.map((s) => ({
|
||||
id: s.id,
|
||||
name: s.name,
|
||||
host: s.host,
|
||||
site: s.site,
|
||||
country: s.country,
|
||||
status: s.status,
|
||||
type: s.type,
|
||||
enabled: s.enabled,
|
||||
meta: String(displayTunnels.filter((t) => t.serverId === s.id).length),
|
||||
}))
|
||||
), [displayServers, displayTunnels])
|
||||
|
||||
const filtered = useMemo(() => {
|
||||
if (!search) return vxlanTunnels
|
||||
if (!search) return scopedTunnels
|
||||
const q = search.toLowerCase()
|
||||
return vxlanTunnels.filter((t) =>
|
||||
return scopedTunnels.filter((t) =>
|
||||
t.name.includes(q) ||
|
||||
String(t.vni).includes(q) ||
|
||||
t.vtepIp.includes(q) ||
|
||||
(serverFor(t.serverId)?.name.toLowerCase().includes(q) ?? false)
|
||||
(serverById[t.serverId]?.name.toLowerCase().includes(q) ?? false),
|
||||
)
|
||||
}, [search])
|
||||
}, [search, scopedTunnels, serverById])
|
||||
|
||||
const upCount = vxlanTunnels.filter((t) => t.status === "up").length
|
||||
const vnis = new Set(vxlanTunnels.map((t) => t.vni)).size
|
||||
const upCount = scopedTunnels.filter((t) => t.status === "up").length
|
||||
const vnis = new Set(scopedTunnels.map((t) => t.vni)).size
|
||||
|
||||
return (
|
||||
<div className="flex flex-col h-full">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "VXLAN" }]}
|
||||
actions={
|
||||
<Button size="sm">
|
||||
<PlusIcon className="size-4" />Новый VXLAN
|
||||
</Button>
|
||||
}
|
||||
/>
|
||||
|
||||
<div className="flex-1 overflow-y-auto p-6">
|
||||
<>
|
||||
<ServerRailLayout
|
||||
items={railItems}
|
||||
selectedId={effectiveServerId}
|
||||
onSelect={setSelectedServerId}
|
||||
showAll
|
||||
allCount={displayServers.length}
|
||||
loading={isLive && loading && displayServers.length === 0}
|
||||
header={
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Управление" }, { label: "VXLAN" }]}
|
||||
actions={
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => { void loadLive() }}
|
||||
disabled={!isLive || loading}
|
||||
>
|
||||
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button size="sm">
|
||||
<PlusIcon className="size-4" />Новый VXLAN
|
||||
</Button>
|
||||
</>
|
||||
}
|
||||
/>
|
||||
}
|
||||
>
|
||||
<div className="flex flex-col gap-5">
|
||||
|
||||
{isLive && liveError && (
|
||||
<Alert variant="warning" className="py-2">
|
||||
<AlertCircleIcon />
|
||||
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
{isLive && !loading && displayTunnels.length === 0 && !liveError && (
|
||||
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
|
||||
На опрошенных серверах нет VXLAN-интерфейсов
|
||||
</div>
|
||||
)}
|
||||
{mode === "mock" && (
|
||||
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
|
||||
Моковые данные
|
||||
</span>
|
||||
)}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка VXLAN"
|
||||
items={[
|
||||
{
|
||||
id: "tunnels",
|
||||
label: "Туннелей",
|
||||
value: vxlanTunnels.length,
|
||||
value: scopedTunnels.length,
|
||||
icon: <NetworkIcon className="size-4" />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
@@ -144,14 +291,13 @@ export default function VxlanPage() {
|
||||
{
|
||||
id: "servers",
|
||||
label: "Серверов",
|
||||
value: new Set(vxlanTunnels.map((t) => t.serverId)).size,
|
||||
value: new Set(scopedTunnels.map((t) => t.serverId)).size,
|
||||
icon: <NetworkIcon className="size-4" />,
|
||||
iconClassName: "text-primary",
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
{/* Info banner */}
|
||||
<div className="flex items-start gap-3 rounded-lg bg-sky-500/5 border border-sky-500/20 px-4 py-3 text-sm">
|
||||
<NetworkIcon className="size-5 text-sky-500 shrink-0 mt-0.5" />
|
||||
<div>
|
||||
@@ -163,7 +309,6 @@ export default function VxlanPage() {
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Table */}
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
search={search}
|
||||
@@ -173,12 +318,11 @@ export default function VxlanPage() {
|
||||
/>
|
||||
<VxlanDataGrid
|
||||
tunnels={filtered}
|
||||
servers={servers}
|
||||
servers={displayServers}
|
||||
onExport={setExportTunnel}
|
||||
/>
|
||||
</DataPageCard>
|
||||
|
||||
{/* Reference */}
|
||||
<OpsPanel title="RouterOS 7 · /interface/vxlan — быстрые команды" contentClassName="px-5 py-4">
|
||||
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
|
||||
{[
|
||||
@@ -232,13 +376,14 @@ export default function VxlanPage() {
|
||||
</OpsPanel>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</ServerRailLayout>
|
||||
|
||||
<ExportSheet
|
||||
open={!!exportTunnel}
|
||||
tunnel={exportTunnel}
|
||||
onClose={() => setExportTunnel(null)}
|
||||
serverById={serverById}
|
||||
/>
|
||||
</div>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -57,10 +57,12 @@ import {
|
||||
type ServerTileItem,
|
||||
} from "@/components/server-tile-rail"
|
||||
import { toast } from "sonner"
|
||||
import { ConfigHistorySheet } from "@/components/config-history-sheet"
|
||||
import type { ConfigRevisionDto } from "@/lib/config-revisions"
|
||||
import {
|
||||
ShieldCheckIcon, PlusIcon, KeyRoundIcon,
|
||||
UsersIcon, ActivityIcon, RefreshCwIcon, UploadIcon, InfoIcon,
|
||||
Trash2Icon, CodeXmlIcon, AlertCircleIcon,
|
||||
Trash2Icon, CodeXmlIcon, AlertCircleIcon, HistoryIcon,
|
||||
} from "lucide-react"
|
||||
|
||||
type WgWorkspaceTab = "interfaces" | "peers" | "cli"
|
||||
@@ -188,6 +190,10 @@ export default function WireGuardPage() {
|
||||
const [exportPeerId, setExportPeerId] = useState<string | null>(null)
|
||||
const [peerIface, setPeerIface] = useState<WgIfaceWithServer | null>(null)
|
||||
const [pendingDelete, setPendingDelete] = useState<PendingDelete | null>(null)
|
||||
const [historyOpen, setHistoryOpen] = useState(false)
|
||||
const [revisions, setRevisions] = useState<ConfigRevisionDto[]>([])
|
||||
const [historyLoading, setHistoryLoading] = useState(false)
|
||||
const [historyRestoring, setHistoryRestoring] = useState(false)
|
||||
const [liveExport, setLiveExport] = useState<{
|
||||
rsc?: string
|
||||
conf?: string
|
||||
@@ -242,6 +248,44 @@ export default function WireGuardPage() {
|
||||
? selectedServerId
|
||||
: ALL_SERVERS_ID
|
||||
|
||||
const historyServerId = effectiveServerId === ALL_SERVERS_ID ? null : effectiveServerId
|
||||
|
||||
const loadRevisions = useCallback(async () => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryLoading(true)
|
||||
try {
|
||||
const res = await requestJson<{ revisions: ConfigRevisionDto[] }>(
|
||||
backendUrl,
|
||||
`/api/wireguard/revisions?serverId=${encodeURIComponent(historyServerId)}`,
|
||||
)
|
||||
setRevisions(res.revisions)
|
||||
} catch (err) {
|
||||
toast.error("Не удалось загрузить историю", { description: String(err) })
|
||||
setRevisions([])
|
||||
} finally {
|
||||
setHistoryLoading(false)
|
||||
}
|
||||
}, [isLive, historyServerId, backendUrl])
|
||||
|
||||
const restoreRevision = useCallback(async (id: string) => {
|
||||
if (!isLive || !historyServerId) return
|
||||
setHistoryRestoring(true)
|
||||
try {
|
||||
await requestJson(
|
||||
backendUrl,
|
||||
`/api/wireguard/revisions/${encodeURIComponent(id)}/restore`,
|
||||
{ method: "POST", body: JSON.stringify({ serverId: historyServerId }) },
|
||||
)
|
||||
toast.success("Версия применена на роутер")
|
||||
await loadLive()
|
||||
await loadRevisions()
|
||||
} catch (err) {
|
||||
toast.error("Не удалось откатить", { description: String(err) })
|
||||
} finally {
|
||||
setHistoryRestoring(false)
|
||||
}
|
||||
}, [isLive, historyServerId, backendUrl, loadLive, loadRevisions])
|
||||
|
||||
const scopedIfaces = useMemo(() => {
|
||||
if (effectiveServerId === ALL_SERVERS_ID) return displayIfaces
|
||||
return displayIfaces.filter((i) => i.serverId === effectiveServerId)
|
||||
@@ -544,6 +588,7 @@ export default function WireGuardPage() {
|
||||
<>
|
||||
<ServerRailMobileButton />
|
||||
{isLive && (
|
||||
<>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
@@ -553,6 +598,20 @@ export default function WireGuardPage() {
|
||||
<RefreshCwIcon className={`size-4 ${loading ? "animate-spin" : ""}`} />
|
||||
Обновить
|
||||
</Button>
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
disabled={loading || !historyServerId}
|
||||
title={!historyServerId ? "Выберите сервер, чтобы смотреть историю" : "История версий и откат на CHR"}
|
||||
onClick={() => {
|
||||
setHistoryOpen(true)
|
||||
void loadRevisions()
|
||||
}}
|
||||
>
|
||||
<HistoryIcon className="size-4" />
|
||||
История
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
<Button size="sm" variant="outline" onClick={() => setImportOpen(true)}>
|
||||
<UploadIcon className="size-4" />
|
||||
@@ -822,6 +881,17 @@ export default function WireGuardPage() {
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
|
||||
<ConfigHistorySheet
|
||||
open={historyOpen}
|
||||
onOpenChange={setHistoryOpen}
|
||||
title="История WireGuard"
|
||||
itemLabel="интерфейсов"
|
||||
revisions={revisions}
|
||||
loading={historyLoading}
|
||||
restoring={historyRestoring}
|
||||
onRestore={restoreRevision}
|
||||
/>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -5,3 +5,4 @@ dist/
|
||||
*.db-wal
|
||||
.env
|
||||
storage/backups/
|
||||
storage/geoip/
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
-- S3-compatible storage for RouterOS backups
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_storage_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
provider TEXT NOT NULL DEFAULT 'local' CHECK (provider IN ('local', 's3')),
|
||||
s3_endpoint TEXT NOT NULL DEFAULT '',
|
||||
s3_region TEXT NOT NULL DEFAULT 'us-east-1',
|
||||
s3_bucket TEXT NOT NULL DEFAULT '',
|
||||
s3_prefix TEXT NOT NULL DEFAULT 'mikrotik',
|
||||
s3_access_key_id TEXT NOT NULL DEFAULT '',
|
||||
s3_secret_access_key TEXT NOT NULL DEFAULT '',
|
||||
s3_force_path_style BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
keep_local_copy BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
last_test_at TIMESTAMPTZ,
|
||||
last_test_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
INSERT INTO backup_storage_settings (id)
|
||||
VALUES (1)
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS storage TEXT NOT NULL DEFAULT 'local';
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS s3_key TEXT;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS s3_etag TEXT;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS upload_error TEXT;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM pg_constraint WHERE conname = 'backup_entries_storage_check'
|
||||
) THEN
|
||||
ALTER TABLE backup_entries
|
||||
ADD CONSTRAINT backup_entries_storage_check
|
||||
CHECK (storage IN ('local', 's3', 'both'));
|
||||
END IF;
|
||||
END $$;
|
||||
@@ -0,0 +1,19 @@
|
||||
-- GeoLite2 mmdb (страна/ASN для netflow): настройки автообновления зеркала P3TERX
|
||||
|
||||
CREATE TABLE IF NOT EXISTS geoip_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
update_interval_sec INTEGER NOT NULL DEFAULT 604800,
|
||||
last_check_at TIMESTAMPTZ,
|
||||
last_success_at TIMESTAMPTZ,
|
||||
last_error TEXT,
|
||||
country_build_at TIMESTAMPTZ,
|
||||
asn_build_at TIMESTAMPTZ,
|
||||
etags_json JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
INSERT INTO geoip_settings (id)
|
||||
VALUES (1)
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
@@ -0,0 +1,28 @@
|
||||
-- Statistics cube: hour + daily facts (server × iface × country × service × ASN).
|
||||
-- Compact types. FILLFACTOR/autovacuum нельзя на partitioned parent (PG 42809) —
|
||||
-- задаются на листовых партициях в ensurePartitionFor.
|
||||
-- Retention: DROP partitions only (see PARTITION_SPECS).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_hour_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_daily_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
day DATE NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, day, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (day);
|
||||
@@ -0,0 +1,16 @@
|
||||
-- История desired/actual снапшотов managed-секций (фильтры, рекурсивные маршруты).
|
||||
-- Retention — prune в сервисе (последние 50 на пару server+section).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS config_revisions (
|
||||
id TEXT PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
section TEXT NOT NULL,
|
||||
source TEXT NOT NULL,
|
||||
fingerprint TEXT NOT NULL,
|
||||
payload JSONB NOT NULL DEFAULT '[]'::jsonb,
|
||||
note TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE INDEX IF NOT EXISTS idx_config_revisions_server_section_created
|
||||
ON config_revisions (server_id, section, created_at DESC);
|
||||
+10
-3
@@ -12,15 +12,21 @@
|
||||
"db:migrate": "drizzle-kit migrate",
|
||||
"db:studio": "drizzle-kit studio",
|
||||
"db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts",
|
||||
"facts:rebuild": "tsx src/scripts/rebuild-flow-facts.ts",
|
||||
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
|
||||
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
|
||||
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-dest.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/traffic-flow-facts-rebuild.test.ts && tsx src/services/statistics-aggregate.test.ts",
|
||||
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg"
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts && tsx src/services/config-revisions.test.ts",
|
||||
"test:config-sync": "tsx src/services/config-apply-plan.test.ts && tsx src/services/entity-snapshots.test.ts",
|
||||
"test:backups": "tsx src/services/s3-backup-client.test.ts",
|
||||
"test:live-maps": "tsx src/services/ospf-route-parse.test.ts && tsx src/services/vxlan-live.test.ts && tsx src/services/containers-live.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups && npm run test:live-maps && npm run test:config-sync",
|
||||
"test:geoip": "tsx src/services/traffic-flow-geoip.test.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.888.0",
|
||||
"@fastify/cors": "^11.2.0",
|
||||
"@fastify/jwt": "^10.2.2",
|
||||
"@fastify/type-provider-zod": "^1.0.0",
|
||||
@@ -31,6 +37,7 @@
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"fastify": "^5.8.5",
|
||||
"fastify-plugin": "^5.1.0",
|
||||
"maxmind": "^5.0.7",
|
||||
"pg": "^8.23.0",
|
||||
"undici": "^8.1.0",
|
||||
"zod": "^4.4.1"
|
||||
|
||||
@@ -8,11 +8,19 @@ export interface PartitionSpec {
|
||||
keepDays: number
|
||||
}
|
||||
|
||||
/** Leaf-only: PG forbids storage params on partitioned parents (SQLSTATE 42809). */
|
||||
const FACT_LEAF_STORAGE =
|
||||
"fillfactor = 70, autovacuum_vacuum_scale_factor = 0.05, autovacuum_vacuum_cost_limit = 2000"
|
||||
|
||||
const FACT_PARENTS = new Set(["flow_hour_facts", "flow_daily_facts"])
|
||||
|
||||
export const PARTITION_SPECS: PartitionSpec[] = [
|
||||
{ parent: "flow_buckets", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_stats", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_dims", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_daily_dims", kind: "month", keepDays: 420 },
|
||||
{ parent: "flow_hour_facts", kind: "day", keepDays: 3 },
|
||||
{ parent: "flow_daily_facts", kind: "month", keepDays: 420 },
|
||||
{ parent: "traffic_samples", kind: "week", keepDays: 21 },
|
||||
{ parent: "servers_rest_ping_samples", kind: "week", keepDays: 35 },
|
||||
{ parent: "uptime_probe_samples", kind: "week", keepDays: 21 },
|
||||
@@ -110,6 +118,9 @@ export async function ensurePartitionFor(
|
||||
await pool.query(
|
||||
`CREATE TABLE IF NOT EXISTS ${name} PARTITION OF ${parent} FOR VALUES FROM ('${from}') TO ('${to}')`,
|
||||
)
|
||||
if (FACT_PARENTS.has(parent)) {
|
||||
await pool.query(`ALTER TABLE ${name} SET (${FACT_LEAF_STORAGE})`)
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
|
||||
@@ -163,6 +163,22 @@ if (!(await withPgOrSkip())) {
|
||||
await dbQuery(`DELETE FROM servers WHERE name = 'pg-wipe-idempotent'`)
|
||||
}
|
||||
|
||||
{
|
||||
const mig = await dbQuery<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = '0006_config_revisions'`,
|
||||
)
|
||||
assert.equal(mig.rows.length, 1, "0006 применена")
|
||||
|
||||
const { rows } = await dbQuery<{ column_name: string; udt_name: string }>(`
|
||||
SELECT column_name, udt_name FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'config_revisions'
|
||||
`)
|
||||
const by = Object.fromEntries(rows.map((r) => [r.column_name, r.udt_name]))
|
||||
assert.equal(by.payload, "jsonb")
|
||||
assert.equal(by.fingerprint, "text")
|
||||
assert.equal(by.section, "text")
|
||||
}
|
||||
|
||||
{
|
||||
const marker = await dbQuery<{ sqlite_imported_at: string | null }>(
|
||||
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
|
||||
|
||||
@@ -93,6 +93,19 @@ export const filterRules = pgTable("filter_rules", {
|
||||
index("idx_filter_rules_server_sort").on(t.serverId, t.sortOrder),
|
||||
])
|
||||
|
||||
export const configRevisions = pgTable("config_revisions", {
|
||||
id: text("id").primaryKey(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
section: text("section", { enum: ["filters", "recursive-routes", "firewall", "wireguard", "gre"] }).notNull(),
|
||||
source: text("source", { enum: ["apply", "rollback", "observed", "copy"] }).notNull(),
|
||||
fingerprint: text("fingerprint").notNull(),
|
||||
payload: jsonb("payload").$type<unknown>().notNull().default(sql`'[]'::jsonb`),
|
||||
note: text("note"),
|
||||
createdAt: ts("created_at").notNull().defaultNow(),
|
||||
}, (t) => [
|
||||
index("idx_config_revisions_server_section_created").on(t.serverId, t.section, t.createdAt),
|
||||
])
|
||||
|
||||
export const recursiveRoutes = pgTable("recursive_routes", {
|
||||
id: idIdentity().primaryKey(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
@@ -208,6 +221,36 @@ export const flowDailyDims = pgTable("flow_daily_dims", {
|
||||
index("idx_flow_daily_dims_day").on(t.day, t.dim),
|
||||
])
|
||||
|
||||
/** Hour-grain traffic cube for statistics (≤48h). No secondary indexes. */
|
||||
export const flowHourFacts = pgTable("flow_hour_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.bucketAt, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
/** Daily-grain traffic cube for statistics (long window). No secondary indexes. */
|
||||
export const flowDailyFacts = pgTable("flow_daily_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
day: date("day", { mode: "string" }).notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.day, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
export const flowBuckets = pgTable("flow_buckets", {
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
@@ -231,6 +274,20 @@ export const flowBuckets = pgTable("flow_buckets", {
|
||||
index("idx_flow_buckets_server_time").on(t.serverId, t.bucketAt),
|
||||
])
|
||||
|
||||
export const geoipSettings = pgTable("geoip_settings", {
|
||||
id: idSingleton(),
|
||||
enabled: boolean("enabled").notNull().default(true),
|
||||
updateIntervalSec: integer("update_interval_sec").notNull().default(604800),
|
||||
lastCheckAt: ts("last_check_at"),
|
||||
lastSuccessAt: ts("last_success_at"),
|
||||
lastError: text("last_error"),
|
||||
countryBuildAt: ts("country_build_at"),
|
||||
asnBuildAt: ts("asn_build_at"),
|
||||
etagsJson: jsonb("etags_json").notNull().default(sql`'{}'::jsonb`),
|
||||
createdAt: ts("created_at").notNull().defaultNow(),
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const flowIpMeta = pgTable("flow_ip_meta", {
|
||||
prefix: text("prefix").primaryKey(),
|
||||
asn: integer("asn").notNull().default(0),
|
||||
@@ -420,6 +477,22 @@ export const backupScheduleSettings = pgTable("backup_schedule_settings", {
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const backupStorageSettings = pgTable("backup_storage_settings", {
|
||||
id: idSingleton(),
|
||||
provider: text("provider", { enum: ["local", "s3"] }).notNull().default("local"),
|
||||
s3Endpoint: text("s3_endpoint").notNull().default(""),
|
||||
s3Region: text("s3_region").notNull().default("us-east-1"),
|
||||
s3Bucket: text("s3_bucket").notNull().default(""),
|
||||
s3Prefix: text("s3_prefix").notNull().default("mikrotik"),
|
||||
s3AccessKeyId: text("s3_access_key_id").notNull().default(""),
|
||||
s3SecretAccessKey: text("s3_secret_access_key").notNull().default(""),
|
||||
s3ForcePathStyle: boolean("s3_force_path_style").notNull().default(true),
|
||||
keepLocalCopy: boolean("keep_local_copy").notNull().default(true),
|
||||
lastTestAt: ts("last_test_at"),
|
||||
lastTestError: text("last_test_error"),
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const backupEntries = pgTable("backup_entries", {
|
||||
id: text("id").primaryKey(),
|
||||
serverId: bigint("server_id", { mode: "number" })
|
||||
@@ -429,6 +502,10 @@ export const backupEntries = pgTable("backup_entries", {
|
||||
sizeBytes: bigint("size_bytes", { mode: "number" }).notNull(),
|
||||
kind: text("kind", { enum: ["manual", "auto"] }).notNull().default("manual"),
|
||||
notes: text("notes"),
|
||||
storage: text("storage", { enum: ["local", "s3", "both"] }).notNull().default("local"),
|
||||
s3Key: text("s3_key"),
|
||||
s3Etag: text("s3_etag"),
|
||||
uploadError: text("upload_error"),
|
||||
createdAt: ts("created_at").notNull(),
|
||||
}, (t) => [
|
||||
uniqueIndex("idx_backup_entries_filename").on(t.filename),
|
||||
@@ -669,6 +746,7 @@ export type ServerInsert = typeof servers.$inferInsert
|
||||
export type Snapshot = typeof serverSnapshots.$inferSelect
|
||||
export type SnapshotInsert = typeof serverSnapshots.$inferInsert
|
||||
export type FilterRuleRow = typeof filterRules.$inferSelect
|
||||
export type ConfigRevisionRow = typeof configRevisions.$inferSelect
|
||||
export type RecursiveRouteRow = typeof recursiveRoutes.$inferSelect
|
||||
export type TrafficSettingsRow = typeof trafficSettings.$inferSelect
|
||||
export type TrafficFlowSettingsRow = typeof trafficFlowSettings.$inferSelect
|
||||
|
||||
@@ -81,6 +81,12 @@ const TABLES: TableCopy[] = [
|
||||
["server_ids_json", "json"], ["last_run_at", "ts"], ["last_duration_ms", "int"],
|
||||
["last_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "backup_storage_settings", upsert: true, columns: [
|
||||
["id", "int"], ["provider", "text"], ["s3_endpoint", "text"], ["s3_region", "text"],
|
||||
["s3_bucket", "text"], ["s3_prefix", "text"], ["s3_access_key_id", "text"],
|
||||
["s3_secret_access_key", "text"], ["s3_force_path_style", "bool"], ["keep_local_copy", "bool"],
|
||||
["last_test_at", "ts"], ["last_test_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "internet_path_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"], ["retention_days", "int"],
|
||||
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"],
|
||||
|
||||
@@ -14,6 +14,7 @@ import filtersRoutes from "./routes/filters.js"
|
||||
import recursiveRoutes from "./routes/recursive-routes.js"
|
||||
import trafficRoutes from "./routes/traffic.js"
|
||||
import trafficFlowRoutes from "./routes/traffic-flow.js"
|
||||
import geoipRoutes from "./routes/geoip.js"
|
||||
import serversApiPingRoutes from "./routes/servers-api-ping.js"
|
||||
import uptimeRoutes from "./routes/uptime.js"
|
||||
import networkRoutes from "./routes/network.js"
|
||||
@@ -28,10 +29,15 @@ import certificatesRoutes from "./routes/certificates.js"
|
||||
import systemDatabaseRoutes from "./routes/system-database.js"
|
||||
import eventsRoutes from "./routes/events.js"
|
||||
import wireguardRoutes from "./routes/wireguard.js"
|
||||
import vxlanRoutes from "./routes/vxlan.js"
|
||||
import containersRoutes from "./routes/containers.js"
|
||||
import firewallRoutes from "./routes/firewall.js"
|
||||
import greRoutes from "./routes/gre.js"
|
||||
import usersRoutes from "./routes/users.js"
|
||||
import statisticsRoutes from "./routes/statistics.js"
|
||||
import { refreshScheduler, stopScheduler } from "./services/scheduler.js"
|
||||
import { getFlowWorkerHealth, startTrafficFlowListener, stopTrafficFlowListener } from "./services/traffic-flow-ingest.js"
|
||||
import { initGeoip } from "./services/traffic-flow-geoip.js"
|
||||
|
||||
const eventLoopDelay = monitorEventLoopDelay({ resolution: 20 })
|
||||
eventLoopDelay.enable()
|
||||
@@ -116,6 +122,7 @@ export async function buildApp(opts?: {
|
||||
await app.register(recursiveRoutes, { prefix: "/api" })
|
||||
await app.register(trafficRoutes, { prefix: "/api" })
|
||||
await app.register(trafficFlowRoutes, { prefix: "/api" })
|
||||
await app.register(geoipRoutes, { prefix: "/api" })
|
||||
await app.register(serversApiPingRoutes, { prefix: "/api" })
|
||||
await app.register(uptimeRoutes, { prefix: "/api" })
|
||||
await app.register(networkRoutes, { prefix: "/api" })
|
||||
@@ -130,11 +137,16 @@ export async function buildApp(opts?: {
|
||||
await app.register(systemDatabaseRoutes, { prefix: "/api" })
|
||||
await app.register(eventsRoutes, { prefix: "/api" })
|
||||
await app.register(wireguardRoutes, { prefix: "/api" })
|
||||
await app.register(vxlanRoutes, { prefix: "/api" })
|
||||
await app.register(containersRoutes, { prefix: "/api" })
|
||||
await app.register(firewallRoutes, { prefix: "/api" })
|
||||
await app.register(greRoutes, { prefix: "/api" })
|
||||
await app.register(usersRoutes, { prefix: "/api" })
|
||||
await app.register(statisticsRoutes, { prefix: "/api" })
|
||||
|
||||
if (opts?.startScheduler !== false) {
|
||||
await refreshScheduler()
|
||||
await initGeoip()
|
||||
await startTrafficFlowListener()
|
||||
app.addHook("onClose", async () => {
|
||||
stopScheduler()
|
||||
|
||||
@@ -18,7 +18,7 @@ assert.equal(
|
||||
"mm:settings:admin",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/traffic/servers/1/live"),
|
||||
permissionForRequest("GET", "/api/statistics"),
|
||||
"mm:traffic:read",
|
||||
)
|
||||
assert.equal(
|
||||
@@ -41,6 +41,14 @@ assert.equal(
|
||||
permissionForRequest("GET", "/api/firewall/all"),
|
||||
"mm:network:read",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/gre/tunnels"),
|
||||
"mm:network:read",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("POST", "/api/gre/tunnels"),
|
||||
"mm:network:write",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/users"),
|
||||
"mm:users:read",
|
||||
|
||||
@@ -107,7 +107,7 @@ const RULES: Rule[] = [
|
||||
},
|
||||
{
|
||||
methods: ["GET"],
|
||||
match: (p) => p.startsWith("/api/traffic"),
|
||||
match: (p) => p.startsWith("/api/traffic") || p.startsWith("/api/statistics"),
|
||||
permission: "mm:traffic:read",
|
||||
},
|
||||
{
|
||||
@@ -155,7 +155,8 @@ const RULES: Rule[] = [
|
||||
p.startsWith("/api/internet-path") ||
|
||||
p.startsWith("/api/exec") ||
|
||||
p.startsWith("/api/wireguard") ||
|
||||
p.startsWith("/api/firewall"),
|
||||
p.startsWith("/api/firewall") ||
|
||||
p.startsWith("/api/gre"),
|
||||
permission: "mm:network:read",
|
||||
},
|
||||
{
|
||||
@@ -168,7 +169,8 @@ const RULES: Rule[] = [
|
||||
p.startsWith("/api/internet-path") ||
|
||||
p.startsWith("/api/exec") ||
|
||||
p.startsWith("/api/wireguard") ||
|
||||
p.startsWith("/api/firewall"),
|
||||
p.startsWith("/api/firewall") ||
|
||||
p.startsWith("/api/gre"),
|
||||
permission: "mm:network:write",
|
||||
},
|
||||
]
|
||||
|
||||
@@ -19,5 +19,31 @@ export function managedComment(label: string): string {
|
||||
}
|
||||
|
||||
export function managedRecursiveComment(comment?: string | null): string {
|
||||
return comment ? `${PRODUCT_NAME}:recursive ${comment}` : `${PRODUCT_NAME}:recursive`
|
||||
const stripped = stripManagedRecursiveComment(comment ?? "")
|
||||
return stripped ? `${PRODUCT_NAME}:recursive ${stripped}` : `${PRODUCT_NAME}:recursive`
|
||||
}
|
||||
|
||||
const LEGACY_RECURSIVE_PREFIX = /^recursive:\s*/i
|
||||
|
||||
export function stripManagedRecursiveComment(comment: string): string {
|
||||
const value = comment.trim()
|
||||
if (!value) return ""
|
||||
const managedPrefixes = [
|
||||
`${PRODUCT_NAME}:recursive`,
|
||||
`${LEGACY_PRODUCT_NAME}:recursive`,
|
||||
]
|
||||
for (const prefix of managedPrefixes) {
|
||||
if (value.startsWith(prefix)) return value.slice(prefix.length).trim()
|
||||
}
|
||||
if (LEGACY_RECURSIVE_PREFIX.test(value)) {
|
||||
return value.replace(LEGACY_RECURSIVE_PREFIX, "").trim()
|
||||
}
|
||||
return value
|
||||
}
|
||||
|
||||
/** Owned recursive route: MM/legacy prefix or old `recursive:` mask. */
|
||||
export function isOwnedRecursiveComment(comment: string | undefined): boolean {
|
||||
if (!comment) return false
|
||||
const value = comment.trim()
|
||||
return hasManagedRecursiveComment(value) || LEGACY_RECURSIVE_PREFIX.test(value)
|
||||
}
|
||||
|
||||
@@ -1,20 +1,26 @@
|
||||
import { randomUUID } from "node:crypto"
|
||||
import { readFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { putBackupScheduleSettingsSchema } from "@mmapp/contracts/backups"
|
||||
import {
|
||||
putBackupScheduleSettingsSchema,
|
||||
putBackupStorageSettingsSchema,
|
||||
} from "@mmapp/contracts/backups"
|
||||
import { listServersRead } from "../modules/servers/service/servers-service.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
import { refreshScheduler } from "../services/scheduler.js"
|
||||
import {
|
||||
deleteBackupRecord,
|
||||
getBackupById,
|
||||
getBackupsDir,
|
||||
getBackupScheduleSettings,
|
||||
getBackupStorageSettings,
|
||||
listBackups,
|
||||
readBackupContent,
|
||||
restoreBackupToDevice,
|
||||
runBackupForServer,
|
||||
syncBackupsFromS3,
|
||||
testBackupStorageConnection,
|
||||
updateBackupScheduleSettings,
|
||||
updateBackupStorageSettings,
|
||||
type BackupMeta,
|
||||
} from "../services/backup-service.js"
|
||||
|
||||
@@ -97,6 +103,39 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
app.get("/backups/storage", async (_req, reply) => {
|
||||
return reply.send(await getBackupStorageSettings())
|
||||
})
|
||||
|
||||
app.put("/backups/storage", async (req, reply) => {
|
||||
const parsed = putBackupStorageSettingsSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(await updateBackupStorageSettings(parsed.data))
|
||||
})
|
||||
|
||||
app.post("/backups/storage/test", async (_req, reply) => {
|
||||
try {
|
||||
return reply.send(await testBackupStorageConnection())
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка проверки S3",
|
||||
settings: await getBackupStorageSettings(),
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/storage/sync", async (_req, reply) => {
|
||||
try {
|
||||
return reply.send(await syncBackupsFromS3())
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка синхронизации S3",
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/create", { schema: { body: CreateBackupBodySchema } }, async (req, reply) => {
|
||||
const inputIds = req.body.serverIds.map((x) => String(x))
|
||||
const notes = req.body.notes?.trim() || undefined
|
||||
@@ -169,12 +208,34 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/backups/:id/download", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
const hit = await getBackupById(req.params.id)
|
||||
if (!hit) return reply.status(404).send({ error: "Бэкап не найден" })
|
||||
const filePath = path.join(getBackupsDir(), hit.filename)
|
||||
const content = await readFile(filePath, "utf8").catch(() => null)
|
||||
if (content == null) return reply.status(404).send({ error: "Файл бэкапа не найден" })
|
||||
reply.header("Content-Type", "text/plain; charset=utf-8")
|
||||
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
|
||||
return reply.send(content)
|
||||
try {
|
||||
const content = await readBackupContent(hit)
|
||||
reply.header("Content-Type", "text/plain; charset=utf-8")
|
||||
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
|
||||
return reply.send(content)
|
||||
} catch {
|
||||
return reply.status(404).send({ error: "Файл бэкапа не найден" })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/:id/restore", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
try {
|
||||
const result = await restoreBackupToDevice(req.params.id)
|
||||
await appendEvent({
|
||||
level: "warning",
|
||||
eventType: "backups.restore",
|
||||
sourceModule: "backups",
|
||||
title: "Восстановление бэкапа",
|
||||
message: `${result.filename} → ${result.serverName}`,
|
||||
entityType: "backup",
|
||||
entityId: req.params.id,
|
||||
})
|
||||
return reply.send(result)
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Не удалось восстановить бэкап",
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.delete("/backups/:id", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import {
|
||||
getEnabledServerById,
|
||||
listContainers,
|
||||
listContainersForServer,
|
||||
removeContainer,
|
||||
restartContainer,
|
||||
startContainer,
|
||||
stopContainer,
|
||||
} from "../services/containers-live.js"
|
||||
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
|
||||
|
||||
const RosIdBodySchema = z.object({
|
||||
rosId: z.string().min(1),
|
||||
})
|
||||
|
||||
type RosIdBody = z.infer<typeof RosIdBodySchema>
|
||||
type MutateFn = typeof startContainer
|
||||
|
||||
const containersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/containers", async (_req, reply) => {
|
||||
const containers = await listContainers()
|
||||
return reply.send({ containers })
|
||||
})
|
||||
|
||||
app.get("/servers/:id/containers", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = await getEnabledServerById(params.id)
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
const containers = await listContainersForServer(server)
|
||||
return reply.send({ containers })
|
||||
})
|
||||
|
||||
function registerMutate(path: string, fn: MutateFn) {
|
||||
app.post(path, { schema: { params: ServerIdParamSchema, body: RosIdBodySchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const body = req.body as RosIdBody
|
||||
const server = await getEnabledServerById(params.id)
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
try {
|
||||
await fn(server, body.rosId)
|
||||
return reply.send({ ok: true })
|
||||
} catch (err) {
|
||||
return reply.status(502).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка RouterOS",
|
||||
})
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
registerMutate("/servers/:id/containers/start", startContainer)
|
||||
registerMutate("/servers/:id/containers/stop", stopContainer)
|
||||
registerMutate("/servers/:id/containers/restart", restartContainer)
|
||||
registerMutate("/servers/:id/containers/remove", removeContainer)
|
||||
}
|
||||
|
||||
export default containersRoutes
|
||||
+204
-259
@@ -1,14 +1,20 @@
|
||||
import { and, asc, eq, inArray } from "drizzle-orm"
|
||||
import { and, asc, eq } from "drizzle-orm"
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { db } from "../db/index.js"
|
||||
import { filterRules, recursiveRoutes, servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
import { managedComment } from "../managed-markers.js"
|
||||
import { planBgpInApply } from "../services/config-apply-plan.js"
|
||||
import {
|
||||
hasManagedCommentPrefix,
|
||||
managedComment,
|
||||
} from "../managed-markers.js"
|
||||
appendRevisionIfChanged,
|
||||
canonicalFilterRules,
|
||||
getRevisionById,
|
||||
listRevisions,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -296,47 +302,6 @@ async function resolveRouteTargets(serverId: number, rule: ApiFilterRule): Promi
|
||||
return { gateway: rule.gateway, outIface: tid }
|
||||
}
|
||||
|
||||
function normalizeCommunity(c: string): string {
|
||||
return (c ?? "").trim()
|
||||
}
|
||||
|
||||
/** Одинаковый эффект на роутере при одинаковой community (blackhole vs gateway + out-interface) */
|
||||
async function ruleEffectSignature(serverId: number, r: ApiFilterRule): Promise<string> {
|
||||
if (r.action === "blackhole") return `bh:${normalizeCommunity(r.community)}`
|
||||
const { gateway, outIface } = await resolveRouteTargets(serverId, r)
|
||||
return `rt:${normalizeCommunity(r.community)}:${gateway}:${outIface}`
|
||||
}
|
||||
|
||||
export type FilterRouterCompareStatus = "synced" | "drift" | "missing"
|
||||
|
||||
async function compareDbRulesWithRouter(
|
||||
serverId: number,
|
||||
dbRules: ApiFilterRule[],
|
||||
remoteRules: ApiFilterRule[],
|
||||
): Promise<Record<string, FilterRouterCompareStatus>> {
|
||||
const remoteSigByComm = new Map<string, string>()
|
||||
for (const rr of remoteRules) {
|
||||
const c = normalizeCommunity(rr.community)
|
||||
if (!remoteSigByComm.has(c)) {
|
||||
remoteSigByComm.set(c, await ruleEffectSignature(serverId, rr))
|
||||
}
|
||||
}
|
||||
const out: Record<string, FilterRouterCompareStatus> = {}
|
||||
for (const dr of dbRules) {
|
||||
const c = normalizeCommunity(dr.community)
|
||||
const sigD = await ruleEffectSignature(serverId, dr)
|
||||
const sigR = remoteSigByComm.get(c)
|
||||
if (sigR === undefined) {
|
||||
out[c] = "missing"
|
||||
} else if (sigR !== sigD) {
|
||||
out[c] = "drift"
|
||||
} else {
|
||||
out[c] = "synced"
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function toRouterRuleBody(serverId: number, rules: ApiFilterRule[]): Promise<string> {
|
||||
if (rules.length === 0) return ""
|
||||
// Группируем по эффекту (action + gateway + out-interface). Communities с одним и тем же
|
||||
@@ -421,44 +386,80 @@ async function replaceDbRules(serverId: number, rules: ApiFilterRule[]) {
|
||||
)
|
||||
}
|
||||
|
||||
const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
/** Сравнение правил в БД с живым bgp-in на MikroTik (один запрос API к роутеру) */
|
||||
app.get("/filters/router-compare", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) {
|
||||
return reply.status(400).send({ error: "serverId is required" })
|
||||
}
|
||||
async function cacheRulesetsForServer(serverId: number): Promise<ApiFilterRule[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(filterRules)
|
||||
.where(eq(filterRules.serverId, serverId))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
return rows.map((r) => ({
|
||||
id: String(r.id),
|
||||
community: r.community,
|
||||
communityName: r.communityName ?? undefined,
|
||||
action: r.action,
|
||||
gateway: r.gateway,
|
||||
gatewayTunnelId: r.gatewayTunnelId,
|
||||
description: r.description,
|
||||
}))
|
||||
}
|
||||
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
async function applyFiltersToServer(
|
||||
server: ServerRow,
|
||||
rules: ApiFilterRule[],
|
||||
source: ConfigRevisionSource,
|
||||
): Promise<{ pushed: number; action: string; conflictsRemoved: number }> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosFilterRule[]>("/routing/filter/rule")
|
||||
const plan = planBgpInApply(existing, rules.length)
|
||||
const managedCommentValue = managedComment(server.name || server.host)
|
||||
|
||||
try {
|
||||
const remote = await fetchServerFilters(server)
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(filterRules)
|
||||
.where(eq(filterRules.serverId, serverId))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
if (plan.action === "patch" && plan.managedId) {
|
||||
const ruleBody = await toRouterRuleBody(server.id, rules)
|
||||
await client.patch(
|
||||
`/routing/filter/rule/${encodeURIComponent(plan.managedId)}`,
|
||||
{
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
disabled: "no",
|
||||
},
|
||||
)
|
||||
} else if (plan.action === "create") {
|
||||
const ruleBody = await toRouterRuleBody(server.id, rules)
|
||||
await client.post("/routing/filter/rule/add", {
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
})
|
||||
} else if (plan.action === "delete" && plan.managedId) {
|
||||
await client.delete(
|
||||
`/routing/filter/rule/${encodeURIComponent(plan.managedId)}`,
|
||||
)
|
||||
}
|
||||
|
||||
const dbRules: ApiFilterRule[] = rows.map(r => ({
|
||||
id: String(r.id),
|
||||
community: r.community,
|
||||
communityName: r.communityName ?? undefined,
|
||||
action: r.action,
|
||||
gateway: r.gateway,
|
||||
gatewayTunnelId: r.gatewayTunnelId,
|
||||
description: r.description,
|
||||
}))
|
||||
for (const id of plan.conflictIds) {
|
||||
await client.delete(`/routing/filter/rule/${encodeURIComponent(id)}`)
|
||||
}
|
||||
|
||||
const byCommunity = await compareDbRulesWithRouter(serverId, dbRules, remote.rules)
|
||||
return reply.send({ byCommunity })
|
||||
} catch (err) {
|
||||
app.log.error({ serverId, err: String(err) }, "filters router-compare failed")
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
await replaceDbRules(server.id, rules)
|
||||
await appendRevisionIfChanged({
|
||||
serverId: server.id,
|
||||
section: "filters",
|
||||
source,
|
||||
payload: canonicalFilterRules(rules),
|
||||
})
|
||||
|
||||
return {
|
||||
pushed: rules.length,
|
||||
action: plan.action,
|
||||
conflictsRemoved: plan.conflictIds.length,
|
||||
}
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
/** GRE с роутеров: один сервер (?serverId) или все включённые (без query) — для /gre, карты сети */
|
||||
app.get("/filters/gre-tunnels", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
@@ -487,74 +488,103 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send({ tunnels: results.flat() })
|
||||
})
|
||||
|
||||
/** Только правила фильтров из БД (без опроса MikroTik за GRE) */
|
||||
app.get("/filters/rules", async (_req, reply) => {
|
||||
/** Без serverId — cache для дашборда. С serverId — live с CHR, cache fallback. */
|
||||
app.get("/filters/rules", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const dbRulesets = await toApiRulesets(allServers)
|
||||
|
||||
return reply.send({
|
||||
rulesets: dbRulesets,
|
||||
greTunnels: [] as LiveGreTunnel[],
|
||||
})
|
||||
})
|
||||
|
||||
app.put("/filters/rules", async (req, reply) => {
|
||||
const body = req.body as { rulesets?: Array<{ serverId: string; rules: ApiFilterRule[] }> }
|
||||
const payload = body.rulesets ?? []
|
||||
const serverIds = payload.map(r => Number.parseInt(r.serverId, 10)).filter(Number.isFinite)
|
||||
if (serverIds.length > 0) {
|
||||
await db.delete(filterRules).where(inArray(filterRules.serverId, serverIds))
|
||||
}
|
||||
for (const rs of payload) {
|
||||
const sid = Number.parseInt(rs.serverId, 10)
|
||||
if (!Number.isFinite(sid)) continue
|
||||
await replaceDbRules(sid, rs.rules ?? [])
|
||||
}
|
||||
return reply.send({ ok: true })
|
||||
})
|
||||
|
||||
app.post("/filters/sync/from-router", async (_req, reply) => {
|
||||
const body = _req.body as { serverId?: string | number } | undefined
|
||||
const rawServerId = body?.serverId
|
||||
const serverId = parseDbServerId(rawServerId)
|
||||
if (serverId === null) {
|
||||
return reply.status(400).send({ error: "serverId is required" })
|
||||
const dbRulesets = await toApiRulesets(allServers)
|
||||
return reply.send({
|
||||
rulesets: dbRulesets,
|
||||
greTunnels: [] as LiveGreTunnel[],
|
||||
live: false,
|
||||
stale: false,
|
||||
})
|
||||
}
|
||||
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
const server = allServers.find((s) => s.id === serverId)
|
||||
?? (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
app.log.info({ serverId, host: server.host }, "Filters sync from router started")
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.from_router.started",
|
||||
sourceModule: "filters",
|
||||
title: "Синхронизация фильтров запущена",
|
||||
message: `${server.name || server.host} → БД`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
})
|
||||
const remote = await fetchServerFilters(server)
|
||||
await replaceDbRules(server.id, remote.rules)
|
||||
app.log.info({ serverId, totalRules: remote.rules.length }, "Filters sync from router completed")
|
||||
await appendRevisionIfChanged({
|
||||
serverId: server.id,
|
||||
section: "filters",
|
||||
source: "observed",
|
||||
payload: canonicalFilterRules(remote.rules),
|
||||
})
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
return reply.send({
|
||||
rulesets: [{ serverId: String(server.id), rules: cached }],
|
||||
greTunnels: remote.tunnels,
|
||||
live: true,
|
||||
stale: false,
|
||||
})
|
||||
} catch (err) {
|
||||
app.log.warn({ serverId, err: String(err) }, "filters live GET failed, serving cache")
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
return reply.send({
|
||||
rulesets: [{ serverId: String(server.id), rules: cached }],
|
||||
greTunnels: [] as LiveGreTunnel[],
|
||||
live: false,
|
||||
stale: true,
|
||||
error: String(err),
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.put("/filters/rules", async (req, reply) => {
|
||||
const body = req.body as {
|
||||
serverId?: string | number
|
||||
rules?: ApiFilterRule[]
|
||||
source?: ConfigRevisionSource
|
||||
}
|
||||
const serverId = parseDbServerId(body.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const rules = body.rules ?? []
|
||||
const source: ConfigRevisionSource = body.source === "copy" ? "copy" : "apply"
|
||||
try {
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.from_router.done",
|
||||
eventType: "filters.apply.started",
|
||||
sourceModule: "filters",
|
||||
title: "Синхронизация фильтров завершена",
|
||||
message: `${server.name || server.host}: ${remote.rules.length} правил`,
|
||||
title: "Применение фильтров на роутер",
|
||||
message: `${server.name || server.host}: ${rules.length} правил`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
})
|
||||
return reply.send({ ok: true, updatedServers: 1, totalRules: remote.rules.length, serverId })
|
||||
const result = await applyFiltersToServer(server, rules, source)
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.apply.done",
|
||||
sourceModule: "filters",
|
||||
title: "Фильтры применены",
|
||||
message: `${server.name || server.host}: ${result.pushed} правил (${result.action})`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
})
|
||||
return reply.send({
|
||||
ok: true,
|
||||
serverId,
|
||||
pushedRules: result.pushed,
|
||||
action: result.action,
|
||||
rules: cached,
|
||||
})
|
||||
} catch (err) {
|
||||
app.log.error({ serverId, err: String(err) }, "Filters sync from router failed")
|
||||
app.log.error({ serverId, err: String(err) }, "filters apply failed")
|
||||
await appendEvent({
|
||||
level: "critical",
|
||||
eventType: "filters.sync.from_router.failed",
|
||||
eventType: "filters.apply.failed",
|
||||
sourceModule: "filters",
|
||||
title: "Ошибка синхронизации фильтров",
|
||||
title: "Ошибка применения фильтров",
|
||||
message: `${server.name || server.host}: ${String(err)}`,
|
||||
entityType: "server",
|
||||
entityId: String(serverId),
|
||||
@@ -563,145 +593,60 @@ const filtersRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/filters/sync/to-router", async (req, reply) => {
|
||||
app.get("/filters/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "filters")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/filters/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const requestedServerId = parseDbServerId(body?.serverId)
|
||||
|
||||
const allServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const targetServers = requestedServerId !== null
|
||||
? allServers.filter(s => s.id === requestedServerId)
|
||||
: allServers
|
||||
|
||||
if (requestedServerId !== null && targetServers.length === 0) {
|
||||
return reply.status(404).send({ error: "Server not found" })
|
||||
const rev = await getRevisionById(id)
|
||||
if (!rev) return reply.status(404).send({ error: "Revision not found" })
|
||||
if (rev.section !== "filters") return reply.status(400).send({ error: "Revision section mismatch" })
|
||||
const requested = parseDbServerId(body?.serverId)
|
||||
if (requested !== null && requested !== rev.serverId) {
|
||||
return reply.status(400).send({ error: "Revision belongs to another server" })
|
||||
}
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, rev.serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
let updatedServers = 0
|
||||
let pushedRules = 0
|
||||
const errors: Array<{ serverId: number; error: string }> = []
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.sync.to_router.started",
|
||||
sourceModule: "filters",
|
||||
title: "Отправка фильтров на роутеры запущена",
|
||||
message: `Целевых серверов: ${targetServers.length}`,
|
||||
payload: { requestedServerId },
|
||||
})
|
||||
|
||||
for (const server of targetServers) {
|
||||
try {
|
||||
app.log.info({ serverId: server.id, host: server.host }, "Filters sync to router started")
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosFilterRule[]>("/routing/filter/rule")
|
||||
|
||||
const isInBgpIn = (r: RosFilterRule) =>
|
||||
(r.chain ?? "").trim().toLowerCase() === "bgp-in"
|
||||
const managedCommentValue = managedComment(server.name || server.host)
|
||||
|
||||
// Уже созданное нами правило — будем PATCH'ить, чтобы сохранить ID/позицию в цепочке.
|
||||
const managedRule = existing.find(
|
||||
r => isInBgpIn(r) && hasManagedCommentPrefix(r.comment ?? ""),
|
||||
)
|
||||
|
||||
// Конфликтующие легаси-правила в bgp-in (без нашего comment, но с bgp-communities) —
|
||||
// удаляем после успешного upsert: иначе старое правило с `else { reject; }`
|
||||
// отрабатывает первым и перебивает наш upsert.
|
||||
const conflictIds = existing
|
||||
.filter(r =>
|
||||
isInBgpIn(r) &&
|
||||
!hasManagedCommentPrefix(r.comment ?? "") &&
|
||||
/bgp-communities/i.test(r.rule ?? ""),
|
||||
)
|
||||
.map(r => r[".id"])
|
||||
.filter((id): id is string => Boolean(id))
|
||||
|
||||
const rows = await db.select().from(filterRules)
|
||||
.where(and(eq(filterRules.serverId, server.id)))
|
||||
.orderBy(asc(filterRules.sortOrder))
|
||||
|
||||
const rules: ApiFilterRule[] = rows.map(r => ({
|
||||
id: String(r.id),
|
||||
community: r.community,
|
||||
communityName: r.communityName ?? undefined,
|
||||
action: r.action,
|
||||
gateway: r.gateway,
|
||||
gatewayTunnelId: r.gatewayTunnelId,
|
||||
description: r.description,
|
||||
}))
|
||||
|
||||
// Upsert: PATCH существующего managed-правила или POST /add нового.
|
||||
// Если ошибка — конфликтные правила НЕ удаляем (роутер не остаётся с пустым bgp-in).
|
||||
// Путь `/routing/filter/rule/add` обязателен: голый POST на коллекцию RouterOS REST
|
||||
// трактует как «вызов команды» и отдаёт 400 «no such command».
|
||||
// См. https://help.mikrotik.com/docs/spaces/ROS/pages/47579162/REST+API
|
||||
if (rules.length > 0) {
|
||||
const ruleBody = await toRouterRuleBody(server.id, rules)
|
||||
if (managedRule && managedRule[".id"]) {
|
||||
await client.patch(
|
||||
`/routing/filter/rule/${encodeURIComponent(managedRule[".id"])}`,
|
||||
{
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
disabled: "no",
|
||||
},
|
||||
)
|
||||
app.log.info({ serverId: server.id, id: managedRule[".id"] }, "bgp-in rule updated")
|
||||
} else {
|
||||
await client.post("/routing/filter/rule/add", {
|
||||
chain: "bgp-in",
|
||||
comment: managedCommentValue,
|
||||
rule: ruleBody,
|
||||
})
|
||||
app.log.info({ serverId: server.id }, "bgp-in rule created")
|
||||
}
|
||||
pushedRules += rules.length
|
||||
} else if (managedRule && managedRule[".id"]) {
|
||||
// В БД нет правил → удаляем наш managed-rule на роутере.
|
||||
await client.delete(
|
||||
`/routing/filter/rule/${encodeURIComponent(managedRule[".id"])}`,
|
||||
)
|
||||
app.log.info({ serverId: server.id }, "bgp-in rule removed (no rules in DB)")
|
||||
}
|
||||
|
||||
for (const id of conflictIds) {
|
||||
await client.delete(`/routing/filter/rule/${encodeURIComponent(id)}`)
|
||||
}
|
||||
|
||||
updatedServers += 1
|
||||
app.log.info(
|
||||
{
|
||||
serverId: server.id,
|
||||
mode: managedRule ? "patch" : "create",
|
||||
conflictsRemoved: conflictIds.length,
|
||||
pushed: rules.length,
|
||||
},
|
||||
"Filters sync to router completed",
|
||||
)
|
||||
} catch (err) {
|
||||
app.log.error({ serverId: server.id, err: String(err) }, "filters sync to-router failed")
|
||||
errors.push({ serverId: server.id, error: String(err) })
|
||||
const raw = Array.isArray(rev.payload) ? rev.payload : []
|
||||
const rules: ApiFilterRule[] = raw.map((item, idx) => {
|
||||
const r = item as Partial<ApiFilterRule>
|
||||
return {
|
||||
id: `rev-${idx}`,
|
||||
community: r.community ?? "",
|
||||
communityName: r.communityName,
|
||||
action: r.action === "blackhole" ? "blackhole" : "route",
|
||||
gateway: r.gateway ?? "",
|
||||
gatewayTunnelId: r.gatewayTunnelId ?? "",
|
||||
description: r.description ?? "",
|
||||
}
|
||||
}
|
||||
})
|
||||
|
||||
await appendEvent({
|
||||
level: errors.length === 0 ? "info" : "warning",
|
||||
eventType: errors.length === 0 ? "filters.sync.to_router.done" : "filters.sync.to_router.partial",
|
||||
sourceModule: "filters",
|
||||
title: errors.length === 0 ? "Отправка фильтров завершена" : "Отправка фильтров завершена с ошибками",
|
||||
message: `Успешно: ${updatedServers}, ошибок: ${errors.length}, правил: ${pushedRules}`,
|
||||
payload: {
|
||||
updatedServers,
|
||||
pushedRules,
|
||||
errors,
|
||||
},
|
||||
})
|
||||
return reply.send({
|
||||
ok: errors.length === 0,
|
||||
updatedServers,
|
||||
pushedRules,
|
||||
errors,
|
||||
})
|
||||
try {
|
||||
const result = await applyFiltersToServer(server, rules, "rollback")
|
||||
const cached = await cacheRulesetsForServer(server.id)
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "filters.rollback.done",
|
||||
sourceModule: "filters",
|
||||
title: "Откат фильтров",
|
||||
message: `${server.name || server.host}: ${result.pushed} правил`,
|
||||
entityType: "server",
|
||||
entityId: String(server.id),
|
||||
})
|
||||
return reply.send({ ok: true, rules: cached, pushedRules: result.pushed })
|
||||
} catch (err) {
|
||||
app.log.error({ serverId: server.id, err: String(err) }, "filters restore failed")
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -2,7 +2,20 @@ import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { z } from "zod"
|
||||
import { MikrotikClient, MikrotikError, encodeRosId, firewallRestPath } from "../services/mikrotik.js"
|
||||
import { getEnabledServerById } from "../services/wireguard-live.js"
|
||||
import { listFirewallAll } from "../services/firewall-live.js"
|
||||
import {
|
||||
captureFirewallSnapshot,
|
||||
fetchFirewallState,
|
||||
listFirewallAll,
|
||||
} from "../services/firewall-live.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
listRevisions,
|
||||
loadRevisionForRestore,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
import { parseFirewallSnapshot, planFirewallRestore } from "../services/entity-snapshots.js"
|
||||
import { executeRosOps } from "../services/ros-ops.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import type { FirewallFamily, FirewallTable } from "../types/server.js"
|
||||
|
||||
const FamilySchema = z.enum(["ip", "ip6"])
|
||||
@@ -120,6 +133,20 @@ async function requireServer(serverId: string) {
|
||||
return await getEnabledServerById(serverId)
|
||||
}
|
||||
|
||||
async function recordFirewall(
|
||||
server: NonNullable<Awaited<ReturnType<typeof requireServer>>>,
|
||||
source: ConfigRevisionSource,
|
||||
) {
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "firewall",
|
||||
source,
|
||||
capture: () => captureFirewallSnapshot(server),
|
||||
})
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/firewall/all", async (_req, reply) => {
|
||||
const data = await listFirewallAll()
|
||||
@@ -138,6 +165,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)
|
||||
try {
|
||||
await client.put(path, ruleToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.status(201).send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -156,6 +184,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family as FirewallFamily, body.table as FirewallTable)}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, ruleToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -174,6 +203,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, { disabled: body.disabled ? "yes" : "no" })
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -192,6 +222,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, body.table)}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.delete(path)
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -213,6 +244,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
numbers: body.rosId,
|
||||
...(body.destinationRosId ? { destination: body.destinationRosId } : {}),
|
||||
})
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -230,6 +262,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await client.put(firewallRestPath(body.family, "address-list"), addressToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.status(201).send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -248,6 +281,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, addressToRos(body))
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -266,6 +300,7 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.patch(path, { disabled: body.disabled ? "yes" : "no" })
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
@@ -284,11 +319,48 @@ const firewallRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const path = `${firewallRestPath(body.family, "address-list")}/${encodeRosId(body.rosId)}`
|
||||
try {
|
||||
await client.delete(path)
|
||||
await recordFirewall(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.get("/firewall/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "firewall")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/firewall/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const loaded = await loadRevisionForRestore({
|
||||
id,
|
||||
section: "firewall",
|
||||
requestedServerId: parseDbServerId(body?.serverId),
|
||||
})
|
||||
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
|
||||
const client = MikrotikClient.fromServer(loaded.server)
|
||||
try {
|
||||
const desired = parseFirewallSnapshot(loaded.row.payload)
|
||||
const state = await fetchFirewallState(loaded.server)
|
||||
const ops = planFirewallRestore(desired, {
|
||||
rules: state.liveRules,
|
||||
addressLists: state.liveLists,
|
||||
})
|
||||
await executeRosOps(client, ops)
|
||||
await recordFirewall(loaded.server, "rollback")
|
||||
const next = await fetchFirewallState(loaded.server)
|
||||
return reply.send({ ok: true, rules: next.rules, addressLists: next.addressLists })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default firewallRoutes
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { geoipSettingsPatchSchema } from "@mmapp/contracts/geoip"
|
||||
import { refreshScheduler } from "../services/scheduler.js"
|
||||
import { getGeoipSettings, updateGeoipSettings } from "../services/geoip-settings.js"
|
||||
import {
|
||||
GEOIP_ASN_FILE,
|
||||
GEOIP_COUNTRY_FILE,
|
||||
geoipReadersStatus,
|
||||
initGeoip,
|
||||
} from "../services/traffic-flow-geoip.js"
|
||||
import { collectGeoipUpdateOnce, getGeoipUpdateState } from "../services/geoip-update-collector.js"
|
||||
|
||||
async function buildGeoipStatus() {
|
||||
await initGeoip()
|
||||
const readers = geoipReadersStatus()
|
||||
return {
|
||||
ready: readers.countryLoaded && readers.asnLoaded,
|
||||
countryLoaded: readers.countryLoaded,
|
||||
asnLoaded: readers.asnLoaded,
|
||||
countryFile: GEOIP_COUNTRY_FILE,
|
||||
asnFile: GEOIP_ASN_FILE,
|
||||
dir: readers.dir,
|
||||
running: getGeoipUpdateState().running,
|
||||
settings: await getGeoipSettings(),
|
||||
}
|
||||
}
|
||||
|
||||
const geoipRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/geoip", async (_req, reply) => {
|
||||
return reply.send(await buildGeoipStatus())
|
||||
})
|
||||
|
||||
app.put("/geoip", async (req, reply) => {
|
||||
const parsed = geoipSettingsPatchSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply
|
||||
.status(400)
|
||||
.send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
await updateGeoipSettings(parsed.data)
|
||||
await refreshScheduler()
|
||||
return reply.send({ ok: true, status: await buildGeoipStatus() })
|
||||
})
|
||||
|
||||
app.post("/geoip/update", async (_req, reply) => {
|
||||
try {
|
||||
const snapshot = await collectGeoipUpdateOnce({ force: true })
|
||||
return reply.send({ ok: !snapshot.fatalError && snapshot.errors.length === 0, snapshot })
|
||||
} catch (e) {
|
||||
const status = (e as { statusCode?: number }).statusCode ?? 502
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
return reply.status(status).send({ error: msg })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default geoipRoutes
|
||||
@@ -0,0 +1,229 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { z } from "zod"
|
||||
import { MikrotikClient, MikrotikError } from "../services/mikrotik.js"
|
||||
import { getEnabledServerById } from "../services/wireguard-live.js"
|
||||
import {
|
||||
captureGreSnapshot,
|
||||
fetchGreState,
|
||||
formatKeepalive,
|
||||
listGreTunnels,
|
||||
parseKeepalive,
|
||||
} from "../services/gre-live.js"
|
||||
import {
|
||||
canonicalGreSnapshot,
|
||||
parseGreSnapshot,
|
||||
planGreCreate,
|
||||
planGreDelete,
|
||||
planGreRestore,
|
||||
} from "../services/entity-snapshots.js"
|
||||
import { executeRosOps } from "../services/ros-ops.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
listRevisions,
|
||||
loadRevisionForRestore,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
|
||||
const TunnelWriteSchema = z.object({
|
||||
serverId: z.string().min(1),
|
||||
name: z.string().min(1),
|
||||
rosId: z.string().optional(),
|
||||
localAddress: z.string().optional(),
|
||||
remoteAddress: z.string().min(1),
|
||||
localInnerIp: z.string().optional(),
|
||||
remoteInnerIp: z.string().optional(),
|
||||
comment: z.string().optional(),
|
||||
enabled: z.boolean().optional(),
|
||||
mtu: z.number().optional(),
|
||||
keepaliveInterval: z.number().optional(),
|
||||
keepaliveRetries: z.number().optional(),
|
||||
dscp: z.union([z.literal("inherit"), z.number(), z.string()]).optional(),
|
||||
clampTcpMss: z.boolean().optional(),
|
||||
allowFastPath: z.boolean().optional(),
|
||||
ipsecSecret: z.string().optional(),
|
||||
})
|
||||
|
||||
const TunnelKeySchema = z.object({
|
||||
serverId: z.string().min(1),
|
||||
rosId: z.string().optional(),
|
||||
name: z.string().optional(),
|
||||
disabled: z.boolean().optional(),
|
||||
})
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
function rosErr(e: unknown): string {
|
||||
if (e instanceof MikrotikError) return e.message
|
||||
if (e instanceof Error) return e.message
|
||||
return String(e)
|
||||
}
|
||||
|
||||
async function recordGre(
|
||||
server: NonNullable<Awaited<ReturnType<typeof getEnabledServerById>>>,
|
||||
source: ConfigRevisionSource,
|
||||
) {
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "gre",
|
||||
source,
|
||||
capture: () => captureGreSnapshot(server),
|
||||
})
|
||||
}
|
||||
|
||||
function tunnelFromBody(body: z.infer<typeof TunnelWriteSchema> & { keepaliveInterval?: number; keepaliveRetries?: number }) {
|
||||
const dscp = body.dscp == null
|
||||
? "inherit"
|
||||
: typeof body.dscp === "number"
|
||||
? String(body.dscp)
|
||||
: body.dscp
|
||||
const keepalive = body.keepaliveInterval === undefined && body.keepaliveRetries === undefined
|
||||
? undefined
|
||||
: formatKeepalive(body.keepaliveInterval ?? 0, body.keepaliveRetries ?? 10)
|
||||
return canonicalGreSnapshot({
|
||||
tunnels: [{
|
||||
name: body.name,
|
||||
localAddress: body.localAddress ?? "",
|
||||
remoteAddress: body.remoteAddress,
|
||||
localInnerIp: body.localInnerIp ?? "",
|
||||
remoteInnerIp: body.remoteInnerIp ?? "",
|
||||
comment: body.comment ?? "",
|
||||
disabled: body.enabled === false,
|
||||
mtu: body.mtu ?? 1476,
|
||||
keepalive: keepalive ?? "0",
|
||||
dscp,
|
||||
clampTcpMss: body.clampTcpMss,
|
||||
allowFastPath: body.allowFastPath,
|
||||
ipsecSecret: body.ipsecSecret ?? "",
|
||||
}],
|
||||
}).tunnels[0]!
|
||||
}
|
||||
|
||||
const greRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/gre/tunnels", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const sid = parseDbServerId(q.serverId)
|
||||
const result = await listGreTunnels({ serverId: sid !== null ? String(sid) : undefined })
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
app.post("/gre/tunnels", async (req, reply) => {
|
||||
const parsed = TunnelWriteSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
const tunnel = tunnelFromBody(body)
|
||||
await executeRosOps(client, planGreCreate(tunnel))
|
||||
await recordGre(server, "apply")
|
||||
const state = await fetchGreState(server)
|
||||
const created = state.tunnels.find((t) => t.name === tunnel.name)
|
||||
return reply.status(201).send(created ?? { ok: true, name: tunnel.name })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.patch("/gre/tunnels", async (req, reply) => {
|
||||
const parsed = TunnelWriteSchema.partial().required({ serverId: true }).safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
try {
|
||||
const state = await fetchGreState(server)
|
||||
const live = state.gre.find((g) =>
|
||||
(body.rosId && g.rosId === body.rosId) || (body.name && g.name === body.name),
|
||||
)
|
||||
if (!live) return reply.status(404).send({ error: "Туннель не найден" })
|
||||
const merged = tunnelFromBody({
|
||||
serverId: body.serverId,
|
||||
name: body.name || live.name,
|
||||
localAddress: body.localAddress ?? live.localAddress,
|
||||
remoteAddress: body.remoteAddress || live.remoteAddress,
|
||||
localInnerIp: body.localInnerIp ?? state.addrs.find((a) => a.interfaceName === live.name)?.address ?? "",
|
||||
remoteInnerIp: body.remoteInnerIp,
|
||||
comment: body.comment ?? live.comment,
|
||||
enabled: body.enabled ?? !live.disabled,
|
||||
mtu: body.mtu ?? live.mtu,
|
||||
keepaliveInterval: body.keepaliveInterval ?? parseKeepalive(live.keepalive).interval,
|
||||
keepaliveRetries: body.keepaliveRetries ?? parseKeepalive(live.keepalive).retries,
|
||||
dscp: body.dscp ?? live.dscp,
|
||||
clampTcpMss: body.clampTcpMss ?? live.clampTcpMss,
|
||||
allowFastPath: body.allowFastPath ?? live.allowFastPath,
|
||||
ipsecSecret: body.ipsecSecret ?? live.ipsecSecret,
|
||||
})
|
||||
const ops = planGreRestore(
|
||||
{ tunnels: state.snapshot.tunnels.map((t) => t.name === live.name ? merged : t) },
|
||||
{ gre: state.gre, addrs: state.addrs },
|
||||
)
|
||||
await executeRosOps(state.client, ops)
|
||||
await recordGre(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.delete("/gre/tunnels", async (req, reply) => {
|
||||
const parsed = TunnelKeySchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
const body = parsed.data
|
||||
const server = await getEnabledServerById(body.serverId)
|
||||
if (!server) return reply.status(404).send({ error: "Сервер не найден" })
|
||||
try {
|
||||
const state = await fetchGreState(server)
|
||||
const live = state.gre.find((g) =>
|
||||
(body.rosId && g.rosId === body.rosId) || (body.name && g.name === body.name),
|
||||
)
|
||||
if (!live) return reply.status(404).send({ error: "Туннель не найден" })
|
||||
await executeRosOps(state.client, planGreDelete(live.name, { gre: state.gre, addrs: state.addrs }))
|
||||
await recordGre(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
|
||||
app.get("/gre/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "gre")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/gre/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const loaded = await loadRevisionForRestore({
|
||||
id,
|
||||
section: "gre",
|
||||
requestedServerId: parseDbServerId(body?.serverId),
|
||||
})
|
||||
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
|
||||
try {
|
||||
const desired = parseGreSnapshot(loaded.row.payload)
|
||||
const state = await fetchGreState(loaded.server)
|
||||
const ops = planGreRestore(desired, { gre: state.gre, addrs: state.addrs })
|
||||
await executeRosOps(state.client, ops)
|
||||
await recordGre(loaded.server, "rollback")
|
||||
const next = await fetchGreState(loaded.server)
|
||||
return reply.send({ ok: true, tunnels: next.tunnels })
|
||||
} catch (e) {
|
||||
return reply.status(502).send({ error: `RouterOS: ${rosErr(e)}` })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default greRoutes
|
||||
@@ -6,9 +6,10 @@ import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
|
||||
import type {
|
||||
RosOspfNeighbor, RosOspfArea, RosOspfInterfaceTemplate, RosOspfInstance,
|
||||
RosBfdSession,
|
||||
OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, BfdSessionRead,
|
||||
RosBfdSession, RosIpRoute,
|
||||
OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, OspfRouteRead, BfdSessionRead,
|
||||
} from "../types/server.js"
|
||||
import { parseOspfGateway, parseOspfRouteType } from "../services/ospf-route-parse.js"
|
||||
import { z } from "zod"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
@@ -74,14 +75,15 @@ function parseAddrIface(addr: string): { ip: string; iface: string } {
|
||||
/** Fetch all OSPF + BFD data for one server */
|
||||
async function fetchServerOspf(server: ServerRow) {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [neighbors, areas, ifaceTemplates, instances, bfdSessions] = await Promise.all([
|
||||
const [neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes] = await Promise.all([
|
||||
client.getOspfNeighbors(),
|
||||
client.getOspfAreas(),
|
||||
client.getOspfInterfaceTemplates(),
|
||||
client.getOspfInstances(),
|
||||
client.getBfdSessions().catch(() => [] as RosBfdSession[]), // BFD is optional
|
||||
client.getIpRoutes().catch(() => [] as RosIpRoute[]),
|
||||
])
|
||||
return { neighbors, areas, ifaceTemplates, instances, bfdSessions }
|
||||
return { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes }
|
||||
}
|
||||
|
||||
// ── BFD parser ────────────────────────────────────────────────────────────────
|
||||
@@ -200,6 +202,29 @@ function parseInstances(
|
||||
}))
|
||||
}
|
||||
|
||||
function parseOspfRoutes(server: ServerRow, routes: RosIpRoute[]): OspfRouteRead[] {
|
||||
const out: OspfRouteRead[] = []
|
||||
for (const [idx, r] of routes.entries()) {
|
||||
const type = parseOspfRouteType(r)
|
||||
if (!type) continue
|
||||
const { nextHop, via } = parseOspfGateway(r)
|
||||
const metric = parseInt(r["ospf-metric"] ?? r.distance ?? "0") || 0
|
||||
out.push({
|
||||
id: r[".id"] ?? String(idx),
|
||||
serverId: server.id,
|
||||
serverName: server.name || server.host,
|
||||
serverSite: server.site,
|
||||
destination: r["dst-address"] ?? "",
|
||||
type,
|
||||
cost: metric,
|
||||
nextHop,
|
||||
via,
|
||||
area: r["ospf-area"] ?? "",
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function calcRouteScore(pingMs: number, dlMbps: number, ulMbps: number, pingWeight: number) {
|
||||
const pingScore = Math.max(0, 100 - pingMs * 0.6)
|
||||
const speedScore = Math.min(100, (dlMbps + ulMbps) / 18)
|
||||
@@ -584,16 +609,17 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const perServer = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
try {
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server)
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
|
||||
const areaMap = buildAreaMap(areas)
|
||||
return {
|
||||
neighbors: parseNeighbors(server, neighbors, areaMap),
|
||||
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
|
||||
instances: parseInstances(server, instances),
|
||||
bfdSessions: parseBfdSessions(server, bfdSessions),
|
||||
routes: parseOspfRoutes(server, ipRoutes),
|
||||
}
|
||||
} catch {
|
||||
return { neighbors: [], interfaces: [], instances: [], bfdSessions: [] }
|
||||
return { neighbors: [], interfaces: [], instances: [], bfdSessions: [], routes: [] }
|
||||
}
|
||||
}),
|
||||
)
|
||||
@@ -603,6 +629,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
interfaces: perServer.flatMap(r => r.interfaces),
|
||||
instances: perServer.flatMap(r => r.instances),
|
||||
bfdSessions: perServer.flatMap(r => r.bfdSessions),
|
||||
routes: perServer.flatMap(r => r.routes),
|
||||
})
|
||||
})
|
||||
|
||||
@@ -634,13 +661,14 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
try {
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server)
|
||||
const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
|
||||
const areaMap = buildAreaMap(areas)
|
||||
return reply.send({
|
||||
neighbors: parseNeighbors(server, neighbors, areaMap),
|
||||
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
|
||||
instances: parseInstances(server, instances),
|
||||
bfdSessions: parseBfdSessions(server, bfdSessions),
|
||||
routes: parseOspfRoutes(server, ipRoutes),
|
||||
areas: areas.map(a => ({ name: a.name, areaId: a["area-id"] ?? "0.0.0.0", type: a.type, disabled: a.disabled === "true", inactive: a.inactive === "true", instance: a.instance })),
|
||||
})
|
||||
} catch (err) {
|
||||
|
||||
@@ -1,13 +1,23 @@
|
||||
import { asc, eq } from "drizzle-orm"
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { db } from "../db/index.js"
|
||||
import { recursiveRoutes, servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import { managedRecursiveComment } from "../managed-markers.js"
|
||||
import {
|
||||
hasManagedRecursiveComment,
|
||||
managedRecursiveComment,
|
||||
} from "../managed-markers.js"
|
||||
mapRosManagedRoutes,
|
||||
planRecursiveApply,
|
||||
userRecursiveComment,
|
||||
} from "../services/config-apply-plan.js"
|
||||
import {
|
||||
appendRevisionIfChanged,
|
||||
canonicalRecursiveRoutes,
|
||||
getRevisionById,
|
||||
listRevisions,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -51,27 +61,6 @@ interface RecursiveRouteDto {
|
||||
disabled: boolean
|
||||
}
|
||||
|
||||
function isIpGateway(gw: string): boolean {
|
||||
return /^\d{1,3}(\.\d{1,3}){3}(?:%\S+)?$/.test(gw.trim())
|
||||
}
|
||||
|
||||
function isRecursiveRoute(r: RosRoute): boolean {
|
||||
if ((r.static ?? "false") !== "true") return false
|
||||
if ((r.dynamic ?? "false") === "true") return false
|
||||
if ((r.blackhole ?? "false") === "true") return false
|
||||
if ((r.unreachable ?? "false") === "true") return false
|
||||
if ((r.prohibit ?? "false") === "true") return false
|
||||
const dst = r["dst-address"] ?? ""
|
||||
const gw = r.gateway ?? ""
|
||||
if (!dst || !gw) return false
|
||||
return isIpGateway(gw)
|
||||
}
|
||||
|
||||
function hasRecursiveCommentMask(comment: string | undefined): boolean {
|
||||
if (!comment) return false
|
||||
return /^recursive:\s*/i.test(comment.trim())
|
||||
}
|
||||
|
||||
function splitGateway(raw: string): { ip: string; name: string } | null {
|
||||
const v = raw.trim()
|
||||
if (!v) return null
|
||||
@@ -102,6 +91,21 @@ async function mapDbRoutes(serverId: number): Promise<RecursiveRouteDto[]> {
|
||||
}))
|
||||
}
|
||||
|
||||
function mergeCachedCountry(
|
||||
live: RecursiveRouteDto[],
|
||||
cached: RecursiveRouteDto[],
|
||||
): RecursiveRouteDto[] {
|
||||
return live.map((row) => {
|
||||
if (row.country) return row
|
||||
const match = cached.find((c) =>
|
||||
c.dstAddress === row.dstAddress &&
|
||||
c.gateway === row.gateway &&
|
||||
c.distance === row.distance,
|
||||
)
|
||||
return match?.country ? { ...row, country: match.country } : row
|
||||
})
|
||||
}
|
||||
|
||||
function toRouterPayload(route: RecursiveRouteDto): Record<string, string> {
|
||||
return {
|
||||
"dst-address": route.dstAddress,
|
||||
@@ -132,7 +136,7 @@ async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
|
||||
routingTable: r.routingTable || "main",
|
||||
checkGateway: r.checkGateway ?? "",
|
||||
country: r.country ?? "",
|
||||
comment: r.comment ?? "",
|
||||
comment: userRecursiveComment(r.comment),
|
||||
disabled: r.disabled,
|
||||
createdAt: now,
|
||||
updatedAt: now,
|
||||
@@ -140,6 +144,34 @@ async function replaceDbRoutes(serverId: number, routes: RecursiveRouteDto[]) {
|
||||
)
|
||||
}
|
||||
|
||||
async function applyRecursiveToServer(
|
||||
server: ServerRow,
|
||||
routes: RecursiveRouteDto[],
|
||||
source: ConfigRevisionSource,
|
||||
): Promise<{ pushed: number; deleted: number }> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosRoute[]>("/ip/route")
|
||||
const { deleteIds } = planRecursiveApply(existing)
|
||||
|
||||
for (const route of routes) {
|
||||
await client.post("/ip/route", toRouterPayload(route))
|
||||
}
|
||||
for (const id of deleteIds) {
|
||||
await client.delete(`/ip/route/${encodeURIComponent(id)}`)
|
||||
}
|
||||
|
||||
await replaceDbRoutes(server.id, routes)
|
||||
await appendRevisionIfChanged({
|
||||
serverId: server.id,
|
||||
section: "recursive-routes",
|
||||
source,
|
||||
payload: canonicalRecursiveRoutes(routes),
|
||||
})
|
||||
return { pushed: routes.length, deleted: deleteIds.length }
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/recursive-routes/gateways", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
@@ -175,79 +207,108 @@ const recursiveRoutesPlugin: FastifyPluginAsyncZod = async (app) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
return reply.send({ routes: await mapDbRoutes(serverId) })
|
||||
})
|
||||
|
||||
app.put("/recursive-routes", async (req, reply) => {
|
||||
const body = req.body as { serverId?: string | number; routes?: RecursiveRouteDto[] }
|
||||
const serverId = parseDbServerId(body.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
await replaceDbRoutes(serverId, body.routes ?? [])
|
||||
return reply.send({ ok: true })
|
||||
})
|
||||
|
||||
app.post("/recursive-routes/sync/from-router", async (req, reply) => {
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const serverId = parseDbServerId(body?.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server: ServerRow | undefined = (await db
|
||||
.select().from(servers)
|
||||
.where(eq(servers.id, serverId))
|
||||
.limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const cached = await mapDbRoutes(serverId)
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const rosRoutes = await client.get<RosRoute[]>("/ip/route")
|
||||
const rec = rosRoutes.filter(r =>
|
||||
isRecursiveRoute(r) && hasRecursiveCommentMask(r.comment),
|
||||
)
|
||||
const mapped: RecursiveRouteDto[] = rec.map((r, i) => ({
|
||||
id: r[".id"] ?? `ros-${i}`,
|
||||
dstAddress: r["dst-address"] ?? "",
|
||||
gateway: r.gateway ?? "",
|
||||
distance: Number.parseInt(r.distance ?? "1", 10) || 1,
|
||||
scope: r.scope ? (Number.parseInt(r.scope, 10) || null) : null,
|
||||
targetScope: r["target-scope"] ? (Number.parseInt(r["target-scope"], 10) || null) : null,
|
||||
routingTable: r["routing-table"] ?? "main",
|
||||
checkGateway: r["check-gateway"] ?? "",
|
||||
country: "",
|
||||
comment: r.comment ?? "",
|
||||
disabled: r.disabled === "true",
|
||||
}))
|
||||
await replaceDbRoutes(serverId, mapped)
|
||||
return reply.send({ ok: true, serverId, totalRoutes: mapped.length })
|
||||
const live = mergeCachedCountry(mapRosManagedRoutes(rosRoutes), cached)
|
||||
await replaceDbRoutes(serverId, live)
|
||||
await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "recursive-routes",
|
||||
source: "observed",
|
||||
payload: canonicalRecursiveRoutes(live),
|
||||
})
|
||||
const stored = await mapDbRoutes(serverId)
|
||||
return reply.send({ routes: stored, live: true, stale: false })
|
||||
} catch (err) {
|
||||
app.log.warn({ serverId, err: String(err) }, "recursive live GET failed, serving cache")
|
||||
return reply.send({
|
||||
routes: cached,
|
||||
live: false,
|
||||
stale: true,
|
||||
error: String(err),
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.put("/recursive-routes", async (req, reply) => {
|
||||
const body = req.body as {
|
||||
serverId?: string | number
|
||||
routes?: RecursiveRouteDto[]
|
||||
source?: ConfigRevisionSource
|
||||
}
|
||||
const serverId = parseDbServerId(body.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
const routes = (body.routes ?? []).map((r) => ({
|
||||
...r,
|
||||
comment: userRecursiveComment(r.comment),
|
||||
}))
|
||||
const source: ConfigRevisionSource = body.source === "copy" ? "copy" : "apply"
|
||||
try {
|
||||
const result = await applyRecursiveToServer(server, routes, source)
|
||||
const stored = await mapDbRoutes(serverId)
|
||||
return reply.send({ ok: true, routes: stored, pushedRoutes: result.pushed })
|
||||
} catch (err) {
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/recursive-routes/sync/to-router", async (req, reply) => {
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const serverId = parseDbServerId(body?.serverId)
|
||||
app.get("/recursive-routes/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, serverId)).limit(1))[0]
|
||||
const revisions = await listRevisions(serverId, "recursive-routes")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/recursive-routes/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const rev = await getRevisionById(id)
|
||||
if (!rev) return reply.status(404).send({ error: "Revision not found" })
|
||||
if (rev.section !== "recursive-routes") {
|
||||
return reply.status(400).send({ error: "Revision section mismatch" })
|
||||
}
|
||||
const requested = parseDbServerId(body?.serverId)
|
||||
if (requested !== null && requested !== rev.serverId) {
|
||||
return reply.status(400).send({ error: "Revision belongs to another server" })
|
||||
}
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, rev.serverId)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
|
||||
const raw = Array.isArray(rev.payload) ? rev.payload : []
|
||||
const routes: RecursiveRouteDto[] = raw.map((item, idx) => {
|
||||
const r = item as Partial<RecursiveRouteDto>
|
||||
return {
|
||||
id: `rev-${idx}`,
|
||||
dstAddress: r.dstAddress ?? "",
|
||||
gateway: r.gateway ?? "",
|
||||
distance: r.distance ?? 1,
|
||||
scope: r.scope ?? null,
|
||||
targetScope: r.targetScope ?? null,
|
||||
routingTable: r.routingTable || "main",
|
||||
checkGateway: r.checkGateway ?? "",
|
||||
country: r.country ?? "",
|
||||
comment: userRecursiveComment(r.comment),
|
||||
disabled: Boolean(r.disabled),
|
||||
}
|
||||
})
|
||||
|
||||
try {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const existing = await client.get<RosRoute[]>("/ip/route")
|
||||
const managed = existing.filter(r => hasManagedRecursiveComment(r.comment ?? ""))
|
||||
for (const r of managed) {
|
||||
if (!r[".id"]) continue
|
||||
await client.delete(`/ip/route/${encodeURIComponent(r[".id"])}`)
|
||||
}
|
||||
|
||||
const dbRows = await mapDbRoutes(serverId)
|
||||
for (const route of dbRows) {
|
||||
await client.post("/ip/route", toRouterPayload(route))
|
||||
}
|
||||
|
||||
return reply.send({ ok: true, serverId, pushedRoutes: dbRows.length })
|
||||
const result = await applyRecursiveToServer(server, routes, "rollback")
|
||||
const stored = await mapDbRoutes(server.id)
|
||||
return reply.send({ ok: true, routes: stored, pushedRoutes: result.pushed })
|
||||
} catch (err) {
|
||||
return reply.status(500).send({ error: String(err) })
|
||||
}
|
||||
|
||||
@@ -2,6 +2,9 @@ import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { count } from "drizzle-orm"
|
||||
import { listCertificatesFromServers } from "../services/certificates-service.js"
|
||||
import { countWireGuardInterfaces } from "../services/wireguard-live.js"
|
||||
import { countVxlanTunnels } from "../services/vxlan-live.js"
|
||||
import { countContainers } from "../services/containers-live.js"
|
||||
import { countBgpSessions } from "../services/bgp-peers-live.js"
|
||||
import { db } from "../db/index.js"
|
||||
import {
|
||||
filterRules,
|
||||
@@ -24,9 +27,12 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const uptimeProbesTotal = await tableCount(uptimeProbes)
|
||||
const uptimeSpeedProbesTotal = await tableCount(uptimeSpeedProbes)
|
||||
const recursiveRoutesTotal = await tableCount(recursiveRoutes)
|
||||
const [certRes, wireguardTotal] = await Promise.all([
|
||||
const [certRes, wireguardTotal, bgpTotal, vxlanTotal, containersTotal] = await Promise.all([
|
||||
listCertificatesFromServers(),
|
||||
countWireGuardInterfaces().catch(() => 0),
|
||||
countBgpSessions().catch(() => 0),
|
||||
countVxlanTunnels().catch(() => 0),
|
||||
countContainers().catch(() => 0),
|
||||
])
|
||||
const certificatesTotal = certRes.certificates.length
|
||||
const usersTotal = (await listUsers()).length
|
||||
@@ -41,6 +47,9 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
certificates: certificatesTotal,
|
||||
wireguard: wireguardTotal,
|
||||
users: usersTotal,
|
||||
bgpSessions: bgpTotal,
|
||||
vxlan: vxlanTotal,
|
||||
containers: containersTotal,
|
||||
})
|
||||
})
|
||||
}
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { statisticsPivotQuerySchema, statisticsQuerySchema } from "@mmapp/contracts/statistics"
|
||||
import { getStatistics, getStatisticsPivot, pivotDimsConflict } from "../services/statistics-aggregate.js"
|
||||
|
||||
const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/statistics", async (req, reply) => {
|
||||
const parsed = statisticsQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или фильтры", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(await getStatistics(parsed.data))
|
||||
})
|
||||
|
||||
app.get("/statistics/pivot", async (req, reply) => {
|
||||
const parsed = statisticsPivotQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или измерения", details: parsed.error.flatten() })
|
||||
}
|
||||
if (pivotDimsConflict(parsed.data.row, parsed.data.col)) {
|
||||
return reply.status(400).send({ error: "Строки и колонки должны отличаться" })
|
||||
}
|
||||
return reply.send(await getStatisticsPivot(parsed.data))
|
||||
})
|
||||
}
|
||||
|
||||
export default statisticsRoutes
|
||||
@@ -27,6 +27,7 @@ import {
|
||||
import { buildFlowMapHops } from "../services/traffic-flow-map-hops.js"
|
||||
import { applyFlowOverlay } from "../services/traffic-flow-overlay.js"
|
||||
import { listTrafficFlowHostFiles } from "../services/traffic-flow-host-files.js"
|
||||
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
|
||||
const LIVE_TICK_MS = 2000
|
||||
@@ -203,6 +204,27 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/rebuild-facts", async (_req, reply) => {
|
||||
try {
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "traffic.flow.rebuild_facts",
|
||||
sourceModule: "traffic",
|
||||
title: "Пересчитан куб NetFlow",
|
||||
message: `Факты ${result.facts} из ${result.buckets} сессий, дней ${result.days.length}`,
|
||||
entityType: "traffic_flow",
|
||||
entityId: "rebuild-facts",
|
||||
payload: { buckets: result.buckets, facts: result.facts, days: result.days },
|
||||
})
|
||||
return reply.send(result)
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
const status = message.includes("уже выполняется") ? 409 : 500
|
||||
return reply.status(status).send({ error: message })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/overlay", applyOverlayHandler)
|
||||
app.post("/traffic/flow-overlay", applyOverlayHandler)
|
||||
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { listVxlanTunnels, listVxlanTunnelsForServer } from "../services/vxlan-live.js"
|
||||
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
|
||||
|
||||
const vxlanRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/vxlan", async (_req, reply) => {
|
||||
const tunnels = await listVxlanTunnels()
|
||||
return reply.send({ tunnels })
|
||||
})
|
||||
|
||||
app.get("/servers/:id/vxlan", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
|
||||
const params = req.params as ServerIdParams
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, params.id)).limit(1))[0]
|
||||
if (!server) return reply.status(404).send({ error: "Server not found" })
|
||||
const tunnels = await listVxlanTunnelsForServer(server)
|
||||
return reply.send({ tunnels })
|
||||
})
|
||||
}
|
||||
|
||||
export default vxlanRoutes
|
||||
@@ -18,6 +18,8 @@ import {
|
||||
type WgParsedConfig,
|
||||
} from "../services/wireguard-config.js"
|
||||
import {
|
||||
captureWireguardSnapshot,
|
||||
fetchWireguardRestoreState,
|
||||
getEnabledServerById,
|
||||
listWireGuardInterfaces,
|
||||
} from "../services/wireguard-live.js"
|
||||
@@ -27,6 +29,16 @@ import {
|
||||
putWireguardPeer,
|
||||
toRosBody,
|
||||
} from "../services/wireguard-ros.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
listRevisions,
|
||||
loadRevisionForRestore,
|
||||
type ConfigRevisionSource,
|
||||
} from "../services/config-revisions.js"
|
||||
import { parseWireguardSnapshot, planWireguardRestore } from "../services/entity-snapshots.js"
|
||||
import { executeRosOps } from "../services/ros-ops.js"
|
||||
import { parseDbServerId } from "../utils/server-id.js"
|
||||
import { z } from "zod"
|
||||
|
||||
function serverIdParam(v: string): string {
|
||||
return decodeURIComponent(v)
|
||||
@@ -137,6 +149,20 @@ function findIface(
|
||||
return list.find((i) => i.serverId === serverId && i.name === interfaceName)
|
||||
}
|
||||
|
||||
async function recordWireguard(
|
||||
server: NonNullable<Awaited<ReturnType<typeof getEnabledServerById>>>,
|
||||
source: ConfigRevisionSource,
|
||||
) {
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "wireguard",
|
||||
source,
|
||||
capture: () => captureWireguardSnapshot(server),
|
||||
})
|
||||
}
|
||||
|
||||
const RevisionIdParamSchema = z.object({ id: z.string().min(1) })
|
||||
|
||||
const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/wireguard", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string; includePrivateKey?: string }
|
||||
@@ -145,6 +171,11 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
serverId: q.serverId,
|
||||
includePrivateKey,
|
||||
})
|
||||
const sid = parseDbServerId(q.serverId)
|
||||
if (sid !== null) {
|
||||
const server = await getEnabledServerById(sid)
|
||||
if (server) await recordWireguard(server, "observed")
|
||||
}
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
@@ -181,6 +212,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
includePrivateKey: true,
|
||||
})
|
||||
const created = list.interfaces.find((i) => i.name === body.name)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.status(201).send(created ?? { ok: true, name: body.name })
|
||||
} catch (e) {
|
||||
const msg = e instanceof MikrotikError ? e.message : e instanceof Error ? e.message : String(e)
|
||||
@@ -210,6 +242,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
disabled: d.disabled === true ? "yes" : d.disabled === false ? "no" : undefined,
|
||||
}),
|
||||
)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -224,6 +257,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await client.delete(`/interface/wireguard/${encodeURIComponent(rosIdParam(rosId))}`)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -242,6 +276,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await putWireguardPeer(client, peerToRosBody(body))
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.status(201).send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -277,6 +312,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
disabled: d.disabled === true ? "yes" : d.disabled === false ? "no" : undefined,
|
||||
}),
|
||||
)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -291,6 +327,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
await client.delete(`/interface/wireguard/peers/${encodeURIComponent(rosIdParam(rosId))}`)
|
||||
await recordWireguard(server, "apply")
|
||||
return reply.send({ ok: true })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -320,6 +357,7 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
const applied = await applyParsedConfig(client, config)
|
||||
await recordWireguard(server, "copy")
|
||||
return reply.send({ dryRun: false, preview, applied })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
@@ -437,6 +475,46 @@ const wireguardRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
content,
|
||||
})
|
||||
})
|
||||
|
||||
app.get("/wireguard/revisions", async (req, reply) => {
|
||||
const q = req.query as { serverId?: string | number }
|
||||
const serverId = parseDbServerId(q.serverId)
|
||||
if (serverId === null) return reply.status(400).send({ error: "serverId is required" })
|
||||
const revisions = await listRevisions(serverId, "wireguard")
|
||||
return reply.send({ revisions })
|
||||
})
|
||||
|
||||
app.post("/wireguard/revisions/:id/restore", {
|
||||
schema: { params: RevisionIdParamSchema },
|
||||
}, async (req, reply) => {
|
||||
const { id } = req.params
|
||||
const body = req.body as { serverId?: string | number } | undefined
|
||||
const loaded = await loadRevisionForRestore({
|
||||
id,
|
||||
section: "wireguard",
|
||||
requestedServerId: parseDbServerId(body?.serverId),
|
||||
})
|
||||
if (!loaded.ok) return reply.status(loaded.status).send({ error: loaded.error })
|
||||
try {
|
||||
const desired = parseWireguardSnapshot(loaded.row.payload)
|
||||
const state = await fetchWireguardRestoreState(loaded.server)
|
||||
const ops = planWireguardRestore(desired, {
|
||||
ifaces: state.ifaces,
|
||||
peers: state.peers,
|
||||
addrs: state.addrs,
|
||||
})
|
||||
await executeRosOps(state.client, ops)
|
||||
await recordWireguard(loaded.server, "rollback")
|
||||
const list = await listWireGuardInterfaces({
|
||||
serverId: String(loaded.server.id),
|
||||
includePrivateKey: true,
|
||||
})
|
||||
return reply.send({ ok: true, interfaces: list.interfaces })
|
||||
} catch (e) {
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
return reply.status(502).send({ error: `RouterOS: ${msg}` })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default wireguardRoutes
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
import { initDatabase } from "../db/bootstrap.js"
|
||||
import { closePool } from "../db/index.js"
|
||||
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
|
||||
|
||||
await initDatabase()
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
console.log(JSON.stringify(result, null, 2))
|
||||
await closePool()
|
||||
@@ -1,14 +1,30 @@
|
||||
import { randomUUID } from "node:crypto"
|
||||
import { mkdir, rm, stat, writeFile } from "node:fs/promises"
|
||||
import { mkdir, rm, stat, writeFile, readFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { desc, eq } from "drizzle-orm"
|
||||
import type { BackupScheduleSettingsDto } from "@mmapp/contracts/backups"
|
||||
import type {
|
||||
BackupScheduleSettingsDto,
|
||||
BackupStorageSettingsDto,
|
||||
PutBackupStorageSettings,
|
||||
} from "@mmapp/contracts/backups"
|
||||
import { db } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { backupEntries, backupScheduleSettings } from "../db/schema.js"
|
||||
import { backupEntries, backupScheduleSettings, backupStorageSettings } from "../db/schema.js"
|
||||
import { getServerRowById } from "../modules/servers/repository/servers-repository.js"
|
||||
import { listServersRead } from "../modules/servers/service/servers-service.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import {
|
||||
buildS3ObjectKey,
|
||||
createS3ClientFromConfig,
|
||||
parseS3ObjectKey,
|
||||
sanitizeServerName,
|
||||
s3DeleteObject,
|
||||
s3GetObject,
|
||||
s3ListObjects,
|
||||
s3PutObject,
|
||||
s3TestConnection,
|
||||
type S3BackupConfig,
|
||||
} from "./s3-backup-client.js"
|
||||
|
||||
const SETTINGS_ID = 1
|
||||
const BACKUPS_DIR = path.resolve(process.cwd(), "storage", "backups")
|
||||
@@ -22,9 +38,14 @@ export type BackupMeta = {
|
||||
createdAt: string
|
||||
kind: "manual" | "auto"
|
||||
notes?: string
|
||||
storage: "local" | "s3" | "both"
|
||||
s3Key?: string | null
|
||||
uploadError?: string | null
|
||||
}
|
||||
|
||||
function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
|
||||
type BackupRow = typeof backupEntries.$inferSelect
|
||||
|
||||
function rowToMeta(row: BackupRow): BackupMeta {
|
||||
return {
|
||||
id: row.id,
|
||||
serverId: row.serverId,
|
||||
@@ -34,6 +55,9 @@ function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
|
||||
createdAt: row.createdAt,
|
||||
kind: row.kind,
|
||||
notes: row.notes ?? undefined,
|
||||
storage: row.storage ?? "local",
|
||||
s3Key: row.s3Key,
|
||||
uploadError: row.uploadError,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,16 +84,36 @@ export async function insertBackup(meta: BackupMeta): Promise<void> {
|
||||
sizeBytes: meta.sizeBytes,
|
||||
kind: meta.kind,
|
||||
notes: meta.notes ?? null,
|
||||
storage: meta.storage,
|
||||
s3Key: meta.s3Key ?? null,
|
||||
s3Etag: null,
|
||||
uploadError: meta.uploadError ?? null,
|
||||
createdAt: meta.createdAt,
|
||||
})
|
||||
}
|
||||
|
||||
async function getBackupRow(id: string): Promise<BackupRow | undefined> {
|
||||
return (await db.select().from(backupEntries).where(eq(backupEntries.id, id)).limit(1))[0]
|
||||
}
|
||||
|
||||
export async function deleteBackupRecord(id: string): Promise<BackupMeta | null> {
|
||||
const hit = await getBackupById(id)
|
||||
if (!hit) return null
|
||||
const row = await getBackupRow(id)
|
||||
if (!row) return null
|
||||
const meta = rowToMeta(row)
|
||||
if (row.s3Key) {
|
||||
try {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (cfg) {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
await s3DeleteObject(client, cfg.bucket, row.s3Key)
|
||||
}
|
||||
} catch {
|
||||
/* объект мог уже отсутствовать */
|
||||
}
|
||||
}
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
return hit
|
||||
await rm(path.join(BACKUPS_DIR, row.filename), { force: true })
|
||||
return meta
|
||||
}
|
||||
|
||||
function fmtTs(d = new Date()): string {
|
||||
@@ -167,6 +211,127 @@ export async function touchBackupScheduleRunMeta(patch: {
|
||||
}).where(eq(backupScheduleSettings.id, SETTINGS_ID))
|
||||
}
|
||||
|
||||
function defaultStorageRow() {
|
||||
return {
|
||||
id: SETTINGS_ID,
|
||||
provider: "local" as const,
|
||||
s3Endpoint: "",
|
||||
s3Region: "us-east-1",
|
||||
s3Bucket: "",
|
||||
s3Prefix: "mikrotik",
|
||||
s3AccessKeyId: "",
|
||||
s3SecretAccessKey: "",
|
||||
s3ForcePathStyle: true,
|
||||
keepLocalCopy: true,
|
||||
lastTestAt: null as string | null,
|
||||
lastTestError: null as string | null,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
async function getBackupStorageSettingsRow() {
|
||||
return (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
?? defaultStorageRow()
|
||||
}
|
||||
|
||||
function toStorageDto(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): BackupStorageSettingsDto {
|
||||
return {
|
||||
provider: row.provider,
|
||||
s3Endpoint: row.s3Endpoint,
|
||||
s3Region: row.s3Region,
|
||||
s3Bucket: row.s3Bucket,
|
||||
s3Prefix: row.s3Prefix,
|
||||
s3AccessKeyId: row.s3AccessKeyId,
|
||||
secretConfigured: Boolean(row.s3SecretAccessKey),
|
||||
s3ForcePathStyle: row.s3ForcePathStyle,
|
||||
keepLocalCopy: row.keepLocalCopy,
|
||||
lastTestAt: row.lastTestAt ?? null,
|
||||
lastTestError: row.lastTestError ?? null,
|
||||
updatedAt: row.updatedAt,
|
||||
}
|
||||
}
|
||||
|
||||
export async function getBackupStorageSettings(): Promise<BackupStorageSettingsDto> {
|
||||
return toStorageDto(await getBackupStorageSettingsRow())
|
||||
}
|
||||
|
||||
export async function updateBackupStorageSettings(
|
||||
patch: PutBackupStorageSettings,
|
||||
): Promise<BackupStorageSettingsDto> {
|
||||
const prev = await getBackupStorageSettingsRow()
|
||||
const now = new Date().toISOString()
|
||||
const secret = patch.s3SecretAccessKey
|
||||
const next = {
|
||||
provider: patch.provider ?? prev.provider,
|
||||
s3Endpoint: patch.s3Endpoint ?? prev.s3Endpoint,
|
||||
s3Region: patch.s3Region ?? prev.s3Region,
|
||||
s3Bucket: patch.s3Bucket ?? prev.s3Bucket,
|
||||
s3Prefix: patch.s3Prefix ?? prev.s3Prefix,
|
||||
s3AccessKeyId: patch.s3AccessKeyId ?? prev.s3AccessKeyId,
|
||||
s3SecretAccessKey: secret && secret.length > 0 ? secret : prev.s3SecretAccessKey,
|
||||
s3ForcePathStyle: patch.s3ForcePathStyle ?? prev.s3ForcePathStyle,
|
||||
keepLocalCopy: patch.keepLocalCopy ?? prev.keepLocalCopy,
|
||||
updatedAt: now,
|
||||
}
|
||||
if ((await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]) {
|
||||
await db.update(backupStorageSettings).set(next).where(eq(backupStorageSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return await getBackupStorageSettings()
|
||||
}
|
||||
|
||||
function rowToS3Config(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): S3BackupConfig | null {
|
||||
if (row.provider !== "s3") return null
|
||||
if (!row.s3Bucket.trim() || !row.s3AccessKeyId.trim() || !row.s3SecretAccessKey) return null
|
||||
return {
|
||||
endpoint: row.s3Endpoint,
|
||||
region: row.s3Region,
|
||||
bucket: row.s3Bucket.trim(),
|
||||
prefix: row.s3Prefix,
|
||||
accessKeyId: row.s3AccessKeyId,
|
||||
secretAccessKey: row.s3SecretAccessKey,
|
||||
forcePathStyle: row.s3ForcePathStyle,
|
||||
}
|
||||
}
|
||||
|
||||
async function getS3ConfigIfEnabled(): Promise<S3BackupConfig | null> {
|
||||
return rowToS3Config(await getBackupStorageSettingsRow())
|
||||
}
|
||||
|
||||
export async function testBackupStorageConnection(): Promise<BackupStorageSettingsDto> {
|
||||
const row = await getBackupStorageSettingsRow()
|
||||
const cfg = rowToS3Config(row)
|
||||
const now = new Date().toISOString()
|
||||
if (!cfg) {
|
||||
const error = row.provider === "s3"
|
||||
? "Заполните bucket, ключ доступа и секрет"
|
||||
: "S3 не выбран"
|
||||
await persistStorageTest(now, error)
|
||||
throw new Error(error)
|
||||
}
|
||||
try {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
await s3TestConnection(client, cfg.bucket)
|
||||
await persistStorageTest(now, null)
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
await persistStorageTest(now, message)
|
||||
throw new Error(message)
|
||||
}
|
||||
return await getBackupStorageSettings()
|
||||
}
|
||||
|
||||
async function persistStorageTest(at: string, error: string | null) {
|
||||
const exists = (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
const patch = { lastTestAt: at, lastTestError: error, updatedAt: at }
|
||||
if (exists) {
|
||||
await db.update(backupStorageSettings).set(patch).where(eq(backupStorageSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...patch })
|
||||
}
|
||||
}
|
||||
|
||||
export async function resolveBackupServerIds(settings: BackupScheduleSettingsDto): Promise<string[]> {
|
||||
const enabled = new Set((await listServersRead()).map((s) => String(s.id)))
|
||||
const requested = settings.serverIds.length > 0 ? settings.serverIds : [...enabled]
|
||||
@@ -214,6 +379,23 @@ export function isBackupDue(now: Date, settings: BackupScheduleSettingsDto, last
|
||||
return true
|
||||
}
|
||||
|
||||
async function uploadBackupToS3(params: {
|
||||
serverName: string
|
||||
filename: string
|
||||
body: string
|
||||
}): Promise<{ key: string; etag?: string } | { error: string }> {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (!cfg) return { error: "S3 не настроен" }
|
||||
try {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
const key = buildS3ObjectKey(cfg.prefix, sanitizeServerName(params.serverName), params.filename)
|
||||
const put = await s3PutObject(client, cfg.bucket, key, params.body)
|
||||
return { key, etag: put.etag }
|
||||
} catch (err) {
|
||||
return { error: err instanceof Error ? err.message : String(err) }
|
||||
}
|
||||
}
|
||||
|
||||
export async function runBackupForServer(
|
||||
id: string,
|
||||
kind: BackupMeta["kind"],
|
||||
@@ -230,12 +412,33 @@ export async function runBackupForServer(
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const script = await client.exportConfigScript()
|
||||
const ts = fmtTs()
|
||||
const safeServer = row.name.replace(/[^a-zA-Z0-9._-]+/g, "_")
|
||||
const safeServer = sanitizeServerName(row.name)
|
||||
const filename = `${safeServer}_${ts}.rsc`
|
||||
const filePath = path.join(BACKUPS_DIR, filename)
|
||||
await ensureBackupStorage()
|
||||
await writeFile(filePath, script, "utf8")
|
||||
const st = await stat(filePath)
|
||||
const storageRow = await getBackupStorageSettingsRow()
|
||||
let storage: BackupMeta["storage"] = "local"
|
||||
let s3Key: string | null = null
|
||||
let s3Etag: string | null = null
|
||||
let uploadError: string | null = null
|
||||
|
||||
if (storageRow.provider === "s3") {
|
||||
const uploaded = await uploadBackupToS3({ serverName: row.name, filename, body: script })
|
||||
if ("key" in uploaded) {
|
||||
s3Key = uploaded.key
|
||||
s3Etag = uploaded.etag ?? null
|
||||
storage = storageRow.keepLocalCopy ? "both" : "s3"
|
||||
if (!storageRow.keepLocalCopy) {
|
||||
await rm(filePath, { force: true })
|
||||
}
|
||||
} else {
|
||||
uploadError = uploaded.error
|
||||
storage = "local"
|
||||
}
|
||||
}
|
||||
|
||||
const meta: BackupMeta = {
|
||||
id: randomUUID(),
|
||||
serverId: row.id,
|
||||
@@ -245,8 +448,24 @@ export async function runBackupForServer(
|
||||
createdAt: new Date().toISOString(),
|
||||
kind,
|
||||
notes,
|
||||
storage,
|
||||
s3Key,
|
||||
uploadError,
|
||||
}
|
||||
await insertBackup(meta)
|
||||
await db.insert(backupEntries).values({
|
||||
id: meta.id,
|
||||
serverId: meta.serverId,
|
||||
serverName: meta.serverName,
|
||||
filename: meta.filename,
|
||||
sizeBytes: meta.sizeBytes,
|
||||
kind: meta.kind,
|
||||
notes: meta.notes ?? null,
|
||||
storage: meta.storage,
|
||||
s3Key: meta.s3Key ?? null,
|
||||
s3Etag,
|
||||
uploadError: meta.uploadError ?? null,
|
||||
createdAt: meta.createdAt,
|
||||
})
|
||||
return meta
|
||||
}
|
||||
|
||||
@@ -257,12 +476,82 @@ export async function pruneBackupsForServer(serverId: string, keepCount: number)
|
||||
if (rows.length <= keepCount) return 0
|
||||
const toDelete = rows.slice(keepCount)
|
||||
for (const hit of toDelete) {
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, hit.id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
await deleteBackupRecord(hit.id)
|
||||
}
|
||||
return toDelete.length
|
||||
}
|
||||
|
||||
export async function readBackupContent(meta: BackupMeta): Promise<Buffer> {
|
||||
if ((meta.storage === "s3" || meta.storage === "both") && meta.s3Key) {
|
||||
try {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (cfg) {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
return await s3GetObject(client, cfg.bucket, meta.s3Key)
|
||||
}
|
||||
} catch {
|
||||
/* fallback: локальная копия, если есть */
|
||||
}
|
||||
}
|
||||
return await readFile(path.join(BACKUPS_DIR, meta.filename))
|
||||
}
|
||||
|
||||
export async function restoreBackupToDevice(id: string): Promise<{ filename: string; serverName: string }> {
|
||||
const meta = await getBackupById(id)
|
||||
if (!meta) throw new Error("Бэкап не найден")
|
||||
if (!meta.serverId) throw new Error("Сервер бэкапа удалён — восстановить нельзя")
|
||||
const row = await getServerRowById(meta.serverId)
|
||||
if (!row) throw new Error("Сервер не найден")
|
||||
const content = await readBackupContent(meta)
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const uploaded = await client.uploadTextFile(meta.filename, content.toString("utf8"), 60_000)
|
||||
await client.importUploadedFile(uploaded)
|
||||
return { filename: meta.filename, serverName: row.name }
|
||||
}
|
||||
|
||||
export async function syncBackupsFromS3(): Promise<{ imported: number; skipped: number }> {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (!cfg) throw new Error("S3 не настроен")
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
const objects = await s3ListObjects(client, cfg.bucket, cfg.prefix)
|
||||
const existing = new Set(
|
||||
(await db.select({ filename: backupEntries.filename, s3Key: backupEntries.s3Key }).from(backupEntries))
|
||||
.flatMap((row) => [row.filename, row.s3Key].filter((v): v is string => Boolean(v))),
|
||||
)
|
||||
let imported = 0
|
||||
let skipped = 0
|
||||
for (const obj of objects) {
|
||||
if (!obj.key.toLowerCase().endsWith(".rsc")) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
const parsed = parseS3ObjectKey(obj.key)
|
||||
if (existing.has(obj.key) || existing.has(parsed.filename)) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
const createdAt = obj.lastModified ?? new Date().toISOString()
|
||||
await db.insert(backupEntries).values({
|
||||
id: randomUUID(),
|
||||
serverId: null,
|
||||
serverName: parsed.serverName,
|
||||
filename: parsed.filename,
|
||||
sizeBytes: obj.size,
|
||||
kind: "auto",
|
||||
notes: "Импорт из S3",
|
||||
storage: "s3",
|
||||
s3Key: obj.key,
|
||||
s3Etag: null,
|
||||
uploadError: null,
|
||||
createdAt,
|
||||
})
|
||||
existing.add(obj.key)
|
||||
existing.add(parsed.filename)
|
||||
imported += 1
|
||||
}
|
||||
return { imported, skipped }
|
||||
}
|
||||
|
||||
export function getBackupsDir(): string {
|
||||
return BACKUPS_DIR
|
||||
}
|
||||
|
||||
@@ -28,3 +28,16 @@ export async function fetchBgpSessionsForAlerts(): Promise<BgpSessionRead[]> {
|
||||
)
|
||||
return results.flat()
|
||||
}
|
||||
|
||||
export async function countBgpSessions(): Promise<number> {
|
||||
try {
|
||||
const result = await Promise.race([
|
||||
fetchBgpSessionsForAlerts(),
|
||||
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
|
||||
])
|
||||
if (!result) return 0
|
||||
return result.length
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
@@ -113,6 +113,15 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
continue
|
||||
}
|
||||
|
||||
const stillOn = await getCertificateRenewSettings()
|
||||
if (!stillOn.enabled) {
|
||||
item.action = "skipped"
|
||||
item.message = "Автообновление выключено"
|
||||
snapshot.skippedTargets += 1
|
||||
snapshot.targets?.push(item)
|
||||
continue
|
||||
}
|
||||
|
||||
const jobId = randomUUID()
|
||||
await createIssueJobRecord({
|
||||
id: jobId,
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { hasManagedCommentPrefix, isOwnedRecursiveComment, managedRecursiveComment, stripManagedRecursiveComment } from "../managed-markers.js"
|
||||
import {
|
||||
planBgpInApply,
|
||||
planRecursiveApply,
|
||||
unmanagedRouteIds,
|
||||
} from "./config-apply-plan.js"
|
||||
import {
|
||||
canonicalFilterRules,
|
||||
fingerprintPayload,
|
||||
} from "./config-revisions.js"
|
||||
import { mapRosManagedRoutes } from "./config-apply-plan.js"
|
||||
|
||||
{
|
||||
const fp1 = fingerprintPayload(canonicalFilterRules([
|
||||
{ community: "65001:100", action: "route", gateway: "10.0.0.1", gatewayTunnelId: "gre1", description: "a" },
|
||||
]))
|
||||
const fp2 = fingerprintPayload(canonicalFilterRules([
|
||||
{ community: "65001:100", action: "route", gateway: "10.0.0.1", gatewayTunnelId: "gre1", description: "a" },
|
||||
]))
|
||||
const fp3 = fingerprintPayload(canonicalFilterRules([
|
||||
{ community: "65001:100", action: "route", gateway: "10.0.0.2", gatewayTunnelId: "gre1", description: "a" },
|
||||
]))
|
||||
assert.equal(fp1, fp2)
|
||||
assert.notEqual(fp1, fp3)
|
||||
}
|
||||
|
||||
{
|
||||
const existing = [
|
||||
{ ".id": "*1", chain: "bgp-in", comment: "MikrotikManager: msk", rule: "if (true) { accept; }" },
|
||||
{ ".id": "*2", chain: "bgp-in", comment: "legacy", rule: "if (bgp-communities includes 1:1) { reject; }" },
|
||||
{ ".id": "*3", chain: "bgp-out", comment: "MikrotikManager: other", rule: "if (bgp-communities includes 1:1) { accept; }" },
|
||||
]
|
||||
const patch = planBgpInApply(existing, 3)
|
||||
assert.equal(patch.action, "patch")
|
||||
assert.equal(patch.managedId, "*1")
|
||||
assert.deepEqual(patch.conflictIds, ["*2"])
|
||||
|
||||
const create = planBgpInApply(existing.filter((r) => r[".id"] !== "*1"), 1)
|
||||
assert.equal(create.action, "create")
|
||||
assert.equal(create.managedId, undefined)
|
||||
|
||||
const del = planBgpInApply(existing, 0)
|
||||
assert.equal(del.action, "delete")
|
||||
assert.equal(del.managedId, "*1")
|
||||
|
||||
const noop = planBgpInApply([], 0)
|
||||
assert.equal(noop.action, "noop")
|
||||
}
|
||||
|
||||
{
|
||||
const routes = [
|
||||
{ ".id": "*10", comment: "MikrotikManager:recursive via de", static: "true", "dst-address": "8.8.8.8/32", gateway: "1.1.1.1" },
|
||||
{ ".id": "*11", comment: "user static", static: "true", "dst-address": "1.1.1.1/32", gateway: "9.9.9.9" },
|
||||
{ ".id": "*12", comment: "recursive: old", static: "true", "dst-address": "9.9.9.9/32", gateway: "1.1.1.1" },
|
||||
]
|
||||
const plan = planRecursiveApply(routes)
|
||||
assert.deepEqual(plan.deleteIds, ["*10", "*12"])
|
||||
assert.deepEqual(unmanagedRouteIds(routes), ["*11"])
|
||||
}
|
||||
|
||||
{
|
||||
assert.equal(stripManagedRecursiveComment("MikrotikManager:recursive via de"), "via de")
|
||||
assert.equal(stripManagedRecursiveComment("recursive: old"), "old")
|
||||
assert.equal(managedRecursiveComment("MikrotikManager:recursive via de"), "MikrotikManager:recursive via de")
|
||||
assert.equal(isOwnedRecursiveComment("MikrotikManager:recursive via de"), true)
|
||||
assert.equal(isOwnedRecursiveComment("recursive: x"), true)
|
||||
assert.equal(isOwnedRecursiveComment("user static"), false)
|
||||
assert.equal(hasManagedCommentPrefix("MikrotikManager: msk"), true)
|
||||
}
|
||||
|
||||
{
|
||||
const mapped = mapRosManagedRoutes([
|
||||
{
|
||||
".id": "*1",
|
||||
static: "true",
|
||||
"dst-address": "10.9.9.2/32",
|
||||
gateway: "1.2.3.4",
|
||||
comment: "MikrotikManager:recursive hop-de",
|
||||
distance: "1",
|
||||
},
|
||||
{
|
||||
".id": "*2",
|
||||
static: "true",
|
||||
"dst-address": "10.9.9.3/32",
|
||||
gateway: "1.2.3.4",
|
||||
comment: "not ours",
|
||||
distance: "1",
|
||||
},
|
||||
])
|
||||
assert.equal(mapped.length, 1)
|
||||
assert.equal(mapped[0]?.dstAddress, "10.9.9.2/32")
|
||||
assert.equal(mapped[0]?.comment, "hop-de")
|
||||
}
|
||||
|
||||
console.log("config-apply-plan.test.ts: ok")
|
||||
@@ -0,0 +1,140 @@
|
||||
import {
|
||||
hasManagedCommentPrefix,
|
||||
isOwnedRecursiveComment,
|
||||
stripManagedRecursiveComment,
|
||||
} from "../managed-markers.js"
|
||||
|
||||
export type RosFilterRuleLike = {
|
||||
".id"?: string
|
||||
chain?: string
|
||||
rule?: string
|
||||
comment?: string
|
||||
}
|
||||
|
||||
export type BgpInApplyAction = "patch" | "create" | "delete" | "noop"
|
||||
|
||||
export interface BgpInApplyPlan {
|
||||
action: BgpInApplyAction
|
||||
managedId?: string
|
||||
conflictIds: string[]
|
||||
}
|
||||
|
||||
function isInBgpIn(rule: RosFilterRuleLike): boolean {
|
||||
return (rule.chain ?? "").trim().toLowerCase() === "bgp-in"
|
||||
}
|
||||
|
||||
export function planBgpInApply(
|
||||
existing: RosFilterRuleLike[],
|
||||
rulesCount: number,
|
||||
): BgpInApplyPlan {
|
||||
const managed = existing.find(
|
||||
(r) => isInBgpIn(r) && hasManagedCommentPrefix(r.comment ?? ""),
|
||||
)
|
||||
const conflictIds = existing
|
||||
.filter((r) =>
|
||||
isInBgpIn(r) &&
|
||||
!hasManagedCommentPrefix(r.comment ?? "") &&
|
||||
/bgp-communities/i.test(r.rule ?? ""),
|
||||
)
|
||||
.map((r) => r[".id"])
|
||||
.filter((id): id is string => Boolean(id))
|
||||
|
||||
if (rulesCount > 0) {
|
||||
return {
|
||||
action: managed?.[".id"] ? "patch" : "create",
|
||||
managedId: managed?.[".id"],
|
||||
conflictIds,
|
||||
}
|
||||
}
|
||||
if (managed?.[".id"]) {
|
||||
return { action: "delete", managedId: managed[".id"], conflictIds }
|
||||
}
|
||||
return { action: "noop", conflictIds }
|
||||
}
|
||||
|
||||
export type RosRouteLike = {
|
||||
".id"?: string
|
||||
comment?: string
|
||||
static?: string
|
||||
dynamic?: string
|
||||
blackhole?: string
|
||||
unreachable?: string
|
||||
prohibit?: string
|
||||
"dst-address"?: string
|
||||
gateway?: string
|
||||
}
|
||||
|
||||
export function planRecursiveApply(existing: RosRouteLike[]): { deleteIds: string[] } {
|
||||
return {
|
||||
deleteIds: existing
|
||||
.filter((r) => isOwnedRecursiveComment(r.comment))
|
||||
.map((r) => r[".id"])
|
||||
.filter((id): id is string => Boolean(id)),
|
||||
}
|
||||
}
|
||||
|
||||
export function unmanagedRouteIds(existing: RosRouteLike[]): string[] {
|
||||
return existing
|
||||
.filter((r) => Boolean(r[".id"]) && !isOwnedRecursiveComment(r.comment))
|
||||
.map((r) => r[".id"] as string)
|
||||
}
|
||||
|
||||
export function userRecursiveComment(comment: string | undefined): string {
|
||||
return stripManagedRecursiveComment(comment ?? "")
|
||||
}
|
||||
|
||||
function isIpGateway(gw: string): boolean {
|
||||
return /^\d{1,3}(\.\d{1,3}){3}(?:%\S+)?$/.test(gw.trim())
|
||||
}
|
||||
|
||||
export function isManagedRecursiveRoute(r: RosRouteLike): boolean {
|
||||
if ((r.static ?? "false") !== "true") return false
|
||||
if ((r.dynamic ?? "false") === "true") return false
|
||||
if ((r.blackhole ?? "false") === "true") return false
|
||||
if ((r.unreachable ?? "false") === "true") return false
|
||||
if ((r.prohibit ?? "false") === "true") return false
|
||||
const dst = r["dst-address"] ?? ""
|
||||
const gw = r.gateway ?? ""
|
||||
if (!dst || !gw) return false
|
||||
if (!isIpGateway(gw)) return false
|
||||
return isOwnedRecursiveComment(r.comment)
|
||||
}
|
||||
|
||||
export interface MappedRecursiveRoute {
|
||||
id: string
|
||||
dstAddress: string
|
||||
gateway: string
|
||||
distance: number
|
||||
scope: number | null
|
||||
targetScope: number | null
|
||||
routingTable: string
|
||||
checkGateway: string
|
||||
country: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
}
|
||||
|
||||
export function mapRosManagedRoutes(
|
||||
rosRoutes: Array<RosRouteLike & {
|
||||
distance?: string
|
||||
scope?: string
|
||||
"target-scope"?: string
|
||||
"routing-table"?: string
|
||||
"check-gateway"?: string
|
||||
disabled?: string
|
||||
}>,
|
||||
): MappedRecursiveRoute[] {
|
||||
return rosRoutes.filter(isManagedRecursiveRoute).map((r, i) => ({
|
||||
id: r[".id"] ?? `ros-${i}`,
|
||||
dstAddress: r["dst-address"] ?? "",
|
||||
gateway: r.gateway ?? "",
|
||||
distance: Number.parseInt(r.distance ?? "1", 10) || 1,
|
||||
scope: r.scope ? (Number.parseInt(r.scope, 10) || null) : null,
|
||||
targetScope: r["target-scope"] ? (Number.parseInt(r["target-scope"], 10) || null) : null,
|
||||
routingTable: r["routing-table"] ?? "main",
|
||||
checkGateway: r["check-gateway"] ?? "",
|
||||
country: "",
|
||||
comment: userRecursiveComment(r.comment),
|
||||
disabled: r.disabled === "true",
|
||||
}))
|
||||
}
|
||||
@@ -0,0 +1,101 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import {
|
||||
appendRevisionIfChanged,
|
||||
fingerprintPayload,
|
||||
getRevisionById,
|
||||
listRevisions,
|
||||
pruneRevisions,
|
||||
} from "./config-revisions.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("config-revisions.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const tag = `rev-test-${Date.now()}`
|
||||
await dbQuery(`INSERT INTO servers (name, host) VALUES ($1, '127.0.0.1')`, [tag])
|
||||
const { rows } = await dbQuery<{ id: number }>(`SELECT id FROM servers WHERE name = $1 LIMIT 1`, [tag])
|
||||
const serverId = rows[0]?.id
|
||||
assert.ok(serverId)
|
||||
|
||||
try {
|
||||
const payloadA = [{ community: "1:1", action: "route" }]
|
||||
const first = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "apply",
|
||||
payload: payloadA,
|
||||
})
|
||||
assert.equal(first.created, true)
|
||||
assert.equal(first.revision.source, "apply")
|
||||
|
||||
const dup = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "observed",
|
||||
payload: payloadA,
|
||||
})
|
||||
assert.equal(dup.created, false)
|
||||
assert.equal(dup.revision.id, first.revision.id)
|
||||
|
||||
const payloadB = [{ community: "1:2", action: "blackhole" }]
|
||||
const second = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "rollback",
|
||||
payload: payloadB,
|
||||
})
|
||||
assert.equal(second.created, true)
|
||||
assert.equal(second.revision.source, "rollback")
|
||||
assert.notEqual(second.revision.fingerprint, first.revision.fingerprint)
|
||||
|
||||
const listed = await listRevisions(serverId, "filters")
|
||||
assert.equal(listed.length, 2)
|
||||
assert.equal(listed[0]?.source, "rollback")
|
||||
|
||||
const stored = await getRevisionById(second.revision.id)
|
||||
assert.ok(stored)
|
||||
assert.equal(fingerprintPayload(stored.payload), second.revision.fingerprint)
|
||||
|
||||
const objPayload = { rules: [{ chain: "input" }], addressLists: [{ list: "vip" }] }
|
||||
const objRev = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "firewall",
|
||||
source: "apply",
|
||||
payload: objPayload,
|
||||
})
|
||||
assert.equal(objRev.created, true)
|
||||
assert.equal(objRev.revision.itemCount, 2)
|
||||
const objStored = await getRevisionById(objRev.revision.id)
|
||||
assert.ok(objStored)
|
||||
assert.ok(!Array.isArray(objStored.payload))
|
||||
assert.equal(fingerprintPayload(objStored.payload), objRev.revision.fingerprint)
|
||||
|
||||
const objDup = await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "firewall",
|
||||
source: "rollback",
|
||||
payload: objPayload,
|
||||
})
|
||||
assert.equal(objDup.created, false)
|
||||
assert.equal(objDup.revision.source, "apply")
|
||||
|
||||
for (let i = 0; i < 4; i++) {
|
||||
await appendRevisionIfChanged({
|
||||
serverId,
|
||||
section: "filters",
|
||||
source: "apply",
|
||||
payload: [{ community: `9:${i}`, action: "route" }],
|
||||
})
|
||||
}
|
||||
const pruned = await pruneRevisions(serverId, "filters", 3)
|
||||
assert.ok(pruned >= 1)
|
||||
const after = await listRevisions(serverId, "filters")
|
||||
assert.equal(after.length, 3)
|
||||
} finally {
|
||||
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
|
||||
}
|
||||
|
||||
console.log("config-revisions.test.ts: ok")
|
||||
@@ -0,0 +1,236 @@
|
||||
import { createHash, randomUUID } from "node:crypto"
|
||||
import { and, desc, eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { configRevisions, servers, type ConfigRevisionRow } from "../db/schema.js"
|
||||
|
||||
export const CONFIG_REVISION_KEEP = 50
|
||||
|
||||
export const CONFIG_SECTIONS = [
|
||||
"filters",
|
||||
"recursive-routes",
|
||||
"firewall",
|
||||
"wireguard",
|
||||
"gre",
|
||||
] as const
|
||||
|
||||
export type ConfigSection = (typeof CONFIG_SECTIONS)[number]
|
||||
export type ConfigRevisionSource = "apply" | "rollback" | "observed" | "copy"
|
||||
|
||||
export interface ConfigRevisionDto {
|
||||
id: string
|
||||
serverId: string
|
||||
section: ConfigSection
|
||||
source: ConfigRevisionSource
|
||||
fingerprint: string
|
||||
createdAt: string
|
||||
note: string | null
|
||||
itemCount: number
|
||||
}
|
||||
|
||||
export function stableStringify(value: unknown): string {
|
||||
if (value === null || typeof value !== "object") return JSON.stringify(value)
|
||||
if (Array.isArray(value)) return `[${value.map(stableStringify).join(",")}]`
|
||||
const obj = value as Record<string, unknown>
|
||||
const keys = Object.keys(obj).sort()
|
||||
return `{${keys.map((k) => `${JSON.stringify(k)}:${stableStringify(obj[k])}`).join(",")}}`
|
||||
}
|
||||
|
||||
export function fingerprintPayload(payload: unknown): string {
|
||||
return createHash("sha256").update(stableStringify(payload)).digest("hex")
|
||||
}
|
||||
|
||||
export function revisionItemCount(payload: unknown): number {
|
||||
if (Array.isArray(payload)) return payload.length
|
||||
if (payload && typeof payload === "object") {
|
||||
let n = 0
|
||||
for (const value of Object.values(payload as Record<string, unknown>)) {
|
||||
if (Array.isArray(value)) n += value.length
|
||||
}
|
||||
return n
|
||||
}
|
||||
return 0
|
||||
}
|
||||
|
||||
export function persistablePayload(payload: unknown): unknown {
|
||||
if (payload === undefined) return []
|
||||
return payload
|
||||
}
|
||||
|
||||
export function canonicalFilterRules(
|
||||
rules: Array<{
|
||||
community?: string
|
||||
action?: string
|
||||
gateway?: string
|
||||
gatewayTunnelId?: string
|
||||
description?: string
|
||||
}>,
|
||||
): unknown[] {
|
||||
return rules.map((r) => ({
|
||||
community: (r.community ?? "").trim(),
|
||||
action: r.action === "blackhole" ? "blackhole" : "route",
|
||||
gateway: r.gateway ?? "",
|
||||
gatewayTunnelId: r.gatewayTunnelId ?? "",
|
||||
description: r.description ?? "",
|
||||
}))
|
||||
}
|
||||
|
||||
export function canonicalRecursiveRoutes(
|
||||
routes: Array<{
|
||||
dstAddress?: string
|
||||
gateway?: string
|
||||
distance?: number
|
||||
scope?: number | null
|
||||
targetScope?: number | null
|
||||
routingTable?: string
|
||||
checkGateway?: string
|
||||
comment?: string
|
||||
disabled?: boolean
|
||||
country?: string
|
||||
}>,
|
||||
): unknown[] {
|
||||
return routes.map((r) => ({
|
||||
dstAddress: (r.dstAddress ?? "").trim(),
|
||||
gateway: r.gateway ?? "",
|
||||
distance: r.distance ?? 1,
|
||||
scope: r.scope ?? null,
|
||||
targetScope: r.targetScope ?? null,
|
||||
routingTable: r.routingTable || "main",
|
||||
checkGateway: r.checkGateway ?? "",
|
||||
comment: r.comment ?? "",
|
||||
disabled: Boolean(r.disabled),
|
||||
country: r.country ?? "",
|
||||
}))
|
||||
}
|
||||
|
||||
export function toRevisionDto(row: ConfigRevisionRow): ConfigRevisionDto {
|
||||
return {
|
||||
id: row.id,
|
||||
serverId: String(row.serverId),
|
||||
section: row.section as ConfigSection,
|
||||
source: row.source,
|
||||
fingerprint: row.fingerprint,
|
||||
createdAt: row.createdAt,
|
||||
note: row.note ?? null,
|
||||
itemCount: revisionItemCount(row.payload),
|
||||
}
|
||||
}
|
||||
|
||||
export async function listRevisions(
|
||||
serverId: number,
|
||||
section: ConfigSection,
|
||||
limit = CONFIG_REVISION_KEEP,
|
||||
): Promise<ConfigRevisionDto[]> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(configRevisions)
|
||||
.where(and(eq(configRevisions.serverId, serverId), eq(configRevisions.section, section)))
|
||||
.orderBy(desc(configRevisions.createdAt))
|
||||
.limit(limit)
|
||||
return rows.map(toRevisionDto)
|
||||
}
|
||||
|
||||
export async function getRevisionById(id: string): Promise<ConfigRevisionRow | undefined> {
|
||||
return (await db.select().from(configRevisions).where(eq(configRevisions.id, id)).limit(1))[0]
|
||||
}
|
||||
|
||||
export async function pruneRevisions(
|
||||
serverId: number,
|
||||
section: ConfigSection,
|
||||
keep = CONFIG_REVISION_KEEP,
|
||||
): Promise<number> {
|
||||
const rows = await db
|
||||
.select({ id: configRevisions.id })
|
||||
.from(configRevisions)
|
||||
.where(and(eq(configRevisions.serverId, serverId), eq(configRevisions.section, section)))
|
||||
.orderBy(desc(configRevisions.createdAt))
|
||||
const extra = rows.slice(keep)
|
||||
if (extra.length === 0) return 0
|
||||
for (const row of extra) {
|
||||
await db.delete(configRevisions).where(eq(configRevisions.id, row.id))
|
||||
}
|
||||
return extra.length
|
||||
}
|
||||
|
||||
export async function appendRevisionIfChanged(input: {
|
||||
serverId: number
|
||||
section: ConfigSection
|
||||
source: ConfigRevisionSource
|
||||
payload: unknown
|
||||
note?: string | null
|
||||
}): Promise<{ created: boolean; revision: ConfigRevisionDto }> {
|
||||
const payload = persistablePayload(input.payload)
|
||||
const fingerprint = fingerprintPayload(payload)
|
||||
const latest = (await db
|
||||
.select()
|
||||
.from(configRevisions)
|
||||
.where(and(
|
||||
eq(configRevisions.serverId, input.serverId),
|
||||
eq(configRevisions.section, input.section),
|
||||
))
|
||||
.orderBy(desc(configRevisions.createdAt))
|
||||
.limit(1))[0]
|
||||
|
||||
if (latest?.fingerprint === fingerprint) {
|
||||
return { created: false, revision: toRevisionDto(latest) }
|
||||
}
|
||||
|
||||
const now = new Date().toISOString()
|
||||
const id = randomUUID()
|
||||
await db.insert(configRevisions).values({
|
||||
id,
|
||||
serverId: input.serverId,
|
||||
section: input.section,
|
||||
source: input.source,
|
||||
fingerprint,
|
||||
payload,
|
||||
note: input.note ?? null,
|
||||
createdAt: now,
|
||||
})
|
||||
await pruneRevisions(input.serverId, input.section)
|
||||
const row = await getRevisionById(id)
|
||||
if (!row) throw new Error("config-revisions: insert vanished")
|
||||
return { created: true, revision: toRevisionDto(row) }
|
||||
}
|
||||
|
||||
/** После успешного mutate: capture live → append, ошибки snapshot не валят мутацию. */
|
||||
export async function captureAndAppendRevision(input: {
|
||||
serverId: number
|
||||
section: ConfigSection
|
||||
source: ConfigRevisionSource
|
||||
capture: () => Promise<unknown>
|
||||
note?: string | null
|
||||
}): Promise<{ created: boolean; revision: ConfigRevisionDto } | null> {
|
||||
try {
|
||||
const payload = await input.capture()
|
||||
return await appendRevisionIfChanged({
|
||||
serverId: input.serverId,
|
||||
section: input.section,
|
||||
source: input.source,
|
||||
payload,
|
||||
note: input.note,
|
||||
})
|
||||
} catch {
|
||||
return null
|
||||
}
|
||||
}
|
||||
|
||||
export async function loadRevisionForRestore(opts: {
|
||||
id: string
|
||||
section: ConfigSection
|
||||
requestedServerId: number | null
|
||||
}): Promise<
|
||||
| { ok: true; row: ConfigRevisionRow; server: typeof servers.$inferSelect }
|
||||
| { ok: false; status: number; error: string }
|
||||
> {
|
||||
const row = await getRevisionById(opts.id)
|
||||
if (!row) return { ok: false, status: 404, error: "Revision not found" }
|
||||
if (row.section !== opts.section) {
|
||||
return { ok: false, status: 400, error: "Revision section mismatch" }
|
||||
}
|
||||
if (opts.requestedServerId !== null && opts.requestedServerId !== row.serverId) {
|
||||
return { ok: false, status: 400, error: "Revision belongs to another server" }
|
||||
}
|
||||
const server = (await db.select().from(servers).where(eq(servers.id, row.serverId)).limit(1))[0]
|
||||
if (!server) return { ok: false, status: 404, error: "Server not found" }
|
||||
return { ok: true, row, server }
|
||||
}
|
||||
@@ -0,0 +1,46 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { mapContainerRow } from "./containers-live.js"
|
||||
|
||||
const server = {
|
||||
id: 3,
|
||||
name: "mt-spb",
|
||||
host: "10.0.1.1",
|
||||
} as Parameters<typeof mapContainerRow>[0]
|
||||
|
||||
const row = mapContainerRow(
|
||||
server,
|
||||
{
|
||||
".id": "*A",
|
||||
name: "adguard",
|
||||
"remote-image": "adguard/adguardhome:latest",
|
||||
interface: "veth-adguard",
|
||||
envlist: "adguard-envs",
|
||||
mounts: "agh-conf,agh-work",
|
||||
status: "running",
|
||||
"start-on-boot": "true",
|
||||
comment: "DNS",
|
||||
},
|
||||
[
|
||||
{ name: "adguard-envs", key: "FOO", value: "bar" },
|
||||
{ name: "other", key: "SKIP", value: "x" },
|
||||
],
|
||||
[
|
||||
{ name: "agh-conf", dst: "/opt/conf", src: "/disk1/conf" },
|
||||
{ name: "agh-work", dst: "/opt/work" },
|
||||
],
|
||||
0,
|
||||
)
|
||||
|
||||
assert.equal(row.rosId, "*A")
|
||||
assert.equal(row.image, "adguard/adguardhome")
|
||||
assert.equal(row.tag, "latest")
|
||||
assert.equal(row.status, "running")
|
||||
assert.deepEqual(row.interfaces, ["veth-adguard"])
|
||||
assert.deepEqual(row.envs, [{ key: "FOO", value: "bar" }])
|
||||
assert.deepEqual(row.mounts, [
|
||||
{ dst: "/opt/conf", src: "/disk1/conf" },
|
||||
{ dst: "/opt/work", src: undefined },
|
||||
])
|
||||
assert.equal(row.startOnBoot, true)
|
||||
|
||||
console.log("containers-live.test.ts: ok")
|
||||
@@ -0,0 +1,215 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient, MikrotikError } from "./mikrotik.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
interface RosContainer {
|
||||
".id"?: string
|
||||
name?: string
|
||||
tag?: string
|
||||
"remote-image"?: string
|
||||
interface?: string
|
||||
envlist?: string
|
||||
mounts?: string
|
||||
cmd?: string
|
||||
"start-on-boot"?: string
|
||||
comment?: string
|
||||
status?: string
|
||||
"memory-high"?: string
|
||||
cpu?: string
|
||||
}
|
||||
|
||||
interface RosContainerEnv {
|
||||
name?: string
|
||||
key?: string
|
||||
value?: string
|
||||
}
|
||||
|
||||
interface RosContainerMount {
|
||||
name?: string
|
||||
src?: string
|
||||
dst?: string
|
||||
}
|
||||
|
||||
export type ContainerLiveStatus = "running" | "stopped" | "error"
|
||||
|
||||
export type ContainerLive = {
|
||||
id: string
|
||||
rosId: string
|
||||
name: string
|
||||
serverId: string
|
||||
image: string
|
||||
tag: string
|
||||
status: ContainerLiveStatus
|
||||
envs: { key: string; value: string }[]
|
||||
mounts: { dst: string; src?: string }[]
|
||||
interfaces: string[]
|
||||
cmd?: string
|
||||
startOnBoot: boolean
|
||||
comment: string
|
||||
uptime?: string
|
||||
cpu?: number
|
||||
memMb?: number
|
||||
}
|
||||
|
||||
function rosYes(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
function mapStatus(raw: string | undefined): ContainerLiveStatus {
|
||||
const s = (raw ?? "").toLowerCase()
|
||||
if (s === "running") return "running"
|
||||
if (s === "error" || s === "failed") return "error"
|
||||
return "stopped"
|
||||
}
|
||||
|
||||
function splitCsv(v: string | undefined): string[] {
|
||||
return (v ?? "")
|
||||
.split(",")
|
||||
.map((x) => x.trim())
|
||||
.filter(Boolean)
|
||||
}
|
||||
|
||||
function parseImageTag(c: RosContainer): { image: string; tag: string } {
|
||||
const remote = (c["remote-image"] ?? "").trim()
|
||||
if (remote) {
|
||||
const idx = remote.lastIndexOf(":")
|
||||
if (idx > 0 && !remote.slice(idx + 1).includes("/")) {
|
||||
return { image: remote.slice(0, idx), tag: remote.slice(idx + 1) }
|
||||
}
|
||||
return { image: remote, tag: (c.tag ?? "latest").trim() || "latest" }
|
||||
}
|
||||
return { image: (c.name ?? "").trim(), tag: (c.tag ?? "latest").trim() || "latest" }
|
||||
}
|
||||
|
||||
function isMissingPackage(err: unknown): boolean {
|
||||
if (err instanceof MikrotikError) {
|
||||
if (err.statusCode === 404) return true
|
||||
const body = err.body.toLowerCase()
|
||||
return body.includes("no such command") || body.includes("not found") || body.includes("unknown")
|
||||
}
|
||||
const msg = err instanceof Error ? err.message.toLowerCase() : String(err).toLowerCase()
|
||||
return msg.includes("no such command") || msg.includes("404")
|
||||
}
|
||||
|
||||
export function mapContainerRow(
|
||||
server: ServerRow,
|
||||
c: RosContainer,
|
||||
envs: RosContainerEnv[],
|
||||
mounts: RosContainerMount[],
|
||||
idx: number,
|
||||
): ContainerLive {
|
||||
const rosId = String(c[".id"] ?? `c-${idx}`)
|
||||
const name = (c.name ?? "").trim() || `container-${idx + 1}`
|
||||
const { image, tag } = parseImageTag(c)
|
||||
const envlist = (c.envlist ?? "").trim()
|
||||
const mountNames = new Set(splitCsv(c.mounts))
|
||||
const envRows = envlist
|
||||
? envs.filter((e) => (e.name ?? "").trim() === envlist && (e.key ?? "").trim())
|
||||
: []
|
||||
const mountRows = mounts.filter((m) => mountNames.has((m.name ?? "").trim()) && (m.dst ?? "").trim())
|
||||
const cpuRaw = Number.parseInt(c.cpu ?? "", 10)
|
||||
const memRaw = Number.parseInt(c["memory-high"] ?? "", 10)
|
||||
return {
|
||||
id: `${server.id}-${rosId}`,
|
||||
rosId,
|
||||
name,
|
||||
serverId: String(server.id),
|
||||
image,
|
||||
tag,
|
||||
status: mapStatus(c.status),
|
||||
envs: envRows.map((e) => ({ key: e.key ?? "", value: e.value ?? "" })),
|
||||
mounts: mountRows.map((m) => ({ dst: m.dst ?? "", src: m.src || undefined })),
|
||||
interfaces: splitCsv(c.interface),
|
||||
cmd: (c.cmd ?? "").trim() || undefined,
|
||||
startOnBoot: rosYes(c["start-on-boot"]),
|
||||
comment: c.comment ?? "",
|
||||
cpu: Number.isFinite(cpuRaw) ? cpuRaw : undefined,
|
||||
memMb: Number.isFinite(memRaw) ? Math.round(memRaw / (1024 * 1024)) || undefined : undefined,
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
try {
|
||||
const [raw, envsRaw, mountsRaw] = await Promise.all([
|
||||
client.get<RosContainer[]>("/container"),
|
||||
client.get<RosContainerEnv[]>("/container/envs").catch(() => [] as RosContainerEnv[]),
|
||||
client.get<RosContainerMount[]>("/container/mounts").catch(() => [] as RosContainerMount[]),
|
||||
])
|
||||
const list = Array.isArray(raw) ? raw : []
|
||||
const envs = Array.isArray(envsRaw) ? envsRaw : []
|
||||
const mounts = Array.isArray(mountsRaw) ? mountsRaw : []
|
||||
return list.map((c, idx) => mapContainerRow(server, c, envs, mounts, idx))
|
||||
} catch (err) {
|
||||
if (isMissingPackage(err)) return []
|
||||
throw err
|
||||
}
|
||||
}
|
||||
|
||||
export async function listContainers(): Promise<ContainerLive[]> {
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const results = await Promise.all(
|
||||
enabledServers.map(async (server) => {
|
||||
try {
|
||||
return await fetchContainersForServer(server)
|
||||
} catch {
|
||||
return [] as ContainerLive[]
|
||||
}
|
||||
}),
|
||||
)
|
||||
return results.flat()
|
||||
}
|
||||
|
||||
export async function listContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
|
||||
try {
|
||||
return await fetchContainersForServer(server)
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
export async function countContainers(): Promise<number> {
|
||||
try {
|
||||
const result = await Promise.race([
|
||||
listContainers(),
|
||||
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
|
||||
])
|
||||
if (!result) return 0
|
||||
return result.length
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
function encodeRosId(rosId: string): string {
|
||||
return encodeURIComponent(rosId)
|
||||
}
|
||||
|
||||
export async function startContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
await client.post("/container/start", { ".id": rosId })
|
||||
}
|
||||
|
||||
export async function stopContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
await client.post("/container/stop", { ".id": rosId })
|
||||
}
|
||||
|
||||
export async function restartContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
await stopContainer(server, rosId)
|
||||
await startContainer(server, rosId)
|
||||
}
|
||||
|
||||
export async function removeContainer(server: ServerRow, rosId: string): Promise<void> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
await client.delete(`/container/${encodeRosId(rosId)}`)
|
||||
}
|
||||
|
||||
export async function getEnabledServerById(serverId: string | number) {
|
||||
const id = typeof serverId === "number" ? serverId : Number.parseInt(String(serverId), 10)
|
||||
if (!Number.isFinite(id)) return null
|
||||
return (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0] ?? null
|
||||
}
|
||||
@@ -0,0 +1,164 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
canonicalFirewallSnapshot,
|
||||
canonicalGreSnapshot,
|
||||
canonicalWireguardSnapshot,
|
||||
opsPaths,
|
||||
opsTouchOnly,
|
||||
planFirewallRestore,
|
||||
planGreCreate,
|
||||
planGreDelete,
|
||||
planGreRestore,
|
||||
planWireguardRestore,
|
||||
} from "./entity-snapshots.js"
|
||||
import { fingerprintPayload, revisionItemCount } from "./config-revisions.js"
|
||||
|
||||
{
|
||||
const a = canonicalFirewallSnapshot({
|
||||
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "ssh" }],
|
||||
addressLists: [{ family: "ip", list: "vip", address: "1.1.1.1" }],
|
||||
})
|
||||
const b = canonicalFirewallSnapshot({
|
||||
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "ssh" }],
|
||||
addressLists: [{ family: "ip", list: "vip", address: "1.1.1.1" }],
|
||||
})
|
||||
assert.equal(fingerprintPayload(a), fingerprintPayload(b))
|
||||
assert.equal(revisionItemCount(a), 2)
|
||||
}
|
||||
|
||||
{
|
||||
const desired = canonicalFirewallSnapshot({
|
||||
rules: [{ family: "ip", table: "filter", chain: "input", action: "accept", comment: "keep" }],
|
||||
addressLists: [],
|
||||
})
|
||||
const ops = planFirewallRestore(desired, {
|
||||
rules: [
|
||||
{
|
||||
...desired.rules[0]!,
|
||||
rosId: "*1",
|
||||
dynamic: false,
|
||||
},
|
||||
{
|
||||
family: "ip",
|
||||
table: "filter",
|
||||
chain: "forward",
|
||||
action: "drop",
|
||||
protocol: "",
|
||||
srcAddress: "",
|
||||
dstAddress: "",
|
||||
srcAddressList: "",
|
||||
dstAddressList: "",
|
||||
srcPort: "",
|
||||
dstPort: "",
|
||||
inInterface: "",
|
||||
outInterface: "",
|
||||
connectionState: "",
|
||||
comment: "extra",
|
||||
disabled: false,
|
||||
log: false,
|
||||
logPrefix: "",
|
||||
tlsHost: "",
|
||||
layer7Proto: "",
|
||||
rosId: "*2",
|
||||
dynamic: false,
|
||||
},
|
||||
],
|
||||
addressLists: [],
|
||||
})
|
||||
assert.ok(ops.some((op) => op.op === "delete" && op.path.includes("/ip/firewall/filter/")))
|
||||
assert.equal(opsTouchOnly(ops, ["/ip/firewall", "/ipv6/firewall"]), true)
|
||||
assert.equal(opsPaths(ops).some((p) => p.startsWith("/ip/route") || p.startsWith("/interface/wireguard")), false)
|
||||
}
|
||||
|
||||
{
|
||||
const snap = canonicalWireguardSnapshot({
|
||||
interfaces: [{
|
||||
name: "wg0",
|
||||
privateKey: "abc",
|
||||
address: "10.8.0.1/24",
|
||||
peers: [{ publicKey: "pk", allowedAddresses: ["10.8.0.2/32"] }],
|
||||
}],
|
||||
})
|
||||
const ops = planWireguardRestore(snap, {
|
||||
ifaces: [{ name: "wg0", rosId: "*w", listenPort: 13231, mtu: 1420, privateKey: "abc", comment: "", disabled: false }],
|
||||
peers: [{
|
||||
rosId: "*p",
|
||||
interfaceName: "wg0",
|
||||
publicKey: "old",
|
||||
allowedAddresses: ["0.0.0.0/0"],
|
||||
endpointAddress: "",
|
||||
endpointPort: "",
|
||||
persistentKeepalive: null,
|
||||
comment: "",
|
||||
name: "",
|
||||
disabled: false,
|
||||
privateKey: "",
|
||||
clientAddress: "",
|
||||
clientDns: "",
|
||||
clientEndpoint: "",
|
||||
}],
|
||||
addrs: [{ rosId: "*a", interfaceName: "wg0", address: "10.8.0.1/24" }],
|
||||
})
|
||||
assert.ok(ops.some((op) => op.op === "delete" && op.path.includes("/interface/wireguard/peers/")))
|
||||
assert.ok(ops.some((op) => op.op === "put" && op.path === "/interface/wireguard/peers"))
|
||||
assert.equal(opsTouchOnly(ops, ["/interface/wireguard", "/ip/address"]), true)
|
||||
assert.equal(opsPaths(ops).some((p) => p.startsWith("/interface/gre") || p.startsWith("/ip/route")), false)
|
||||
}
|
||||
|
||||
{
|
||||
const tunnel = canonicalGreSnapshot({
|
||||
tunnels: [{
|
||||
name: "gre-a",
|
||||
remoteAddress: "203.0.113.1",
|
||||
localInnerIp: "10.200.0.1/30",
|
||||
ipsecSecret: "psk-secret",
|
||||
}],
|
||||
}).tunnels[0]!
|
||||
const create = planGreCreate(tunnel)
|
||||
assert.deepEqual(create.map((op) => op.op), ["put", "put"])
|
||||
assert.equal(create[0]?.path, "/interface/gre")
|
||||
assert.equal(create[1]?.path, "/ip/address")
|
||||
assert.equal(create[1] && create[1].op === "put" ? create[1].body.interface : "", "gre-a")
|
||||
assert.equal(opsPaths(create).some((p) => p.includes("gre-b")), false)
|
||||
|
||||
const del = planGreDelete("gre-a", {
|
||||
gre: [
|
||||
{ name: "gre-a", rosId: "*1", localAddress: "", remoteAddress: "203.0.113.1", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
|
||||
{ name: "gre-b", rosId: "*2", localAddress: "", remoteAddress: "203.0.113.2", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
|
||||
],
|
||||
addrs: [
|
||||
{ rosId: "*a1", interfaceName: "gre-a", address: "10.200.0.1/30" },
|
||||
{ rosId: "*a2", interfaceName: "gre-b", address: "10.200.0.5/30" },
|
||||
],
|
||||
})
|
||||
assert.ok(del.some((op) => op.path === "/ip/address/*a1"))
|
||||
assert.ok(del.some((op) => op.path === "/interface/gre/*1"))
|
||||
assert.equal(opsPaths(del).some((p) => p.includes("*2") || p.includes("*a2")), false)
|
||||
|
||||
const restore = planGreRestore(
|
||||
canonicalGreSnapshot({ tunnels: [tunnel] }),
|
||||
{
|
||||
gre: [
|
||||
{ name: "gre-a", rosId: "*1", localAddress: "", remoteAddress: "203.0.113.1", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "psk-secret" },
|
||||
{ name: "gre-b", rosId: "*2", localAddress: "", remoteAddress: "203.0.113.2", comment: "", disabled: false, mtu: 1476, keepalive: "0", dscp: "inherit", clampTcpMss: true, allowFastPath: true, ipsecSecret: "" },
|
||||
],
|
||||
addrs: [
|
||||
{ rosId: "*a1", interfaceName: "gre-a", address: "10.200.0.1/30" },
|
||||
{ rosId: "*a2", interfaceName: "gre-b", address: "10.200.0.5/30" },
|
||||
],
|
||||
},
|
||||
)
|
||||
assert.ok(restore.some((op) => op.path === "/interface/gre/*2"))
|
||||
assert.ok(restore.some((op) => op.path === "/ip/address/*a2"))
|
||||
assert.equal(opsTouchOnly(restore, ["/interface/gre", "/ip/address"]), true)
|
||||
}
|
||||
|
||||
{
|
||||
const p1 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1476 }] })
|
||||
const p2 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1476 }] })
|
||||
const p3 = fingerprintPayload({ tunnels: [{ name: "gre-a", mtu: 1400 }] })
|
||||
assert.equal(p1, p2)
|
||||
assert.notEqual(p1, p3)
|
||||
}
|
||||
|
||||
console.log("entity-snapshots.test.ts: ok")
|
||||
@@ -0,0 +1,649 @@
|
||||
/** Канонические снапшоты и планы restore для firewall / WireGuard / GRE. */
|
||||
|
||||
export type FirewallFamily = "ip" | "ip6"
|
||||
export type FirewallTable = "filter" | "nat" | "mangle" | "raw"
|
||||
|
||||
export type RosWriteOp =
|
||||
| { op: "put"; path: string; body: Record<string, string> }
|
||||
| { op: "post"; path: string; body: Record<string, string> }
|
||||
| { op: "patch"; path: string; body: Record<string, string> }
|
||||
| { op: "delete"; path: string }
|
||||
| { op: "move"; path: string; body: Record<string, string> }
|
||||
|
||||
export interface FirewallSnapshotRule {
|
||||
family: FirewallFamily
|
||||
table: FirewallTable
|
||||
chain: string
|
||||
action: string
|
||||
protocol: string
|
||||
srcAddress: string
|
||||
dstAddress: string
|
||||
srcAddressList: string
|
||||
dstAddressList: string
|
||||
srcPort: string
|
||||
dstPort: string
|
||||
inInterface: string
|
||||
outInterface: string
|
||||
connectionState: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
log: boolean
|
||||
logPrefix: string
|
||||
tlsHost: string
|
||||
layer7Proto: string
|
||||
}
|
||||
|
||||
export interface FirewallSnapshotList {
|
||||
family: FirewallFamily
|
||||
list: string
|
||||
address: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
timeout: string
|
||||
}
|
||||
|
||||
export interface FirewallSnapshot {
|
||||
rules: FirewallSnapshotRule[]
|
||||
addressLists: FirewallSnapshotList[]
|
||||
}
|
||||
|
||||
export interface FirewallLiveRule extends FirewallSnapshotRule {
|
||||
rosId: string
|
||||
dynamic: boolean
|
||||
}
|
||||
|
||||
export interface FirewallLiveList extends FirewallSnapshotList {
|
||||
rosId: string
|
||||
dynamic: boolean
|
||||
}
|
||||
|
||||
export interface WgSnapshotPeer {
|
||||
publicKey: string
|
||||
allowedAddresses: string[]
|
||||
endpointAddress: string
|
||||
endpointPort: string
|
||||
persistentKeepalive: number | null
|
||||
comment: string
|
||||
name: string
|
||||
disabled: boolean
|
||||
privateKey: string
|
||||
clientAddress: string
|
||||
clientDns: string
|
||||
clientEndpoint: string
|
||||
}
|
||||
|
||||
export interface WgSnapshotIface {
|
||||
name: string
|
||||
listenPort: number
|
||||
mtu: number
|
||||
privateKey: string
|
||||
address: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
peers: WgSnapshotPeer[]
|
||||
}
|
||||
|
||||
export interface WgSnapshot {
|
||||
interfaces: WgSnapshotIface[]
|
||||
}
|
||||
|
||||
export interface WgLiveIface {
|
||||
name: string
|
||||
rosId: string
|
||||
listenPort: number
|
||||
mtu: number
|
||||
privateKey: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
}
|
||||
|
||||
export interface WgLivePeer {
|
||||
rosId: string
|
||||
interfaceName: string
|
||||
publicKey: string
|
||||
allowedAddresses: string[]
|
||||
endpointAddress: string
|
||||
endpointPort: string
|
||||
persistentKeepalive: number | null
|
||||
comment: string
|
||||
name: string
|
||||
disabled: boolean
|
||||
privateKey: string
|
||||
clientAddress: string
|
||||
clientDns: string
|
||||
clientEndpoint: string
|
||||
}
|
||||
|
||||
export interface WgLiveAddr {
|
||||
rosId: string
|
||||
interfaceName: string
|
||||
address: string
|
||||
}
|
||||
|
||||
export interface GreSnapshotTunnel {
|
||||
name: string
|
||||
localAddress: string
|
||||
remoteAddress: string
|
||||
localInnerIp: string
|
||||
remoteInnerIp: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
mtu: number
|
||||
keepalive: string
|
||||
dscp: string
|
||||
clampTcpMss: boolean
|
||||
allowFastPath: boolean
|
||||
ipsecSecret: string
|
||||
}
|
||||
|
||||
export interface GreSnapshot {
|
||||
tunnels: GreSnapshotTunnel[]
|
||||
}
|
||||
|
||||
export interface GreLiveIface {
|
||||
name: string
|
||||
rosId: string
|
||||
localAddress: string
|
||||
remoteAddress: string
|
||||
comment: string
|
||||
disabled: boolean
|
||||
mtu: number
|
||||
keepalive: string
|
||||
dscp: string
|
||||
clampTcpMss: boolean
|
||||
allowFastPath: boolean
|
||||
ipsecSecret: string
|
||||
}
|
||||
|
||||
export interface GreLiveAddr {
|
||||
rosId: string
|
||||
interfaceName: string
|
||||
address: string
|
||||
}
|
||||
|
||||
function str(v: unknown): string {
|
||||
return String(v ?? "").trim()
|
||||
}
|
||||
|
||||
function bool(v: unknown): boolean {
|
||||
if (typeof v === "boolean") return v
|
||||
const s = str(v).toLowerCase()
|
||||
return s === "true" || s === "yes" || s === "1"
|
||||
}
|
||||
|
||||
function num(v: unknown, fallback: number): number {
|
||||
const n = typeof v === "number" ? v : Number.parseInt(str(v), 10)
|
||||
return Number.isFinite(n) ? n : fallback
|
||||
}
|
||||
|
||||
export function isHiddenSecret(value: string | undefined): boolean {
|
||||
const s = str(value)
|
||||
if (!s) return true
|
||||
if (s === "(hidden)") return true
|
||||
return /^\*+$/.test(s)
|
||||
}
|
||||
|
||||
export function firewallRestPath(
|
||||
family: FirewallFamily,
|
||||
table: FirewallTable | "address-list",
|
||||
): string {
|
||||
const root = family === "ip6" ? "/ipv6/firewall" : "/ip/firewall"
|
||||
return `${root}/${table}`
|
||||
}
|
||||
|
||||
function rosYesNo(v: boolean | undefined): string | undefined {
|
||||
if (v === true) return "yes"
|
||||
if (v === false) return "no"
|
||||
return undefined
|
||||
}
|
||||
|
||||
function compactBody(obj: Record<string, string | undefined>): Record<string, string> {
|
||||
const out: Record<string, string> = {}
|
||||
for (const [k, v] of Object.entries(obj)) {
|
||||
if (v !== undefined && v !== "") out[k] = v
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function canonicalFirewallSnapshot(input: {
|
||||
rules?: Array<Partial<FirewallSnapshotRule>>
|
||||
addressLists?: Array<Partial<FirewallSnapshotList>>
|
||||
}): FirewallSnapshot {
|
||||
const rules = (input.rules ?? []).map((r) => ({
|
||||
family: r.family === "ip6" ? "ip6" as const : "ip" as const,
|
||||
table: (["filter", "nat", "mangle", "raw"] as const).includes(r.table as FirewallTable)
|
||||
? (r.table as FirewallTable)
|
||||
: "filter",
|
||||
chain: str(r.chain),
|
||||
action: str(r.action),
|
||||
protocol: str(r.protocol),
|
||||
srcAddress: str(r.srcAddress),
|
||||
dstAddress: str(r.dstAddress),
|
||||
srcAddressList: str(r.srcAddressList),
|
||||
dstAddressList: str(r.dstAddressList),
|
||||
srcPort: str(r.srcPort),
|
||||
dstPort: str(r.dstPort),
|
||||
inInterface: str(r.inInterface),
|
||||
outInterface: str(r.outInterface),
|
||||
connectionState: str(r.connectionState),
|
||||
comment: str(r.comment),
|
||||
disabled: Boolean(r.disabled),
|
||||
log: Boolean(r.log),
|
||||
logPrefix: str(r.logPrefix),
|
||||
tlsHost: str(r.tlsHost),
|
||||
layer7Proto: str(r.layer7Proto),
|
||||
}))
|
||||
const addressLists = (input.addressLists ?? []).map((e) => ({
|
||||
family: e.family === "ip6" ? "ip6" as const : "ip" as const,
|
||||
list: str(e.list),
|
||||
address: str(e.address),
|
||||
comment: str(e.comment),
|
||||
disabled: Boolean(e.disabled),
|
||||
timeout: str(e.timeout),
|
||||
}))
|
||||
return { rules, addressLists }
|
||||
}
|
||||
|
||||
export function parseFirewallSnapshot(payload: unknown): FirewallSnapshot {
|
||||
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
|
||||
return { rules: [], addressLists: [] }
|
||||
}
|
||||
const o = payload as Record<string, unknown>
|
||||
return canonicalFirewallSnapshot({
|
||||
rules: Array.isArray(o.rules) ? o.rules as Partial<FirewallSnapshotRule>[] : [],
|
||||
addressLists: Array.isArray(o.addressLists) ? o.addressLists as Partial<FirewallSnapshotList>[] : [],
|
||||
})
|
||||
}
|
||||
|
||||
function firewallRuleKey(r: FirewallSnapshotRule): string {
|
||||
return [
|
||||
r.family, r.table, r.chain, r.action, r.protocol,
|
||||
r.srcAddress, r.dstAddress, r.srcAddressList, r.dstAddressList,
|
||||
r.srcPort, r.dstPort, r.inInterface, r.outInterface, r.connectionState,
|
||||
r.comment, r.disabled ? "1" : "0", r.log ? "1" : "0", r.logPrefix, r.tlsHost, r.layer7Proto,
|
||||
].join("\0")
|
||||
}
|
||||
|
||||
function firewallListKey(e: FirewallSnapshotList): string {
|
||||
return [e.family, e.list, e.address, e.comment, e.disabled ? "1" : "0", e.timeout].join("\0")
|
||||
}
|
||||
|
||||
function firewallRuleBody(r: FirewallSnapshotRule): Record<string, string> {
|
||||
return compactBody({
|
||||
chain: r.chain,
|
||||
action: r.action,
|
||||
protocol: r.protocol && r.protocol !== "all" ? r.protocol : undefined,
|
||||
"src-address": r.srcAddress,
|
||||
"dst-address": r.dstAddress,
|
||||
"src-address-list": r.srcAddressList,
|
||||
"dst-address-list": r.dstAddressList,
|
||||
"src-port": r.srcPort,
|
||||
"dst-port": r.dstPort,
|
||||
"in-interface": r.inInterface,
|
||||
"out-interface": r.outInterface,
|
||||
"connection-state": r.connectionState,
|
||||
comment: r.comment,
|
||||
disabled: rosYesNo(r.disabled),
|
||||
log: rosYesNo(r.log),
|
||||
"log-prefix": r.logPrefix,
|
||||
"tls-host": r.tlsHost,
|
||||
"layer7-protocol": r.layer7Proto,
|
||||
})
|
||||
}
|
||||
|
||||
export function planFirewallRestore(
|
||||
desiredInput: FirewallSnapshot,
|
||||
current: { rules: FirewallLiveRule[]; addressLists: FirewallLiveList[] },
|
||||
): RosWriteOp[] {
|
||||
const desired = canonicalFirewallSnapshot(desiredInput)
|
||||
const ops: RosWriteOp[] = []
|
||||
const usedRules = new Set<string>()
|
||||
const usedLists = new Set<string>()
|
||||
|
||||
for (const live of current.rules) {
|
||||
if (live.dynamic) continue
|
||||
const key = firewallRuleKey(live)
|
||||
const stillWanted = desired.rules.some((d) => firewallRuleKey(d) === key)
|
||||
if (!stillWanted) {
|
||||
ops.push({
|
||||
op: "delete",
|
||||
path: `${firewallRestPath(live.family, live.table)}/${live.rosId}`,
|
||||
})
|
||||
} else {
|
||||
usedRules.add(key)
|
||||
}
|
||||
}
|
||||
|
||||
for (const live of current.addressLists) {
|
||||
if (live.dynamic) continue
|
||||
const key = firewallListKey(live)
|
||||
const stillWanted = desired.addressLists.some((d) => firewallListKey(d) === key)
|
||||
if (!stillWanted) {
|
||||
ops.push({
|
||||
op: "delete",
|
||||
path: `${firewallRestPath(live.family, "address-list")}/${live.rosId}`,
|
||||
})
|
||||
} else {
|
||||
usedLists.add(key)
|
||||
}
|
||||
}
|
||||
|
||||
for (const rule of desired.rules) {
|
||||
if (usedRules.has(firewallRuleKey(rule))) continue
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: firewallRestPath(rule.family, rule.table),
|
||||
body: firewallRuleBody(rule),
|
||||
})
|
||||
}
|
||||
|
||||
for (const entry of desired.addressLists) {
|
||||
if (usedLists.has(firewallListKey(entry))) continue
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: firewallRestPath(entry.family, "address-list"),
|
||||
body: compactBody({
|
||||
list: entry.list,
|
||||
address: entry.address,
|
||||
comment: entry.comment,
|
||||
timeout: entry.timeout,
|
||||
disabled: rosYesNo(entry.disabled),
|
||||
}),
|
||||
})
|
||||
}
|
||||
|
||||
return ops
|
||||
}
|
||||
|
||||
function canonicalPeer(p: Partial<WgSnapshotPeer>): WgSnapshotPeer {
|
||||
const allowed = Array.isArray(p.allowedAddresses)
|
||||
? p.allowedAddresses.map((a) => str(a)).filter(Boolean)
|
||||
: str((p as { allowedIps?: unknown }).allowedIps)
|
||||
.split(",")
|
||||
.map((s) => s.trim())
|
||||
.filter(Boolean)
|
||||
return {
|
||||
publicKey: str(p.publicKey),
|
||||
allowedAddresses: allowed,
|
||||
endpointAddress: str(p.endpointAddress),
|
||||
endpointPort: str(p.endpointPort),
|
||||
persistentKeepalive: p.persistentKeepalive == null ? null : num(p.persistentKeepalive, 0) || null,
|
||||
comment: str(p.comment),
|
||||
name: str(p.name),
|
||||
disabled: Boolean(p.disabled),
|
||||
privateKey: str(p.privateKey),
|
||||
clientAddress: str(p.clientAddress),
|
||||
clientDns: str(p.clientDns),
|
||||
clientEndpoint: str(p.clientEndpoint),
|
||||
}
|
||||
}
|
||||
|
||||
export function canonicalWireguardSnapshot(input: {
|
||||
interfaces?: Array<Partial<WgSnapshotIface> & { peers?: Array<Partial<WgSnapshotPeer>> }>
|
||||
}): WgSnapshot {
|
||||
const interfaces = (input.interfaces ?? [])
|
||||
.map((iface) => ({
|
||||
name: str(iface.name),
|
||||
listenPort: num(iface.listenPort, 13231),
|
||||
mtu: num(iface.mtu, 1420),
|
||||
privateKey: str(iface.privateKey),
|
||||
address: str(iface.address),
|
||||
comment: str(iface.comment),
|
||||
disabled: Boolean(iface.disabled),
|
||||
peers: (iface.peers ?? []).map(canonicalPeer).sort((a, b) => a.publicKey.localeCompare(b.publicKey)),
|
||||
}))
|
||||
.filter((i) => i.name)
|
||||
.sort((a, b) => a.name.localeCompare(b.name))
|
||||
return { interfaces }
|
||||
}
|
||||
|
||||
export function parseWireguardSnapshot(payload: unknown): WgSnapshot {
|
||||
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
|
||||
return { interfaces: [] }
|
||||
}
|
||||
const o = payload as Record<string, unknown>
|
||||
return canonicalWireguardSnapshot({
|
||||
interfaces: Array.isArray(o.interfaces)
|
||||
? o.interfaces as Array<Partial<WgSnapshotIface> & { peers?: Array<Partial<WgSnapshotPeer>> }>
|
||||
: [],
|
||||
})
|
||||
}
|
||||
|
||||
function peerBody(interfaceName: string, p: WgSnapshotPeer): Record<string, string> {
|
||||
return compactBody({
|
||||
interface: interfaceName,
|
||||
"public-key": p.publicKey,
|
||||
"allowed-address": p.allowedAddresses.join(","),
|
||||
"endpoint-address": p.endpointAddress,
|
||||
"endpoint-port": p.endpointPort,
|
||||
"persistent-keepalive": p.persistentKeepalive != null ? String(p.persistentKeepalive) : undefined,
|
||||
comment: p.comment,
|
||||
name: p.name,
|
||||
"private-key": isHiddenSecret(p.privateKey) ? undefined : p.privateKey,
|
||||
"client-address": p.clientAddress,
|
||||
"client-dns": p.clientDns,
|
||||
"client-endpoint": p.clientEndpoint,
|
||||
disabled: rosYesNo(p.disabled),
|
||||
})
|
||||
}
|
||||
|
||||
export function planWireguardRestore(
|
||||
desiredInput: WgSnapshot,
|
||||
current: { ifaces: WgLiveIface[]; peers: WgLivePeer[]; addrs: WgLiveAddr[] },
|
||||
): RosWriteOp[] {
|
||||
const desired = canonicalWireguardSnapshot(desiredInput)
|
||||
const wantedNames = new Set(desired.interfaces.map((i) => i.name))
|
||||
const ops: RosWriteOp[] = []
|
||||
|
||||
for (const peer of current.peers) {
|
||||
const iface = desired.interfaces.find((i) => i.name === peer.interfaceName)
|
||||
const keep = iface?.peers.some((p) => p.publicKey === peer.publicKey)
|
||||
if (!keep) {
|
||||
ops.push({ op: "delete", path: `/interface/wireguard/peers/${peer.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
for (const addr of current.addrs) {
|
||||
if (!wantedNames.has(addr.interfaceName)) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${addr.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
for (const iface of current.ifaces) {
|
||||
if (!wantedNames.has(iface.name)) {
|
||||
ops.push({ op: "delete", path: `/interface/wireguard/${iface.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
for (const want of desired.interfaces) {
|
||||
const live = current.ifaces.find((i) => i.name === want.name)
|
||||
const ifaceBody = compactBody({
|
||||
name: want.name,
|
||||
"listen-port": String(want.listenPort),
|
||||
mtu: String(want.mtu),
|
||||
"private-key": isHiddenSecret(want.privateKey) ? undefined : want.privateKey,
|
||||
comment: want.comment,
|
||||
disabled: rosYesNo(want.disabled),
|
||||
})
|
||||
if (!live) {
|
||||
ops.push({ op: "put", path: "/interface/wireguard", body: ifaceBody })
|
||||
} else {
|
||||
ops.push({
|
||||
op: "patch",
|
||||
path: `/interface/wireguard/${live.rosId}`,
|
||||
body: ifaceBody,
|
||||
})
|
||||
}
|
||||
|
||||
const liveAddr = current.addrs.find((a) => a.interfaceName === want.name)
|
||||
if (want.address) {
|
||||
if (!liveAddr) {
|
||||
ops.push({ op: "put", path: "/ip/address", body: { address: want.address, interface: want.name } })
|
||||
} else if (liveAddr.address !== want.address) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
ops.push({ op: "put", path: "/ip/address", body: { address: want.address, interface: want.name } })
|
||||
}
|
||||
} else if (liveAddr) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
}
|
||||
|
||||
for (const peer of want.peers) {
|
||||
if (!peer.publicKey) continue
|
||||
const livePeer = current.peers.find(
|
||||
(p) => p.interfaceName === want.name && p.publicKey === peer.publicKey,
|
||||
)
|
||||
const body = peerBody(want.name, peer)
|
||||
if (!livePeer) {
|
||||
ops.push({ op: "put", path: "/interface/wireguard/peers", body })
|
||||
} else {
|
||||
ops.push({
|
||||
op: "patch",
|
||||
path: `/interface/wireguard/peers/${livePeer.rosId}`,
|
||||
body,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
return ops
|
||||
}
|
||||
|
||||
export function canonicalGreSnapshot(input: {
|
||||
tunnels?: Array<Partial<GreSnapshotTunnel>>
|
||||
}): GreSnapshot {
|
||||
const tunnels = (input.tunnels ?? [])
|
||||
.map((t) => ({
|
||||
name: str(t.name),
|
||||
localAddress: str(t.localAddress),
|
||||
remoteAddress: str(t.remoteAddress),
|
||||
localInnerIp: str(t.localInnerIp),
|
||||
remoteInnerIp: str(t.remoteInnerIp),
|
||||
comment: str(t.comment),
|
||||
disabled: Boolean(t.disabled),
|
||||
mtu: num(t.mtu, 1476),
|
||||
keepalive: str(t.keepalive) || "0",
|
||||
dscp: str(t.dscp) || "inherit",
|
||||
clampTcpMss: t.clampTcpMss !== false,
|
||||
allowFastPath: t.allowFastPath !== false,
|
||||
ipsecSecret: str(t.ipsecSecret),
|
||||
}))
|
||||
.filter((t) => t.name)
|
||||
.sort((a, b) => a.name.localeCompare(b.name))
|
||||
return { tunnels }
|
||||
}
|
||||
|
||||
export function parseGreSnapshot(payload: unknown): GreSnapshot {
|
||||
if (!payload || typeof payload !== "object" || Array.isArray(payload)) {
|
||||
return { tunnels: [] }
|
||||
}
|
||||
const o = payload as Record<string, unknown>
|
||||
return canonicalGreSnapshot({
|
||||
tunnels: Array.isArray(o.tunnels) ? o.tunnels as Array<Partial<GreSnapshotTunnel>> : [],
|
||||
})
|
||||
}
|
||||
|
||||
export function greInterfaceBody(t: GreSnapshotTunnel): Record<string, string> {
|
||||
return compactBody({
|
||||
name: t.name,
|
||||
"local-address": t.localAddress && t.localAddress !== "0.0.0.0" ? t.localAddress : undefined,
|
||||
"remote-address": t.remoteAddress,
|
||||
mtu: String(t.mtu),
|
||||
keepalive: t.keepalive,
|
||||
dscp: t.dscp,
|
||||
"clamp-tcp-mss": t.clampTcpMss ? "yes" : "no",
|
||||
"allow-fast-path": t.allowFastPath ? "yes" : "no",
|
||||
comment: t.comment,
|
||||
disabled: rosYesNo(t.disabled),
|
||||
"ipsec-secret": isHiddenSecret(t.ipsecSecret) ? undefined : t.ipsecSecret,
|
||||
})
|
||||
}
|
||||
|
||||
export function planGreCreate(tunnel: GreSnapshotTunnel): RosWriteOp[] {
|
||||
const t = canonicalGreSnapshot({ tunnels: [tunnel] }).tunnels[0]
|
||||
if (!t) return []
|
||||
const ops: RosWriteOp[] = [
|
||||
{ op: "put", path: "/interface/gre", body: greInterfaceBody(t) },
|
||||
]
|
||||
if (t.localInnerIp) {
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: "/ip/address",
|
||||
body: { address: t.localInnerIp, interface: t.name },
|
||||
})
|
||||
}
|
||||
return ops
|
||||
}
|
||||
|
||||
export function planGreDelete(
|
||||
name: string,
|
||||
current: { gre: GreLiveIface[]; addrs: GreLiveAddr[] },
|
||||
): RosWriteOp[] {
|
||||
const want = str(name)
|
||||
const ops: RosWriteOp[] = []
|
||||
for (const addr of current.addrs) {
|
||||
if (addr.interfaceName === want) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${addr.rosId}` })
|
||||
}
|
||||
}
|
||||
for (const gre of current.gre) {
|
||||
if (gre.name === want) {
|
||||
ops.push({ op: "delete", path: `/interface/gre/${gre.rosId}` })
|
||||
}
|
||||
}
|
||||
return ops
|
||||
}
|
||||
|
||||
export function planGreRestore(
|
||||
desiredInput: GreSnapshot,
|
||||
current: { gre: GreLiveIface[]; addrs: GreLiveAddr[] },
|
||||
): RosWriteOp[] {
|
||||
const desired = canonicalGreSnapshot(desiredInput)
|
||||
const wanted = new Set(desired.tunnels.map((t) => t.name))
|
||||
const ops: RosWriteOp[] = []
|
||||
|
||||
for (const gre of current.gre) {
|
||||
if (!wanted.has(gre.name)) {
|
||||
ops.push(...planGreDelete(gre.name, current))
|
||||
}
|
||||
}
|
||||
|
||||
for (const want of desired.tunnels) {
|
||||
const live = current.gre.find((g) => g.name === want.name)
|
||||
const body = greInterfaceBody(want)
|
||||
if (!live) {
|
||||
ops.push({ op: "put", path: "/interface/gre", body })
|
||||
} else {
|
||||
ops.push({ op: "patch", path: `/interface/gre/${live.rosId}`, body })
|
||||
}
|
||||
|
||||
const liveAddr = current.addrs.find((a) => a.interfaceName === want.name)
|
||||
if (want.localInnerIp) {
|
||||
if (!liveAddr) {
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: "/ip/address",
|
||||
body: { address: want.localInnerIp, interface: want.name },
|
||||
})
|
||||
} else if (liveAddr.address !== want.localInnerIp) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
ops.push({
|
||||
op: "put",
|
||||
path: "/ip/address",
|
||||
body: { address: want.localInnerIp, interface: want.name },
|
||||
})
|
||||
}
|
||||
} else if (liveAddr) {
|
||||
ops.push({ op: "delete", path: `/ip/address/${liveAddr.rosId}` })
|
||||
}
|
||||
}
|
||||
|
||||
return ops
|
||||
}
|
||||
|
||||
export function opsPaths(ops: RosWriteOp[]): string[] {
|
||||
return ops.map((op) => op.path)
|
||||
}
|
||||
|
||||
export function opsTouchOnly(ops: RosWriteOp[], prefixes: string[]): boolean {
|
||||
return ops.every((op) => prefixes.some((p) => op.path === p || op.path.startsWith(`${p}/`)))
|
||||
}
|
||||
@@ -5,12 +5,21 @@ import {
|
||||
MikrotikClient,
|
||||
firewallRestPath,
|
||||
} from "./mikrotik.js"
|
||||
import {
|
||||
captureAndAppendRevision,
|
||||
} from "./config-revisions.js"
|
||||
import type {
|
||||
FirewallFamily,
|
||||
FirewallTable,
|
||||
RosFirewallAddressList,
|
||||
RosFirewallFilter,
|
||||
} from "../types/server.js"
|
||||
import {
|
||||
canonicalFirewallSnapshot,
|
||||
type FirewallLiveList,
|
||||
type FirewallLiveRule,
|
||||
type FirewallSnapshot,
|
||||
} from "./entity-snapshots.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -150,29 +159,121 @@ async function safeGet<T>(fn: () => Promise<T[]>, fallback: T[] = []): Promise<T
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchServerFirewall(server: ServerRow): Promise<{
|
||||
function rosYes(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
export function mapFirewallSnapshotRule(
|
||||
family: FirewallFamily,
|
||||
table: FirewallTable,
|
||||
raw: RosFirewallFilter,
|
||||
): FirewallLiveRule {
|
||||
return {
|
||||
rosId: raw[".id"] || "",
|
||||
dynamic: rosYes(raw.dynamic),
|
||||
family,
|
||||
table,
|
||||
chain: raw.chain || "",
|
||||
action: raw.action || "",
|
||||
protocol: raw.protocol || "",
|
||||
srcAddress: raw["src-address"] ?? "",
|
||||
dstAddress: raw["dst-address"] ?? "",
|
||||
srcAddressList: raw["src-address-list"] ?? "",
|
||||
dstAddressList: raw["dst-address-list"] ?? "",
|
||||
srcPort: raw["src-port"] ?? "",
|
||||
dstPort: raw["dst-port"] ?? "",
|
||||
inInterface: raw["in-interface"] ?? "",
|
||||
outInterface: raw["out-interface"] ?? "",
|
||||
connectionState: raw["connection-state"] ?? "",
|
||||
comment: raw.comment ?? "",
|
||||
disabled: rosDisabled(raw.disabled),
|
||||
log: rosYes(raw.log),
|
||||
logPrefix: raw["log-prefix"] ?? "",
|
||||
tlsHost: raw["tls-host"] ?? "",
|
||||
layer7Proto: raw["layer7-protocol"] ?? "",
|
||||
}
|
||||
}
|
||||
|
||||
export function mapFirewallSnapshotList(
|
||||
family: FirewallFamily,
|
||||
raw: RosFirewallAddressList,
|
||||
): FirewallLiveList {
|
||||
return {
|
||||
rosId: raw[".id"] || "",
|
||||
dynamic: rosYes(raw.dynamic),
|
||||
family,
|
||||
list: raw.list || "",
|
||||
address: raw.address || "",
|
||||
comment: raw.comment ?? "",
|
||||
disabled: rosDisabled(raw.disabled),
|
||||
timeout: raw.timeout ?? "",
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchFirewallState(server: ServerRow): Promise<{
|
||||
rules: FirewallRuleDto[]
|
||||
addressLists: FirewallAddressListDto[]
|
||||
liveRules: FirewallLiveRule[]
|
||||
liveLists: FirewallLiveList[]
|
||||
snapshot: FirewallSnapshot
|
||||
}> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const ruleJobs = FAMILIES.flatMap((family) =>
|
||||
TABLES.map(async (table) => {
|
||||
const raw = await safeGet(() => client.getFirewallRules(family, table))
|
||||
return raw.map((row, idx) => mapFirewallRule(server, family, table, row, idx))
|
||||
return { family, table, raw }
|
||||
}),
|
||||
)
|
||||
const listJobs = FAMILIES.map(async (family) => {
|
||||
const raw = await safeGet(() => client.getFirewallAddressList(family))
|
||||
return raw.map((row, idx) => mapAddressList(server, family, row, idx))
|
||||
return { family, raw }
|
||||
})
|
||||
const [ruleChunks, listChunks] = await Promise.all([
|
||||
Promise.all(ruleJobs),
|
||||
Promise.all(listJobs),
|
||||
])
|
||||
return {
|
||||
rules: ruleChunks.flat(),
|
||||
addressLists: listChunks.flat(),
|
||||
|
||||
const rules: FirewallRuleDto[] = []
|
||||
const liveRules: FirewallLiveRule[] = []
|
||||
for (const chunk of ruleChunks) {
|
||||
chunk.raw.forEach((row, idx) => {
|
||||
rules.push(mapFirewallRule(server, chunk.family, chunk.table, row, idx))
|
||||
liveRules.push(mapFirewallSnapshotRule(chunk.family, chunk.table, row))
|
||||
})
|
||||
}
|
||||
|
||||
const addressLists: FirewallAddressListDto[] = []
|
||||
const liveLists: FirewallLiveList[] = []
|
||||
for (const chunk of listChunks) {
|
||||
chunk.raw.forEach((row, idx) => {
|
||||
addressLists.push(mapAddressList(server, chunk.family, row, idx))
|
||||
liveLists.push(mapFirewallSnapshotList(chunk.family, row))
|
||||
})
|
||||
}
|
||||
|
||||
return {
|
||||
rules,
|
||||
addressLists,
|
||||
liveRules,
|
||||
liveLists,
|
||||
snapshot: canonicalFirewallSnapshot({
|
||||
rules: liveRules.filter((r) => !r.dynamic),
|
||||
addressLists: liveLists.filter((e) => !e.dynamic),
|
||||
}),
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchServerFirewall(server: ServerRow): Promise<{
|
||||
rules: FirewallRuleDto[]
|
||||
addressLists: FirewallAddressListDto[]
|
||||
}> {
|
||||
const state = await fetchFirewallState(server)
|
||||
return { rules: state.rules, addressLists: state.addressLists }
|
||||
}
|
||||
|
||||
export async function captureFirewallSnapshot(server: ServerRow): Promise<FirewallSnapshot> {
|
||||
const state = await fetchFirewallState(server)
|
||||
return state.snapshot
|
||||
}
|
||||
|
||||
export async function listFirewallAll(): Promise<{
|
||||
@@ -183,7 +284,14 @@ export async function listFirewallAll(): Promise<{
|
||||
const perServer = await Promise.all(
|
||||
allServers.map(async (server) => {
|
||||
try {
|
||||
return await fetchServerFirewall(server)
|
||||
const state = await fetchFirewallState(server)
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "firewall",
|
||||
source: "observed",
|
||||
capture: async () => state.snapshot,
|
||||
})
|
||||
return { rules: state.rules, addressLists: state.addressLists }
|
||||
} catch {
|
||||
return { rules: [] as FirewallRuleDto[], addressLists: [] as FirewallAddressListDto[] }
|
||||
}
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { geoipSettings } from "../db/schema.js"
|
||||
import type { GeoipSettingsDto, GeoipSettingsPatch } from "@mmapp/contracts/geoip"
|
||||
|
||||
const SETTINGS_ID = 1
|
||||
const DEFAULT_INTERVAL_SEC = 604800
|
||||
|
||||
let dbEnabled = true
|
||||
|
||||
/** Тесты без PostgreSQL: геттеры отдают дефолты, touch/update — no-op. */
|
||||
export function disableGeoipDbForTests(): void {
|
||||
dbEnabled = false
|
||||
}
|
||||
|
||||
export function resetGeoipSettingsForTests(): void {
|
||||
dbEnabled = true
|
||||
}
|
||||
|
||||
type GeoipSettingsRow = typeof geoipSettings.$inferSelect
|
||||
|
||||
async function getGeoipSettingsRow(): Promise<GeoipSettingsRow | undefined> {
|
||||
if (!dbEnabled) return undefined
|
||||
return (
|
||||
(await db.select().from(geoipSettings).where(eq(geoipSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
)
|
||||
}
|
||||
|
||||
function toDto(row: GeoipSettingsRow | undefined): GeoipSettingsDto {
|
||||
return {
|
||||
enabled: row?.enabled ?? true,
|
||||
updateIntervalSec: row?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
|
||||
lastCheckAt: row?.lastCheckAt ?? null,
|
||||
lastSuccessAt: row?.lastSuccessAt ?? null,
|
||||
lastError: row?.lastError ?? null,
|
||||
countryBuildAt: row?.countryBuildAt ?? null,
|
||||
asnBuildAt: row?.asnBuildAt ?? null,
|
||||
updatedAt: row?.updatedAt ?? new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
export async function getGeoipSettings(): Promise<GeoipSettingsDto> {
|
||||
return toDto(await getGeoipSettingsRow())
|
||||
}
|
||||
|
||||
/** ETag'и зеркала для conditional GET (ключ — имя файла базы). */
|
||||
export async function getGeoipEtags(): Promise<Record<string, string>> {
|
||||
const row = await getGeoipSettingsRow()
|
||||
if (!row) return {}
|
||||
const raw = row?.etagsJson
|
||||
if (!raw || typeof raw !== "object") return {}
|
||||
return Object.fromEntries(
|
||||
Object.entries(raw as Record<string, unknown>).filter(
|
||||
(entry): entry is [string, string] => typeof entry[1] === "string",
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
export async function updateGeoipSettings(patch: GeoipSettingsPatch): Promise<GeoipSettingsDto> {
|
||||
const prev = await getGeoipSettingsRow()
|
||||
const next = {
|
||||
enabled: patch.enabled ?? prev?.enabled ?? true,
|
||||
updateIntervalSec: patch.updateIntervalSec ?? prev?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
if (prev) {
|
||||
await db.update(geoipSettings).set(next).where(eq(geoipSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return getGeoipSettings()
|
||||
}
|
||||
|
||||
export async function touchGeoipRunMeta(patch: {
|
||||
lastCheckAt?: string
|
||||
lastSuccessAt?: string | null
|
||||
lastError?: string | null
|
||||
countryBuildAt?: string | null
|
||||
asnBuildAt?: string | null
|
||||
etags?: Record<string, string>
|
||||
}): Promise<void> {
|
||||
const prev = await getGeoipSettingsRow()
|
||||
const set: Partial<typeof geoipSettings.$inferInsert> = {
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
if (patch.lastCheckAt !== undefined) set.lastCheckAt = patch.lastCheckAt
|
||||
if (patch.lastSuccessAt !== undefined) set.lastSuccessAt = patch.lastSuccessAt
|
||||
if (patch.lastError !== undefined) set.lastError = patch.lastError
|
||||
if (patch.countryBuildAt !== undefined) set.countryBuildAt = patch.countryBuildAt
|
||||
if (patch.asnBuildAt !== undefined) set.asnBuildAt = patch.asnBuildAt
|
||||
if (patch.etags !== undefined) {
|
||||
const prevEtags = (prev?.etagsJson as Record<string, string> | null) ?? {}
|
||||
set.etagsJson = { ...prevEtags, ...patch.etags }
|
||||
}
|
||||
if (prev) {
|
||||
await db.update(geoipSettings).set(set).where(eq(geoipSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...set })
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
import { rename, rm, mkdir, writeFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { open, type AsnResponse, type CountryResponse } from "maxmind"
|
||||
import type { GeoipUpdateRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
|
||||
import { getGeoipEtags, getGeoipSettings, touchGeoipRunMeta } from "./geoip-settings.js"
|
||||
import {
|
||||
GEOIP_ASN_FILE,
|
||||
GEOIP_COUNTRY_FILE,
|
||||
geoipDir,
|
||||
reloadGeoipReaders,
|
||||
} from "./traffic-flow-geoip.js"
|
||||
|
||||
/** Зеркало GeoLite2 без регистрации и ключей (см. README: GeoIP). */
|
||||
const MIRROR_BASE = "https://github.com/P3TERX/GeoLite.mmdb/raw/download"
|
||||
const DOWNLOAD_TIMEOUT_MS = 120_000
|
||||
/** Пробный IP для валидации скачанной базы: Google DNS. */
|
||||
const PROBE_IP = "8.8.8.8"
|
||||
|
||||
type GeoipDbKind = "country" | "asn"
|
||||
|
||||
let updating = false
|
||||
let fetchImpl: typeof fetch = globalThis.fetch.bind(globalThis)
|
||||
|
||||
export function getGeoipUpdateState(): { running: boolean } {
|
||||
return { running: updating }
|
||||
}
|
||||
|
||||
async function validateCountryFile(filePath: string): Promise<string> {
|
||||
const reader = await open<CountryResponse>(filePath)
|
||||
const rec = reader.get(PROBE_IP)
|
||||
const iso = rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
|
||||
if (iso !== "US") {
|
||||
throw new Error(`база Country не распознала ${PROBE_IP} как US (${iso || "нет записи"})`)
|
||||
}
|
||||
return reader.metadata.buildEpoch.toISOString()
|
||||
}
|
||||
|
||||
async function validateAsnFile(filePath: string): Promise<string> {
|
||||
const reader = await open<AsnResponse>(filePath)
|
||||
const rec = reader.get(PROBE_IP)
|
||||
const asn = rec?.autonomous_system_number ?? 0
|
||||
if (asn !== 15169) {
|
||||
throw new Error(`база ASN не распознала ${PROBE_IP} как AS15169 (${asn ? `AS${asn}` : "нет записи"})`)
|
||||
}
|
||||
return reader.metadata.buildEpoch.toISOString()
|
||||
}
|
||||
|
||||
let validateCountry = validateCountryFile
|
||||
let validateAsn = validateAsnFile
|
||||
|
||||
export function setGeoipFetchForTests(fn: typeof fetch): void {
|
||||
fetchImpl = fn
|
||||
}
|
||||
|
||||
export function setGeoipValidateForTests(opts: {
|
||||
country?: (filePath: string) => Promise<string>
|
||||
asn?: (filePath: string) => Promise<string>
|
||||
}): void {
|
||||
validateCountry = opts.country ?? validateCountryFile
|
||||
validateAsn = opts.asn ?? validateAsnFile
|
||||
}
|
||||
|
||||
export function resetGeoipUpdateForTests(): void {
|
||||
updating = false
|
||||
fetchImpl = globalThis.fetch.bind(globalThis)
|
||||
validateCountry = validateCountryFile
|
||||
validateAsn = validateAsnFile
|
||||
}
|
||||
|
||||
/**
|
||||
* Разовая проверка/доставка баз с зеркала P3TERX. Conditional GET по ETag
|
||||
* (304 = не меняем файл), валидация пробоем 8.8.8.8, атомарная подмена через rename.
|
||||
*/
|
||||
export async function collectGeoipUpdateOnce(
|
||||
opts: { force?: boolean } = {},
|
||||
): Promise<GeoipUpdateRunSnapshot> {
|
||||
const sampledAt = new Date().toISOString()
|
||||
if (updating) {
|
||||
if (opts.force) {
|
||||
throw Object.assign(new Error("Обновление GeoIP уже выполняется"), { statusCode: 409 })
|
||||
}
|
||||
return emptySnapshot(sampledAt, true)
|
||||
}
|
||||
|
||||
const settings = await getGeoipSettings()
|
||||
if (!settings.enabled && !opts.force) {
|
||||
return emptySnapshot(sampledAt, true)
|
||||
}
|
||||
|
||||
updating = true
|
||||
const snapshot: GeoipUpdateRunSnapshot = {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "geoip_update",
|
||||
sampledAt,
|
||||
checked: 0,
|
||||
downloaded: 0,
|
||||
skippedUnchanged: 0,
|
||||
bytes: 0,
|
||||
errors: [],
|
||||
}
|
||||
|
||||
try {
|
||||
const dir = geoipDir()
|
||||
await mkdir(dir, { recursive: true })
|
||||
const storedEtags = await getGeoipEtags()
|
||||
const etags: Record<string, string> = {}
|
||||
const buildAt: Partial<Record<GeoipDbKind, string>> = {}
|
||||
|
||||
for (const kind of ["country", "asn"] as const) {
|
||||
snapshot.checked += 1
|
||||
const file = kind === "country" ? GEOIP_COUNTRY_FILE : GEOIP_ASN_FILE
|
||||
const target = path.join(dir, file)
|
||||
const tmp = `${target}.tmp`
|
||||
const prevEtag = storedEtags[file]
|
||||
try {
|
||||
const ac = new AbortController()
|
||||
const timer = setTimeout(() => ac.abort(), DOWNLOAD_TIMEOUT_MS)
|
||||
let res: Response
|
||||
try {
|
||||
res = await fetchImpl(`${MIRROR_BASE}/${file}`, {
|
||||
headers: prevEtag ? { "If-None-Match": prevEtag } : {},
|
||||
signal: ac.signal,
|
||||
})
|
||||
} finally {
|
||||
clearTimeout(timer)
|
||||
}
|
||||
if (res.status === 304) {
|
||||
snapshot.skippedUnchanged += 1
|
||||
if (prevEtag) etags[file] = prevEtag
|
||||
continue
|
||||
}
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`)
|
||||
const etag = res.headers.get("etag") ?? ""
|
||||
const body = Buffer.from(await res.arrayBuffer())
|
||||
snapshot.bytes += body.byteLength
|
||||
await writeFile(tmp, body)
|
||||
buildAt[kind] =
|
||||
kind === "country" ? await validateCountry(tmp) : await validateAsn(tmp)
|
||||
|
||||
const prevFile = `${target}.prev`
|
||||
await rm(prevFile, { force: true })
|
||||
await rename(target, prevFile).catch(() => {
|
||||
/* текущего файла могло ещё не быть */
|
||||
})
|
||||
await rename(tmp, target)
|
||||
snapshot.downloaded += 1
|
||||
if (etag) etags[file] = etag
|
||||
} catch (e) {
|
||||
await rm(tmp, { force: true }).catch(() => {
|
||||
/* best-effort */
|
||||
})
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.errors.push(`${file}: ${message}`)
|
||||
}
|
||||
}
|
||||
|
||||
if (snapshot.downloaded > 0) {
|
||||
await reloadGeoipReaders()
|
||||
}
|
||||
|
||||
await touchGeoipRunMeta({
|
||||
lastCheckAt: sampledAt,
|
||||
lastSuccessAt: snapshot.errors.length ? null : sampledAt,
|
||||
lastError: snapshot.errors.length ? snapshot.errors.join("; ") : null,
|
||||
countryBuildAt: buildAt.country,
|
||||
asnBuildAt: buildAt.asn,
|
||||
etags,
|
||||
})
|
||||
return snapshot
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.fatalError = message
|
||||
await touchGeoipRunMeta({
|
||||
lastCheckAt: sampledAt,
|
||||
lastError: message,
|
||||
}).catch(() => {
|
||||
/* best-effort */
|
||||
})
|
||||
return snapshot
|
||||
} finally {
|
||||
updating = false
|
||||
}
|
||||
}
|
||||
|
||||
function emptySnapshot(sampledAt: string, skipped: boolean): GeoipUpdateRunSnapshot {
|
||||
return {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "geoip_update",
|
||||
sampledAt,
|
||||
skipped,
|
||||
checked: 0,
|
||||
downloaded: 0,
|
||||
skippedUnchanged: 0,
|
||||
bytes: 0,
|
||||
errors: [],
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,253 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import {
|
||||
canonicalGreSnapshot,
|
||||
type GreLiveAddr,
|
||||
type GreLiveIface,
|
||||
type GreSnapshot,
|
||||
} from "./entity-snapshots.js"
|
||||
import { captureAndAppendRevision } from "./config-revisions.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
export interface RosGre {
|
||||
".id"?: string
|
||||
name?: string
|
||||
"local-address"?: string
|
||||
"remote-address"?: string
|
||||
"allow-fast-path"?: string
|
||||
"clamp-tcp-mss"?: string
|
||||
mtu?: string
|
||||
keepalive?: string
|
||||
dscp?: string
|
||||
running?: string
|
||||
disabled?: string
|
||||
comment?: string
|
||||
"ipsec-secret"?: string
|
||||
}
|
||||
|
||||
interface RosIpAddress {
|
||||
".id"?: string
|
||||
address?: string
|
||||
interface?: string
|
||||
disabled?: string
|
||||
network?: string
|
||||
}
|
||||
|
||||
export interface LiveGreTunnel {
|
||||
id: string
|
||||
rosId: string
|
||||
name: string
|
||||
serverId: string
|
||||
localAddress: string
|
||||
remoteAddress: string
|
||||
localInnerIp: string
|
||||
remoteInnerIp: string
|
||||
poolId: string
|
||||
ipsec: { secret: string } | null
|
||||
mtu: number
|
||||
keepaliveInterval: number
|
||||
keepaliveRetries: number
|
||||
dscp: "inherit" | number
|
||||
clampTcpMss: boolean
|
||||
allowFastPath: boolean
|
||||
comment: string
|
||||
enabled: boolean
|
||||
status: "up" | "down" | "degraded"
|
||||
}
|
||||
|
||||
export function parseKeepalive(value: string | undefined): { interval: number; retries: number } {
|
||||
if (!value || value.toLowerCase() === "none") return { interval: 0, retries: 0 }
|
||||
const [intervalRaw, retriesRaw] = value.split(",")
|
||||
const interval = Number.parseInt((intervalRaw ?? "").trim(), 10)
|
||||
const retries = Number.parseInt((retriesRaw ?? "").trim(), 10)
|
||||
return {
|
||||
interval: Number.isFinite(interval) ? interval : 0,
|
||||
retries: Number.isFinite(retries) ? retries : 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function formatKeepalive(interval: number, retries: number): string {
|
||||
if (!interval || interval <= 0) return "0"
|
||||
return `${interval}s,${retries > 0 ? retries : 10}`
|
||||
}
|
||||
|
||||
function parseDscp(value: string | undefined): "inherit" | number {
|
||||
if (!value || value === "inherit") return "inherit"
|
||||
const n = Number.parseInt(value, 10)
|
||||
return Number.isFinite(n) ? n : "inherit"
|
||||
}
|
||||
|
||||
function parseInnerFromComment(comment: string | undefined): { localInnerIp: string; remoteInnerIp: string } {
|
||||
if (!comment) return { localInnerIp: "", remoteInnerIp: "" }
|
||||
const local = comment.match(/address\s*=\s*([0-9.]+\/\d+)/)?.[1] ?? ""
|
||||
const remote = comment.match(/(?:network|gateway)\s*=\s*([0-9.]+\/\d+)/)?.[1] ?? ""
|
||||
return { localInnerIp: local, remoteInnerIp: remote }
|
||||
}
|
||||
|
||||
function rosDisabled(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
export function mapGreLive(
|
||||
server: ServerRow,
|
||||
greRaw: RosGre[],
|
||||
addrsRaw: RosIpAddress[],
|
||||
): {
|
||||
tunnels: LiveGreTunnel[]
|
||||
snapshot: GreSnapshot
|
||||
gre: GreLiveIface[]
|
||||
addrs: GreLiveAddr[]
|
||||
} {
|
||||
const addrsByIface = new Map<string, { address: string; rosId: string }[]>()
|
||||
for (const a of addrsRaw) {
|
||||
if (rosDisabled(a.disabled)) continue
|
||||
const iface = (a.interface ?? "").trim()
|
||||
const address = (a.address ?? "").trim()
|
||||
const rosId = String(a[".id"] ?? "")
|
||||
if (!iface || !address || !rosId) continue
|
||||
const list = addrsByIface.get(iface) ?? []
|
||||
list.push({ address, rosId })
|
||||
addrsByIface.set(iface, list)
|
||||
}
|
||||
|
||||
const gre: GreLiveIface[] = []
|
||||
const addrs: GreLiveAddr[] = []
|
||||
const tunnels: LiveGreTunnel[] = []
|
||||
|
||||
greRaw.forEach((g, idx) => {
|
||||
const rosId = String(g[".id"] ?? g.name ?? `gre-${idx}`)
|
||||
const name = (g.name ?? "").trim() || `gre-${idx + 1}`
|
||||
const keepalive = parseKeepalive(g.keepalive)
|
||||
const fromComment = parseInnerFromComment(g.comment)
|
||||
const ifaceAddrs = addrsByIface.get(name) ?? []
|
||||
const localInnerIp = ifaceAddrs[0]?.address || fromComment.localInnerIp
|
||||
const secret = (g["ipsec-secret"] ?? "").trim()
|
||||
const disabled = rosDisabled(g.disabled)
|
||||
const running = g.running === "true" || g.running === "yes"
|
||||
|
||||
gre.push({
|
||||
name,
|
||||
rosId,
|
||||
localAddress: g["local-address"] ?? "",
|
||||
remoteAddress: g["remote-address"] ?? "",
|
||||
comment: g.comment ?? "",
|
||||
disabled,
|
||||
mtu: Number.parseInt(g.mtu ?? "1476", 10) || 1476,
|
||||
keepalive: g.keepalive ?? "0",
|
||||
dscp: g.dscp ?? "inherit",
|
||||
clampTcpMss: g["clamp-tcp-mss"] !== "false" && g["clamp-tcp-mss"] !== "no",
|
||||
allowFastPath: g["allow-fast-path"] !== "false" && g["allow-fast-path"] !== "no",
|
||||
ipsecSecret: secret,
|
||||
})
|
||||
|
||||
for (const a of ifaceAddrs) {
|
||||
addrs.push({ rosId: a.rosId, interfaceName: name, address: a.address })
|
||||
}
|
||||
|
||||
tunnels.push({
|
||||
id: rosId || `${server.id}:${name}`,
|
||||
rosId,
|
||||
name,
|
||||
serverId: String(server.id),
|
||||
localAddress: g["local-address"] ?? "",
|
||||
remoteAddress: g["remote-address"] ?? "",
|
||||
localInnerIp,
|
||||
remoteInnerIp: fromComment.remoteInnerIp,
|
||||
poolId: "live",
|
||||
ipsec: secret ? { secret } : null,
|
||||
mtu: Number.parseInt(g.mtu ?? "1476", 10) || 1476,
|
||||
keepaliveInterval: keepalive.interval,
|
||||
keepaliveRetries: keepalive.retries,
|
||||
dscp: parseDscp(g.dscp),
|
||||
clampTcpMss: g["clamp-tcp-mss"] !== "false" && g["clamp-tcp-mss"] !== "no",
|
||||
allowFastPath: g["allow-fast-path"] !== "false" && g["allow-fast-path"] !== "no",
|
||||
comment: g.comment ?? "",
|
||||
enabled: !disabled,
|
||||
status: disabled ? "down" : running ? "up" : "degraded",
|
||||
})
|
||||
})
|
||||
|
||||
return {
|
||||
tunnels,
|
||||
snapshot: canonicalGreSnapshot({
|
||||
tunnels: gre.map((g) => ({
|
||||
name: g.name,
|
||||
localAddress: g.localAddress,
|
||||
remoteAddress: g.remoteAddress,
|
||||
localInnerIp: addrs.find((a) => a.interfaceName === g.name)?.address ?? "",
|
||||
remoteInnerIp: "",
|
||||
comment: g.comment,
|
||||
disabled: g.disabled,
|
||||
mtu: g.mtu,
|
||||
keepalive: g.keepalive,
|
||||
dscp: g.dscp,
|
||||
clampTcpMss: g.clampTcpMss,
|
||||
allowFastPath: g.allowFastPath,
|
||||
ipsecSecret: g.ipsecSecret,
|
||||
})),
|
||||
}),
|
||||
gre,
|
||||
addrs,
|
||||
}
|
||||
}
|
||||
|
||||
export async function fetchGreState(server: ServerRow) {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [greRaw, addrsRaw] = await Promise.all([
|
||||
client.get<RosGre[]>("/interface/gre"),
|
||||
client.get<RosIpAddress[]>("/ip/address").catch(() => [] as RosIpAddress[]),
|
||||
])
|
||||
return {
|
||||
client,
|
||||
...mapGreLive(server, Array.isArray(greRaw) ? greRaw : [], Array.isArray(addrsRaw) ? addrsRaw : []),
|
||||
}
|
||||
}
|
||||
|
||||
export async function captureGreSnapshot(server: ServerRow): Promise<GreSnapshot> {
|
||||
const state = await fetchGreState(server)
|
||||
return state.snapshot
|
||||
}
|
||||
|
||||
export async function listGreTunnels(opts?: { serverId?: string }): Promise<{
|
||||
tunnels: LiveGreTunnel[]
|
||||
failures: Array<{ serverId: string; serverName?: string; error: string }>
|
||||
}> {
|
||||
let serverRows: ServerRow[]
|
||||
if (opts?.serverId) {
|
||||
const id = Number.parseInt(String(opts.serverId), 10)
|
||||
if (!Number.isFinite(id)) {
|
||||
return { tunnels: [], failures: [{ serverId: String(opts.serverId), error: "Некорректный serverId" }] }
|
||||
}
|
||||
const row = (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0]
|
||||
serverRows = row ? [row] : []
|
||||
} else {
|
||||
serverRows = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
}
|
||||
|
||||
const failures: Array<{ serverId: string; serverName?: string; error: string }> = []
|
||||
const chunks = await Promise.all(
|
||||
serverRows.map(async (server) => {
|
||||
try {
|
||||
const state = await fetchGreState(server)
|
||||
await captureAndAppendRevision({
|
||||
serverId: server.id,
|
||||
section: "gre",
|
||||
source: "observed",
|
||||
capture: async () => state.snapshot,
|
||||
})
|
||||
return state.tunnels
|
||||
} catch (e) {
|
||||
failures.push({
|
||||
serverId: String(server.id),
|
||||
serverName: server.name ?? undefined,
|
||||
error: e instanceof Error ? e.message : String(e),
|
||||
})
|
||||
return [] as LiveGreTunnel[]
|
||||
}
|
||||
}),
|
||||
)
|
||||
return { tunnels: chunks.flat(), failures }
|
||||
}
|
||||
@@ -586,6 +586,14 @@ export class MikrotikClient {
|
||||
: new Error(`Не удалось загрузить файл ${normalized} на RouterOS`)
|
||||
}
|
||||
|
||||
async importUploadedFile(fileName: string): Promise<unknown> {
|
||||
try {
|
||||
return await this.post("/import", { "file-name": fileName }, 120_000)
|
||||
} catch {
|
||||
return await this.post("/execute", { script: `/import file-name="${fileName}"` }, 120_000)
|
||||
}
|
||||
}
|
||||
|
||||
async importCertificate(params: {
|
||||
fileName: string
|
||||
name: string
|
||||
|
||||
@@ -0,0 +1,22 @@
|
||||
import assert from "node:assert/strict"
|
||||
import type { RosIpRoute } from "../types/server.js"
|
||||
import { parseOspfGateway, parseOspfRouteType } from "./ospf-route-parse.js"
|
||||
|
||||
function route(partial: Partial<RosIpRoute>): RosIpRoute {
|
||||
return { ".id": "*1", "dst-address": "10.0.0.0/8", ...partial }
|
||||
}
|
||||
|
||||
assert.equal(parseOspfRouteType(route({ static: "true" })), null)
|
||||
assert.equal(parseOspfRouteType(route({ bgp: "true" })), null)
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true" })), "O")
|
||||
assert.equal(parseOspfRouteType(route({ "ospf-type": "intra-area" })), "O")
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "inter-area" })), "O IA")
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "ext-type-1" })), "O E1")
|
||||
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "type-2" })), "O E2")
|
||||
|
||||
assert.deepEqual(parseOspfGateway(route({ gateway: "10.200.0.1%gre-msk-spb" })), {
|
||||
nextHop: "10.200.0.1",
|
||||
via: "gre-msk-spb",
|
||||
})
|
||||
|
||||
console.log("ospf-route-parse.test.ts: ok")
|
||||
@@ -0,0 +1,28 @@
|
||||
import type { RosIpRoute } from "../types/server.js"
|
||||
|
||||
export type OspfRouteKind = "O" | "O IA" | "O E1" | "O E2"
|
||||
|
||||
/** RouterOS /ip/route → тип OSPF-маршрута UI, либо null если маршрут не OSPF. */
|
||||
export function parseOspfRouteType(r: RosIpRoute): OspfRouteKind | null {
|
||||
const ospfFlag = r.ospf === "true" || r.ospf === "yes"
|
||||
const raw = `${r["ospf-type"] ?? ""} ${r.type ?? ""}`.toLowerCase()
|
||||
const looksOspf = ospfFlag || raw.includes("ospf") || Boolean(r["ospf-type"])
|
||||
if (!looksOspf) return null
|
||||
if (raw.includes("inter")) return "O IA"
|
||||
if (raw.includes("e1") || raw.includes("type-1") || raw.includes("ext-1") || raw.includes("nssa-ext-type-1")) {
|
||||
return "O E1"
|
||||
}
|
||||
if (raw.includes("e2") || raw.includes("type-2") || raw.includes("ext-2") || raw.includes("nssa-ext-type-2")) {
|
||||
return "O E2"
|
||||
}
|
||||
return "O"
|
||||
}
|
||||
|
||||
export function parseOspfGateway(r: RosIpRoute): { nextHop: string; via: string } {
|
||||
const gw = (r.gateway ?? r["immediate-gw"] ?? "").trim()
|
||||
const [ip, iface = ""] = gw.split("%")
|
||||
return {
|
||||
nextHop: ip || gw || "—",
|
||||
via: iface || (r.interface ?? "—"),
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,32 @@
|
||||
import type { MikrotikClient } from "./mikrotik.js"
|
||||
import type { RosWriteOp } from "./entity-snapshots.js"
|
||||
|
||||
function encodeIdSegment(path: string): string {
|
||||
const i = path.lastIndexOf("/")
|
||||
if (i < 0) return path
|
||||
const last = path.slice(i + 1)
|
||||
if (!last.startsWith("*")) return path
|
||||
return `${path.slice(0, i + 1)}${encodeURIComponent(last)}`
|
||||
}
|
||||
|
||||
export async function executeRosOps(client: MikrotikClient, ops: RosWriteOp[]): Promise<void> {
|
||||
for (const op of ops) {
|
||||
if (op.op === "put") {
|
||||
await client.put(op.path, op.body)
|
||||
continue
|
||||
}
|
||||
if (op.op === "post") {
|
||||
await client.post(encodeIdSegment(op.path), op.body)
|
||||
continue
|
||||
}
|
||||
if (op.op === "patch") {
|
||||
await client.patch(encodeIdSegment(op.path), op.body)
|
||||
continue
|
||||
}
|
||||
if (op.op === "delete") {
|
||||
await client.delete(encodeIdSegment(op.path))
|
||||
continue
|
||||
}
|
||||
await client.post(encodeIdSegment(op.path), op.body)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,89 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadBucketCommand,
|
||||
ListObjectsV2Command,
|
||||
PutObjectCommand,
|
||||
type S3Client,
|
||||
} from "@aws-sdk/client-s3"
|
||||
import {
|
||||
buildS3ObjectKey,
|
||||
normalizeS3Prefix,
|
||||
parseS3ObjectKey,
|
||||
sanitizeServerName,
|
||||
s3DeleteObject,
|
||||
s3GetObject,
|
||||
s3ListObjects,
|
||||
s3PutObject,
|
||||
s3TestConnection,
|
||||
} from "./s3-backup-client.js"
|
||||
|
||||
assert.equal(normalizeS3Prefix("/mikrotik/backups/"), "mikrotik/backups")
|
||||
assert.equal(sanitizeServerName("MSK CHR 01"), "MSK_CHR_01")
|
||||
assert.equal(
|
||||
buildS3ObjectKey("mikrotik", "msk-chr01", "chr_2026-09-08_03-00-00.rsc"),
|
||||
"mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc",
|
||||
)
|
||||
assert.deepEqual(
|
||||
parseS3ObjectKey("mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc"),
|
||||
{ filename: "chr_2026-09-08_03-00-00.rsc", serverName: "msk-chr01" },
|
||||
)
|
||||
|
||||
const store = new Map<string, Buffer>()
|
||||
let lastCommand = ""
|
||||
|
||||
const fake = {
|
||||
send: async (command: { input?: Record<string, unknown> }) => {
|
||||
const name = command.constructor.name
|
||||
lastCommand = name
|
||||
const input = command.input ?? {}
|
||||
if (command instanceof HeadBucketCommand || name === "HeadBucketCommand") {
|
||||
if (input.Bucket !== "backups") throw new Error("no bucket")
|
||||
return {}
|
||||
}
|
||||
if (command instanceof PutObjectCommand || name === "PutObjectCommand") {
|
||||
const key = String(input.Key)
|
||||
const body = input.Body
|
||||
store.set(key, Buffer.isBuffer(body) ? body : Buffer.from(String(body)))
|
||||
return { ETag: '"etag-1"' }
|
||||
}
|
||||
if (command instanceof GetObjectCommand || name === "GetObjectCommand") {
|
||||
const key = String(input.Key)
|
||||
const body = store.get(key)
|
||||
if (!body) throw new Error("not found")
|
||||
return { Body: { transformToByteArray: async () => new Uint8Array(body) } }
|
||||
}
|
||||
if (command instanceof DeleteObjectCommand || name === "DeleteObjectCommand") {
|
||||
store.delete(String(input.Key))
|
||||
return {}
|
||||
}
|
||||
if (command instanceof ListObjectsV2Command || name === "ListObjectsV2Command") {
|
||||
const prefix = String(input.Prefix ?? "")
|
||||
const contents = [...store.entries()]
|
||||
.filter(([key]) => !prefix || key.startsWith(prefix))
|
||||
.map(([key, buf]) => ({ Key: key, Size: buf.length, LastModified: new Date("2026-09-08T00:00:00Z") }))
|
||||
return { Contents: contents, IsTruncated: false }
|
||||
}
|
||||
throw new Error(`unexpected command ${name}`)
|
||||
},
|
||||
} as unknown as S3Client
|
||||
|
||||
await s3TestConnection(fake, "backups")
|
||||
assert.equal(lastCommand === "HeadBucketCommand" || lastCommand.includes("Head"), true)
|
||||
|
||||
const put = await s3PutObject(fake, "backups", "mikrotik/a/file.rsc", "hello")
|
||||
assert.equal(put.etag, '"etag-1"')
|
||||
|
||||
const got = await s3GetObject(fake, "backups", "mikrotik/a/file.rsc")
|
||||
assert.equal(got.toString("utf8"), "hello")
|
||||
|
||||
const listed = await s3ListObjects(fake, "backups", "mikrotik")
|
||||
assert.equal(listed.length, 1)
|
||||
assert.equal(listed[0]?.key, "mikrotik/a/file.rsc")
|
||||
|
||||
await s3DeleteObject(fake, "backups", "mikrotik/a/file.rsc")
|
||||
const after = await s3ListObjects(fake, "backups", "mikrotik")
|
||||
assert.equal(after.length, 0)
|
||||
|
||||
console.log("s3-backup-client.test.ts: ok")
|
||||
@@ -0,0 +1,128 @@
|
||||
import {
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadBucketCommand,
|
||||
ListObjectsV2Command,
|
||||
PutObjectCommand,
|
||||
S3Client,
|
||||
type S3ClientConfig,
|
||||
} from "@aws-sdk/client-s3"
|
||||
|
||||
export type S3BackupConfig = {
|
||||
endpoint: string
|
||||
region: string
|
||||
bucket: string
|
||||
prefix: string
|
||||
accessKeyId: string
|
||||
secretAccessKey: string
|
||||
forcePathStyle: boolean
|
||||
}
|
||||
|
||||
export type S3ListedObject = {
|
||||
key: string
|
||||
size: number
|
||||
lastModified?: string
|
||||
}
|
||||
|
||||
export function normalizeS3Prefix(prefix: string): string {
|
||||
return prefix.trim().replace(/^\/+|\/+$/g, "")
|
||||
}
|
||||
|
||||
export function sanitizeServerName(name: string): string {
|
||||
const safe = name.replace(/[^a-zA-Z0-9._-]+/g, "_").replace(/^_+|_+$/g, "")
|
||||
return safe || "server"
|
||||
}
|
||||
|
||||
export function buildS3ObjectKey(prefix: string, serverSafe: string, filename: string): string {
|
||||
const parts = [normalizeS3Prefix(prefix), sanitizeServerName(serverSafe), filename]
|
||||
.filter((part) => part.length > 0)
|
||||
return parts.join("/")
|
||||
}
|
||||
|
||||
export function parseS3ObjectKey(key: string): { filename: string; serverName: string } {
|
||||
const parts = key.split("/").filter(Boolean)
|
||||
const filename = parts.pop() ?? key
|
||||
const folder = parts.pop() ?? ""
|
||||
const base = filename.replace(/\.(rsc|backup)$/i, "")
|
||||
const fromFilename = base.replace(/_\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}$/, "")
|
||||
return { filename, serverName: folder || fromFilename || filename }
|
||||
}
|
||||
|
||||
export function createS3ClientFromConfig(cfg: S3BackupConfig): S3Client {
|
||||
const options: S3ClientConfig = {
|
||||
region: cfg.region.trim() || "us-east-1",
|
||||
credentials: {
|
||||
accessKeyId: cfg.accessKeyId,
|
||||
secretAccessKey: cfg.secretAccessKey,
|
||||
},
|
||||
forcePathStyle: cfg.forcePathStyle,
|
||||
}
|
||||
const endpoint = cfg.endpoint.trim()
|
||||
if (endpoint) options.endpoint = endpoint
|
||||
return new S3Client(options)
|
||||
}
|
||||
|
||||
export async function s3TestConnection(client: S3Client, bucket: string): Promise<void> {
|
||||
try {
|
||||
await client.send(new HeadBucketCommand({ Bucket: bucket }))
|
||||
} catch {
|
||||
await client.send(new ListObjectsV2Command({ Bucket: bucket, MaxKeys: 1 }))
|
||||
}
|
||||
}
|
||||
|
||||
export async function s3PutObject(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
key: string,
|
||||
body: Buffer | string,
|
||||
): Promise<{ etag?: string }> {
|
||||
const out = await client.send(new PutObjectCommand({
|
||||
Bucket: bucket,
|
||||
Key: key,
|
||||
Body: body,
|
||||
ContentType: "text/plain; charset=utf-8",
|
||||
}))
|
||||
return { etag: out.ETag }
|
||||
}
|
||||
|
||||
export async function s3GetObject(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
key: string,
|
||||
): Promise<Buffer> {
|
||||
const out = await client.send(new GetObjectCommand({ Bucket: bucket, Key: key }))
|
||||
const bytes = await out.Body?.transformToByteArray()
|
||||
if (!bytes) throw new Error("Пустой объект S3")
|
||||
return Buffer.from(bytes)
|
||||
}
|
||||
|
||||
export async function s3DeleteObject(client: S3Client, bucket: string, key: string): Promise<void> {
|
||||
await client.send(new DeleteObjectCommand({ Bucket: bucket, Key: key }))
|
||||
}
|
||||
|
||||
export async function s3ListObjects(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
prefix: string,
|
||||
): Promise<S3ListedObject[]> {
|
||||
const items: S3ListedObject[] = []
|
||||
let token: string | undefined
|
||||
const normalized = normalizeS3Prefix(prefix)
|
||||
do {
|
||||
const out = await client.send(new ListObjectsV2Command({
|
||||
Bucket: bucket,
|
||||
Prefix: normalized ? `${normalized}/` : undefined,
|
||||
ContinuationToken: token,
|
||||
}))
|
||||
for (const obj of out.Contents ?? []) {
|
||||
if (!obj.Key) continue
|
||||
items.push({
|
||||
key: obj.Key,
|
||||
size: obj.Size ?? 0,
|
||||
lastModified: obj.LastModified?.toISOString(),
|
||||
})
|
||||
}
|
||||
token = out.IsTruncated ? out.NextContinuationToken : undefined
|
||||
} while (token)
|
||||
return items
|
||||
}
|
||||
@@ -46,6 +46,8 @@ import { collectCertificatesRenewOnce } from "./certificate-renew-collector.js"
|
||||
import { getCertificateRenewSettings } from "./certificates-service.js"
|
||||
import { collectScheduledBackupsOnce } from "./backup-scheduler-collector.js"
|
||||
import { getBackupScheduleSettings } from "./backup-service.js"
|
||||
import { collectGeoipUpdateOnce } from "./geoip-update-collector.js"
|
||||
import { getGeoipSettings } from "./geoip-settings.js"
|
||||
import {
|
||||
endSchedulerJob,
|
||||
isSchedulerJobRunning,
|
||||
@@ -63,6 +65,7 @@ export const JOB_KEYS = [
|
||||
"gre_bgp",
|
||||
"certificates_renew",
|
||||
"backups",
|
||||
"geoip_update",
|
||||
"alert_engine",
|
||||
] as const
|
||||
export type SchedulerJobKey = (typeof JOB_KEYS)[number]
|
||||
@@ -148,6 +151,9 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
case "backups":
|
||||
snapshot = await collectScheduledBackupsOnce()
|
||||
break
|
||||
case "geoip_update":
|
||||
snapshot = await collectGeoipUpdateOnce()
|
||||
break
|
||||
case "alert_engine": {
|
||||
const r = await runAlertEngineOnce()
|
||||
snapshot = {
|
||||
@@ -377,6 +383,18 @@ export async function refreshScheduler(): Promise<void> {
|
||||
)
|
||||
}
|
||||
|
||||
const geoip = await getGeoipSettings()
|
||||
if (geoip.enabled) {
|
||||
const geoipMs = Math.max(6 * 3600_000, geoip.updateIntervalSec * 1000)
|
||||
void executeSchedulerJob("geoip_update").catch(() => {})
|
||||
timers.set(
|
||||
"geoip_update",
|
||||
setInterval(() => {
|
||||
void executeSchedulerJob("geoip_update").catch(() => {})
|
||||
}, geoipMs),
|
||||
)
|
||||
}
|
||||
|
||||
const alertMs = 20_000
|
||||
void executeSchedulerJob("alert_engine").catch(() => {})
|
||||
timers.set(
|
||||
@@ -409,6 +427,7 @@ export async function getSchedulerStatus() {
|
||||
const internetPath = await getInternetPathSettings()
|
||||
const certRenew = await getCertificateRenewSettings()
|
||||
const backupSchedule = await getBackupScheduleSettings()
|
||||
const geoip = await getGeoipSettings()
|
||||
|
||||
const resOn = uptime.resourcesEnabled ?? uptime.enabled
|
||||
const pingOn = uptime.pingEnabled ?? uptime.enabled
|
||||
@@ -424,6 +443,7 @@ export async function getSchedulerStatus() {
|
||||
gre_bgp: { enabled: true, intervalSec: 30 },
|
||||
certificates_renew: { enabled: certRenew.enabled, intervalSec: certRenew.intervalSec },
|
||||
backups: { enabled: backupSchedule.enabled, intervalSec: 60 },
|
||||
geoip_update: { enabled: geoip.enabled, intervalSec: geoip.updateIntervalSec },
|
||||
alert_engine: { enabled: true, intervalSec: 20 },
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { getStatistics, getStatisticsPivot, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
|
||||
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { ensurePartitionFor } from "../db/partitions.js"
|
||||
import { pool } from "../db/index.js"
|
||||
import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
|
||||
|
||||
{
|
||||
const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10")
|
||||
assert.ok(sameDay)
|
||||
assert.equal(sameDay.fromDay, "2026-09-10")
|
||||
assert.equal(sameDay.toDayExclusive, "2026-09-11")
|
||||
assert.equal(sameDay.grain, "hour")
|
||||
const month = parseStatisticsPeriod("2026-08-01", "2026-08-31")
|
||||
assert.ok(month)
|
||||
assert.equal(month.grain, "day")
|
||||
assert.equal(month.toDayExclusive, "2026-09-01")
|
||||
assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null)
|
||||
assert.equal(pivotDimsConflict("country", "country"), true)
|
||||
assert.equal(pivotDimsConflict("country", "service"), false)
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("statistics-aggregate.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const inserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type, wan_uplinks)
|
||||
VALUES ('stats-cube', '127.0.0.1', 'jump-host', '[{"iface":"wan1"}]'::jsonb)
|
||||
RETURNING id
|
||||
`)
|
||||
const serverId = inserted.rows[0]?.id
|
||||
if (serverId == null) throw new Error("no server")
|
||||
|
||||
const enInserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type)
|
||||
VALUES ('stats-en', '198.51.100.1', 'exit-node')
|
||||
RETURNING id
|
||||
`)
|
||||
const enId = enInserted.rows[0]?.id
|
||||
if (enId == null) throw new Error("no en server")
|
||||
|
||||
await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z"))
|
||||
await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z"))
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login, role, active)
|
||||
VALUES ('u-stats-1', 'Клиент', 'stats-user', 'viewer', TRUE)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-stats-1', 'u-stats-1', $1, 'gre-client', 'gre')
|
||||
`, [serverId])
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
|
||||
VALUES
|
||||
($1, '2026-09-10T12:00:00Z', 'online', $3::jsonb),
|
||||
($2, '2026-09-10T12:00:00Z', 'online', $4::jsonb)
|
||||
`, [
|
||||
serverId,
|
||||
enId,
|
||||
JSON.stringify([
|
||||
{ name: "gre-client", type: "gre-tunnel" },
|
||||
{ name: "wan1", type: "ether" },
|
||||
{ name: "gre-en", type: "gre-tunnel" },
|
||||
{ name: "NSK-SERVHOST-RTK", type: "gre-tunnel" },
|
||||
{ name: "wg-mesh", type: "wg" },
|
||||
{ name: "wg-server", type: "wg" },
|
||||
{ name: "wg-flow", type: "wg" },
|
||||
]),
|
||||
JSON.stringify([
|
||||
{ name: "ether1", type: "ether" },
|
||||
{ name: "gre-jh", type: "gre-tunnel" },
|
||||
]),
|
||||
])
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
rememberServerIfaces(serverId, [
|
||||
{ name: "gre-client", ifindex: "2" },
|
||||
{ name: "wan1", ifindex: "8" },
|
||||
{ name: "gre-en", ifindex: "9" },
|
||||
{ name: "NSK-SERVHOST-RTK" },
|
||||
{ name: "wg-mesh" },
|
||||
{ name: "wg-server" },
|
||||
{ name: "wg-flow" },
|
||||
])
|
||||
rememberServerIfaces(enId, [
|
||||
{ name: "ether1", ifindex: "2" },
|
||||
{ name: "gre-jh", ifindex: "5" },
|
||||
])
|
||||
setRefreshIfacesForTests(async () => {})
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
|
||||
VALUES
|
||||
($1, '2026-09-10', '2', 'US', 'https', 15169, 800, 10),
|
||||
($1, '2026-09-10', '2', 'DE', 'dns', 15133, 200, 4),
|
||||
($1, '2026-09-10', 'wan1', 'NL', 'other', 0, 70, 1),
|
||||
($1, '2026-09-10', '0', 'US', 'https', 0, 999, 3),
|
||||
($1, '2026-09-10', 'gre-en', 'US', 'https', 15169, 400, 2),
|
||||
($1, '2026-09-10', 'NSK-SERVHOST-RTK', 'US', 'https', 15169, 300, 2),
|
||||
($1, '2026-09-10', 'wg-mesh', 'US', 'https', 0, 250, 2),
|
||||
($1, '2026-09-10', 'wg-flow', 'US', 'https', 0, 80, 1),
|
||||
($2, '2026-09-10', 'gre-jh', 'US', 'https', 15169, 500, 5),
|
||||
($2, '2026-09-10', 'ether1', 'US', 'https', 15169, 200, 2)
|
||||
`, [serverId, enId])
|
||||
|
||||
try {
|
||||
const unique = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
|
||||
assert.equal(unique.grain, "day")
|
||||
assert.equal(unique.kpis.bytes, 1000)
|
||||
const uniqueAsnSum = unique.asns.reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(uniqueAsnSum, unique.kpis.bytes, "unique KPI = SUM dest ASN")
|
||||
assert.equal(unique.kpis.users, 1)
|
||||
assert.ok(unique.countries.some((r) => r.id === "US"))
|
||||
assert.ok(unique.users.some((r) => r.id === "u-stats-1"))
|
||||
assert.equal(unique.users.find((r) => r.id === STATISTICS_UNBOUND_USER_ID), undefined)
|
||||
assert.ok(unique.servers.some((r) => r.id === String(serverId)))
|
||||
assert.ok(!unique.servers.some((r) => r.id === String(enId)), "EN-транзит не в сетевом KPI")
|
||||
const greIface = unique.interfaces.find((r) => r.label.includes("gre-client"))
|
||||
assert.ok(greIface)
|
||||
assert.equal(greIface.bytes, 1000)
|
||||
assert.equal(greIface.id, `${serverId}:gre-client`)
|
||||
assert.ok(!unique.interfaces.some((r) => /· (?:#)?\d+$/.test(r.label)))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes(" · —") || r.label.endsWith("· —")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("gre-en")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("NSK-SERVHOST-RTK")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-mesh")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
assert.equal(unique.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined, "unique без WAN")
|
||||
|
||||
const allPlanes = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "all" })
|
||||
assert.equal(allPlanes.kpis.bytes, 1000, "KPI unique и all одинаковый")
|
||||
const wanRow = allPlanes.interfaces.find((r) => r.id === `${serverId}:wan1`)
|
||||
assert.ok(wanRow)
|
||||
assert.ok(wanRow.label.includes("WAN · интернет"))
|
||||
assert.equal(wanRow.bytes, 70)
|
||||
assert.equal(wanRow.percent, 0)
|
||||
const overlayGre = allPlanes.interfaces.find((r) => r.id === `${serverId}:gre-en`)
|
||||
assert.ok(overlayGre)
|
||||
assert.ok(overlayGre.label.includes("дубль"))
|
||||
assert.equal(overlayGre.percent, 0)
|
||||
const overlayCustom = allPlanes.interfaces.find((r) => r.label.includes("NSK-SERVHOST-RTK"))
|
||||
assert.ok(overlayCustom)
|
||||
assert.ok(overlayCustom.label.includes("дубль"))
|
||||
const overlayWg = allPlanes.interfaces.find((r) => r.label.includes("wg-mesh"))
|
||||
assert.ok(overlayWg)
|
||||
assert.ok(overlayWg.label.includes("дубль"))
|
||||
assert.ok(!allPlanes.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
|
||||
const wanSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
iface: "wan1",
|
||||
})
|
||||
assert.equal(wanSlice.kpis.bytes, 70)
|
||||
|
||||
const nodeSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
planes: "unique",
|
||||
})
|
||||
assert.equal(nodeSlice.kpis.bytes, 1000)
|
||||
assert.equal(nodeSlice.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined)
|
||||
assert.ok(!nodeSlice.users.some((r) => r.id === STATISTICS_UNBOUND_USER_ID))
|
||||
|
||||
const nodeAll = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
planes: "all",
|
||||
})
|
||||
assert.equal(nodeAll.kpis.bytes, 1000)
|
||||
const nodeWan = nodeAll.interfaces.find((r) => r.id === `${serverId}:wan1`)
|
||||
assert.ok(nodeWan)
|
||||
assert.equal(nodeWan.percent, 0)
|
||||
assert.ok(nodeWan.label.includes("WAN · интернет"))
|
||||
|
||||
const enSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId: enId,
|
||||
planes: "unique",
|
||||
})
|
||||
assert.equal(enSlice.kpis.bytes, 0)
|
||||
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("gre-jh")))
|
||||
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("WAN · интернет")))
|
||||
|
||||
const enAll = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId: enId,
|
||||
planes: "all",
|
||||
})
|
||||
assert.equal(enAll.kpis.bytes, 0)
|
||||
assert.ok(enAll.interfaces.some((r) => r.label.includes("WAN · интернет") && r.label.includes("ether1") && r.percent === 0))
|
||||
assert.ok(enAll.interfaces.some((r) => r.label.includes("gre-jh") && r.label.includes("дубль")))
|
||||
|
||||
const sliced = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
})
|
||||
assert.equal(sliced.kpis.bytes, 800)
|
||||
assert.equal(sliced.countries.length, 1)
|
||||
assert.equal(sliced.countries[0]?.id, "US")
|
||||
assert.ok(sliced.users.some((r) => r.id === "u-stats-1"))
|
||||
|
||||
const byUser = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
userId: "u-stats-1",
|
||||
})
|
||||
assert.equal(byUser.kpis.bytes, 1000)
|
||||
|
||||
const pivot = await getStatisticsPivot({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
row: "country",
|
||||
col: "service",
|
||||
metric: "bytes",
|
||||
})
|
||||
const us = pivot.rows.find((r) => r.id === "US")
|
||||
const de = pivot.rows.find((r) => r.id === "DE")
|
||||
assert.ok(us)
|
||||
assert.ok(de)
|
||||
assert.equal(us.cells.https, 800)
|
||||
assert.equal(de.cells.dns, 200)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-stats-wg', 'u-stats-1', $1, 'wg-server', 'wg')
|
||||
`, [serverId])
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
|
||||
VALUES ($1, '2026-09-10', 'wg-server', 'US', 'https', 15169, 150, 2)
|
||||
`, [serverId])
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
const withWg = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
|
||||
assert.equal(withWg.kpis.bytes, 1150)
|
||||
assert.ok(withWg.interfaces.some((r) => r.label.includes("wg-server") && r.bytes === 150))
|
||||
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-mesh")))
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
VALUES ($1, '2026-09-10T10:00:00Z', '2', 'US', 'https', 15169, 40, 2)
|
||||
`, [serverId])
|
||||
const hourly = await getStatistics({
|
||||
from: "2026-09-10T00:00:00.000Z",
|
||||
to: "2026-09-10T23:00:00.000Z",
|
||||
})
|
||||
assert.equal(hourly.grain, "hour")
|
||||
assert.equal(hourly.kpis.bytes, 40)
|
||||
assert.ok(hourly.users.some((r) => r.id === "u-stats-1"))
|
||||
} finally {
|
||||
setRefreshIfacesForTests(null)
|
||||
resetIfaceCacheForTests()
|
||||
invalidateFlowCatalogCache()
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM servers WHERE id IN ($1, $2)`, [serverId, enId])
|
||||
}
|
||||
|
||||
console.log("statistics-aggregate.test.ts: ok")
|
||||
@@ -0,0 +1,885 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import {
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsBreakdownRow,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDim,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsPivotQuery,
|
||||
type StatisticsQuery,
|
||||
} from "@mmapp/contracts/statistics"
|
||||
import {
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
listCachedIfaceNames,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
import { refreshServerIfaces } from "./traffic-flow-ifaces.js"
|
||||
import {
|
||||
isDashDisplayIface,
|
||||
isJunkFactIface,
|
||||
isOverlayTunnelIface,
|
||||
isWanFactIface,
|
||||
overlayDupLabel,
|
||||
wanIfaceLabel,
|
||||
} from "./traffic-flow-facts-filter.js"
|
||||
import { getServerCatalog, loadFlowTopology, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
const TOP_N = 200
|
||||
const HOUR_WINDOW_MS = 48 * 3600_000
|
||||
const PIVOT_ROW_CAP = 50
|
||||
const PIVOT_COL_CAP = 15
|
||||
const PIVOT_OTHER_ID = "__other__"
|
||||
|
||||
export interface ParsedPeriod {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
grain: "hour" | "day"
|
||||
windowSec: number
|
||||
}
|
||||
|
||||
function pad2(n: number): string {
|
||||
return String(n).padStart(2, "0")
|
||||
}
|
||||
|
||||
function toUtcDay(d: Date): string {
|
||||
return `${d.getUTCFullYear()}-${pad2(d.getUTCMonth() + 1)}-${pad2(d.getUTCDate())}`
|
||||
}
|
||||
|
||||
function addUtcDays(day: string, n: number): string {
|
||||
const d = new Date(`${day}T00:00:00Z`)
|
||||
d.setUTCDate(d.getUTCDate() + n)
|
||||
return toUtcDay(d)
|
||||
}
|
||||
|
||||
/** Parse from/to. Date-only `to` is inclusive (end of that UTC day). */
|
||||
export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPeriod | null {
|
||||
const from = Date.parse(fromRaw.includes("T") ? fromRaw : `${fromRaw}T00:00:00Z`)
|
||||
const toHasTime = toRaw.includes("T")
|
||||
const to = Date.parse(toHasTime ? toRaw : `${toRaw}T00:00:00Z`)
|
||||
if (!Number.isFinite(from) || !Number.isFinite(to)) return null
|
||||
const fromDate = new Date(from)
|
||||
let toDate = new Date(to)
|
||||
let toDayExclusive: string
|
||||
if (toHasTime) {
|
||||
toDayExclusive = toUtcDay(toDate)
|
||||
if (toDate.getUTCHours() !== 0 || toDate.getUTCMinutes() !== 0 || toDate.getUTCSeconds() !== 0) {
|
||||
toDayExclusive = addUtcDays(toDayExclusive, 1)
|
||||
}
|
||||
} else {
|
||||
toDayExclusive = addUtcDays(toUtcDay(toDate), 1)
|
||||
toDate = new Date(`${toDayExclusive}T00:00:00Z`)
|
||||
}
|
||||
if (toDate.getTime() <= from) return null
|
||||
const windowSec = Math.max(1, Math.round((toDate.getTime() - from) / 1000))
|
||||
const grain: "hour" | "day" = toDate.getTime() - from <= HOUR_WINDOW_MS ? "hour" : "day"
|
||||
return {
|
||||
fromIso: fromDate.toISOString(),
|
||||
toIso: toDate.toISOString(),
|
||||
fromDay: toUtcDay(fromDate),
|
||||
toDayExclusive,
|
||||
grain,
|
||||
windowSec,
|
||||
}
|
||||
}
|
||||
|
||||
type FactScope = "unique" | "wan" | "overlay"
|
||||
|
||||
interface FilterCtx {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
serverId?: number
|
||||
iface?: string
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: number
|
||||
planes: "unique" | "all"
|
||||
userIfaces: Array<{ serverId: number; iface: string }> | null
|
||||
unboundOnly: boolean
|
||||
boundIfaces: Array<{ serverId: number; iface: string }>
|
||||
overlayIfaces: Array<{ serverId: number; iface: string }>
|
||||
wanIfaces: Array<{ serverId: number; iface: string }>
|
||||
excludeServerIds: number[]
|
||||
topo: FlowTopology | null
|
||||
}
|
||||
|
||||
function ifaceFilterAliases(iface: string, serverId?: number): string[] {
|
||||
return factIfaceAliases(iface.trim(), serverId)
|
||||
}
|
||||
|
||||
function looksLikeIfIndex(iface: string): boolean {
|
||||
const raw = iface.trim()
|
||||
return /^\d+$/.test(raw) || /^#\d+$/.test(raw)
|
||||
}
|
||||
|
||||
async function warmIfaceCache(ids: Iterable<number>): Promise<void> {
|
||||
const uniq = [...new Set(ids)].filter((id) => Number.isFinite(id) && id > 0)
|
||||
if (!uniq.length) return
|
||||
await Promise.all(uniq.map((id) => refreshServerIfaces(id)))
|
||||
}
|
||||
|
||||
async function warmBindingIfaceCache(): Promise<void> {
|
||||
const rows = await db.select({ serverId: userInterfaceBindings.serverId }).from(userInterfaceBindings)
|
||||
await warmIfaceCache(rows.map((r) => r.serverId))
|
||||
}
|
||||
|
||||
function canonicalIfaceDimId(id: string): string {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (!Number.isFinite(sid)) return id
|
||||
return `${sid}:${displayFactIface(sid, id.slice(colon + 1))}`
|
||||
}
|
||||
|
||||
function pushIfaceTuples(
|
||||
parts: string[],
|
||||
params: unknown[],
|
||||
alias: string,
|
||||
tuples: Array<{ serverId: number; iface: string }>,
|
||||
op: "IN" | "NOT IN",
|
||||
): void {
|
||||
if (!tuples.length) {
|
||||
if (op === "IN") parts.push("FALSE")
|
||||
return
|
||||
}
|
||||
const sql = tuples.map(() => "(?, ?)").join(", ")
|
||||
parts.push(`(${alias}.server_id, ${alias}.iface) ${op} (${sql})`)
|
||||
for (const t of tuples) {
|
||||
params.push(t.serverId, t.iface)
|
||||
}
|
||||
}
|
||||
|
||||
function factWhere(
|
||||
alias: string,
|
||||
grain: "hour" | "day",
|
||||
ctx: FilterCtx,
|
||||
scope: FactScope = "unique",
|
||||
): { sql: string; params: unknown[] } {
|
||||
const params: unknown[] = []
|
||||
const parts: string[] = []
|
||||
if (grain === "hour") {
|
||||
params.push(ctx.fromIso, ctx.toIso)
|
||||
parts.push(`${alias}.bucket_at >= ? AND ${alias}.bucket_at < ?`)
|
||||
} else {
|
||||
params.push(ctx.fromDay, ctx.toDayExclusive)
|
||||
parts.push(`${alias}.day >= ? AND ${alias}.day < ?`)
|
||||
}
|
||||
if (ctx.serverId != null) {
|
||||
parts.push(`${alias}.server_id = ?`)
|
||||
params.push(ctx.serverId)
|
||||
}
|
||||
if (ctx.country) {
|
||||
parts.push(`${alias}.country = ?`)
|
||||
params.push(ctx.country.toUpperCase())
|
||||
}
|
||||
if (ctx.service) {
|
||||
parts.push(`${alias}.service = ?`)
|
||||
params.push(ctx.service)
|
||||
}
|
||||
if (ctx.asn != null) {
|
||||
parts.push(`${alias}.asn = ?`)
|
||||
params.push(ctx.asn)
|
||||
}
|
||||
parts.push(`${alias}.iface NOT IN ('0', '—', '__unknown__', 'wg-flow', '')`)
|
||||
|
||||
if (scope === "wan") {
|
||||
pushIfaceTuples(parts, params, alias, ctx.wanIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (scope === "overlay") {
|
||||
pushIfaceTuples(parts, params, alias, ctx.overlayIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
|
||||
if (ctx.iface) {
|
||||
const aliases = ifaceFilterAliases(ctx.iface, ctx.serverId)
|
||||
if (aliases.length <= 1) {
|
||||
parts.push(`${alias}.iface = ?`)
|
||||
params.push(aliases[0] ?? ctx.iface)
|
||||
} else {
|
||||
parts.push(`${alias}.iface IN (${aliases.map(() => "?").join(", ")})`)
|
||||
params.push(...aliases)
|
||||
}
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (ctx.userIfaces) {
|
||||
pushIfaceTuples(parts, params, alias, ctx.userIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (ctx.unboundOnly) {
|
||||
parts.push("FALSE")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
pushIfaceTuples(parts, params, alias, ctx.boundIfaces, "IN")
|
||||
if (ctx.excludeServerIds.length) {
|
||||
parts.push(`${alias}.server_id NOT IN (${ctx.excludeServerIds.map(() => "?").join(", ")})`)
|
||||
params.push(...ctx.excludeServerIds)
|
||||
}
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
|
||||
function emptyDto(period: ParsedPeriod): StatisticsDto {
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "—",
|
||||
topService: "—",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
}
|
||||
|
||||
function toBreakdown(
|
||||
rows: Array<{ id: string; label: string; bytes: number; packets: number }>,
|
||||
totalBytes: number,
|
||||
windowSec: number,
|
||||
): StatisticsBreakdownRow[] {
|
||||
const denom = totalBytes || 1
|
||||
return rows
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
.slice(0, TOP_N)
|
||||
.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.label,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
bps: (r.bytes * 8) / windowSec,
|
||||
percent: (r.bytes / denom) * 100,
|
||||
}))
|
||||
}
|
||||
|
||||
interface UserBindTuple {
|
||||
userId: string
|
||||
serverId: number
|
||||
iface: string
|
||||
}
|
||||
|
||||
async function loadBindUserTuples(): Promise<UserBindTuple[]> {
|
||||
const binds = await db.select().from(userInterfaceBindings)
|
||||
const seen = new Set<string>()
|
||||
const out: UserBindTuple[] = []
|
||||
for (const b of binds) {
|
||||
for (const iface of factIfaceAliases(b.interfaceName, b.serverId)) {
|
||||
const k = `${b.userId}\0${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ userId: b.userId, serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function uniqueBoundIfaces(tuples: UserBindTuple[]): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const t of tuples) {
|
||||
const k = `${t.serverId}\0${t.iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: t.serverId, iface: t.iface })
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> {
|
||||
if (!userId || userId === STATISTICS_UNBOUND_USER_ID) return null
|
||||
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
|
||||
return expandBindingIfaces(binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })))
|
||||
}
|
||||
|
||||
function userBindJoinSql(tuples: UserBindTuple[]): { sql: string; params: unknown[] } {
|
||||
const values = tuples.map(() => "(?::text, ?::int, ?::text)").join(", ")
|
||||
const params = tuples.flatMap((t) => [t.userId, t.serverId, t.iface])
|
||||
return {
|
||||
sql: `JOIN (VALUES ${values}) AS b(user_id, server_id, iface) ON b.server_id = f.server_id AND b.iface = f.iface`,
|
||||
params,
|
||||
}
|
||||
}
|
||||
|
||||
function expandIfaceTuples(
|
||||
items: Array<{ serverId: number; iface: string }>,
|
||||
): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const t of items) {
|
||||
for (const iface of factIfaceAliases(t.iface, t.serverId)) {
|
||||
const k = `${t.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: t.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function loadPayloadScope(serverId?: number): Promise<{
|
||||
overlayIfaces: Array<{ serverId: number; iface: string }>
|
||||
wanIfaces: Array<{ serverId: number; iface: string }>
|
||||
excludeServerIds: number[]
|
||||
topo: FlowTopology
|
||||
}> {
|
||||
const topo = await loadFlowTopology()
|
||||
const catalog = await getServerCatalog()
|
||||
await warmIfaceCache(catalog.list.map((s) => s.id))
|
||||
const overlayRaw: Array<{ serverId: number; iface: string }> = []
|
||||
const wanRaw: Array<{ serverId: number; iface: string }> = []
|
||||
for (const s of catalog.list) {
|
||||
if (serverId != null && s.id !== serverId) continue
|
||||
const wanSet = topo.wanIfaces.get(s.id)
|
||||
const wanNames = wanSet && wanSet.size > 0
|
||||
? [...wanSet]
|
||||
: s.type === "home-router" ? [] : ["ether1"]
|
||||
for (const name of wanNames) wanRaw.push({ serverId: s.id, iface: name })
|
||||
const names = new Set(listCachedIfaceNames(s.id))
|
||||
for (const name of topo.tunnelIfaces?.get(s.id) ?? []) names.add(name)
|
||||
for (const name of names) {
|
||||
if (isOverlayTunnelIface(topo, s.id, name)) overlayRaw.push({ serverId: s.id, iface: name })
|
||||
}
|
||||
}
|
||||
return {
|
||||
overlayIfaces: expandIfaceTuples(overlayRaw),
|
||||
wanIfaces: expandIfaceTuples(wanRaw),
|
||||
excludeServerIds: serverId != null
|
||||
? []
|
||||
: catalog.list.filter((s) => s.type === "exit-node").map((s) => s.id),
|
||||
topo,
|
||||
}
|
||||
}
|
||||
|
||||
async function buildFilterCtx(query: StatisticsQuery, period: ParsedPeriod): Promise<FilterCtx | null> {
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const boundIfaces = uniqueBoundIfaces(bindTuples)
|
||||
const unboundOnly = query.userId === STATISTICS_UNBOUND_USER_ID
|
||||
const userIfaces = unboundOnly ? null : await resolveUserIfaces(query.userId)
|
||||
if (userIfaces && userIfaces.length === 0) return null
|
||||
if (unboundOnly) return null
|
||||
const scope = await loadPayloadScope(query.serverId)
|
||||
return {
|
||||
...period,
|
||||
serverId: query.serverId,
|
||||
iface: query.iface,
|
||||
country: query.country,
|
||||
service: query.service,
|
||||
asn: query.asn,
|
||||
planes: query.planes ?? "unique",
|
||||
userIfaces,
|
||||
unboundOnly,
|
||||
boundIfaces,
|
||||
overlayIfaces: scope.overlayIfaces,
|
||||
wanIfaces: scope.wanIfaces,
|
||||
excludeServerIds: scope.excludeServerIds,
|
||||
topo: scope.topo,
|
||||
}
|
||||
}
|
||||
|
||||
export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> {
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyDto({
|
||||
fromIso: query.from,
|
||||
toIso: query.to,
|
||||
fromDay: query.from.slice(0, 10),
|
||||
toDayExclusive: query.to.slice(0, 10),
|
||||
grain: "day",
|
||||
windowSec: 1,
|
||||
})
|
||||
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyDto(period)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const timeCol = period.grain === "hour" ? "bucket_at" : "day"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
|
||||
const totals = await dbAll<{ bytes: number; packets: number; servers: number }>(`
|
||||
SELECT
|
||||
COALESCE(SUM(f.bytes), 0) AS bytes,
|
||||
COALESCE(SUM(f.packets), 0) AS packets,
|
||||
COUNT(DISTINCT f.server_id)::int AS servers
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
`, where.params)
|
||||
|
||||
const bytes = Number(totals[0]?.bytes) || 0
|
||||
const packets = Number(totals[0]?.packets) || 0
|
||||
const serverCount = Number(totals[0]?.servers) || 0
|
||||
|
||||
const seriesRows = await dbAll<{ t: string; bytes: number }>(`
|
||||
SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY ${timeCol}
|
||||
ORDER BY ${timeCol}
|
||||
`, where.params)
|
||||
|
||||
const countryRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.country AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.country
|
||||
`, where.params)
|
||||
|
||||
const serviceRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.service AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.service
|
||||
`, where.params)
|
||||
|
||||
const asnRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.asn AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.asn
|
||||
`, where.params)
|
||||
|
||||
const serverRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id
|
||||
`, where.params)
|
||||
|
||||
const ifaceRowsRaw = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, where.params)
|
||||
await warmIfaceCache(ifaceRowsRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId))
|
||||
const ifaceRows = collapseServerIfaceRows(ifaceRowsRaw).filter((r) => {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) return false
|
||||
if (ctx.iface) return true
|
||||
if (ctx.topo && isOverlayTunnelIface(ctx.topo, r.serverId, r.iface)) return false
|
||||
if (ctx.topo && isWanFactIface(ctx.topo, r.serverId, r.iface)) return false
|
||||
return true
|
||||
})
|
||||
const ifaceCount = ifaceRows.length
|
||||
|
||||
let dupeIfaceRows: Array<{ serverId: number; iface: string; bytes: number; packets: number; kind: "wan" | "overlay" }> = []
|
||||
if (ctx.planes === "all" && !ctx.iface) {
|
||||
const wanWhere = factWhere("f", period.grain, ctx, "wan")
|
||||
const overlayWhere = factWhere("f", period.grain, ctx, "overlay")
|
||||
const [wanRaw, overlayRaw] = await Promise.all([
|
||||
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${wanWhere.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, wanWhere.params),
|
||||
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${overlayWhere.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, overlayWhere.params),
|
||||
])
|
||||
await warmIfaceCache([
|
||||
...wanRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
|
||||
...overlayRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
|
||||
])
|
||||
const seen = new Set(ifaceRows.map((r) => `${r.serverId}:${r.iface}`))
|
||||
for (const r of collapseServerIfaceRows(wanRaw)) {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
|
||||
const key = `${r.serverId}:${r.iface}`
|
||||
if (seen.has(key)) continue
|
||||
seen.add(key)
|
||||
dupeIfaceRows.push({ ...r, kind: "wan" })
|
||||
}
|
||||
for (const r of collapseServerIfaceRows(overlayRaw)) {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
|
||||
const key = `${r.serverId}:${r.iface}`
|
||||
if (seen.has(key)) continue
|
||||
seen.add(key)
|
||||
dupeIfaceRows.push({ ...r, kind: "overlay" })
|
||||
}
|
||||
}
|
||||
|
||||
let userRows: Array<{ id: string; bytes: number; packets: number }> = []
|
||||
if (bindTuples.length && !ctx.unboundOnly) {
|
||||
const join = userBindJoinSql(bindTuples)
|
||||
userRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY b.user_id
|
||||
`, [...join.params, ...where.params])
|
||||
}
|
||||
|
||||
const serverNames = new Map<number, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(s.id, s.name || s.host)
|
||||
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
|
||||
const asnHolders = new Map<number, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(a.asn, a.holder)
|
||||
|
||||
const countries = toBreakdown(
|
||||
countryRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id === "XX" ? "Неизвестно" : r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const services = toBreakdown(
|
||||
serviceRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const asns = toBreakdown(
|
||||
asnRows.map((r) => {
|
||||
const id = Number(r.id) || 0
|
||||
const holder = asnHolders.get(id)
|
||||
return {
|
||||
id: String(id),
|
||||
label: id === 0 ? "other" : holder ? `AS${id} · ${holder}` : `AS${id}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
}
|
||||
}),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const serverBreakdown = toBreakdown(
|
||||
serverRows.map((r) => ({
|
||||
id: String(r.id),
|
||||
label: serverNames.get(r.id) || String(r.id),
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const uniqueInterfaces = toBreakdown(
|
||||
ifaceRows.map((r) => {
|
||||
const serverName = serverNames.get(r.serverId) || String(r.serverId)
|
||||
const wan = ctx.topo ? isWanFactIface(ctx.topo, r.serverId, r.iface) : false
|
||||
return {
|
||||
id: `${r.serverId}:${r.iface}`,
|
||||
label: wan ? wanIfaceLabel(serverName, r.iface) : `${serverName} · ${r.iface}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
}
|
||||
}),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const dupeInterfaces: StatisticsBreakdownRow[] = dupeIfaceRows.map((r) => {
|
||||
const serverName = serverNames.get(r.serverId) || String(r.serverId)
|
||||
const rowBytes = Number(r.bytes) || 0
|
||||
const rowPackets = Number(r.packets) || 0
|
||||
return {
|
||||
id: `${r.serverId}:${r.iface}`,
|
||||
label: r.kind === "wan" ? wanIfaceLabel(serverName, r.iface) : overlayDupLabel(serverName, r.iface),
|
||||
bytes: rowBytes,
|
||||
packets: rowPackets,
|
||||
bps: (rowBytes * 8) / period.windowSec,
|
||||
percent: 0,
|
||||
}
|
||||
})
|
||||
const interfaces = [...uniqueInterfaces, ...dupeInterfaces]
|
||||
const matchedUsers = toBreakdown(
|
||||
userRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: userNames.get(r.id) || r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
|
||||
const users = [...matchedUsers]
|
||||
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes,
|
||||
packets,
|
||||
avgBps: (bytes * 8) / period.windowSec,
|
||||
users: matchedUsers.length,
|
||||
servers: serverCount,
|
||||
ifaces: ifaceCount,
|
||||
topCountry: countries[0]?.label || "—",
|
||||
topService: services[0]?.label || "—",
|
||||
},
|
||||
series: seriesRows.map((r) => ({ t: r.t, bytes: Number(r.bytes) || 0 })),
|
||||
users,
|
||||
servers: serverBreakdown,
|
||||
interfaces,
|
||||
countries,
|
||||
services,
|
||||
asns,
|
||||
}
|
||||
}
|
||||
|
||||
function dimSql(dim: StatisticsPivotDim, factAlias: string, bindAlias: string): string {
|
||||
if (dim === "country") return `${factAlias}.country`
|
||||
if (dim === "service") return `${factAlias}.service`
|
||||
if (dim === "asn") return `${factAlias}.asn::text`
|
||||
if (dim === "server") return `${factAlias}.server_id::text`
|
||||
if (dim === "iface") return `(${factAlias}.server_id::text || ':' || ${factAlias}.iface)`
|
||||
return `${bindAlias}.user_id`
|
||||
}
|
||||
|
||||
function emptyPivot(query: StatisticsPivotQuery): StatisticsPivotDto {
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric: query.metric,
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function pivotDimsConflict(row: StatisticsPivotDim, col: StatisticsPivotDim): boolean {
|
||||
return row === col
|
||||
}
|
||||
|
||||
export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<StatisticsPivotDto> {
|
||||
if (pivotDimsConflict(query.row, query.col)) return emptyPivot(query)
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyPivot(query)
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyPivot(query)
|
||||
const needsUser = query.row === "user" || query.col === "user"
|
||||
if (needsUser && bindTuples.length === 0) return emptyPivot(query)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
const rowExpr = dimSql(query.row, "f", "b")
|
||||
const colExpr = dimSql(query.col, "f", "b")
|
||||
const join = needsUser ? userBindJoinSql(bindTuples) : { sql: "", params: [] as unknown[] }
|
||||
|
||||
const raw = await dbAll<{ row_id: string; col_id: string; bytes: number; packets: number }>(`
|
||||
SELECT ${rowExpr} AS row_id, ${colExpr} AS col_id,
|
||||
SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY 1, 2
|
||||
`, [...join.params, ...where.params])
|
||||
|
||||
if (query.row === "iface" || query.col === "iface") {
|
||||
const ifaceServerIds: number[] = []
|
||||
for (const r of raw) {
|
||||
for (const dim of [query.row, query.col] as const) {
|
||||
if (dim !== "iface") continue
|
||||
const id = dim === query.row ? String(r.row_id ?? "") : String(r.col_id ?? "")
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) continue
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (looksLikeIfIndex(id.slice(colon + 1)) && Number.isFinite(sid)) ifaceServerIds.push(sid)
|
||||
}
|
||||
}
|
||||
await warmIfaceCache(ifaceServerIds)
|
||||
for (const r of raw) {
|
||||
if (query.row === "iface") r.row_id = canonicalIfaceDimId(String(r.row_id ?? ""))
|
||||
if (query.col === "iface") r.col_id = canonicalIfaceDimId(String(r.col_id ?? ""))
|
||||
}
|
||||
}
|
||||
|
||||
const metric = query.metric
|
||||
type Acc = { bytes: number; packets: number }
|
||||
const cell = new Map<string, Map<string, Acc>>()
|
||||
const colTotals = new Map<string, number>()
|
||||
for (const r of raw) {
|
||||
const rid = String(r.row_id ?? "")
|
||||
const cid = String(r.col_id ?? "")
|
||||
const acc: Acc = { bytes: Number(r.bytes) || 0, packets: Number(r.packets) || 0 }
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
let rowMap = cell.get(rid)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
cell.set(rid, rowMap)
|
||||
}
|
||||
const prev = rowMap.get(cid)
|
||||
if (prev) {
|
||||
prev.bytes += acc.bytes
|
||||
prev.packets += acc.packets
|
||||
} else {
|
||||
rowMap.set(cid, acc)
|
||||
}
|
||||
colTotals.set(cid, (colTotals.get(cid) ?? 0) + val)
|
||||
}
|
||||
|
||||
const topCols = [...colTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_COL_CAP)
|
||||
.map(([id]) => id)
|
||||
const topColSet = new Set(topCols)
|
||||
const folded = new Map<string, Map<string, number>>()
|
||||
const foldedColTotals = new Map<string, number>()
|
||||
let otherBytes = 0
|
||||
for (const [rid, cols] of cell) {
|
||||
const rowMap = new Map<string, number>()
|
||||
for (const [cid, acc] of cols) {
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
const dest = topColSet.has(cid) ? cid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) otherBytes += val
|
||||
rowMap.set(dest, (rowMap.get(dest) ?? 0) + val)
|
||||
foldedColTotals.set(dest, (foldedColTotals.get(dest) ?? 0) + val)
|
||||
}
|
||||
folded.set(rid, rowMap)
|
||||
}
|
||||
|
||||
const rowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
rowTotals.set(rid, t)
|
||||
}
|
||||
const topRows = [...rowTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_ROW_CAP)
|
||||
.map(([id]) => id)
|
||||
const topRowSet = new Set(topRows)
|
||||
const finalRows = new Map<string, Map<string, number>>()
|
||||
const finalRowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
const dest = topRowSet.has(rid) ? rid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) {
|
||||
for (const [cid, v] of cols) {
|
||||
if (cid !== PIVOT_OTHER_ID) otherBytes += v
|
||||
}
|
||||
}
|
||||
let rowMap = finalRows.get(dest)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
finalRows.set(dest, rowMap)
|
||||
}
|
||||
for (const [cid, v] of cols) {
|
||||
rowMap.set(cid, (rowMap.get(cid) ?? 0) + v)
|
||||
}
|
||||
}
|
||||
for (const [rid, cols] of finalRows) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
finalRowTotals.set(rid, t)
|
||||
}
|
||||
|
||||
const colIds = [...topCols]
|
||||
if (foldedColTotals.has(PIVOT_OTHER_ID)) colIds.push(PIVOT_OTHER_ID)
|
||||
const rowIds = [...topRows]
|
||||
if (finalRows.has(PIVOT_OTHER_ID) && !topRowSet.has(PIVOT_OTHER_ID)) rowIds.push(PIVOT_OTHER_ID)
|
||||
|
||||
const labels = await loadPivotLabels(query.row, query.col, rowIds, colIds)
|
||||
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric,
|
||||
columns: colIds.map((id) => ({
|
||||
id,
|
||||
label: labels.col.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: foldedColTotals.get(id) ?? 0,
|
||||
})),
|
||||
rows: rowIds.map((id) => {
|
||||
const cols = finalRows.get(id) ?? new Map()
|
||||
const cells: Record<string, number> = {}
|
||||
for (const cid of colIds) cells[cid] = cols.get(cid) ?? 0
|
||||
return {
|
||||
id,
|
||||
label: labels.row.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: finalRowTotals.get(id) ?? 0,
|
||||
cells,
|
||||
}
|
||||
}),
|
||||
otherBytes,
|
||||
}
|
||||
}
|
||||
|
||||
async function loadPivotLabels(
|
||||
rowDim: StatisticsPivotDim,
|
||||
colDim: StatisticsPivotDim,
|
||||
rowIds: string[],
|
||||
colIds: string[],
|
||||
): Promise<{ row: Map<string, string>; col: Map<string, string> }> {
|
||||
const topo = await loadFlowTopology()
|
||||
const serverNames = new Map<string, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(String(s.id), s.name || s.host)
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
const asnHolders = new Map<string, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(String(a.asn), a.holder)
|
||||
|
||||
function label(dim: StatisticsPivotDim, id: string): string {
|
||||
if (id === PIVOT_OTHER_ID) return "Прочие"
|
||||
if (dim === "country") return id === "XX" ? "Неизвестно" : id
|
||||
if (dim === "server") return serverNames.get(id) || id
|
||||
if (dim === "user") return userNames.get(id) || id
|
||||
if (dim === "asn") {
|
||||
if (id === "0") return "other"
|
||||
const holder = asnHolders.get(id)
|
||||
return holder ? `AS${id} · ${holder}` : `AS${id}`
|
||||
}
|
||||
if (dim === "iface") {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = id.slice(0, colon)
|
||||
const iface = id.slice(colon + 1)
|
||||
const sidNum = Number(sid)
|
||||
const name = Number.isFinite(sidNum) ? displayFactIface(sidNum, iface) : iface
|
||||
const serverName = serverNames.get(sid) || sid
|
||||
if (Number.isFinite(sidNum) && isWanFactIface(topo, sidNum, name)) {
|
||||
return wanIfaceLabel(serverName, name)
|
||||
}
|
||||
if (Number.isFinite(sidNum) && isOverlayTunnelIface(topo, sidNum, name)) {
|
||||
return overlayDupLabel(serverName, name)
|
||||
}
|
||||
return `${serverName} · ${name}`
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
const row = new Map<string, string>()
|
||||
const col = new Map<string, string>()
|
||||
for (const id of rowIds) row.set(id, label(rowDim, id))
|
||||
for (const id of colIds) col.set(id, label(colDim, id))
|
||||
return { row, col }
|
||||
}
|
||||
@@ -26,11 +26,10 @@ import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-settings.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached } from "./traffic-flow-ripe.js"
|
||||
import { classifyFlowDst, refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { enqueueRipeMisses } from "./traffic-flow-ripe.js"
|
||||
import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import { refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
enGreIfaceNames,
|
||||
getServerCatalog,
|
||||
@@ -252,11 +251,20 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
totalPackets += r.packets
|
||||
srcs.add(r.src)
|
||||
dsts.add(r.dst)
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
peers.add(peer)
|
||||
const destMeta = resolveInternetDest({
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
serverId: r.serverId,
|
||||
inIface: resolved.name,
|
||||
topo,
|
||||
})
|
||||
if (destMeta.dest) peers.add(destMeta.dest)
|
||||
const app = applicationName(r.proto, r.dstPort, r.srcPort)
|
||||
const ripe = lookupRipeCached(peer)
|
||||
const classified = classifyFlowDst(peer, r.proto, r.dstPort, r.srcPort, ripe)
|
||||
const ripe = destMeta.ripe
|
||||
const classified = destMeta.classified
|
||||
bump(applications, app, r.bytes, r.packets)
|
||||
bump(protocols, protoName(r.proto), r.bytes, r.packets)
|
||||
bump(sources, r.src, r.bytes, r.packets)
|
||||
@@ -268,7 +276,7 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
const asnLabel = ripe.holder ? `AS${ripe.asn} ${ripe.holder}` : `AS${ripe.asn}`
|
||||
bump(asns, asnId, r.bytes, r.packets, asnLabel)
|
||||
}
|
||||
const dstCountry = ripe?.ok && isIsoCountry(ripe.country) ? ripe.country : ""
|
||||
const dstCountry = destMeta.country && destMeta.country !== "unknown" ? destMeta.country : ""
|
||||
if (dstCountry) {
|
||||
bump(countries, dstCountry, r.bytes, r.packets)
|
||||
}
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
brandByAsn,
|
||||
brandByHolder,
|
||||
countryFromHolder,
|
||||
isSteamGamePort,
|
||||
lookupBrand,
|
||||
OTHER_SERVICE,
|
||||
isNamedInternetService,
|
||||
mapServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
resolveRipeCountry,
|
||||
} from "./traffic-flow-brands.js"
|
||||
|
||||
@@ -39,4 +42,35 @@ assert.equal(isNamedInternetService("DNS", "DNS"), false)
|
||||
assert.equal(mapServiceNodeId("AWS"), "svc:aws")
|
||||
assert.equal(mapServiceNodeId("Cloudflare"), "svc:cloudflare")
|
||||
|
||||
assert.equal(brandByAsn(714)?.service, "Apple")
|
||||
assert.equal(brandByAsn(714)?.category, "CDN")
|
||||
assert.equal(brandByAsn(36459)?.service, "GitHub")
|
||||
assert.equal(brandByAsn(395701)?.service, "Epic")
|
||||
assert.equal(brandByAsn(6507)?.service, "Riot")
|
||||
assert.equal(brandByAsn(33353)?.service, "PlayStation")
|
||||
assert.equal(brandByAsn(14061)?.service, "DigitalOcean")
|
||||
assert.equal(brandByAsn(24940)?.service, "Hetzner")
|
||||
assert.equal(brandByAsn(8403)?.service, "Spotify")
|
||||
assert.equal(brandByAsn(13414)?.service, "X")
|
||||
assert.equal(brandByAsn(47541)?.service, "VK")
|
||||
assert.equal(brandByAsn(47764)?.service, "VK")
|
||||
assert.equal(brandByAsn(30103)?.service, "Zoom")
|
||||
assert.equal(brandByAsn(19281)?.service, "Quad9")
|
||||
assert.equal(brandByAsn(9059)?.service, "AWS")
|
||||
assert.equal(brandByAsn(396982)?.service, "Google")
|
||||
assert.equal(brandByAsn(400645)?.service, "ChatGPT")
|
||||
|
||||
assert.equal(brandByHolder("VALVE-CORPORATION")?.service, "Steam")
|
||||
assert.equal(brandByHolder("OpenAI, LLC")?.service, "ChatGPT")
|
||||
assert.equal(brandByHolder("YouTube LLC")?.service, "YouTube")
|
||||
assert.equal(brandByHolder("AMAZON-AES - Amazon.com, Inc."), null)
|
||||
|
||||
assert.equal(isSteamGamePort(17, 27015, 50000), true)
|
||||
assert.equal(isSteamGamePort(6, 443, 50000), false)
|
||||
|
||||
assert.equal(resolveFlowBrand("104.18.35.51", 32590, "VALVE-CORPORATION", 6, 443, 1)?.service, "Cloudflare")
|
||||
assert.equal(resolveFlowBrand("203.0.113.9", 32590, "", 17, 27015, 50000)?.service, "Steam")
|
||||
assert.equal(resolveRipeCountry("", 9059, ""), "IE")
|
||||
assert.equal(resolveRipeCountry("", 24940, ""), "DE")
|
||||
|
||||
console.log("traffic-flow-brands.test.ts: ok")
|
||||
|
||||
@@ -7,59 +7,162 @@ export interface BrandHit {
|
||||
category: string
|
||||
}
|
||||
|
||||
const CDN = { category: "CDN" } as const
|
||||
const WEB = { category: "Веб" } as const
|
||||
const VIDEO = { category: "Видео / стриминг" } as const
|
||||
const GAMES = { category: "Игры" } as const
|
||||
const VOICE = { category: "Голос" } as const
|
||||
const AI = { category: "ИИ" } as const
|
||||
const DNS = { category: "DNS" } as const
|
||||
|
||||
const CLOUDFLARE: BrandHit = { service: "Cloudflare", ...CDN }
|
||||
const FASTLY: BrandHit = { service: "Fastly", ...CDN }
|
||||
const AKAMAI: BrandHit = { service: "Akamai", ...CDN }
|
||||
const AWS: BrandHit = { service: "AWS", ...CDN }
|
||||
const MICROSOFT: BrandHit = { service: "Microsoft", ...CDN }
|
||||
const YANDEX: BrandHit = { service: "Yandex", ...CDN }
|
||||
const APPLE: BrandHit = { service: "Apple", ...CDN }
|
||||
const DIGITALOCEAN: BrandHit = { service: "DigitalOcean", ...CDN }
|
||||
const HETZNER: BrandHit = { service: "Hetzner", ...CDN }
|
||||
const OVH: BrandHit = { service: "OVH", ...CDN }
|
||||
const ORACLE: BrandHit = { service: "Oracle", ...CDN }
|
||||
const LINODE: BrandHit = { service: "Linode", ...CDN }
|
||||
const VULTR: BrandHit = { service: "Vultr", ...CDN }
|
||||
const SCALEWAY: BrandHit = { service: "Scaleway", ...CDN }
|
||||
const IBM_CLOUD: BrandHit = { service: "IBM Cloud", ...CDN }
|
||||
const ALIBABA: BrandHit = { service: "Alibaba", ...CDN }
|
||||
const TENCENT: BrandHit = { service: "Tencent", ...CDN }
|
||||
const GCORE: BrandHit = { service: "G-Core", ...CDN }
|
||||
const CDN77: BrandHit = { service: "CDN77", ...CDN }
|
||||
const SELECTEL: BrandHit = { service: "Selectel", ...CDN }
|
||||
const TIMEWEB: BrandHit = { service: "Timeweb", ...CDN }
|
||||
const BEGET: BrandHit = { service: "Beget", ...CDN }
|
||||
const DDOS_GUARD: BrandHit = { service: "DDoS-Guard", ...CDN }
|
||||
const META: BrandHit = { service: "Meta", ...CDN }
|
||||
|
||||
const GOOGLE: BrandHit = { service: "Google", ...WEB }
|
||||
const GITHUB: BrandHit = { service: "GitHub", ...WEB }
|
||||
const GITLAB: BrandHit = { service: "GitLab", ...WEB }
|
||||
const X: BrandHit = { service: "X", ...WEB }
|
||||
const LINKEDIN: BrandHit = { service: "LinkedIn", ...WEB }
|
||||
const VK: BrandHit = { service: "VK", ...WEB }
|
||||
const REDDIT: BrandHit = { service: "Reddit", ...WEB }
|
||||
const DROPBOX: BrandHit = { service: "Dropbox", ...WEB }
|
||||
const SNAP: BrandHit = { service: "Snap", ...WEB }
|
||||
const WIKIPEDIA: BrandHit = { service: "Wikipedia", ...WEB }
|
||||
const PAYPAL: BrandHit = { service: "PayPal", ...WEB }
|
||||
const SALESFORCE: BrandHit = { service: "Salesforce", ...WEB }
|
||||
|
||||
const YOUTUBE: BrandHit = { service: "YouTube", ...VIDEO }
|
||||
const NETFLIX: BrandHit = { service: "Netflix", ...VIDEO }
|
||||
const TWITCH: BrandHit = { service: "Twitch", ...VIDEO }
|
||||
const TIKTOK: BrandHit = { service: "TikTok", ...VIDEO }
|
||||
const SPOTIFY: BrandHit = { service: "Spotify", ...VIDEO }
|
||||
|
||||
const STEAM: BrandHit = { service: "Steam", ...GAMES }
|
||||
const BLIZZARD: BrandHit = { service: "Blizzard", ...GAMES }
|
||||
const EPIC: BrandHit = { service: "Epic", ...GAMES }
|
||||
const RIOT: BrandHit = { service: "Riot", ...GAMES }
|
||||
const PLAYSTATION: BrandHit = { service: "PlayStation", ...GAMES }
|
||||
const ROBLOX: BrandHit = { service: "Roblox", ...GAMES }
|
||||
const UBISOFT: BrandHit = { service: "Ubisoft", ...GAMES }
|
||||
|
||||
const DISCORD: BrandHit = { service: "Discord", ...VOICE }
|
||||
const TELEGRAM: BrandHit = { service: "Telegram", ...VOICE }
|
||||
const ZOOM: BrandHit = { service: "Zoom", ...VOICE }
|
||||
|
||||
const CHATGPT: BrandHit = { service: "ChatGPT", ...AI }
|
||||
|
||||
const QUAD9: BrandHit = { service: "Quad9", ...DNS }
|
||||
const OPENDNS: BrandHit = { service: "OpenDNS", ...DNS }
|
||||
|
||||
function brandEntries(hit: BrandHit, asns: number[]): Array<[number, BrandHit]> {
|
||||
return asns.map((asn) => [asn, hit])
|
||||
}
|
||||
|
||||
function hqEntries(cc: string, asns: number[]): Array<[number, string]> {
|
||||
return asns.map((asn) => [asn, cc])
|
||||
}
|
||||
|
||||
const ASN_BRANDS = new Map<number, BrandHit>([
|
||||
[13335, { service: "Cloudflare", category: "CDN" }],
|
||||
[209242, { service: "Cloudflare", category: "CDN" }],
|
||||
[54113, { service: "Fastly", category: "CDN" }],
|
||||
[20940, { service: "Akamai", category: "CDN" }],
|
||||
[16509, { service: "AWS", category: "CDN" }],
|
||||
[14618, { service: "AWS", category: "CDN" }],
|
||||
[8075, { service: "Microsoft", category: "CDN" }],
|
||||
[13238, { service: "Yandex", category: "CDN" }],
|
||||
[32590, { service: "Steam", category: "Игры" }],
|
||||
[57976, { service: "Blizzard", category: "Игры" }],
|
||||
[2906, { service: "Netflix", category: "Видео / стриминг" }],
|
||||
[40027, { service: "Netflix", category: "Видео / стриминг" }],
|
||||
[15169, { service: "Google", category: "Веб" }],
|
||||
[36040, { service: "YouTube", category: "Видео / стриминг" }],
|
||||
[46489, { service: "Twitch", category: "Видео / стриминг" }],
|
||||
[401115, { service: "ChatGPT", category: "ИИ" }],
|
||||
[49544, { service: "Discord", category: "Голос" }],
|
||||
[62041, { service: "Telegram", category: "Голос" }],
|
||||
[59930, { service: "Telegram", category: "Голос" }],
|
||||
[211157, { service: "Telegram", category: "Голос" }],
|
||||
[32934, { service: "Meta", category: "CDN" }],
|
||||
[396986, { service: "TikTok", category: "Видео / стриминг" }],
|
||||
...brandEntries(CLOUDFLARE, [13335, 209242]),
|
||||
...brandEntries(FASTLY, [54113]),
|
||||
...brandEntries(AKAMAI, [20940, 16625, 32787, 35994, 16702, 24319]),
|
||||
...brandEntries(AWS, [16509, 14618, 8987, 7224, 9059]),
|
||||
...brandEntries(MICROSOFT, [8075, 8068, 8069, 8070]),
|
||||
...brandEntries(YANDEX, [13238]),
|
||||
...brandEntries(APPLE, [714, 6185]),
|
||||
...brandEntries(DIGITALOCEAN, [14061]),
|
||||
...brandEntries(HETZNER, [24940, 213230]),
|
||||
...brandEntries(OVH, [16276]),
|
||||
...brandEntries(ORACLE, [31898]),
|
||||
...brandEntries(LINODE, [63949]),
|
||||
...brandEntries(VULTR, [20473]),
|
||||
...brandEntries(SCALEWAY, [12876]),
|
||||
...brandEntries(IBM_CLOUD, [36351]),
|
||||
...brandEntries(ALIBABA, [45102]),
|
||||
...brandEntries(TENCENT, [132203]),
|
||||
...brandEntries(GCORE, [199524]),
|
||||
...brandEntries(CDN77, [60068]),
|
||||
...brandEntries(SELECTEL, [50340, 49505]),
|
||||
...brandEntries(TIMEWEB, [9123]),
|
||||
...brandEntries(BEGET, [198610]),
|
||||
...brandEntries(DDOS_GUARD, [57724]),
|
||||
...brandEntries(META, [32934, 63293, 54115]),
|
||||
...brandEntries(GOOGLE, [15169, 396982]),
|
||||
...brandEntries(GITHUB, [36459]),
|
||||
...brandEntries(GITLAB, [54876]),
|
||||
...brandEntries(X, [13414]),
|
||||
...brandEntries(LINKEDIN, [14413, 40793]),
|
||||
...brandEntries(VK, [47541, 47764]),
|
||||
...brandEntries(REDDIT, [394706]),
|
||||
...brandEntries(DROPBOX, [19679]),
|
||||
...brandEntries(SNAP, [19750]),
|
||||
...brandEntries(WIKIPEDIA, [14907]),
|
||||
...brandEntries(PAYPAL, [17012, 26101]),
|
||||
...brandEntries(SALESFORCE, [14340]),
|
||||
...brandEntries(YOUTUBE, [36040, 43515]),
|
||||
...brandEntries(NETFLIX, [2906, 40027]),
|
||||
...brandEntries(TWITCH, [46489]),
|
||||
...brandEntries(TIKTOK, [396986, 138699]),
|
||||
...brandEntries(SPOTIFY, [8403, 34081]),
|
||||
...brandEntries(STEAM, [32590]),
|
||||
...brandEntries(BLIZZARD, [57976]),
|
||||
...brandEntries(EPIC, [395701]),
|
||||
...brandEntries(RIOT, [6507, 62830]),
|
||||
...brandEntries(PLAYSTATION, [33353]),
|
||||
...brandEntries(ROBLOX, [22697]),
|
||||
...brandEntries(UBISOFT, [197922]),
|
||||
...brandEntries(DISCORD, [49544, 394141]),
|
||||
...brandEntries(TELEGRAM, [62041, 59930, 211157]),
|
||||
...brandEntries(ZOOM, [30103]),
|
||||
...brandEntries(CHATGPT, [401115, 400645]),
|
||||
...brandEntries(QUAD9, [19281]),
|
||||
...brandEntries(OPENDNS, [36692]),
|
||||
])
|
||||
|
||||
const ASN_HQ_COUNTRY = new Map<number, string>([
|
||||
[13335, "US"],
|
||||
[209242, "US"],
|
||||
[54113, "US"],
|
||||
[20940, "US"],
|
||||
[16509, "US"],
|
||||
[14618, "US"],
|
||||
[8075, "US"],
|
||||
[15169, "US"],
|
||||
[32590, "US"],
|
||||
[57976, "US"],
|
||||
[2906, "US"],
|
||||
[40027, "US"],
|
||||
[36040, "US"],
|
||||
[46489, "US"],
|
||||
[401115, "US"],
|
||||
[49544, "US"],
|
||||
[32934, "US"],
|
||||
[13238, "RU"],
|
||||
[62041, "NL"],
|
||||
[59930, "NL"],
|
||||
[211157, "NL"],
|
||||
...hqEntries("US", [
|
||||
13335, 209242, 54113, 20940, 16625, 32787, 35994, 16702, 24319,
|
||||
16509, 14618, 8987, 7224, 8075, 8068, 8069, 8070, 15169, 396982,
|
||||
32590, 57976, 2906, 40027, 36040, 43515, 46489, 401115, 400645,
|
||||
49544, 394141, 32934, 63293, 54115, 714, 6185, 36459, 54876, 14061,
|
||||
31898, 63949, 20473, 36351, 13414, 14413, 40793, 394706, 19679, 19750,
|
||||
14907, 17012, 26101, 14340, 30103, 36692, 395701, 6507, 62830, 33353, 22697,
|
||||
]),
|
||||
...hqEntries("IE", [9059]),
|
||||
...hqEntries("SG", [138699]),
|
||||
...hqEntries("DE", [24940, 213230]),
|
||||
...hqEntries("FR", [16276, 12876, 197922]),
|
||||
...hqEntries("CN", [45102, 132203]),
|
||||
...hqEntries("LU", [199524]),
|
||||
...hqEntries("CZ", [60068]),
|
||||
...hqEntries("RU", [13238, 50340, 49505, 9123, 198610, 57724, 47541, 47764]),
|
||||
...hqEntries("SE", [8403, 34081]),
|
||||
...hqEntries("NL", [62041, 59930, 211157]),
|
||||
...hqEntries("CH", [19281]),
|
||||
])
|
||||
|
||||
const GOOGLE: BrandHit = { service: "Google", category: "Веб" }
|
||||
const CLOUDFLARE: BrandHit = { service: "Cloudflare", category: "CDN" }
|
||||
const YOUTUBE: BrandHit = { service: "YouTube", category: "Видео / стриминг" }
|
||||
|
||||
const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
|
||||
{ cidr: "104.16.0.0/13", prefixLen: 13, hit: CLOUDFLARE },
|
||||
{ cidr: "104.24.0.0/14", prefixLen: 14, hit: CLOUDFLARE },
|
||||
@@ -75,8 +178,25 @@ const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
|
||||
{ cidr: "208.117.224.0/19", prefixLen: 19, hit: YOUTUBE },
|
||||
].sort((a, b) => b.prefixLen - a.prefixLen)
|
||||
|
||||
const HOLDER_BRANDS: Array<{ re: RegExp; hit: BrandHit }> = [
|
||||
{ re: /youtube/i, hit: YOUTUBE },
|
||||
{ re: /valve|\bsteam\b/i, hit: STEAM },
|
||||
{ re: /blizzard|battle.?net/i, hit: BLIZZARD },
|
||||
{ re: /openai/i, hit: CHATGPT },
|
||||
{ re: /riot games/i, hit: RIOT },
|
||||
{ re: /epic games/i, hit: EPIC },
|
||||
{ re: /\bapple\b/i, hit: APPLE },
|
||||
{ re: /github/i, hit: GITHUB },
|
||||
{ re: /spotify/i, hit: SPOTIFY },
|
||||
{ re: /twitter|\bx corp\b/i, hit: X },
|
||||
{ re: /dropbox/i, hit: DROPBOX },
|
||||
{ re: /akamai/i, hit: AKAMAI },
|
||||
]
|
||||
|
||||
const NON_ISO = new Set(["EU", "AP", "ZZ", "XX", "A1", "A2", "O1"])
|
||||
|
||||
const STEAM_ASN = 32590
|
||||
|
||||
export function isIsoCountry(code: string): boolean {
|
||||
const c = String(code ?? "").trim().toUpperCase()
|
||||
return /^[A-Z]{2}$/.test(c) && !NON_ISO.has(c)
|
||||
@@ -114,10 +234,51 @@ export function brandByCidr(ip: string): BrandHit | null {
|
||||
return null
|
||||
}
|
||||
|
||||
export function brandByHolder(holder: string): BrandHit | null {
|
||||
const h = String(holder ?? "").trim()
|
||||
if (!h) return null
|
||||
for (const row of HOLDER_BRANDS) {
|
||||
if (row.re.test(h)) return row.hit
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/** Игровые порты Steam — только вместе с AS32590, никогда :80/:443. */
|
||||
export function isSteamGamePort(proto: number, dstPort: number, srcPort: number): boolean {
|
||||
if (proto !== 6 && proto !== 17) return false
|
||||
const port = dstPort || srcPort
|
||||
if (!port || port === 80 || port === 443) return false
|
||||
if (port === 4380 || port === 3478) return true
|
||||
return port >= 27000 && port <= 27100
|
||||
}
|
||||
|
||||
export function lookupBrand(ip: string, asn: number): BrandHit | null {
|
||||
return brandByCidr(ip) || brandByAsn(asn)
|
||||
}
|
||||
|
||||
/**
|
||||
* Cloudflare CIDR бьёт holder (витрина на CF не становится Steam).
|
||||
* Holder (YouTube и др.) бьёт остальные CIDR/ASN.
|
||||
* Порты Steam — только AS32590 и не выше Cloudflare CIDR.
|
||||
*/
|
||||
export function resolveFlowBrand(
|
||||
ip: string,
|
||||
asn: number,
|
||||
holder: string,
|
||||
proto = 0,
|
||||
dstPort = 0,
|
||||
srcPort = 0,
|
||||
): BrandHit | null {
|
||||
const cidrBrand = brandByCidr(ip)
|
||||
if (cidrBrand?.service === "Cloudflare") return cidrBrand
|
||||
const holderBrand = brandByHolder(holder)
|
||||
if (holderBrand) return holderBrand
|
||||
const fromLookup = cidrBrand || brandByAsn(asn)
|
||||
if (fromLookup) return fromLookup
|
||||
if (asn === STEAM_ASN && isSteamGamePort(proto, dstPort, srcPort)) return STEAM
|
||||
return null
|
||||
}
|
||||
|
||||
const SKIP_MAP_SERVICES = new Set([
|
||||
OTHER_SERVICE,
|
||||
"GRE",
|
||||
|
||||
@@ -53,6 +53,71 @@ const youtube = classifyFlowDst("173.194.160.163", 6, 443, 1, {
|
||||
assert.equal(youtube.service, "YouTube")
|
||||
assert.equal(youtube.category, "Видео / стриминг")
|
||||
|
||||
const valve = classifyFlowDst("203.0.113.40", 17, 27015, 50000, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 64501,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "VALVE-CORPORATION",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(valve.service, "Steam")
|
||||
assert.equal(valve.category, "Игры")
|
||||
|
||||
const openaiHolder = classifyFlowDst("203.0.113.41", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 64502,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "OPENAI, US",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(openaiHolder.service, "ChatGPT")
|
||||
assert.equal(openaiHolder.category, "ИИ")
|
||||
|
||||
const cfNotSteam = classifyFlowDst("104.18.35.51", 6, 443, 1, {
|
||||
prefix: "104.18.0.0/16",
|
||||
asn: 32590,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "VALVE-CORPORATION",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(cfNotSteam.service, "Cloudflare")
|
||||
assert.notEqual(cfNotSteam.service, "Steam")
|
||||
|
||||
const awsIeu = classifyFlowDst("203.0.113.42", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 9059,
|
||||
country: "IE",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "AMAZON-02",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(awsIeu.service, "AWS")
|
||||
assert.equal(awsIeu.category, "CDN")
|
||||
|
||||
const googleCloud = classifyFlowDst("203.0.113.43", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 396982,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE-CLOUD",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(googleCloud.service, "Google")
|
||||
assert.equal(googleCloud.category, "Веб")
|
||||
|
||||
const gre = classifyFlowDst("198.51.100.1", 47, 0, 0, null)
|
||||
assert.equal(gre.service, "GRE")
|
||||
assert.equal(gre.category, "Туннель")
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { lookupBrand, OTHER_SERVICE } from "./traffic-flow-brands.js"
|
||||
import { OTHER_SERVICE, resolveFlowBrand } from "./traffic-flow-brands.js"
|
||||
import { db } from "../db/index.js"
|
||||
import { evobgpSettings } from "../db/schema.js"
|
||||
import { ipInCidrV4, parseCidrV4 } from "./traffic-flow-ip.js"
|
||||
@@ -47,12 +47,13 @@ export function seedFlowCatalogForTests(input: {
|
||||
|
||||
export function categoryFromPurpose(purpose: string, proto: number, dstPort: number, srcPort: number): string {
|
||||
const p = purpose.toLowerCase()
|
||||
if (/gaming|steam|epic|riot/.test(p)) return "Игры"
|
||||
if (/streaming|youtube|netflix|twitch|video/.test(p)) return "Видео / стриминг"
|
||||
if (/cdn|cloudflare|akamai|fastly/.test(p)) return "CDN"
|
||||
if (/gaming|steam|epic|riot|playstation|roblox|ubisoft/.test(p)) return "Игры"
|
||||
if (/streaming|youtube|netflix|twitch|video|spotify/.test(p)) return "Видео / стриминг"
|
||||
if (/cdn|cloudflare|akamai|fastly|hetzner|ovh|apple/.test(p)) return "CDN"
|
||||
if (/voip|discord|zoom/.test(p)) return "Голос"
|
||||
if (/openai|chatgpt|\bai\b/.test(p)) return "ИИ"
|
||||
if (/веб|web|google/.test(p)) return "Веб"
|
||||
if (/quad9|opendns/.test(p)) return "DNS"
|
||||
if (/веб|web|google|github|paypal|vk|linkedin/.test(p)) return "Веб"
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "DNS" || app === "SSH" || app === "BGP") return app
|
||||
if (app === "GRE" || app === "ESP" || app === "WireGuard") return "Туннель"
|
||||
@@ -79,10 +80,7 @@ export function classifyFlowDst(
|
||||
if (app === "WireGuard") return { service: "WireGuard", category: "Туннель" }
|
||||
const hit = matchCidr(dst)
|
||||
const holder = ripe?.holder ?? ""
|
||||
const youtubeHolder = /youtube/i.test(holder)
|
||||
const brand = youtubeHolder
|
||||
? { service: "YouTube", category: "Видео / стриминг" }
|
||||
: lookupBrand(dst, ripe?.asn ?? 0)
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, holder, proto, dstPort, srcPort)
|
||||
const asnName = ripe?.asn ? asnPurpose.get(ripe.asn) : undefined
|
||||
const service = (hit?.purpose || brand?.service || asnName || OTHER_SERVICE).trim() || OTHER_SERVICE
|
||||
const category = hit
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import {
|
||||
dedupFlowRowsAcrossExporters,
|
||||
dedupFlowRowsMaxBytes,
|
||||
flowConversationKey,
|
||||
flowTupleKey,
|
||||
} from "./traffic-flow-dedup.js"
|
||||
|
||||
const a = {
|
||||
serverId: 7,
|
||||
@@ -22,4 +27,13 @@ assert.equal(flowTupleKey(a), flowTupleKey(b))
|
||||
const sameIface = dedupFlowRowsMaxBytes([a, { ...a, bytes: 3_000, packets: 2 }])
|
||||
assert.equal(sameIface[0]?.bytes, 15_000)
|
||||
|
||||
const jh = { ...a, serverId: 7, bytes: 9_000 }
|
||||
const en = { ...a, serverId: 9, bytes: 11_000, inIface: "1" }
|
||||
assert.equal(flowConversationKey(jh), flowConversationKey(en))
|
||||
assert.notEqual(flowTupleKey(jh), flowTupleKey(en))
|
||||
const across = dedupFlowRowsAcrossExporters([en, jh], (x, y) => (x.serverId === 7 ? x : y))
|
||||
assert.equal(across.length, 1)
|
||||
assert.equal(across[0]?.serverId, 7)
|
||||
assert.equal(across[0]?.bytes, 9_000)
|
||||
|
||||
console.log("traffic-flow-dedup.test.ts: ok")
|
||||
|
||||
@@ -14,6 +14,32 @@ export function flowTupleKey(r: Pick<FlowTupleRow, "serverId" | "src" | "dst" |
|
||||
return `${r.serverId}|${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/** Один разговор на всех экспортёрах (JH+EN), без serverId. */
|
||||
export function flowConversationKey(r: Pick<FlowTupleRow, "src" | "dst" | "proto" | "srcPort" | "dstPort">): string {
|
||||
return `${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Схлопнуть копии одного 5-tuple с разных серверов.
|
||||
* `prefer` выбирает ряд (клиент на JH важнее голого EN).
|
||||
*/
|
||||
export function dedupFlowRowsAcrossExporters<T extends FlowTupleRow>(
|
||||
rows: T[],
|
||||
prefer: (a: T, b: T) => T,
|
||||
): T[] {
|
||||
const byConv = new Map<string, T>()
|
||||
for (const row of rows) {
|
||||
const key = flowConversationKey(row)
|
||||
const prev = byConv.get(key)
|
||||
if (!prev) {
|
||||
byConv.set(key, row)
|
||||
continue
|
||||
}
|
||||
byConv.set(key, prefer(prev, row))
|
||||
}
|
||||
return [...byConv.values()]
|
||||
}
|
||||
|
||||
function ifaceKey(r: FlowTupleRow): string {
|
||||
return `${flowTupleKey(r)}|${r.inIface}`
|
||||
}
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
ingestParsedFlowsForServerForTests,
|
||||
resetEngineForTests,
|
||||
} from "./traffic-flow-engine.js"
|
||||
import { factsSnapshotForTests } from "./traffic-flow-facts.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetEngineForTests()
|
||||
resetIfaceCacheForTests()
|
||||
|
||||
seedRipeCacheForTests({
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "95.167.0.0/16",
|
||||
asn: 12389,
|
||||
country: "RU",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "ROSTELECOM-AS",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
|
||||
const topo: FlowTopology = {
|
||||
clientIfaces: new Map([[1, new Set(["gre-client"])]]),
|
||||
clientByIface: new Map([["1|gre-client", {
|
||||
userId: "u-rost",
|
||||
login: "alice",
|
||||
name: "Alice",
|
||||
serverId: 1,
|
||||
interfaceName: "gre-client",
|
||||
}]]),
|
||||
enNodes: [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
wanIfaces: new Map([[1, new Set(["ether1"])]]),
|
||||
plane: {
|
||||
clientIfaceNames: new Set(["gre-client"]),
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
},
|
||||
}
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
|
||||
|
||||
ingestParsedFlowsForServerForTests(1, [
|
||||
{
|
||||
src: "95.167.1.10",
|
||||
dst: "10.200.100.53",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 100,
|
||||
packets: 2,
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
},
|
||||
{
|
||||
src: "95.167.1.10",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 50,
|
||||
packets: 1,
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
},
|
||||
])
|
||||
|
||||
const facts = factsSnapshotForTests()
|
||||
const total = facts.reduce((s, r) => s + r.bytes, 0)
|
||||
const asnBytes = facts.reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(total, 150)
|
||||
assert.equal(asnBytes, 150, "unique bytes = SUM dest ASN")
|
||||
assert.equal(facts.some((r) => r.asn === 12389), false, "ASN клиента не в кубе")
|
||||
const google = facts.find((r) => r.asn === 15169)
|
||||
assert.ok(google)
|
||||
assert.equal(google.bytes, 50)
|
||||
const other = facts.filter((r) => r.asn === 0).reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(other, 100)
|
||||
|
||||
resetEngineForTests()
|
||||
seedFlowTopologyForTests(null)
|
||||
resetRipeCacheForTests()
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-dest.test.ts: ok")
|
||||
@@ -0,0 +1,60 @@
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { classifyFlowDst, type FlowClassification } from "./traffic-flow-classify.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { pickInternetDest, type InternetDestCtx } from "./traffic-flow-ip.js"
|
||||
import type { FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import {
|
||||
flowOursHosts,
|
||||
resolveClient,
|
||||
type FlowTopology,
|
||||
} from "./traffic-flow-topology.js"
|
||||
|
||||
export interface InternetDestMeta {
|
||||
dest: string
|
||||
ripe: FlowIpMeta | null
|
||||
classified: FlowClassification
|
||||
country: string
|
||||
asn: number
|
||||
}
|
||||
|
||||
export function destCtxForIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
inIface: string,
|
||||
): InternetDestCtx {
|
||||
const name = canonicalFactIface(serverId, inIface) || String(inIface ?? "").trim()
|
||||
return {
|
||||
ours: flowOursHosts(topo),
|
||||
boundClient: Boolean(topo && name && resolveClient(topo, serverId, name)),
|
||||
}
|
||||
}
|
||||
|
||||
export function resolveInternetDest(opts: {
|
||||
src: string
|
||||
dst: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
serverId: number
|
||||
inIface: string
|
||||
topo?: FlowTopology | null
|
||||
}): InternetDestMeta {
|
||||
const dest = pickInternetDest(
|
||||
opts.src,
|
||||
opts.dst,
|
||||
opts.srcPort,
|
||||
opts.dstPort,
|
||||
destCtxForIface(opts.topo, opts.serverId, opts.inIface),
|
||||
)
|
||||
const ripe = dest ? resolveFlowIp(dest) : null
|
||||
const classified = classifyFlowDst(dest || opts.dst, opts.proto, opts.dstPort, opts.srcPort, ripe)
|
||||
if (!dest) {
|
||||
return { dest: "", ripe: null, classified, country: "", asn: 0 }
|
||||
}
|
||||
const country = ripe?.ok && isIsoCountry(ripe.country)
|
||||
? ripe.country
|
||||
: (ripe?.ok ? "" : "unknown")
|
||||
const asn = ripe?.ok && ripe.asn ? ripe.asn : 0
|
||||
return { dest, ripe, classified, country, asn }
|
||||
}
|
||||
@@ -4,12 +4,25 @@ import { normalizeParsedFlow, parseFlowPacket, protoName, type ParsedFlow, type
|
||||
import { classifyFlowPlaneLite } from "./traffic-flow-planes.js"
|
||||
import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js"
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { enqueueRipeMisses, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import {
|
||||
getServerCatalog,
|
||||
loadFlowTopology,
|
||||
peekFlowTopology,
|
||||
peekServerCatalog,
|
||||
} from "./traffic-flow-topology.js"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
factsPendingSize,
|
||||
flushFlowFacts,
|
||||
hourBucketIso,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
export const TICK_MS = 2_000
|
||||
export const PERSIST_MS = 10_000
|
||||
@@ -327,20 +340,33 @@ export function getEngineStats(): EngineStats {
|
||||
export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): void {
|
||||
if (flows.length) bumpDataEpoch()
|
||||
const bucketAt = minuteBucketIso()
|
||||
const hourAt = hourBucketIso()
|
||||
const ripeMisses: string[] = []
|
||||
const topo = peekFlowTopology()
|
||||
const catalog = peekServerCatalog()
|
||||
if (!topo) void loadFlowTopology().catch(() => {})
|
||||
if (!catalog) void getServerCatalog().catch(() => {})
|
||||
const serverType = catalog?.byId.get(serverId)?.type
|
||||
for (const raw of flows) {
|
||||
const flow = normalizeParsedFlow(raw)
|
||||
addToTick(serverId, flow, flow.bytes)
|
||||
bumpRollup(serverId, bucketAt, flow, flow.bytes, flow.packets)
|
||||
const peer = pickInternetPeer(flow.src, flow.dst, flow.srcPort, flow.dstPort)
|
||||
const ripe = lookupRipeCached(peer)
|
||||
if (peer && !ripe) ripeMisses.push(peer)
|
||||
const classified = classifyFlowDst(peer, flow.proto, flow.dstPort, flow.srcPort, ripe)
|
||||
const destMeta = resolveInternetDest({
|
||||
src: flow.src,
|
||||
dst: flow.dst,
|
||||
proto: flow.proto,
|
||||
srcPort: flow.srcPort,
|
||||
dstPort: flow.dstPort,
|
||||
serverId,
|
||||
inIface: flow.inIface,
|
||||
topo,
|
||||
})
|
||||
const ripe = destMeta.ripe
|
||||
if (destMeta.dest && !ripe) ripeMisses.push(destMeta.dest)
|
||||
const classified = destMeta.classified
|
||||
const app = applicationName(flow.proto, flow.dstPort, flow.srcPort)
|
||||
const country = ripe?.ok && isIsoCountry(ripe.country)
|
||||
? ripe.country
|
||||
: (ripe?.ok ? "" : "unknown")
|
||||
const asnKey = ripe?.ok && ripe.asn ? String(ripe.asn) : "unknown"
|
||||
const country = destMeta.country
|
||||
const asnKey = destMeta.asn ? String(destMeta.asn) : "unknown"
|
||||
bumpDim(serverId, bucketAt, "proto", protoName(flow.proto), flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "app", app, flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "iface", flow.inIface || "__unknown__", flow.bytes, flow.packets)
|
||||
@@ -348,6 +374,29 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
|
||||
bumpDim(serverId, bucketAt, "service", classified.service, flow.bytes, flow.packets)
|
||||
if (country) bumpDim(serverId, bucketAt, "country", country, flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "asn", asnKey, flow.bytes, flow.packets)
|
||||
if (shouldWriteFlowFact({
|
||||
serverId,
|
||||
serverType,
|
||||
inIface: flow.inIface,
|
||||
outIface: flow.outIface,
|
||||
proto: flow.proto,
|
||||
srcPort: flow.srcPort,
|
||||
dstPort: flow.dstPort,
|
||||
src: flow.src,
|
||||
dst: flow.dst,
|
||||
topo,
|
||||
})) {
|
||||
bumpFlowFact({
|
||||
serverId,
|
||||
bucketAt: hourAt,
|
||||
iface: canonicalFactIface(serverId, flow.inIface),
|
||||
country: country || "XX",
|
||||
service: classified.service,
|
||||
asn: destMeta.asn,
|
||||
bytes: flow.bytes,
|
||||
packets: flow.packets,
|
||||
})
|
||||
}
|
||||
|
||||
const key = pendingKey(serverId, bucketAt, flow)
|
||||
const prev = pending.get(key)
|
||||
@@ -822,11 +871,12 @@ async function upsertFlowBuckets(rows: PendingFlowRow[]): Promise<number> {
|
||||
}
|
||||
}
|
||||
|
||||
export async function flushPending(opts?: { force?: boolean }): Promise<void> {
|
||||
export async function flushPending(opts?: { force?: boolean; prune?: boolean }): Promise<void> {
|
||||
pruneRecent()
|
||||
rollFlowRings()
|
||||
const force = Boolean(opts?.force)
|
||||
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0
|
||||
const doPrune = opts?.prune !== false
|
||||
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0 || factsPendingSize() > 0
|
||||
const due = persistDue(force, hasWork)
|
||||
try {
|
||||
await persistListenerStats(force)
|
||||
@@ -835,7 +885,7 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
|
||||
}
|
||||
|
||||
if (!hasWork) {
|
||||
if (force) {
|
||||
if (force && doPrune) {
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
@@ -885,9 +935,16 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
|
||||
/* rollup best-effort */
|
||||
}
|
||||
try {
|
||||
await pruneStored()
|
||||
await flushFlowFacts()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
/* statistics cube best-effort */
|
||||
}
|
||||
if (doPrune) {
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -895,8 +952,12 @@ export function lastFlushUsedTransactionForTests(): boolean {
|
||||
return lastFlushUsedTransaction
|
||||
}
|
||||
|
||||
export async function flushEngineNow(opts?: { prune?: boolean }): Promise<void> {
|
||||
await flushPending({ force: true, prune: opts?.prune })
|
||||
}
|
||||
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending({ force: true })
|
||||
await flushEngineNow()
|
||||
}
|
||||
|
||||
export function onEngineTick(): void {
|
||||
@@ -915,6 +976,7 @@ export function resetEngineForTests(): void {
|
||||
rings.clear()
|
||||
minuteRollup.clear()
|
||||
minuteDims.clear()
|
||||
resetFactsForTests()
|
||||
packetsReceived = 0
|
||||
lastExporterIp = null
|
||||
lastError = ""
|
||||
@@ -939,3 +1001,25 @@ export function pendingSizeForTests(): number {
|
||||
export function droppedForTests(): number {
|
||||
return dropped
|
||||
}
|
||||
|
||||
/** Снимок минутных dims (dim → key → bytes) для тестов обогащения потоков. */
|
||||
export function minuteDimsSnapshotForTests(): Map<string, Map<string, { bytes: number; packets: number }>> {
|
||||
const out = new Map<string, Map<string, { bytes: number; packets: number }>>()
|
||||
for (const [k, acc] of minuteDims) {
|
||||
// dimKey: serverId\0bucketAt\0dim\0key
|
||||
const parts = k.split("\0")
|
||||
const dim = parts[2] ?? ""
|
||||
const key = parts.slice(3).join("\0")
|
||||
let byKey = out.get(dim)
|
||||
if (!byKey) {
|
||||
byKey = new Map()
|
||||
out.set(dim, byKey)
|
||||
}
|
||||
const prev = byKey.get(key)
|
||||
byKey.set(key, {
|
||||
bytes: (prev?.bytes ?? 0) + acc.bytes,
|
||||
packets: (prev?.packets ?? 0) + acc.packets,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
isJunkFactIface,
|
||||
isOverlayGreIface,
|
||||
isOverlayTunnelIface,
|
||||
isWanFactIface,
|
||||
shouldWriteFlowFact,
|
||||
} from "./traffic-flow-facts-filter.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
|
||||
function topo(partial: Partial<FlowTopology> = {}): FlowTopology {
|
||||
const wanIfaces = partial.wanIfaces ?? new Map([[1, new Set(["ether1"])]])
|
||||
const clientIfaces = partial.clientIfaces ?? new Map([[1, new Set(["gre-client"])]])
|
||||
const clientByIface = partial.clientByIface ?? new Map()
|
||||
const enHosts = partial.enHosts ?? new Set(["198.51.100.1"])
|
||||
const jhHosts = partial.jhHosts ?? new Set(["203.0.113.10"])
|
||||
return {
|
||||
clientIfaces,
|
||||
clientByIface,
|
||||
enNodes: partial.enNodes ?? [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
|
||||
enHosts,
|
||||
jhHosts,
|
||||
wanIfaces,
|
||||
tunnelIfaces: partial.tunnelIfaces,
|
||||
plane: partial.plane ?? {
|
||||
clientIfaceNames: new Set(["gre-client"]),
|
||||
enHosts,
|
||||
jhHosts,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
rememberServerIfaces(1, [{ name: "ether1", ifindex: "2" }])
|
||||
seedFlowTopologyForTests(topo())
|
||||
|
||||
assert.equal(isJunkFactIface("0"), true)
|
||||
assert.equal(isJunkFactIface(""), true)
|
||||
assert.equal(isJunkFactIface("wg-flow"), true)
|
||||
assert.equal(isJunkFactIface("ether1"), false)
|
||||
assert.equal(isWanFactIface(topo(), 1, "ether1"), true)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "gre-en"), true)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "gre-client"), false)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "ether1"), false)
|
||||
|
||||
const typed = topo({
|
||||
clientIfaces: new Map([[1, new Set(["gre-client", "wg-server"])]]),
|
||||
tunnelIfaces: new Map([[1, new Set(["gre-en", "NSK-SERVHOST-RTK", "wg-jh-en", "wg-server"])]]),
|
||||
plane: {
|
||||
clientIfaceNames: new Set(["gre-client", "wg-server"]),
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
},
|
||||
})
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "NSK-SERVHOST-RTK"), true, "кастомное GRE overlay по type")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-jh-en"), true, "WG overlay по type")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-server"), false, "клиентский WG с binding")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-flow"), false, "wg-flow не overlay")
|
||||
assert.equal(isOverlayTunnelIface(topo(), 1, "NSK-SERVHOST-RTK"), false, "без type в снимке — не overlay")
|
||||
|
||||
const overlayOuter = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "ether1",
|
||||
outIface: "gre-en",
|
||||
proto: 47,
|
||||
srcPort: 0,
|
||||
dstPort: 0,
|
||||
src: "203.0.113.10",
|
||||
dst: "198.51.100.1",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(overlayOuter, false, "overlay proto 47 на ether1 не в facts")
|
||||
|
||||
const payloadGre = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "gre-client",
|
||||
outIface: "gre-en",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(payloadGre, true, "payload на GRE — да")
|
||||
|
||||
const payloadWan = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "ether1",
|
||||
outIface: "gre-client",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 51234,
|
||||
src: "8.8.8.8",
|
||||
dst: "10.100.1.17",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(payloadWan, true, "payload на ether1 WAN — да")
|
||||
|
||||
const junkZero = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "0",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 80,
|
||||
src: "1.1.1.1",
|
||||
dst: "8.8.8.8",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(junkZero, false)
|
||||
|
||||
const enTopo = topo({
|
||||
clientIfaces: new Map([[2, new Set()]]),
|
||||
wanIfaces: new Map([[2, new Set(["ether1"])]]),
|
||||
})
|
||||
const enTransit = shouldWriteFlowFact({
|
||||
serverId: 2,
|
||||
serverType: "exit-node",
|
||||
inIface: "gre-jh",
|
||||
outIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
topo: enTopo,
|
||||
})
|
||||
assert.equal(enTransit, false, "EN-транзит без клиента — нет")
|
||||
|
||||
const enWan = shouldWriteFlowFact({
|
||||
serverId: 2,
|
||||
serverType: "exit-node",
|
||||
inIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 80,
|
||||
src: "8.8.8.8",
|
||||
dst: "198.51.100.1",
|
||||
topo: enTopo,
|
||||
})
|
||||
assert.equal(enWan, true, "WAN payload на EN — да")
|
||||
|
||||
seedFlowTopologyForTests(null)
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-facts-filter.test.ts: ok")
|
||||
@@ -0,0 +1,119 @@
|
||||
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
|
||||
import { STATISTICS_DUP_MARK, STATISTICS_WAN_MARK } from "@mmapp/contracts/statistics"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { classifyFlowPlane } from "./traffic-flow-planes.js"
|
||||
import { resolveClient, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
const JUNK_IFACE = new Set(["", "0", "—", "__unknown__", "wg-flow"])
|
||||
|
||||
export { STATISTICS_WAN_MARK, STATISTICS_DUP_MARK }
|
||||
|
||||
export function isJunkFactIface(iface: string | null | undefined): boolean {
|
||||
const n = String(iface ?? "").trim()
|
||||
if (JUNK_IFACE.has(n)) return true
|
||||
return /^#?0$/.test(n)
|
||||
}
|
||||
|
||||
export function isDashDisplayIface(iface: string): boolean {
|
||||
return String(iface ?? "").trim() === "—"
|
||||
}
|
||||
|
||||
export function isMgmtIface(name: string): boolean {
|
||||
const n = String(name ?? "").trim().toLowerCase()
|
||||
return n === "wg-flow" || n.endsWith("/wg-flow") || n.includes("wg-flow")
|
||||
}
|
||||
|
||||
/** GRE или WG по снимку RouterOS, иначе по имени. */
|
||||
export function isTunnelIfaceName(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isJunkFactIface(name) || isMgmtIface(name)) return false
|
||||
const typed = topo?.tunnelIfaces?.get(serverId)
|
||||
if (typed && typed.size > 0) return typed.has(name)
|
||||
const t = mapRosInterfaceType("", name)
|
||||
return t === "gre" || t === "wg"
|
||||
}
|
||||
|
||||
/** WAN uplink: `wanIfaces` топологии, иначе ether1 у JH/EN без wan_uplinks. */
|
||||
export function isWanFactIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isJunkFactIface(name) || isDashDisplayIface(name)) return false
|
||||
const wan = topo?.wanIfaces.get(serverId)
|
||||
if (wan && wan.size > 0) return wan.has(name)
|
||||
return /^ether1$/i.test(name)
|
||||
}
|
||||
|
||||
/** Overlay JH↔EN: GRE/WG не клиент, не WAN, не wg-flow. */
|
||||
export function isOverlayTunnelIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isWanFactIface(topo, serverId, name) || isMgmtIface(name)) return false
|
||||
if (topo?.clientIfaces.get(serverId)?.has(name)) return false
|
||||
return isTunnelIfaceName(topo, serverId, name)
|
||||
}
|
||||
|
||||
/** @deprecated используйте isOverlayTunnelIface (GRE и WG). */
|
||||
export function isOverlayGreIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
return isOverlayTunnelIface(topo, serverId, iface)
|
||||
}
|
||||
|
||||
export function shouldWriteFlowFact(opts: {
|
||||
serverId: number
|
||||
serverType?: string
|
||||
inIface: string
|
||||
outIface?: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
src: string
|
||||
dst: string
|
||||
topo?: FlowTopology | null
|
||||
}): boolean {
|
||||
const inName = canonicalFactIface(opts.serverId, opts.inIface) || String(opts.inIface ?? "").trim()
|
||||
if (isJunkFactIface(inName) || isJunkFactIface(opts.inIface)) return false
|
||||
const outRaw = String(opts.outIface ?? "").trim()
|
||||
const outName = outRaw ? (canonicalFactIface(opts.serverId, outRaw) || outRaw) : ""
|
||||
const plane = classifyFlowPlane({
|
||||
src: opts.src,
|
||||
dst: opts.dst,
|
||||
proto: opts.proto,
|
||||
srcPort: opts.srcPort,
|
||||
dstPort: opts.dstPort,
|
||||
inIface: inName,
|
||||
outIface: outName || undefined,
|
||||
}, opts.topo?.plane)
|
||||
if (plane !== "payload") return false
|
||||
if (opts.serverType === "exit-node" && opts.topo) {
|
||||
const client =
|
||||
resolveClient(opts.topo, opts.serverId, inName)
|
||||
?? (outName ? resolveClient(opts.topo, opts.serverId, outName) : null)
|
||||
if (!client && isOverlayTunnelIface(opts.topo, opts.serverId, inName)) return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
export function wanIfaceLabel(serverName: string, iface: string): string {
|
||||
return `${serverName} · ${iface} · ${STATISTICS_WAN_MARK}`
|
||||
}
|
||||
|
||||
export function overlayDupLabel(serverName: string, iface: string): string {
|
||||
return `${serverName} · ${iface} · ${STATISTICS_DUP_MARK}`
|
||||
}
|
||||
|
||||
export function isNonUniqueShareLabel(label: string): boolean {
|
||||
return label.includes(STATISTICS_WAN_MARK) || label.includes(`· ${STATISTICS_DUP_MARK}`)
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { ensurePartitionFor } from "../db/partitions.js"
|
||||
import { pool } from "../db/index.js"
|
||||
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
import { resetEngineForTests } from "./traffic-flow-engine.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-facts-rebuild.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const nServers = (await dbQuery<{ n: number }>(`SELECT COUNT(*)::int AS n FROM servers`)).rows[0]?.n ?? 0
|
||||
if (nServers > 10) {
|
||||
console.warn("traffic-flow-facts-rebuild.test.ts: skip (не пустая БД)")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetEngineForTests()
|
||||
resetIfaceCacheForTests()
|
||||
|
||||
seedRipeCacheForTests({
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "95.167.0.0/16",
|
||||
asn: 12389,
|
||||
country: "RU",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "ROSTELECOM-AS",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
|
||||
const inserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type, wan_uplinks)
|
||||
VALUES ('rebuild-facts-jh', '203.0.113.10', 'jump-host', '[{"iface":"ether1"}]'::jsonb)
|
||||
RETURNING id
|
||||
`)
|
||||
const serverId = inserted.rows[0]?.id
|
||||
if (serverId == null) throw new Error("no server")
|
||||
|
||||
const ts = new Date()
|
||||
await ensurePartitionFor(pool, "flow_buckets", "day", ts)
|
||||
await ensurePartitionFor(pool, "flow_hour_facts", "day", ts)
|
||||
await ensurePartitionFor(pool, "flow_daily_facts", "month", ts)
|
||||
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login, role, active)
|
||||
VALUES ('u-rebuild-1', 'Клиент', 'rebuild-user', 'viewer', TRUE)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-rebuild-1', 'u-rebuild-1', $1, 'gre-client', 'gre')
|
||||
`, [serverId])
|
||||
await dbQuery(`
|
||||
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
|
||||
VALUES ($1, now(), 'online', $2::jsonb)
|
||||
`, [serverId, JSON.stringify([{ name: "gre-client", type: "gre-tunnel" }, { name: "ether1", type: "ether" }])])
|
||||
|
||||
rememberServerIfaces(serverId, [{ name: "gre-client", ifindex: "2" }])
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
const bucketAt = new Date(Date.UTC(
|
||||
ts.getUTCFullYear(),
|
||||
ts.getUTCMonth(),
|
||||
ts.getUTCDate(),
|
||||
ts.getUTCHours(),
|
||||
0, 0, 0,
|
||||
)).toISOString()
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_buckets (server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface)
|
||||
VALUES
|
||||
($1, $2, '95.167.1.10', '10.200.100.53', 6, 51234, 443, 100, 2, 'gre-client', 'ether1'),
|
||||
($1, $2, '95.167.1.10', '8.8.8.8', 6, 51234, 443, 50, 1, 'gre-client', 'ether1')
|
||||
`, [serverId, bucketAt])
|
||||
|
||||
try {
|
||||
const { rebuildFlowFactsFromBuckets } = await import("./traffic-flow-facts-rebuild.js")
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
assert.equal(result.ok, true)
|
||||
assert.ok(result.buckets >= 2)
|
||||
|
||||
const rows = await dbQuery<{ asn: number; bytes: number }>(`
|
||||
SELECT asn, SUM(bytes)::bigint AS bytes
|
||||
FROM flow_hour_facts
|
||||
WHERE server_id = $1
|
||||
GROUP BY asn
|
||||
`, [serverId])
|
||||
const byAsn = new Map(rows.rows.map((r) => [Number(r.asn), Number(r.bytes)]))
|
||||
const total = [...byAsn.values()].reduce((s, n) => s + n, 0)
|
||||
assert.equal(total, 150)
|
||||
assert.equal(byAsn.get(12389), undefined, "ASN клиента не в hour facts")
|
||||
assert.equal(byAsn.get(15169), 50)
|
||||
assert.equal(byAsn.get(0), 100)
|
||||
} finally {
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
|
||||
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
|
||||
resetEngineForTests()
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-facts-rebuild.test.ts: ok")
|
||||
@@ -0,0 +1,130 @@
|
||||
import { dbAll, dbGet, dbQuery, withAdvisoryLock } from "../db/index.js"
|
||||
import { flushEngineNow } from "./traffic-flow-engine.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
discardPendingFacts,
|
||||
flushFlowFacts,
|
||||
hourBucketIso,
|
||||
} from "./traffic-flow-facts.js"
|
||||
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { getServerCatalog, loadFlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
export const FACT_REBUILD_LOCK_KEY = 8_723_104
|
||||
const BATCH = 4_000
|
||||
|
||||
export interface FlowFactsRebuildResult {
|
||||
ok: true
|
||||
buckets: number
|
||||
facts: number
|
||||
days: string[]
|
||||
}
|
||||
|
||||
function hourFromBucket(raw: Date | string): string {
|
||||
const iso = raw instanceof Date ? raw.toISOString() : String(raw)
|
||||
const ms = new Date(iso).getTime()
|
||||
return hourBucketIso(Number.isFinite(ms) ? ms : Date.now())
|
||||
}
|
||||
|
||||
export async function rebuildFlowFactsFromBuckets(): Promise<FlowFactsRebuildResult> {
|
||||
return await withAdvisoryLock(FACT_REBUILD_LOCK_KEY, async () => {
|
||||
await flushEngineNow({ prune: false })
|
||||
discardPendingFacts()
|
||||
|
||||
const days = await dbAll<{ day: string }>(`
|
||||
SELECT DISTINCT (bucket_at AT TIME ZONE 'UTC')::date::text AS day
|
||||
FROM flow_buckets
|
||||
ORDER BY 1
|
||||
`)
|
||||
const dayList = days.map((r) => r.day).filter(Boolean)
|
||||
if (dayList.length === 0) {
|
||||
return { ok: true as const, buckets: 0, facts: 0, days: [] }
|
||||
}
|
||||
|
||||
await dbQuery(
|
||||
`DELETE FROM flow_hour_facts WHERE (bucket_at AT TIME ZONE 'UTC')::date = ANY(?::date[])`,
|
||||
[dayList],
|
||||
)
|
||||
await dbQuery(
|
||||
`DELETE FROM flow_daily_facts WHERE day = ANY(?::date[])`,
|
||||
[dayList],
|
||||
)
|
||||
|
||||
const topo = await loadFlowTopology()
|
||||
const catalog = await getServerCatalog()
|
||||
let offset = 0
|
||||
let buckets = 0
|
||||
|
||||
for (;;) {
|
||||
const rows = await dbAll<{
|
||||
serverId: number
|
||||
bucketAt: Date | string
|
||||
src: string
|
||||
dst: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
bytes: number
|
||||
packets: number
|
||||
inIface: string
|
||||
outIface: string
|
||||
}>(`
|
||||
SELECT server_id AS "serverId", bucket_at AS "bucketAt",
|
||||
host(src) AS src, host(dst) AS dst, proto, src_port AS "srcPort", dst_port AS "dstPort",
|
||||
bytes, packets, in_iface AS "inIface", COALESCE(out_iface, '') AS "outIface"
|
||||
FROM flow_buckets
|
||||
ORDER BY bucket_at, server_id
|
||||
LIMIT ? OFFSET ?
|
||||
`, [BATCH, offset])
|
||||
if (rows.length === 0) break
|
||||
for (const row of rows) {
|
||||
buckets += 1
|
||||
const serverType = catalog.byId.get(row.serverId)?.type
|
||||
if (!shouldWriteFlowFact({
|
||||
serverId: row.serverId,
|
||||
serverType,
|
||||
inIface: row.inIface,
|
||||
outIface: row.outIface,
|
||||
proto: Number(row.proto) || 0,
|
||||
srcPort: Number(row.srcPort) || 0,
|
||||
dstPort: Number(row.dstPort) || 0,
|
||||
src: row.src,
|
||||
dst: row.dst,
|
||||
topo,
|
||||
})) continue
|
||||
const destMeta = resolveInternetDest({
|
||||
src: row.src,
|
||||
dst: row.dst,
|
||||
proto: Number(row.proto) || 0,
|
||||
srcPort: Number(row.srcPort) || 0,
|
||||
dstPort: Number(row.dstPort) || 0,
|
||||
serverId: row.serverId,
|
||||
inIface: row.inIface,
|
||||
topo,
|
||||
})
|
||||
bumpFlowFact({
|
||||
serverId: row.serverId,
|
||||
bucketAt: hourFromBucket(row.bucketAt),
|
||||
iface: canonicalFactIface(row.serverId, row.inIface),
|
||||
country: destMeta.country || "XX",
|
||||
service: destMeta.classified.service,
|
||||
asn: destMeta.asn,
|
||||
bytes: Number(row.bytes) || 0,
|
||||
packets: Number(row.packets) || 0,
|
||||
})
|
||||
}
|
||||
offset += rows.length
|
||||
if (rows.length < BATCH) break
|
||||
}
|
||||
|
||||
const facts = await flushFlowFacts()
|
||||
const range = await dbGet<{ n: number }>(`SELECT COUNT(*)::int AS n FROM flow_buckets`)
|
||||
return {
|
||||
ok: true as const,
|
||||
buckets: Number(range?.n) || buckets,
|
||||
facts,
|
||||
days: dayList,
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
collectCappedFacts,
|
||||
FACT_ASN_TOP,
|
||||
FACT_TUPLE_CAP,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
resetFactsForTests()
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 100,
|
||||
packets: 2,
|
||||
})
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "us",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 50,
|
||||
packets: 1,
|
||||
})
|
||||
const merged = collectCappedFacts()
|
||||
assert.equal(merged.length, 1)
|
||||
assert.equal(merged[0]?.bytes, 150)
|
||||
assert.equal(merged[0]?.country, "US")
|
||||
assert.equal(merged[0]?.asn, 15169)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 1; i <= FACT_ASN_TOP + 20; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 2,
|
||||
bucketAt: "2026-09-10T11:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "DE",
|
||||
service: "https",
|
||||
asn: i,
|
||||
bytes: FACT_ASN_TOP + 21 - i,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedAsn = collectCappedFacts()
|
||||
const asns = new Set(cappedAsn.map((r) => r.asn))
|
||||
assert.ok(asns.has(0))
|
||||
assert.ok(asns.size <= FACT_ASN_TOP + 1)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 0; i < FACT_TUPLE_CAP + 30; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 3,
|
||||
bucketAt: "2026-09-10T12:00:00.000Z",
|
||||
iface: `ether${i % 3}`,
|
||||
country: "NL",
|
||||
service: `svc-${i}`,
|
||||
asn: 1,
|
||||
bytes: 10,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedTuples = collectCappedFacts()
|
||||
assert.ok(cappedTuples.length <= FACT_TUPLE_CAP + 3)
|
||||
|
||||
console.log("traffic-flow-facts.test.ts: ok")
|
||||
@@ -0,0 +1,289 @@
|
||||
import { pool } from "../db/index.js"
|
||||
import { ensurePartitionFor, specForParent } from "../db/partitions.js"
|
||||
|
||||
export const FACT_ASN_TOP = 200
|
||||
export const FACT_TUPLE_CAP = 8000
|
||||
export const FACT_SERVICE_MAX_LEN = 64
|
||||
export const UNKNOWN_COUNTRY = "XX"
|
||||
export const OTHER_SERVICE = "other"
|
||||
export const UNKNOWN_IFACE = "__unknown__"
|
||||
|
||||
export interface FactAcc {
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
export interface FactRow {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
const hourFacts = new Map<string, FactAcc>()
|
||||
const ensuredParts = new Set<string>()
|
||||
|
||||
export function hourBucketIso(at = Date.now()): string {
|
||||
const d = new Date(at)
|
||||
d.setMinutes(0, 0, 0)
|
||||
return d.toISOString()
|
||||
}
|
||||
|
||||
export function normalizeFactCountry(raw: string): string {
|
||||
const iso = raw.trim().toUpperCase()
|
||||
if (/^[A-Z]{2}$/.test(iso)) return iso
|
||||
return UNKNOWN_COUNTRY
|
||||
}
|
||||
|
||||
export function normalizeFactService(raw: string): string {
|
||||
const s = raw.trim().slice(0, FACT_SERVICE_MAX_LEN)
|
||||
return s || OTHER_SERVICE
|
||||
}
|
||||
|
||||
export function normalizeFactIface(raw: string): string {
|
||||
return raw.trim() || UNKNOWN_IFACE
|
||||
}
|
||||
|
||||
export function normalizeFactAsn(raw: number): number {
|
||||
if (!Number.isFinite(raw) || raw <= 0) return 0
|
||||
return Math.trunc(raw)
|
||||
}
|
||||
|
||||
function factKey(
|
||||
serverId: number,
|
||||
bucketAt: string,
|
||||
iface: string,
|
||||
country: string,
|
||||
service: string,
|
||||
asn: number,
|
||||
): string {
|
||||
return `${serverId}\0${bucketAt}\0${iface}\0${country}\0${service}\0${asn}`
|
||||
}
|
||||
|
||||
function parseFactKey(k: string, acc: FactAcc): FactRow | null {
|
||||
const parts = k.split("\0")
|
||||
if (parts.length !== 6) return null
|
||||
const serverId = Number(parts[0])
|
||||
const asn = Number(parts[5])
|
||||
if (!Number.isFinite(serverId) || !Number.isFinite(asn)) return null
|
||||
return {
|
||||
serverId,
|
||||
bucketAt: parts[1] ?? "",
|
||||
iface: parts[2] ?? UNKNOWN_IFACE,
|
||||
country: parts[3] ?? UNKNOWN_COUNTRY,
|
||||
service: parts[4] ?? OTHER_SERVICE,
|
||||
asn,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
}
|
||||
}
|
||||
|
||||
export function bumpFlowFact(row: {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}): void {
|
||||
const iface = normalizeFactIface(row.iface)
|
||||
const country = normalizeFactCountry(row.country)
|
||||
const service = normalizeFactService(row.service)
|
||||
const asn = normalizeFactAsn(row.asn)
|
||||
const k = factKey(row.serverId, row.bucketAt, iface, country, service, asn)
|
||||
const prev = hourFacts.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
hourFacts.set(k, { bytes: row.bytes, packets: row.packets })
|
||||
}
|
||||
|
||||
function groupKey(row: FactRow): string {
|
||||
return `${row.serverId}\0${row.bucketAt}`
|
||||
}
|
||||
|
||||
function mergeRow(map: Map<string, FactRow>, row: FactRow): void {
|
||||
const k = factKey(row.serverId, row.bucketAt, row.iface, row.country, row.service, row.asn)
|
||||
const prev = map.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
map.set(k, { ...row })
|
||||
}
|
||||
|
||||
/** Cap ASN tail and tuple count per server×hour before persist. */
|
||||
export function collectCappedFacts(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
hourFacts.clear()
|
||||
|
||||
const groups = new Map<string, FactRow[]>()
|
||||
for (const row of parsed) {
|
||||
const g = groupKey(row)
|
||||
const list = groups.get(g) ?? []
|
||||
list.push(row)
|
||||
groups.set(g, list)
|
||||
}
|
||||
|
||||
const out = new Map<string, FactRow>()
|
||||
for (const list of groups.values()) {
|
||||
const byAsn = new Map<number, number>()
|
||||
for (const row of list) {
|
||||
byAsn.set(row.asn, (byAsn.get(row.asn) ?? 0) + row.bytes)
|
||||
}
|
||||
const asnKeep = new Set(
|
||||
[...byAsn.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, FACT_ASN_TOP)
|
||||
.map(([asn]) => asn),
|
||||
)
|
||||
const afterAsn: FactRow[] = []
|
||||
for (const row of list) {
|
||||
if (asnKeep.has(row.asn) || row.asn === 0) {
|
||||
afterAsn.push(row)
|
||||
continue
|
||||
}
|
||||
afterAsn.push({ ...row, asn: 0 })
|
||||
}
|
||||
const collapsed = new Map<string, FactRow>()
|
||||
for (const row of afterAsn) mergeRow(collapsed, row)
|
||||
const tuples = [...collapsed.values()].sort((a, b) => b.bytes - a.bytes)
|
||||
const keep = tuples.slice(0, FACT_TUPLE_CAP)
|
||||
const tail = tuples.slice(FACT_TUPLE_CAP)
|
||||
for (const row of keep) mergeRow(out, row)
|
||||
for (const row of tail) {
|
||||
mergeRow(out, {
|
||||
...row,
|
||||
country: UNKNOWN_COUNTRY,
|
||||
service: OTHER_SERVICE,
|
||||
asn: 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
return [...out.values()]
|
||||
}
|
||||
|
||||
async function ensureParentPartition(parent: string, ts: string): Promise<void> {
|
||||
const spec = specForParent(parent)
|
||||
if (!spec) return
|
||||
const iso = ts.length === 10 ? `${ts}T00:00:00Z` : ts
|
||||
const key = `${parent}:${iso.slice(0, 10)}`
|
||||
if (ensuredParts.has(key)) return
|
||||
await ensurePartitionFor(pool, parent, spec.kind, new Date(iso))
|
||||
ensuredParts.add(key)
|
||||
}
|
||||
|
||||
function dayKey(bucketAt: string): string {
|
||||
return bucketAt.slice(0, 10)
|
||||
}
|
||||
|
||||
export async function flushFlowFacts(): Promise<number> {
|
||||
const rows = collectCappedFacts()
|
||||
if (rows.length === 0) return 0
|
||||
const hours = new Set(rows.map((r) => r.bucketAt))
|
||||
const days = new Set(rows.map((r) => dayKey(r.bucketAt)))
|
||||
for (const h of hours) await ensureParentPartition("flow_hour_facts", h)
|
||||
for (const d of days) await ensureParentPartition("flow_daily_facts", d)
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_hour_facts (
|
||||
server_id, bucket_at, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::timestamptz[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, bucket_at, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_hour_facts.bytes + excluded.bytes,
|
||||
packets = flow_hour_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => r.bucketAt),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_daily_facts (
|
||||
server_id, day, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::date[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, day, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, day, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_daily_facts.bytes + excluded.bytes,
|
||||
packets = flow_daily_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => dayKey(r.bucketAt)),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
return rows.length
|
||||
}
|
||||
|
||||
export function factsPendingSize(): number {
|
||||
return hourFacts.size
|
||||
}
|
||||
|
||||
export function resetFactsForTests(): void {
|
||||
hourFacts.clear()
|
||||
ensuredParts.clear()
|
||||
}
|
||||
|
||||
export function discardPendingFacts(): void {
|
||||
hourFacts.clear()
|
||||
}
|
||||
|
||||
export function factsSnapshotForTests(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"
|
||||
import { tmpdir } from "node:os"
|
||||
import path from "node:path"
|
||||
import type { AsnResponse, CountryResponse, Reader } from "maxmind"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import {
|
||||
lookupGeoip,
|
||||
resetGeoipForTests,
|
||||
resolveFlowIp,
|
||||
setGeoipReadersForTests,
|
||||
} from "./traffic-flow-geoip.js"
|
||||
import {
|
||||
resetEngineForTests,
|
||||
ingestParsedFlowsForServerForTests,
|
||||
minuteDimsSnapshotForTests,
|
||||
} from "./traffic-flow-engine.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { disableGeoipDbForTests } from "./geoip-settings.js"
|
||||
import {
|
||||
collectGeoipUpdateOnce,
|
||||
resetGeoipUpdateForTests,
|
||||
setGeoipFetchForTests,
|
||||
setGeoipValidateForTests,
|
||||
} from "./geoip-update-collector.js"
|
||||
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetGeoipForTests()
|
||||
|
||||
// ── lookupGeoip: приватные IP → negative без ридеров ──────────────────────────
|
||||
assert.equal(lookupGeoip("10.1.1.8")?.ok, false)
|
||||
assert.equal(lookupGeoip("192.168.0.1")?.prefix, "192.168.0.1/32")
|
||||
assert.equal(lookupGeoip("100.64.1.2")?.ok, false)
|
||||
assert.equal(lookupGeoip("fe80::1")?.prefix, "fe80::1/128")
|
||||
|
||||
// ── без ридеров публичный IP → null, resolveFlowIp уходит в RIPE-кэш ─────────
|
||||
assert.equal(lookupGeoip("1.2.3.10"), null)
|
||||
seedRipeCacheForTests({
|
||||
prefix: "1.2.3.0/24",
|
||||
asn: 64500,
|
||||
country: "NL",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "TEST",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.country, "NL")
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
|
||||
|
||||
// ── fake-ридеры: geoip приоритетнее RIPE ──────────────────────────────────────
|
||||
function fakeCountryReader(byIp: Record<string, string>): Reader<CountryResponse> {
|
||||
return {
|
||||
get(ip: string) {
|
||||
const iso = byIp[ip]
|
||||
return iso ? ({ country: { iso_code: iso } } as CountryResponse) : null
|
||||
},
|
||||
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
|
||||
} as unknown as Reader<CountryResponse>
|
||||
}
|
||||
|
||||
function fakeAsnReader(byIp: Record<string, { asn: number; org: string }>): Reader<AsnResponse> {
|
||||
return {
|
||||
get(ip: string) {
|
||||
const hit = byIp[ip]
|
||||
return hit
|
||||
? ({ autonomous_system_number: hit.asn, autonomous_system_organization: hit.org } as AsnResponse)
|
||||
: null
|
||||
},
|
||||
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
|
||||
} as unknown as Reader<AsnResponse>
|
||||
}
|
||||
|
||||
setGeoipReadersForTests({
|
||||
country: fakeCountryReader({ "8.8.8.8": "US", "6.6.6.6": "EU" }),
|
||||
asn: fakeAsnReader({
|
||||
"8.8.8.8": { asn: 15169, org: "GOOGLE" },
|
||||
"6.6.6.6": { asn: 15169, org: "GOOGLE" },
|
||||
}),
|
||||
})
|
||||
|
||||
const hit = resolveFlowIp("8.8.8.8")
|
||||
assert.equal(hit?.country, "US")
|
||||
assert.equal(hit?.asn, 15169)
|
||||
assert.equal(hit?.holder, "GOOGLE")
|
||||
assert.equal(hit?.ok, true)
|
||||
|
||||
// 1.2.3.10 в fake-ридерах нет — по-прежнему из RIPE-кэша
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
|
||||
|
||||
// EU не ISO-страна: отфильтрована, страна выведена из ASN (HQ Google → US)
|
||||
assert.equal(lookupGeoip("6.6.6.6")?.country, "US")
|
||||
|
||||
// geoip-мета совместима с classifyFlowDst (бренд по ASN 15169)
|
||||
const classified = classifyFlowDst("8.8.8.8", 6, 443, 51504, hit)
|
||||
assert.equal(classified.service, "Google")
|
||||
|
||||
// ── движок: dims country/asn наполняются из geoip-ридеров ────────────────────
|
||||
resetEngineForTests()
|
||||
ingestParsedFlowsForServerForTests(1, [{
|
||||
src: "192.168.88.10",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51504,
|
||||
dstPort: 443,
|
||||
bytes: 1000,
|
||||
packets: 10,
|
||||
inIface: "wg-flow",
|
||||
outIface: "",
|
||||
nextHop: "",
|
||||
flowStartMs: 0,
|
||||
flowEndMs: 0,
|
||||
natSrc: "",
|
||||
natDst: "",
|
||||
}])
|
||||
const dims = minuteDimsSnapshotForTests()
|
||||
assert.equal(dims.get("country")?.get("US")?.bytes, 1000)
|
||||
assert.equal(dims.get("asn")?.get("15169")?.bytes, 1000)
|
||||
|
||||
// ── коллектор: 304 → обе базы без изменений ───────────────────────────────────
|
||||
disableGeoipDbForTests()
|
||||
resetGeoipUpdateForTests()
|
||||
const geoipDir = mkdtempSync(path.join(tmpdir(), "mm-geoip-test-"))
|
||||
process.env.GEOIP_DIR = geoipDir
|
||||
|
||||
setGeoipFetchForTests(async () => new Response(null, { status: 304 }))
|
||||
let snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.skippedUnchanged, 2)
|
||||
assert.equal(snap.downloaded, 0)
|
||||
assert.equal(existsSync(path.join(geoipDir, "GeoLite2-Country.mmdb")), false)
|
||||
|
||||
// ── коллектор: 200 + валидация ok → подмена, старый файл в .prev ─────────────
|
||||
const countryPath = path.join(geoipDir, "GeoLite2-Country.mmdb")
|
||||
const asnPath = path.join(geoipDir, "GeoLite2-ASN.mmdb")
|
||||
writeFileSync(countryPath, "old-country")
|
||||
|
||||
setGeoipFetchForTests(async () =>
|
||||
new Response(new Uint8Array([1, 2, 3]), { status: 200, headers: { etag: '"v1"' } }))
|
||||
setGeoipValidateForTests({
|
||||
country: async (p) => {
|
||||
assert.ok(p.endsWith(".tmp"), "валидация должна идти по tmp-файлу")
|
||||
return "2026-09-08T00:00:00.000Z"
|
||||
},
|
||||
asn: async () => "2026-09-08T00:00:00.000Z",
|
||||
})
|
||||
snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.downloaded, 2)
|
||||
assert.equal(snap.errors.length, 0)
|
||||
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]))
|
||||
assert.equal(readFileSync(`${countryPath}.prev`, "utf8"), "old-country")
|
||||
assert.equal(existsSync(`${asnPath}.prev`), false, "prev у asn не бывает при первой загрузке")
|
||||
assert.equal(existsSync(`${countryPath}.tmp`), false)
|
||||
|
||||
// ── коллектор: битая база → подмены нет, старый файл цел, tmp удалён ─────────
|
||||
writeFileSync(asnPath, "good-asn")
|
||||
setGeoipFetchForTests(async () =>
|
||||
new Response(new Uint8Array([9, 9]), { status: 200 }))
|
||||
setGeoipValidateForTests({
|
||||
country: async () => {
|
||||
throw new Error("битая база")
|
||||
},
|
||||
asn: async () => {
|
||||
throw new Error("битая база")
|
||||
},
|
||||
})
|
||||
snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.downloaded, 0)
|
||||
assert.equal(snap.errors.length, 2)
|
||||
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]), "country не тронута")
|
||||
assert.equal(readFileSync(asnPath, "utf8"), "good-asn", "asn не тронут")
|
||||
assert.equal(existsSync(`${countryPath}.tmp`), false)
|
||||
assert.equal(existsSync(`${asnPath}.tmp`), false)
|
||||
|
||||
rmSync(geoipDir, { recursive: true, force: true })
|
||||
delete process.env.GEOIP_DIR
|
||||
resetGeoipUpdateForTests()
|
||||
resetGeoipForTests()
|
||||
|
||||
console.log("traffic-flow-geoip.test.ts: ok")
|
||||
@@ -0,0 +1,162 @@
|
||||
import { existsSync } from "node:fs"
|
||||
import path from "node:path"
|
||||
import { open, type AsnResponse, type CountryResponse, type Reader } from "maxmind"
|
||||
import { isNonPublicIp } from "./traffic-flow-ip.js"
|
||||
import { isIsoCountry, resolveRipeCountry } from "./traffic-flow-brands.js"
|
||||
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
|
||||
export const GEOIP_COUNTRY_FILE = "GeoLite2-Country.mmdb"
|
||||
export const GEOIP_ASN_FILE = "GeoLite2-ASN.mmdb"
|
||||
|
||||
/** Каталог баз: `storage/geoip` рядом со storage/backups; переопределяется GEOIP_DIR. */
|
||||
export function geoipDir(): string {
|
||||
return path.resolve(process.env.GEOIP_DIR ?? path.join(process.cwd(), "storage", "geoip"))
|
||||
}
|
||||
|
||||
export function geoipCountryPath(): string {
|
||||
return path.join(geoipDir(), GEOIP_COUNTRY_FILE)
|
||||
}
|
||||
|
||||
export function geoipAsnPath(): string {
|
||||
return path.join(geoipDir(), GEOIP_ASN_FILE)
|
||||
}
|
||||
|
||||
export interface GeoipReaders {
|
||||
country: Reader<CountryResponse> | null
|
||||
asn: Reader<AsnResponse> | null
|
||||
}
|
||||
|
||||
let readers: GeoipReaders = { country: null, asn: null }
|
||||
let initPromise: Promise<GeoipReaders> | null = null
|
||||
|
||||
/** Открывает оба файла best-effort: отсутствующий/битый файл не мешает второму. */
|
||||
export async function openGeoipReaders(dir = geoipDir()): Promise<GeoipReaders> {
|
||||
const next: GeoipReaders = { country: null, asn: null }
|
||||
if (existsSync(path.join(dir, GEOIP_COUNTRY_FILE))) {
|
||||
try {
|
||||
next.country = await open<CountryResponse>(path.join(dir, GEOIP_COUNTRY_FILE))
|
||||
} catch {
|
||||
/* битый файл — работаем без country */
|
||||
}
|
||||
}
|
||||
if (existsSync(path.join(dir, GEOIP_ASN_FILE))) {
|
||||
try {
|
||||
next.asn = await open<AsnResponse>(path.join(dir, GEOIP_ASN_FILE))
|
||||
} catch {
|
||||
/* битый файл — работаем без ASN */
|
||||
}
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
/** Открывает ридеры при старте; файлы есть — работают, нет — lookup уходит в RIPE-fallback. */
|
||||
export async function initGeoip(): Promise<GeoipReaders> {
|
||||
if (!initPromise) {
|
||||
initPromise = openGeoipReaders().then((next) => {
|
||||
readers = next
|
||||
return next
|
||||
})
|
||||
}
|
||||
return initPromise
|
||||
}
|
||||
|
||||
/** Переоткрывает ридеры после обновления файлов (атомарная замена ссылок). */
|
||||
export async function reloadGeoipReaders(): Promise<GeoipReaders> {
|
||||
const next = await openGeoipReaders()
|
||||
readers = next
|
||||
initPromise = Promise.resolve(next)
|
||||
return next
|
||||
}
|
||||
|
||||
export function setGeoipReadersForTests(next: Partial<GeoipReaders>): void {
|
||||
readers = { country: next.country ?? null, asn: next.asn ?? null }
|
||||
}
|
||||
|
||||
export function resetGeoipForTests(): void {
|
||||
readers = { country: null, asn: null }
|
||||
initPromise = null
|
||||
}
|
||||
|
||||
function negativeMeta(ip: string): FlowIpMeta {
|
||||
const v6 = ip.includes(":")
|
||||
return {
|
||||
prefix: `${ip}/${v6 ? 128 : 32}`,
|
||||
asn: 0,
|
||||
country: "—",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "",
|
||||
ok: false,
|
||||
fetchedAt: Date.now(),
|
||||
}
|
||||
}
|
||||
|
||||
function safeCountryIso(reader: Reader<CountryResponse>, ip: string): string {
|
||||
try {
|
||||
const rec = reader.get(ip)
|
||||
return rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
|
||||
} catch {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
function safeAsn(reader: Reader<AsnResponse>, ip: string): { asn: number; holder: string } {
|
||||
try {
|
||||
const rec = reader.get(ip)
|
||||
return {
|
||||
asn: rec?.autonomous_system_number ?? 0,
|
||||
holder: rec?.autonomous_system_organization ?? "",
|
||||
}
|
||||
} catch {
|
||||
return { asn: 0, holder: "" }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Синхронный lookup по локальным GeoLite2. Возвращает FlowIpMeta в семантике RIPE-кэша
|
||||
* (ok=true когда есть страна или ASN; null — данных нет, пусть пробует RIPE).
|
||||
*/
|
||||
export function lookupGeoip(ip: string): FlowIpMeta | null {
|
||||
const trimmed = String(ip ?? "").trim()
|
||||
if (!trimmed) return null
|
||||
if (isNonPublicIp(trimmed)) return negativeMeta(trimmed)
|
||||
const { country: countryReader, asn: asnReader } = readers
|
||||
if (!countryReader && !asnReader) return null
|
||||
const iso = countryReader ? safeCountryIso(countryReader, trimmed) : ""
|
||||
const country = iso && isIsoCountry(iso) ? iso : ""
|
||||
const { asn, holder } = asnReader ? safeAsn(asnReader, trimmed) : { asn: 0, holder: "" }
|
||||
if (!asn && !country) return null
|
||||
return {
|
||||
prefix: `${trimmed}/${trimmed.includes(":") ? 128 : 32}`,
|
||||
asn,
|
||||
country: resolveRipeCountry(country, asn, holder) || "—",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder,
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
}
|
||||
}
|
||||
|
||||
/** Главный вход для потребителей пайплайна: локальные базы первыми, RIPE-кэш fallback. */
|
||||
export function resolveFlowIp(ip: string): FlowIpMeta | null {
|
||||
return lookupGeoip(ip) ?? lookupRipeCached(ip)
|
||||
}
|
||||
|
||||
export interface GeoipReadersStatus {
|
||||
countryLoaded: boolean
|
||||
asnLoaded: boolean
|
||||
countryBuildAt: string | null
|
||||
asnBuildAt: string | null
|
||||
dir: string
|
||||
}
|
||||
|
||||
export function geoipReadersStatus(): GeoipReadersStatus {
|
||||
return {
|
||||
countryLoaded: Boolean(readers.country),
|
||||
asnLoaded: Boolean(readers.asn),
|
||||
countryBuildAt: readers.country?.metadata.buildEpoch.toISOString() ?? null,
|
||||
asnBuildAt: readers.asn?.metadata.buildEpoch.toISOString() ?? null,
|
||||
dir: geoipDir(),
|
||||
}
|
||||
}
|
||||
@@ -22,8 +22,9 @@ rememberServerIfaces(7, [
|
||||
{ ".id": "*A", name: "wg-flow" },
|
||||
{ ".id": "*D", name: "bridge" },
|
||||
])
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "#2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "13").name, "bridge")
|
||||
assert.equal(resolveIfaceName(7, "0").name, "—")
|
||||
assert.equal(resolveIfaceName(7, "ether1").name, "ether1")
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bindingIfaceAliases,
|
||||
bindingIfaceAliasesAllServers,
|
||||
canonicalFactIface,
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
rememberServerIfaces,
|
||||
resetIfaceCacheForTests,
|
||||
resolveIfaceName,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
assert.equal(canonicalFactIface(1, "2"), "2")
|
||||
assert.deepEqual(bindingIfaceAliases(1, "gre-client"), ["gre-client"])
|
||||
|
||||
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
|
||||
assert.equal(canonicalFactIface(1, "2"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "gre-client"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "9"), "9")
|
||||
assert.equal(resolveIfaceName(1, "9").name, "#9")
|
||||
assert.equal(resolveIfaceName(1, "2").name, "gre-client")
|
||||
assert.equal(resolveIfaceName(1, "#2").name, "gre-client")
|
||||
assert.equal(displayFactIface(1, "2"), "gre-client")
|
||||
|
||||
const aliases = bindingIfaceAliases(1, "gre-client")
|
||||
assert.ok(aliases.includes("gre-client"))
|
||||
assert.ok(aliases.includes("2"))
|
||||
assert.ok(aliases.includes("#2"))
|
||||
|
||||
const fromIndex = factIfaceAliases("2", 1)
|
||||
assert.ok(fromIndex.includes("gre-client"))
|
||||
assert.ok(fromIndex.includes("2"))
|
||||
assert.ok(fromIndex.includes("#2"))
|
||||
|
||||
const all = bindingIfaceAliasesAllServers("gre-client")
|
||||
assert.ok(all.includes("2"))
|
||||
|
||||
const expanded = expandBindingIfaces([{ serverId: 1, iface: "gre-client" }])
|
||||
assert.ok(expanded.some((x) => x.iface === "2"))
|
||||
assert.ok(expanded.some((x) => x.iface === "gre-client"))
|
||||
|
||||
const collapsed = collapseServerIfaceRows([
|
||||
{ serverId: 1, iface: "2", bytes: 10, packets: 1 },
|
||||
{ serverId: 1, iface: "gre-client", bytes: 5, packets: 2 },
|
||||
{ serverId: 1, iface: "wan1", bytes: 3, packets: 1 },
|
||||
])
|
||||
assert.equal(collapsed.length, 2)
|
||||
const gre = collapsed.find((r) => r.iface === "gre-client")
|
||||
assert.ok(gre)
|
||||
assert.equal(gre.bytes, 15)
|
||||
assert.equal(gre.packets, 3)
|
||||
assert.ok(collapsed.some((r) => r.iface === "wan1"))
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-ifindex.test.ts: ok")
|
||||
@@ -36,12 +36,135 @@ export function rememberServerIfaces(serverId: number, rows: RosIfaceIndexRow[])
|
||||
|
||||
export function resolveIfaceName(serverId: number, indexOrName: string): { name: string; index: string } {
|
||||
const trimmed = String(indexOrName ?? "").trim()
|
||||
if (!trimmed || trimmed === "0") return { name: "—", index: trimmed }
|
||||
if (!/^\d+$/.test(trimmed)) return { name: trimmed, index: "" }
|
||||
const idx = Number(trimmed)
|
||||
const name = cache.get(serverId)?.get(idx)
|
||||
if (name) return { name, index: trimmed }
|
||||
return { name: `#${trimmed}`, index: trimmed }
|
||||
const asIndex = trimmed.startsWith("#") && /^\d+$/.test(trimmed.slice(1)) ? trimmed.slice(1) : trimmed
|
||||
if (!asIndex || asIndex === "0") return { name: "—", index: asIndex }
|
||||
if (!/^\d+$/.test(asIndex)) return { name: trimmed, index: "" }
|
||||
const name = cache.get(serverId)?.get(Number(asIndex))
|
||||
if (name) return { name, index: asIndex }
|
||||
return { name: `#${asIndex}`, index: asIndex }
|
||||
}
|
||||
|
||||
/** Имя iface для факта куба: ifIndex→имя, без `#13` при пустом кэше. */
|
||||
export function canonicalFactIface(serverId: number, inIface: string): string {
|
||||
const trimmed = String(inIface ?? "").trim()
|
||||
if (!trimmed) return trimmed
|
||||
if (!/^\d+$/.test(trimmed)) return trimmed
|
||||
const name = cache.get(serverId)?.get(Number(trimmed))
|
||||
return name || trimmed
|
||||
}
|
||||
|
||||
function numericIfaceIndex(iface: string): string | null {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (/^\d+$/.test(raw)) return raw
|
||||
if (raw.startsWith("#") && /^\d+$/.test(raw.slice(1))) return raw.slice(1)
|
||||
return null
|
||||
}
|
||||
|
||||
/** Имя для UI: ifIndex → RouterOS name; `0` → «—»; miss → `#n`. */
|
||||
export function displayFactIface(serverId: number, iface: string): string {
|
||||
return resolveIfaceName(serverId, iface).name
|
||||
}
|
||||
|
||||
/** Склеить факты `2` + `ether1` в одну строку после резолва ifIndex. */
|
||||
export function collapseServerIfaceRows(
|
||||
rows: Array<{ serverId: number; iface: string; bytes: number; packets: number }>,
|
||||
): Array<{ serverId: number; iface: string; bytes: number; packets: number }> {
|
||||
const acc = new Map<string, { serverId: number; iface: string; bytes: number; packets: number }>()
|
||||
for (const r of rows) {
|
||||
const name = displayFactIface(r.serverId, r.iface)
|
||||
const k = `${r.serverId}\0${name}`
|
||||
const prev = acc.get(k)
|
||||
const bytes = Number(r.bytes) || 0
|
||||
const packets = Number(r.packets) || 0
|
||||
if (prev) {
|
||||
prev.bytes += bytes
|
||||
prev.packets += packets
|
||||
} else {
|
||||
acc.set(k, { serverId: r.serverId, iface: name, bytes, packets })
|
||||
}
|
||||
}
|
||||
return [...acc.values()]
|
||||
}
|
||||
|
||||
/** Ключи факта для фильтра: имя, ifIndex и `#n`. */
|
||||
export function factIfaceAliases(iface: string, serverId?: number): string[] {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (!raw) return []
|
||||
const out = new Set<string>([raw])
|
||||
const idx = numericIfaceIndex(raw)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
const n = Number(idx)
|
||||
if (serverId != null) {
|
||||
const name = cache.get(serverId)?.get(n)
|
||||
if (name) out.add(name)
|
||||
} else {
|
||||
for (const map of cache.values()) {
|
||||
const name = map.get(n)
|
||||
if (name) out.add(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (serverId != null) {
|
||||
for (const a of bindingIfaceAliases(serverId, raw)) out.add(a)
|
||||
} else {
|
||||
for (const a of bindingIfaceAliasesAllServers(raw)) out.add(a)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
/** Имя + ifIndex + `#n` — тот же матч, что карта `/traffic`. */
|
||||
export function bindingIfaceAliases(serverId: number, interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
if (!name) return []
|
||||
const out = new Set<string>([name])
|
||||
const map = cache.get(serverId)
|
||||
const idx = numericIfaceIndex(name)
|
||||
const canonical = (idx && map?.get(Number(idx))) || name
|
||||
out.add(canonical)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
}
|
||||
if (!map) return [...out]
|
||||
for (const [i, n] of map) {
|
||||
if (n !== canonical && n !== name) continue
|
||||
out.add(String(i))
|
||||
out.add(`#${i}`)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function bindingIfaceAliasesAllServers(interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
const out = new Set<string>(name ? [name] : [])
|
||||
for (const serverId of cache.keys()) {
|
||||
for (const alias of bindingIfaceAliases(serverId, name)) out.add(alias)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function expandBindingIfaces(
|
||||
binds: Array<{ serverId: number; iface: string }>,
|
||||
): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const b of binds) {
|
||||
for (const iface of bindingIfaceAliases(b.serverId, b.iface)) {
|
||||
const k = `${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function listCachedIfaceNames(serverId: number): string[] {
|
||||
const map = cache.get(serverId)
|
||||
if (!map) return []
|
||||
return [...new Set(map.values())]
|
||||
}
|
||||
|
||||
export function ifaceCacheHas(serverId: number): boolean {
|
||||
|
||||
@@ -467,11 +467,15 @@ export async function purgeTrafficFlowStore(): Promise<FlowPurgeDto> {
|
||||
minuteStats: await tableCount("flow_minute_stats"),
|
||||
minuteDims: await tableCount("flow_minute_dims"),
|
||||
dailyDims: await tableCount("flow_daily_dims"),
|
||||
hourFacts: await tableCount("flow_hour_facts"),
|
||||
dailyFacts: await tableCount("flow_daily_facts"),
|
||||
}
|
||||
await dbQuery(`DELETE FROM flow_buckets`)
|
||||
await dbQuery(`DELETE FROM flow_minute_stats`)
|
||||
await dbQuery(`DELETE FROM flow_minute_dims`)
|
||||
await dbQuery(`DELETE FROM flow_daily_dims`)
|
||||
await dbQuery(`DELETE FROM flow_hour_facts`)
|
||||
await dbQuery(`DELETE FROM flow_daily_facts`)
|
||||
await resetFlowIngestCounters()
|
||||
await dropExpiredPartitions(pool)
|
||||
try {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { isNonPublicIp, pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { isNonPublicIp, pickInternetDest, pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
|
||||
assert.equal(isNonPublicIp("10.200.100.53"), true)
|
||||
assert.equal(isNonPublicIp("173.194.151.65"), false)
|
||||
@@ -22,4 +22,39 @@ assert.equal(
|
||||
)
|
||||
assert.equal(pickInternetPeer("10.1.1.1", "10.2.2.2", 443, 80), "10.2.2.2")
|
||||
|
||||
const rost = "95.167.1.10"
|
||||
const ours = new Set(["198.51.100.1", "203.0.113.10"])
|
||||
const client = { ours, boundClient: true }
|
||||
|
||||
assert.equal(
|
||||
pickInternetDest("10.200.100.53", "104.18.35.51", 53880, 443, client),
|
||||
"104.18.35.51",
|
||||
"RFC1918 → CF на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("173.194.151.65", "10.200.100.53", 443, 57182, client),
|
||||
"173.194.151.65",
|
||||
"Google:443 → RFC1918 на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "10.200.100.53", 51234, 443, client),
|
||||
"",
|
||||
"Rostelecom → overlay 10.x: не dest ASN клиента",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "8.8.8.8", 51234, 443, client),
|
||||
"8.8.8.8",
|
||||
"Rostelecom → Google:443 на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "1.1.1.1", 51234, 40000, client),
|
||||
"1.1.1.1",
|
||||
"оба публичные без well-known на client GRE → dst",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("8.8.8.8", "198.51.100.1", 443, 51234, { ours }),
|
||||
"8.8.8.8",
|
||||
"ours как dst: dest = публичный src",
|
||||
)
|
||||
|
||||
console.log("traffic-flow-ip.test.ts: ok")
|
||||
|
||||
@@ -55,13 +55,46 @@ export function isNonPublicIp(ip: string): boolean {
|
||||
|
||||
const PEER_WELL_KNOWN_PORTS = new Set([80, 443, 53, 853])
|
||||
|
||||
export interface InternetDestCtx {
|
||||
/** WAN IP узлов сети (EN/JH) — не интернет-назначение. */
|
||||
ours?: ReadonlySet<string>
|
||||
/** Ingress с bound GRE/WG клиента: dest = нелокальный IP, не ASN клиента. */
|
||||
boundClient?: boolean
|
||||
}
|
||||
|
||||
export function isLocalIp(ip: string, ours?: ReadonlySet<string>): boolean {
|
||||
if (isNonPublicIp(ip)) return true
|
||||
return Boolean(ours?.has(String(ip ?? "").trim()))
|
||||
}
|
||||
|
||||
/**
|
||||
* Интернет-сторона потока: у IPFIX сервис часто в src (Google:443 → RFC1918:ephemeral).
|
||||
* Классифицировать этот IP, не слепой dst.
|
||||
* Интернет-назначение потока для ASN/страны/сервиса.
|
||||
* Пустая строка — dest нет (не GeoIP IP клиента).
|
||||
*/
|
||||
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
|
||||
const srcPub = !isNonPublicIp(src)
|
||||
const dstPub = !isNonPublicIp(dst)
|
||||
export function pickInternetDest(
|
||||
src: string,
|
||||
dst: string,
|
||||
srcPort: number,
|
||||
dstPort: number,
|
||||
ctx?: InternetDestCtx,
|
||||
): string {
|
||||
const ours = ctx?.ours
|
||||
const srcLocal = isLocalIp(src, ours)
|
||||
const dstLocal = isLocalIp(dst, ours)
|
||||
const srcPub = !srcLocal
|
||||
const dstPub = !dstLocal
|
||||
|
||||
if (ctx?.boundClient) {
|
||||
if (dstPub) return dst
|
||||
if (srcPub && dstLocal) {
|
||||
const srcWk = PEER_WELL_KNOWN_PORTS.has(srcPort)
|
||||
const dstWk = PEER_WELL_KNOWN_PORTS.has(dstPort)
|
||||
if (srcWk && !dstWk) return src
|
||||
return ""
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
if (srcPub && !dstPub) return src
|
||||
if (dstPub && !srcPub) return dst
|
||||
if (srcPub && dstPub) {
|
||||
@@ -72,3 +105,11 @@ export function pickInternetPeer(src: string, dst: string, srcPort: number, dstP
|
||||
}
|
||||
return dst
|
||||
}
|
||||
|
||||
/**
|
||||
* Интернет-сторона потока без топологии: у IPFIX сервис часто в src (Google:443 → RFC1918).
|
||||
* Для куба статистики используйте pickInternetDest.
|
||||
*/
|
||||
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
|
||||
return pickInternetDest(src, dst, srcPort, dstPort) || dst
|
||||
}
|
||||
|
||||
@@ -4,7 +4,13 @@ import {
|
||||
ingestParsedFlowsForServerForTests,
|
||||
resetFlowRingsForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { buildFlowMapHops, resetFlowMapHopsCacheForTests } from "./traffic-flow-map-hops.js"
|
||||
import {
|
||||
buildFlowMapHops,
|
||||
MAP_SERVICE_MIN_NODES,
|
||||
MAP_SERVICE_NODE_CAP,
|
||||
pickMapServices,
|
||||
resetFlowMapHopsCacheForTests,
|
||||
} from "./traffic-flow-map-hops.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
@@ -15,6 +21,55 @@ import {
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
|
||||
{
|
||||
const googleOnly = pickMapServices(
|
||||
[{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 1 }],
|
||||
5,
|
||||
)
|
||||
assert.equal(googleOnly.length, 1)
|
||||
assert.equal(googleOnly[0]?.share, 1)
|
||||
|
||||
const twoNamed = pickMapServices(
|
||||
[
|
||||
{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 0.5 },
|
||||
{ id: "svc:cloudflare", label: "Cloudflare", category: "CDN", bytes: 400, bps: 0, share: 0.5 },
|
||||
],
|
||||
5,
|
||||
)
|
||||
assert.equal(twoNamed.length, 2)
|
||||
|
||||
const tinyTail = pickMapServices(
|
||||
[
|
||||
{ id: "svc:google", label: "Google", category: "Веб", bytes: 9000, bps: 0, share: 0.9 },
|
||||
...Array.from({ length: 9 }, (_, i) => ({
|
||||
id: `svc:t${i}`,
|
||||
label: `T${i}`,
|
||||
category: "Веб",
|
||||
bytes: 100,
|
||||
bps: 0,
|
||||
share: 0.01,
|
||||
})),
|
||||
],
|
||||
5,
|
||||
)
|
||||
assert.equal(tinyTail.length, MAP_SERVICE_MIN_NODES)
|
||||
assert.equal(tinyTail.at(-1)?.id, "svc:t6")
|
||||
|
||||
const allOff = pickMapServices(
|
||||
Array.from({ length: 25 }, (_, i) => ({
|
||||
id: `svc:n${i}`,
|
||||
label: `N${i}`,
|
||||
category: "Веб",
|
||||
bytes: 25 - i,
|
||||
bps: 0,
|
||||
share: 0.04,
|
||||
})),
|
||||
0,
|
||||
)
|
||||
assert.equal(allOff.length, MAP_SERVICE_NODE_CAP)
|
||||
console.log("traffic-flow-map-hops.test.ts: pickMapServices ok")
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-map-hops.test.ts: skip")
|
||||
process.exit(0)
|
||||
@@ -178,6 +233,19 @@ function googleRipe() {
|
||||
})
|
||||
}
|
||||
|
||||
function seedRipeAsn(ip: string, asn: number, holder: string) {
|
||||
seedRipeCacheForTests({
|
||||
prefix: `${ip}/32`,
|
||||
asn,
|
||||
country: "US",
|
||||
lat: 37.4,
|
||||
lng: -122.1,
|
||||
holder,
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
}
|
||||
|
||||
function payloadFlow(dst: string, bytes: number) {
|
||||
return {
|
||||
src: "10.100.1.17",
|
||||
@@ -241,10 +309,106 @@ try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const four = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.equal(four.totalBytes, 10_000)
|
||||
assert.ok(!(four.services ?? []).some((s) => s.id === "svc:google"), "Google < 5% hidden")
|
||||
const googleFour = four.services?.find((s) => s.id === "svc:google")
|
||||
assert.ok(googleFour, "единственный бренд виден при 4% от окна")
|
||||
assert.ok(googleFour.share >= 0.99, "доля среди брендов ≈ 1")
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const off = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google 4%")
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 400),
|
||||
payloadFlow("104.18.35.51", 400),
|
||||
payloadFlow("203.0.113.50", 9200),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const two = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const googleTwo = two.services?.find((s) => s.id === "svc:google")
|
||||
const cfTwo = two.services?.find((s) => s.id === "svc:cloudflare")
|
||||
assert.ok(googleTwo, "Google среди брендов")
|
||||
assert.ok(cfTwo, "Cloudflare среди брендов")
|
||||
assert.ok(googleTwo.share >= 0.05)
|
||||
assert.ok(cfTwo.share >= 0.05)
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
seedRipeAsn("162.254.192.71", 32590, "VALVE-CORP")
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("162.254.192.71", 2000),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const steam = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.ok(steam.services?.some((s) => s.id === "svc:steam"), "Steam AS32590 на карте")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
const smallBrands: Array<{ ip: string; asn: number; holder: string; bytes: number; id: string }> = [
|
||||
{ ip: "203.0.113.1", asn: 714, holder: "APPLE-ENGINEERING", bytes: 400, id: "svc:apple" },
|
||||
{ ip: "203.0.113.2", asn: 36459, holder: "GITHUB", bytes: 390, id: "svc:github" },
|
||||
{ ip: "203.0.113.3", asn: 54876, holder: "GITLAB", bytes: 380, id: "svc:gitlab" },
|
||||
{ ip: "203.0.113.4", asn: 8403, holder: "SPOTIFY", bytes: 370, id: "svc:spotify" },
|
||||
{ ip: "203.0.113.5", asn: 13414, holder: "TWITTER", bytes: 360, id: "svc:x" },
|
||||
{ ip: "203.0.113.6", asn: 47541, holder: "VKONTAKTE", bytes: 350, id: "svc:vk" },
|
||||
{ ip: "203.0.113.7", asn: 30103, holder: "ZOOM", bytes: 340, id: "svc:zoom" },
|
||||
{ ip: "203.0.113.8", asn: 395701, holder: "EPIC-GAMES", bytes: 330, id: "svc:epic" },
|
||||
{ ip: "203.0.113.9", asn: 6507, holder: "RIOT-GAMES", bytes: 320, id: "svc:riot" },
|
||||
]
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
for (const b of smallBrands) seedRipeAsn(b.ip, b.asn, b.holder)
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 5000),
|
||||
...smallBrands.map((b) => payloadFlow(b.ip, b.bytes)),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const top = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const ids = new Set((top.services ?? []).map((s) => s.id))
|
||||
assert.equal(top.services?.length, MAP_SERVICE_MIN_NODES, "топ-8 брендов на карте")
|
||||
assert.ok(ids.has("svc:google"))
|
||||
for (const b of smallBrands.slice(0, 7)) assert.ok(ids.has(b.id), b.id)
|
||||
assert.ok(!ids.has("svc:epic"), "хвост ниже ранга 8 скрыт")
|
||||
assert.ok(!ids.has("svc:riot"))
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
@@ -420,4 +584,98 @@ try {
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
rememberServerIfaces(9, [
|
||||
{ ".id": "*1", name: "ether1" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [payloadFlow("8.8.8.8", 12_000)])
|
||||
ingestParsedFlowsForServerForTests(9, [{
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 12_000,
|
||||
packets: 10,
|
||||
inIface: "1",
|
||||
outIface: "1",
|
||||
nextHop: "",
|
||||
}])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const dual = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = dual.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE JH→EN сохранён")
|
||||
assert.equal(gre.bytes, 12_000)
|
||||
const googlePaths = (dual.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "один путь без копии EN")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.equal(googlePaths[0]?.viaId, "7")
|
||||
const googleEdge = dual.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 8_000),
|
||||
{
|
||||
src: "8.8.8.8",
|
||||
dst: "10.100.1.17",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 51234,
|
||||
bytes: 4_000,
|
||||
packets: 8,
|
||||
inIface: "3",
|
||||
outIface: "2",
|
||||
nextHop: "",
|
||||
},
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const bothDir = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = bothDir.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE-hop при fwd/rev")
|
||||
const googlePaths = (bothDir.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "fwd+rev — один клиент")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.ok(!(bothDir.servicePaths ?? []).some((p) => p.serviceId === "svc:google" && p.clientId === "—"))
|
||||
const googleEdge = bothDir.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-map-hops.test.ts: ok")
|
||||
|
||||
@@ -5,21 +5,25 @@ import { userInterfaceBindings } from "../db/schema.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import {
|
||||
isNamedInternetService,
|
||||
lookupBrand,
|
||||
mapServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
} from "./traffic-flow-brands.js"
|
||||
import { dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { dedupFlowRowsAcrossExporters, dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js"
|
||||
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { destCtxForIface } from "./traffic-flow-dest.js"
|
||||
import { pickInternetDest } from "./traffic-flow-ip.js"
|
||||
import { type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog } from "./traffic-flow-topology.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { flowDataEpoch } from "./traffic-flow-engine.js"
|
||||
|
||||
export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5
|
||||
export const MAP_SERVICE_NODE_CAP = 20
|
||||
/** Минимум узлов-брендов на карте, даже если доля ниже порога. */
|
||||
export const MAP_SERVICE_MIN_NODES = 8
|
||||
const HOPS_CACHE_TTL_MS = 2000
|
||||
|
||||
export interface FlowMapHopsQuery {
|
||||
@@ -99,6 +103,15 @@ export function clampMapServiceMinSharePct(n: unknown): number {
|
||||
return Math.min(100, Math.max(0, v))
|
||||
}
|
||||
|
||||
/** Доля среди именованных брендов; порог ИЛИ топ-N, затем cap. */
|
||||
export function pickMapServices(ranked: FlowMapService[], minSharePct: number): FlowMapService[] {
|
||||
if (minSharePct <= 0) return ranked.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
const minShare = minSharePct / 100
|
||||
return ranked
|
||||
.filter((s, i) => s.share >= minShare || i < MAP_SERVICE_MIN_NODES)
|
||||
.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
}
|
||||
|
||||
function hopsQueryKey(q: FlowMapHopsQuery, minSharePct: number): string {
|
||||
return JSON.stringify({
|
||||
epoch: flowDataEpoch(),
|
||||
@@ -129,6 +142,16 @@ function ifaceUsable(name: string): boolean {
|
||||
return Boolean(name) && name !== "—"
|
||||
}
|
||||
|
||||
function resolveMapClient(
|
||||
topo: FlowTopology,
|
||||
serverId: number,
|
||||
inName: string,
|
||||
outName: string,
|
||||
) {
|
||||
return resolveClient(topo, serverId, inName)
|
||||
?? (ifaceUsable(outName) ? resolveClient(topo, serverId, outName) : null)
|
||||
}
|
||||
|
||||
function bump(acc: Map<string, HopAcc>, key: string, seed: Omit<HopAcc, "bytes" | "bytesFwd" | "bytesRev">, bytes: number, dir: "fwd" | "rev" | "both"): void {
|
||||
const prev = acc.get(key)
|
||||
const addFwd = dir === "fwd" || dir === "both" ? bytes : 0
|
||||
@@ -174,10 +197,7 @@ function classifyMapDstLite(
|
||||
if (proto === 47 || proto === 50) return null
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "WireGuard" || app === "DNS" || app === "SSH" || app === "BGP") return null
|
||||
if (/youtube/i.test(ripe?.holder ?? "")) {
|
||||
return { service: "YouTube", category: "Видео / стриминг" }
|
||||
}
|
||||
const brand = lookupBrand(dst, ripe?.asn ?? 0)
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, ripe?.holder ?? "", proto, dstPort, srcPort)
|
||||
if (!brand || !isNamedInternetService(brand.service, brand.category)) return null
|
||||
return brand
|
||||
}
|
||||
@@ -229,6 +249,21 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const enIds = new Set(topo.enNodes.map((n) => n.id))
|
||||
let totalBytes = 0
|
||||
|
||||
function rowClient(r: (typeof working)[number]) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
return resolveMapClient(topo, r.serverId, inName, outName)
|
||||
}
|
||||
|
||||
const payloadRows = wantDedup
|
||||
? dedupFlowRowsAcrossExporters(working, (a, b) => {
|
||||
const aCli = Boolean(rowClient(a))
|
||||
const bCli = Boolean(rowClient(b))
|
||||
if (aCli !== bCli) return aCli ? a : b
|
||||
return a.bytes >= b.bytes ? a : b
|
||||
})
|
||||
: working
|
||||
|
||||
for (const r of working) {
|
||||
const inRes = resolveIfaceName(r.serverId, r.inIface)
|
||||
const outRes = resolveIfaceName(r.serverId, r.outIface)
|
||||
@@ -314,11 +349,22 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}, r.bytes, "fwd")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const r of payloadRows) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
totalBytes += r.bytes
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
const client = resolveClient(topo, r.serverId, inName)
|
||||
const prevDst = dstAcc.get(peer)
|
||||
const dest = pickInternetDest(
|
||||
r.src,
|
||||
r.dst,
|
||||
r.srcPort,
|
||||
r.dstPort,
|
||||
destCtxForIface(topo, r.serverId, inName),
|
||||
)
|
||||
if (!dest) continue
|
||||
const client = resolveMapClient(topo, r.serverId, inName, outName)
|
||||
const prevDst = dstAcc.get(dest)
|
||||
if (prevDst) {
|
||||
prevDst.bytes += r.bytes
|
||||
bumpFrom(prevDst, String(r.serverId), r.bytes, client)
|
||||
@@ -331,7 +377,7 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
fromBytes: new Map(),
|
||||
}
|
||||
bumpFrom(acc, String(r.serverId), r.bytes, client)
|
||||
dstAcc.set(peer, acc)
|
||||
dstAcc.set(dest, acc)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -386,7 +432,7 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}
|
||||
|
||||
for (const [dst, acc] of dstAcc) {
|
||||
const ripe = lookupRipeCached(dst)
|
||||
const ripe = resolveFlowIp(dst)
|
||||
const classified = classifyMapDstLite(dst, acc.proto, acc.dstPort, acc.srcPort, ripe)
|
||||
if (!classified) continue
|
||||
const toId = mapServiceNodeId(classified.service)
|
||||
@@ -419,10 +465,15 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const enName = nodeName(fromId)
|
||||
const viaName = nodeName(exporterId)
|
||||
for (const [clientId, c] of from.clients) {
|
||||
const pathKey = `${clientId}|${exporterId}|${fromId}|${toId}`
|
||||
const pathKey = `${clientId}|${fromId}|${toId}`
|
||||
const prevPath = svcPaths.get(pathKey)
|
||||
if (prevPath) {
|
||||
prevPath.bytes += c.bytes
|
||||
if (exporterId !== fromId && prevPath.viaId === fromId) {
|
||||
prevPath.viaId = exporterId
|
||||
prevPath.viaName = viaName
|
||||
}
|
||||
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
|
||||
} else {
|
||||
svcPaths.set(pathKey, {
|
||||
clientId,
|
||||
@@ -439,21 +490,20 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}
|
||||
}
|
||||
|
||||
const minShare = minSharePct / 100
|
||||
let services: FlowMapService[] = [...svcTotals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: totalBytes > 0 ? s.bytes / totalBytes : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
if (minSharePct > 0) {
|
||||
services = services.filter((s) => s.share >= minShare)
|
||||
}
|
||||
services = services.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
const namedBytes = [...svcTotals.values()].reduce((n, s) => n + s.bytes, 0)
|
||||
const services = pickMapServices(
|
||||
[...svcTotals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: namedBytes > 0 ? s.bytes / namedBytes : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes),
|
||||
minSharePct,
|
||||
)
|
||||
const keepSvc = new Set(services.map((s) => s.id))
|
||||
const serviceEdges: FlowMapServiceEdge[] = [...svcEdges.values()]
|
||||
.filter((e) => keepSvc.has(e.toId))
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { appUsers, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
|
||||
import { mapRosInterfaceType, parseRawInterfaces } from "../modules/users/iface-type.js"
|
||||
import type { PlaneTopology } from "./traffic-flow-planes.js"
|
||||
|
||||
export interface FlowClientBinding {
|
||||
@@ -25,6 +25,8 @@ export interface FlowTopology {
|
||||
enHosts: Set<string>
|
||||
jhHosts: Set<string>
|
||||
wanIfaces: Map<number, Set<string>>
|
||||
/** GRE/WG из последнего снимка RouterOS (`type`), без mgmt. */
|
||||
tunnelIfaces?: Map<number, Set<string>>
|
||||
plane: PlaneTopology
|
||||
}
|
||||
|
||||
@@ -48,6 +50,15 @@ export function invalidateFlowCatalogCache(): void {
|
||||
serverCatalogCache = null
|
||||
}
|
||||
|
||||
export function peekFlowTopology(): FlowTopology | null {
|
||||
if (seeded) return seeded
|
||||
return topologyCache?.topo ?? null
|
||||
}
|
||||
|
||||
export function peekServerCatalog(): { list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> } | null {
|
||||
return serverCatalogCache
|
||||
}
|
||||
|
||||
export async function getServerCatalog(): Promise<{ list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> }> {
|
||||
const now = Date.now()
|
||||
if (serverCatalogCache && now - serverCatalogCache.at < CATALOG_TTL_MS) {
|
||||
@@ -76,6 +87,25 @@ function ifaceKey(serverId: number, name: string): string {
|
||||
return `${serverId}|${name}`
|
||||
}
|
||||
|
||||
async function loadTunnelIfacesFromSnapshots(): Promise<Map<number, Set<string>>> {
|
||||
const rows = await dbAll<{ serverId: number; rawInterfaces: unknown }>(`
|
||||
SELECT DISTINCT ON (server_id) server_id AS "serverId", raw_interfaces AS "rawInterfaces"
|
||||
FROM server_snapshots
|
||||
ORDER BY server_id, polled_at DESC
|
||||
`)
|
||||
const map = new Map<number, Set<string>>()
|
||||
for (const r of rows) {
|
||||
const set = new Set<string>()
|
||||
for (const iface of parseRawInterfaces(r.rawInterfaces)) {
|
||||
if (iface.type !== "gre" && iface.type !== "wg") continue
|
||||
if (iface.name.toLowerCase() === "wg-flow") continue
|
||||
set.add(iface.name)
|
||||
}
|
||||
if (set.size) map.set(r.serverId, set)
|
||||
}
|
||||
return map
|
||||
}
|
||||
|
||||
export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
if (seeded) return seeded
|
||||
const now = Date.now()
|
||||
@@ -118,6 +148,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
for (const h of hosts) jhHosts.add(h)
|
||||
}
|
||||
}
|
||||
const tunnelIfaces = await loadTunnelIfacesFromSnapshots()
|
||||
const topo: FlowTopology = {
|
||||
clientIfaces,
|
||||
clientByIface,
|
||||
@@ -125,6 +156,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
enHosts,
|
||||
jhHosts,
|
||||
wanIfaces,
|
||||
tunnelIfaces,
|
||||
plane: {
|
||||
clientIfaceNames: allClientNames,
|
||||
enHosts,
|
||||
@@ -140,6 +172,18 @@ export function seedFlowTopologyForTests(topo: FlowTopology | null): void {
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
export function flowOursHosts(topo: FlowTopology | null | undefined): Set<string> {
|
||||
const ours = new Set<string>()
|
||||
if (!topo) return ours
|
||||
for (const h of topo.enHosts) {
|
||||
if (h) ours.add(h)
|
||||
}
|
||||
for (const h of topo.jhHosts) {
|
||||
if (h) ours.add(h)
|
||||
}
|
||||
return ours
|
||||
}
|
||||
|
||||
export function resolveClient(
|
||||
topo: FlowTopology,
|
||||
serverId: number,
|
||||
@@ -168,10 +212,14 @@ export function resolveEn(
|
||||
|
||||
export function enGreIfaceNames(topo: FlowTopology, serverId: number, ifaceNames: string[]): string[] {
|
||||
const client = topo.clientIfaces.get(serverId) ?? new Set<string>()
|
||||
const wan = topo.wanIfaces.get(serverId) ?? new Set<string>()
|
||||
const typed = topo.tunnelIfaces?.get(serverId)
|
||||
return ifaceNames.filter((name) => {
|
||||
if (client.has(name)) return false
|
||||
if (client.has(name) || wan.has(name)) return false
|
||||
if (name === "wg-flow") return false
|
||||
return mapRosInterfaceType("", name) === "gre"
|
||||
if (typed && typed.size > 0) return typed.has(name)
|
||||
const t = mapRosInterfaceType("", name)
|
||||
return t === "gre" || t === "wg"
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,43 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { mapVxlanRow } from "./vxlan-live.js"
|
||||
|
||||
const server = {
|
||||
id: 7,
|
||||
name: "mt-msk",
|
||||
host: "10.0.0.1",
|
||||
site: "MSK",
|
||||
country: "RU",
|
||||
} as Parameters<typeof mapVxlanRow>[0]
|
||||
|
||||
const row = mapVxlanRow(
|
||||
server,
|
||||
{
|
||||
".id": "*3",
|
||||
name: "vxlan-10",
|
||||
vni: "10010",
|
||||
port: "8472",
|
||||
"local-address": "10.0.0.1",
|
||||
running: "true",
|
||||
disabled: "false",
|
||||
l2mtu: "1500",
|
||||
"mac-learning": "true",
|
||||
"arp-proxy": "true",
|
||||
comment: "overlay",
|
||||
},
|
||||
[
|
||||
{ interface: "vxlan-10", "remote-ip": "10.0.1.1" },
|
||||
{ interface: "other", "remote-ip": "1.1.1.1" },
|
||||
{ interface: "vxlan-10", "remote-ip": "10.0.2.1" },
|
||||
],
|
||||
0,
|
||||
)
|
||||
|
||||
assert.equal(row.serverId, "7")
|
||||
assert.equal(row.vni, 10010)
|
||||
assert.equal(row.dstPort, 8472)
|
||||
assert.equal(row.status, "up")
|
||||
assert.equal(row.enabled, true)
|
||||
assert.deepEqual(row.remoteVteps, ["10.0.1.1", "10.0.2.1"])
|
||||
assert.equal(row.vtepIp, "10.0.0.1")
|
||||
|
||||
console.log("vxlan-live.test.ts: ok")
|
||||
@@ -0,0 +1,136 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
interface RosVxlan {
|
||||
".id"?: string
|
||||
name?: string
|
||||
vni?: string
|
||||
port?: string
|
||||
"local-address"?: string
|
||||
"vtep-address"?: string
|
||||
running?: string
|
||||
disabled?: string
|
||||
comment?: string
|
||||
l2mtu?: string
|
||||
arp?: string
|
||||
"arp-proxy"?: string
|
||||
"mac-learning"?: string
|
||||
learning?: string
|
||||
}
|
||||
|
||||
interface RosVxlanVtep {
|
||||
".id"?: string
|
||||
interface?: string
|
||||
"remote-ip"?: string
|
||||
}
|
||||
|
||||
export type VxlanTunnelLive = {
|
||||
id: string
|
||||
rosId: string
|
||||
name: string
|
||||
vni: number
|
||||
port: number
|
||||
dstPort: number
|
||||
serverId: string
|
||||
vtepIp: string
|
||||
remoteVteps: string[]
|
||||
l2mtu: number
|
||||
arpProxy: boolean
|
||||
macLearning: boolean
|
||||
comment: string
|
||||
enabled: boolean
|
||||
status: "up" | "down"
|
||||
}
|
||||
|
||||
function rosYes(v: string | undefined): boolean {
|
||||
return v === "true" || v === "yes"
|
||||
}
|
||||
|
||||
function parseIntSafe(v: string | undefined, fallback: number): number {
|
||||
const n = Number.parseInt(v ?? "", 10)
|
||||
return Number.isFinite(n) ? n : fallback
|
||||
}
|
||||
|
||||
export function mapVxlanRow(
|
||||
server: ServerRow,
|
||||
vx: RosVxlan,
|
||||
vteps: RosVxlanVtep[],
|
||||
idx: number,
|
||||
): VxlanTunnelLive {
|
||||
const name = (vx.name ?? "").trim() || `vxlan-${idx + 1}`
|
||||
const rosId = String(vx[".id"] ?? name)
|
||||
const disabled = rosYes(vx.disabled)
|
||||
const running = rosYes(vx.running)
|
||||
const port = parseIntSafe(vx.port, 8472)
|
||||
const remoteVteps = vteps
|
||||
.filter((v) => (v.interface ?? "").trim() === name)
|
||||
.map((v) => (v["remote-ip"] ?? "").trim())
|
||||
.filter(Boolean)
|
||||
return {
|
||||
id: `${server.id}-${rosId}`,
|
||||
rosId,
|
||||
name,
|
||||
vni: parseIntSafe(vx.vni, 0),
|
||||
port: 0,
|
||||
dstPort: port,
|
||||
serverId: String(server.id),
|
||||
vtepIp: (vx["local-address"] ?? vx["vtep-address"] ?? "").trim(),
|
||||
remoteVteps,
|
||||
l2mtu: parseIntSafe(vx.l2mtu, 1500),
|
||||
arpProxy: rosYes(vx["arp-proxy"]) || vx.arp === "proxy-arp" || vx.arp === "enabled",
|
||||
macLearning: vx["mac-learning"] != null ? rosYes(vx["mac-learning"]) : vx.learning !== "false",
|
||||
comment: vx.comment ?? "",
|
||||
enabled: !disabled,
|
||||
status: !disabled && running ? "up" : "down",
|
||||
}
|
||||
}
|
||||
|
||||
async function fetchVxlanForServer(server: ServerRow): Promise<VxlanTunnelLive[]> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [vxRaw, vtepRaw] = await Promise.all([
|
||||
client.get<RosVxlan[]>("/interface/vxlan"),
|
||||
client.get<RosVxlanVtep[]>("/interface/vxlan/vteps").catch(() => [] as RosVxlanVtep[]),
|
||||
])
|
||||
const list = Array.isArray(vxRaw) ? vxRaw : []
|
||||
const vteps = Array.isArray(vtepRaw) ? vtepRaw : []
|
||||
return list.map((vx, idx) => mapVxlanRow(server, vx, vteps, idx))
|
||||
}
|
||||
|
||||
export async function listVxlanTunnels(): Promise<VxlanTunnelLive[]> {
|
||||
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const results = await Promise.all(
|
||||
enabledServers.map(async (server) => {
|
||||
try {
|
||||
return await fetchVxlanForServer(server)
|
||||
} catch {
|
||||
return [] as VxlanTunnelLive[]
|
||||
}
|
||||
}),
|
||||
)
|
||||
return results.flat()
|
||||
}
|
||||
|
||||
export async function listVxlanTunnelsForServer(server: ServerRow): Promise<VxlanTunnelLive[]> {
|
||||
try {
|
||||
return await fetchVxlanForServer(server)
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
export async function countVxlanTunnels(): Promise<number> {
|
||||
try {
|
||||
const result = await Promise.race([
|
||||
listVxlanTunnels(),
|
||||
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
|
||||
])
|
||||
if (!result) return 0
|
||||
return result.length
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
@@ -3,6 +3,13 @@ import { db } from "../db/index.js"
|
||||
import { servers } from "../db/schema.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import type { WgIfaceDto, WgPeerDto } from "@mmapp/contracts/wireguard"
|
||||
import {
|
||||
canonicalWireguardSnapshot,
|
||||
type WgLiveAddr,
|
||||
type WgLiveIface,
|
||||
type WgLivePeer,
|
||||
type WgSnapshot,
|
||||
} from "./entity-snapshots.js"
|
||||
|
||||
type ServerRow = typeof servers.$inferSelect
|
||||
|
||||
@@ -35,6 +42,7 @@ interface RosWireGuardPeer {
|
||||
"client-address"?: string
|
||||
"client-dns"?: string
|
||||
"client-endpoint"?: string
|
||||
"private-key"?: string
|
||||
}
|
||||
|
||||
interface RosIpAddress {
|
||||
@@ -152,6 +160,85 @@ async function fetchForServer(
|
||||
})
|
||||
}
|
||||
|
||||
export async function fetchWireguardRestoreState(server: ServerRow): Promise<{
|
||||
client: MikrotikClient
|
||||
ifaces: WgLiveIface[]
|
||||
peers: WgLivePeer[]
|
||||
addrs: WgLiveAddr[]
|
||||
snapshot: WgSnapshot
|
||||
}> {
|
||||
const client = MikrotikClient.fromServer(server)
|
||||
const [ifacesRaw, peersRaw, addrsRaw] = await Promise.all([
|
||||
client.get<RosWireGuard[]>("/interface/wireguard"),
|
||||
client.get<RosWireGuardPeer[]>("/interface/wireguard/peers"),
|
||||
client.get<RosIpAddress[]>("/ip/address").catch(() => [] as RosIpAddress[]),
|
||||
])
|
||||
|
||||
const ifaces: WgLiveIface[] = (Array.isArray(ifacesRaw) ? ifacesRaw : []).map((w) => ({
|
||||
name: (w.name ?? "").trim(),
|
||||
rosId: String(w[".id"] ?? w.name ?? ""),
|
||||
listenPort: Number.parseInt(w["listen-port"] ?? "13231", 10) || 13231,
|
||||
mtu: Number.parseInt(w.mtu ?? "1420", 10) || 1420,
|
||||
privateKey: w["private-key"] ?? "",
|
||||
comment: w.comment ?? "",
|
||||
disabled: w.disabled === "true" || w.disabled === "yes",
|
||||
}))
|
||||
|
||||
const peers: WgLivePeer[] = (Array.isArray(peersRaw) ? peersRaw : []).map((p, idx) => {
|
||||
const mapped = mapPeer(p, idx)
|
||||
const ep = (p["endpoint-address"] ?? "").trim()
|
||||
const port = (p["endpoint-port"] ?? "").trim()
|
||||
const ka = p["persistent-keepalive"] ? Number.parseInt(p["persistent-keepalive"], 10) : NaN
|
||||
return {
|
||||
rosId: mapped.rosId,
|
||||
interfaceName: (p.interface ?? "").trim(),
|
||||
publicKey: mapped.publicKey,
|
||||
allowedAddresses: mapped.allowedIps,
|
||||
endpointAddress: ep,
|
||||
endpointPort: port,
|
||||
persistentKeepalive: Number.isFinite(ka) ? ka : null,
|
||||
comment: mapped.comment ?? "",
|
||||
name: mapped.name ?? "",
|
||||
disabled: mapped.disabled === true,
|
||||
privateKey: p["private-key"] ?? "",
|
||||
clientAddress: mapped.clientAddress ?? "",
|
||||
clientDns: mapped.clientDns ?? "",
|
||||
clientEndpoint: mapped.clientEndpoint ?? "",
|
||||
}
|
||||
})
|
||||
|
||||
const addrs: WgLiveAddr[] = []
|
||||
for (const a of Array.isArray(addrsRaw) ? addrsRaw : []) {
|
||||
if (a.disabled === "true" || a.disabled === "yes") continue
|
||||
const iface = (a.interface ?? "").trim()
|
||||
const address = (a.address ?? "").trim()
|
||||
const rosId = String(a[".id"] ?? "")
|
||||
if (!iface || !address || !rosId) continue
|
||||
if (!ifaces.some((i) => i.name === iface)) continue
|
||||
addrs.push({ rosId, interfaceName: iface, address })
|
||||
}
|
||||
|
||||
const snapshot = canonicalWireguardSnapshot({
|
||||
interfaces: ifaces.map((iface) => ({
|
||||
name: iface.name,
|
||||
listenPort: iface.listenPort,
|
||||
mtu: iface.mtu,
|
||||
privateKey: iface.privateKey,
|
||||
address: addrs.find((a) => a.interfaceName === iface.name)?.address ?? "",
|
||||
comment: iface.comment,
|
||||
disabled: iface.disabled,
|
||||
peers: peers.filter((p) => p.interfaceName === iface.name),
|
||||
})),
|
||||
})
|
||||
|
||||
return { client, ifaces, peers, addrs, snapshot }
|
||||
}
|
||||
|
||||
export async function captureWireguardSnapshot(server: ServerRow): Promise<WgSnapshot> {
|
||||
const state = await fetchWireguardRestoreState(server)
|
||||
return state.snapshot
|
||||
}
|
||||
|
||||
export type WgListResult = {
|
||||
interfaces: WgIfaceDto[]
|
||||
failures: Array<{ serverId: string; serverName?: string; error: string }>
|
||||
|
||||
@@ -226,6 +226,19 @@ export interface BackupsRunSnapshot {
|
||||
fatalError?: string
|
||||
}
|
||||
|
||||
export interface GeoipUpdateRunSnapshot {
|
||||
v: typeof SCHEDULER_RUN_SNAPSHOT_VERSION
|
||||
job: "geoip_update"
|
||||
sampledAt: string
|
||||
skipped?: boolean
|
||||
fatalError?: string
|
||||
checked: number
|
||||
downloaded: number
|
||||
skippedUnchanged: number
|
||||
bytes: number
|
||||
errors: string[]
|
||||
}
|
||||
|
||||
export type SchedulerRunSnapshot =
|
||||
| TrafficRunSnapshot
|
||||
| ResourcesRunSnapshot
|
||||
@@ -236,4 +249,5 @@ export type SchedulerRunSnapshot =
|
||||
| InternetPathRunSnapshot
|
||||
| CertificatesRenewRunSnapshot
|
||||
| BackupsRunSnapshot
|
||||
| GeoipUpdateRunSnapshot
|
||||
| AlertEngineRunSnapshot
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user