Compare commits

..
9 Commits
Author SHA1 Message Date
DenozordecandCursor 9c0ee7940e feat(bgp, vxlan, ospf): enhance server data handling and introduce new routes
Docker images / prepare-release (push) Successful in 7s
Docker images / backend-test (push) Successful in 1m40s
Docker images / frontend-image (push) Successful in 2m50s
Docker images / updater-image (push) Successful in 42s
Docker images / backend-image (push) Successful in 2m40s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
- Added support for fetching and displaying server data in BGP, VXLAN, and OSPF pages, improving the overall user experience.
- Introduced new backend routes for OSPF and VXLAN, allowing for better data management and retrieval.
- Implemented mapping functions for backend server data to frontend types, ensuring consistency across components.
- Enhanced the sidebar to display counts for BGP sessions, VXLAN tunnels, and containers, providing users with quick insights into their network status.
- Updated tests to cover new functionalities and ensure reliability.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 11:18:04 +07:00
DenozordecandCursor 5750590b68 feat(traffic-flow): add rebuild facts endpoint and enhance traffic flow analytics
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m6s
Docker images / frontend-image (push) Successful in 3m15s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m50s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 9s
- Introduced a new endpoint `/traffic/flow/rebuild-facts` to rebuild flow facts from buckets, improving data accuracy and management.
- Updated traffic flow analytics to utilize the new `resolveInternetDest` function for better destination resolution.
- Enhanced tests for traffic flow IP handling and added new utility functions for managing internet destinations.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 10:32:51 +07:00
DenozordecandCursor 3c42c114f5 refactor(statistics): improve component structure and enhance state management
Docker images / prepare-release (push) Successful in 11s
Docker images / backend-test (push) Successful in 2m17s
Docker images / frontend-image (push) Successful in 4m11s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 2m18s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 8s
- Introduced Suspense for lazy loading in StatisticsPage to optimize rendering.
- Refactored StatisticsPage to separate inner logic into StatisticsPageInner for better readability.
- Updated PeriodSelector to utilize useCallback for handling state changes, improving performance and clarity.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 09:38:31 +07:00
DenozordecandCursor 5aef419582 fix(statistics): считать уникальный payload без дублей hops
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m10s
Docker images / frontend-image (push) Successful in 3m13s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 2m52s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 10s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 01:27:40 +07:00
DenozordecandCursor 0c0dfa1df7 fix(statistics): не двоить overlay и транзит в кубе
Docker images / prepare-release (push) Successful in 8s
Docker images / backend-test (push) Successful in 2m18s
Docker images / frontend-image (push) Successful in 3m25s
Docker images / updater-image (push) Successful in 48s
Docker images / backend-image (push) Successful in 3m9s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 14s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-11 00:46:25 +07:00
DenozordecandCursor c162a41bc0 fix(network-map): подписать unbound-пути парой серверов
Docker images / prepare-release (push) Successful in 7s
Docker images / backend-test (push) Successful in 2m22s
Docker images / frontend-image (push) Successful in 3m29s
Docker images / updater-image (push) Successful in 52s
Docker images / backend-image (push) Successful in 2m41s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 12s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-10 22:19:05 +07:00
DenozordecandCursor 97e43b2335 feat(statistics): enhance interface handling and data aggregation
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m29s
Docker images / frontend-image (push) Successful in 3m24s
Docker images / updater-image (push) Successful in 44s
Docker images / backend-image (push) Successful in 2m39s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 15s
Updated the statistics aggregation service to improve interface resolution and data handling. Introduced new functions for managing interface aliases and collapsing server interface rows, ensuring accurate data representation. Enhanced test coverage for interface resolution and added checks for new functionality.

- Implemented `factIfaceAliases` and `collapseServerIfaceRows` for better interface data management.
- Updated `resolveIfaceName` to handle additional cases for interface indexing.
- Enhanced tests for interface resolution and aggregation logic.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-10 21:49:50 +07:00
DenozordecandCursor db21e1217c feat(network-map): enhance service selection logic and UI updates
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 2m19s
Docker images / frontend-image (push) Successful in 2m52s
Docker images / updater-image (push) Successful in 46s
Docker images / backend-image (push) Successful in 2m20s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 11s
Refactor the service selection mechanism to utilize a live reference for selected services, ensuring accurate display of service details. Update UI components to reflect changes in service selection, including dynamic updates for service metrics and paths. Additionally, introduce new utility functions for deduplicating flow rows across exporters in the backend, improving data handling in traffic flow analysis.

- Updated service selection logic in NetworkMapPage component.
- Enhanced UI to display live service data.
- Added deduplication functions in traffic-flow-dedup module for improved data integrity.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-10 19:54:16 +07:00
DenozordecandCursor 5bb9066be8 feat(statistics): добавить BI-разрез и сводную матрицу
Docker images / prepare-release (push) Successful in 10s
Docker images / backend-test (push) Successful in 2m23s
Docker images / frontend-image (push) Successful in 3m30s
Docker images / updater-image (push) Successful in 52s
Docker images / backend-image (push) Successful in 3m8s
Docker images / notify-webhook (push) Skipped
Docker images / publish-release (push) Successful in 14s
Сопоставить клиентов с ifIndex как на карте трафика, чтобы KPI пользователей не обнулялся. На экране — разрез остальных измерений и сводная матрица.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-10 16:07:25 +07:00
64 changed files with 4356 additions and 524 deletions

No files matched your search

@@ -0,0 +1,63 @@
# Локальные GeoLite2-базы (Country + ASN) для потоков по странам и ASN
## Контекст
Сейчас страна и ASN для netflow-потоков резолвятся через внешний RIPEstat API (`backend/src/services/traffic-flow-ripe.ts`): лимит 30 новых префиксов/мин, очередь на 90, кэш в PG `flow_ip_meta`. Новые IP «дозревают» с задержкой, IPv6 не покрывается (кэш индексируется только по IPv4). Локальные mmdb-базы дают мгновенный синхронный lookup всех IP без внешних вызовов.
Решения (подтверждены):
- Источник — **MaxMind GeoLite2 через P3TERX-зеркало**: `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb` и `.../GeoLite2-ASN.mmdb`. Без регистрации, ключей и tar-распаковки. Точность по стране у GeoLite2 и IPinfo паритетная (<1% ошибок у обоих, arXiv 2026); выбран P3TERX за надёжность зеркала (5.2k звёзд) и преемственность: текущий RIPE-путь и так читает GeoLite (`maxmind-geo-lite`), история в кэше остаётся консистентной.
- **RIPEstat остаётся fallback** (до первой загрузки баз / если lookup не дал результата).
- City-базу не качаем (lat/lng фронтенд не использует).
## Изменения
### 1. Зависимость
- `npm install -w mikrotik-manager-backend maxmind` — sync-чтение mmdb, встроенные TS-типы, без транзитивных зависимостей, Node 22 ок.
### 2. Новый сервис `backend/src/services/traffic-flow-geoip.ts`
(по конвенциям окружения traffic-flow-*: контракт → маршрут → сервис, без БД-логики в маршрутах)
- Каталог: `backend/storage/geoip/` (конвенция `storage/backups`), файлы `GeoLite2-Country.mmdb`, `GeoLite2-ASN.mmdb`.
- `initGeoip()` — открыть ридеры best-effort при старте (из `index.ts` рядом с `startTrafficFlowListener`), независимо от настроек автообновления: файлы есть — работают.
- `lookupGeoip(ip): FlowIpMeta | null` — синхронно: `country.iso_code` (fallback `registered_country.iso_code`) с валидацией `isIsoCountry`, ASN = `autonomous_system_number`, holder = `autonomous_system_organization`; приватные IP → negative-запись как в RIPE (`isNonPublicIp`); в PG не пишем (lookup и так быстрый). IPv6 поддержан ридером.
- `resolveFlowIp(ip)` — фасад: `lookupGeoip(ip) ?? lookupRipeCached(ip)`; главный экспорт для потребителей.
- `geoipStatus()` — loaded, даты сборки баз (метаданные mmdb). Тест-хук `setGeoipReadersForTests`. Смена ридеров после обновления — атомарная замена ссылок.
### 3. Коллектор `backend/src/services/geoip-update-collector.ts`
`collectGeoipUpdateOnce()` по образцу `certificate-renew-collector.ts`:
1. Conditional GET с ETag/If-None-Match из настроек → 304 = skip (фолбэк-сравнение: размер/содержимое).
2. Скачивание в `*.tmp` через глобальный `fetch` + AbortController с таймаутом (внешний HTTP из service-слоя — по правилу fastify-backend-drizzle).
3. Валидация: открыть ридер из tmp-файла, пробой 8.8.8.8 (страна US, ASN 15169).
4. `fs.rename` атомарная подмена, старый файл → `*.prev` (откат, если новый ридер не открылся).
5. Перезагрузка ридеров, статус в настройках; snapshot для `scheduler_runs` (checked/downloaded/skipped/bytes/error).
### 4. Планировщик (`backend/src/services/scheduler.ts`)
- `JOB_KEYS` += `geoip_update`; case в `runSchedulerJobBody`; блок в `refreshScheduler()` по образцу `certificates_renew`: интервал `Math.max(6ч, updateIntervalSec*1000)`, по умолчанию 7 дней (upstream обновляется еженедельно) + немедленный первый запуск при включённой настройке.
### 5. Схема и миграция
- `backend/src/db/schema.ts`: singleton `geoip_settings` — `enabled` (default true), `updateIntervalSec` (default 604800), `lastCheckAt`, `lastSuccessAt`, `lastError`, `countryBuildAt`, `asnBuildAt`, `etagsJson` (jsonb), `createdAt/updatedAt`.
- Миграция: `npm run db:generate` → файл в `backend/drizzle/`.
### 6. API + контракты
- `packages/contracts/src/geoip.ts`: zod-схемы настроек/статуса (все входы — Zod, по правилам проекта).
- Новый `backend/src/routes/geoip.ts`, регистрация в `index.ts` с prefix `/api`:
- `GET /api/geoip` — настройки + статус (ready, даты сборки, последняя проверка/ошибка);
- `PUT /api/geoip` — сохранить настройки, затем `refreshScheduler()`;
- `POST /api/geoip/update` — запустить загрузку сейчас (409, если уже идёт; флаг-гард как в коллекторах).
### 7. Интеграция в пайплайн (geoip-first, RIPE-fallback)
- `traffic-flow-engine.ts` (`queueParsedFlows`, ~строка 336): `lookupRipeCached` → `resolveFlowIp`. Логика misses не меняется: при готовом mmdb публичные IP (v4+v6) резолвятся сразу, очередь RIPE пустеет; до скачивания баз — прежнее поведение.
- Остальные вызовы `lookupRipeCached` → `resolveFlowIp` (grep: как минимум `traffic-flow-analytics.ts` ~258–271).
- `classifyFlowDst`/бренды не трогаем: holder из mmdb (org name) встаёт в существующие `HOLDER_BRANDS`-регулярки как есть.
### 8. Frontend (по next-shadcn-production / ui-guardian: только переиспользование)
- Секция «GeoIP-базы (GeoLite2)» внутри существующей `components/traffic/netflow-settings-panel.tsx`: статус (готово/не скачано, даты сборки Country/ASN, последняя проверка, ошибка), тумблер автообновления, интервал, кнопка «Обновить сейчас» с индикатором. Только уже используемые в панели примитивы (Switch/Button/поля) — никаких новых визуальных паттернов и Card-shell. API-клиент через существующие http-хелперы.
### 9. Хаускипинг, тесты, проверки
- `backend/.gitignore`: `storage/geoip/`.
- Тесты `backend/src/services/traffic-flow-geoip.test.ts` + скрипт `test:geoip` (по образцу `test:traffic-flow`): приоритет фасада (geoip hit → RIPE не зовётся; miss → fallback), negative на приватных IP, фильтрация EU/ZZ через `isIsoCountry`, коллектор с мокнутым fetch (304-skip, битый файл → подмены нет, `.prev` сохранён), dims по стране/ASN с засеянным ридером.
- Проверки после реализации (обязательно по правилам): типы/сборка бэка (`npm run build -w mikrotik-manager-backend`), типы фронта при правке UI (`npx tsc --noEmit`), `npm run test:geoip` и `test:traffic-flow`; предупреждения не игнорировать.
- Коммит: `feat(netflow): <subject по-русски>` — новая пользовательская фича (мгновенные страна/ASN в потоках), по commit-messages-ru.
- README: короткий раздел о GeoIP; примечание, что в Docker `storage/geoip` ephemeral без тома — базы перекачаются после пересоздания контейнера (~17 МБ); при желании смонтировать volume.
## Что это даёт
Страна и ASN появляются у потока мгновенно при ingest (включая IPv6), без ограничения скорости RIPE; dims `country`/`asn` в `flow_daily_dims`, аналитика (карта, топы, monthly) становятся полными сразу. Внешняя зависимость от stat.ripe.net остаётся только как fallback до первой загрузки баз.
+130 -42
View File
@@ -1,6 +1,6 @@
"use client" "use client"
import { Fragment, useState, useMemo, useEffect } from "react" import { useState, useMemo, useEffect } from "react"
import { PageHeader } from "@/components/page-header" import { PageHeader } from "@/components/page-header"
import { DataPageToolbar } from "@/components/data-page-toolbar" import { DataPageToolbar } from "@/components/data-page-toolbar"
import { BgpSessionsDataGrid } from "@/components/data-grids/bgp-sessions-data-grid" import { BgpSessionsDataGrid } from "@/components/data-grids/bgp-sessions-data-grid"
@@ -25,6 +25,9 @@ import {
} from "lucide-react" } from "lucide-react"
import { useDataSource } from "@/lib/data-source" import { useDataSource } from "@/lib/data-source"
import { requestJson } from "@/shared/api/http-client" import { requestJson } from "@/shared/api/http-client"
import { servers as mockServers, type Server } from "@/lib/data"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
// ─── types ──────────────────────────────────────────────────────────────────── // ─── types ────────────────────────────────────────────────────────────────────
@@ -233,6 +236,37 @@ interface BackendBgpSession {
capabilities: string[]; lastError: string | null capabilities: string[]; lastError: string | null
} }
interface BackendServer {
id: number
name: string
host: string
type?: Server["type"]
site?: string
country: string
asn?: string
enabled: boolean
status?: Server["status"]
latency?: number | null
}
function mapBackendServer(s: BackendServer): Server {
return {
id: String(s.id),
name: s.name || s.host,
host: s.host,
model: "—",
os: "—",
site: s.site ?? "",
country: s.country || "UN",
asn: s.asn ?? "",
type: s.type ?? "exit-node",
enabled: s.enabled,
status: s.status ?? "online",
latency: s.latency ?? null,
sessions: 0,
}
}
function backendToFrontend(b: BackendBgpSession): BgpSession { function backendToFrontend(b: BackendBgpSession): BgpSession {
return { return {
id: `${b.serverId}-${b.id}`, id: `${b.serverId}-${b.id}`,
@@ -623,27 +657,40 @@ const TABS: Array<{ id: BgpTab; label: string; icon: React.ReactNode }> = [
export default function BgpPage() { export default function BgpPage() {
const [activeTab, setActiveTab] = useState<BgpTab>("sessions") const [activeTab, setActiveTab] = useState<BgpTab>("sessions")
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const { mode, backendUrl } = useDataSource() const { mode, backendUrl } = useDataSource()
const isLive = mode === "live" const isLive = mode === "live"
const [liveSessions, setLiveSessions] = useState<BgpSession[]>([]) const [liveSessions, setLiveSessions] = useState<BgpSession[]>([])
const [liveServers, setLiveServers] = useState<Server[]>([])
const [loading, setLoading] = useState(false) const [loading, setLoading] = useState(false)
const [fetchedAt, setFetchedAt] = useState<Date | null>(null) const [fetchedAt, setFetchedAt] = useState<Date | null>(null)
const [liveError, setLiveError] = useState<string | null>(null) const [liveError, setLiveError] = useState<string | null>(null)
const [fetchTick, setFetchTick] = useState(0) const [fetchTick, setFetchTick] = useState(0)
useEffect(() => { useEffect(() => {
if (!isLive) return if (!isLive) {
queueMicrotask(() => {
setLiveSessions([])
setLiveServers([])
setLiveError(null)
})
return
}
let cancelled = false let cancelled = false
queueMicrotask(() => { queueMicrotask(() => {
if (cancelled) return if (cancelled) return
setLoading(true) setLoading(true)
setLiveError(null) setLiveError(null)
void requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions") void Promise.all([
.then(data => { requestJson<BackendBgpSession[]>(backendUrl, "/api/bgp/sessions"),
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
])
.then(([data, servers]) => {
if (cancelled) return if (cancelled) return
setLiveSessions(data.map(backendToFrontend)) setLiveSessions(data.map(backendToFrontend))
setLiveServers(servers.filter((s) => s.enabled).map(mapBackendServer))
setFetchedAt(new Date()) setFetchedAt(new Date())
setLoading(false) setLoading(false)
}) })
@@ -656,50 +703,87 @@ export default function BgpPage() {
return () => { cancelled = true } return () => { cancelled = true }
}, [isLive, backendUrl, fetchTick]) }, [isLive, backendUrl, fetchTick])
// Use live or mock data for all tabs and KPI const allSessions = isLive ? liveSessions : SESSIONS
const sessions = isLive ? liveSessions : SESSIONS const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
const effectiveServerId =
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
? selectedServerId
: ALL_SERVERS_ID
const sessions = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return allSessions
return allSessions.filter((s) => s.serverId === effectiveServerId)
}, [allSessions, effectiveServerId])
const railItems = useMemo<ServerTileItem[]>(() => {
const counts = new Map<string, number>()
for (const s of allSessions) {
counts.set(s.serverId, (counts.get(s.serverId) ?? 0) + 1)
}
return displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
site: s.site,
country: s.country,
status: s.status,
type: s.type,
count: counts.get(s.id) ?? 0,
enabled: s.enabled,
title: [s.name, s.host, s.asn].filter(Boolean).join(" · "),
}))
}, [displayServers, allSessions])
const established = sessions.filter(s => s.state === "Established").length const established = sessions.filter(s => s.state === "Established").length
const notEstab = sessions.length - established const notEstab = sessions.length - established
const totalRx = sessions.reduce((a, s) => a + s.prefixesRx, 0) const totalRx = sessions.reduce((a, s) => a + s.prefixesRx, 0)
const serverCount = useMemo( const serverCount = useMemo(
() => new Set(liveSessions.map(s => s.serverId)).size, () => new Set(sessions.map(s => s.serverId)).size,
[liveSessions], [sessions],
) )
return ( return (
<div className="flex flex-col h-full"> <ServerRailLayout
<PageHeader items={railItems}
crumbs={[{ label: "Управление" }, { label: "BGP" }]} selectedId={effectiveServerId}
actions={ onSelect={setSelectedServerId}
<> showAll
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}> allCount={displayServers.length}
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} /> loading={isLive && loading && displayServers.length === 0}
Обновить header={
</Button> <PageHeader
<Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button> crumbs={[{ label: "Управление" }, { label: "BGP" }]}
</> actions={
} <>
/> <ServerRailMobileButton />
<Button variant="outline" size="sm" onClick={() => setFetchTick(t => t + 1)}>
{/* tab bar */} <RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
<div className="border-b bg-background shrink-0"> Обновить
<div className="flex items-center px-6"> </Button>
{TABS.map(t => ( <Button variant="outline" size="sm"><DownloadIcon className="size-4" />Экспорт</Button>
<button key={t.id} onClick={() => setActiveTab(t.id)} </>
className={cn( }
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px", />
activeTab === t.id }
? "border-primary text-foreground" banner={
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border", <div className="border-b bg-background shrink-0">
)}> <div className="flex items-center px-6">
{t.icon}{t.label} {TABS.map(t => (
</button> <button key={t.id} onClick={() => setActiveTab(t.id)}
))} className={cn(
"flex items-center gap-2 px-4 py-3 text-sm font-medium border-b-2 transition-colors -mb-px",
activeTab === t.id
? "border-primary text-foreground"
: "border-transparent text-muted-foreground hover:text-foreground hover:border-border",
)}>
{t.icon}{t.label}
</button>
))}
</div>
</div> </div>
</div> }
>
<div className="flex-1 overflow-y-auto p-6">
<div className="flex flex-col gap-5"> <div className="flex flex-col gap-5">
{/* data source banner */} {/* data source banner */}
@@ -729,11 +813,16 @@ export default function BgpPage() {
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription> <AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
</Alert> </Alert>
)} )}
{isLive && !loading && liveSessions.length === 0 && !liveError && fetchedAt && ( {isLive && !loading && allSessions.length === 0 && !liveError && fetchedAt && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground"> <div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
BGP не настроен ни на одном сервере BGP не настроен ни на одном сервере
</div> </div>
)} )}
{isLive && !loading && allSessions.length > 0 && sessions.length === 0 && !liveError && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
На выбранном сервере нет BGP-сессий
</div>
)}
{mode === "mock" && ( {mode === "mock" && (
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground"> <span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
Моковые данные Моковые данные
@@ -794,7 +883,6 @@ export default function BgpPage() {
{activeTab === "analytics" && <AnalyticsTab sessions={sessions} />} {activeTab === "analytics" && <AnalyticsTab sessions={sessions} />}
</div> </div>
</div> </ServerRailLayout>
</div>
) )
} }
+253 -70
View File
@@ -1,15 +1,17 @@
"use client" "use client"
import { useMemo, useState } from "react" import { useCallback, useEffect, useMemo, useState } from "react"
import { PageHeader } from "@/components/page-header" import { PageHeader } from "@/components/page-header"
import { routerContainers, servers } from "@/lib/data" import { routerContainers as mockContainers, servers as mockServers } from "@/lib/data"
import type { RouterContainer } from "@/lib/data" import type { RouterContainer, Server } from "@/lib/data"
import { Flag } from "@/components/flag" import { Flag } from "@/components/flag"
import { Frame, FramePanel } from "@/components/reui/frame" import { Frame, FramePanel } from "@/components/reui/frame"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid" import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { OpsPanel } from "@/components/ops-panel" import { OpsPanel } from "@/components/ops-panel"
import { Button } from "@/components/ui/button" import { Button } from "@/components/ui/button"
import { Alert, AlertDescription } from "@/components/ui/alert"
import { cn } from "@/lib/utils" import { cn } from "@/lib/utils"
import { toast } from "sonner"
import { import {
DropdownMenu, DropdownMenuTrigger, DropdownMenuContent, DropdownMenu, DropdownMenuTrigger, DropdownMenuContent,
DropdownMenuItem, DropdownMenuSeparator, DropdownMenuItem, DropdownMenuSeparator,
@@ -18,18 +20,47 @@ import {
BoxIcon, PlayIcon, StopCircleIcon, SearchIcon, BoxIcon, PlayIcon, StopCircleIcon, SearchIcon,
MoreHorizontalIcon, Trash2Icon, PencilIcon, PowerIcon, MoreHorizontalIcon, Trash2Icon, PencilIcon, PowerIcon,
CodeXmlIcon, ActivityIcon, ServerIcon, CodeXmlIcon, ActivityIcon, ServerIcon,
TerminalIcon, AlertCircleIcon, TerminalIcon, AlertCircleIcon, RefreshCwIcon,
} from "lucide-react" } from "lucide-react"
import {
Sheet, SheetContent, SheetHeader, SheetTitle,
SheetDescription, SheetFooter, SheetClose,
} from "@/components/ui/sheet"
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet" import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
import { useDataSource } from "@/lib/data-source"
import { requestJson } from "@/shared/api/http-client"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
// ─── helpers ────────────────────────────────────────────────────────────────── interface BackendServer {
id: number
name: string
host: string
type?: Server["type"]
site?: string
country: string
asn?: string
enabled: boolean
status?: Server["status"]
latency?: number | null
}
function serverFor(id: string) { interface ContainersApiResponse {
return servers.find((s) => s.id === id) containers: RouterContainer[]
}
function mapBackendServer(s: BackendServer): Server {
return {
id: String(s.id),
name: s.name || s.host,
host: s.host,
model: "—",
os: "—",
site: s.site ?? "",
country: s.country || "UN",
asn: s.asn ?? "",
type: s.type ?? "exit-node",
enabled: s.enabled,
status: s.status ?? "online",
latency: s.latency ?? null,
sessions: 0,
}
} }
function statusConfig(status: RouterContainer["status"]) { function statusConfig(status: RouterContainer["status"]) {
@@ -52,10 +83,8 @@ function statusConfig(status: RouterContainer["status"]) {
}[status] }[status]
} }
// ─── RSC generator ──────────────────────────────────────────────────────────── function generateContainerRsc(c: RouterContainer, serverById: Record<string, Server>): string {
const srv = serverById[c.serverId]
function generateContainerRsc(c: RouterContainer): string {
const srv = serverFor(c.serverId)
const lines: string[] = [] const lines: string[] = []
lines.push(`# RouterOS Container — ${c.name}`) lines.push(`# RouterOS Container — ${c.name}`)
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`) if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
@@ -63,13 +92,11 @@ function generateContainerRsc(c: RouterContainer): string {
lines.push(`# RouterOS 7.4+ · /container`) lines.push(`# RouterOS 7.4+ · /container`)
lines.push(``) lines.push(``)
// interface
for (const iface of c.interfaces) { for (const iface of c.interfaces) {
lines.push(`/interface/veth/add name=${iface} address=172.17.0.2/24 gateway=172.17.0.1`) lines.push(`/interface/veth/add name=${iface} address=172.17.0.2/24 gateway=172.17.0.1`)
} }
lines.push(``) lines.push(``)
// envs
if (c.envs.length > 0) { if (c.envs.length > 0) {
lines.push(`/container/envs/add name=${c.name}-envs \\`) lines.push(`/container/envs/add name=${c.name}-envs \\`)
for (const { key, value } of c.envs) { for (const { key, value } of c.envs) {
@@ -78,7 +105,6 @@ function generateContainerRsc(c: RouterContainer): string {
lines.push(``) lines.push(``)
} }
// mounts
for (const m of c.mounts) { for (const m of c.mounts) {
lines.push(`/container/mounts/add name=${c.name}-mount-${m.dst.replace(/\//g, "-").slice(1)} \\`) lines.push(`/container/mounts/add name=${c.name}-mount-${m.dst.replace(/\//g, "-").slice(1)} \\`)
if (m.src) lines.push(` src=${m.src} \\`) if (m.src) lines.push(` src=${m.src} \\`)
@@ -86,7 +112,6 @@ function generateContainerRsc(c: RouterContainer): string {
lines.push(``) lines.push(``)
} }
// container
lines.push(`/container/add \\`) lines.push(`/container/add \\`)
lines.push(` remote-image=${c.image}:${c.tag} \\`) lines.push(` remote-image=${c.image}:${c.tag} \\`)
lines.push(` interface=${c.interfaces[0] ?? "veth-container"} \\`) lines.push(` interface=${c.interfaces[0] ?? "veth-container"} \\`)
@@ -100,12 +125,18 @@ function generateContainerRsc(c: RouterContainer): string {
return lines.join("\n") return lines.join("\n")
} }
// ─── Export Sheet ───────────────────────────────────────────────────────────── function ExportSheet({
open, container, onClose, serverById,
function ExportSheet({ open, container, onClose }: { }: {
open: boolean; container: RouterContainer | null; onClose: () => void open: boolean
container: RouterContainer | null
onClose: () => void
serverById: Record<string, Server>
}) { }) {
const code = useMemo(() => container ? generateContainerRsc(container) : "", [container]) const code = useMemo(
() => (container ? generateContainerRsc(container, serverById) : ""),
[container, serverById],
)
return ( return (
<CodeExportSheet <CodeExportSheet
@@ -125,17 +156,29 @@ function ExportSheet({ open, container, onClose }: {
) )
} }
// ─── Container card ───────────────────────────────────────────────────────────
function ContainerCard({ function ContainerCard({
container, container,
server,
live,
busy,
onExport, onExport,
onStart,
onStop,
onRestart,
onRemove,
}: { }: {
container: RouterContainer container: RouterContainer
server?: Server
live: boolean
busy: boolean
onExport: () => void onExport: () => void
onStart: () => void
onStop: () => void
onRestart: () => void
onRemove: () => void
}) { }) {
const srv = serverFor(container.serverId)
const cfg = statusConfig(container.status) const cfg = statusConfig(container.status)
const canMutate = live && Boolean(container.rosId)
return ( return (
<Frame dense className="w-full overflow-hidden"> <Frame dense className="w-full overflow-hidden">
@@ -150,29 +193,36 @@ function ContainerCard({
</div> </div>
<DropdownMenu> <DropdownMenu>
<DropdownMenuTrigger render={ <DropdownMenuTrigger render={
<Button variant="ghost" size="icon" className="size-7 shrink-0"> <Button variant="ghost" size="icon" className="size-7 shrink-0" disabled={busy}>
<MoreHorizontalIcon className="size-4" /> <MoreHorizontalIcon className="size-4" />
</Button> </Button>
} /> } />
<DropdownMenuContent side="bottom" align="end"> <DropdownMenuContent side="bottom" align="end">
{container.status === "running" ? ( {container.status === "running" ? (
<DropdownMenuItem><StopCircleIcon className="size-4 text-amber-500" />Остановить</DropdownMenuItem> <DropdownMenuItem disabled={!canMutate} onClick={onStop}>
<StopCircleIcon className="size-4 text-amber-500" />Остановить
</DropdownMenuItem>
) : ( ) : (
<DropdownMenuItem><PlayIcon className="size-4 text-emerald-500" />Запустить</DropdownMenuItem> <DropdownMenuItem disabled={!canMutate} onClick={onStart}>
<PlayIcon className="size-4 text-emerald-500" />Запустить
</DropdownMenuItem>
)} )}
<DropdownMenuItem><TerminalIcon className="size-4" />Логи</DropdownMenuItem> <DropdownMenuItem disabled><TerminalIcon className="size-4" />Логи</DropdownMenuItem>
<DropdownMenuItem><PencilIcon className="size-4" />Редактировать</DropdownMenuItem> <DropdownMenuItem disabled><PencilIcon className="size-4" />Редактировать</DropdownMenuItem>
<DropdownMenuItem onClick={onExport}><CodeXmlIcon className="size-4" />Экспорт .rsc</DropdownMenuItem> <DropdownMenuItem onClick={onExport}><CodeXmlIcon className="size-4" />Экспорт .rsc</DropdownMenuItem>
<DropdownMenuSeparator /> <DropdownMenuSeparator />
<DropdownMenuItem><PowerIcon className="size-4" />Перезапустить</DropdownMenuItem> <DropdownMenuItem disabled={!canMutate} onClick={onRestart}>
<PowerIcon className="size-4" />Перезапустить
</DropdownMenuItem>
<DropdownMenuSeparator /> <DropdownMenuSeparator />
<DropdownMenuItem variant="destructive"><Trash2Icon className="size-4" />Удалить</DropdownMenuItem> <DropdownMenuItem variant="destructive" disabled={!canMutate} onClick={onRemove}>
<Trash2Icon className="size-4" />Удалить
</DropdownMenuItem>
</DropdownMenuContent> </DropdownMenuContent>
</DropdownMenu> </DropdownMenu>
</div> </div>
<div className="px-4 py-3 flex flex-col gap-3"> <div className="px-4 py-3 flex flex-col gap-3">
{/* image */}
<div className="flex items-center gap-2"> <div className="flex items-center gap-2">
<BoxIcon className="size-3.5 text-muted-foreground shrink-0" /> <BoxIcon className="size-3.5 text-muted-foreground shrink-0" />
<span className="font-mono text-xs text-foreground/80"> <span className="font-mono text-xs text-foreground/80">
@@ -180,17 +230,15 @@ function ContainerCard({
</span> </span>
</div> </div>
{/* server */} {server && (
{srv && (
<div className="flex items-center gap-2 text-xs text-muted-foreground"> <div className="flex items-center gap-2 text-xs text-muted-foreground">
<ServerIcon className="size-3.5 shrink-0" /> <ServerIcon className="size-3.5 shrink-0" />
<Flag code={srv.country} size={12} /> <Flag code={server.country} size={12} />
<span className="font-mono">{srv.name}</span> <span className="font-mono">{server.name}</span>
</div> </div>
)} )}
{/* uptime + stats */} {container.status === "running" && (container.uptime || container.cpu !== undefined || container.memMb !== undefined) && (
{container.status === "running" && (
<div className="flex items-center gap-4 text-xs text-muted-foreground border-t pt-2.5"> <div className="flex items-center gap-4 text-xs text-muted-foreground border-t pt-2.5">
{container.uptime && ( {container.uptime && (
<div className="flex items-center gap-1"> <div className="flex items-center gap-1">
@@ -216,7 +264,6 @@ function ContainerCard({
</div> </div>
)} )}
{/* interfaces */}
{container.interfaces.length > 0 && ( {container.interfaces.length > 0 && (
<div className="flex flex-wrap gap-1"> <div className="flex flex-wrap gap-1">
{container.interfaces.map((i) => ( {container.interfaces.map((i) => (
@@ -227,7 +274,6 @@ function ContainerCard({
</div> </div>
)} )}
{/* mounts */}
{container.mounts.length > 0 && ( {container.mounts.length > 0 && (
<div className="flex flex-col gap-1"> <div className="flex flex-col gap-1">
{container.mounts.map((m, idx) => ( {container.mounts.map((m, idx) => (
@@ -249,50 +295,183 @@ function ContainerCard({
) )
} }
// ════════════════════════════════════════════════════════════════════════════
export default function ContainersPage() { export default function ContainersPage() {
const [search, setSearch] = useState("") const { mode, backendUrl } = useDataSource()
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all") const isLive = mode === "live"
const [search, setSearch] = useState("")
const [statusFilter, setStatusFilter] = useState<RouterContainer["status"] | "all">("all")
const [exportContainer, setExportContainer] = useState<RouterContainer | null>(null) const [exportContainer, setExportContainer] = useState<RouterContainer | null>(null)
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const [liveContainers, setLiveContainers] = useState<RouterContainer[]>([])
const [liveServers, setLiveServers] = useState<Server[]>([])
const [loading, setLoading] = useState(false)
const [busyId, setBusyId] = useState<string | null>(null)
const [liveError, setLiveError] = useState<string | null>(null)
const loadLive = useCallback(async () => {
if (!isLive) return
setLoading(true)
setLiveError(null)
try {
const [cRes, sRes] = await Promise.all([
requestJson<ContainersApiResponse>(backendUrl, "/api/containers"),
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
])
setLiveContainers(cRes.containers ?? [])
setLiveServers(sRes.filter((s) => s.enabled).map(mapBackendServer))
} catch (e) {
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
setLiveContainers([])
} finally {
setLoading(false)
}
}, [isLive, backendUrl])
useEffect(() => {
if (!isLive) {
queueMicrotask(() => {
setLiveContainers([])
setLiveServers([])
setLiveError(null)
})
return
}
queueMicrotask(() => {
void loadLive()
})
}, [isLive, loadLive])
const displayContainers = isLive ? liveContainers : mockContainers
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
const effectiveServerId =
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
? selectedServerId
: ALL_SERVERS_ID
const scoped = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return displayContainers
return displayContainers.filter((c) => c.serverId === effectiveServerId)
}, [displayContainers, effectiveServerId])
const serverById = useMemo(
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
[displayServers],
)
const railItems = useMemo<ServerTileItem[]>(() => (
displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
site: s.site,
country: s.country,
status: s.status,
type: s.type,
enabled: s.enabled,
meta: String(displayContainers.filter((c) => c.serverId === s.id).length),
}))
), [displayServers, displayContainers])
const filtered = useMemo(() => { const filtered = useMemo(() => {
return routerContainers.filter((c) => { return scoped.filter((c) => {
if (statusFilter !== "all" && c.status !== statusFilter) return false if (statusFilter !== "all" && c.status !== statusFilter) return false
if (!search) return true if (!search) return true
const q = search.toLowerCase() const q = search.toLowerCase()
return ( return (
c.name.toLowerCase().includes(q) || c.name.toLowerCase().includes(q) ||
c.image.toLowerCase().includes(q) || c.image.toLowerCase().includes(q) ||
(serverFor(c.serverId)?.name.toLowerCase().includes(q) ?? false) (serverById[c.serverId]?.name.toLowerCase().includes(q) ?? false)
) )
}) })
}, [search, statusFilter]) }, [search, statusFilter, scoped, serverById])
const running = routerContainers.filter((c) => c.status === "running").length const running = scoped.filter((c) => c.status === "running").length
const stopped = routerContainers.filter((c) => c.status === "stopped").length const stopped = scoped.filter((c) => c.status === "stopped").length
const errors = routerContainers.filter((c) => c.status === "error").length const errors = scoped.filter((c) => c.status === "error").length
async function mutate(c: RouterContainer, action: "start" | "stop" | "restart" | "remove") {
if (!isLive || !c.rosId) {
toast.info("Действие доступно только в live-режиме")
return
}
if (action === "remove" && !window.confirm(`Удалить контейнер ${c.name}?`)) return
setBusyId(c.id)
try {
await requestJson(backendUrl, `/api/servers/${c.serverId}/containers/${action}`, {
method: "POST",
body: JSON.stringify({ rosId: c.rosId }),
})
const labels = { start: "запущен", stop: "остановлен", restart: "перезапущен", remove: "удалён" }
toast.success(`${c.name}: ${labels[action]}`)
await loadLive()
} catch (e) {
toast.error(e instanceof Error ? e.message : "Ошибка RouterOS")
} finally {
setBusyId(null)
}
}
return ( return (
<div className="flex flex-col h-full"> <>
<PageHeader <ServerRailLayout
crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]} items={railItems}
actions={ selectedId={effectiveServerId}
<Button size="sm"> onSelect={setSelectedServerId}
<BoxIcon className="size-4" />Новый контейнер showAll
</Button> allCount={displayServers.length}
} loading={isLive && loading && displayServers.length === 0}
/> header={
<PageHeader
<div className="flex-1 overflow-y-auto p-6"> crumbs={[{ label: "Управление" }, { label: "Контейнеры" }]}
actions={
<>
<ServerRailMobileButton />
<Button
variant="outline"
size="sm"
onClick={() => { void loadLive() }}
disabled={!isLive || loading}
>
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
Обновить
</Button>
<Button size="sm">
<BoxIcon className="size-4" />Новый контейнер
</Button>
</>
}
/>
}
>
<div className="flex flex-col gap-5"> <div className="flex flex-col gap-5">
{isLive && liveError && (
<Alert variant="warning" className="py-2">
<AlertCircleIcon />
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
</Alert>
)}
{isLive && !loading && displayContainers.length === 0 && !liveError && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
Контейнеры не найдены. Нужен пакет container (RouterOS 7.4+).
</div>
)}
{mode === "mock" && (
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
Моковые данные
</span>
)}
<KpiStatGrid <KpiStatGrid
aria-label="Сводка контейнеров" aria-label="Сводка контейнеров"
items={[ items={[
{ {
id: "all", id: "all",
label: "Всего", label: "Всего",
value: routerContainers.length, value: scoped.length,
icon: <BoxIcon className="size-4" />, icon: <BoxIcon className="size-4" />,
iconClassName: "text-muted-foreground", iconClassName: "text-muted-foreground",
}, },
@@ -321,7 +500,6 @@ export default function ContainersPage() {
]} ]}
/> />
{/* Info banner */}
<div className="flex items-start gap-3 rounded-lg bg-violet-500/5 border border-violet-500/20 px-4 py-3 text-sm"> <div className="flex items-start gap-3 rounded-lg bg-violet-500/5 border border-violet-500/20 px-4 py-3 text-sm">
<BoxIcon className="size-5 text-violet-500 shrink-0 mt-0.5" /> <BoxIcon className="size-5 text-violet-500 shrink-0 mt-0.5" />
<div> <div>
@@ -333,7 +511,6 @@ export default function ContainersPage() {
</div> </div>
</div> </div>
{/* Toolbar */}
<div className="flex items-center gap-3 flex-wrap"> <div className="flex items-center gap-3 flex-wrap">
<div className="flex items-center gap-2 h-8 px-3 border border-input rounded-md bg-background min-w-[240px]"> <div className="flex items-center gap-2 h-8 px-3 border border-input rounded-md bg-background min-w-[240px]">
<SearchIcon className="size-3.5 text-muted-foreground shrink-0" /> <SearchIcon className="size-3.5 text-muted-foreground shrink-0" />
@@ -363,7 +540,6 @@ export default function ContainersPage() {
<span className="text-sm text-muted-foreground ml-auto">{filtered.length} контейнеров</span> <span className="text-sm text-muted-foreground ml-auto">{filtered.length} контейнеров</span>
</div> </div>
{/* Grid */}
{filtered.length === 0 ? ( {filtered.length === 0 ? (
<div className="flex flex-col items-center justify-center py-16 text-center text-muted-foreground"> <div className="flex flex-col items-center justify-center py-16 text-center text-muted-foreground">
<BoxIcon className="size-10 mb-3 opacity-20" /> <BoxIcon className="size-10 mb-3 opacity-20" />
@@ -376,13 +552,19 @@ export default function ContainersPage() {
<ContainerCard <ContainerCard
key={c.id} key={c.id}
container={c} container={c}
server={serverById[c.serverId]}
live={isLive}
busy={busyId === c.id}
onExport={() => setExportContainer(c)} onExport={() => setExportContainer(c)}
onStart={() => { void mutate(c, "start") }}
onStop={() => { void mutate(c, "stop") }}
onRestart={() => { void mutate(c, "restart") }}
onRemove={() => { void mutate(c, "remove") }}
/> />
))} ))}
</div> </div>
)} )}
{/* RouterOS reference */}
<OpsPanel title="RouterOS 7.4+ · /container — быстрые команды" contentClassName="px-5 py-4"> <OpsPanel title="RouterOS 7.4+ · /container — быстрые команды" contentClassName="px-5 py-4">
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono"> <div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
{[ {[
@@ -442,13 +624,14 @@ export default function ContainersPage() {
</OpsPanel> </OpsPanel>
</div> </div>
</div> </ServerRailLayout>
<ExportSheet <ExportSheet
open={!!exportContainer} open={!!exportContainer}
container={exportContainer} container={exportContainer}
onClose={() => setExportContainer(null)} onClose={() => setExportContainer(null)}
serverById={serverById}
/> />
</div> </>
) )
} }
+48 -33
View File
@@ -67,6 +67,7 @@ import {
CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon, CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon,
} from "lucide-react" } from "lucide-react"
import { cn } from "@/lib/utils" import { cn } from "@/lib/utils"
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
import Link from "next/link" import Link from "next/link"
import { Flag } from "@/components/flag" import { Flag } from "@/components/flag"
@@ -820,9 +821,15 @@ function ServicePathList({
{paths.map((p) => { {paths.map((p) => {
const rowKey = servicePathKey(p) const rowKey = servicePathKey(p)
const via = servers.find((s) => s.id === p.viaId) const via = servers.find((s) => s.id === p.viaId)
const viaLabel = via?.site || p.viaName const en = servers.find((s) => s.id === p.enId)
const svc = services.find((s) => s.id === p.serviceId) const svc = services.find((s) => s.id === p.serviceId)
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId) const label = formatServicePathLabel(p, viaMode, {
viaName: via?.name,
viaSite: via?.site,
enName: en?.name,
serviceLabel: svc?.label,
})
const title = formatServicePathTitle(label, svc?.label ?? p.serviceId)
const active = Boolean( const active = Boolean(
highlight highlight
&& highlight.viaId === p.viaId && highlight.viaId === p.viaId
@@ -833,13 +840,14 @@ function ServicePathList({
<button <button
key={rowKey} key={rowKey}
type="button" type="button"
title={title}
onClick={() => onToggle(p)} onClick={() => onToggle(p)}
className={cn( className={cn(
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors", "flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50", active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
)} )}
> >
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span> <span className="font-mono truncate min-w-0">{label}</span>
<span className="font-mono text-emerald-400 tabular-nums shrink-0"> <span className="font-mono text-emerald-400 tabular-nums shrink-0">
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })} {formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
</span> </span>
@@ -1216,6 +1224,9 @@ export default function NetworkMapPage() {
// ── Interaction ───────────────────────────────────────────────────────────── // ── Interaction ─────────────────────────────────────────────────────────────
const [selected, setSelected] = useState<Server | null>(null) const [selected, setSelected] = useState<Server | null>(null)
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null) const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
const liveSelectedService = selectedService
? (mapServices.find((s) => s.id === selectedService.id) ?? selectedService)
: null
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null) const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
const [selWanIdx, setSelWanIdx] = useState<number | null>(null) const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
const [hoveredId, setHoveredId] = useState<string | null>(null) const [hoveredId, setHoveredId] = useState<string | null>(null)
@@ -2702,16 +2713,16 @@ export default function NetworkMapPage() {
})()} })()}
</div> </div>
</> </>
) : selectedService ? ( ) : liveSelectedService ? (
<> <>
<div className="flex items-start gap-2 px-4 py-3 border-b"> <div className="flex items-start gap-2 px-4 py-3 border-b">
<div className="mt-0.5"> <div className="mt-0.5">
<ServiceBrandIcon label={selectedService.label} size={22} /> <ServiceBrandIcon label={liveSelectedService.label} size={22} />
</div> </div>
<div className="flex-1 min-w-0"> <div className="flex-1 min-w-0">
<p className="font-mono font-semibold text-sm truncate">{selectedService.label}</p> <p className="font-mono font-semibold text-sm truncate">{liveSelectedService.label}</p>
<p className="text-xs text-muted-foreground mt-0.5"> <p className="text-xs text-muted-foreground mt-0.5">
Конечный сервис · {selectedService.category} Конечный сервис · {liveSelectedService.category}
</p> </p>
</div> </div>
<button <button
@@ -2726,15 +2737,15 @@ export default function NetworkMapPage() {
<div className="flex flex-col gap-0"> <div className="flex flex-col gap-0">
<div className="flex items-center justify-between py-2 border-b border-border/50"> <div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Доля окна</span> <span className="text-xs text-muted-foreground">Доля окна</span>
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedService.share)}</span> <span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(liveSelectedService.share)}</span>
</div> </div>
<div className="flex items-center justify-between py-2 border-b border-border/50"> <div className="flex items-center justify-between py-2 border-b border-border/50">
<span className="text-xs text-muted-foreground">Скорость</span> <span className="text-xs text-muted-foreground">Скорость</span>
<span className="text-xs font-mono font-medium"> <span className="text-xs font-mono font-medium">
{formatNetflowRate({ {formatNetflowRate({
bytes: selectedService.bytes, bytes: liveSelectedService.bytes,
bps: selectedService.bps, bps: liveSelectedService.bps,
bpsFwd: selectedService.bps, bpsFwd: liveSelectedService.bps,
bpsRev: 0, bpsRev: 0,
})} })}
</span> </span>
@@ -2742,34 +2753,34 @@ export default function NetworkMapPage() {
</div> </div>
<div> <div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p> <p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
<div className="flex flex-col gap-1.5"> <div className="flex flex-col gap-3">
{visibleServiceEdges.filter((e) => e.toId === selectedService.id).map((e) => { {visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
const src = mapServers.find((s) => s.id === e.fromId) const src = mapServers.find((s) => s.id === e.fromId)
const enPaths = mapServicePaths
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
.slice()
.sort((a, b) => b.bps - a.bps)
return ( return (
<div key={`${e.fromId}|${e.toId}`} className="flex items-center justify-between text-xs"> <div key={`${e.fromId}|${e.toId}`} className="flex flex-col gap-1.5">
<span className="font-mono truncate">{src?.name ?? e.fromId}</span> <div className="flex items-center justify-between text-xs">
<span className="font-mono text-emerald-400 tabular-nums"> <span className="font-mono truncate">{src?.name ?? e.fromId}</span>
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })} <span className="font-mono text-emerald-400 tabular-nums">
</span> {formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
</span>
</div>
<ServicePathList
paths={enPaths}
servers={mapServers}
services={mapServices}
highlight={highlightedPath}
viaMode="via"
onToggle={togglePathHighlight}
/>
</div> </div>
) )
})} })}
</div> </div>
</div> </div>
<div>
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
<ServicePathList
paths={mapServicePaths
.filter((p) => p.serviceId === selectedService.id)
.slice()
.sort((a, b) => b.bps - a.bps)}
servers={mapServers}
services={mapServices}
highlight={highlightedPath}
viaMode="via"
onToggle={togglePathHighlight}
/>
</div>
</div> </div>
</> </>
) : selected ? ( ) : selected ? (
@@ -3002,7 +3013,11 @@ export default function NetworkMapPage() {
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p> <p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
<ServicePathList <ServicePathList
paths={mapServicePaths paths={mapServicePaths
.filter((p) => p.viaId === selected.id || p.enId === selected.id) .filter((p) => (
selected.type === "exit-node"
? p.enId === selected.id
: p.viaId === selected.id
))
.slice() .slice()
.sort((a, b) => b.bps - a.bps)} .sort((a, b) => b.bps - a.bps)}
servers={mapServers} servers={mapServers}
+29 -3
View File
@@ -122,11 +122,17 @@ interface BackendBfdSession {
packetsRx: number; packetsTx: number; stateChanges: number packetsRx: number; packetsTx: number; stateChanges: number
} }
interface BackendOspfRoute {
id: string; serverId: number; serverName: string; serverSite: string
destination: string; type: OspfRoute["type"]; cost: number; nextHop: string; via: string; area: string
}
interface BackendOspfAll { interface BackendOspfAll {
neighbors: BackendNeighbor[] neighbors: BackendNeighbor[]
interfaces: BackendInterface[] interfaces: BackendInterface[]
instances: BackendInstance[] instances: BackendInstance[]
bfdSessions: BackendBfdSession[] bfdSessions: BackendBfdSession[]
routes?: BackendOspfRoute[]
} }
function isRefInterfaceName(name: string): boolean { function isRefInterfaceName(name: string): boolean {
@@ -213,6 +219,22 @@ function backendToBfdSession(b: BackendBfdSession): BfdSession {
} }
} }
function backendToRoute(b: BackendOspfRoute): OspfRoute {
const allowed: OspfRoute["type"][] = ["O", "O IA", "O E1", "O E2"]
const type = allowed.includes(b.type) ? b.type : "O"
return {
id: `${b.serverId}-${b.id}`,
destination: b.destination,
type,
cost: b.cost,
nextHop: b.nextHop,
via: b.via,
serverId: String(b.serverId),
serverLabel: b.serverName,
area: b.area || "—",
}
}
// ─── mock data ──────────────────────────────────────────────────────────────── // ─── mock data ────────────────────────────────────────────────────────────────
const COST_STEP = 10 const COST_STEP = 10
@@ -1172,7 +1194,7 @@ export default function OspfPage() {
}, [isLive, backendUrl, fetchTick]) }, [isLive, backendUrl, fetchTick])
// Derive frontend types from backend data or use mocks // Derive frontend types from backend data or use mocks
const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions } = useMemo(() => { const { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes } = useMemo(() => {
if (isLive && liveData) { if (isLive && liveData) {
// Build interface→cost map for neighbor cost lookup // Build interface→cost map for neighbor cost lookup
const ifaceMap = new Map<string, number>() const ifaceMap = new Map<string, number>()
@@ -1185,6 +1207,7 @@ export default function OspfPage() {
.filter((item) => !isRefInterfaceName(item.interfaceName)) .filter((item) => !isRefInterfaceName(item.interfaceName))
const neighbors = liveData.neighbors.map(b => backendToNeighbor(b, ifaceMap)) const neighbors = liveData.neighbors.map(b => backendToNeighbor(b, ifaceMap))
const bfdSessions = (liveData.bfdSessions ?? []).map(backendToBfdSession) const bfdSessions = (liveData.bfdSessions ?? []).map(backendToBfdSession)
const routes = (liveData.routes ?? []).map(backendToRoute)
// Build routerIds from instances // Build routerIds from instances
const routerIds: Record<string, string> = {} const routerIds: Record<string, string> = {}
@@ -1195,7 +1218,7 @@ export default function OspfPage() {
} }
const { nodes: graphNodes, edges: graphEdges } = buildLiveGraph(neighbors) const { nodes: graphNodes, edges: graphEdges } = buildLiveGraph(neighbors)
return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions } return { items, neighbors, graphNodes, graphEdges, routerIds, bfdSessions, routes }
} }
if (isLive) { if (isLive) {
return { return {
@@ -1205,6 +1228,7 @@ export default function OspfPage() {
graphEdges: [], graphEdges: [],
routerIds: {}, routerIds: {},
bfdSessions: [], bfdSessions: [],
routes: [],
} }
} }
return { return {
@@ -1214,6 +1238,7 @@ export default function OspfPage() {
graphEdges: MOCK_GRAPH_EDGES, graphEdges: MOCK_GRAPH_EDGES,
routerIds: MOCK_ROUTER_IDS, routerIds: MOCK_ROUTER_IDS,
bfdSessions: MOCK_BFD, bfdSessions: MOCK_BFD,
routes: MOCK_ROUTES,
} }
}, [isLive, liveData]) }, [isLive, liveData])
@@ -1257,6 +1282,7 @@ export default function OspfPage() {
const displayItems = filterServerId === ALL_SERVERS_ID ? items : items.filter(i => i.routerKey === filterServerId) const displayItems = filterServerId === ALL_SERVERS_ID ? items : items.filter(i => i.routerKey === filterServerId)
const displayNeighbors = filterServerId === ALL_SERVERS_ID ? neighbors : neighbors.filter(n => n.localRouter === filterServerId) const displayNeighbors = filterServerId === ALL_SERVERS_ID ? neighbors : neighbors.filter(n => n.localRouter === filterServerId)
const displayBfdSessions = filterServerId === ALL_SERVERS_ID ? bfdSessions : bfdSessions.filter(b => b.serverId === filterServerId) const displayBfdSessions = filterServerId === ALL_SERVERS_ID ? bfdSessions : bfdSessions.filter(b => b.serverId === filterServerId)
const displayRoutes = filterServerId === ALL_SERVERS_ID ? routes : routes.filter(r => r.serverId === filterServerId)
const ospfRailItems = useMemo<ServerTileItem[]>(() => ( const ospfRailItems = useMemo<ServerTileItem[]>(() => (
ospfServers.map((s) => { ospfServers.map((s) => {
@@ -1398,7 +1424,7 @@ export default function OspfPage() {
routerIds={routerIds} routerIds={routerIds}
/> />
)} )}
{activeTab === "routes" && <RoutesTab routes={isLive ? [] : MOCK_ROUTES} />} {activeTab === "routes" && <RoutesTab routes={displayRoutes} />}
{activeTab === "bfd" && <BfdTab sessions={displayBfdSessions} />} {activeTab === "bfd" && <BfdTab sessions={displayBfdSessions} />}
</div> </div>
+282 -151
View File
@@ -1,13 +1,11 @@
"use client" "use client"
import { useCallback, useEffect, useMemo, useState } from "react" import { Suspense, useCallback, useEffect, useMemo, useState } from "react"
import { useRouter, useSearchParams } from "next/navigation" import { useSearchParams } from "next/navigation"
import { import {
ActivityIcon, ActivityIcon,
CableIcon,
DatabaseIcon, DatabaseIcon,
GaugeIcon, GaugeIcon,
GlobeIcon,
ServerIcon, ServerIcon,
UsersIcon, UsersIcon,
} from "lucide-react" } from "lucide-react"
@@ -15,38 +13,45 @@ import { PageHeader } from "@/components/page-header"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid" import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { DataPageCard } from "@/components/data-page-card" import { DataPageCard } from "@/components/data-page-card"
import { DataPageToolbar } from "@/components/data-page-toolbar" import { DataPageToolbar } from "@/components/data-page-toolbar"
import { SegmentedControl } from "@/components/form-kit"
import { EmptyState } from "@/components/empty-state" import { EmptyState } from "@/components/empty-state"
import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector" import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector"
import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart" import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart"
import { DimensionSelect, PivotDimSelect } from "@/components/statistics/dimension-select"
import { SliceChips } from "@/components/statistics/slice-chips"
import { BreakdownDashboard } from "@/components/statistics/breakdown-dashboard"
import { StatisticsPivotGrid } from "@/components/statistics/statistics-pivot-grid"
import { import {
StatisticsBreakdownDataGrid, StatisticsBreakdownDataGrid,
type StatisticsSliceKind, type StatisticsSliceKind,
} from "@/components/data-grids/statistics-breakdown-data-grid" } from "@/components/data-grids/statistics-breakdown-data-grid"
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert" import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
import { Badge } from "@/components/reui/badge"
import type { Filter } from "@/components/reui/filters" import type { Filter } from "@/components/reui/filters"
import { Tabs, TabsContent, TabsList, TabsTrigger } from "@/components/ui/tabs"
import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields" import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields"
import {
isStatisticsPivotDim,
isStatisticsSliceKind,
STATISTICS_DIMS,
} from "@/lib/statistics-dims"
import { useDataSource } from "@/lib/data-source" import { useDataSource } from "@/lib/data-source"
import { fmtBps, formatBytes } from "@/lib/fmt-rate" import { fmtBps, formatBytes } from "@/lib/fmt-rate"
import { getStatistics, type StatisticsDto, type StatisticsQuery } from "@/shared/api/statistics" import {
getStatistics,
getStatisticsPivot,
STATISTICS_UNBOUND_USER_ID,
type StatisticsDto,
type StatisticsPivotDto,
type StatisticsQuery,
} from "@/shared/api/statistics"
import type { StatisticsBreakdownRow, StatisticsPivotDim } from "@mmapp/contracts/statistics"
/** /**
* Отчётный куб трафика — KPI + период + график + табы-гриды. * BI-куб трафика: критерий → остальные разрезы + pivot.
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12 * Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
* · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/chart-23 * · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/solution-analytics-8
* · https://reui.io/preview/base/components/c-date-selector-2 · https://reui.io/preview/base/empty-state-12 * · https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/data-grid
*/ */
const TABS: { id: StatisticsSliceKind; label: string }[] = [
{ id: "users", label: "Пользователи" },
{ id: "servers", label: "Серверы" },
{ id: "interfaces", label: "Интерфейсы" },
{ id: "countries", label: "Страны" },
{ id: "services", label: "Сервисы" },
{ id: "asns", label: "ASN" },
]
const EMPTY: StatisticsDto = { const EMPTY: StatisticsDto = {
from: "", from: "",
to: "", to: "",
@@ -70,6 +75,15 @@ const EMPTY: StatisticsDto = {
asns: [], asns: [],
} }
const EMPTY_PIVOT: StatisticsPivotDto = {
rowDim: "country",
colDim: "service",
metric: "bytes",
columns: [],
rows: [],
otherBytes: 0,
}
interface CubeSlices { interface CubeSlices {
country?: string country?: string
service?: string service?: string
@@ -88,9 +102,22 @@ function readRange(sp: URLSearchParams): DateRangeYmd {
return rangeForPreset("7d") return rangeForPreset("7d")
} }
function readTab(sp: URLSearchParams): StatisticsSliceKind { function readDim(sp: URLSearchParams): StatisticsSliceKind {
const t = sp.get("tab") const t = sp.get("dim") ?? sp.get("tab")
return TABS.some((x) => x.id === t) ? (t as StatisticsSliceKind) : "users" return t && isStatisticsSliceKind(t) ? t : "users"
}
function readView(sp: URLSearchParams): "explore" | "pivot" {
return sp.get("view") === "pivot" ? "pivot" : "explore"
}
function readPlanes(sp: URLSearchParams): "unique" | "all" {
return sp.get("planes") === "all" ? "all" : "unique"
}
function readPivotDim(sp: URLSearchParams, key: string, fallback: StatisticsPivotDim): StatisticsPivotDim {
const v = sp.get(key)
return v && isStatisticsPivotDim(v) ? v : fallback
} }
function readSlices(sp: URLSearchParams): CubeSlices { function readSlices(sp: URLSearchParams): CubeSlices {
@@ -125,7 +152,7 @@ function filtersToSlices(filters: Filter[]): CubeSlices {
return next return next
} }
function toQuery(range: DateRangeYmd, slices: CubeSlices): StatisticsQuery { function toQuery(range: DateRangeYmd, slices: CubeSlices, planes: "unique" | "all"): StatisticsQuery {
const serverId = slices.serverId ? Number(slices.serverId) : undefined const serverId = slices.serverId ? Number(slices.serverId) : undefined
const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined
return { return {
@@ -137,33 +164,97 @@ function toQuery(range: DateRangeYmd, slices: CubeSlices): StatisticsQuery {
country: slices.country && slices.country.length === 2 ? slices.country : undefined, country: slices.country && slices.country.length === 2 ? slices.country : undefined,
service: slices.service, service: slices.service,
asn: Number.isFinite(asn) ? asn : undefined, asn: Number.isFinite(asn) ? asn : undefined,
planes,
} }
} }
function selectedIdForTab(tab: StatisticsSliceKind, slices: CubeSlices): string | undefined { function selectedIdForKind(kind: StatisticsSliceKind, slices: CubeSlices): string | undefined {
if (tab === "users") return slices.userId if (kind === "users") return slices.userId
if (tab === "servers") return slices.serverId if (kind === "servers") return slices.serverId
if (tab === "countries") return slices.country if (kind === "countries") return slices.country
if (tab === "services") return slices.service if (kind === "services") return slices.service
if (tab === "asns") return slices.asn if (kind === "asns") return slices.asn
if (tab === "interfaces" && slices.serverId && slices.iface) { if (kind === "interfaces" && slices.serverId && slices.iface) {
return `${slices.serverId}:${slices.iface}` return `${slices.serverId}:${slices.iface}`
} }
if (tab === "interfaces") return slices.iface if (kind === "interfaces") return slices.iface
return undefined return undefined
} }
function rowsForTab(data: StatisticsDto, tab: StatisticsSliceKind) { function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind) {
if (tab === "users") return data.users if (kind === "users") return data.users
if (tab === "servers") return data.servers if (kind === "servers") return data.servers
if (tab === "interfaces") return data.interfaces if (kind === "interfaces") return data.interfaces
if (tab === "countries") return data.countries if (kind === "countries") return data.countries
if (tab === "services") return data.services if (kind === "services") return data.services
return data.asns return data.asns
} }
export default function StatisticsPage() { function hasAnySlice(slices: CubeSlices): boolean {
const router = useRouter() return SLICE_KEYS.some((k) => Boolean(slices[k]))
}
function hiddenKinds(slices: CubeSlices): Set<StatisticsSliceKind> {
const hidden = new Set<StatisticsSliceKind>()
if (slices.userId) hidden.add("users")
if (slices.serverId) hidden.add("servers")
if (slices.iface) hidden.add("interfaces")
if (slices.country) hidden.add("countries")
if (slices.service) hidden.add("services")
if (slices.asn) hidden.add("asns")
return hidden
}
function applyDimValue(slices: CubeSlices, kind: StatisticsSliceKind, rowId: string): CubeSlices {
const next: CubeSlices = { ...slices }
if (kind === "users") {
if (rowId === STATISTICS_UNBOUND_USER_ID) return next
if (next.userId === rowId) delete next.userId
else next.userId = rowId
} else if (kind === "servers") {
if (next.serverId === rowId) delete next.serverId
else next.serverId = rowId
} else if (kind === "countries") {
if (next.country === rowId) delete next.country
else next.country = rowId
} else if (kind === "services") {
if (next.service === rowId) delete next.service
else next.service = rowId
} else if (kind === "asns") {
if (next.asn === rowId) delete next.asn
else next.asn = rowId
} else {
const colon = rowId.indexOf(":")
const sid = colon >= 0 ? rowId.slice(0, colon) : undefined
const iface = colon >= 0 ? rowId.slice(colon + 1) : rowId
if (next.iface === iface && next.serverId === sid) {
delete next.iface
delete next.serverId
} else {
next.iface = iface
if (sid) next.serverId = sid
}
}
return next
}
function applyPivotDim(slices: CubeSlices, dim: StatisticsPivotDim, id: string): CubeSlices {
const kind = STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
return applyDimValue(slices, kind, id)
}
function chipList(slices: CubeSlices): { key: string; label: string }[] {
const chips: { key: string; label: string }[] = []
if (slices.country) chips.push({ key: "country", label: `страна ${slices.country}` })
if (slices.service) chips.push({ key: "service", label: `сервис ${slices.service}` })
if (slices.asn) chips.push({ key: "asn", label: `ASN ${slices.asn}` })
if (slices.serverId) chips.push({ key: "serverId", label: `сервер ${slices.serverId}` })
if (slices.userId) chips.push({ key: "userId", label: `пользователь ${slices.userId}` })
if (slices.iface) chips.push({ key: "iface", label: `iface ${slices.iface}` })
return chips
}
function StatisticsPageInner() {
const searchParams = useSearchParams() const searchParams = useSearchParams()
const { mode, backendUrl, prefsHydrated } = useDataSource() const { mode, backendUrl, prefsHydrated } = useDataSource()
const isLive = mode === "live" const isLive = mode === "live"
@@ -171,9 +262,14 @@ export default function StatisticsPage() {
const range = useMemo(() => readRange(searchParams), [searchParams]) const range = useMemo(() => readRange(searchParams), [searchParams])
const slices = useMemo(() => readSlices(searchParams), [searchParams]) const slices = useMemo(() => readSlices(searchParams), [searchParams])
const filters = useMemo(() => slicesToFilters(slices), [slices]) const filters = useMemo(() => slicesToFilters(slices), [slices])
const [tab, setTab] = useState<StatisticsSliceKind>(() => readTab(searchParams)) const dim = useMemo(() => readDim(searchParams), [searchParams])
const view = useMemo(() => readView(searchParams), [searchParams])
const planes = useMemo(() => readPlanes(searchParams), [searchParams])
const pivotRow = useMemo(() => readPivotDim(searchParams, "pivotRow", "country"), [searchParams])
const pivotCol = useMemo(() => readPivotDim(searchParams, "pivotCol", "service"), [searchParams])
const [data, setData] = useState<StatisticsDto>(EMPTY) const [data, setData] = useState<StatisticsDto>(EMPTY)
const [pivot, setPivot] = useState<StatisticsPivotDto>(EMPTY_PIVOT)
const [loading, setLoading] = useState(false) const [loading, setLoading] = useState(false)
const [error, setError] = useState<string | null>(null) const [error, setError] = useState<string | null>(null)
@@ -185,9 +281,10 @@ export default function StatisticsPage() {
else sp.delete(k) else sp.delete(k)
} }
const qs = sp.toString() const qs = sp.toString()
router.replace(qs ? `/statistics?${qs}` : "/statistics") if (qs === searchParams.toString()) return
window.history.replaceState(null, "", qs ? `/statistics?${qs}` : "/statistics")
}, },
[router, searchParams], [searchParams],
) )
const setRange = useCallback( const setRange = useCallback(
@@ -218,11 +315,22 @@ export default function StatisticsPage() {
setLoading(true) setLoading(true)
setError(null) setError(null)
try { try {
const dto = await getStatistics(backendUrl, toQuery(range, slices)) const query = toQuery(range, slices, planes)
const dto = await getStatistics(backendUrl, query)
if (!cancelled) setData(dto) if (!cancelled) setData(dto)
if (view === "pivot" && pivotRow !== pivotCol) {
const matrix = await getStatisticsPivot(backendUrl, {
...query,
row: pivotRow,
col: pivotCol,
metric: "bytes",
})
if (!cancelled) setPivot(matrix)
}
} catch (e: unknown) { } catch (e: unknown) {
if (!cancelled) { if (!cancelled) {
setData(EMPTY) setData(EMPTY)
setPivot(EMPTY_PIVOT)
setError(e instanceof Error ? e.message : "Не удалось загрузить статистику") setError(e instanceof Error ? e.message : "Не удалось загрузить статистику")
} }
} finally { } finally {
@@ -232,46 +340,41 @@ export default function StatisticsPage() {
return () => { return () => {
cancelled = true cancelled = true
} }
}, [backendUrl, isLive, prefsHydrated, range, slices]) }, [backendUrl, isLive, prefsHydrated, range, slices, view, pivotRow, pivotCol, planes])
const rows = rowsForTab(isLive ? data : EMPTY, tab) const viewData = isLive ? data : EMPTY
const selectedId = selectedIdForTab(tab, slices) const sliced = hasAnySlice(slices)
const view = isLive ? data : EMPTY const emptyCube = !isLive || (!loading && viewData.kpis.bytes === 0 && viewData.interfaces.length === 0)
function handleRowClick(kind: StatisticsSliceKind, row: { id: string }) { function handleRowClick(kind: StatisticsSliceKind, row: StatisticsBreakdownRow) {
const next: CubeSlices = { ...slices } if (kind === "users" && row.id === STATISTICS_UNBOUND_USER_ID) return
if (kind === "users") { if (kind === "interfaces" && row.label.includes("· дубль")) return
if (next.userId === row.id) delete next.userId setSlices(applyDimValue(slices, kind, row.id))
else next.userId = row.id
} else if (kind === "servers") {
if (next.serverId === row.id) delete next.serverId
else next.serverId = row.id
} else if (kind === "countries") {
if (next.country === row.id) delete next.country
else next.country = row.id
} else if (kind === "services") {
if (next.service === row.id) delete next.service
else next.service = row.id
} else if (kind === "asns") {
if (next.asn === row.id) delete next.asn
else next.asn = row.id
} else {
const colon = row.id.indexOf(":")
const sid = colon >= 0 ? row.id.slice(0, colon) : undefined
const iface = colon >= 0 ? row.id.slice(colon + 1) : row.id
if (next.iface === iface && next.serverId === sid) {
delete next.iface
delete next.serverId
} else {
next.iface = iface
if (sid) next.serverId = sid
}
}
setSlices(next)
} }
const kpis = view.kpis function handlePivotCell(rowId: string, colId: string) {
const emptyCube = !isLive || (!loading && kpis.bytes === 0) if (rowId === "__other__" || colId === "__other__") return
let next = applyPivotDim(slices, pivotRow, rowId)
next = applyPivotDim(next, pivotCol, colId)
replaceParams({
country: next.country,
service: next.service,
asn: next.asn,
serverId: next.serverId,
userId: next.userId,
iface: next.iface,
view: "explore",
})
}
const kpis = viewData.kpis
const chips = chipList(slices)
const countLabel =
view === "pivot"
? `${pivot.rows.length} × ${pivot.columns.length}`
: sliced
? `${STATISTICS_DIMS.filter((d) => !hiddenKinds(slices).has(d.id)).length} разрезов`
: `${rowsForKind(viewData, dim).length} строк`
return ( return (
<div className="flex h-full flex-col"> <div className="flex h-full flex-col">
@@ -297,6 +400,15 @@ export default function StatisticsPage() {
</Alert> </Alert>
) : null} ) : null}
{!slices.serverId && isLive && !emptyCube ? (
<Alert>
<AlertTitle>Уникальный объём</AlertTitle>
<AlertDescription>
Объём — трафик клиентов на GRE/WG, без повторного учёта JH↔EN и WAN.
</AlertDescription>
</Alert>
) : null}
<KpiStatGrid <KpiStatGrid
aria-label="Сводка трафика" aria-label="Сводка трафика"
isLoading={loading} isLoading={loading}
@@ -306,7 +418,7 @@ export default function StatisticsPage() {
id: "bytes", id: "bytes",
label: "Объём", label: "Объём",
value: formatBytes(kpis.bytes), value: formatBytes(kpis.bytes),
hint: kpis.topCountry ? `топ: ${kpis.topCountry}` : undefined, hint: "GRE/WG клиентов, без hops",
icon: <DatabaseIcon />, icon: <DatabaseIcon />,
iconClassName: "text-muted-foreground", iconClassName: "text-muted-foreground",
}, },
@@ -336,92 +448,111 @@ export default function StatisticsPage() {
id: "servers", id: "servers",
label: "Серверы", label: "Серверы",
value: String(kpis.servers), value: String(kpis.servers),
hint: kpis.ifaces ? `${kpis.ifaces} iface` : undefined, hint: slices.serverId
? (kpis.ifaces ? `${kpis.ifaces} iface` : undefined)
: planes === "all"
? "WAN и дубли в списке"
: "без WAN и overlay",
icon: <ServerIcon />, icon: <ServerIcon />,
iconClassName: "text-muted-foreground", iconClassName: "text-muted-foreground",
}, },
]} ]}
/> />
<StatisticsVolumeChart series={view.series} grain={view.grain} /> <StatisticsVolumeChart series={viewData.series} grain={viewData.grain} />
<DataPageCard> <DataPageCard>
<DataPageToolbar <DataPageToolbar
leading={
<div className="flex flex-wrap items-center gap-3">
<SegmentedControl
value={view}
onChange={(next) => replaceParams({ view: next === "pivot" ? "pivot" : "explore" })}
options={[
{ value: "explore", label: "Разрез" },
{ value: "pivot", label: "Сводка" },
]}
/>
<SegmentedControl
value={planes}
onChange={(next) => replaceParams({ planes: next === "all" ? "all" : undefined })}
options={[
{ value: "unique", label: "Уникальный" },
{ value: "all", label: "Все плоскости" },
]}
/>
{view === "explore" && !sliced ? (
<DimensionSelect
label="Критерий"
value={dim}
onChange={(next) => replaceParams({ dim: next })}
/>
) : null}
{view === "pivot" ? (
<>
<PivotDimSelect
label="Строки"
value={pivotRow}
exclude={pivotCol}
onChange={(next) => replaceParams({ pivotRow: next })}
/>
<PivotDimSelect
label="Колонки"
value={pivotCol}
exclude={pivotRow}
onChange={(next) => replaceParams({ pivotCol: next })}
/>
</>
) : null}
</div>
}
filters={filters} filters={filters}
onFiltersChange={(next) => setSlices(filtersToSlices(next))} onFiltersChange={(next) => setSlices(filtersToSlices(next))}
filterFields={STATISTICS_FILTER_FIELDS} filterFields={STATISTICS_FILTER_FIELDS}
countLabel={`${rows.length} строк`} countLabel={countLabel}
/> />
{SLICE_KEYS.some((k) => slices[k]) ? ( <SliceChips
<div className="flex flex-wrap items-center gap-1.5 border-b px-5 py-2"> chips={chips}
{slices.country ? ( onRemove={(key) => {
<Badge variant="outline" size="sm">страна {slices.country}</Badge> const next = { ...slices }
) : null} delete next[key as keyof CubeSlices]
{slices.service ? ( setSlices(next)
<Badge variant="outline" size="sm">сервис {slices.service}</Badge> }}
) : null} />
{slices.asn ? ( {emptyCube ? (
<Badge variant="outline" size="sm">ASN {slices.asn}</Badge> <EmptyState
) : null} title="Нет данных куба"
{slices.serverId ? ( description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
<Badge variant="outline" size="sm">сервер {slices.serverId}</Badge> />
) : null} ) : view === "pivot" ? (
{slices.userId ? ( <StatisticsPivotGrid data={isLive ? pivot : EMPTY_PIVOT} onCellClick={handlePivotCell} isLoading={loading} />
<Badge variant="outline" size="sm">пользователь {slices.userId}</Badge> ) : sliced ? (
) : null} <BreakdownDashboard
{slices.iface ? ( data={viewData}
<Badge variant="outline" size="sm">iface {slices.iface}</Badge> hidden={hiddenKinds(slices)}
) : null} selectedIdFor={(kind) => selectedIdForKind(kind, slices)}
</div> onRowClick={handleRowClick}
) : null} isLoading={loading}
<Tabs />
value={tab} ) : (
onValueChange={(v) => { <StatisticsBreakdownDataGrid
const next = String(v) as StatisticsSliceKind rows={rowsForKind(viewData, dim)}
setTab(next) kind={dim}
replaceParams({ tab: next }) selectedId={selectedIdForKind(dim, slices)}
}} onRowClick={(row) => handleRowClick(dim, row)}
className="gap-0" isLoading={loading}
> />
<div className="px-5 pt-2"> )}
<TabsList variant="line" className="w-fit">
<TabsTrigger value="users">
<UsersIcon /> Пользователи
</TabsTrigger>
<TabsTrigger value="servers">
<ServerIcon /> Серверы
</TabsTrigger>
<TabsTrigger value="interfaces">
<CableIcon /> Интерфейсы
</TabsTrigger>
<TabsTrigger value="countries">
<GlobeIcon /> Страны
</TabsTrigger>
<TabsTrigger value="services">Сервисы</TabsTrigger>
<TabsTrigger value="asns">ASN</TabsTrigger>
</TabsList>
</div>
{TABS.map((t) => (
<TabsContent key={t.id} value={t.id}>
{emptyCube ? (
<EmptyState
title="Нет данных куба"
description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
/>
) : (
<StatisticsBreakdownDataGrid
rows={rowsForTab(view, t.id)}
kind={t.id}
selectedId={t.id === tab ? selectedId : undefined}
onRowClick={(row) => handleRowClick(t.id, row)}
isLoading={loading}
/>
)}
</TabsContent>
))}
</Tabs>
</DataPageCard> </DataPageCard>
</div> </div>
</div> </div>
) )
} }
export default function StatisticsPage() {
return (
<Suspense fallback={null}>
<StatisticsPageInner />
</Suspense>
)
}
+190 -45
View File
@@ -1,30 +1,63 @@
"use client" "use client"
import { useMemo, useState } from "react" import { useCallback, useEffect, useMemo, useState } from "react"
import { PageHeader } from "@/components/page-header" import { PageHeader } from "@/components/page-header"
import { vxlanTunnels, servers } from "@/lib/data" import { vxlanTunnels as mockVxlanTunnels, servers as mockServers } from "@/lib/data"
import type { VxlanTunnel } from "@/lib/data" import type { Server, VxlanTunnel } from "@/lib/data"
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid" import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
import { OpsPanel } from "@/components/ops-panel" import { OpsPanel } from "@/components/ops-panel"
import { DataPageCard } from "@/components/data-page-card" import { DataPageCard } from "@/components/data-page-card"
import { DataPageToolbar } from "@/components/data-page-toolbar" import { DataPageToolbar } from "@/components/data-page-toolbar"
import { VxlanDataGrid } from "@/components/data-grids/vxlan-data-grid" import { VxlanDataGrid } from "@/components/data-grids/vxlan-data-grid"
import { Button } from "@/components/ui/button" import { Button } from "@/components/ui/button"
import { Alert, AlertDescription } from "@/components/ui/alert"
import { import {
NetworkIcon, PlusIcon, CodeXmlIcon, LayersIcon, NetworkIcon, PlusIcon, LayersIcon, RefreshCwIcon, AlertCircleIcon,
} from "lucide-react" } from "lucide-react"
import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet" import { CodeExportSheet } from "@/components/reui-kit/code-export-sheet"
import { useDataSource } from "@/lib/data-source"
import { requestJson } from "@/shared/api/http-client"
import { cn } from "@/lib/utils"
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
// ─── helpers ────────────────────────────────────────────────────────────────── interface BackendServer {
id: number
function serverFor(id: string) { name: string
return servers.find((s) => s.id === id) host: string
type?: Server["type"]
site?: string
country: string
asn?: string
enabled: boolean
status?: Server["status"]
latency?: number | null
} }
// ─── RSC generator ─────────────────────────────────────────────────────────── interface VxlanApiResponse {
tunnels: VxlanTunnel[]
}
function generateVxlanRsc(t: VxlanTunnel): string { function mapBackendServer(s: BackendServer): Server {
const srv = serverFor(t.serverId) return {
id: String(s.id),
name: s.name || s.host,
host: s.host,
model: "—",
os: "—",
site: s.site ?? "",
country: s.country || "UN",
asn: s.asn ?? "",
type: s.type ?? "exit-node",
enabled: s.enabled,
status: s.status ?? "online",
latency: s.latency ?? null,
sessions: 0,
}
}
function generateVxlanRsc(t: VxlanTunnel, serverById: Record<string, Server>): string {
const srv = serverById[t.serverId]
const lines: string[] = [] const lines: string[] = []
lines.push(`# VXLAN — ${t.name} · VNI ${t.vni}`) lines.push(`# VXLAN — ${t.name} · VNI ${t.vni}`)
if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`) if (srv) lines.push(`# Сервер: ${srv.name} (${srv.host})`)
@@ -42,7 +75,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
if (!t.enabled) lines.push(` disabled=yes \\`) if (!t.enabled) lines.push(` disabled=yes \\`)
lines.push(``) lines.push(``)
// FDB entries for remote VTEPs
for (const vtep of t.remoteVteps) { for (const vtep of t.remoteVteps) {
lines.push(`/interface/vxlan/vteps/add \\`) lines.push(`/interface/vxlan/vteps/add \\`)
lines.push(` interface=${t.name} \\`) lines.push(` interface=${t.name} \\`)
@@ -50,7 +82,6 @@ function generateVxlanRsc(t: VxlanTunnel): string {
lines.push(``) lines.push(``)
} }
// Bridge
lines.push(`# Добавить в bridge:`) lines.push(`# Добавить в bridge:`)
lines.push(`/interface/bridge/port/add \\`) lines.push(`/interface/bridge/port/add \\`)
lines.push(` bridge=bridge-overlay \\`) lines.push(` bridge=bridge-overlay \\`)
@@ -59,12 +90,18 @@ function generateVxlanRsc(t: VxlanTunnel): string {
return lines.join("\n") return lines.join("\n")
} }
// ─── Export Sheet ───────────────────────────────────────────────────────────── function ExportSheet({
open, tunnel, onClose, serverById,
function ExportSheet({ open, tunnel, onClose }: { }: {
open: boolean; tunnel: VxlanTunnel | null; onClose: () => void open: boolean
tunnel: VxlanTunnel | null
onClose: () => void
serverById: Record<string, Server>
}) { }) {
const code = useMemo(() => tunnel ? generateVxlanRsc(tunnel) : "", [tunnel]) const code = useMemo(
() => (tunnel ? generateVxlanRsc(tunnel, serverById) : ""),
[tunnel, serverById],
)
return ( return (
<CodeExportSheet <CodeExportSheet
@@ -84,46 +121,156 @@ function ExportSheet({ open, tunnel, onClose }: {
) )
} }
// ─── Export Sheet ─────────────────────────────────────────────────────────────
export default function VxlanPage() { export default function VxlanPage() {
const [search, setSearch] = useState("") const { mode, backendUrl } = useDataSource()
const isLive = mode === "live"
const [search, setSearch] = useState("")
const [exportTunnel, setExportTunnel] = useState<VxlanTunnel | null>(null) const [exportTunnel, setExportTunnel] = useState<VxlanTunnel | null>(null)
const [selectedServerId, setSelectedServerId] = useState(ALL_SERVERS_ID)
const [liveTunnels, setLiveTunnels] = useState<VxlanTunnel[]>([])
const [liveServers, setLiveServers] = useState<Server[]>([])
const [loading, setLoading] = useState(false)
const [liveError, setLiveError] = useState<string | null>(null)
const loadLive = useCallback(async () => {
if (!isLive) return
setLoading(true)
setLiveError(null)
try {
const [tunnelsRes, serversRes] = await Promise.all([
requestJson<VxlanApiResponse>(backendUrl, "/api/vxlan"),
requestJson<BackendServer[]>(backendUrl, "/api/servers"),
])
setLiveTunnels(tunnelsRes.tunnels ?? [])
setLiveServers(serversRes.filter((s) => s.enabled).map(mapBackendServer))
} catch (e) {
setLiveError(e instanceof Error ? e.message : "Ошибка загрузки")
setLiveTunnels([])
} finally {
setLoading(false)
}
}, [isLive, backendUrl])
useEffect(() => {
if (!isLive) {
queueMicrotask(() => {
setLiveTunnels([])
setLiveServers([])
setLiveError(null)
})
return
}
queueMicrotask(() => {
void loadLive()
})
}, [isLive, loadLive])
const displayTunnels = isLive ? liveTunnels : mockVxlanTunnels
const displayServers = isLive ? liveServers : mockServers.filter((s) => s.enabled)
const effectiveServerId =
selectedServerId === ALL_SERVERS_ID || displayServers.some((s) => s.id === selectedServerId)
? selectedServerId
: ALL_SERVERS_ID
const scopedTunnels = useMemo(() => {
if (effectiveServerId === ALL_SERVERS_ID) return displayTunnels
return displayTunnels.filter((t) => t.serverId === effectiveServerId)
}, [displayTunnels, effectiveServerId])
const serverById = useMemo(
() => Object.fromEntries(displayServers.map((s) => [s.id, s])),
[displayServers],
)
const railItems = useMemo<ServerTileItem[]>(() => (
displayServers.map((s) => ({
id: s.id,
name: s.name,
host: s.host,
site: s.site,
country: s.country,
status: s.status,
type: s.type,
enabled: s.enabled,
meta: String(displayTunnels.filter((t) => t.serverId === s.id).length),
}))
), [displayServers, displayTunnels])
const filtered = useMemo(() => { const filtered = useMemo(() => {
if (!search) return vxlanTunnels if (!search) return scopedTunnels
const q = search.toLowerCase() const q = search.toLowerCase()
return vxlanTunnels.filter((t) => return scopedTunnels.filter((t) =>
t.name.includes(q) || t.name.includes(q) ||
String(t.vni).includes(q) || String(t.vni).includes(q) ||
t.vtepIp.includes(q) || t.vtepIp.includes(q) ||
(serverFor(t.serverId)?.name.toLowerCase().includes(q) ?? false) (serverById[t.serverId]?.name.toLowerCase().includes(q) ?? false),
) )
}, [search]) }, [search, scopedTunnels, serverById])
const upCount = vxlanTunnels.filter((t) => t.status === "up").length const upCount = scopedTunnels.filter((t) => t.status === "up").length
const vnis = new Set(vxlanTunnels.map((t) => t.vni)).size const vnis = new Set(scopedTunnels.map((t) => t.vni)).size
return ( return (
<div className="flex flex-col h-full"> <>
<PageHeader <ServerRailLayout
crumbs={[{ label: "Управление" }, { label: "VXLAN" }]} items={railItems}
actions={ selectedId={effectiveServerId}
<Button size="sm"> onSelect={setSelectedServerId}
<PlusIcon className="size-4" />Новый VXLAN showAll
</Button> allCount={displayServers.length}
} loading={isLive && loading && displayServers.length === 0}
/> header={
<PageHeader
<div className="flex-1 overflow-y-auto p-6"> crumbs={[{ label: "Управление" }, { label: "VXLAN" }]}
actions={
<>
<ServerRailMobileButton />
<Button
variant="outline"
size="sm"
onClick={() => { void loadLive() }}
disabled={!isLive || loading}
>
<RefreshCwIcon className={cn("size-4", loading && "animate-spin")} />
Обновить
</Button>
<Button size="sm">
<PlusIcon className="size-4" />Новый VXLAN
</Button>
</>
}
/>
}
>
<div className="flex flex-col gap-5"> <div className="flex flex-col gap-5">
{isLive && liveError && (
<Alert variant="warning" className="py-2">
<AlertCircleIcon />
<AlertDescription className="text-xs">Ошибка загрузки: {liveError}</AlertDescription>
</Alert>
)}
{isLive && !loading && displayTunnels.length === 0 && !liveError && (
<div className="rounded-md border border-border bg-muted/30 px-4 py-6 text-center text-sm text-muted-foreground">
На опрошенных серверах нет VXLAN-интерфейсов
</div>
)}
{mode === "mock" && (
<span className="inline-flex w-fit items-center gap-1.5 rounded-full border border-border bg-muted/40 px-2.5 py-0.5 text-[11px] font-medium text-muted-foreground">
Моковые данные
</span>
)}
<KpiStatGrid <KpiStatGrid
aria-label="Сводка VXLAN" aria-label="Сводка VXLAN"
items={[ items={[
{ {
id: "tunnels", id: "tunnels",
label: "Туннелей", label: "Туннелей",
value: vxlanTunnels.length, value: scopedTunnels.length,
icon: <NetworkIcon className="size-4" />, icon: <NetworkIcon className="size-4" />,
iconClassName: "text-muted-foreground", iconClassName: "text-muted-foreground",
}, },
@@ -144,14 +291,13 @@ export default function VxlanPage() {
{ {
id: "servers", id: "servers",
label: "Серверов", label: "Серверов",
value: new Set(vxlanTunnels.map((t) => t.serverId)).size, value: new Set(scopedTunnels.map((t) => t.serverId)).size,
icon: <NetworkIcon className="size-4" />, icon: <NetworkIcon className="size-4" />,
iconClassName: "text-primary", iconClassName: "text-primary",
}, },
]} ]}
/> />
{/* Info banner */}
<div className="flex items-start gap-3 rounded-lg bg-sky-500/5 border border-sky-500/20 px-4 py-3 text-sm"> <div className="flex items-start gap-3 rounded-lg bg-sky-500/5 border border-sky-500/20 px-4 py-3 text-sm">
<NetworkIcon className="size-5 text-sky-500 shrink-0 mt-0.5" /> <NetworkIcon className="size-5 text-sky-500 shrink-0 mt-0.5" />
<div> <div>
@@ -163,7 +309,6 @@ export default function VxlanPage() {
</div> </div>
</div> </div>
{/* Table */}
<DataPageCard> <DataPageCard>
<DataPageToolbar <DataPageToolbar
search={search} search={search}
@@ -173,12 +318,11 @@ export default function VxlanPage() {
/> />
<VxlanDataGrid <VxlanDataGrid
tunnels={filtered} tunnels={filtered}
servers={servers} servers={displayServers}
onExport={setExportTunnel} onExport={setExportTunnel}
/> />
</DataPageCard> </DataPageCard>
{/* Reference */}
<OpsPanel title="RouterOS 7 · /interface/vxlan — быстрые команды" contentClassName="px-5 py-4"> <OpsPanel title="RouterOS 7 · /interface/vxlan — быстрые команды" contentClassName="px-5 py-4">
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono"> <div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
{[ {[
@@ -232,13 +376,14 @@ export default function VxlanPage() {
</OpsPanel> </OpsPanel>
</div> </div>
</div> </ServerRailLayout>
<ExportSheet <ExportSheet
open={!!exportTunnel} open={!!exportTunnel}
tunnel={exportTunnel} tunnel={exportTunnel}
onClose={() => setExportTunnel(null)} onClose={() => setExportTunnel(null)}
serverById={serverById}
/> />
</div> </>
) )
} }
+4 -2
View File
@@ -12,14 +12,16 @@
"db:migrate": "drizzle-kit migrate", "db:migrate": "drizzle-kit migrate",
"db:studio": "drizzle-kit studio", "db:studio": "drizzle-kit studio",
"db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts", "db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts",
"facts:rebuild": "tsx src/scripts/rebuild-flow-facts.ts",
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts", "test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts", "test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts", "test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/statistics-aggregate.test.ts", "test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-dest.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/traffic-flow-facts-rebuild.test.ts && tsx src/services/statistics-aggregate.test.ts",
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts", "test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts", "test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts",
"test:backups": "tsx src/services/s3-backup-client.test.ts", "test:backups": "tsx src/services/s3-backup-client.test.ts",
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups", "test:live-maps": "tsx src/services/ospf-route-parse.test.ts && tsx src/services/vxlan-live.test.ts && tsx src/services/containers-live.test.ts",
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups && npm run test:live-maps",
"test:geoip": "tsx src/services/traffic-flow-geoip.test.ts" "test:geoip": "tsx src/services/traffic-flow-geoip.test.ts"
}, },
"dependencies": { "dependencies": {
+4
View File
@@ -29,6 +29,8 @@ import certificatesRoutes from "./routes/certificates.js"
import systemDatabaseRoutes from "./routes/system-database.js" import systemDatabaseRoutes from "./routes/system-database.js"
import eventsRoutes from "./routes/events.js" import eventsRoutes from "./routes/events.js"
import wireguardRoutes from "./routes/wireguard.js" import wireguardRoutes from "./routes/wireguard.js"
import vxlanRoutes from "./routes/vxlan.js"
import containersRoutes from "./routes/containers.js"
import firewallRoutes from "./routes/firewall.js" import firewallRoutes from "./routes/firewall.js"
import usersRoutes from "./routes/users.js" import usersRoutes from "./routes/users.js"
import statisticsRoutes from "./routes/statistics.js" import statisticsRoutes from "./routes/statistics.js"
@@ -134,6 +136,8 @@ export async function buildApp(opts?: {
await app.register(systemDatabaseRoutes, { prefix: "/api" }) await app.register(systemDatabaseRoutes, { prefix: "/api" })
await app.register(eventsRoutes, { prefix: "/api" }) await app.register(eventsRoutes, { prefix: "/api" })
await app.register(wireguardRoutes, { prefix: "/api" }) await app.register(wireguardRoutes, { prefix: "/api" })
await app.register(vxlanRoutes, { prefix: "/api" })
await app.register(containersRoutes, { prefix: "/api" })
await app.register(firewallRoutes, { prefix: "/api" }) await app.register(firewallRoutes, { prefix: "/api" })
await app.register(usersRoutes, { prefix: "/api" }) await app.register(usersRoutes, { prefix: "/api" })
await app.register(statisticsRoutes, { prefix: "/api" }) await app.register(statisticsRoutes, { prefix: "/api" })
+58
View File
@@ -0,0 +1,58 @@
import { z } from "zod"
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import {
getEnabledServerById,
listContainers,
listContainersForServer,
removeContainer,
restartContainer,
startContainer,
stopContainer,
} from "../services/containers-live.js"
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
const RosIdBodySchema = z.object({
rosId: z.string().min(1),
})
type RosIdBody = z.infer<typeof RosIdBodySchema>
type MutateFn = typeof startContainer
const containersRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/containers", async (_req, reply) => {
const containers = await listContainers()
return reply.send({ containers })
})
app.get("/servers/:id/containers", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
const params = req.params as ServerIdParams
const server = await getEnabledServerById(params.id)
if (!server) return reply.status(404).send({ error: "Server not found" })
const containers = await listContainersForServer(server)
return reply.send({ containers })
})
function registerMutate(path: string, fn: MutateFn) {
app.post(path, { schema: { params: ServerIdParamSchema, body: RosIdBodySchema } }, async (req, reply) => {
const params = req.params as ServerIdParams
const body = req.body as RosIdBody
const server = await getEnabledServerById(params.id)
if (!server) return reply.status(404).send({ error: "Server not found" })
try {
await fn(server, body.rosId)
return reply.send({ ok: true })
} catch (err) {
return reply.status(502).send({
error: err instanceof Error ? err.message : "Ошибка RouterOS",
})
}
})
}
registerMutate("/servers/:id/containers/start", startContainer)
registerMutate("/servers/:id/containers/stop", stopContainer)
registerMutate("/servers/:id/containers/restart", restartContainer)
registerMutate("/servers/:id/containers/remove", removeContainer)
}
export default containersRoutes
+35 -7
View File
@@ -6,9 +6,10 @@ import { MikrotikClient } from "../services/mikrotik.js"
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js" import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
import type { import type {
RosOspfNeighbor, RosOspfArea, RosOspfInterfaceTemplate, RosOspfInstance, RosOspfNeighbor, RosOspfArea, RosOspfInterfaceTemplate, RosOspfInstance,
RosBfdSession, RosBfdSession, RosIpRoute,
OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, BfdSessionRead, OspfNeighborRead, OspfInterfaceRead, OspfInstanceRead, OspfRouteRead, BfdSessionRead,
} from "../types/server.js" } from "../types/server.js"
import { parseOspfGateway, parseOspfRouteType } from "../services/ospf-route-parse.js"
import { z } from "zod" import { z } from "zod"
type ServerRow = typeof servers.$inferSelect type ServerRow = typeof servers.$inferSelect
@@ -74,14 +75,15 @@ function parseAddrIface(addr: string): { ip: string; iface: string } {
/** Fetch all OSPF + BFD data for one server */ /** Fetch all OSPF + BFD data for one server */
async function fetchServerOspf(server: ServerRow) { async function fetchServerOspf(server: ServerRow) {
const client = MikrotikClient.fromServer(server) const client = MikrotikClient.fromServer(server)
const [neighbors, areas, ifaceTemplates, instances, bfdSessions] = await Promise.all([ const [neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes] = await Promise.all([
client.getOspfNeighbors(), client.getOspfNeighbors(),
client.getOspfAreas(), client.getOspfAreas(),
client.getOspfInterfaceTemplates(), client.getOspfInterfaceTemplates(),
client.getOspfInstances(), client.getOspfInstances(),
client.getBfdSessions().catch(() => [] as RosBfdSession[]), // BFD is optional client.getBfdSessions().catch(() => [] as RosBfdSession[]), // BFD is optional
client.getIpRoutes().catch(() => [] as RosIpRoute[]),
]) ])
return { neighbors, areas, ifaceTemplates, instances, bfdSessions } return { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes }
} }
// ── BFD parser ──────────────────────────────────────────────────────────────── // ── BFD parser ────────────────────────────────────────────────────────────────
@@ -200,6 +202,29 @@ function parseInstances(
})) }))
} }
function parseOspfRoutes(server: ServerRow, routes: RosIpRoute[]): OspfRouteRead[] {
const out: OspfRouteRead[] = []
for (const [idx, r] of routes.entries()) {
const type = parseOspfRouteType(r)
if (!type) continue
const { nextHop, via } = parseOspfGateway(r)
const metric = parseInt(r["ospf-metric"] ?? r.distance ?? "0") || 0
out.push({
id: r[".id"] ?? String(idx),
serverId: server.id,
serverName: server.name || server.host,
serverSite: server.site,
destination: r["dst-address"] ?? "",
type,
cost: metric,
nextHop,
via,
area: r["ospf-area"] ?? "",
})
}
return out
}
function calcRouteScore(pingMs: number, dlMbps: number, ulMbps: number, pingWeight: number) { function calcRouteScore(pingMs: number, dlMbps: number, ulMbps: number, pingWeight: number) {
const pingScore = Math.max(0, 100 - pingMs * 0.6) const pingScore = Math.max(0, 100 - pingMs * 0.6)
const speedScore = Math.min(100, (dlMbps + ulMbps) / 18) const speedScore = Math.min(100, (dlMbps + ulMbps) / 18)
@@ -584,16 +609,17 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
const perServer = await Promise.all( const perServer = await Promise.all(
allServers.map(async (server) => { allServers.map(async (server) => {
try { try {
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server) const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
const areaMap = buildAreaMap(areas) const areaMap = buildAreaMap(areas)
return { return {
neighbors: parseNeighbors(server, neighbors, areaMap), neighbors: parseNeighbors(server, neighbors, areaMap),
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap), interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
instances: parseInstances(server, instances), instances: parseInstances(server, instances),
bfdSessions: parseBfdSessions(server, bfdSessions), bfdSessions: parseBfdSessions(server, bfdSessions),
routes: parseOspfRoutes(server, ipRoutes),
} }
} catch { } catch {
return { neighbors: [], interfaces: [], instances: [], bfdSessions: [] } return { neighbors: [], interfaces: [], instances: [], bfdSessions: [], routes: [] }
} }
}), }),
) )
@@ -603,6 +629,7 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
interfaces: perServer.flatMap(r => r.interfaces), interfaces: perServer.flatMap(r => r.interfaces),
instances: perServer.flatMap(r => r.instances), instances: perServer.flatMap(r => r.instances),
bfdSessions: perServer.flatMap(r => r.bfdSessions), bfdSessions: perServer.flatMap(r => r.bfdSessions),
routes: perServer.flatMap(r => r.routes),
}) })
}) })
@@ -634,13 +661,14 @@ const ospfRoutes: FastifyPluginAsyncZod = async (app) => {
if (!server) return reply.status(404).send({ error: "Server not found" }) if (!server) return reply.status(404).send({ error: "Server not found" })
try { try {
const { neighbors, areas, ifaceTemplates, instances, bfdSessions } = await fetchServerOspf(server) const { neighbors, areas, ifaceTemplates, instances, bfdSessions, ipRoutes } = await fetchServerOspf(server)
const areaMap = buildAreaMap(areas) const areaMap = buildAreaMap(areas)
return reply.send({ return reply.send({
neighbors: parseNeighbors(server, neighbors, areaMap), neighbors: parseNeighbors(server, neighbors, areaMap),
interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap), interfaces: parseInterfaces(server, ifaceTemplates, areas, instances, areaMap),
instances: parseInstances(server, instances), instances: parseInstances(server, instances),
bfdSessions: parseBfdSessions(server, bfdSessions), bfdSessions: parseBfdSessions(server, bfdSessions),
routes: parseOspfRoutes(server, ipRoutes),
areas: areas.map(a => ({ name: a.name, areaId: a["area-id"] ?? "0.0.0.0", type: a.type, disabled: a.disabled === "true", inactive: a.inactive === "true", instance: a.instance })), areas: areas.map(a => ({ name: a.name, areaId: a["area-id"] ?? "0.0.0.0", type: a.type, disabled: a.disabled === "true", inactive: a.inactive === "true", instance: a.instance })),
}) })
} catch (err) { } catch (err) {
+10 -1
View File
@@ -2,6 +2,9 @@ import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { count } from "drizzle-orm" import { count } from "drizzle-orm"
import { listCertificatesFromServers } from "../services/certificates-service.js" import { listCertificatesFromServers } from "../services/certificates-service.js"
import { countWireGuardInterfaces } from "../services/wireguard-live.js" import { countWireGuardInterfaces } from "../services/wireguard-live.js"
import { countVxlanTunnels } from "../services/vxlan-live.js"
import { countContainers } from "../services/containers-live.js"
import { countBgpSessions } from "../services/bgp-peers-live.js"
import { db } from "../db/index.js" import { db } from "../db/index.js"
import { import {
filterRules, filterRules,
@@ -24,9 +27,12 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
const uptimeProbesTotal = await tableCount(uptimeProbes) const uptimeProbesTotal = await tableCount(uptimeProbes)
const uptimeSpeedProbesTotal = await tableCount(uptimeSpeedProbes) const uptimeSpeedProbesTotal = await tableCount(uptimeSpeedProbes)
const recursiveRoutesTotal = await tableCount(recursiveRoutes) const recursiveRoutesTotal = await tableCount(recursiveRoutes)
const [certRes, wireguardTotal] = await Promise.all([ const [certRes, wireguardTotal, bgpTotal, vxlanTotal, containersTotal] = await Promise.all([
listCertificatesFromServers(), listCertificatesFromServers(),
countWireGuardInterfaces().catch(() => 0), countWireGuardInterfaces().catch(() => 0),
countBgpSessions().catch(() => 0),
countVxlanTunnels().catch(() => 0),
countContainers().catch(() => 0),
]) ])
const certificatesTotal = certRes.certificates.length const certificatesTotal = certRes.certificates.length
const usersTotal = (await listUsers()).length const usersTotal = (await listUsers()).length
@@ -41,6 +47,9 @@ const sidebarCountsRoutes: FastifyPluginAsyncZod = async (app) => {
certificates: certificatesTotal, certificates: certificatesTotal,
wireguard: wireguardTotal, wireguard: wireguardTotal,
users: usersTotal, users: usersTotal,
bgpSessions: bgpTotal,
vxlan: vxlanTotal,
containers: containersTotal,
}) })
}) })
} }
+13 -2
View File
@@ -1,6 +1,6 @@
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod" import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { statisticsQuerySchema } from "@mmapp/contracts/statistics" import { statisticsPivotQuerySchema, statisticsQuerySchema } from "@mmapp/contracts/statistics"
import { getStatistics } from "../services/statistics-aggregate.js" import { getStatistics, getStatisticsPivot, pivotDimsConflict } from "../services/statistics-aggregate.js"
const statisticsRoutes: FastifyPluginAsyncZod = async (app) => { const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/statistics", async (req, reply) => { app.get("/statistics", async (req, reply) => {
@@ -10,6 +10,17 @@ const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
} }
return reply.send(await getStatistics(parsed.data)) return reply.send(await getStatistics(parsed.data))
}) })
app.get("/statistics/pivot", async (req, reply) => {
const parsed = statisticsPivotQuerySchema.safeParse(req.query ?? {})
if (!parsed.success) {
return reply.status(400).send({ error: "Некорректный период или измерения", details: parsed.error.flatten() })
}
if (pivotDimsConflict(parsed.data.row, parsed.data.col)) {
return reply.status(400).send({ error: "Строки и колонки должны отличаться" })
}
return reply.send(await getStatisticsPivot(parsed.data))
})
} }
export default statisticsRoutes export default statisticsRoutes
+22
View File
@@ -27,6 +27,7 @@ import {
import { buildFlowMapHops } from "../services/traffic-flow-map-hops.js" import { buildFlowMapHops } from "../services/traffic-flow-map-hops.js"
import { applyFlowOverlay } from "../services/traffic-flow-overlay.js" import { applyFlowOverlay } from "../services/traffic-flow-overlay.js"
import { listTrafficFlowHostFiles } from "../services/traffic-flow-host-files.js" import { listTrafficFlowHostFiles } from "../services/traffic-flow-host-files.js"
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
import { appendEvent } from "../modules/events/service/events-service.js" import { appendEvent } from "../modules/events/service/events-service.js"
const LIVE_TICK_MS = 2000 const LIVE_TICK_MS = 2000
@@ -203,6 +204,27 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
} }
}) })
app.post("/traffic/flow/rebuild-facts", async (_req, reply) => {
try {
const result = await rebuildFlowFactsFromBuckets()
await appendEvent({
level: "info",
eventType: "traffic.flow.rebuild_facts",
sourceModule: "traffic",
title: "Пересчитан куб NetFlow",
message: `Факты ${result.facts} из ${result.buckets} сессий, дней ${result.days.length}`,
entityType: "traffic_flow",
entityId: "rebuild-facts",
payload: { buckets: result.buckets, facts: result.facts, days: result.days },
})
return reply.send(result)
} catch (err) {
const message = err instanceof Error ? err.message : String(err)
const status = message.includes("уже выполняется") ? 409 : 500
return reply.status(status).send({ error: message })
}
})
app.post("/traffic/flow/overlay", applyOverlayHandler) app.post("/traffic/flow/overlay", applyOverlayHandler)
app.post("/traffic/flow-overlay", applyOverlayHandler) app.post("/traffic/flow-overlay", applyOverlayHandler)
+23
View File
@@ -0,0 +1,23 @@
import { eq } from "drizzle-orm"
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
import { db } from "../db/index.js"
import { servers } from "../db/schema.js"
import { listVxlanTunnels, listVxlanTunnelsForServer } from "../services/vxlan-live.js"
import { ServerIdParamSchema, type ServerIdParams } from "../types/server.js"
const vxlanRoutes: FastifyPluginAsyncZod = async (app) => {
app.get("/vxlan", async (_req, reply) => {
const tunnels = await listVxlanTunnels()
return reply.send({ tunnels })
})
app.get("/servers/:id/vxlan", { schema: { params: ServerIdParamSchema } }, async (req, reply) => {
const params = req.params as ServerIdParams
const server = (await db.select().from(servers).where(eq(servers.id, params.id)).limit(1))[0]
if (!server) return reply.status(404).send({ error: "Server not found" })
const tunnels = await listVxlanTunnelsForServer(server)
return reply.send({ tunnels })
})
}
export default vxlanRoutes
@@ -0,0 +1,8 @@
import { initDatabase } from "../db/bootstrap.js"
import { closePool } from "../db/index.js"
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
await initDatabase()
const result = await rebuildFlowFactsFromBuckets()
console.log(JSON.stringify(result, null, 2))
await closePool()
+13
View File
@@ -28,3 +28,16 @@ export async function fetchBgpSessionsForAlerts(): Promise<BgpSessionRead[]> {
) )
return results.flat() return results.flat()
} }
export async function countBgpSessions(): Promise<number> {
try {
const result = await Promise.race([
fetchBgpSessionsForAlerts(),
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
])
if (!result) return 0
return result.length
} catch {
return 0
}
}
@@ -0,0 +1,46 @@
import assert from "node:assert/strict"
import { mapContainerRow } from "./containers-live.js"
const server = {
id: 3,
name: "mt-spb",
host: "10.0.1.1",
} as Parameters<typeof mapContainerRow>[0]
const row = mapContainerRow(
server,
{
".id": "*A",
name: "adguard",
"remote-image": "adguard/adguardhome:latest",
interface: "veth-adguard",
envlist: "adguard-envs",
mounts: "agh-conf,agh-work",
status: "running",
"start-on-boot": "true",
comment: "DNS",
},
[
{ name: "adguard-envs", key: "FOO", value: "bar" },
{ name: "other", key: "SKIP", value: "x" },
],
[
{ name: "agh-conf", dst: "/opt/conf", src: "/disk1/conf" },
{ name: "agh-work", dst: "/opt/work" },
],
0,
)
assert.equal(row.rosId, "*A")
assert.equal(row.image, "adguard/adguardhome")
assert.equal(row.tag, "latest")
assert.equal(row.status, "running")
assert.deepEqual(row.interfaces, ["veth-adguard"])
assert.deepEqual(row.envs, [{ key: "FOO", value: "bar" }])
assert.deepEqual(row.mounts, [
{ dst: "/opt/conf", src: "/disk1/conf" },
{ dst: "/opt/work", src: undefined },
])
assert.equal(row.startOnBoot, true)
console.log("containers-live.test.ts: ok")
+215
View File
@@ -0,0 +1,215 @@
import { eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { servers } from "../db/schema.js"
import { MikrotikClient, MikrotikError } from "./mikrotik.js"
type ServerRow = typeof servers.$inferSelect
interface RosContainer {
".id"?: string
name?: string
tag?: string
"remote-image"?: string
interface?: string
envlist?: string
mounts?: string
cmd?: string
"start-on-boot"?: string
comment?: string
status?: string
"memory-high"?: string
cpu?: string
}
interface RosContainerEnv {
name?: string
key?: string
value?: string
}
interface RosContainerMount {
name?: string
src?: string
dst?: string
}
export type ContainerLiveStatus = "running" | "stopped" | "error"
export type ContainerLive = {
id: string
rosId: string
name: string
serverId: string
image: string
tag: string
status: ContainerLiveStatus
envs: { key: string; value: string }[]
mounts: { dst: string; src?: string }[]
interfaces: string[]
cmd?: string
startOnBoot: boolean
comment: string
uptime?: string
cpu?: number
memMb?: number
}
function rosYes(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
function mapStatus(raw: string | undefined): ContainerLiveStatus {
const s = (raw ?? "").toLowerCase()
if (s === "running") return "running"
if (s === "error" || s === "failed") return "error"
return "stopped"
}
function splitCsv(v: string | undefined): string[] {
return (v ?? "")
.split(",")
.map((x) => x.trim())
.filter(Boolean)
}
function parseImageTag(c: RosContainer): { image: string; tag: string } {
const remote = (c["remote-image"] ?? "").trim()
if (remote) {
const idx = remote.lastIndexOf(":")
if (idx > 0 && !remote.slice(idx + 1).includes("/")) {
return { image: remote.slice(0, idx), tag: remote.slice(idx + 1) }
}
return { image: remote, tag: (c.tag ?? "latest").trim() || "latest" }
}
return { image: (c.name ?? "").trim(), tag: (c.tag ?? "latest").trim() || "latest" }
}
function isMissingPackage(err: unknown): boolean {
if (err instanceof MikrotikError) {
if (err.statusCode === 404) return true
const body = err.body.toLowerCase()
return body.includes("no such command") || body.includes("not found") || body.includes("unknown")
}
const msg = err instanceof Error ? err.message.toLowerCase() : String(err).toLowerCase()
return msg.includes("no such command") || msg.includes("404")
}
export function mapContainerRow(
server: ServerRow,
c: RosContainer,
envs: RosContainerEnv[],
mounts: RosContainerMount[],
idx: number,
): ContainerLive {
const rosId = String(c[".id"] ?? `c-${idx}`)
const name = (c.name ?? "").trim() || `container-${idx + 1}`
const { image, tag } = parseImageTag(c)
const envlist = (c.envlist ?? "").trim()
const mountNames = new Set(splitCsv(c.mounts))
const envRows = envlist
? envs.filter((e) => (e.name ?? "").trim() === envlist && (e.key ?? "").trim())
: []
const mountRows = mounts.filter((m) => mountNames.has((m.name ?? "").trim()) && (m.dst ?? "").trim())
const cpuRaw = Number.parseInt(c.cpu ?? "", 10)
const memRaw = Number.parseInt(c["memory-high"] ?? "", 10)
return {
id: `${server.id}-${rosId}`,
rosId,
name,
serverId: String(server.id),
image,
tag,
status: mapStatus(c.status),
envs: envRows.map((e) => ({ key: e.key ?? "", value: e.value ?? "" })),
mounts: mountRows.map((m) => ({ dst: m.dst ?? "", src: m.src || undefined })),
interfaces: splitCsv(c.interface),
cmd: (c.cmd ?? "").trim() || undefined,
startOnBoot: rosYes(c["start-on-boot"]),
comment: c.comment ?? "",
cpu: Number.isFinite(cpuRaw) ? cpuRaw : undefined,
memMb: Number.isFinite(memRaw) ? Math.round(memRaw / (1024 * 1024)) || undefined : undefined,
}
}
async function fetchContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
const client = MikrotikClient.fromServer(server)
try {
const [raw, envsRaw, mountsRaw] = await Promise.all([
client.get<RosContainer[]>("/container"),
client.get<RosContainerEnv[]>("/container/envs").catch(() => [] as RosContainerEnv[]),
client.get<RosContainerMount[]>("/container/mounts").catch(() => [] as RosContainerMount[]),
])
const list = Array.isArray(raw) ? raw : []
const envs = Array.isArray(envsRaw) ? envsRaw : []
const mounts = Array.isArray(mountsRaw) ? mountsRaw : []
return list.map((c, idx) => mapContainerRow(server, c, envs, mounts, idx))
} catch (err) {
if (isMissingPackage(err)) return []
throw err
}
}
export async function listContainers(): Promise<ContainerLive[]> {
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
const results = await Promise.all(
enabledServers.map(async (server) => {
try {
return await fetchContainersForServer(server)
} catch {
return [] as ContainerLive[]
}
}),
)
return results.flat()
}
export async function listContainersForServer(server: ServerRow): Promise<ContainerLive[]> {
try {
return await fetchContainersForServer(server)
} catch {
return []
}
}
export async function countContainers(): Promise<number> {
try {
const result = await Promise.race([
listContainers(),
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
])
if (!result) return 0
return result.length
} catch {
return 0
}
}
function encodeRosId(rosId: string): string {
return encodeURIComponent(rosId)
}
export async function startContainer(server: ServerRow, rosId: string): Promise<void> {
const client = MikrotikClient.fromServer(server)
await client.post("/container/start", { ".id": rosId })
}
export async function stopContainer(server: ServerRow, rosId: string): Promise<void> {
const client = MikrotikClient.fromServer(server)
await client.post("/container/stop", { ".id": rosId })
}
export async function restartContainer(server: ServerRow, rosId: string): Promise<void> {
await stopContainer(server, rosId)
await startContainer(server, rosId)
}
export async function removeContainer(server: ServerRow, rosId: string): Promise<void> {
const client = MikrotikClient.fromServer(server)
await client.delete(`/container/${encodeRosId(rosId)}`)
}
export async function getEnabledServerById(serverId: string | number) {
const id = typeof serverId === "number" ? serverId : Number.parseInt(String(serverId), 10)
if (!Number.isFinite(id)) return null
return (await db.select().from(servers).where(eq(servers.id, id)).limit(1))[0] ?? null
}
@@ -0,0 +1,22 @@
import assert from "node:assert/strict"
import type { RosIpRoute } from "../types/server.js"
import { parseOspfGateway, parseOspfRouteType } from "./ospf-route-parse.js"
function route(partial: Partial<RosIpRoute>): RosIpRoute {
return { ".id": "*1", "dst-address": "10.0.0.0/8", ...partial }
}
assert.equal(parseOspfRouteType(route({ static: "true" })), null)
assert.equal(parseOspfRouteType(route({ bgp: "true" })), null)
assert.equal(parseOspfRouteType(route({ ospf: "true" })), "O")
assert.equal(parseOspfRouteType(route({ "ospf-type": "intra-area" })), "O")
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "inter-area" })), "O IA")
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "ext-type-1" })), "O E1")
assert.equal(parseOspfRouteType(route({ ospf: "true", "ospf-type": "type-2" })), "O E2")
assert.deepEqual(parseOspfGateway(route({ gateway: "10.200.0.1%gre-msk-spb" })), {
nextHop: "10.200.0.1",
via: "gre-msk-spb",
})
console.log("ospf-route-parse.test.ts: ok")
+28
View File
@@ -0,0 +1,28 @@
import type { RosIpRoute } from "../types/server.js"
export type OspfRouteKind = "O" | "O IA" | "O E1" | "O E2"
/** RouterOS /ip/route → тип OSPF-маршрута UI, либо null если маршрут не OSPF. */
export function parseOspfRouteType(r: RosIpRoute): OspfRouteKind | null {
const ospfFlag = r.ospf === "true" || r.ospf === "yes"
const raw = `${r["ospf-type"] ?? ""} ${r.type ?? ""}`.toLowerCase()
const looksOspf = ospfFlag || raw.includes("ospf") || Boolean(r["ospf-type"])
if (!looksOspf) return null
if (raw.includes("inter")) return "O IA"
if (raw.includes("e1") || raw.includes("type-1") || raw.includes("ext-1") || raw.includes("nssa-ext-type-1")) {
return "O E1"
}
if (raw.includes("e2") || raw.includes("type-2") || raw.includes("ext-2") || raw.includes("nssa-ext-type-2")) {
return "O E2"
}
return "O"
}
export function parseOspfGateway(r: RosIpRoute): { nextHop: string; via: string } {
const gw = (r.gateway ?? r["immediate-gw"] ?? "").trim()
const [ip, iface = ""] = gw.split("%")
return {
nextHop: ip || gw || "—",
via: iface || (r.interface ?? "—"),
}
}
+211 -18
View File
@@ -1,9 +1,13 @@
import assert from "node:assert/strict" import assert from "node:assert/strict"
import { getStatistics, parseStatisticsPeriod } from "./statistics-aggregate.js" import { getStatistics, getStatisticsPivot, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
import { withPgOrSkip } from "../test/pg.js" import { withPgOrSkip } from "../test/pg.js"
import { dbQuery } from "../db/index.js" import { dbQuery } from "../db/index.js"
import { ensurePartitionFor } from "../db/partitions.js" import { ensurePartitionFor } from "../db/partitions.js"
import { pool } from "../db/index.js" import { pool } from "../db/index.js"
import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
{ {
const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10") const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10")
@@ -16,6 +20,8 @@ import { pool } from "../db/index.js"
assert.equal(month.grain, "day") assert.equal(month.grain, "day")
assert.equal(month.toDayExclusive, "2026-09-01") assert.equal(month.toDayExclusive, "2026-09-01")
assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null) assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null)
assert.equal(pivotDimsConflict("country", "country"), true)
assert.equal(pivotDimsConflict("country", "service"), false)
} }
if (!(await withPgOrSkip())) { if (!(await withPgOrSkip())) {
@@ -24,16 +30,27 @@ if (!(await withPgOrSkip())) {
} }
const inserted = await dbQuery<{ id: number }>(` const inserted = await dbQuery<{ id: number }>(`
INSERT INTO servers (name, host) VALUES ('stats-cube', '127.0.0.1') RETURNING id INSERT INTO servers (name, host, type, wan_uplinks)
VALUES ('stats-cube', '127.0.0.1', 'jump-host', '[{"iface":"wan1"}]'::jsonb)
RETURNING id
`) `)
const serverId = inserted.rows[0]?.id const serverId = inserted.rows[0]?.id
if (serverId == null) throw new Error("no server") if (serverId == null) throw new Error("no server")
const enInserted = await dbQuery<{ id: number }>(`
INSERT INTO servers (name, host, type)
VALUES ('stats-en', '198.51.100.1', 'exit-node')
RETURNING id
`)
const enId = enInserted.rows[0]?.id
if (enId == null) throw new Error("no en server")
await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z")) await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z"))
await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z")) await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z"))
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId]) await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId]) await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId]) await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`) await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`)
await dbQuery(` await dbQuery(`
@@ -43,23 +60,156 @@ await dbQuery(`
`) `)
await dbQuery(` await dbQuery(`
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type) INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
VALUES ('bind-stats-1', 'u-stats-1', $1, 'ether1', 'ether') VALUES ('bind-stats-1', 'u-stats-1', $1, 'gre-client', 'gre')
`, [serverId]) `, [serverId])
await dbQuery(`
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
VALUES
($1, '2026-09-10T12:00:00Z', 'online', $3::jsonb),
($2, '2026-09-10T12:00:00Z', 'online', $4::jsonb)
`, [
serverId,
enId,
JSON.stringify([
{ name: "gre-client", type: "gre-tunnel" },
{ name: "wan1", type: "ether" },
{ name: "gre-en", type: "gre-tunnel" },
{ name: "NSK-SERVHOST-RTK", type: "gre-tunnel" },
{ name: "wg-mesh", type: "wg" },
{ name: "wg-server", type: "wg" },
{ name: "wg-flow", type: "wg" },
]),
JSON.stringify([
{ name: "ether1", type: "ether" },
{ name: "gre-jh", type: "gre-tunnel" },
]),
])
resetIfaceCacheForTests()
rememberServerIfaces(serverId, [
{ name: "gre-client", ifindex: "2" },
{ name: "wan1", ifindex: "8" },
{ name: "gre-en", ifindex: "9" },
{ name: "NSK-SERVHOST-RTK" },
{ name: "wg-mesh" },
{ name: "wg-server" },
{ name: "wg-flow" },
])
rememberServerIfaces(enId, [
{ name: "ether1", ifindex: "2" },
{ name: "gre-jh", ifindex: "5" },
])
setRefreshIfacesForTests(async () => {})
invalidateFlowCatalogCache()
await dbQuery(` await dbQuery(`
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets) INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
VALUES VALUES
($1, '2026-09-10', 'ether1', 'US', 'https', 15169, 800, 10), ($1, '2026-09-10', '2', 'US', 'https', 15169, 800, 10),
($1, '2026-09-10', 'ether1', 'DE', 'dns', 15133, 200, 4) ($1, '2026-09-10', '2', 'DE', 'dns', 15133, 200, 4),
`, [serverId]) ($1, '2026-09-10', 'wan1', 'NL', 'other', 0, 70, 1),
($1, '2026-09-10', '0', 'US', 'https', 0, 999, 3),
($1, '2026-09-10', 'gre-en', 'US', 'https', 15169, 400, 2),
($1, '2026-09-10', 'NSK-SERVHOST-RTK', 'US', 'https', 15169, 300, 2),
($1, '2026-09-10', 'wg-mesh', 'US', 'https', 0, 250, 2),
($1, '2026-09-10', 'wg-flow', 'US', 'https', 0, 80, 1),
($2, '2026-09-10', 'gre-jh', 'US', 'https', 15169, 500, 5),
($2, '2026-09-10', 'ether1', 'US', 'https', 15169, 200, 2)
`, [serverId, enId])
try { try {
const all = await getStatistics({ from: "2026-09-01", to: "2026-09-30" }) const unique = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
assert.equal(all.grain, "day") assert.equal(unique.grain, "day")
assert.equal(all.kpis.bytes, 1000) assert.equal(unique.kpis.bytes, 1000)
assert.ok(all.countries.some((r) => r.id === "US")) const uniqueAsnSum = unique.asns.reduce((s, r) => s + r.bytes, 0)
assert.ok(all.users.some((r) => r.id === "u-stats-1")) assert.equal(uniqueAsnSum, unique.kpis.bytes, "unique KPI = SUM dest ASN")
assert.ok(all.servers.some((r) => r.id === String(serverId))) assert.equal(unique.kpis.users, 1)
assert.ok(unique.countries.some((r) => r.id === "US"))
assert.ok(unique.users.some((r) => r.id === "u-stats-1"))
assert.equal(unique.users.find((r) => r.id === STATISTICS_UNBOUND_USER_ID), undefined)
assert.ok(unique.servers.some((r) => r.id === String(serverId)))
assert.ok(!unique.servers.some((r) => r.id === String(enId)), "EN-транзит не в сетевом KPI")
const greIface = unique.interfaces.find((r) => r.label.includes("gre-client"))
assert.ok(greIface)
assert.equal(greIface.bytes, 1000)
assert.equal(greIface.id, `${serverId}:gre-client`)
assert.ok(!unique.interfaces.some((r) => /· (?:#)?\d+$/.test(r.label)))
assert.ok(!unique.interfaces.some((r) => r.label.includes(" · —") || r.label.endsWith("· —")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("gre-en")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("NSK-SERVHOST-RTK")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-mesh")))
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-flow")))
assert.equal(unique.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined, "unique без WAN")
const allPlanes = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "all" })
assert.equal(allPlanes.kpis.bytes, 1000, "KPI unique и all одинаковый")
const wanRow = allPlanes.interfaces.find((r) => r.id === `${serverId}:wan1`)
assert.ok(wanRow)
assert.ok(wanRow.label.includes("WAN · интернет"))
assert.equal(wanRow.bytes, 70)
assert.equal(wanRow.percent, 0)
const overlayGre = allPlanes.interfaces.find((r) => r.id === `${serverId}:gre-en`)
assert.ok(overlayGre)
assert.ok(overlayGre.label.includes("дубль"))
assert.equal(overlayGre.percent, 0)
const overlayCustom = allPlanes.interfaces.find((r) => r.label.includes("NSK-SERVHOST-RTK"))
assert.ok(overlayCustom)
assert.ok(overlayCustom.label.includes("дубль"))
const overlayWg = allPlanes.interfaces.find((r) => r.label.includes("wg-mesh"))
assert.ok(overlayWg)
assert.ok(overlayWg.label.includes("дубль"))
assert.ok(!allPlanes.interfaces.some((r) => r.label.includes("wg-flow")))
const wanSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId,
iface: "wan1",
})
assert.equal(wanSlice.kpis.bytes, 70)
const nodeSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId,
planes: "unique",
})
assert.equal(nodeSlice.kpis.bytes, 1000)
assert.equal(nodeSlice.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined)
assert.ok(!nodeSlice.users.some((r) => r.id === STATISTICS_UNBOUND_USER_ID))
const nodeAll = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId,
planes: "all",
})
assert.equal(nodeAll.kpis.bytes, 1000)
const nodeWan = nodeAll.interfaces.find((r) => r.id === `${serverId}:wan1`)
assert.ok(nodeWan)
assert.equal(nodeWan.percent, 0)
assert.ok(nodeWan.label.includes("WAN · интернет"))
const enSlice = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId: enId,
planes: "unique",
})
assert.equal(enSlice.kpis.bytes, 0)
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("gre-jh")))
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("WAN · интернет")))
const enAll = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
serverId: enId,
planes: "all",
})
assert.equal(enAll.kpis.bytes, 0)
assert.ok(enAll.interfaces.some((r) => r.label.includes("WAN · интернет") && r.label.includes("ether1") && r.percent === 0))
assert.ok(enAll.interfaces.some((r) => r.label.includes("gre-jh") && r.label.includes("дубль")))
const sliced = await getStatistics({ const sliced = await getStatistics({
from: "2026-09-01", from: "2026-09-01",
@@ -73,9 +223,46 @@ try {
assert.equal(sliced.countries[0]?.id, "US") assert.equal(sliced.countries[0]?.id, "US")
assert.ok(sliced.users.some((r) => r.id === "u-stats-1")) assert.ok(sliced.users.some((r) => r.id === "u-stats-1"))
const byUser = await getStatistics({
from: "2026-09-01",
to: "2026-09-30",
userId: "u-stats-1",
})
assert.equal(byUser.kpis.bytes, 1000)
const pivot = await getStatisticsPivot({
from: "2026-09-01",
to: "2026-09-30",
row: "country",
col: "service",
metric: "bytes",
})
const us = pivot.rows.find((r) => r.id === "US")
const de = pivot.rows.find((r) => r.id === "DE")
assert.ok(us)
assert.ok(de)
assert.equal(us.cells.https, 800)
assert.equal(de.cells.dns, 200)
await dbQuery(`
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
VALUES ('bind-stats-wg', 'u-stats-1', $1, 'wg-server', 'wg')
`, [serverId])
await dbQuery(`
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
VALUES ($1, '2026-09-10', 'wg-server', 'US', 'https', 15169, 150, 2)
`, [serverId])
invalidateFlowCatalogCache()
const withWg = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
assert.equal(withWg.kpis.bytes, 1150)
assert.ok(withWg.interfaces.some((r) => r.label.includes("wg-server") && r.bytes === 150))
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-flow")))
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-mesh")))
await dbQuery(` await dbQuery(`
INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets) INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets)
VALUES ($1, '2026-09-10T10:00:00Z', 'ether1', 'US', 'https', 15169, 40, 2) VALUES ($1, '2026-09-10T10:00:00Z', '2', 'US', 'https', 15169, 40, 2)
`, [serverId]) `, [serverId])
const hourly = await getStatistics({ const hourly = await getStatistics({
from: "2026-09-10T00:00:00.000Z", from: "2026-09-10T00:00:00.000Z",
@@ -83,10 +270,16 @@ try {
}) })
assert.equal(hourly.grain, "hour") assert.equal(hourly.grain, "hour")
assert.equal(hourly.kpis.bytes, 40) assert.equal(hourly.kpis.bytes, 40)
assert.ok(hourly.users.some((r) => r.id === "u-stats-1"))
} finally { } finally {
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId]) setRefreshIfacesForTests(null)
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId]) resetIfaceCacheForTests()
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId]) invalidateFlowCatalogCache()
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
await dbQuery(`DELETE FROM servers WHERE id IN ($1, $2)`, [serverId, enId])
} }
console.log("statistics-aggregate.test.ts: ok") console.log("statistics-aggregate.test.ts: ok")
+570 -54
View File
@@ -1,14 +1,38 @@
import { eq } from "drizzle-orm" import { eq } from "drizzle-orm"
import { db, dbAll } from "../db/index.js" import { db, dbAll } from "../db/index.js"
import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js" import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js"
import type { import {
StatisticsBreakdownRow, STATISTICS_UNBOUND_USER_ID,
StatisticsDto, type StatisticsBreakdownRow,
StatisticsQuery, type StatisticsDto,
type StatisticsPivotDim,
type StatisticsPivotDto,
type StatisticsPivotQuery,
type StatisticsQuery,
} from "@mmapp/contracts/statistics" } from "@mmapp/contracts/statistics"
import {
collapseServerIfaceRows,
displayFactIface,
expandBindingIfaces,
factIfaceAliases,
listCachedIfaceNames,
} from "./traffic-flow-ifindex.js"
import { refreshServerIfaces } from "./traffic-flow-ifaces.js"
import {
isDashDisplayIface,
isJunkFactIface,
isOverlayTunnelIface,
isWanFactIface,
overlayDupLabel,
wanIfaceLabel,
} from "./traffic-flow-facts-filter.js"
import { getServerCatalog, loadFlowTopology, type FlowTopology } from "./traffic-flow-topology.js"
const TOP_N = 200 const TOP_N = 200
const HOUR_WINDOW_MS = 48 * 3600_000 const HOUR_WINDOW_MS = 48 * 3600_000
const PIVOT_ROW_CAP = 50
const PIVOT_COL_CAP = 15
const PIVOT_OTHER_ID = "__other__"
export interface ParsedPeriod { export interface ParsedPeriod {
fromIso: string fromIso: string
@@ -64,6 +88,8 @@ export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPer
} }
} }
type FactScope = "unique" | "wan" | "overlay"
interface FilterCtx { interface FilterCtx {
fromIso: string fromIso: string
toIso: string toIso: string
@@ -74,10 +100,68 @@ interface FilterCtx {
country?: string country?: string
service?: string service?: string
asn?: number asn?: number
planes: "unique" | "all"
userIfaces: Array<{ serverId: number; iface: string }> | null userIfaces: Array<{ serverId: number; iface: string }> | null
unboundOnly: boolean
boundIfaces: Array<{ serverId: number; iface: string }>
overlayIfaces: Array<{ serverId: number; iface: string }>
wanIfaces: Array<{ serverId: number; iface: string }>
excludeServerIds: number[]
topo: FlowTopology | null
} }
function factWhere(alias: string, grain: "hour" | "day", ctx: FilterCtx): { sql: string; params: unknown[] } { function ifaceFilterAliases(iface: string, serverId?: number): string[] {
return factIfaceAliases(iface.trim(), serverId)
}
function looksLikeIfIndex(iface: string): boolean {
const raw = iface.trim()
return /^\d+$/.test(raw) || /^#\d+$/.test(raw)
}
async function warmIfaceCache(ids: Iterable<number>): Promise<void> {
const uniq = [...new Set(ids)].filter((id) => Number.isFinite(id) && id > 0)
if (!uniq.length) return
await Promise.all(uniq.map((id) => refreshServerIfaces(id)))
}
async function warmBindingIfaceCache(): Promise<void> {
const rows = await db.select({ serverId: userInterfaceBindings.serverId }).from(userInterfaceBindings)
await warmIfaceCache(rows.map((r) => r.serverId))
}
function canonicalIfaceDimId(id: string): string {
const colon = id.indexOf(":")
if (colon < 0) return id
const sid = Number(id.slice(0, colon))
if (!Number.isFinite(sid)) return id
return `${sid}:${displayFactIface(sid, id.slice(colon + 1))}`
}
function pushIfaceTuples(
parts: string[],
params: unknown[],
alias: string,
tuples: Array<{ serverId: number; iface: string }>,
op: "IN" | "NOT IN",
): void {
if (!tuples.length) {
if (op === "IN") parts.push("FALSE")
return
}
const sql = tuples.map(() => "(?, ?)").join(", ")
parts.push(`(${alias}.server_id, ${alias}.iface) ${op} (${sql})`)
for (const t of tuples) {
params.push(t.serverId, t.iface)
}
}
function factWhere(
alias: string,
grain: "hour" | "day",
ctx: FilterCtx,
scope: FactScope = "unique",
): { sql: string; params: unknown[] } {
const params: unknown[] = [] const params: unknown[] = []
const parts: string[] = [] const parts: string[] = []
if (grain === "hour") { if (grain === "hour") {
@@ -91,10 +175,6 @@ function factWhere(alias: string, grain: "hour" | "day", ctx: FilterCtx): { sql:
parts.push(`${alias}.server_id = ?`) parts.push(`${alias}.server_id = ?`)
params.push(ctx.serverId) params.push(ctx.serverId)
} }
if (ctx.iface) {
parts.push(`${alias}.iface = ?`)
params.push(ctx.iface)
}
if (ctx.country) { if (ctx.country) {
parts.push(`${alias}.country = ?`) parts.push(`${alias}.country = ?`)
params.push(ctx.country.toUpperCase()) params.push(ctx.country.toUpperCase())
@@ -107,22 +187,44 @@ function factWhere(alias: string, grain: "hour" | "day", ctx: FilterCtx): { sql:
parts.push(`${alias}.asn = ?`) parts.push(`${alias}.asn = ?`)
params.push(ctx.asn) params.push(ctx.asn)
} }
if (ctx.userIfaces) { parts.push(`${alias}.iface NOT IN ('0', '—', '__unknown__', 'wg-flow', '')`)
if (ctx.userIfaces.length === 0) {
parts.push("FALSE") if (scope === "wan") {
pushIfaceTuples(parts, params, alias, ctx.wanIfaces, "IN")
return { sql: parts.join(" AND "), params }
}
if (scope === "overlay") {
pushIfaceTuples(parts, params, alias, ctx.overlayIfaces, "IN")
return { sql: parts.join(" AND "), params }
}
if (ctx.iface) {
const aliases = ifaceFilterAliases(ctx.iface, ctx.serverId)
if (aliases.length <= 1) {
parts.push(`${alias}.iface = ?`)
params.push(aliases[0] ?? ctx.iface)
} else { } else {
const tuples = ctx.userIfaces.map(() => "(?, ?)").join(", ") parts.push(`${alias}.iface IN (${aliases.map(() => "?").join(", ")})`)
parts.push(`(${alias}.server_id, ${alias}.iface) IN (${tuples})`) params.push(...aliases)
for (const u of ctx.userIfaces) {
params.push(u.serverId, u.iface)
}
} }
return { sql: parts.join(" AND "), params }
}
if (ctx.userIfaces) {
pushIfaceTuples(parts, params, alias, ctx.userIfaces, "IN")
return { sql: parts.join(" AND "), params }
}
if (ctx.unboundOnly) {
parts.push("FALSE")
return { sql: parts.join(" AND "), params }
}
pushIfaceTuples(parts, params, alias, ctx.boundIfaces, "IN")
if (ctx.excludeServerIds.length) {
parts.push(`${alias}.server_id NOT IN (${ctx.excludeServerIds.map(() => "?").join(", ")})`)
params.push(...ctx.excludeServerIds)
} }
return { sql: parts.join(" AND "), params } return { sql: parts.join(" AND "), params }
} }
type FilterCtxFull = FilterCtx
function emptyDto(period: ParsedPeriod): StatisticsDto { function emptyDto(period: ParsedPeriod): StatisticsDto {
return { return {
from: period.fromIso, from: period.fromIso,
@@ -167,10 +269,128 @@ function toBreakdown(
})) }))
} }
interface UserBindTuple {
userId: string
serverId: number
iface: string
}
async function loadBindUserTuples(): Promise<UserBindTuple[]> {
const binds = await db.select().from(userInterfaceBindings)
const seen = new Set<string>()
const out: UserBindTuple[] = []
for (const b of binds) {
for (const iface of factIfaceAliases(b.interfaceName, b.serverId)) {
const k = `${b.userId}\0${b.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ userId: b.userId, serverId: b.serverId, iface })
}
}
return out
}
function uniqueBoundIfaces(tuples: UserBindTuple[]): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const t of tuples) {
const k = `${t.serverId}\0${t.iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: t.serverId, iface: t.iface })
}
return out
}
async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> { async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> {
if (!userId) return null if (!userId || userId === STATISTICS_UNBOUND_USER_ID) return null
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId)) const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
return binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })) return expandBindingIfaces(binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })))
}
function userBindJoinSql(tuples: UserBindTuple[]): { sql: string; params: unknown[] } {
const values = tuples.map(() => "(?::text, ?::int, ?::text)").join(", ")
const params = tuples.flatMap((t) => [t.userId, t.serverId, t.iface])
return {
sql: `JOIN (VALUES ${values}) AS b(user_id, server_id, iface) ON b.server_id = f.server_id AND b.iface = f.iface`,
params,
}
}
function expandIfaceTuples(
items: Array<{ serverId: number; iface: string }>,
): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const t of items) {
for (const iface of factIfaceAliases(t.iface, t.serverId)) {
const k = `${t.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: t.serverId, iface })
}
}
return out
}
async function loadPayloadScope(serverId?: number): Promise<{
overlayIfaces: Array<{ serverId: number; iface: string }>
wanIfaces: Array<{ serverId: number; iface: string }>
excludeServerIds: number[]
topo: FlowTopology
}> {
const topo = await loadFlowTopology()
const catalog = await getServerCatalog()
await warmIfaceCache(catalog.list.map((s) => s.id))
const overlayRaw: Array<{ serverId: number; iface: string }> = []
const wanRaw: Array<{ serverId: number; iface: string }> = []
for (const s of catalog.list) {
if (serverId != null && s.id !== serverId) continue
const wanSet = topo.wanIfaces.get(s.id)
const wanNames = wanSet && wanSet.size > 0
? [...wanSet]
: s.type === "home-router" ? [] : ["ether1"]
for (const name of wanNames) wanRaw.push({ serverId: s.id, iface: name })
const names = new Set(listCachedIfaceNames(s.id))
for (const name of topo.tunnelIfaces?.get(s.id) ?? []) names.add(name)
for (const name of names) {
if (isOverlayTunnelIface(topo, s.id, name)) overlayRaw.push({ serverId: s.id, iface: name })
}
}
return {
overlayIfaces: expandIfaceTuples(overlayRaw),
wanIfaces: expandIfaceTuples(wanRaw),
excludeServerIds: serverId != null
? []
: catalog.list.filter((s) => s.type === "exit-node").map((s) => s.id),
topo,
}
}
async function buildFilterCtx(query: StatisticsQuery, period: ParsedPeriod): Promise<FilterCtx | null> {
const bindTuples = await loadBindUserTuples()
const boundIfaces = uniqueBoundIfaces(bindTuples)
const unboundOnly = query.userId === STATISTICS_UNBOUND_USER_ID
const userIfaces = unboundOnly ? null : await resolveUserIfaces(query.userId)
if (userIfaces && userIfaces.length === 0) return null
if (unboundOnly) return null
const scope = await loadPayloadScope(query.serverId)
return {
...period,
serverId: query.serverId,
iface: query.iface,
country: query.country,
service: query.service,
asn: query.asn,
planes: query.planes ?? "unique",
userIfaces,
unboundOnly,
boundIfaces,
overlayIfaces: scope.overlayIfaces,
wanIfaces: scope.wanIfaces,
excludeServerIds: scope.excludeServerIds,
topo: scope.topo,
}
} }
export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> { export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> {
@@ -184,28 +404,21 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
windowSec: 1, windowSec: 1,
}) })
const userIfaces = await resolveUserIfaces(query.userId) await warmBindingIfaceCache()
const ctx: FilterCtxFull = { if (query.serverId) await warmIfaceCache([query.serverId])
...period, const bindTuples = await loadBindUserTuples()
serverId: query.serverId, const ctx = await buildFilterCtx(query, period)
iface: query.iface, if (!ctx) return emptyDto(period)
country: query.country,
service: query.service,
asn: query.asn,
userIfaces,
}
if (userIfaces && userIfaces.length === 0) return emptyDto(period)
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts" const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
const timeCol = period.grain === "hour" ? "bucket_at" : "day" const timeCol = period.grain === "hour" ? "bucket_at" : "day"
const where = factWhere("f", period.grain, ctx) const where = factWhere("f", period.grain, ctx)
const totals = await dbAll<{ bytes: number; packets: number; servers: number; ifaces: number }>(` const totals = await dbAll<{ bytes: number; packets: number; servers: number }>(`
SELECT SELECT
COALESCE(SUM(f.bytes), 0) AS bytes, COALESCE(SUM(f.bytes), 0) AS bytes,
COALESCE(SUM(f.packets), 0) AS packets, COALESCE(SUM(f.packets), 0) AS packets,
COUNT(DISTINCT f.server_id)::int AS servers, COUNT(DISTINCT f.server_id)::int AS servers
COUNT(DISTINCT (f.server_id::text || ':' || f.iface))::int AS ifaces
FROM ${table} f FROM ${table} f
WHERE ${where.sql} WHERE ${where.sql}
`, where.params) `, where.params)
@@ -213,7 +426,6 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
const bytes = Number(totals[0]?.bytes) || 0 const bytes = Number(totals[0]?.bytes) || 0
const packets = Number(totals[0]?.packets) || 0 const packets = Number(totals[0]?.packets) || 0
const serverCount = Number(totals[0]?.servers) || 0 const serverCount = Number(totals[0]?.servers) || 0
const ifaceCount = Number(totals[0]?.ifaces) || 0
const seriesRows = await dbAll<{ t: string; bytes: number }>(` const seriesRows = await dbAll<{ t: string; bytes: number }>(`
SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes
@@ -251,21 +463,72 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
GROUP BY f.server_id GROUP BY f.server_id
`, where.params) `, where.params)
const ifaceRows = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(` const ifaceRowsRaw = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f FROM ${table} f
WHERE ${where.sql} WHERE ${where.sql}
GROUP BY f.server_id, f.iface GROUP BY f.server_id, f.iface
`, where.params) `, where.params)
await warmIfaceCache(ifaceRowsRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId))
const ifaceRows = collapseServerIfaceRows(ifaceRowsRaw).filter((r) => {
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) return false
if (ctx.iface) return true
if (ctx.topo && isOverlayTunnelIface(ctx.topo, r.serverId, r.iface)) return false
if (ctx.topo && isWanFactIface(ctx.topo, r.serverId, r.iface)) return false
return true
})
const ifaceCount = ifaceRows.length
const userRows = await dbAll<{ id: string; bytes: number; packets: number }>(` let dupeIfaceRows: Array<{ serverId: number; iface: string; bytes: number; packets: number; kind: "wan" | "overlay" }> = []
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets if (ctx.planes === "all" && !ctx.iface) {
FROM ${table} f const wanWhere = factWhere("f", period.grain, ctx, "wan")
JOIN user_interface_bindings b const overlayWhere = factWhere("f", period.grain, ctx, "overlay")
ON b.server_id = f.server_id AND b.interface_name = f.iface const [wanRaw, overlayRaw] = await Promise.all([
WHERE ${where.sql} dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
GROUP BY b.user_id SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
`, where.params) FROM ${table} f
WHERE ${wanWhere.sql}
GROUP BY f.server_id, f.iface
`, wanWhere.params),
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
WHERE ${overlayWhere.sql}
GROUP BY f.server_id, f.iface
`, overlayWhere.params),
])
await warmIfaceCache([
...wanRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
...overlayRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
])
const seen = new Set(ifaceRows.map((r) => `${r.serverId}:${r.iface}`))
for (const r of collapseServerIfaceRows(wanRaw)) {
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
const key = `${r.serverId}:${r.iface}`
if (seen.has(key)) continue
seen.add(key)
dupeIfaceRows.push({ ...r, kind: "wan" })
}
for (const r of collapseServerIfaceRows(overlayRaw)) {
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
const key = `${r.serverId}:${r.iface}`
if (seen.has(key)) continue
seen.add(key)
dupeIfaceRows.push({ ...r, kind: "overlay" })
}
}
let userRows: Array<{ id: string; bytes: number; packets: number }> = []
if (bindTuples.length && !ctx.unboundOnly) {
const join = userBindJoinSql(bindTuples)
userRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
${join.sql}
WHERE ${where.sql}
GROUP BY b.user_id
`, [...join.params, ...where.params])
}
const serverNames = new Map<number, string>() const serverNames = new Map<number, string>()
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers) const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
@@ -323,17 +586,35 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
bytes, bytes,
period.windowSec, period.windowSec,
) )
const interfaces = toBreakdown( const uniqueInterfaces = toBreakdown(
ifaceRows.map((r) => ({ ifaceRows.map((r) => {
id: `${r.serverId}:${r.iface}`, const serverName = serverNames.get(r.serverId) || String(r.serverId)
label: `${serverNames.get(r.serverId) || r.serverId} · ${r.iface}`, const wan = ctx.topo ? isWanFactIface(ctx.topo, r.serverId, r.iface) : false
bytes: Number(r.bytes) || 0, return {
packets: Number(r.packets) || 0, id: `${r.serverId}:${r.iface}`,
})), label: wan ? wanIfaceLabel(serverName, r.iface) : `${serverName} · ${r.iface}`,
bytes: Number(r.bytes) || 0,
packets: Number(r.packets) || 0,
}
}),
bytes, bytes,
period.windowSec, period.windowSec,
) )
const users = toBreakdown( const dupeInterfaces: StatisticsBreakdownRow[] = dupeIfaceRows.map((r) => {
const serverName = serverNames.get(r.serverId) || String(r.serverId)
const rowBytes = Number(r.bytes) || 0
const rowPackets = Number(r.packets) || 0
return {
id: `${r.serverId}:${r.iface}`,
label: r.kind === "wan" ? wanIfaceLabel(serverName, r.iface) : overlayDupLabel(serverName, r.iface),
bytes: rowBytes,
packets: rowPackets,
bps: (rowBytes * 8) / period.windowSec,
percent: 0,
}
})
const interfaces = [...uniqueInterfaces, ...dupeInterfaces]
const matchedUsers = toBreakdown(
userRows.map((r) => ({ userRows.map((r) => ({
id: r.id, id: r.id,
label: userNames.get(r.id) || r.id, label: userNames.get(r.id) || r.id,
@@ -344,6 +625,8 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
period.windowSec, period.windowSec,
) )
const users = [...matchedUsers]
return { return {
from: period.fromIso, from: period.fromIso,
to: period.toIso, to: period.toIso,
@@ -352,7 +635,7 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
bytes, bytes,
packets, packets,
avgBps: (bytes * 8) / period.windowSec, avgBps: (bytes * 8) / period.windowSec,
users: users.length, users: matchedUsers.length,
servers: serverCount, servers: serverCount,
ifaces: ifaceCount, ifaces: ifaceCount,
topCountry: countries[0]?.label || "—", topCountry: countries[0]?.label || "—",
@@ -367,3 +650,236 @@ export async function getStatistics(query: StatisticsQuery): Promise<StatisticsD
asns, asns,
} }
} }
function dimSql(dim: StatisticsPivotDim, factAlias: string, bindAlias: string): string {
if (dim === "country") return `${factAlias}.country`
if (dim === "service") return `${factAlias}.service`
if (dim === "asn") return `${factAlias}.asn::text`
if (dim === "server") return `${factAlias}.server_id::text`
if (dim === "iface") return `(${factAlias}.server_id::text || ':' || ${factAlias}.iface)`
return `${bindAlias}.user_id`
}
function emptyPivot(query: StatisticsPivotQuery): StatisticsPivotDto {
return {
rowDim: query.row,
colDim: query.col,
metric: query.metric,
columns: [],
rows: [],
otherBytes: 0,
}
}
export function pivotDimsConflict(row: StatisticsPivotDim, col: StatisticsPivotDim): boolean {
return row === col
}
export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<StatisticsPivotDto> {
if (pivotDimsConflict(query.row, query.col)) return emptyPivot(query)
const period = parseStatisticsPeriod(query.from, query.to)
if (!period) return emptyPivot(query)
await warmBindingIfaceCache()
if (query.serverId) await warmIfaceCache([query.serverId])
const bindTuples = await loadBindUserTuples()
const ctx = await buildFilterCtx(query, period)
if (!ctx) return emptyPivot(query)
const needsUser = query.row === "user" || query.col === "user"
if (needsUser && bindTuples.length === 0) return emptyPivot(query)
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
const where = factWhere("f", period.grain, ctx)
const rowExpr = dimSql(query.row, "f", "b")
const colExpr = dimSql(query.col, "f", "b")
const join = needsUser ? userBindJoinSql(bindTuples) : { sql: "", params: [] as unknown[] }
const raw = await dbAll<{ row_id: string; col_id: string; bytes: number; packets: number }>(`
SELECT ${rowExpr} AS row_id, ${colExpr} AS col_id,
SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
FROM ${table} f
${join.sql}
WHERE ${where.sql}
GROUP BY 1, 2
`, [...join.params, ...where.params])
if (query.row === "iface" || query.col === "iface") {
const ifaceServerIds: number[] = []
for (const r of raw) {
for (const dim of [query.row, query.col] as const) {
if (dim !== "iface") continue
const id = dim === query.row ? String(r.row_id ?? "") : String(r.col_id ?? "")
const colon = id.indexOf(":")
if (colon < 0) continue
const sid = Number(id.slice(0, colon))
if (looksLikeIfIndex(id.slice(colon + 1)) && Number.isFinite(sid)) ifaceServerIds.push(sid)
}
}
await warmIfaceCache(ifaceServerIds)
for (const r of raw) {
if (query.row === "iface") r.row_id = canonicalIfaceDimId(String(r.row_id ?? ""))
if (query.col === "iface") r.col_id = canonicalIfaceDimId(String(r.col_id ?? ""))
}
}
const metric = query.metric
type Acc = { bytes: number; packets: number }
const cell = new Map<string, Map<string, Acc>>()
const colTotals = new Map<string, number>()
for (const r of raw) {
const rid = String(r.row_id ?? "")
const cid = String(r.col_id ?? "")
const acc: Acc = { bytes: Number(r.bytes) || 0, packets: Number(r.packets) || 0 }
const val = metric === "packets" ? acc.packets : acc.bytes
let rowMap = cell.get(rid)
if (!rowMap) {
rowMap = new Map()
cell.set(rid, rowMap)
}
const prev = rowMap.get(cid)
if (prev) {
prev.bytes += acc.bytes
prev.packets += acc.packets
} else {
rowMap.set(cid, acc)
}
colTotals.set(cid, (colTotals.get(cid) ?? 0) + val)
}
const topCols = [...colTotals.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, PIVOT_COL_CAP)
.map(([id]) => id)
const topColSet = new Set(topCols)
const folded = new Map<string, Map<string, number>>()
const foldedColTotals = new Map<string, number>()
let otherBytes = 0
for (const [rid, cols] of cell) {
const rowMap = new Map<string, number>()
for (const [cid, acc] of cols) {
const val = metric === "packets" ? acc.packets : acc.bytes
const dest = topColSet.has(cid) ? cid : PIVOT_OTHER_ID
if (dest === PIVOT_OTHER_ID) otherBytes += val
rowMap.set(dest, (rowMap.get(dest) ?? 0) + val)
foldedColTotals.set(dest, (foldedColTotals.get(dest) ?? 0) + val)
}
folded.set(rid, rowMap)
}
const rowTotals = new Map<string, number>()
for (const [rid, cols] of folded) {
let t = 0
for (const v of cols.values()) t += v
rowTotals.set(rid, t)
}
const topRows = [...rowTotals.entries()]
.sort((a, b) => b[1] - a[1])
.slice(0, PIVOT_ROW_CAP)
.map(([id]) => id)
const topRowSet = new Set(topRows)
const finalRows = new Map<string, Map<string, number>>()
const finalRowTotals = new Map<string, number>()
for (const [rid, cols] of folded) {
const dest = topRowSet.has(rid) ? rid : PIVOT_OTHER_ID
if (dest === PIVOT_OTHER_ID) {
for (const [cid, v] of cols) {
if (cid !== PIVOT_OTHER_ID) otherBytes += v
}
}
let rowMap = finalRows.get(dest)
if (!rowMap) {
rowMap = new Map()
finalRows.set(dest, rowMap)
}
for (const [cid, v] of cols) {
rowMap.set(cid, (rowMap.get(cid) ?? 0) + v)
}
}
for (const [rid, cols] of finalRows) {
let t = 0
for (const v of cols.values()) t += v
finalRowTotals.set(rid, t)
}
const colIds = [...topCols]
if (foldedColTotals.has(PIVOT_OTHER_ID)) colIds.push(PIVOT_OTHER_ID)
const rowIds = [...topRows]
if (finalRows.has(PIVOT_OTHER_ID) && !topRowSet.has(PIVOT_OTHER_ID)) rowIds.push(PIVOT_OTHER_ID)
const labels = await loadPivotLabels(query.row, query.col, rowIds, colIds)
return {
rowDim: query.row,
colDim: query.col,
metric,
columns: colIds.map((id) => ({
id,
label: labels.col.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
total: foldedColTotals.get(id) ?? 0,
})),
rows: rowIds.map((id) => {
const cols = finalRows.get(id) ?? new Map()
const cells: Record<string, number> = {}
for (const cid of colIds) cells[cid] = cols.get(cid) ?? 0
return {
id,
label: labels.row.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
total: finalRowTotals.get(id) ?? 0,
cells,
}
}),
otherBytes,
}
}
async function loadPivotLabels(
rowDim: StatisticsPivotDim,
colDim: StatisticsPivotDim,
rowIds: string[],
colIds: string[],
): Promise<{ row: Map<string, string>; col: Map<string, string> }> {
const topo = await loadFlowTopology()
const serverNames = new Map<string, string>()
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
for (const s of allServers) serverNames.set(String(s.id), s.name || s.host)
const userNames = new Map<string, string>()
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
const asnHolders = new Map<string, string>()
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
for (const a of asnMeta) asnHolders.set(String(a.asn), a.holder)
function label(dim: StatisticsPivotDim, id: string): string {
if (id === PIVOT_OTHER_ID) return "Прочие"
if (dim === "country") return id === "XX" ? "Неизвестно" : id
if (dim === "server") return serverNames.get(id) || id
if (dim === "user") return userNames.get(id) || id
if (dim === "asn") {
if (id === "0") return "other"
const holder = asnHolders.get(id)
return holder ? `AS${id} · ${holder}` : `AS${id}`
}
if (dim === "iface") {
const colon = id.indexOf(":")
if (colon < 0) return id
const sid = id.slice(0, colon)
const iface = id.slice(colon + 1)
const sidNum = Number(sid)
const name = Number.isFinite(sidNum) ? displayFactIface(sidNum, iface) : iface
const serverName = serverNames.get(sid) || sid
if (Number.isFinite(sidNum) && isWanFactIface(topo, sidNum, name)) {
return wanIfaceLabel(serverName, name)
}
if (Number.isFinite(sidNum) && isOverlayTunnelIface(topo, sidNum, name)) {
return overlayDupLabel(serverName, name)
}
return `${serverName} · ${name}`
}
return id
}
const row = new Map<string, string>()
const col = new Map<string, string>()
for (const id of rowIds) row.set(id, label(rowDim, id))
for (const id of colIds) col.set(id, label(colDim, id))
return { row, col }
}
+16 -9
View File
@@ -27,11 +27,9 @@ import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-setting
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js" import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js" import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
import { enqueueRipeMisses } from "./traffic-flow-ripe.js" import { enqueueRipeMisses } from "./traffic-flow-ripe.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { classifyFlowDst, refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
import { isIsoCountry } from "./traffic-flow-brands.js"
import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js" import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js"
import { pickInternetPeer } from "./traffic-flow-ip.js" import { resolveInternetDest } from "./traffic-flow-dest.js"
import { refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
import { import {
enGreIfaceNames, enGreIfaceNames,
getServerCatalog, getServerCatalog,
@@ -253,11 +251,20 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
totalPackets += r.packets totalPackets += r.packets
srcs.add(r.src) srcs.add(r.src)
dsts.add(r.dst) dsts.add(r.dst)
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort) const destMeta = resolveInternetDest({
peers.add(peer) src: r.src,
dst: r.dst,
proto: r.proto,
srcPort: r.srcPort,
dstPort: r.dstPort,
serverId: r.serverId,
inIface: resolved.name,
topo,
})
if (destMeta.dest) peers.add(destMeta.dest)
const app = applicationName(r.proto, r.dstPort, r.srcPort) const app = applicationName(r.proto, r.dstPort, r.srcPort)
const ripe = resolveFlowIp(peer) const ripe = destMeta.ripe
const classified = classifyFlowDst(peer, r.proto, r.dstPort, r.srcPort, ripe) const classified = destMeta.classified
bump(applications, app, r.bytes, r.packets) bump(applications, app, r.bytes, r.packets)
bump(protocols, protoName(r.proto), r.bytes, r.packets) bump(protocols, protoName(r.proto), r.bytes, r.packets)
bump(sources, r.src, r.bytes, r.packets) bump(sources, r.src, r.bytes, r.packets)
@@ -269,7 +276,7 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
const asnLabel = ripe.holder ? `AS${ripe.asn} ${ripe.holder}` : `AS${ripe.asn}` const asnLabel = ripe.holder ? `AS${ripe.asn} ${ripe.holder}` : `AS${ripe.asn}`
bump(asns, asnId, r.bytes, r.packets, asnLabel) bump(asns, asnId, r.bytes, r.packets, asnLabel)
} }
const dstCountry = ripe?.ok && isIsoCountry(ripe.country) ? ripe.country : "" const dstCountry = destMeta.country && destMeta.country !== "unknown" ? destMeta.country : ""
if (dstCountry) { if (dstCountry) {
bump(countries, dstCountry, r.bytes, r.packets) bump(countries, dstCountry, r.bytes, r.packets)
} }
@@ -1,5 +1,10 @@
import assert from "node:assert/strict" import assert from "node:assert/strict"
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js" import {
dedupFlowRowsAcrossExporters,
dedupFlowRowsMaxBytes,
flowConversationKey,
flowTupleKey,
} from "./traffic-flow-dedup.js"
const a = { const a = {
serverId: 7, serverId: 7,
@@ -22,4 +27,13 @@ assert.equal(flowTupleKey(a), flowTupleKey(b))
const sameIface = dedupFlowRowsMaxBytes([a, { ...a, bytes: 3_000, packets: 2 }]) const sameIface = dedupFlowRowsMaxBytes([a, { ...a, bytes: 3_000, packets: 2 }])
assert.equal(sameIface[0]?.bytes, 15_000) assert.equal(sameIface[0]?.bytes, 15_000)
const jh = { ...a, serverId: 7, bytes: 9_000 }
const en = { ...a, serverId: 9, bytes: 11_000, inIface: "1" }
assert.equal(flowConversationKey(jh), flowConversationKey(en))
assert.notEqual(flowTupleKey(jh), flowTupleKey(en))
const across = dedupFlowRowsAcrossExporters([en, jh], (x, y) => (x.serverId === 7 ? x : y))
assert.equal(across.length, 1)
assert.equal(across[0]?.serverId, 7)
assert.equal(across[0]?.bytes, 9_000)
console.log("traffic-flow-dedup.test.ts: ok") console.log("traffic-flow-dedup.test.ts: ok")
@@ -14,6 +14,32 @@ export function flowTupleKey(r: Pick<FlowTupleRow, "serverId" | "src" | "dst" |
return `${r.serverId}|${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}` return `${r.serverId}|${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
} }
/** Один разговор на всех экспортёрах (JH+EN), без serverId. */
export function flowConversationKey(r: Pick<FlowTupleRow, "src" | "dst" | "proto" | "srcPort" | "dstPort">): string {
return `${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
}
/**
* Схлопнуть копии одного 5-tuple с разных серверов.
* `prefer` выбирает ряд (клиент на JH важнее голого EN).
*/
export function dedupFlowRowsAcrossExporters<T extends FlowTupleRow>(
rows: T[],
prefer: (a: T, b: T) => T,
): T[] {
const byConv = new Map<string, T>()
for (const row of rows) {
const key = flowConversationKey(row)
const prev = byConv.get(key)
if (!prev) {
byConv.set(key, row)
continue
}
byConv.set(key, prefer(prev, row))
}
return [...byConv.values()]
}
function ifaceKey(r: FlowTupleRow): string { function ifaceKey(r: FlowTupleRow): string {
return `${flowTupleKey(r)}|${r.inIface}` return `${flowTupleKey(r)}|${r.inIface}`
} }
@@ -0,0 +1,109 @@
import assert from "node:assert/strict"
import {
ingestParsedFlowsForServerForTests,
resetEngineForTests,
} from "./traffic-flow-engine.js"
import { factsSnapshotForTests } from "./traffic-flow-facts.js"
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
import {
disableRipeEnqueueForTests,
disableRipePersistForTests,
resetRipeCacheForTests,
seedRipeCacheForTests,
} from "./traffic-flow-ripe.js"
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
disableCatalogFetchForTests()
resetFlowCatalogForTests()
disableRipePersistForTests()
disableRipeEnqueueForTests()
resetRipeCacheForTests()
resetEngineForTests()
resetIfaceCacheForTests()
seedRipeCacheForTests({
prefix: "8.8.8.0/24",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
seedRipeCacheForTests({
prefix: "95.167.0.0/16",
asn: 12389,
country: "RU",
lat: null,
lng: null,
holder: "ROSTELECOM-AS",
ok: true,
fetchedAt: Date.now(),
})
const topo: FlowTopology = {
clientIfaces: new Map([[1, new Set(["gre-client"])]]),
clientByIface: new Map([["1|gre-client", {
userId: "u-rost",
login: "alice",
name: "Alice",
serverId: 1,
interfaceName: "gre-client",
}]]),
enNodes: [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
enHosts: new Set(["198.51.100.1"]),
jhHosts: new Set(["203.0.113.10"]),
wanIfaces: new Map([[1, new Set(["ether1"])]]),
plane: {
clientIfaceNames: new Set(["gre-client"]),
enHosts: new Set(["198.51.100.1"]),
jhHosts: new Set(["203.0.113.10"]),
},
}
seedFlowTopologyForTests(topo)
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
ingestParsedFlowsForServerForTests(1, [
{
src: "95.167.1.10",
dst: "10.200.100.53",
proto: 6,
srcPort: 51234,
dstPort: 443,
bytes: 100,
packets: 2,
inIface: "gre-client",
outIface: "ether1",
},
{
src: "95.167.1.10",
dst: "8.8.8.8",
proto: 6,
srcPort: 51234,
dstPort: 443,
bytes: 50,
packets: 1,
inIface: "gre-client",
outIface: "ether1",
},
])
const facts = factsSnapshotForTests()
const total = facts.reduce((s, r) => s + r.bytes, 0)
const asnBytes = facts.reduce((s, r) => s + r.bytes, 0)
assert.equal(total, 150)
assert.equal(asnBytes, 150, "unique bytes = SUM dest ASN")
assert.equal(facts.some((r) => r.asn === 12389), false, "ASN клиента не в кубе")
const google = facts.find((r) => r.asn === 15169)
assert.ok(google)
assert.equal(google.bytes, 50)
const other = facts.filter((r) => r.asn === 0).reduce((s, r) => s + r.bytes, 0)
assert.equal(other, 100)
resetEngineForTests()
seedFlowTopologyForTests(null)
resetRipeCacheForTests()
resetIfaceCacheForTests()
console.log("traffic-flow-dest.test.ts: ok")
+60
View File
@@ -0,0 +1,60 @@
import { isIsoCountry } from "./traffic-flow-brands.js"
import { classifyFlowDst, type FlowClassification } from "./traffic-flow-classify.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { pickInternetDest, type InternetDestCtx } from "./traffic-flow-ip.js"
import type { FlowIpMeta } from "./traffic-flow-ripe.js"
import {
flowOursHosts,
resolveClient,
type FlowTopology,
} from "./traffic-flow-topology.js"
export interface InternetDestMeta {
dest: string
ripe: FlowIpMeta | null
classified: FlowClassification
country: string
asn: number
}
export function destCtxForIface(
topo: FlowTopology | null | undefined,
serverId: number,
inIface: string,
): InternetDestCtx {
const name = canonicalFactIface(serverId, inIface) || String(inIface ?? "").trim()
return {
ours: flowOursHosts(topo),
boundClient: Boolean(topo && name && resolveClient(topo, serverId, name)),
}
}
export function resolveInternetDest(opts: {
src: string
dst: string
proto: number
srcPort: number
dstPort: number
serverId: number
inIface: string
topo?: FlowTopology | null
}): InternetDestMeta {
const dest = pickInternetDest(
opts.src,
opts.dst,
opts.srcPort,
opts.dstPort,
destCtxForIface(opts.topo, opts.serverId, opts.inIface),
)
const ripe = dest ? resolveFlowIp(dest) : null
const classified = classifyFlowDst(dest || opts.dst, opts.proto, opts.dstPort, opts.srcPort, ripe)
if (!dest) {
return { dest: "", ripe: null, classified, country: "", asn: 0 }
}
const country = ripe?.ok && isIsoCountry(ripe.country)
? ripe.country
: (ripe?.ok ? "" : "unknown")
const asn = ripe?.ok && ripe.asn ? ripe.asn : 0
return { dest, ripe, classified, country, asn }
}
+65 -28
View File
@@ -4,13 +4,18 @@ import { normalizeParsedFlow, parseFlowPacket, protoName, type ParsedFlow, type
import { classifyFlowPlaneLite } from "./traffic-flow-planes.js" import { classifyFlowPlaneLite } from "./traffic-flow-planes.js"
import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js" import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js"
import { applicationName } from "./traffic-flow-apps.js" import { applicationName } from "./traffic-flow-apps.js"
import { classifyFlowDst } from "./traffic-flow-classify.js"
import { enqueueRipeMisses, pruneRipeSqlite } from "./traffic-flow-ripe.js" import { enqueueRipeMisses, pruneRipeSqlite } from "./traffic-flow-ripe.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js" import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
import { isIsoCountry } from "./traffic-flow-brands.js"
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js" import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
import { pickInternetPeer } from "./traffic-flow-ip.js" import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
import { resolveInternetDest } from "./traffic-flow-dest.js"
import {
getServerCatalog,
loadFlowTopology,
peekFlowTopology,
peekServerCatalog,
} from "./traffic-flow-topology.js"
import { import {
bumpFlowFact, bumpFlowFact,
factsPendingSize, factsPendingSize,
@@ -337,19 +342,31 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
const bucketAt = minuteBucketIso() const bucketAt = minuteBucketIso()
const hourAt = hourBucketIso() const hourAt = hourBucketIso()
const ripeMisses: string[] = [] const ripeMisses: string[] = []
const topo = peekFlowTopology()
const catalog = peekServerCatalog()
if (!topo) void loadFlowTopology().catch(() => {})
if (!catalog) void getServerCatalog().catch(() => {})
const serverType = catalog?.byId.get(serverId)?.type
for (const raw of flows) { for (const raw of flows) {
const flow = normalizeParsedFlow(raw) const flow = normalizeParsedFlow(raw)
addToTick(serverId, flow, flow.bytes) addToTick(serverId, flow, flow.bytes)
bumpRollup(serverId, bucketAt, flow, flow.bytes, flow.packets) bumpRollup(serverId, bucketAt, flow, flow.bytes, flow.packets)
const peer = pickInternetPeer(flow.src, flow.dst, flow.srcPort, flow.dstPort) const destMeta = resolveInternetDest({
const ripe = resolveFlowIp(peer) src: flow.src,
if (peer && !ripe) ripeMisses.push(peer) dst: flow.dst,
const classified = classifyFlowDst(peer, flow.proto, flow.dstPort, flow.srcPort, ripe) proto: flow.proto,
srcPort: flow.srcPort,
dstPort: flow.dstPort,
serverId,
inIface: flow.inIface,
topo,
})
const ripe = destMeta.ripe
if (destMeta.dest && !ripe) ripeMisses.push(destMeta.dest)
const classified = destMeta.classified
const app = applicationName(flow.proto, flow.dstPort, flow.srcPort) const app = applicationName(flow.proto, flow.dstPort, flow.srcPort)
const country = ripe?.ok && isIsoCountry(ripe.country) const country = destMeta.country
? ripe.country const asnKey = destMeta.asn ? String(destMeta.asn) : "unknown"
: (ripe?.ok ? "" : "unknown")
const asnKey = ripe?.ok && ripe.asn ? String(ripe.asn) : "unknown"
bumpDim(serverId, bucketAt, "proto", protoName(flow.proto), flow.bytes, flow.packets) bumpDim(serverId, bucketAt, "proto", protoName(flow.proto), flow.bytes, flow.packets)
bumpDim(serverId, bucketAt, "app", app, flow.bytes, flow.packets) bumpDim(serverId, bucketAt, "app", app, flow.bytes, flow.packets)
bumpDim(serverId, bucketAt, "iface", flow.inIface || "__unknown__", flow.bytes, flow.packets) bumpDim(serverId, bucketAt, "iface", flow.inIface || "__unknown__", flow.bytes, flow.packets)
@@ -357,16 +374,29 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
bumpDim(serverId, bucketAt, "service", classified.service, flow.bytes, flow.packets) bumpDim(serverId, bucketAt, "service", classified.service, flow.bytes, flow.packets)
if (country) bumpDim(serverId, bucketAt, "country", country, flow.bytes, flow.packets) if (country) bumpDim(serverId, bucketAt, "country", country, flow.bytes, flow.packets)
bumpDim(serverId, bucketAt, "asn", asnKey, flow.bytes, flow.packets) bumpDim(serverId, bucketAt, "asn", asnKey, flow.bytes, flow.packets)
bumpFlowFact({ if (shouldWriteFlowFact({
serverId, serverId,
bucketAt: hourAt, serverType,
iface: flow.inIface, inIface: flow.inIface,
country: country || "XX", outIface: flow.outIface,
service: classified.service, proto: flow.proto,
asn: ripe?.ok && ripe.asn ? ripe.asn : 0, srcPort: flow.srcPort,
bytes: flow.bytes, dstPort: flow.dstPort,
packets: flow.packets, src: flow.src,
}) dst: flow.dst,
topo,
})) {
bumpFlowFact({
serverId,
bucketAt: hourAt,
iface: canonicalFactIface(serverId, flow.inIface),
country: country || "XX",
service: classified.service,
asn: destMeta.asn,
bytes: flow.bytes,
packets: flow.packets,
})
}
const key = pendingKey(serverId, bucketAt, flow) const key = pendingKey(serverId, bucketAt, flow)
const prev = pending.get(key) const prev = pending.get(key)
@@ -841,10 +871,11 @@ async function upsertFlowBuckets(rows: PendingFlowRow[]): Promise<number> {
} }
} }
export async function flushPending(opts?: { force?: boolean }): Promise<void> { export async function flushPending(opts?: { force?: boolean; prune?: boolean }): Promise<void> {
pruneRecent() pruneRecent()
rollFlowRings() rollFlowRings()
const force = Boolean(opts?.force) const force = Boolean(opts?.force)
const doPrune = opts?.prune !== false
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0 || factsPendingSize() > 0 const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0 || factsPendingSize() > 0
const due = persistDue(force, hasWork) const due = persistDue(force, hasWork)
try { try {
@@ -854,7 +885,7 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
} }
if (!hasWork) { if (!hasWork) {
if (force) { if (force && doPrune) {
try { try {
await pruneStored() await pruneStored()
} catch { } catch {
@@ -908,10 +939,12 @@ export async function flushPending(opts?: { force?: boolean }): Promise<void> {
} catch { } catch {
/* statistics cube best-effort */ /* statistics cube best-effort */
} }
try { if (doPrune) {
await pruneStored() try {
} catch { await pruneStored()
/* prune best-effort */ } catch {
/* prune best-effort */
}
} }
} }
@@ -919,8 +952,12 @@ export function lastFlushUsedTransactionForTests(): boolean {
return lastFlushUsedTransaction return lastFlushUsedTransaction
} }
export async function flushEngineNow(opts?: { prune?: boolean }): Promise<void> {
await flushPending({ force: true, prune: opts?.prune })
}
export async function flushPendingForTests(): Promise<void> { export async function flushPendingForTests(): Promise<void> {
await flushPending({ force: true }) await flushEngineNow()
} }
export function onEngineTick(): void { export function onEngineTick(): void {
@@ -0,0 +1,150 @@
import assert from "node:assert/strict"
import {
isJunkFactIface,
isOverlayGreIface,
isOverlayTunnelIface,
isWanFactIface,
shouldWriteFlowFact,
} from "./traffic-flow-facts-filter.js"
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
function topo(partial: Partial<FlowTopology> = {}): FlowTopology {
const wanIfaces = partial.wanIfaces ?? new Map([[1, new Set(["ether1"])]])
const clientIfaces = partial.clientIfaces ?? new Map([[1, new Set(["gre-client"])]])
const clientByIface = partial.clientByIface ?? new Map()
const enHosts = partial.enHosts ?? new Set(["198.51.100.1"])
const jhHosts = partial.jhHosts ?? new Set(["203.0.113.10"])
return {
clientIfaces,
clientByIface,
enNodes: partial.enNodes ?? [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
enHosts,
jhHosts,
wanIfaces,
tunnelIfaces: partial.tunnelIfaces,
plane: partial.plane ?? {
clientIfaceNames: new Set(["gre-client"]),
enHosts,
jhHosts,
},
}
}
resetIfaceCacheForTests()
rememberServerIfaces(1, [{ name: "ether1", ifindex: "2" }])
seedFlowTopologyForTests(topo())
assert.equal(isJunkFactIface("0"), true)
assert.equal(isJunkFactIface(""), true)
assert.equal(isJunkFactIface("wg-flow"), true)
assert.equal(isJunkFactIface("ether1"), false)
assert.equal(isWanFactIface(topo(), 1, "ether1"), true)
assert.equal(isOverlayGreIface(topo(), 1, "gre-en"), true)
assert.equal(isOverlayGreIface(topo(), 1, "gre-client"), false)
assert.equal(isOverlayGreIface(topo(), 1, "ether1"), false)
const typed = topo({
clientIfaces: new Map([[1, new Set(["gre-client", "wg-server"])]]),
tunnelIfaces: new Map([[1, new Set(["gre-en", "NSK-SERVHOST-RTK", "wg-jh-en", "wg-server"])]]),
plane: {
clientIfaceNames: new Set(["gre-client", "wg-server"]),
enHosts: new Set(["198.51.100.1"]),
jhHosts: new Set(["203.0.113.10"]),
},
})
assert.equal(isOverlayTunnelIface(typed, 1, "NSK-SERVHOST-RTK"), true, "кастомное GRE overlay по type")
assert.equal(isOverlayTunnelIface(typed, 1, "wg-jh-en"), true, "WG overlay по type")
assert.equal(isOverlayTunnelIface(typed, 1, "wg-server"), false, "клиентский WG с binding")
assert.equal(isOverlayTunnelIface(typed, 1, "wg-flow"), false, "wg-flow не overlay")
assert.equal(isOverlayTunnelIface(topo(), 1, "NSK-SERVHOST-RTK"), false, "без type в снимке — не overlay")
const overlayOuter = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "ether1",
outIface: "gre-en",
proto: 47,
srcPort: 0,
dstPort: 0,
src: "203.0.113.10",
dst: "198.51.100.1",
topo: topo(),
})
assert.equal(overlayOuter, false, "overlay proto 47 на ether1 не в facts")
const payloadGre = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "gre-client",
outIface: "gre-en",
proto: 6,
srcPort: 51234,
dstPort: 443,
src: "10.100.1.17",
dst: "8.8.8.8",
topo: topo(),
})
assert.equal(payloadGre, true, "payload на GRE — да")
const payloadWan = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "ether1",
outIface: "gre-client",
proto: 6,
srcPort: 443,
dstPort: 51234,
src: "8.8.8.8",
dst: "10.100.1.17",
topo: topo(),
})
assert.equal(payloadWan, true, "payload на ether1 WAN — да")
const junkZero = shouldWriteFlowFact({
serverId: 1,
serverType: "jump-host",
inIface: "0",
proto: 6,
srcPort: 443,
dstPort: 80,
src: "1.1.1.1",
dst: "8.8.8.8",
topo: topo(),
})
assert.equal(junkZero, false)
const enTopo = topo({
clientIfaces: new Map([[2, new Set()]]),
wanIfaces: new Map([[2, new Set(["ether1"])]]),
})
const enTransit = shouldWriteFlowFact({
serverId: 2,
serverType: "exit-node",
inIface: "gre-jh",
outIface: "ether1",
proto: 6,
srcPort: 51234,
dstPort: 443,
src: "10.100.1.17",
dst: "8.8.8.8",
topo: enTopo,
})
assert.equal(enTransit, false, "EN-транзит без клиента — нет")
const enWan = shouldWriteFlowFact({
serverId: 2,
serverType: "exit-node",
inIface: "ether1",
proto: 6,
srcPort: 443,
dstPort: 80,
src: "8.8.8.8",
dst: "198.51.100.1",
topo: enTopo,
})
assert.equal(enWan, true, "WAN payload на EN — да")
seedFlowTopologyForTests(null)
resetIfaceCacheForTests()
console.log("traffic-flow-facts-filter.test.ts: ok")
@@ -0,0 +1,119 @@
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
import { STATISTICS_DUP_MARK, STATISTICS_WAN_MARK } from "@mmapp/contracts/statistics"
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { classifyFlowPlane } from "./traffic-flow-planes.js"
import { resolveClient, type FlowTopology } from "./traffic-flow-topology.js"
const JUNK_IFACE = new Set(["", "0", "—", "__unknown__", "wg-flow"])
export { STATISTICS_WAN_MARK, STATISTICS_DUP_MARK }
export function isJunkFactIface(iface: string | null | undefined): boolean {
const n = String(iface ?? "").trim()
if (JUNK_IFACE.has(n)) return true
return /^#?0$/.test(n)
}
export function isDashDisplayIface(iface: string): boolean {
return String(iface ?? "").trim() === "—"
}
export function isMgmtIface(name: string): boolean {
const n = String(name ?? "").trim().toLowerCase()
return n === "wg-flow" || n.endsWith("/wg-flow") || n.includes("wg-flow")
}
/** GRE или WG по снимку RouterOS, иначе по имени. */
export function isTunnelIfaceName(
topo: FlowTopology | null | undefined,
serverId: number,
iface: string,
): boolean {
const name = String(iface ?? "").trim()
if (!name || isJunkFactIface(name) || isMgmtIface(name)) return false
const typed = topo?.tunnelIfaces?.get(serverId)
if (typed && typed.size > 0) return typed.has(name)
const t = mapRosInterfaceType("", name)
return t === "gre" || t === "wg"
}
/** WAN uplink: `wanIfaces` топологии, иначе ether1 у JH/EN без wan_uplinks. */
export function isWanFactIface(
topo: FlowTopology | null | undefined,
serverId: number,
iface: string,
): boolean {
const name = String(iface ?? "").trim()
if (!name || isJunkFactIface(name) || isDashDisplayIface(name)) return false
const wan = topo?.wanIfaces.get(serverId)
if (wan && wan.size > 0) return wan.has(name)
return /^ether1$/i.test(name)
}
/** Overlay JH↔EN: GRE/WG не клиент, не WAN, не wg-flow. */
export function isOverlayTunnelIface(
topo: FlowTopology | null | undefined,
serverId: number,
iface: string,
): boolean {
const name = String(iface ?? "").trim()
if (!name || isWanFactIface(topo, serverId, name) || isMgmtIface(name)) return false
if (topo?.clientIfaces.get(serverId)?.has(name)) return false
return isTunnelIfaceName(topo, serverId, name)
}
/** @deprecated используйте isOverlayTunnelIface (GRE и WG). */
export function isOverlayGreIface(
topo: FlowTopology | null | undefined,
serverId: number,
iface: string,
): boolean {
return isOverlayTunnelIface(topo, serverId, iface)
}
export function shouldWriteFlowFact(opts: {
serverId: number
serverType?: string
inIface: string
outIface?: string
proto: number
srcPort: number
dstPort: number
src: string
dst: string
topo?: FlowTopology | null
}): boolean {
const inName = canonicalFactIface(opts.serverId, opts.inIface) || String(opts.inIface ?? "").trim()
if (isJunkFactIface(inName) || isJunkFactIface(opts.inIface)) return false
const outRaw = String(opts.outIface ?? "").trim()
const outName = outRaw ? (canonicalFactIface(opts.serverId, outRaw) || outRaw) : ""
const plane = classifyFlowPlane({
src: opts.src,
dst: opts.dst,
proto: opts.proto,
srcPort: opts.srcPort,
dstPort: opts.dstPort,
inIface: inName,
outIface: outName || undefined,
}, opts.topo?.plane)
if (plane !== "payload") return false
if (opts.serverType === "exit-node" && opts.topo) {
const client =
resolveClient(opts.topo, opts.serverId, inName)
?? (outName ? resolveClient(opts.topo, opts.serverId, outName) : null)
if (!client && isOverlayTunnelIface(opts.topo, opts.serverId, inName)) return false
}
return true
}
export function wanIfaceLabel(serverName: string, iface: string): string {
return `${serverName} · ${iface} · ${STATISTICS_WAN_MARK}`
}
export function overlayDupLabel(serverName: string, iface: string): string {
return `${serverName} · ${iface} · ${STATISTICS_DUP_MARK}`
}
export function isNonUniqueShareLabel(label: string): boolean {
return label.includes(STATISTICS_WAN_MARK) || label.includes(`· ${STATISTICS_DUP_MARK}`)
}
@@ -0,0 +1,138 @@
import assert from "node:assert/strict"
import { dbQuery } from "../db/index.js"
import { withPgOrSkip } from "../test/pg.js"
import { ensurePartitionFor } from "../db/partitions.js"
import { pool } from "../db/index.js"
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
import {
disableRipeEnqueueForTests,
disableRipePersistForTests,
resetRipeCacheForTests,
seedRipeCacheForTests,
} from "./traffic-flow-ripe.js"
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
import { resetEngineForTests } from "./traffic-flow-engine.js"
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
if (!(await withPgOrSkip())) {
console.log("traffic-flow-facts-rebuild.test.ts: skip")
process.exit(0)
}
const nServers = (await dbQuery<{ n: number }>(`SELECT COUNT(*)::int AS n FROM servers`)).rows[0]?.n ?? 0
if (nServers > 10) {
console.warn("traffic-flow-facts-rebuild.test.ts: skip (не пустая БД)")
process.exit(0)
}
disableCatalogFetchForTests()
resetFlowCatalogForTests()
disableRipePersistForTests()
disableRipeEnqueueForTests()
resetRipeCacheForTests()
resetEngineForTests()
resetIfaceCacheForTests()
seedRipeCacheForTests({
prefix: "8.8.8.0/24",
asn: 15169,
country: "US",
lat: null,
lng: null,
holder: "GOOGLE",
ok: true,
fetchedAt: Date.now(),
})
seedRipeCacheForTests({
prefix: "95.167.0.0/16",
asn: 12389,
country: "RU",
lat: null,
lng: null,
holder: "ROSTELECOM-AS",
ok: true,
fetchedAt: Date.now(),
})
const inserted = await dbQuery<{ id: number }>(`
INSERT INTO servers (name, host, type, wan_uplinks)
VALUES ('rebuild-facts-jh', '203.0.113.10', 'jump-host', '[{"iface":"ether1"}]'::jsonb)
RETURNING id
`)
const serverId = inserted.rows[0]?.id
if (serverId == null) throw new Error("no server")
const ts = new Date()
await ensurePartitionFor(pool, "flow_buckets", "day", ts)
await ensurePartitionFor(pool, "flow_hour_facts", "day", ts)
await ensurePartitionFor(pool, "flow_daily_facts", "month", ts)
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
await dbQuery(`
INSERT INTO app_users (id, name, login, role, active)
VALUES ('u-rebuild-1', 'Клиент', 'rebuild-user', 'viewer', TRUE)
ON CONFLICT (id) DO NOTHING
`)
await dbQuery(`
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
VALUES ('bind-rebuild-1', 'u-rebuild-1', $1, 'gre-client', 'gre')
`, [serverId])
await dbQuery(`
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
VALUES ($1, now(), 'online', $2::jsonb)
`, [serverId, JSON.stringify([{ name: "gre-client", type: "gre-tunnel" }, { name: "ether1", type: "ether" }])])
rememberServerIfaces(serverId, [{ name: "gre-client", ifindex: "2" }])
invalidateFlowCatalogCache()
const bucketAt = new Date(Date.UTC(
ts.getUTCFullYear(),
ts.getUTCMonth(),
ts.getUTCDate(),
ts.getUTCHours(),
0, 0, 0,
)).toISOString()
await dbQuery(`
INSERT INTO flow_buckets (server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface)
VALUES
($1, $2, '95.167.1.10', '10.200.100.53', 6, 51234, 443, 100, 2, 'gre-client', 'ether1'),
($1, $2, '95.167.1.10', '8.8.8.8', 6, 51234, 443, 50, 1, 'gre-client', 'ether1')
`, [serverId, bucketAt])
try {
const { rebuildFlowFactsFromBuckets } = await import("./traffic-flow-facts-rebuild.js")
const result = await rebuildFlowFactsFromBuckets()
assert.equal(result.ok, true)
assert.ok(result.buckets >= 2)
const rows = await dbQuery<{ asn: number; bytes: number }>(`
SELECT asn, SUM(bytes)::bigint AS bytes
FROM flow_hour_facts
WHERE server_id = $1
GROUP BY asn
`, [serverId])
const byAsn = new Map(rows.rows.map((r) => [Number(r.asn), Number(r.bytes)]))
const total = [...byAsn.values()].reduce((s, n) => s + n, 0)
assert.equal(total, 150)
assert.equal(byAsn.get(12389), undefined, "ASN клиента не в hour facts")
assert.equal(byAsn.get(15169), 50)
assert.equal(byAsn.get(0), 100)
} finally {
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM server_snapshots WHERE server_id = $1`, [serverId])
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
resetEngineForTests()
invalidateFlowCatalogCache()
}
console.log("traffic-flow-facts-rebuild.test.ts: ok")
@@ -0,0 +1,130 @@
import { dbAll, dbGet, dbQuery, withAdvisoryLock } from "../db/index.js"
import { flushEngineNow } from "./traffic-flow-engine.js"
import { resolveInternetDest } from "./traffic-flow-dest.js"
import {
bumpFlowFact,
discardPendingFacts,
flushFlowFacts,
hourBucketIso,
} from "./traffic-flow-facts.js"
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
import { getServerCatalog, loadFlowTopology } from "./traffic-flow-topology.js"
export const FACT_REBUILD_LOCK_KEY = 8_723_104
const BATCH = 4_000
export interface FlowFactsRebuildResult {
ok: true
buckets: number
facts: number
days: string[]
}
function hourFromBucket(raw: Date | string): string {
const iso = raw instanceof Date ? raw.toISOString() : String(raw)
const ms = new Date(iso).getTime()
return hourBucketIso(Number.isFinite(ms) ? ms : Date.now())
}
export async function rebuildFlowFactsFromBuckets(): Promise<FlowFactsRebuildResult> {
return await withAdvisoryLock(FACT_REBUILD_LOCK_KEY, async () => {
await flushEngineNow({ prune: false })
discardPendingFacts()
const days = await dbAll<{ day: string }>(`
SELECT DISTINCT (bucket_at AT TIME ZONE 'UTC')::date::text AS day
FROM flow_buckets
ORDER BY 1
`)
const dayList = days.map((r) => r.day).filter(Boolean)
if (dayList.length === 0) {
return { ok: true as const, buckets: 0, facts: 0, days: [] }
}
await dbQuery(
`DELETE FROM flow_hour_facts WHERE (bucket_at AT TIME ZONE 'UTC')::date = ANY(?::date[])`,
[dayList],
)
await dbQuery(
`DELETE FROM flow_daily_facts WHERE day = ANY(?::date[])`,
[dayList],
)
const topo = await loadFlowTopology()
const catalog = await getServerCatalog()
let offset = 0
let buckets = 0
for (;;) {
const rows = await dbAll<{
serverId: number
bucketAt: Date | string
src: string
dst: string
proto: number
srcPort: number
dstPort: number
bytes: number
packets: number
inIface: string
outIface: string
}>(`
SELECT server_id AS "serverId", bucket_at AS "bucketAt",
host(src) AS src, host(dst) AS dst, proto, src_port AS "srcPort", dst_port AS "dstPort",
bytes, packets, in_iface AS "inIface", COALESCE(out_iface, '') AS "outIface"
FROM flow_buckets
ORDER BY bucket_at, server_id
LIMIT ? OFFSET ?
`, [BATCH, offset])
if (rows.length === 0) break
for (const row of rows) {
buckets += 1
const serverType = catalog.byId.get(row.serverId)?.type
if (!shouldWriteFlowFact({
serverId: row.serverId,
serverType,
inIface: row.inIface,
outIface: row.outIface,
proto: Number(row.proto) || 0,
srcPort: Number(row.srcPort) || 0,
dstPort: Number(row.dstPort) || 0,
src: row.src,
dst: row.dst,
topo,
})) continue
const destMeta = resolveInternetDest({
src: row.src,
dst: row.dst,
proto: Number(row.proto) || 0,
srcPort: Number(row.srcPort) || 0,
dstPort: Number(row.dstPort) || 0,
serverId: row.serverId,
inIface: row.inIface,
topo,
})
bumpFlowFact({
serverId: row.serverId,
bucketAt: hourFromBucket(row.bucketAt),
iface: canonicalFactIface(row.serverId, row.inIface),
country: destMeta.country || "XX",
service: destMeta.classified.service,
asn: destMeta.asn,
bytes: Number(row.bytes) || 0,
packets: Number(row.packets) || 0,
})
}
offset += rows.length
if (rows.length < BATCH) break
}
const facts = await flushFlowFacts()
const range = await dbGet<{ n: number }>(`SELECT COUNT(*)::int AS n FROM flow_buckets`)
return {
ok: true as const,
buckets: Number(range?.n) || buckets,
facts,
days: dayList,
}
})
}
@@ -275,6 +275,10 @@ export function resetFactsForTests(): void {
ensuredParts.clear() ensuredParts.clear()
} }
export function discardPendingFacts(): void {
hourFacts.clear()
}
export function factsSnapshotForTests(): FactRow[] { export function factsSnapshotForTests(): FactRow[] {
const parsed: FactRow[] = [] const parsed: FactRow[] = []
for (const [k, acc] of hourFacts) { for (const [k, acc] of hourFacts) {
@@ -22,8 +22,9 @@ rememberServerIfaces(7, [
{ ".id": "*A", name: "wg-flow" }, { ".id": "*A", name: "wg-flow" },
{ ".id": "*D", name: "bridge" }, { ".id": "*D", name: "bridge" },
]) ])
assert.equal(resolveIfaceName(7, "2").name, "ether1") assert.equal(resolveIfaceName(7, "2").name, "ether1")
assert.equal(resolveIfaceName(7, "10").name, "wg-flow") assert.equal(resolveIfaceName(7, "#2").name, "ether1")
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
assert.equal(resolveIfaceName(7, "13").name, "bridge") assert.equal(resolveIfaceName(7, "13").name, "bridge")
assert.equal(resolveIfaceName(7, "0").name, "—") assert.equal(resolveIfaceName(7, "0").name, "—")
assert.equal(resolveIfaceName(7, "ether1").name, "ether1") assert.equal(resolveIfaceName(7, "ether1").name, "ether1")
@@ -0,0 +1,58 @@
import assert from "node:assert/strict"
import {
bindingIfaceAliases,
bindingIfaceAliasesAllServers,
canonicalFactIface,
collapseServerIfaceRows,
displayFactIface,
expandBindingIfaces,
factIfaceAliases,
rememberServerIfaces,
resetIfaceCacheForTests,
resolveIfaceName,
} from "./traffic-flow-ifindex.js"
resetIfaceCacheForTests()
assert.equal(canonicalFactIface(1, "2"), "2")
assert.deepEqual(bindingIfaceAliases(1, "gre-client"), ["gre-client"])
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
assert.equal(canonicalFactIface(1, "2"), "gre-client")
assert.equal(canonicalFactIface(1, "gre-client"), "gre-client")
assert.equal(canonicalFactIface(1, "9"), "9")
assert.equal(resolveIfaceName(1, "9").name, "#9")
assert.equal(resolveIfaceName(1, "2").name, "gre-client")
assert.equal(resolveIfaceName(1, "#2").name, "gre-client")
assert.equal(displayFactIface(1, "2"), "gre-client")
const aliases = bindingIfaceAliases(1, "gre-client")
assert.ok(aliases.includes("gre-client"))
assert.ok(aliases.includes("2"))
assert.ok(aliases.includes("#2"))
const fromIndex = factIfaceAliases("2", 1)
assert.ok(fromIndex.includes("gre-client"))
assert.ok(fromIndex.includes("2"))
assert.ok(fromIndex.includes("#2"))
const all = bindingIfaceAliasesAllServers("gre-client")
assert.ok(all.includes("2"))
const expanded = expandBindingIfaces([{ serverId: 1, iface: "gre-client" }])
assert.ok(expanded.some((x) => x.iface === "2"))
assert.ok(expanded.some((x) => x.iface === "gre-client"))
const collapsed = collapseServerIfaceRows([
{ serverId: 1, iface: "2", bytes: 10, packets: 1 },
{ serverId: 1, iface: "gre-client", bytes: 5, packets: 2 },
{ serverId: 1, iface: "wan1", bytes: 3, packets: 1 },
])
assert.equal(collapsed.length, 2)
const gre = collapsed.find((r) => r.iface === "gre-client")
assert.ok(gre)
assert.equal(gre.bytes, 15)
assert.equal(gre.packets, 3)
assert.ok(collapsed.some((r) => r.iface === "wan1"))
resetIfaceCacheForTests()
console.log("traffic-flow-ifindex.test.ts: ok")
+129 -6
View File
@@ -36,12 +36,135 @@ export function rememberServerIfaces(serverId: number, rows: RosIfaceIndexRow[])
export function resolveIfaceName(serverId: number, indexOrName: string): { name: string; index: string } { export function resolveIfaceName(serverId: number, indexOrName: string): { name: string; index: string } {
const trimmed = String(indexOrName ?? "").trim() const trimmed = String(indexOrName ?? "").trim()
if (!trimmed || trimmed === "0") return { name: "—", index: trimmed } const asIndex = trimmed.startsWith("#") && /^\d+$/.test(trimmed.slice(1)) ? trimmed.slice(1) : trimmed
if (!/^\d+$/.test(trimmed)) return { name: trimmed, index: "" } if (!asIndex || asIndex === "0") return { name: "—", index: asIndex }
const idx = Number(trimmed) if (!/^\d+$/.test(asIndex)) return { name: trimmed, index: "" }
const name = cache.get(serverId)?.get(idx) const name = cache.get(serverId)?.get(Number(asIndex))
if (name) return { name, index: trimmed } if (name) return { name, index: asIndex }
return { name: `#${trimmed}`, index: trimmed } return { name: `#${asIndex}`, index: asIndex }
}
/** Имя iface для факта куба: ifIndex→имя, без `#13` при пустом кэше. */
export function canonicalFactIface(serverId: number, inIface: string): string {
const trimmed = String(inIface ?? "").trim()
if (!trimmed) return trimmed
if (!/^\d+$/.test(trimmed)) return trimmed
const name = cache.get(serverId)?.get(Number(trimmed))
return name || trimmed
}
function numericIfaceIndex(iface: string): string | null {
const raw = String(iface ?? "").trim()
if (/^\d+$/.test(raw)) return raw
if (raw.startsWith("#") && /^\d+$/.test(raw.slice(1))) return raw.slice(1)
return null
}
/** Имя для UI: ifIndex → RouterOS name; `0` → «—»; miss → `#n`. */
export function displayFactIface(serverId: number, iface: string): string {
return resolveIfaceName(serverId, iface).name
}
/** Склеить факты `2` + `ether1` в одну строку после резолва ifIndex. */
export function collapseServerIfaceRows(
rows: Array<{ serverId: number; iface: string; bytes: number; packets: number }>,
): Array<{ serverId: number; iface: string; bytes: number; packets: number }> {
const acc = new Map<string, { serverId: number; iface: string; bytes: number; packets: number }>()
for (const r of rows) {
const name = displayFactIface(r.serverId, r.iface)
const k = `${r.serverId}\0${name}`
const prev = acc.get(k)
const bytes = Number(r.bytes) || 0
const packets = Number(r.packets) || 0
if (prev) {
prev.bytes += bytes
prev.packets += packets
} else {
acc.set(k, { serverId: r.serverId, iface: name, bytes, packets })
}
}
return [...acc.values()]
}
/** Ключи факта для фильтра: имя, ifIndex и `#n`. */
export function factIfaceAliases(iface: string, serverId?: number): string[] {
const raw = String(iface ?? "").trim()
if (!raw) return []
const out = new Set<string>([raw])
const idx = numericIfaceIndex(raw)
if (idx) {
out.add(idx)
out.add(`#${idx}`)
const n = Number(idx)
if (serverId != null) {
const name = cache.get(serverId)?.get(n)
if (name) out.add(name)
} else {
for (const map of cache.values()) {
const name = map.get(n)
if (name) out.add(name)
}
}
}
if (serverId != null) {
for (const a of bindingIfaceAliases(serverId, raw)) out.add(a)
} else {
for (const a of bindingIfaceAliasesAllServers(raw)) out.add(a)
}
return [...out]
}
/** Имя + ifIndex + `#n` — тот же матч, что карта `/traffic`. */
export function bindingIfaceAliases(serverId: number, interfaceName: string): string[] {
const name = String(interfaceName ?? "").trim()
if (!name) return []
const out = new Set<string>([name])
const map = cache.get(serverId)
const idx = numericIfaceIndex(name)
const canonical = (idx && map?.get(Number(idx))) || name
out.add(canonical)
if (idx) {
out.add(idx)
out.add(`#${idx}`)
}
if (!map) return [...out]
for (const [i, n] of map) {
if (n !== canonical && n !== name) continue
out.add(String(i))
out.add(`#${i}`)
}
return [...out]
}
export function bindingIfaceAliasesAllServers(interfaceName: string): string[] {
const name = String(interfaceName ?? "").trim()
const out = new Set<string>(name ? [name] : [])
for (const serverId of cache.keys()) {
for (const alias of bindingIfaceAliases(serverId, name)) out.add(alias)
}
return [...out]
}
export function expandBindingIfaces(
binds: Array<{ serverId: number; iface: string }>,
): Array<{ serverId: number; iface: string }> {
const seen = new Set<string>()
const out: Array<{ serverId: number; iface: string }> = []
for (const b of binds) {
for (const iface of bindingIfaceAliases(b.serverId, b.iface)) {
const k = `${b.serverId}\0${iface}`
if (seen.has(k)) continue
seen.add(k)
out.push({ serverId: b.serverId, iface })
}
}
return out
}
export function listCachedIfaceNames(serverId: number): string[] {
const map = cache.get(serverId)
if (!map) return []
return [...new Set(map.values())]
} }
export function ifaceCacheHas(serverId: number): boolean { export function ifaceCacheHas(serverId: number): boolean {
+36 -1
View File
@@ -1,5 +1,5 @@
import assert from "node:assert/strict" import assert from "node:assert/strict"
import { isNonPublicIp, pickInternetPeer } from "./traffic-flow-ip.js" import { isNonPublicIp, pickInternetDest, pickInternetPeer } from "./traffic-flow-ip.js"
assert.equal(isNonPublicIp("10.200.100.53"), true) assert.equal(isNonPublicIp("10.200.100.53"), true)
assert.equal(isNonPublicIp("173.194.151.65"), false) assert.equal(isNonPublicIp("173.194.151.65"), false)
@@ -22,4 +22,39 @@ assert.equal(
) )
assert.equal(pickInternetPeer("10.1.1.1", "10.2.2.2", 443, 80), "10.2.2.2") assert.equal(pickInternetPeer("10.1.1.1", "10.2.2.2", 443, 80), "10.2.2.2")
const rost = "95.167.1.10"
const ours = new Set(["198.51.100.1", "203.0.113.10"])
const client = { ours, boundClient: true }
assert.equal(
pickInternetDest("10.200.100.53", "104.18.35.51", 53880, 443, client),
"104.18.35.51",
"RFC1918 → CF на client GRE",
)
assert.equal(
pickInternetDest("173.194.151.65", "10.200.100.53", 443, 57182, client),
"173.194.151.65",
"Google:443 → RFC1918 на client GRE",
)
assert.equal(
pickInternetDest(rost, "10.200.100.53", 51234, 443, client),
"",
"Rostelecom → overlay 10.x: не dest ASN клиента",
)
assert.equal(
pickInternetDest(rost, "8.8.8.8", 51234, 443, client),
"8.8.8.8",
"Rostelecom → Google:443 на client GRE",
)
assert.equal(
pickInternetDest(rost, "1.1.1.1", 51234, 40000, client),
"1.1.1.1",
"оба публичные без well-known на client GRE → dst",
)
assert.equal(
pickInternetDest("8.8.8.8", "198.51.100.1", 443, 51234, { ours }),
"8.8.8.8",
"ours как dst: dest = публичный src",
)
console.log("traffic-flow-ip.test.ts: ok") console.log("traffic-flow-ip.test.ts: ok")
+46 -5
View File
@@ -55,13 +55,46 @@ export function isNonPublicIp(ip: string): boolean {
const PEER_WELL_KNOWN_PORTS = new Set([80, 443, 53, 853]) const PEER_WELL_KNOWN_PORTS = new Set([80, 443, 53, 853])
export interface InternetDestCtx {
/** WAN IP узлов сети (EN/JH) — не интернет-назначение. */
ours?: ReadonlySet<string>
/** Ingress с bound GRE/WG клиента: dest = нелокальный IP, не ASN клиента. */
boundClient?: boolean
}
export function isLocalIp(ip: string, ours?: ReadonlySet<string>): boolean {
if (isNonPublicIp(ip)) return true
return Boolean(ours?.has(String(ip ?? "").trim()))
}
/** /**
* Интернет-сторона потока: у IPFIX сервис часто в src (Google:443 → RFC1918:ephemeral). * Интернет-назначение потока для ASN/страны/сервиса.
* Классифицировать этот IP, не слепой dst. * Пустая строка — dest нет (не GeoIP IP клиента).
*/ */
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string { export function pickInternetDest(
const srcPub = !isNonPublicIp(src) src: string,
const dstPub = !isNonPublicIp(dst) dst: string,
srcPort: number,
dstPort: number,
ctx?: InternetDestCtx,
): string {
const ours = ctx?.ours
const srcLocal = isLocalIp(src, ours)
const dstLocal = isLocalIp(dst, ours)
const srcPub = !srcLocal
const dstPub = !dstLocal
if (ctx?.boundClient) {
if (dstPub) return dst
if (srcPub && dstLocal) {
const srcWk = PEER_WELL_KNOWN_PORTS.has(srcPort)
const dstWk = PEER_WELL_KNOWN_PORTS.has(dstPort)
if (srcWk && !dstWk) return src
return ""
}
return ""
}
if (srcPub && !dstPub) return src if (srcPub && !dstPub) return src
if (dstPub && !srcPub) return dst if (dstPub && !srcPub) return dst
if (srcPub && dstPub) { if (srcPub && dstPub) {
@@ -72,3 +105,11 @@ export function pickInternetPeer(src: string, dst: string, srcPort: number, dstP
} }
return dst return dst
} }
/**
* Интернет-сторона потока без топологии: у IPFIX сервис часто в src (Google:443 → RFC1918).
* Для куба статистики используйте pickInternetDest.
*/
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
return pickInternetDest(src, dst, srcPort, dstPort) || dst
}
@@ -584,4 +584,98 @@ try {
resetFlowCatalogForTests() resetFlowCatalogForTests()
} }
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
rememberServerIfaces(9, [
{ ".id": "*1", name: "ether1" },
])
googleRipe()
ingestParsedFlowsForServerForTests(7, [payloadFlow("8.8.8.8", 12_000)])
ingestParsedFlowsForServerForTests(9, [{
src: "10.100.1.17",
dst: "8.8.8.8",
proto: 6,
srcPort: 51234,
dstPort: 443,
bytes: 12_000,
packets: 10,
inIface: "1",
outIface: "1",
nextHop: "",
}])
try {
resetFlowMapHopsCacheForTests()
const dual = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
const gre = dual.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
assert.ok(gre, "GRE JH→EN сохранён")
assert.equal(gre.bytes, 12_000)
const googlePaths = (dual.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
assert.equal(googlePaths.length, 1, "один путь без копии EN")
assert.equal(googlePaths[0]?.clientId, "u1")
assert.equal(googlePaths[0]?.viaId, "7")
const googleEdge = dual.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
assert.ok(googleEdge)
assert.equal(googleEdge.bytes, 12_000)
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
disableRipeEnqueueForTests()
seedFlowTopologyForTests(topo)
rememberServerIfaces(7, [
{ ".id": "*2", name: "gre-client" },
{ ".id": "*3", name: "gre-jh-en" },
])
googleRipe()
ingestParsedFlowsForServerForTests(7, [
payloadFlow("8.8.8.8", 8_000),
{
src: "8.8.8.8",
dst: "10.100.1.17",
proto: 6,
srcPort: 443,
dstPort: 51234,
bytes: 4_000,
packets: 8,
inIface: "3",
outIface: "2",
nextHop: "",
},
])
try {
resetFlowMapHopsCacheForTests()
const bothDir = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
const gre = bothDir.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
assert.ok(gre, "GRE-hop при fwd/rev")
const googlePaths = (bothDir.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
assert.equal(googlePaths.length, 1, "fwd+rev — один клиент")
assert.equal(googlePaths[0]?.clientId, "u1")
assert.ok(!(bothDir.servicePaths ?? []).some((p) => p.serviceId === "svc:google" && p.clientId === "—"))
const googleEdge = bothDir.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
assert.ok(googleEdge)
assert.equal(googleEdge.bytes, 12_000)
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
} finally {
seedFlowTopologyForTests(null)
resetFlowRingsForTests()
resetIfaceCacheForTests()
resetRipeCacheForTests()
resetFlowCatalogForTests()
}
console.log("traffic-flow-map-hops.test.ts: ok") console.log("traffic-flow-map-hops.test.ts: ok")
+50 -8
View File
@@ -8,15 +8,16 @@ import {
mapServiceNodeId, mapServiceNodeId,
resolveFlowBrand, resolveFlowBrand,
} from "./traffic-flow-brands.js" } from "./traffic-flow-brands.js"
import { dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js" import { dedupFlowRowsAcrossExporters, dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js" import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js"
import { resolveIfaceName } from "./traffic-flow-ifaces.js" import { resolveIfaceName } from "./traffic-flow-ifaces.js"
import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js" import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js"
import { pickInternetPeer } from "./traffic-flow-ip.js" import { destCtxForIface } from "./traffic-flow-dest.js"
import { pickInternetDest } from "./traffic-flow-ip.js"
import { type FlowIpMeta } from "./traffic-flow-ripe.js" import { type FlowIpMeta } from "./traffic-flow-ripe.js"
import { resolveFlowIp } from "./traffic-flow-geoip.js" import { resolveFlowIp } from "./traffic-flow-geoip.js"
import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js" import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js"
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog } from "./traffic-flow-topology.js" import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog, type FlowTopology } from "./traffic-flow-topology.js"
import { flowDataEpoch } from "./traffic-flow-engine.js" import { flowDataEpoch } from "./traffic-flow-engine.js"
export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5 export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5
@@ -141,6 +142,16 @@ function ifaceUsable(name: string): boolean {
return Boolean(name) && name !== "—" return Boolean(name) && name !== "—"
} }
function resolveMapClient(
topo: FlowTopology,
serverId: number,
inName: string,
outName: string,
) {
return resolveClient(topo, serverId, inName)
?? (ifaceUsable(outName) ? resolveClient(topo, serverId, outName) : null)
}
function bump(acc: Map<string, HopAcc>, key: string, seed: Omit<HopAcc, "bytes" | "bytesFwd" | "bytesRev">, bytes: number, dir: "fwd" | "rev" | "both"): void { function bump(acc: Map<string, HopAcc>, key: string, seed: Omit<HopAcc, "bytes" | "bytesFwd" | "bytesRev">, bytes: number, dir: "fwd" | "rev" | "both"): void {
const prev = acc.get(key) const prev = acc.get(key)
const addFwd = dir === "fwd" || dir === "both" ? bytes : 0 const addFwd = dir === "fwd" || dir === "both" ? bytes : 0
@@ -238,6 +249,21 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
const enIds = new Set(topo.enNodes.map((n) => n.id)) const enIds = new Set(topo.enNodes.map((n) => n.id))
let totalBytes = 0 let totalBytes = 0
function rowClient(r: (typeof working)[number]) {
const inName = resolveIfaceName(r.serverId, r.inIface).name
const outName = resolveIfaceName(r.serverId, r.outIface).name
return resolveMapClient(topo, r.serverId, inName, outName)
}
const payloadRows = wantDedup
? dedupFlowRowsAcrossExporters(working, (a, b) => {
const aCli = Boolean(rowClient(a))
const bCli = Boolean(rowClient(b))
if (aCli !== bCli) return aCli ? a : b
return a.bytes >= b.bytes ? a : b
})
: working
for (const r of working) { for (const r of working) {
const inRes = resolveIfaceName(r.serverId, r.inIface) const inRes = resolveIfaceName(r.serverId, r.inIface)
const outRes = resolveIfaceName(r.serverId, r.outIface) const outRes = resolveIfaceName(r.serverId, r.outIface)
@@ -323,11 +349,22 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
}, r.bytes, "fwd") }, r.bytes, "fwd")
} }
} }
}
for (const r of payloadRows) {
const inName = resolveIfaceName(r.serverId, r.inIface).name
const outName = resolveIfaceName(r.serverId, r.outIface).name
totalBytes += r.bytes totalBytes += r.bytes
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort) const dest = pickInternetDest(
const client = resolveClient(topo, r.serverId, inName) r.src,
const prevDst = dstAcc.get(peer) r.dst,
r.srcPort,
r.dstPort,
destCtxForIface(topo, r.serverId, inName),
)
if (!dest) continue
const client = resolveMapClient(topo, r.serverId, inName, outName)
const prevDst = dstAcc.get(dest)
if (prevDst) { if (prevDst) {
prevDst.bytes += r.bytes prevDst.bytes += r.bytes
bumpFrom(prevDst, String(r.serverId), r.bytes, client) bumpFrom(prevDst, String(r.serverId), r.bytes, client)
@@ -340,7 +377,7 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
fromBytes: new Map(), fromBytes: new Map(),
} }
bumpFrom(acc, String(r.serverId), r.bytes, client) bumpFrom(acc, String(r.serverId), r.bytes, client)
dstAcc.set(peer, acc) dstAcc.set(dest, acc)
} }
} }
@@ -428,10 +465,15 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
const enName = nodeName(fromId) const enName = nodeName(fromId)
const viaName = nodeName(exporterId) const viaName = nodeName(exporterId)
for (const [clientId, c] of from.clients) { for (const [clientId, c] of from.clients) {
const pathKey = `${clientId}|${exporterId}|${fromId}|${toId}` const pathKey = `${clientId}|${fromId}|${toId}`
const prevPath = svcPaths.get(pathKey) const prevPath = svcPaths.get(pathKey)
if (prevPath) { if (prevPath) {
prevPath.bytes += c.bytes prevPath.bytes += c.bytes
if (exporterId !== fromId && prevPath.viaId === fromId) {
prevPath.viaId = exporterId
prevPath.viaName = viaName
}
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
} else { } else {
svcPaths.set(pathKey, { svcPaths.set(pathKey, {
clientId, clientId,
+51 -3
View File
@@ -1,7 +1,7 @@
import { db, dbAll } from "../db/index.js" import { db, dbAll } from "../db/index.js"
import { parseJsonArray } from "../db/json.js" import { parseJsonArray } from "../db/json.js"
import { appUsers, servers, userInterfaceBindings } from "../db/schema.js" import { appUsers, servers, userInterfaceBindings } from "../db/schema.js"
import { mapRosInterfaceType } from "../modules/users/iface-type.js" import { mapRosInterfaceType, parseRawInterfaces } from "../modules/users/iface-type.js"
import type { PlaneTopology } from "./traffic-flow-planes.js" import type { PlaneTopology } from "./traffic-flow-planes.js"
export interface FlowClientBinding { export interface FlowClientBinding {
@@ -25,6 +25,8 @@ export interface FlowTopology {
enHosts: Set<string> enHosts: Set<string>
jhHosts: Set<string> jhHosts: Set<string>
wanIfaces: Map<number, Set<string>> wanIfaces: Map<number, Set<string>>
/** GRE/WG из последнего снимка RouterOS (`type`), без mgmt. */
tunnelIfaces?: Map<number, Set<string>>
plane: PlaneTopology plane: PlaneTopology
} }
@@ -48,6 +50,15 @@ export function invalidateFlowCatalogCache(): void {
serverCatalogCache = null serverCatalogCache = null
} }
export function peekFlowTopology(): FlowTopology | null {
if (seeded) return seeded
return topologyCache?.topo ?? null
}
export function peekServerCatalog(): { list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> } | null {
return serverCatalogCache
}
export async function getServerCatalog(): Promise<{ list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> }> { export async function getServerCatalog(): Promise<{ list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> }> {
const now = Date.now() const now = Date.now()
if (serverCatalogCache && now - serverCatalogCache.at < CATALOG_TTL_MS) { if (serverCatalogCache && now - serverCatalogCache.at < CATALOG_TTL_MS) {
@@ -76,6 +87,25 @@ function ifaceKey(serverId: number, name: string): string {
return `${serverId}|${name}` return `${serverId}|${name}`
} }
async function loadTunnelIfacesFromSnapshots(): Promise<Map<number, Set<string>>> {
const rows = await dbAll<{ serverId: number; rawInterfaces: unknown }>(`
SELECT DISTINCT ON (server_id) server_id AS "serverId", raw_interfaces AS "rawInterfaces"
FROM server_snapshots
ORDER BY server_id, polled_at DESC
`)
const map = new Map<number, Set<string>>()
for (const r of rows) {
const set = new Set<string>()
for (const iface of parseRawInterfaces(r.rawInterfaces)) {
if (iface.type !== "gre" && iface.type !== "wg") continue
if (iface.name.toLowerCase() === "wg-flow") continue
set.add(iface.name)
}
if (set.size) map.set(r.serverId, set)
}
return map
}
export async function loadFlowTopology(): Promise<FlowTopology> { export async function loadFlowTopology(): Promise<FlowTopology> {
if (seeded) return seeded if (seeded) return seeded
const now = Date.now() const now = Date.now()
@@ -118,6 +148,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
for (const h of hosts) jhHosts.add(h) for (const h of hosts) jhHosts.add(h)
} }
} }
const tunnelIfaces = await loadTunnelIfacesFromSnapshots()
const topo: FlowTopology = { const topo: FlowTopology = {
clientIfaces, clientIfaces,
clientByIface, clientByIface,
@@ -125,6 +156,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
enHosts, enHosts,
jhHosts, jhHosts,
wanIfaces, wanIfaces,
tunnelIfaces,
plane: { plane: {
clientIfaceNames: allClientNames, clientIfaceNames: allClientNames,
enHosts, enHosts,
@@ -140,6 +172,18 @@ export function seedFlowTopologyForTests(topo: FlowTopology | null): void {
invalidateFlowCatalogCache() invalidateFlowCatalogCache()
} }
export function flowOursHosts(topo: FlowTopology | null | undefined): Set<string> {
const ours = new Set<string>()
if (!topo) return ours
for (const h of topo.enHosts) {
if (h) ours.add(h)
}
for (const h of topo.jhHosts) {
if (h) ours.add(h)
}
return ours
}
export function resolveClient( export function resolveClient(
topo: FlowTopology, topo: FlowTopology,
serverId: number, serverId: number,
@@ -168,10 +212,14 @@ export function resolveEn(
export function enGreIfaceNames(topo: FlowTopology, serverId: number, ifaceNames: string[]): string[] { export function enGreIfaceNames(topo: FlowTopology, serverId: number, ifaceNames: string[]): string[] {
const client = topo.clientIfaces.get(serverId) ?? new Set<string>() const client = topo.clientIfaces.get(serverId) ?? new Set<string>()
const wan = topo.wanIfaces.get(serverId) ?? new Set<string>()
const typed = topo.tunnelIfaces?.get(serverId)
return ifaceNames.filter((name) => { return ifaceNames.filter((name) => {
if (client.has(name)) return false if (client.has(name) || wan.has(name)) return false
if (name === "wg-flow") return false if (name === "wg-flow") return false
return mapRosInterfaceType("", name) === "gre" if (typed && typed.size > 0) return typed.has(name)
const t = mapRosInterfaceType("", name)
return t === "gre" || t === "wg"
}) })
} }
+43
View File
@@ -0,0 +1,43 @@
import assert from "node:assert/strict"
import { mapVxlanRow } from "./vxlan-live.js"
const server = {
id: 7,
name: "mt-msk",
host: "10.0.0.1",
site: "MSK",
country: "RU",
} as Parameters<typeof mapVxlanRow>[0]
const row = mapVxlanRow(
server,
{
".id": "*3",
name: "vxlan-10",
vni: "10010",
port: "8472",
"local-address": "10.0.0.1",
running: "true",
disabled: "false",
l2mtu: "1500",
"mac-learning": "true",
"arp-proxy": "true",
comment: "overlay",
},
[
{ interface: "vxlan-10", "remote-ip": "10.0.1.1" },
{ interface: "other", "remote-ip": "1.1.1.1" },
{ interface: "vxlan-10", "remote-ip": "10.0.2.1" },
],
0,
)
assert.equal(row.serverId, "7")
assert.equal(row.vni, 10010)
assert.equal(row.dstPort, 8472)
assert.equal(row.status, "up")
assert.equal(row.enabled, true)
assert.deepEqual(row.remoteVteps, ["10.0.1.1", "10.0.2.1"])
assert.equal(row.vtepIp, "10.0.0.1")
console.log("vxlan-live.test.ts: ok")
+136
View File
@@ -0,0 +1,136 @@
import { eq } from "drizzle-orm"
import { db } from "../db/index.js"
import { servers } from "../db/schema.js"
import { MikrotikClient } from "./mikrotik.js"
type ServerRow = typeof servers.$inferSelect
interface RosVxlan {
".id"?: string
name?: string
vni?: string
port?: string
"local-address"?: string
"vtep-address"?: string
running?: string
disabled?: string
comment?: string
l2mtu?: string
arp?: string
"arp-proxy"?: string
"mac-learning"?: string
learning?: string
}
interface RosVxlanVtep {
".id"?: string
interface?: string
"remote-ip"?: string
}
export type VxlanTunnelLive = {
id: string
rosId: string
name: string
vni: number
port: number
dstPort: number
serverId: string
vtepIp: string
remoteVteps: string[]
l2mtu: number
arpProxy: boolean
macLearning: boolean
comment: string
enabled: boolean
status: "up" | "down"
}
function rosYes(v: string | undefined): boolean {
return v === "true" || v === "yes"
}
function parseIntSafe(v: string | undefined, fallback: number): number {
const n = Number.parseInt(v ?? "", 10)
return Number.isFinite(n) ? n : fallback
}
export function mapVxlanRow(
server: ServerRow,
vx: RosVxlan,
vteps: RosVxlanVtep[],
idx: number,
): VxlanTunnelLive {
const name = (vx.name ?? "").trim() || `vxlan-${idx + 1}`
const rosId = String(vx[".id"] ?? name)
const disabled = rosYes(vx.disabled)
const running = rosYes(vx.running)
const port = parseIntSafe(vx.port, 8472)
const remoteVteps = vteps
.filter((v) => (v.interface ?? "").trim() === name)
.map((v) => (v["remote-ip"] ?? "").trim())
.filter(Boolean)
return {
id: `${server.id}-${rosId}`,
rosId,
name,
vni: parseIntSafe(vx.vni, 0),
port: 0,
dstPort: port,
serverId: String(server.id),
vtepIp: (vx["local-address"] ?? vx["vtep-address"] ?? "").trim(),
remoteVteps,
l2mtu: parseIntSafe(vx.l2mtu, 1500),
arpProxy: rosYes(vx["arp-proxy"]) || vx.arp === "proxy-arp" || vx.arp === "enabled",
macLearning: vx["mac-learning"] != null ? rosYes(vx["mac-learning"]) : vx.learning !== "false",
comment: vx.comment ?? "",
enabled: !disabled,
status: !disabled && running ? "up" : "down",
}
}
async function fetchVxlanForServer(server: ServerRow): Promise<VxlanTunnelLive[]> {
const client = MikrotikClient.fromServer(server)
const [vxRaw, vtepRaw] = await Promise.all([
client.get<RosVxlan[]>("/interface/vxlan"),
client.get<RosVxlanVtep[]>("/interface/vxlan/vteps").catch(() => [] as RosVxlanVtep[]),
])
const list = Array.isArray(vxRaw) ? vxRaw : []
const vteps = Array.isArray(vtepRaw) ? vtepRaw : []
return list.map((vx, idx) => mapVxlanRow(server, vx, vteps, idx))
}
export async function listVxlanTunnels(): Promise<VxlanTunnelLive[]> {
const enabledServers = await db.select().from(servers).where(eq(servers.enabled, true))
const results = await Promise.all(
enabledServers.map(async (server) => {
try {
return await fetchVxlanForServer(server)
} catch {
return [] as VxlanTunnelLive[]
}
}),
)
return results.flat()
}
export async function listVxlanTunnelsForServer(server: ServerRow): Promise<VxlanTunnelLive[]> {
try {
return await fetchVxlanForServer(server)
} catch {
return []
}
}
export async function countVxlanTunnels(): Promise<number> {
try {
const result = await Promise.race([
listVxlanTunnels(),
new Promise<null>((resolve) => setTimeout(() => resolve(null), 8_000)),
])
if (!result) return 0
return result.length
} catch {
return 0
}
}
+17
View File
@@ -285,6 +285,19 @@ export interface OspfInterfaceRead {
useBfd: boolean useBfd: boolean
} }
export interface OspfRouteRead {
id: string
serverId: number
serverName: string
serverSite: string
destination: string
type: "O" | "O IA" | "O E1" | "O E2"
cost: number
nextHop: string
via: string
area: string
}
export interface OspfInstanceRead { export interface OspfInstanceRead {
id: string id: string
serverId: number serverId: number
@@ -306,6 +319,7 @@ export interface RosIpRoute {
"dst-address": string "dst-address": string
"pref-src"?: string "pref-src"?: string
"gateway"?: string "gateway"?: string
"immediate-gw"?: string
"distance"?: string "distance"?: string
"scope"?: string "scope"?: string
"active"?: string // "true" "active"?: string // "true"
@@ -314,6 +328,9 @@ export interface RosIpRoute {
"connect"?: string "connect"?: string
"bgp"?: string "bgp"?: string
"ospf"?: string "ospf"?: string
"ospf-type"?: string
"ospf-metric"?: string
"ospf-area"?: string
"rip"?: string "rip"?: string
"blackhole"?: string "blackhole"?: string
"unreachable"?: string "unreachable"?: string
+12 -5
View File
@@ -106,7 +106,15 @@ const navStructure: { label: string; items: NavItemBase[] }[] = [
}, },
] ]
type LiveSidebarCounts = SidebarCountsDto & { greTunnels?: number; certificates?: number; wireguard?: number; users?: number } type LiveSidebarCounts = SidebarCountsDto & {
greTunnels?: number
certificates?: number
wireguard?: number
users?: number
bgpSessions?: number
vxlan?: number
containers?: number
}
export function AppSidebar({ ...props }: React.ComponentProps<typeof Sidebar>) { export function AppSidebar({ ...props }: React.ComponentProps<typeof Sidebar>) {
const { mode, backendUrl, prefsHydrated } = useDataSource() const { mode, backendUrl, prefsHydrated } = useDataSource()
@@ -171,10 +179,9 @@ export function AppSidebar({ ...props }: React.ComponentProps<typeof Sidebar>) {
if (url === "/gre") return formatSidebarBadgeCount(liveCounts.greTunnels ?? 0) if (url === "/gre") return formatSidebarBadgeCount(liveCounts.greTunnels ?? 0)
if (url === "/certificates") return formatSidebarBadgeCount(liveCounts.certificates ?? 0) if (url === "/certificates") return formatSidebarBadgeCount(liveCounts.certificates ?? 0)
if (url === "/wireguard") return formatSidebarBadgeCount(liveCounts.wireguard ?? 0) if (url === "/wireguard") return formatSidebarBadgeCount(liveCounts.wireguard ?? 0)
if (url === "/bgp") return formatSidebarBadgeCount(liveCounts.bgpSessions ?? 0)
if (url === "/containers" || url === "/bgp") { if (url === "/vxlan") return formatSidebarBadgeCount(liveCounts.vxlan ?? 0)
return undefined if (url === "/containers") return formatSidebarBadgeCount(liveCounts.containers ?? 0)
}
return undefined return undefined
} }
@@ -5,7 +5,12 @@ import { Flag } from "@/components/flag"
import { Badge } from "@/components/reui/badge" import { Badge } from "@/components/reui/badge"
import { fmtBps, formatBytes } from "@/lib/fmt-rate" import { fmtBps, formatBytes } from "@/lib/fmt-rate"
import { cn } from "@/lib/utils" import { cn } from "@/lib/utils"
import type { StatisticsBreakdownRow } from "@mmapp/contracts/statistics" import {
STATISTICS_DUP_MARK,
STATISTICS_UNBOUND_USER_ID,
STATISTICS_WAN_MARK,
type StatisticsBreakdownRow,
} from "@mmapp/contracts/statistics"
export type StatisticsSliceKind = "users" | "servers" | "interfaces" | "countries" | "services" | "asns" export type StatisticsSliceKind = "users" | "servers" | "interfaces" | "countries" | "services" | "asns"
@@ -15,13 +20,16 @@ export function StatisticsBreakdownDataGrid({
selectedId, selectedId,
onRowClick, onRowClick,
isLoading, isLoading,
density = "full",
}: { }: {
rows: StatisticsBreakdownRow[] rows: StatisticsBreakdownRow[]
kind: StatisticsSliceKind kind: StatisticsSliceKind
selectedId?: string selectedId?: string
onRowClick?: (row: StatisticsBreakdownRow) => void onRowClick?: (row: StatisticsBreakdownRow) => void
isLoading?: boolean isLoading?: boolean
density?: "full" | "mini"
}) { }) {
const mini = density === "mini"
const columns: CompactDataGridColumn<StatisticsBreakdownRow>[] = [ const columns: CompactDataGridColumn<StatisticsBreakdownRow>[] = [
{ {
id: "label", id: "label",
@@ -29,7 +37,9 @@ export function StatisticsBreakdownDataGrid({
accessorKey: "label", accessorKey: "label",
cell: (row) => ( cell: (row) => (
<span className={cn("flex items-center gap-2", selectedId === row.id && "font-medium")}> <span className={cn("flex items-center gap-2", selectedId === row.id && "font-medium")}>
{kind === "countries" && row.id !== "XX" ? <Flag code={row.id} size={16} /> : null} {kind === "countries" && row.id !== "XX" && row.id !== STATISTICS_UNBOUND_USER_ID ? (
<Flag code={row.id} size={16} />
) : null}
<span className="truncate">{row.label || row.id}</span> <span className="truncate">{row.label || row.id}</span>
{selectedId === row.id ? ( {selectedId === row.id ? (
<Badge variant="outline" size="sm"> <Badge variant="outline" size="sm">
@@ -45,23 +55,36 @@ export function StatisticsBreakdownDataGrid({
accessorKey: "bytes", accessorKey: "bytes",
cell: (row) => <span className="tabular-nums">{formatBytes(row.bytes)}</span>, cell: (row) => <span className="tabular-nums">{formatBytes(row.bytes)}</span>,
}, },
{ ...(!mini
id: "packets", ? [
header: "Пакеты", {
accessorKey: "packets", id: "packets",
cell: (row) => <span className="tabular-nums">{row.packets.toLocaleString("ru-RU")}</span>, header: "Пакеты",
}, accessorKey: "packets" as const,
{ cell: (row: StatisticsBreakdownRow) => (
id: "bps", <span className="tabular-nums">{row.packets.toLocaleString("ru-RU")}</span>
header: "Средний bitrate", ),
accessorKey: "bps", },
cell: (row) => <span className="tabular-nums">{fmtBps(row.bps)}</span>, {
}, id: "bps",
header: "Средний bitrate",
accessorKey: "bps" as const,
cell: (row: StatisticsBreakdownRow) => <span className="tabular-nums">{fmtBps(row.bps)}</span>,
},
]
: []),
{ {
id: "percent", id: "percent",
header: "Доля", header: "Доля",
accessorKey: "percent", accessorKey: "percent",
cell: (row) => <span className="tabular-nums">{row.percent.toFixed(1)}%</span>, cell: (row) => (
<span className="tabular-nums">
{row.percent === 0
&& (row.label.includes(STATISTICS_WAN_MARK) || row.label.includes(`· ${STATISTICS_DUP_MARK}`))
? "—"
: `${row.percent.toFixed(1)}%`}
</span>
),
}, },
] ]
@@ -0,0 +1,57 @@
"use client"
import { Frame, FrameHeader, FramePanel, FrameTitle } from "@/components/reui/frame"
import {
StatisticsBreakdownDataGrid,
type StatisticsSliceKind,
} from "@/components/data-grids/statistics-breakdown-data-grid"
import { STATISTICS_DIMS } from "@/lib/statistics-dims"
import type { StatisticsBreakdownRow, StatisticsDto } from "@mmapp/contracts/statistics"
const MINI_ROWS = 12
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind): StatisticsBreakdownRow[] {
if (kind === "users") return data.users
if (kind === "servers") return data.servers
if (kind === "interfaces") return data.interfaces
if (kind === "countries") return data.countries
if (kind === "services") return data.services
return data.asns
}
export function BreakdownDashboard({
data,
hidden,
selectedIdFor,
onRowClick,
isLoading,
}: {
data: StatisticsDto
hidden: Set<StatisticsSliceKind>
selectedIdFor: (kind: StatisticsSliceKind) => string | undefined
onRowClick: (kind: StatisticsSliceKind, row: StatisticsBreakdownRow) => void
isLoading?: boolean
}) {
const dims = STATISTICS_DIMS.filter((d) => !hidden.has(d.id))
return (
<div className="grid grid-cols-1 gap-4 p-4 md:grid-cols-2">
{dims.map((d) => (
<Frame key={d.id} dense>
<FrameHeader className="border-b">
<FrameTitle>{d.label}</FrameTitle>
</FrameHeader>
<FramePanel className="max-h-80 overflow-auto p-0">
<StatisticsBreakdownDataGrid
rows={rowsForKind(data, d.id).slice(0, MINI_ROWS)}
kind={d.id}
selectedId={selectedIdFor(d.id)}
onRowClick={(row) => onRowClick(d.id, row)}
isLoading={isLoading}
density="mini"
/>
</FramePanel>
</Frame>
))}
</div>
)
}
@@ -0,0 +1,73 @@
"use client"
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from "@/components/ui/select"
import { STATISTICS_DIMS } from "@/lib/statistics-dims"
import type { StatisticsPivotDim } from "@mmapp/contracts/statistics"
import type { StatisticsSliceKind } from "@/components/data-grids/statistics-breakdown-data-grid"
export function DimensionSelect({
value,
onChange,
label,
}: {
value: StatisticsSliceKind
onChange: (value: StatisticsSliceKind) => void
label?: string
}) {
const current = STATISTICS_DIMS.find((d) => d.id === value)
return (
<label className="flex items-center gap-2">
{label ? <span className="text-muted-foreground text-xs whitespace-nowrap">{label}</span> : null}
<Select value={value} onValueChange={(v) => onChange(String(v ?? value) as StatisticsSliceKind)}>
<SelectTrigger size="sm" className="min-w-40">
<SelectValue>{current?.label ?? "Измерение"}</SelectValue>
</SelectTrigger>
<SelectContent align="start">
{STATISTICS_DIMS.map((d) => (
<SelectItem key={d.id} value={d.id}>
{d.label}
</SelectItem>
))}
</SelectContent>
</Select>
</label>
)
}
export function PivotDimSelect({
value,
onChange,
exclude,
label,
}: {
value: StatisticsPivotDim
onChange: (value: StatisticsPivotDim) => void
exclude?: StatisticsPivotDim
label: string
}) {
const options = STATISTICS_DIMS.filter((d) => d.pivot !== exclude)
const current = STATISTICS_DIMS.find((d) => d.pivot === value)
return (
<label className="flex items-center gap-2">
<span className="text-muted-foreground text-xs whitespace-nowrap">{label}</span>
<Select value={value} onValueChange={(v) => onChange(String(v ?? value) as StatisticsPivotDim)}>
<SelectTrigger size="sm" className="min-w-40">
<SelectValue>{current?.label ?? label}</SelectValue>
</SelectTrigger>
<SelectContent align="start">
{options.map((d) => (
<SelectItem key={d.pivot} value={d.pivot}>
{d.label}
</SelectItem>
))}
</SelectContent>
</Select>
</label>
)
}
+13 -8
View File
@@ -1,6 +1,6 @@
"use client" "use client"
import { useMemo } from "react" import { useCallback, useMemo, useState } from "react"
import { format } from "date-fns" import { format } from "date-fns"
import { ru } from "date-fns/locale" import { ru } from "date-fns/locale"
import { CalendarIcon } from "lucide-react" import { CalendarIcon } from "lucide-react"
@@ -130,14 +130,19 @@ export function PeriodSelector({
range: DateRangeYmd range: DateRangeYmd
onChange: (next: DateRangeYmd) => void onChange: (next: DateRangeYmd) => void
}) { }) {
const [open, setOpen] = useState(false)
const selectorValue = useMemo(() => rangeToSelector(range), [range]) const selectorValue = useMemo(() => rangeToSelector(range), [range])
function handleSelectorChange(value: DateSelectorValue) { const handleSelectorChange = useCallback(
const next = dateSelectorToRange(value) (value: DateSelectorValue) => {
if (!next) return const next = dateSelectorToRange(value)
if (next.from === range.from && next.to === range.to) return if (!next) return
onChange(next) if (next.from === range.from && next.to === range.to) return
} onChange(next)
setOpen(false)
},
[onChange, range.from, range.to],
)
const activePreset = PRESETS.find((p) => { const activePreset = PRESETS.find((p) => {
const r = rangeForPreset(p.id) const r = rangeForPreset(p.id)
@@ -157,7 +162,7 @@ export function PeriodSelector({
{p.label} {p.label}
</Button> </Button>
))} ))}
<Popover> <Popover open={open} onOpenChange={setOpen} modal={false}>
<PopoverTrigger <PopoverTrigger
render={ render={
<Button type="button" variant="outline" size="sm" className="min-w-40 justify-between" /> <Button type="button" variant="outline" size="sm" className="min-w-40 justify-between" />
+38
View File
@@ -0,0 +1,38 @@
"use client"
import { XIcon } from "lucide-react"
import { Badge } from "@/components/reui/badge"
import { Button } from "@/components/ui/button"
export interface SliceChip {
key: string
label: string
}
export function SliceChips({
chips,
onRemove,
}: {
chips: SliceChip[]
onRemove: (key: string) => void
}) {
if (!chips.length) return null
return (
<div className="flex flex-wrap items-center gap-1.5 border-b px-5 py-2">
{chips.map((chip) => (
<Badge key={chip.key} variant="outline" size="sm" className="gap-1 pr-0.5">
{chip.label}
<Button
type="button"
variant="ghost"
size="icon-xs"
aria-label={`Снять ${chip.label}`}
onClick={() => onRemove(chip.key)}
>
<XIcon />
</Button>
</Badge>
))}
</div>
)
}
@@ -0,0 +1,89 @@
"use client"
import { useMemo } from "react"
import { CompactDataGrid, type CompactDataGridColumn } from "@/components/data-grids/compact-data-grid"
import { formatBytes } from "@/lib/fmt-rate"
import { cn } from "@/lib/utils"
import type { StatisticsPivotDto } from "@mmapp/contracts/statistics"
interface PivotGridRow {
id: string
label: string
total: number
[key: string]: string | number
}
export function StatisticsPivotGrid({
data,
onCellClick,
isLoading,
}: {
data: StatisticsPivotDto
onCellClick?: (rowId: string, colId: string) => void
isLoading?: boolean
}) {
const rows: PivotGridRow[] = useMemo(
() =>
data.rows.map((r) => {
const next: PivotGridRow = { id: r.id, label: r.label, total: r.total }
for (const col of data.columns) {
next[`c:${col.id}`] = r.cells[col.id] ?? 0
}
return next
}),
[data],
)
const columns: CompactDataGridColumn<PivotGridRow>[] = [
{
id: "label",
header: "Измерение",
accessorKey: "label",
cell: (row) => <span className="truncate font-medium">{row.label}</span>,
},
...data.columns.map((col) => ({
id: `c:${col.id}`,
header: col.label,
accessorKey: `c:${col.id}` as const,
cell: (row: PivotGridRow) => {
const value = Number(row[`c:${col.id}`] ?? 0)
return (
<button
type="button"
className={cn(
"tabular-nums text-left hover:underline",
col.id === "__other__" || row.id === "__other__" ? "text-muted-foreground" : "",
)}
onClick={(e) => {
e.stopPropagation()
if (row.id === "__other__" || col.id === "__other__") return
onCellClick?.(row.id, col.id)
}}
>
{data.metric === "packets" ? value.toLocaleString("ru-RU") : formatBytes(value)}
</button>
)
},
})),
{
id: "total",
header: "Итого",
accessorKey: "total",
cell: (row) => (
<span className="tabular-nums font-medium">
{data.metric === "packets" ? row.total.toLocaleString("ru-RU") : formatBytes(row.total)}
</span>
),
},
]
return (
<CompactDataGrid
data={rows}
columns={columns}
isLoading={isLoading}
emptyTitle="Нет данных сводной"
emptyDescription="Выберите разные измерения строк и колонок."
/>
)
}
+13 -2
View File
@@ -264,7 +264,7 @@ export function useDashboardLive() {
setInternetPathLoading(true) setInternetPathLoading(true)
} }
try { try {
const [overviewRes, serversRes, fr, br, ipRes, greRes, wgRes, trafficRes] = await Promise.allSettled([ const [overviewRes, serversRes, fr, br, ipRes, greRes, wgRes, trafficRes, vxRes] = await Promise.allSettled([
apiFetch<{ probes: PingProbe[] }>("/api/uptime/overview?range=1h"), apiFetch<{ probes: PingProbe[] }>("/api/uptime/overview?range=1h"),
apiFetch<BackendServerRow[]>("/api/servers"), apiFetch<BackendServerRow[]>("/api/servers"),
apiFetch<{ rulesets: Array<{ rules?: unknown[] }> }>("/api/filters/rules"), apiFetch<{ rulesets: Array<{ rules?: unknown[] }> }>("/api/filters/rules"),
@@ -273,6 +273,7 @@ export function useDashboardLive() {
apiFetch<{ tunnels?: ApiGreTunnelRow[] }>("/api/filters/gre-tunnels"), apiFetch<{ tunnels?: ApiGreTunnelRow[] }>("/api/filters/gre-tunnels"),
listWireGuard(backendUrl), listWireGuard(backendUrl),
apiFetch<{ servers?: TrafficServerRow[] }>("/api/traffic/servers?range=1h"), apiFetch<{ servers?: TrafficServerRow[] }>("/api/traffic/servers?range=1h"),
apiFetch<{ tunnels?: Array<{ id: string; name: string; status: OverlayItem["status"] }> }>("/api/vxlan"),
]) ])
const hardFail = overviewRes.status === "rejected" && serversRes.status === "rejected" const hardFail = overviewRes.status === "rejected" && serversRes.status === "rejected"
@@ -333,7 +334,17 @@ export function useDashboardLive() {
status: iface.status, status: iface.status,
})) }))
: [] : []
setOverlayItems([...greItems, ...wgItems]) const vxItems: OverlayItem[] =
vxRes.status === "fulfilled"
? (vxRes.value.tunnels ?? []).map((t) => ({
id: `vx-${t.id}`,
name: t.name,
kind: "vxlan" as const,
href: "/vxlan",
status: t.status === "up" ? "up" : "down",
}))
: []
setOverlayItems([...greItems, ...wgItems, ...vxItems])
if (trafficRes.status === "fulfilled") { if (trafficRes.status === "fulfilled") {
const rows = trafficRes.value.servers ?? [] const rows = trafficRes.value.servers ?? []
+1
View File
@@ -233,6 +233,7 @@ export interface VrfInstance {
export interface RouterContainer { export interface RouterContainer {
id: string id: string
rosId?: string
name: string name: string
serverId: string serverId: string
image: string // e.g. "nginx:alpine" image: string // e.g. "nginx:alpine"
+62
View File
@@ -0,0 +1,62 @@
import assert from "node:assert/strict"
import {
formatServicePathLabel,
formatServicePathTitle,
isUnboundServicePath,
} from "./format-service-path-label.ts"
const bound = {
clientId: "u1",
clientName: "D",
viaId: "7",
viaName: "nsk-gw01",
enId: "9",
enName: "arn-gw01",
serviceId: "svc:cdn",
}
const jhEn = {
clientId: "—",
clientName: "—",
viaId: "7",
viaName: "msk-gw01",
enId: "9",
enName: "arn-gw01",
serviceId: "svc:cdn",
}
const enOnly = {
clientId: "—",
clientName: "—",
viaId: "9",
viaName: "arn-gw01",
enId: "9",
enName: "arn-gw01",
serviceId: "svc:cdn",
}
assert.equal(isUnboundServicePath(jhEn), true)
assert.equal(isUnboundServicePath(bound), false)
assert.equal(
formatServicePathLabel(jhEn, "via", { viaName: "msk-gw01.rtnt.top", enName: "arn-gw01.rtnt.top" }),
"msk-gw01.rtnt.top → arn-gw01.rtnt.top",
)
assert.equal(formatServicePathLabel(enOnly, "via"), "arn-gw01 · без привязки")
assert.equal(formatServicePathLabel(enOnly, "service"), "arn-gw01 · без привязки")
assert.equal(
formatServicePathLabel(bound, "via", { viaSite: "NSK" }),
"D · NSK",
)
assert.equal(
formatServicePathLabel(bound, "service", { serviceLabel: "CDN" }),
"D · CDN",
)
assert.equal(
formatServicePathTitle("msk-gw01 → arn-gw01", "CDN"),
"msk-gw01 → arn-gw01 · CDN",
)
console.log("format-service-path-label.test.ts: ok")
+45
View File
@@ -0,0 +1,45 @@
export const UNBOUND_PATH_CLIENT = "—"
export interface ServicePathLabelInput {
clientId: string
clientName: string
viaId: string
viaName: string
enId: string
enName: string
serviceId: string
}
export interface ServicePathLabelNames {
viaName?: string
viaSite?: string
enName?: string
serviceLabel?: string
}
export function isUnboundServicePath(p: Pick<ServicePathLabelInput, "clientId" | "clientName">): boolean {
return p.clientId === UNBOUND_PATH_CLIENT || p.clientName === UNBOUND_PATH_CLIENT
}
export function formatServicePathLabel(
p: ServicePathLabelInput,
viaMode: "via" | "service",
names: ServicePathLabelNames = {},
): string {
const viaName = names.viaName || p.viaName
const enName = names.enName || p.enName
if (isUnboundServicePath(p)) {
if (p.viaId !== p.enId) return `${viaName} → ${enName}`
return `${enName} · без привязки`
}
const viaLabel = names.viaSite || p.viaName
const mid = viaMode === "via" ? viaLabel : (names.serviceLabel || p.serviceId)
return `${p.clientName} · ${mid}`
}
export function formatServicePathTitle(label: string, serviceLabel: string): string {
const svc = serviceLabel.trim()
if (!svc) return label
if (label.includes(svc)) return label
return `${label} · ${svc}`
}
+5
View File
@@ -9,6 +9,7 @@ import {
routerCertificates, routerCertificates,
routerContainers, routerContainers,
servers, servers,
vxlanTunnels,
} from "@/lib/data" } from "@/lib/data"
/** Число мок-сессий BGP (см. `SESSIONS` в `app/(main)/bgp/page.tsx`). */ /** Число мок-сессий BGP (см. `SESSIONS` в `app/(main)/bgp/page.tsx`). */
@@ -41,6 +42,7 @@ export function mockSidebarBadgesByUrl(): Record<string, string> {
"/filters": formatSidebarBadgeCount(filters.length), "/filters": formatSidebarBadgeCount(filters.length),
"/wireguard": formatSidebarBadgeCount(mockWireGuardIfacesCount()), "/wireguard": formatSidebarBadgeCount(mockWireGuardIfacesCount()),
"/gre": formatSidebarBadgeCount(greTunnels.length), "/gre": formatSidebarBadgeCount(greTunnels.length),
"/vxlan": formatSidebarBadgeCount(vxlanTunnels.length),
"/containers": formatSidebarBadgeCount(routerContainers.length), "/containers": formatSidebarBadgeCount(routerContainers.length),
"/certificates": formatSidebarBadgeCount(routerCertificates.length), "/certificates": formatSidebarBadgeCount(routerCertificates.length),
"/bgp": formatSidebarBadgeCount(MOCK_BGP_SESSION_COUNT), "/bgp": formatSidebarBadgeCount(MOCK_BGP_SESSION_COUNT),
@@ -57,4 +59,7 @@ export interface SidebarCountsDto {
certificates?: number certificates?: number
wireguard?: number wireguard?: number
users?: number users?: number
bgpSessions?: number
vxlan?: number
containers?: number
} }
+31
View File
@@ -0,0 +1,31 @@
import type { StatisticsPivotDim } from "@mmapp/contracts/statistics"
import type { StatisticsSliceKind } from "@/components/data-grids/statistics-breakdown-data-grid"
export const STATISTICS_DIMS: {
id: StatisticsSliceKind
pivot: StatisticsPivotDim
label: string
}[] = [
{ id: "users", pivot: "user", label: "Пользователи" },
{ id: "servers", pivot: "server", label: "Серверы" },
{ id: "interfaces", pivot: "iface", label: "Интерфейсы" },
{ id: "countries", pivot: "country", label: "Страны" },
{ id: "services", pivot: "service", label: "Сервисы" },
{ id: "asns", pivot: "asn", label: "ASN" },
]
export function isStatisticsSliceKind(v: string): v is StatisticsSliceKind {
return STATISTICS_DIMS.some((d) => d.id === v)
}
export function isStatisticsPivotDim(v: string): v is StatisticsPivotDim {
return STATISTICS_DIMS.some((d) => d.pivot === v)
}
export function sliceKindToPivot(kind: StatisticsSliceKind): StatisticsPivotDim {
return STATISTICS_DIMS.find((d) => d.id === kind)?.pivot ?? "user"
}
export function pivotToSliceKind(dim: StatisticsPivotDim): StatisticsSliceKind {
return STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
}
+15
View File
@@ -14875,6 +14875,21 @@
"dependencies": { "dependencies": {
"zod": "^4.4.1" "zod": "^4.4.1"
} }
},
"node_modules/@next/swc-win32-x64-msvc": {
"version": "16.2.4",
"resolved": "https://registry.npmjs.org/@next/swc-win32-x64-msvc/-/swc-win32-x64-msvc-16.2.4.tgz",
"integrity": "sha512-kMVGgsqhO5YTYODD9IPGGhA6iprWidQckK3LmPeW08PIFENRmgfb4MjXHO+p//d+ts2rpjvK5gXWzXSMrPl9cw==",
"cpu": [
"x64"
],
"optional": true,
"os": [
"win32"
],
"engines": {
"node": ">= 10"
}
} }
} }
} }
+45
View File
@@ -1,5 +1,18 @@
import { z } from "zod" import { z } from "zod"
export const STATISTICS_UNBOUND_USER_ID = "__unbound__"
export const STATISTICS_WAN_MARK = "WAN · интернет"
export const STATISTICS_DUP_MARK = "дубль"
export const statisticsPivotDimSchema = z.enum([
"country",
"service",
"asn",
"server",
"user",
"iface",
])
export const statisticsBreakdownRowSchema = z.object({ export const statisticsBreakdownRowSchema = z.object({
id: z.string(), id: z.string(),
label: z.string(), label: z.string(),
@@ -34,6 +47,7 @@ export const statisticsQuerySchema = z.object({
country: z.string().min(2).max(2).optional(), country: z.string().min(2).max(2).optional(),
service: z.string().min(1).optional(), service: z.string().min(1).optional(),
asn: z.coerce.number().int().optional(), asn: z.coerce.number().int().optional(),
planes: z.enum(["unique", "all"]).default("unique"),
}) })
export const statisticsDtoSchema = z.object({ export const statisticsDtoSchema = z.object({
@@ -50,8 +64,39 @@ export const statisticsDtoSchema = z.object({
asns: z.array(statisticsBreakdownRowSchema), asns: z.array(statisticsBreakdownRowSchema),
}) })
export const statisticsPivotQuerySchema = statisticsQuerySchema.extend({
row: statisticsPivotDimSchema,
col: statisticsPivotDimSchema,
metric: z.enum(["bytes", "packets"]).default("bytes"),
})
export const statisticsPivotColumnSchema = z.object({
id: z.string(),
label: z.string(),
total: z.number().nonnegative(),
})
export const statisticsPivotRowSchema = z.object({
id: z.string(),
label: z.string(),
total: z.number().nonnegative(),
cells: z.record(z.string(), z.number().nonnegative()),
})
export const statisticsPivotDtoSchema = z.object({
rowDim: statisticsPivotDimSchema,
colDim: statisticsPivotDimSchema,
metric: z.enum(["bytes", "packets"]),
columns: z.array(statisticsPivotColumnSchema),
rows: z.array(statisticsPivotRowSchema),
otherBytes: z.number().nonnegative(),
})
export type StatisticsBreakdownRow = z.infer<typeof statisticsBreakdownRowSchema> export type StatisticsBreakdownRow = z.infer<typeof statisticsBreakdownRowSchema>
export type StatisticsSeriesPoint = z.infer<typeof statisticsSeriesPointSchema> export type StatisticsSeriesPoint = z.infer<typeof statisticsSeriesPointSchema>
export type StatisticsKpis = z.infer<typeof statisticsKpisSchema> export type StatisticsKpis = z.infer<typeof statisticsKpisSchema>
export type StatisticsQuery = z.infer<typeof statisticsQuerySchema> export type StatisticsQuery = z.infer<typeof statisticsQuerySchema>
export type StatisticsDto = z.infer<typeof statisticsDtoSchema> export type StatisticsDto = z.infer<typeof statisticsDtoSchema>
export type StatisticsPivotDim = z.infer<typeof statisticsPivotDimSchema>
export type StatisticsPivotQuery = z.infer<typeof statisticsPivotQuerySchema>
export type StatisticsPivotDto = z.infer<typeof statisticsPivotDtoSchema>
+8
View File
@@ -252,6 +252,13 @@ export const flowPurgeDtoSchema = z.object({
vacuumed: z.boolean(), vacuumed: z.boolean(),
}) })
export const flowFactsRebuildDtoSchema = z.object({
ok: z.literal(true),
buckets: z.number().int().nonnegative(),
facts: z.number().int().nonnegative(),
days: z.array(z.string()),
})
export const flowMapHopKindSchema = z.enum(["gre", "wan", "iface"]) export const flowMapHopKindSchema = z.enum(["gre", "wan", "iface"])
export const flowMapHopDtoSchema = z.object({ export const flowMapHopDtoSchema = z.object({
@@ -330,6 +337,7 @@ export type FlowExportersDto = z.infer<typeof flowExportersDtoSchema>
export type FlowClientsDto = z.infer<typeof flowClientsDtoSchema> export type FlowClientsDto = z.infer<typeof flowClientsDtoSchema>
export type FlowMonthlyDto = z.infer<typeof flowMonthlyDtoSchema> export type FlowMonthlyDto = z.infer<typeof flowMonthlyDtoSchema>
export type FlowPurgeDto = z.infer<typeof flowPurgeDtoSchema> export type FlowPurgeDto = z.infer<typeof flowPurgeDtoSchema>
export type FlowFactsRebuildDto = z.infer<typeof flowFactsRebuildDtoSchema>
export type FlowMapHopKind = z.infer<typeof flowMapHopKindSchema> export type FlowMapHopKind = z.infer<typeof flowMapHopKindSchema>
export type FlowMapHop = z.infer<typeof flowMapHopDtoSchema> export type FlowMapHop = z.infer<typeof flowMapHopDtoSchema>
export type FlowMapService = z.infer<typeof flowMapServiceDtoSchema> export type FlowMapService = z.infer<typeof flowMapServiceDtoSchema>
+29 -2
View File
@@ -1,7 +1,13 @@
import type { StatisticsDto, StatisticsQuery } from "@mmapp/contracts/statistics" import type {
StatisticsDto,
StatisticsPivotDto,
StatisticsPivotQuery,
StatisticsQuery,
} from "@mmapp/contracts/statistics"
import { requestJson } from "@/shared/api/http-client" import { requestJson } from "@/shared/api/http-client"
export type { StatisticsDto, StatisticsQuery } export type { StatisticsDto, StatisticsQuery, StatisticsPivotDto, StatisticsPivotQuery }
export { STATISTICS_UNBOUND_USER_ID, STATISTICS_WAN_MARK, STATISTICS_DUP_MARK } from "@mmapp/contracts/statistics"
export async function getStatistics( export async function getStatistics(
baseUrl: string, baseUrl: string,
@@ -16,5 +22,26 @@ export async function getStatistics(
if (query.country) params.set("country", query.country) if (query.country) params.set("country", query.country)
if (query.service) params.set("service", query.service) if (query.service) params.set("service", query.service)
if (query.asn != null) params.set("asn", String(query.asn)) if (query.asn != null) params.set("asn", String(query.asn))
if (query.planes && query.planes !== "unique") params.set("planes", query.planes)
return requestJson<StatisticsDto>(baseUrl, `/api/statistics?${params.toString()}`) return requestJson<StatisticsDto>(baseUrl, `/api/statistics?${params.toString()}`)
} }
export async function getStatisticsPivot(
baseUrl: string,
query: StatisticsPivotQuery,
): Promise<StatisticsPivotDto> {
const params = new URLSearchParams()
params.set("from", query.from)
params.set("to", query.to)
params.set("row", query.row)
params.set("col", query.col)
params.set("metric", query.metric)
if (query.serverId != null) params.set("serverId", String(query.serverId))
if (query.userId) params.set("userId", query.userId)
if (query.iface) params.set("iface", query.iface)
if (query.country) params.set("country", query.country)
if (query.service) params.set("service", query.service)
if (query.asn != null) params.set("asn", String(query.asn))
if (query.planes && query.planes !== "unique") params.set("planes", query.planes)
return requestJson<StatisticsPivotDto>(baseUrl, `/api/statistics/pivot?${params.toString()}`)
}
+9
View File
@@ -141,4 +141,13 @@ export async function purgeTrafficFlowData(baseUrl: string): Promise<FlowPurgeDt
return requestJson<FlowPurgeDto>(baseUrl, "/api/traffic/flow/purge", { method: "POST" }) return requestJson<FlowPurgeDto>(baseUrl, "/api/traffic/flow/purge", { method: "POST" })
} }
export async function rebuildTrafficFlowFacts(baseUrl: string): Promise<{
ok: true
buckets: number
facts: number
days: string[]
}> {
return requestJson(baseUrl, "/api/traffic/flow/rebuild-facts", { method: "POST" })
}
export { flowQuery } export { flowQuery }
+1 -1
View File
@@ -1 +1 @@
{"fileNames":["./node_modules/typescript/lib/lib.es5.d.ts","./node_modules/typescript/lib/lib.es2015.d.ts","./node_modules/typescript/lib/lib.es2016.d.ts","./node_modules/typescript/lib/lib.es2017.d.ts","./node_modules/typescript/lib/lib.es2018.d.ts","./node_modules/typescript/lib/lib.es2019.d.ts","./node_modules/typescript/lib/lib.es2020.d.ts","./node_modules/typescript/lib/lib.es2021.d.ts","./node_modules/typescript/lib/lib.es2022.d.ts","./node_modules/typescript/lib/lib.es2023.d.ts","./node_modules/typescript/lib/lib.es2024.d.ts","./node_modules/typescript/lib/lib.esnext.d.ts","./node_modules/typescript/lib/lib.dom.d.ts","./node_modules/typescript/lib/lib.dom.iterable.d.ts","./node_modules/typescript/lib/lib.es2015.core.d.ts","./node_modules/typescript/lib/lib.es2015.collection.d.ts","./node_modules/typescript/lib/lib.es2015.generator.d.ts","./node_modules/typescript/lib/lib.es2015.iterable.d.ts","./node_modules/typescript/lib/lib.es2015.promise.d.ts","./node_modules/typescript/lib/lib.es2015.proxy.d.ts","./node_modules/typescript/lib/lib.es2015.reflect.d.ts","./node_modules/typescript/lib/lib.es2015.symbol.d.ts","./node_modules/typescript/lib/lib.es2015.symbol.wellknown.d.ts","./node_modules/typescript/lib/lib.es2016.array.include.d.ts","./node_modules/typescript/lib/lib.es2016.intl.d.ts","./node_modules/typescript/lib/lib.es2017.arraybuffer.d.ts","./node_modules/typescript/lib/lib.es2017.date.d.ts","./node_modules/typescript/lib/lib.es2017.object.d.ts","./node_modules/typescript/lib/lib.es2017.sharedmemory.d.ts","./node_modules/typescript/lib/lib.es2017.string.d.ts","./node_modules/typescript/lib/lib.es2017.intl.d.ts","./node_modules/typescript/lib/lib.es2017.typedarrays.d.ts","./node_modules/typescript/lib/lib.es2018.asyncgenerator.d.ts","./node_modules/typescript/lib/lib.es2018.asynciterable.d.ts","./node_modules/typescript/lib/lib.es2018.intl.d.ts","./node_modules/typescript/lib/lib.es2018.promise.d.ts","./node_modules/typescript/lib/lib.es2018.regexp.d.ts","./node_modules/typescript/lib/lib.es2019.array.d.ts","./node_modules/typescript/lib/lib.es2019.object.d.ts","./node_modules/typescript/lib/lib.es2019.string.d.ts","./node_modules/typescript/lib/lib.es2019.symbol.d.ts","./node_modules/typescript/lib/lib.es2019.intl.d.ts","./node_modules/typescript/lib/lib.es2020.bigint.d.ts","./node_modules/typescript/lib/lib.es2020.date.d.ts","./node_modules/typescript/lib/lib.es2020.promise.d.ts","./node_modules/typescript/lib/lib.es2020.sharedmemory.d.ts","./node_modules/typescript/lib/lib.es2020.string.d.ts","./node_modules/typescript/lib/lib.es2020.symbol.wellknown.d.ts","./node_modules/typescript/lib/lib.es2020.intl.d.ts","./node_modules/typescript/lib/lib.es2020.number.d.ts","./node_modules/typescript/lib/lib.es2021.promise.d.ts","./node_modules/typescript/lib/lib.es2021.string.d.ts","./node_modules/typescript/lib/lib.es2021.weakref.d.ts","./node_modules/typescript/lib/lib.es2021.intl.d.ts","./node_modules/typescript/lib/lib.es2022.array.d.ts","./node_modules/typescript/lib/lib.es2022.error.d.ts","./node_modules/typescript/lib/lib.es2022.intl.d.ts","./node_modules/typescript/lib/lib.es2022.object.d.ts","./node_modules/typescript/lib/lib.es2022.string.d.ts","./node_modules/typescript/lib/lib.es2022.regexp.d.ts","./node_modules/typescript/lib/lib.es2023.array.d.ts","./node_modules/typescript/lib/lib.es2023.collection.d.ts","./node_modules/typescript/lib/lib.es2023.intl.d.ts","./node_modules/typescript/lib/lib.es2024.arraybuffer.d.ts","./node_modules/typescript/lib/lib.es2024.collection.d.ts","./node_modules/typescript/lib/lib.es2024.object.d.ts","./node_modules/typescript/lib/lib.es2024.promise.d.ts","./node_modules/typescript/lib/lib.es2024.regexp.d.ts","./node_modules/typescript/lib/lib.es2024.sharedmemory.d.ts","./node_modules/typescript/lib/lib.es2024.string.d.ts","./node_modules/typescript/lib/lib.esnext.array.d.ts","./node_modules/typescript/lib/lib.esnext.collection.d.ts","./node_modules/typescript/lib/lib.esnext.intl.d.ts","./node_modules/typescript/lib/lib.esnext.disposable.d.ts","./node_modules/typescript/lib/lib.esnext.promise.d.ts","./node_modules/typescript/lib/lib.esnext.decorators.d.ts","./node_modules/typescript/lib/lib.esnext.iterator.d.ts","./node_modules/typescript/lib/lib.esnext.float16.d.ts","./node_modules/typescript/lib/lib.esnext.error.d.ts","./node_modules/typescript/lib/lib.esnext.sharedmemory.d.ts","./node_modules/typescript/lib/lib.decorators.d.ts","./node_modules/typescript/lib/lib.decorators.legacy.d.ts","./node_modules/@types/react/global.d.ts","./node_modules/csstype/index.d.ts","./node_modules/@types/react/index.d.ts","./node_modules/next/dist/styled-jsx/types/css.d.ts","./node_modules/next/dist/styled-jsx/types/macro.d.ts","./node_modules/next/dist/styled-jsx/types/style.d.ts","./node_modules/next/dist/styled-jsx/types/global.d.ts","./node_modules/next/dist/styled-jsx/types/index.d.ts","./node_modules/next/dist/server/get-page-files.d.ts","./node_modules/@types/node/compatiLine truncated {"fileNames":["./node_modules/typescript/lib/lib.es5.d.ts","./node_modules/typescript/lib/lib.es2015.d.ts","./node_modules/typescript/lib/lib.es2016.d.ts","./node_modules/typescript/lib/lib.es2017.d.ts","./node_modules/typescript/lib/lib.es2018.d.ts","./node_modules/typescript/lib/lib.es2019.d.ts","./node_modules/typescript/lib/lib.es2020.d.ts","./node_modules/typescript/lib/lib.es2021.d.ts","./node_modules/typescript/lib/lib.es2022.d.ts","./node_modules/typescript/lib/lib.es2023.d.ts","./node_modules/typescript/lib/lib.es2024.d.ts","./node_modules/typescript/lib/lib.esnext.d.ts","./node_modules/typescript/lib/lib.dom.d.ts","./node_modules/typescript/lib/lib.dom.iterable.d.ts","./node_modules/typescript/lib/lib.es2015.core.d.ts","./node_modules/typescript/lib/lib.es2015.collection.d.ts","./node_modules/typescript/lib/lib.es2015.generator.d.ts","./node_modules/typescript/lib/lib.es2015.iterable.d.ts","./node_modules/typescript/lib/lib.es2015.promise.d.ts","./node_modules/typescript/lib/lib.es2015.proxy.d.ts","./node_modules/typescript/lib/lib.es2015.reflect.d.ts","./node_modules/typescript/lib/lib.es2015.symbol.d.ts","./node_modules/typescript/lib/lib.es2015.symbol.wellknown.d.ts","./node_modules/typescript/lib/lib.es2016.array.include.d.ts","./node_modules/typescript/lib/lib.es2016.intl.d.ts","./node_modules/typescript/lib/lib.es2017.arraybuffer.d.ts","./node_modules/typescript/lib/lib.es2017.date.d.ts","./node_modules/typescript/lib/lib.es2017.object.d.ts","./node_modules/typescript/lib/lib.es2017.sharedmemory.d.ts","./node_modules/typescript/lib/lib.es2017.string.d.ts","./node_modules/typescript/lib/lib.es2017.intl.d.ts","./node_modules/typescript/lib/lib.es2017.typedarrays.d.ts","./node_modules/typescript/lib/lib.es2018.asyncgenerator.d.ts","./node_modules/typescript/lib/lib.es2018.asynciterable.d.ts","./node_modules/typescript/lib/lib.es2018.intl.d.ts","./node_modules/typescript/lib/lib.es2018.promise.d.ts","./node_modules/typescript/lib/lib.es2018.regexp.d.ts","./node_modules/typescript/lib/lib.es2019.array.d.ts","./node_modules/typescript/lib/lib.es2019.object.d.ts","./node_modules/typescript/lib/lib.es2019.string.d.ts","./node_modules/typescript/lib/lib.es2019.symbol.d.ts","./node_modules/typescript/lib/lib.es2019.intl.d.ts","./node_modules/typescript/lib/lib.es2020.bigint.d.ts","./node_modules/typescript/lib/lib.es2020.date.d.ts","./node_modules/typescript/lib/lib.es2020.promise.d.ts","./node_modules/typescript/lib/lib.es2020.sharedmemory.d.ts","./node_modules/typescript/lib/lib.es2020.string.d.ts","./node_modules/typescript/lib/lib.es2020.symbol.wellknown.d.ts","./node_modules/typescript/lib/lib.es2020.intl.d.ts","./node_modules/typescript/lib/lib.es2020.number.d.ts","./node_modules/typescript/lib/lib.es2021.promise.d.ts","./node_modules/typescript/lib/lib.es2021.string.d.ts","./node_modules/typescript/lib/lib.es2021.weakref.d.ts","./node_modules/typescript/lib/lib.es2021.intl.d.ts","./node_modules/typescript/lib/lib.es2022.array.d.ts","./node_modules/typescript/lib/lib.es2022.error.d.ts","./node_modules/typescript/lib/lib.es2022.intl.d.ts","./node_modules/typescript/lib/lib.es2022.object.d.ts","./node_modules/typescript/lib/lib.es2022.string.d.ts","./node_modules/typescript/lib/lib.es2022.regexp.d.ts","./node_modules/typescript/lib/lib.es2023.array.d.ts","./node_modules/typescript/lib/lib.es2023.collection.d.ts","./node_modules/typescript/lib/lib.es2023.intl.d.ts","./node_modules/typescript/lib/lib.es2024.arraybuffer.d.ts","./node_modules/typescript/lib/lib.es2024.collection.d.ts","./node_modules/typescript/lib/lib.es2024.object.d.ts","./node_modules/typescript/lib/lib.es2024.promise.d.ts","./node_modules/typescript/lib/lib.es2024.regexp.d.ts","./node_modules/typescript/lib/lib.es2024.sharedmemory.d.ts","./node_modules/typescript/lib/lib.es2024.string.d.ts","./node_modules/typescript/lib/lib.esnext.array.d.ts","./node_modules/typescript/lib/lib.esnext.collection.d.ts","./node_modules/typescript/lib/lib.esnext.intl.d.ts","./node_modules/typescript/lib/lib.esnext.disposable.d.ts","./node_modules/typescript/lib/lib.esnext.promise.d.ts","./node_modules/typescript/lib/lib.esnext.decorators.d.ts","./node_modules/typescript/lib/lib.esnext.iterator.d.ts","./node_modules/typescript/lib/lib.esnext.float16.d.ts","./node_modules/typescript/lib/lib.esnext.error.d.ts","./node_modules/typescript/lib/lib.esnext.sharedmemory.d.ts","./node_modules/typescript/lib/lib.decorators.d.ts","./node_modules/typescript/lib/lib.decorators.legacy.d.ts","./node_modules/@types/react/global.d.ts","./node_modules/csstype/index.d.ts","./node_modules/@types/react/index.d.ts","./node_modules/next/dist/styled-jsx/types/css.d.ts","./node_modules/next/dist/styled-jsx/types/macro.d.ts","./node_modules/next/dist/styled-jsx/types/style.d.ts","./node_modules/next/dist/styled-jsx/types/global.d.ts","./node_modules/next/dist/styled-jsx/types/index.d.ts","./node_modules/next/dist/server/get-page-files.d.ts","./node_modules/@types/node/compatiLine truncated