Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5750590b68 | ||
|
|
3c42c114f5 | ||
|
|
5aef419582 | ||
|
|
0c0dfa1df7 | ||
|
|
c162a41bc0 | ||
|
|
97e43b2335 | ||
|
|
db21e1217c | ||
|
|
5bb9066be8 | ||
|
|
b1fd259f10 | ||
|
|
3687bb8fa2 | ||
|
|
a80caf5676 | ||
|
|
d39e3454aa | ||
|
|
a8f2055c77 | ||
|
|
cdeb97d841 | ||
|
|
36c5305db7 | ||
|
|
5f2b4e2d40 | ||
|
|
cff26813b9 | ||
|
|
39c8ec4a02 | ||
|
|
c6c859a495 |
@@ -0,0 +1,63 @@
|
||||
# Локальные GeoLite2-базы (Country + ASN) для потоков по странам и ASN
|
||||
|
||||
## Контекст
|
||||
|
||||
Сейчас страна и ASN для netflow-потоков резолвятся через внешний RIPEstat API (`backend/src/services/traffic-flow-ripe.ts`): лимит 30 новых префиксов/мин, очередь на 90, кэш в PG `flow_ip_meta`. Новые IP «дозревают» с задержкой, IPv6 не покрывается (кэш индексируется только по IPv4). Локальные mmdb-базы дают мгновенный синхронный lookup всех IP без внешних вызовов.
|
||||
|
||||
Решения (подтверждены):
|
||||
- Источник — **MaxMind GeoLite2 через P3TERX-зеркало**: `https://github.com/P3TERX/GeoLite.mmdb/raw/download/GeoLite2-Country.mmdb` и `.../GeoLite2-ASN.mmdb`. Без регистрации, ключей и tar-распаковки. Точность по стране у GeoLite2 и IPinfo паритетная (<1% ошибок у обоих, arXiv 2026); выбран P3TERX за надёжность зеркала (5.2k звёзд) и преемственность: текущий RIPE-путь и так читает GeoLite (`maxmind-geo-lite`), история в кэше остаётся консистентной.
|
||||
- **RIPEstat остаётся fallback** (до первой загрузки баз / если lookup не дал результата).
|
||||
- City-базу не качаем (lat/lng фронтенд не использует).
|
||||
|
||||
## Изменения
|
||||
|
||||
### 1. Зависимость
|
||||
- `npm install -w mikrotik-manager-backend maxmind` — sync-чтение mmdb, встроенные TS-типы, без транзитивных зависимостей, Node 22 ок.
|
||||
|
||||
### 2. Новый сервис `backend/src/services/traffic-flow-geoip.ts`
|
||||
(по конвенциям окружения traffic-flow-*: контракт → маршрут → сервис, без БД-логики в маршрутах)
|
||||
- Каталог: `backend/storage/geoip/` (конвенция `storage/backups`), файлы `GeoLite2-Country.mmdb`, `GeoLite2-ASN.mmdb`.
|
||||
- `initGeoip()` — открыть ридеры best-effort при старте (из `index.ts` рядом с `startTrafficFlowListener`), независимо от настроек автообновления: файлы есть — работают.
|
||||
- `lookupGeoip(ip): FlowIpMeta | null` — синхронно: `country.iso_code` (fallback `registered_country.iso_code`) с валидацией `isIsoCountry`, ASN = `autonomous_system_number`, holder = `autonomous_system_organization`; приватные IP → negative-запись как в RIPE (`isNonPublicIp`); в PG не пишем (lookup и так быстрый). IPv6 поддержан ридером.
|
||||
- `resolveFlowIp(ip)` — фасад: `lookupGeoip(ip) ?? lookupRipeCached(ip)`; главный экспорт для потребителей.
|
||||
- `geoipStatus()` — loaded, даты сборки баз (метаданные mmdb). Тест-хук `setGeoipReadersForTests`. Смена ридеров после обновления — атомарная замена ссылок.
|
||||
|
||||
### 3. Коллектор `backend/src/services/geoip-update-collector.ts`
|
||||
`collectGeoipUpdateOnce()` по образцу `certificate-renew-collector.ts`:
|
||||
1. Conditional GET с ETag/If-None-Match из настроек → 304 = skip (фолбэк-сравнение: размер/содержимое).
|
||||
2. Скачивание в `*.tmp` через глобальный `fetch` + AbortController с таймаутом (внешний HTTP из service-слоя — по правилу fastify-backend-drizzle).
|
||||
3. Валидация: открыть ридер из tmp-файла, пробой 8.8.8.8 (страна US, ASN 15169).
|
||||
4. `fs.rename` атомарная подмена, старый файл → `*.prev` (откат, если новый ридер не открылся).
|
||||
5. Перезагрузка ридеров, статус в настройках; snapshot для `scheduler_runs` (checked/downloaded/skipped/bytes/error).
|
||||
|
||||
### 4. Планировщик (`backend/src/services/scheduler.ts`)
|
||||
- `JOB_KEYS` += `geoip_update`; case в `runSchedulerJobBody`; блок в `refreshScheduler()` по образцу `certificates_renew`: интервал `Math.max(6ч, updateIntervalSec*1000)`, по умолчанию 7 дней (upstream обновляется еженедельно) + немедленный первый запуск при включённой настройке.
|
||||
|
||||
### 5. Схема и миграция
|
||||
- `backend/src/db/schema.ts`: singleton `geoip_settings` — `enabled` (default true), `updateIntervalSec` (default 604800), `lastCheckAt`, `lastSuccessAt`, `lastError`, `countryBuildAt`, `asnBuildAt`, `etagsJson` (jsonb), `createdAt/updatedAt`.
|
||||
- Миграция: `npm run db:generate` → файл в `backend/drizzle/`.
|
||||
|
||||
### 6. API + контракты
|
||||
- `packages/contracts/src/geoip.ts`: zod-схемы настроек/статуса (все входы — Zod, по правилам проекта).
|
||||
- Новый `backend/src/routes/geoip.ts`, регистрация в `index.ts` с prefix `/api`:
|
||||
- `GET /api/geoip` — настройки + статус (ready, даты сборки, последняя проверка/ошибка);
|
||||
- `PUT /api/geoip` — сохранить настройки, затем `refreshScheduler()`;
|
||||
- `POST /api/geoip/update` — запустить загрузку сейчас (409, если уже идёт; флаг-гард как в коллекторах).
|
||||
|
||||
### 7. Интеграция в пайплайн (geoip-first, RIPE-fallback)
|
||||
- `traffic-flow-engine.ts` (`queueParsedFlows`, ~строка 336): `lookupRipeCached` → `resolveFlowIp`. Логика misses не меняется: при готовом mmdb публичные IP (v4+v6) резолвятся сразу, очередь RIPE пустеет; до скачивания баз — прежнее поведение.
|
||||
- Остальные вызовы `lookupRipeCached` → `resolveFlowIp` (grep: как минимум `traffic-flow-analytics.ts` ~258–271).
|
||||
- `classifyFlowDst`/бренды не трогаем: holder из mmdb (org name) встаёт в существующие `HOLDER_BRANDS`-регулярки как есть.
|
||||
|
||||
### 8. Frontend (по next-shadcn-production / ui-guardian: только переиспользование)
|
||||
- Секция «GeoIP-базы (GeoLite2)» внутри существующей `components/traffic/netflow-settings-panel.tsx`: статус (готово/не скачано, даты сборки Country/ASN, последняя проверка, ошибка), тумблер автообновления, интервал, кнопка «Обновить сейчас» с индикатором. Только уже используемые в панели примитивы (Switch/Button/поля) — никаких новых визуальных паттернов и Card-shell. API-клиент через существующие http-хелперы.
|
||||
|
||||
### 9. Хаускипинг, тесты, проверки
|
||||
- `backend/.gitignore`: `storage/geoip/`.
|
||||
- Тесты `backend/src/services/traffic-flow-geoip.test.ts` + скрипт `test:geoip` (по образцу `test:traffic-flow`): приоритет фасада (geoip hit → RIPE не зовётся; miss → fallback), negative на приватных IP, фильтрация EU/ZZ через `isIsoCountry`, коллектор с мокнутым fetch (304-skip, битый файл → подмены нет, `.prev` сохранён), dims по стране/ASN с засеянным ридером.
|
||||
- Проверки после реализации (обязательно по правилам): типы/сборка бэка (`npm run build -w mikrotik-manager-backend`), типы фронта при правке UI (`npx tsc --noEmit`), `npm run test:geoip` и `test:traffic-flow`; предупреждения не игнорировать.
|
||||
- Коммит: `feat(netflow): <subject по-русски>` — новая пользовательская фича (мгновенные страна/ASN в потоках), по commit-messages-ru.
|
||||
- README: короткий раздел о GeoIP; примечание, что в Docker `storage/geoip` ephemeral без тома — базы перекачаются после пересоздания контейнера (~17 МБ); при желании смонтировать volume.
|
||||
|
||||
## Что это даёт
|
||||
Страна и ASN появляются у потока мгновенно при ingest (включая IPv6), без ограничения скорости RIPE; dims `country`/`asn` в `flow_daily_dims`, аналитика (карта, топы, monthly) становятся полными сразу. Внешняя зависимость от stat.ripe.net остаётся только как fallback до первой загрузки баз.
|
||||
@@ -190,6 +190,14 @@ npm run build -w @mmapp/contracts
|
||||
npm --prefix backend run db:migrate-from-sqlite
|
||||
```
|
||||
|
||||
### GeoIP-базы GeoLite2 (страны и ASN для NetFlow)
|
||||
|
||||
Backend держит локальные mmdb-базы MaxMind GeoLite2 (Country + ASN) в `backend/storage/geoip/` и скачивает их с зеркала [P3TERX/GeoLite.mmdb](https://github.com/P3TERX/GeoLite.mmdb) — без регистрации и ключей. Lookup страны/ASN потока при ingest становится мгновенным (включая IPv6) и не упирается в лимиты RIPEstat; пока базы не скачаны или lookup промахнулся, работает прежний RIPE-fallback.
|
||||
|
||||
Управление — секция «GeoIP-базы (GeoLite2)» в настройках NetFlow (страница «Сбор данных»): автообновление (по умолчанию проверка раз в 7 дней, upstream обновляется еженедельно), статус сборки баз и кнопка «Обновить сейчас». Джоба планировщика — `geoip_update`. Атрибуция: данные MaxMind GeoLite2, CC BY-SA 4.0.
|
||||
|
||||
Примечание для Docker: каталог `storage/geoip` внутри контейнера ephemeral — без смонтированного volume базы (~17 МБ) перекачаются после пересоздания контейнера. Каталог переопределяется переменной `GEOIP_DIR`.
|
||||
|
||||
## CI/CD (Gitea Actions)
|
||||
|
||||
Файл: `.gitea/workflows/docker.yml` (имя workflow: **Docker images**).
|
||||
|
||||
+309
-583
File diff suppressed because it is too large
Load Diff
@@ -11,6 +11,7 @@ import { OpsPanel } from "@/components/ops-panel"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { CertificatesDataGrid } from "@/components/data-grids/certificates-data-grid"
|
||||
import { CertificateRenewSettingsPanel } from "@/components/certificates/certificate-renew-settings"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { ServerRailLayout, ServerRailMobileButton } from "@/components/server-rail-layout"
|
||||
import { ALL_SERVERS_ID, type ServerTileItem } from "@/components/server-tile-rail"
|
||||
@@ -243,7 +244,7 @@ function CertPartReference() {
|
||||
return (
|
||||
<OpsPanel
|
||||
title="RouterOS 7 · /certificate — справка CLI"
|
||||
description="RouterOS 7.22+ · публичные LE для Cloudflare через backend DNS-01, не через /certificate add-acme на устройстве."
|
||||
description="RouterOS 7 умеет обновлять Let's Encrypt сам. Этот CLI — справка; автообновление MM включается панелью выше."
|
||||
contentClassName="px-5 py-4"
|
||||
>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-3 gap-4 text-xs font-mono">
|
||||
@@ -715,6 +716,8 @@ export default function CertificatesPage() {
|
||||
|
||||
<CertPartKpi displayCerts={scopedCerts} expiring={expiring} expired={expired} />
|
||||
|
||||
<CertificateRenewSettingsPanel backendUrl={backendUrl} liveReady={liveReady} />
|
||||
|
||||
{liveReady && (
|
||||
<CertPartAcmeSettings
|
||||
acmeDirectoryUrl={acmeDirectoryUrl}
|
||||
|
||||
+260
-851
File diff suppressed because it is too large
Load Diff
@@ -34,6 +34,7 @@ import {
|
||||
type InternetPathRunSnapshot,
|
||||
type CertificatesRenewRunSnapshot,
|
||||
type BackupsRunSnapshot,
|
||||
type GeoipUpdateRunSnapshot,
|
||||
type PingRunSnapshot,
|
||||
type ResourcesRunSnapshot,
|
||||
type SchedulerRunSnapshot,
|
||||
@@ -340,6 +341,41 @@ function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (snap.job === "geoip_update") {
|
||||
const g = snap as GeoipUpdateRunSnapshot
|
||||
return (
|
||||
<div className="flex flex-col gap-3">
|
||||
{g.skipped ? (
|
||||
<p className="text-xs text-amber-600 dark:text-amber-400">Прогон пропущен: обновление уже выполнялось или задача отключена.</p>
|
||||
) : null}
|
||||
<dl className="grid grid-cols-2 gap-3 text-xs sm:grid-cols-4">
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Баз проверено</dt>
|
||||
<dd className="font-mono font-medium">{g.checked}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Скачано</dt>
|
||||
<dd className="font-mono font-medium">{g.downloaded}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Без изменений</dt>
|
||||
<dd className="font-mono font-medium">{g.skippedUnchanged}</dd>
|
||||
</div>
|
||||
<div>
|
||||
<dt className="text-muted-foreground">Объём, МБ</dt>
|
||||
<dd className="font-mono font-medium">{(g.bytes / 1024 / 1024).toFixed(1)}</dd>
|
||||
</div>
|
||||
</dl>
|
||||
{g.errors.length ? (
|
||||
<div className="rounded-md border border-amber-500/40 bg-amber-500/10 px-3 py-2 text-xs text-amber-800 dark:text-amber-200 flex flex-col gap-1">
|
||||
{g.errors.map((e, i) => (
|
||||
<p key={i} className="break-words">{e}</p>
|
||||
))}
|
||||
</div>
|
||||
) : null}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (snap.job === "alert_engine") {
|
||||
const a = snap as AlertEngineRunSnapshot
|
||||
return (
|
||||
@@ -1127,7 +1163,11 @@ export default function DataCollectionPage() {
|
||||
onChange={(e) => setRenewBeforeDaysDraft(e.target.value)}
|
||||
className="h-8 text-sm"
|
||||
inputMode="numeric"
|
||||
disabled={!draftCertRenewEnabled}
|
||||
/>
|
||||
<p className="text-[11px] text-muted-foreground mt-1">
|
||||
Вкл/выкл автообновления MM — также на странице «Сертификаты». Не включайте вместе со встроенным ACME RouterOS.
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
<p className="text-[11px] text-muted-foreground leading-snug">
|
||||
|
||||
@@ -67,6 +67,7 @@ import {
|
||||
CableIcon, CopyIcon, ActivityIcon, ExternalLinkIcon,
|
||||
} from "lucide-react"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { formatServicePathLabel, formatServicePathTitle } from "@/lib/format-service-path-label"
|
||||
import Link from "next/link"
|
||||
import { Flag } from "@/components/flag"
|
||||
|
||||
@@ -820,9 +821,15 @@ function ServicePathList({
|
||||
{paths.map((p) => {
|
||||
const rowKey = servicePathKey(p)
|
||||
const via = servers.find((s) => s.id === p.viaId)
|
||||
const viaLabel = via?.site || p.viaName
|
||||
const en = servers.find((s) => s.id === p.enId)
|
||||
const svc = services.find((s) => s.id === p.serviceId)
|
||||
const mid = viaMode === "via" ? viaLabel : (svc?.label ?? p.serviceId)
|
||||
const label = formatServicePathLabel(p, viaMode, {
|
||||
viaName: via?.name,
|
||||
viaSite: via?.site,
|
||||
enName: en?.name,
|
||||
serviceLabel: svc?.label,
|
||||
})
|
||||
const title = formatServicePathTitle(label, svc?.label ?? p.serviceId)
|
||||
const active = Boolean(
|
||||
highlight
|
||||
&& highlight.viaId === p.viaId
|
||||
@@ -833,13 +840,14 @@ function ServicePathList({
|
||||
<button
|
||||
key={rowKey}
|
||||
type="button"
|
||||
title={title}
|
||||
onClick={() => onToggle(p)}
|
||||
className={cn(
|
||||
"flex items-center justify-between gap-2 rounded-md px-2 py-1.5 text-left text-xs transition-colors",
|
||||
active ? "bg-cyan-500/15 ring-1 ring-cyan-500/40" : "hover:bg-muted/50",
|
||||
)}
|
||||
>
|
||||
<span className="font-mono truncate min-w-0">{p.clientName} · {mid}</span>
|
||||
<span className="font-mono truncate min-w-0">{label}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums shrink-0">
|
||||
{formatNetflowRate({ bytes: p.bytes, bps: p.bps, bpsFwd: p.bps, bpsRev: 0 })}
|
||||
</span>
|
||||
@@ -1216,6 +1224,9 @@ export default function NetworkMapPage() {
|
||||
// ── Interaction ─────────────────────────────────────────────────────────────
|
||||
const [selected, setSelected] = useState<Server | null>(null)
|
||||
const [selectedService, setSelectedService] = useState<FlowMapService | null>(null)
|
||||
const liveSelectedService = selectedService
|
||||
? (mapServices.find((s) => s.id === selectedService.id) ?? selectedService)
|
||||
: null
|
||||
const [highlightedPath, setHighlightedPath] = useState<{ viaId: string; enId: string; serviceId: string } | null>(null)
|
||||
const [selWanIdx, setSelWanIdx] = useState<number | null>(null)
|
||||
const [hoveredId, setHoveredId] = useState<string | null>(null)
|
||||
@@ -2702,16 +2713,16 @@ export default function NetworkMapPage() {
|
||||
})()}
|
||||
</div>
|
||||
</>
|
||||
) : selectedService ? (
|
||||
) : liveSelectedService ? (
|
||||
<>
|
||||
<div className="flex items-start gap-2 px-4 py-3 border-b">
|
||||
<div className="mt-0.5">
|
||||
<ServiceBrandIcon label={selectedService.label} size={22} />
|
||||
<ServiceBrandIcon label={liveSelectedService.label} size={22} />
|
||||
</div>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="font-mono font-semibold text-sm truncate">{selectedService.label}</p>
|
||||
<p className="font-mono font-semibold text-sm truncate">{liveSelectedService.label}</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">
|
||||
Конечный сервис · {selectedService.category}
|
||||
Конечный сервис · {liveSelectedService.category}
|
||||
</p>
|
||||
</div>
|
||||
<button
|
||||
@@ -2726,15 +2737,15 @@ export default function NetworkMapPage() {
|
||||
<div className="flex flex-col gap-0">
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Доля окна</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(selectedService.share)}</span>
|
||||
<span className="text-xs font-mono font-medium text-cyan-400">{serviceSharePct(liveSelectedService.share)}</span>
|
||||
</div>
|
||||
<div className="flex items-center justify-between py-2 border-b border-border/50">
|
||||
<span className="text-xs text-muted-foreground">Скорость</span>
|
||||
<span className="text-xs font-mono font-medium">
|
||||
{formatNetflowRate({
|
||||
bytes: selectedService.bytes,
|
||||
bps: selectedService.bps,
|
||||
bpsFwd: selectedService.bps,
|
||||
bytes: liveSelectedService.bytes,
|
||||
bps: liveSelectedService.bps,
|
||||
bpsFwd: liveSelectedService.bps,
|
||||
bpsRev: 0,
|
||||
})}
|
||||
</span>
|
||||
@@ -2742,34 +2753,34 @@ export default function NetworkMapPage() {
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Выход</p>
|
||||
<div className="flex flex-col gap-1.5">
|
||||
{visibleServiceEdges.filter((e) => e.toId === selectedService.id).map((e) => {
|
||||
<div className="flex flex-col gap-3">
|
||||
{visibleServiceEdges.filter((e) => e.toId === liveSelectedService.id).map((e) => {
|
||||
const src = mapServers.find((s) => s.id === e.fromId)
|
||||
const enPaths = mapServicePaths
|
||||
.filter((p) => p.serviceId === liveSelectedService.id && p.enId === e.fromId)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)
|
||||
return (
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
<div key={`${e.fromId}|${e.toId}`} className="flex flex-col gap-1.5">
|
||||
<div className="flex items-center justify-between text-xs">
|
||||
<span className="font-mono truncate">{src?.name ?? e.fromId}</span>
|
||||
<span className="font-mono text-emerald-400 tabular-nums">
|
||||
{formatNetflowRate({ bytes: e.bytes, bps: e.bps, bpsFwd: e.bpsFwd, bpsRev: e.bpsRev })}
|
||||
</span>
|
||||
</div>
|
||||
<ServicePathList
|
||||
paths={enPaths}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.serviceId === selectedService.id)
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
services={mapServices}
|
||||
highlight={highlightedPath}
|
||||
viaMode="via"
|
||||
onToggle={togglePathHighlight}
|
||||
/>
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
) : selected ? (
|
||||
@@ -3002,7 +3013,11 @@ export default function NetworkMapPage() {
|
||||
<p className="text-xs font-semibold text-muted-foreground uppercase tracking-wider mb-2">Пути</p>
|
||||
<ServicePathList
|
||||
paths={mapServicePaths
|
||||
.filter((p) => p.viaId === selected.id || p.enId === selected.id)
|
||||
.filter((p) => (
|
||||
selected.type === "exit-node"
|
||||
? p.enId === selected.id
|
||||
: p.viaId === selected.id
|
||||
))
|
||||
.slice()
|
||||
.sort((a, b) => b.bps - a.bps)}
|
||||
servers={mapServers}
|
||||
|
||||
@@ -0,0 +1,558 @@
|
||||
"use client"
|
||||
|
||||
import { Suspense, useCallback, useEffect, useMemo, useState } from "react"
|
||||
import { useSearchParams } from "next/navigation"
|
||||
import {
|
||||
ActivityIcon,
|
||||
DatabaseIcon,
|
||||
GaugeIcon,
|
||||
ServerIcon,
|
||||
UsersIcon,
|
||||
} from "lucide-react"
|
||||
import { PageHeader } from "@/components/page-header"
|
||||
import { KpiStatGrid } from "@/components/reui-kit/kpi-stat-grid"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { SegmentedControl } from "@/components/form-kit"
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import { PeriodSelector, rangeForPreset, type DateRangeYmd } from "@/components/statistics/period-selector"
|
||||
import { StatisticsVolumeChart } from "@/components/statistics/statistics-volume-chart"
|
||||
import { DimensionSelect, PivotDimSelect } from "@/components/statistics/dimension-select"
|
||||
import { SliceChips } from "@/components/statistics/slice-chips"
|
||||
import { BreakdownDashboard } from "@/components/statistics/breakdown-dashboard"
|
||||
import { StatisticsPivotGrid } from "@/components/statistics/statistics-pivot-grid"
|
||||
import {
|
||||
StatisticsBreakdownDataGrid,
|
||||
type StatisticsSliceKind,
|
||||
} from "@/components/data-grids/statistics-breakdown-data-grid"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import type { Filter } from "@/components/reui/filters"
|
||||
import { STATISTICS_FILTER_FIELDS } from "@/lib/data-filters/statistics-filter-fields"
|
||||
import {
|
||||
isStatisticsPivotDim,
|
||||
isStatisticsSliceKind,
|
||||
STATISTICS_DIMS,
|
||||
} from "@/lib/statistics-dims"
|
||||
import { useDataSource } from "@/lib/data-source"
|
||||
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
|
||||
import {
|
||||
getStatistics,
|
||||
getStatisticsPivot,
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsQuery,
|
||||
} from "@/shared/api/statistics"
|
||||
import type { StatisticsBreakdownRow, StatisticsPivotDim } from "@mmapp/contracts/statistics"
|
||||
|
||||
/**
|
||||
* BI-куб трафика: критерий → остальные разрезы + pivot.
|
||||
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
|
||||
* · https://reui.io/preview/base/data-grid-filtering-2 · https://reui.io/preview/base/solution-analytics-8
|
||||
* · https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/data-grid
|
||||
*/
|
||||
|
||||
const EMPTY: StatisticsDto = {
|
||||
from: "",
|
||||
to: "",
|
||||
grain: "day",
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "",
|
||||
topService: "",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
|
||||
const EMPTY_PIVOT: StatisticsPivotDto = {
|
||||
rowDim: "country",
|
||||
colDim: "service",
|
||||
metric: "bytes",
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
|
||||
interface CubeSlices {
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: string
|
||||
serverId?: string
|
||||
userId?: string
|
||||
iface?: string
|
||||
}
|
||||
|
||||
const SLICE_KEYS = ["country", "service", "asn", "serverId", "userId", "iface"] as const
|
||||
|
||||
function readRange(sp: URLSearchParams): DateRangeYmd {
|
||||
const from = sp.get("from")
|
||||
const to = sp.get("to")
|
||||
if (from && to && from <= to) return { from, to }
|
||||
return rangeForPreset("7d")
|
||||
}
|
||||
|
||||
function readDim(sp: URLSearchParams): StatisticsSliceKind {
|
||||
const t = sp.get("dim") ?? sp.get("tab")
|
||||
return t && isStatisticsSliceKind(t) ? t : "users"
|
||||
}
|
||||
|
||||
function readView(sp: URLSearchParams): "explore" | "pivot" {
|
||||
return sp.get("view") === "pivot" ? "pivot" : "explore"
|
||||
}
|
||||
|
||||
function readPlanes(sp: URLSearchParams): "unique" | "all" {
|
||||
return sp.get("planes") === "all" ? "all" : "unique"
|
||||
}
|
||||
|
||||
function readPivotDim(sp: URLSearchParams, key: string, fallback: StatisticsPivotDim): StatisticsPivotDim {
|
||||
const v = sp.get(key)
|
||||
return v && isStatisticsPivotDim(v) ? v : fallback
|
||||
}
|
||||
|
||||
function readSlices(sp: URLSearchParams): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const key of SLICE_KEYS) {
|
||||
const v = sp.get(key)?.trim()
|
||||
if (v) next[key] = v
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function slicesToFilters(slices: CubeSlices): Filter[] {
|
||||
return SLICE_KEYS.flatMap((key) => {
|
||||
const val = slices[key]
|
||||
if (!val) return []
|
||||
return [{ id: key, field: key, operator: "is", values: [val] }]
|
||||
})
|
||||
}
|
||||
|
||||
function filtersToSlices(filters: Filter[]): CubeSlices {
|
||||
const next: CubeSlices = {}
|
||||
for (const f of filters) {
|
||||
const raw = String(f.values[0] ?? "").trim()
|
||||
if (!raw) continue
|
||||
if (f.field === "country") next.country = raw.toUpperCase().slice(0, 2)
|
||||
else if (f.field === "service") next.service = raw
|
||||
else if (f.field === "asn") next.asn = raw.replace(/[^\d]/g, "")
|
||||
else if (f.field === "serverId") next.serverId = raw
|
||||
else if (f.field === "userId") next.userId = raw
|
||||
else if (f.field === "iface") next.iface = raw
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function toQuery(range: DateRangeYmd, slices: CubeSlices, planes: "unique" | "all"): StatisticsQuery {
|
||||
const serverId = slices.serverId ? Number(slices.serverId) : undefined
|
||||
const asn = slices.asn != null && slices.asn !== "" ? Number(slices.asn) : undefined
|
||||
return {
|
||||
from: range.from,
|
||||
to: range.to,
|
||||
serverId: Number.isFinite(serverId) && (serverId ?? 0) > 0 ? serverId : undefined,
|
||||
userId: slices.userId,
|
||||
iface: slices.iface,
|
||||
country: slices.country && slices.country.length === 2 ? slices.country : undefined,
|
||||
service: slices.service,
|
||||
asn: Number.isFinite(asn) ? asn : undefined,
|
||||
planes,
|
||||
}
|
||||
}
|
||||
|
||||
function selectedIdForKind(kind: StatisticsSliceKind, slices: CubeSlices): string | undefined {
|
||||
if (kind === "users") return slices.userId
|
||||
if (kind === "servers") return slices.serverId
|
||||
if (kind === "countries") return slices.country
|
||||
if (kind === "services") return slices.service
|
||||
if (kind === "asns") return slices.asn
|
||||
if (kind === "interfaces" && slices.serverId && slices.iface) {
|
||||
return `${slices.serverId}:${slices.iface}`
|
||||
}
|
||||
if (kind === "interfaces") return slices.iface
|
||||
return undefined
|
||||
}
|
||||
|
||||
function rowsForKind(data: StatisticsDto, kind: StatisticsSliceKind) {
|
||||
if (kind === "users") return data.users
|
||||
if (kind === "servers") return data.servers
|
||||
if (kind === "interfaces") return data.interfaces
|
||||
if (kind === "countries") return data.countries
|
||||
if (kind === "services") return data.services
|
||||
return data.asns
|
||||
}
|
||||
|
||||
function hasAnySlice(slices: CubeSlices): boolean {
|
||||
return SLICE_KEYS.some((k) => Boolean(slices[k]))
|
||||
}
|
||||
|
||||
function hiddenKinds(slices: CubeSlices): Set<StatisticsSliceKind> {
|
||||
const hidden = new Set<StatisticsSliceKind>()
|
||||
if (slices.userId) hidden.add("users")
|
||||
if (slices.serverId) hidden.add("servers")
|
||||
if (slices.iface) hidden.add("interfaces")
|
||||
if (slices.country) hidden.add("countries")
|
||||
if (slices.service) hidden.add("services")
|
||||
if (slices.asn) hidden.add("asns")
|
||||
return hidden
|
||||
}
|
||||
|
||||
function applyDimValue(slices: CubeSlices, kind: StatisticsSliceKind, rowId: string): CubeSlices {
|
||||
const next: CubeSlices = { ...slices }
|
||||
if (kind === "users") {
|
||||
if (rowId === STATISTICS_UNBOUND_USER_ID) return next
|
||||
if (next.userId === rowId) delete next.userId
|
||||
else next.userId = rowId
|
||||
} else if (kind === "servers") {
|
||||
if (next.serverId === rowId) delete next.serverId
|
||||
else next.serverId = rowId
|
||||
} else if (kind === "countries") {
|
||||
if (next.country === rowId) delete next.country
|
||||
else next.country = rowId
|
||||
} else if (kind === "services") {
|
||||
if (next.service === rowId) delete next.service
|
||||
else next.service = rowId
|
||||
} else if (kind === "asns") {
|
||||
if (next.asn === rowId) delete next.asn
|
||||
else next.asn = rowId
|
||||
} else {
|
||||
const colon = rowId.indexOf(":")
|
||||
const sid = colon >= 0 ? rowId.slice(0, colon) : undefined
|
||||
const iface = colon >= 0 ? rowId.slice(colon + 1) : rowId
|
||||
if (next.iface === iface && next.serverId === sid) {
|
||||
delete next.iface
|
||||
delete next.serverId
|
||||
} else {
|
||||
next.iface = iface
|
||||
if (sid) next.serverId = sid
|
||||
}
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
function applyPivotDim(slices: CubeSlices, dim: StatisticsPivotDim, id: string): CubeSlices {
|
||||
const kind = STATISTICS_DIMS.find((d) => d.pivot === dim)?.id ?? "users"
|
||||
return applyDimValue(slices, kind, id)
|
||||
}
|
||||
|
||||
function chipList(slices: CubeSlices): { key: string; label: string }[] {
|
||||
const chips: { key: string; label: string }[] = []
|
||||
if (slices.country) chips.push({ key: "country", label: `страна ${slices.country}` })
|
||||
if (slices.service) chips.push({ key: "service", label: `сервис ${slices.service}` })
|
||||
if (slices.asn) chips.push({ key: "asn", label: `ASN ${slices.asn}` })
|
||||
if (slices.serverId) chips.push({ key: "serverId", label: `сервер ${slices.serverId}` })
|
||||
if (slices.userId) chips.push({ key: "userId", label: `пользователь ${slices.userId}` })
|
||||
if (slices.iface) chips.push({ key: "iface", label: `iface ${slices.iface}` })
|
||||
return chips
|
||||
}
|
||||
|
||||
function StatisticsPageInner() {
|
||||
const searchParams = useSearchParams()
|
||||
const { mode, backendUrl, prefsHydrated } = useDataSource()
|
||||
const isLive = mode === "live"
|
||||
|
||||
const range = useMemo(() => readRange(searchParams), [searchParams])
|
||||
const slices = useMemo(() => readSlices(searchParams), [searchParams])
|
||||
const filters = useMemo(() => slicesToFilters(slices), [slices])
|
||||
const dim = useMemo(() => readDim(searchParams), [searchParams])
|
||||
const view = useMemo(() => readView(searchParams), [searchParams])
|
||||
const planes = useMemo(() => readPlanes(searchParams), [searchParams])
|
||||
const pivotRow = useMemo(() => readPivotDim(searchParams, "pivotRow", "country"), [searchParams])
|
||||
const pivotCol = useMemo(() => readPivotDim(searchParams, "pivotCol", "service"), [searchParams])
|
||||
|
||||
const [data, setData] = useState<StatisticsDto>(EMPTY)
|
||||
const [pivot, setPivot] = useState<StatisticsPivotDto>(EMPTY_PIVOT)
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [error, setError] = useState<string | null>(null)
|
||||
|
||||
const replaceParams = useCallback(
|
||||
(patch: Record<string, string | undefined>) => {
|
||||
const sp = new URLSearchParams(searchParams.toString())
|
||||
for (const [k, v] of Object.entries(patch)) {
|
||||
if (v) sp.set(k, v)
|
||||
else sp.delete(k)
|
||||
}
|
||||
const qs = sp.toString()
|
||||
if (qs === searchParams.toString()) return
|
||||
window.history.replaceState(null, "", qs ? `/statistics?${qs}` : "/statistics")
|
||||
},
|
||||
[searchParams],
|
||||
)
|
||||
|
||||
const setRange = useCallback(
|
||||
(next: DateRangeYmd) => {
|
||||
replaceParams({ from: next.from, to: next.to })
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
const setSlices = useCallback(
|
||||
(next: CubeSlices) => {
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
})
|
||||
},
|
||||
[replaceParams],
|
||||
)
|
||||
|
||||
useEffect(() => {
|
||||
if (!prefsHydrated || !isLive) return
|
||||
let cancelled = false
|
||||
void (async () => {
|
||||
setLoading(true)
|
||||
setError(null)
|
||||
try {
|
||||
const query = toQuery(range, slices, planes)
|
||||
const dto = await getStatistics(backendUrl, query)
|
||||
if (!cancelled) setData(dto)
|
||||
if (view === "pivot" && pivotRow !== pivotCol) {
|
||||
const matrix = await getStatisticsPivot(backendUrl, {
|
||||
...query,
|
||||
row: pivotRow,
|
||||
col: pivotCol,
|
||||
metric: "bytes",
|
||||
})
|
||||
if (!cancelled) setPivot(matrix)
|
||||
}
|
||||
} catch (e: unknown) {
|
||||
if (!cancelled) {
|
||||
setData(EMPTY)
|
||||
setPivot(EMPTY_PIVOT)
|
||||
setError(e instanceof Error ? e.message : "Не удалось загрузить статистику")
|
||||
}
|
||||
} finally {
|
||||
if (!cancelled) setLoading(false)
|
||||
}
|
||||
})()
|
||||
return () => {
|
||||
cancelled = true
|
||||
}
|
||||
}, [backendUrl, isLive, prefsHydrated, range, slices, view, pivotRow, pivotCol, planes])
|
||||
|
||||
const viewData = isLive ? data : EMPTY
|
||||
const sliced = hasAnySlice(slices)
|
||||
const emptyCube = !isLive || (!loading && viewData.kpis.bytes === 0 && viewData.interfaces.length === 0)
|
||||
|
||||
function handleRowClick(kind: StatisticsSliceKind, row: StatisticsBreakdownRow) {
|
||||
if (kind === "users" && row.id === STATISTICS_UNBOUND_USER_ID) return
|
||||
if (kind === "interfaces" && row.label.includes("· дубль")) return
|
||||
setSlices(applyDimValue(slices, kind, row.id))
|
||||
}
|
||||
|
||||
function handlePivotCell(rowId: string, colId: string) {
|
||||
if (rowId === "__other__" || colId === "__other__") return
|
||||
let next = applyPivotDim(slices, pivotRow, rowId)
|
||||
next = applyPivotDim(next, pivotCol, colId)
|
||||
replaceParams({
|
||||
country: next.country,
|
||||
service: next.service,
|
||||
asn: next.asn,
|
||||
serverId: next.serverId,
|
||||
userId: next.userId,
|
||||
iface: next.iface,
|
||||
view: "explore",
|
||||
})
|
||||
}
|
||||
|
||||
const kpis = viewData.kpis
|
||||
const chips = chipList(slices)
|
||||
const countLabel =
|
||||
view === "pivot"
|
||||
? `${pivot.rows.length} × ${pivot.columns.length}`
|
||||
: sliced
|
||||
? `${STATISTICS_DIMS.filter((d) => !hiddenKinds(slices).has(d.id)).length} разрезов`
|
||||
: `${rowsForKind(viewData, dim).length} строк`
|
||||
|
||||
return (
|
||||
<div className="flex h-full flex-col">
|
||||
<PageHeader
|
||||
crumbs={[{ label: "Обзор", href: "/dashboard" }, { label: "Статистика" }]}
|
||||
actions={<PeriodSelector range={range} onChange={setRange} />}
|
||||
/>
|
||||
|
||||
<div className="flex flex-1 flex-col gap-4 overflow-y-auto px-4 py-4 md:gap-6 md:px-6 md:py-5">
|
||||
{!isLive ? (
|
||||
<Alert>
|
||||
<AlertTitle>Живые данные выключены</AlertTitle>
|
||||
<AlertDescription>
|
||||
Куб статистики строится из IPFIX. Переключитесь на живой источник, чтобы увидеть отчёт.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{error ? (
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>Ошибка загрузки</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
{!slices.serverId && isLive && !emptyCube ? (
|
||||
<Alert>
|
||||
<AlertTitle>Уникальный объём</AlertTitle>
|
||||
<AlertDescription>
|
||||
Объём — трафик клиентов на GRE/WG, без повторного учёта JH↔EN и WAN.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
|
||||
<KpiStatGrid
|
||||
aria-label="Сводка трафика"
|
||||
isLoading={loading}
|
||||
skeletonCount={5}
|
||||
items={[
|
||||
{
|
||||
id: "bytes",
|
||||
label: "Объём",
|
||||
value: formatBytes(kpis.bytes),
|
||||
hint: "GRE/WG клиентов, без hops",
|
||||
icon: <DatabaseIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "packets",
|
||||
label: "Пакеты",
|
||||
value: kpis.packets.toLocaleString("ru-RU"),
|
||||
hint: kpis.topService ? `топ: ${kpis.topService}` : undefined,
|
||||
icon: <ActivityIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "bps",
|
||||
label: "Средний bitrate",
|
||||
value: fmtBps(kpis.avgBps),
|
||||
icon: <GaugeIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "users",
|
||||
label: "Пользователи",
|
||||
value: String(kpis.users),
|
||||
icon: <UsersIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
{
|
||||
id: "servers",
|
||||
label: "Серверы",
|
||||
value: String(kpis.servers),
|
||||
hint: slices.serverId
|
||||
? (kpis.ifaces ? `${kpis.ifaces} iface` : undefined)
|
||||
: planes === "all"
|
||||
? "WAN и дубли в списке"
|
||||
: "без WAN и overlay",
|
||||
icon: <ServerIcon />,
|
||||
iconClassName: "text-muted-foreground",
|
||||
},
|
||||
]}
|
||||
/>
|
||||
|
||||
<StatisticsVolumeChart series={viewData.series} grain={viewData.grain} />
|
||||
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
leading={
|
||||
<div className="flex flex-wrap items-center gap-3">
|
||||
<SegmentedControl
|
||||
value={view}
|
||||
onChange={(next) => replaceParams({ view: next === "pivot" ? "pivot" : "explore" })}
|
||||
options={[
|
||||
{ value: "explore", label: "Разрез" },
|
||||
{ value: "pivot", label: "Сводка" },
|
||||
]}
|
||||
/>
|
||||
<SegmentedControl
|
||||
value={planes}
|
||||
onChange={(next) => replaceParams({ planes: next === "all" ? "all" : undefined })}
|
||||
options={[
|
||||
{ value: "unique", label: "Уникальный" },
|
||||
{ value: "all", label: "Все плоскости" },
|
||||
]}
|
||||
/>
|
||||
{view === "explore" && !sliced ? (
|
||||
<DimensionSelect
|
||||
label="Критерий"
|
||||
value={dim}
|
||||
onChange={(next) => replaceParams({ dim: next })}
|
||||
/>
|
||||
) : null}
|
||||
{view === "pivot" ? (
|
||||
<>
|
||||
<PivotDimSelect
|
||||
label="Строки"
|
||||
value={pivotRow}
|
||||
exclude={pivotCol}
|
||||
onChange={(next) => replaceParams({ pivotRow: next })}
|
||||
/>
|
||||
<PivotDimSelect
|
||||
label="Колонки"
|
||||
value={pivotCol}
|
||||
exclude={pivotRow}
|
||||
onChange={(next) => replaceParams({ pivotCol: next })}
|
||||
/>
|
||||
</>
|
||||
) : null}
|
||||
</div>
|
||||
}
|
||||
filters={filters}
|
||||
onFiltersChange={(next) => setSlices(filtersToSlices(next))}
|
||||
filterFields={STATISTICS_FILTER_FIELDS}
|
||||
countLabel={countLabel}
|
||||
/>
|
||||
<SliceChips
|
||||
chips={chips}
|
||||
onRemove={(key) => {
|
||||
const next = { ...slices }
|
||||
delete next[key as keyof CubeSlices]
|
||||
setSlices(next)
|
||||
}}
|
||||
/>
|
||||
{emptyCube ? (
|
||||
<EmptyState
|
||||
title="Нет данных куба"
|
||||
description="За выбранный период нет IPFIX-фактов. Куб заполняется с момента деплоя, без бэкфилла за год."
|
||||
/>
|
||||
) : view === "pivot" ? (
|
||||
<StatisticsPivotGrid data={isLive ? pivot : EMPTY_PIVOT} onCellClick={handlePivotCell} isLoading={loading} />
|
||||
) : sliced ? (
|
||||
<BreakdownDashboard
|
||||
data={viewData}
|
||||
hidden={hiddenKinds(slices)}
|
||||
selectedIdFor={(kind) => selectedIdForKind(kind, slices)}
|
||||
onRowClick={handleRowClick}
|
||||
isLoading={loading}
|
||||
/>
|
||||
) : (
|
||||
<StatisticsBreakdownDataGrid
|
||||
rows={rowsForKind(viewData, dim)}
|
||||
kind={dim}
|
||||
selectedId={selectedIdForKind(dim, slices)}
|
||||
onRowClick={(row) => handleRowClick(dim, row)}
|
||||
isLoading={loading}
|
||||
/>
|
||||
)}
|
||||
</DataPageCard>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
export default function StatisticsPage() {
|
||||
return (
|
||||
<Suspense fallback={null}>
|
||||
<StatisticsPageInner />
|
||||
</Suspense>
|
||||
)
|
||||
}
|
||||
@@ -952,10 +952,6 @@ export default function TrafficPage() {
|
||||
setFlowAnalytics(null)
|
||||
return
|
||||
}
|
||||
if (range === "5m") {
|
||||
setFlowAnalytics(null)
|
||||
return
|
||||
}
|
||||
if (range === "30d") {
|
||||
const month = new Date().toISOString().slice(0, 7)
|
||||
void getFlowMonthly(backendUrl, {
|
||||
@@ -1128,7 +1124,7 @@ export default function TrafficPage() {
|
||||
const ingestLine = flowIngestLine(flowStats)
|
||||
const collectorAlive = Boolean(flowStats?.listenerBound || flowStats?.packetsReceived)
|
||||
const flowError = liveError
|
||||
|| (flowLiveError && !(collectorAlive && /live HTTP 500/.test(flowLiveError)) ? flowLiveError : null)
|
||||
|| flowLiveError
|
||||
|| (displayedFlow?.degraded ? "Коллектор перегружен: упрощённая аналитика" : null)
|
||||
|
||||
const flowKpiItems = [
|
||||
|
||||
@@ -5,3 +5,4 @@ dist/
|
||||
*.db-wal
|
||||
.env
|
||||
storage/backups/
|
||||
storage/geoip/
|
||||
|
||||
@@ -0,0 +1,620 @@
|
||||
-- Compact PostgreSQL 18 schema: wipe all app tables (not schema_migrations),
|
||||
-- recreate with smaller time-series rows. SQLite is re-imported after this.
|
||||
|
||||
DO $$
|
||||
DECLARE
|
||||
r record;
|
||||
BEGIN
|
||||
FOR r IN
|
||||
SELECT tablename
|
||||
FROM pg_tables
|
||||
WHERE schemaname = 'public'
|
||||
AND tablename <> 'schema_migrations'
|
||||
LOOP
|
||||
EXECUTE format('DROP TABLE IF EXISTS public.%I CASCADE', r.tablename);
|
||||
END LOOP;
|
||||
END $$;
|
||||
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
id TEXT PRIMARY KEY,
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
host TEXT NOT NULL,
|
||||
port INTEGER NOT NULL DEFAULT 443,
|
||||
username TEXT NOT NULL DEFAULT 'admin',
|
||||
password TEXT NOT NULL DEFAULT '',
|
||||
use_ssl BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
verify_ssl BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
type TEXT NOT NULL DEFAULT 'home-router'
|
||||
CHECK (type IN ('jump-host', 'exit-node', 'home-router')),
|
||||
site TEXT NOT NULL DEFAULT '',
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
asn TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
lan_subnet TEXT NOT NULL DEFAULT '',
|
||||
wan_uplinks JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
mgmt_tunnel_ip TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 30,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers_api_ping_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 120,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_flow_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
collector_ip TEXT NOT NULL DEFAULT '10.255.254.1',
|
||||
flow_listen_port INTEGER NOT NULL DEFAULT 4739,
|
||||
wg_listen_port INTEGER NOT NULL DEFAULT 51821,
|
||||
prefix TEXT NOT NULL DEFAULT '10.255.254.0/24',
|
||||
public_endpoint TEXT NOT NULL DEFAULT '',
|
||||
host_public_key TEXT NOT NULL DEFAULT '',
|
||||
host_private_key TEXT NOT NULL DEFAULT '',
|
||||
hub_server_id BIGINT,
|
||||
retention_hours INTEGER NOT NULL DEFAULT 24,
|
||||
top_n INTEGER NOT NULL DEFAULT 200,
|
||||
map_service_min_share_pct DOUBLE PRECISION NOT NULL DEFAULT 5,
|
||||
last_datagram_at TIMESTAMPTZ,
|
||||
last_exporter_ip TEXT,
|
||||
last_error TEXT,
|
||||
packets_received BIGINT NOT NULL DEFAULT 0,
|
||||
peers_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
resources_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
ping_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
speed_enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 15,
|
||||
probe_interval_sec INTEGER NOT NULL DEFAULT 15,
|
||||
speed_interval_sec INTEGER NOT NULL DEFAULT 60,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS evobgp_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
base_url TEXT NOT NULL DEFAULT '',
|
||||
api_key TEXT NOT NULL DEFAULT '',
|
||||
enabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_telegram_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
bot_token TEXT NOT NULL DEFAULT '',
|
||||
chat_id TEXT NOT NULL DEFAULT '',
|
||||
message_thread_id INTEGER,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS acme_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
directory_url TEXT NOT NULL DEFAULT 'https://acme-v02.api.letsencrypt.org/directory',
|
||||
cloudflare_api_token TEXT NOT NULL DEFAULT '',
|
||||
default_zone_id TEXT NOT NULL DEFAULT '',
|
||||
account_private_key TEXT NOT NULL DEFAULT '',
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS certificate_renew_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 21600,
|
||||
renew_before_days INTEGER NOT NULL DEFAULT 30,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_schedule_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
frequency TEXT NOT NULL DEFAULT 'daily' CHECK (frequency IN ('daily', 'weekly', 'monthly')),
|
||||
hour INTEGER NOT NULL DEFAULT 3,
|
||||
minute INTEGER NOT NULL DEFAULT 0,
|
||||
week_day INTEGER NOT NULL DEFAULT 0,
|
||||
month_day INTEGER NOT NULL DEFAULT 1,
|
||||
keep_count INTEGER NOT NULL DEFAULT 7,
|
||||
format TEXT NOT NULL DEFAULT 'rsc' CHECK (format IN ('rsc', 'backup')),
|
||||
server_ids_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
last_run_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS internet_path_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 300,
|
||||
retention_days INTEGER NOT NULL DEFAULT 14,
|
||||
last_collected_at TIMESTAMPTZ,
|
||||
last_duration_ms INTEGER,
|
||||
last_error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_cursor (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
last_source_finished_at TIMESTAMPTZ,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS data_migration (
|
||||
id INTEGER PRIMARY KEY CHECK (id = 1),
|
||||
sqlite_imported_at TIMESTAMPTZ,
|
||||
sqlite_path TEXT,
|
||||
sqlite_sha256 TEXT,
|
||||
report_json JSONB COMPRESSION lz4
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS filter_rules (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
community TEXT NOT NULL,
|
||||
community_name TEXT,
|
||||
action TEXT NOT NULL DEFAULT 'route' CHECK (action IN ('route', 'blackhole')),
|
||||
gateway TEXT NOT NULL DEFAULT '',
|
||||
gateway_tunnel_id TEXT NOT NULL DEFAULT '',
|
||||
description TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_filter_rules_server_sort ON filter_rules(server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS recursive_routes (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
dst_address TEXT NOT NULL,
|
||||
gateway TEXT NOT NULL,
|
||||
distance INTEGER NOT NULL DEFAULT 1,
|
||||
scope INTEGER,
|
||||
target_scope INTEGER,
|
||||
routing_table TEXT NOT NULL DEFAULT 'main',
|
||||
check_gateway TEXT NOT NULL DEFAULT '',
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
disabled BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_recursive_routes_server_sort ON recursive_routes(server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS server_snapshots (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
polled_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('online', 'offline')),
|
||||
latency_ms DOUBLE PRECISION,
|
||||
ros_version TEXT,
|
||||
board_name TEXT,
|
||||
uptime TEXT,
|
||||
cpu_load INTEGER,
|
||||
free_memory BIGINT,
|
||||
total_memory BIGINT,
|
||||
identity_name TEXT,
|
||||
raw_interfaces JSONB COMPRESSION lz4,
|
||||
raw_ip_addresses JSONB COMPRESSION lz4,
|
||||
PRIMARY KEY (id, polled_at)
|
||||
) PARTITION BY RANGE (polled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_server_snapshots_server_time ON server_snapshots(server_id, polled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS traffic_samples (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
interface_name TEXT NOT NULL,
|
||||
peer_public_key TEXT NOT NULL DEFAULT '',
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
rx_bytes BIGINT NOT NULL DEFAULT 0,
|
||||
tx_bytes BIGINT NOT NULL DEFAULT 0,
|
||||
rx_bps BIGINT NOT NULL DEFAULT 0,
|
||||
tx_bps BIGINT NOT NULL DEFAULT 0,
|
||||
flags SMALLINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, sampled_at, interface_name, peer_public_key)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_traffic_samples_server_iface_time ON traffic_samples(server_id, interface_name, sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS servers_rest_ping_samples (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
ok BOOLEAN NOT NULL,
|
||||
latency_ms INTEGER,
|
||||
error TEXT,
|
||||
PRIMARY KEY (server_id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_buckets (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
src INET NOT NULL,
|
||||
dst INET NOT NULL,
|
||||
proto SMALLINT NOT NULL DEFAULT 0,
|
||||
src_port INTEGER NOT NULL DEFAULT 0,
|
||||
dst_port INTEGER NOT NULL DEFAULT 0,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
in_iface TEXT NOT NULL DEFAULT '',
|
||||
out_iface TEXT NOT NULL DEFAULT '',
|
||||
next_hop INET,
|
||||
flow_start_ms BIGINT NOT NULL DEFAULT 0,
|
||||
flow_end_ms BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_buckets_server_time ON flow_buckets(server_id, bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_minute_stats (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
unique_src INTEGER NOT NULL DEFAULT 0,
|
||||
unique_dst INTEGER NOT NULL DEFAULT 0,
|
||||
conversations INTEGER NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_minute_dims (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
dim TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, dim, key)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_minute_dims_time ON flow_minute_dims(bucket_at, dim);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_daily_dims (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
day DATE NOT NULL,
|
||||
dim TEXT NOT NULL,
|
||||
key TEXT NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, day, dim, key)
|
||||
) PARTITION BY RANGE (day);
|
||||
CREATE INDEX IF NOT EXISTS idx_flow_daily_dims_day ON flow_daily_dims(day, dim);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_ip_meta (
|
||||
prefix TEXT PRIMARY KEY,
|
||||
asn INTEGER NOT NULL DEFAULT 0,
|
||||
country TEXT NOT NULL DEFAULT '',
|
||||
lat DOUBLE PRECISION,
|
||||
lng DOUBLE PRECISION,
|
||||
holder TEXT NOT NULL DEFAULT '',
|
||||
ok INTEGER NOT NULL DEFAULT 1,
|
||||
fetched_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_asn_meta (
|
||||
asn INTEGER PRIMARY KEY,
|
||||
holder TEXT NOT NULL DEFAULT '',
|
||||
fetched_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_probes (
|
||||
id TEXT PRIMARY KEY,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
name TEXT NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
probe_filter TEXT NOT NULL DEFAULT '—',
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
interval_sec INTEGER NOT NULL DEFAULT 0,
|
||||
show_on_dashboard BOOLEAN NOT NULL DEFAULT FALSE,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_probes_server_sort ON uptime_probes(src_server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_probe_samples (
|
||||
probe_id TEXT NOT NULL REFERENCES uptime_probes(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
rtt_ms INTEGER,
|
||||
loss_pct INTEGER NOT NULL DEFAULT 0,
|
||||
status TEXT NOT NULL DEFAULT 'down' CHECK (status IN ('up', 'warn', 'down')),
|
||||
PRIMARY KEY (probe_id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_resource_samples (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL DEFAULT 'offline' CHECK (status IN ('online', 'offline')),
|
||||
cpu_load INTEGER NOT NULL DEFAULT 0,
|
||||
free_memory BIGINT NOT NULL DEFAULT 0,
|
||||
total_memory BIGINT NOT NULL DEFAULT 0,
|
||||
free_hdd_space BIGINT NOT NULL DEFAULT 0,
|
||||
total_hdd_space BIGINT NOT NULL DEFAULT 0,
|
||||
uptime_seconds BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_speed_probes (
|
||||
id TEXT PRIMARY KEY,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
dst_interface TEXT NOT NULL DEFAULT '',
|
||||
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
|
||||
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
|
||||
duration_sec INTEGER NOT NULL DEFAULT 10,
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
last_run_at TIMESTAMPTZ,
|
||||
last_tx_avg_mbps DOUBLE PRECISION,
|
||||
last_rx_avg_mbps DOUBLE PRECISION,
|
||||
last_status TEXT CHECK (last_status IN ('done', 'error')),
|
||||
last_error TEXT,
|
||||
last_ping_rtt_ms INTEGER,
|
||||
last_ping_loss_pct INTEGER,
|
||||
last_ping_at TIMESTAMPTZ,
|
||||
last_ping_error TEXT,
|
||||
sort_order INTEGER NOT NULL DEFAULT 0,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_speed_probes_src_sort ON uptime_speed_probes(src_server_id, sort_order);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS uptime_speed_test_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
probe_id TEXT,
|
||||
src_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
dst_server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
src_interface TEXT NOT NULL DEFAULT '',
|
||||
dst_interface TEXT NOT NULL DEFAULT '',
|
||||
src_address TEXT,
|
||||
dst_address TEXT,
|
||||
src_interface_address TEXT,
|
||||
dst_interface_address TEXT,
|
||||
protocol TEXT NOT NULL DEFAULT 'tcp' CHECK (protocol IN ('tcp', 'udp')),
|
||||
direction TEXT NOT NULL DEFAULT 'both' CHECK (direction IN ('transmit', 'receive', 'both')),
|
||||
duration_sec INTEGER NOT NULL DEFAULT 10,
|
||||
tx_avg_mbps DOUBLE PRECISION,
|
||||
rx_avg_mbps DOUBLE PRECISION,
|
||||
ping_rtt_ms INTEGER,
|
||||
ping_loss_pct INTEGER,
|
||||
ping_error TEXT,
|
||||
status TEXT NOT NULL DEFAULT 'done' CHECK (status IN ('done', 'error')),
|
||||
error TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_uptime_speed_test_runs_created_at ON uptime_speed_test_runs(created_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS certificate_issue_jobs (
|
||||
id TEXT PRIMARY KEY,
|
||||
status TEXT NOT NULL DEFAULT 'queued' CHECK (status IN ('queued', 'running', 'done', 'failed')),
|
||||
step TEXT NOT NULL DEFAULT 'queued',
|
||||
source TEXT NOT NULL DEFAULT 'manual' CHECK (source IN ('manual', 'scheduler')),
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
cert_name TEXT NOT NULL,
|
||||
domain_names JSONB COMPRESSION lz4 NOT NULL,
|
||||
key_type TEXT NOT NULL DEFAULT 'rsa2048',
|
||||
trust_store TEXT NOT NULL DEFAULT 'www,api',
|
||||
requested_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
started_at TIMESTAMPTZ,
|
||||
finished_at TIMESTAMPTZ,
|
||||
error TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_entries (
|
||||
id TEXT PRIMARY KEY,
|
||||
server_id BIGINT REFERENCES servers(id) ON DELETE SET NULL,
|
||||
server_name TEXT NOT NULL,
|
||||
filename TEXT NOT NULL,
|
||||
size_bytes BIGINT NOT NULL,
|
||||
kind TEXT NOT NULL DEFAULT 'manual' CHECK (kind IN ('manual', 'auto')),
|
||||
notes TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_backup_entries_filename ON backup_entries(filename);
|
||||
CREATE INDEX IF NOT EXISTS idx_backup_entries_server_created ON backup_entries(server_id, created_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_groups (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
combine_mode TEXT NOT NULL DEFAULT 'any' CHECK (combine_mode IN ('any', 'all')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
cooldown_override TEXT,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rules (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL,
|
||||
type TEXT NOT NULL,
|
||||
target TEXT NOT NULL,
|
||||
condition TEXT NOT NULL,
|
||||
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
cooldown TEXT NOT NULL DEFAULT '5м',
|
||||
rule_chat_id TEXT NOT NULL DEFAULT '',
|
||||
confirm_stability_sec INTEGER,
|
||||
recovery_mode TEXT NOT NULL DEFAULT 'always' CHECK (recovery_mode IN ('always', 'never', 'conditional')),
|
||||
recovery_stability_sec INTEGER,
|
||||
group_id TEXT REFERENCES alert_groups(id) ON DELETE SET NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rule_targets (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
|
||||
target TEXT NOT NULL,
|
||||
sort_index INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_rule_targets_rule ON alert_rule_targets(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_rule_conditions (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT NOT NULL REFERENCES alert_rules(id) ON DELETE CASCADE,
|
||||
condition_line TEXT NOT NULL,
|
||||
sort_index INTEGER NOT NULL DEFAULT 0
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_rule_conditions_rule ON alert_rule_conditions(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_state (
|
||||
scope_key TEXT PRIMARY KEY,
|
||||
last_fired_at TEXT NOT NULL DEFAULT '',
|
||||
last_payload_hash TEXT
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_prev_live (
|
||||
kind TEXT PRIMARY KEY CHECK (kind IN ('gre', 'bgp')),
|
||||
payload_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '{}'::jsonb,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_engine_confirm_pending (
|
||||
rule_id TEXT PRIMARY KEY,
|
||||
payload_hash TEXT NOT NULL,
|
||||
since_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_history (
|
||||
id TEXT PRIMARY KEY,
|
||||
rule_id TEXT,
|
||||
group_id TEXT,
|
||||
rule_name TEXT NOT NULL,
|
||||
severity TEXT NOT NULL CHECK (severity IN ('critical', 'warning', 'info')),
|
||||
message TEXT NOT NULL,
|
||||
sent_ok BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
fired_at TIMESTAMPTZ NOT NULL
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_history_fired_at ON alert_history(fired_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_history_rule_id ON alert_history(rule_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS alert_outbox (
|
||||
id TEXT PRIMARY KEY,
|
||||
dedupe_key TEXT NOT NULL,
|
||||
channel TEXT NOT NULL DEFAULT 'telegram' CHECK (channel IN ('telegram')),
|
||||
status TEXT NOT NULL DEFAULT 'pending' CHECK (status IN ('pending', 'sent', 'failed')),
|
||||
retry_count INTEGER NOT NULL DEFAULT 0,
|
||||
max_retries INTEGER NOT NULL DEFAULT 3,
|
||||
next_attempt_at TIMESTAMPTZ NOT NULL,
|
||||
payload_json JSONB COMPRESSION lz4 NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
sent_at TIMESTAMPTZ,
|
||||
last_error TEXT
|
||||
);
|
||||
CREATE UNIQUE INDEX IF NOT EXISTS idx_alert_outbox_dedupe ON alert_outbox(dedupe_key);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_outbox_status_next_attempt ON alert_outbox(status, next_attempt_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_alert_outbox_pending_next ON alert_outbox(next_attempt_at) WHERE status = 'pending';
|
||||
|
||||
CREATE TABLE IF NOT EXISTS scheduler_runs (
|
||||
id TEXT PRIMARY KEY,
|
||||
job_key TEXT NOT NULL,
|
||||
started_at TIMESTAMPTZ NOT NULL,
|
||||
finished_at TIMESTAMPTZ NOT NULL,
|
||||
status TEXT NOT NULL CHECK (status IN ('ok', 'error')),
|
||||
error TEXT,
|
||||
duration_ms INTEGER NOT NULL DEFAULT 0,
|
||||
result_json JSONB COMPRESSION lz4
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_scheduler_runs_job_time ON scheduler_runs(job_key, started_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS events (
|
||||
id TEXT PRIMARY KEY,
|
||||
created_at TIMESTAMPTZ NOT NULL,
|
||||
level TEXT NOT NULL CHECK (level IN ('critical', 'warning', 'info')),
|
||||
event_type TEXT NOT NULL,
|
||||
source_module TEXT NOT NULL,
|
||||
title TEXT NOT NULL,
|
||||
message TEXT NOT NULL,
|
||||
entity_type TEXT,
|
||||
entity_id TEXT,
|
||||
payload_json JSONB COMPRESSION lz4
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_created_at ON events(created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_level_created_at ON events(level, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_source_created_at ON events(source_module, created_at DESC);
|
||||
CREATE INDEX IF NOT EXISTS idx_events_event_type_created_at ON events(event_type, created_at DESC);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS app_users (
|
||||
id TEXT PRIMARY KEY,
|
||||
name TEXT NOT NULL DEFAULT '',
|
||||
login TEXT NOT NULL UNIQUE,
|
||||
email TEXT NOT NULL DEFAULT '',
|
||||
role TEXT NOT NULL DEFAULT 'viewer' CHECK (role IN ('admin', 'operator', 'viewer')),
|
||||
active BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
avatar TEXT NOT NULL DEFAULT '',
|
||||
last_seen TIMESTAMPTZ,
|
||||
sections_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
servers_json JSONB COMPRESSION lz4 NOT NULL DEFAULT '[]'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS user_interface_bindings (
|
||||
id TEXT PRIMARY KEY,
|
||||
user_id TEXT NOT NULL REFERENCES app_users(id) ON DELETE CASCADE,
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
interface_name TEXT NOT NULL,
|
||||
interface_type TEXT NOT NULL DEFAULT 'other' CHECK (interface_type IN ('ether', 'gre', 'wg', 'other')),
|
||||
peer_public_key TEXT NOT NULL DEFAULT '',
|
||||
peer_name TEXT NOT NULL DEFAULT '',
|
||||
comment TEXT NOT NULL DEFAULT '',
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
UNIQUE (server_id, interface_name, peer_public_key)
|
||||
);
|
||||
CREATE INDEX IF NOT EXISTS idx_user_iface_bind_user ON user_interface_bindings(user_id);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS internet_path_snapshots (
|
||||
id BIGINT GENERATED BY DEFAULT AS IDENTITY,
|
||||
sampled_at TIMESTAMPTZ NOT NULL,
|
||||
payload_json JSONB COMPRESSION lz4 NOT NULL,
|
||||
PRIMARY KEY (id, sampled_at)
|
||||
) PARTITION BY RANGE (sampled_at);
|
||||
CREATE INDEX IF NOT EXISTS idx_internet_path_snapshots_sampled ON internet_path_snapshots(sampled_at);
|
||||
|
||||
INSERT INTO traffic_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO traffic_flow_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO uptime_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO evobgp_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO servers_api_ping_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO internet_path_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO alert_telegram_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO acme_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO certificate_renew_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO backup_schedule_settings (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO alert_engine_cursor (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
INSERT INTO data_migration (id) VALUES (1) ON CONFLICT (id) DO NOTHING;
|
||||
@@ -0,0 +1,44 @@
|
||||
-- S3-compatible storage for RouterOS backups
|
||||
|
||||
CREATE TABLE IF NOT EXISTS backup_storage_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
provider TEXT NOT NULL DEFAULT 'local' CHECK (provider IN ('local', 's3')),
|
||||
s3_endpoint TEXT NOT NULL DEFAULT '',
|
||||
s3_region TEXT NOT NULL DEFAULT 'us-east-1',
|
||||
s3_bucket TEXT NOT NULL DEFAULT '',
|
||||
s3_prefix TEXT NOT NULL DEFAULT 'mikrotik',
|
||||
s3_access_key_id TEXT NOT NULL DEFAULT '',
|
||||
s3_secret_access_key TEXT NOT NULL DEFAULT '',
|
||||
s3_force_path_style BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
keep_local_copy BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
last_test_at TIMESTAMPTZ,
|
||||
last_test_error TEXT,
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
INSERT INTO backup_storage_settings (id)
|
||||
VALUES (1)
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS storage TEXT NOT NULL DEFAULT 'local';
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS s3_key TEXT;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS s3_etag TEXT;
|
||||
|
||||
ALTER TABLE backup_entries
|
||||
ADD COLUMN IF NOT EXISTS upload_error TEXT;
|
||||
|
||||
DO $$
|
||||
BEGIN
|
||||
IF NOT EXISTS (
|
||||
SELECT 1 FROM pg_constraint WHERE conname = 'backup_entries_storage_check'
|
||||
) THEN
|
||||
ALTER TABLE backup_entries
|
||||
ADD CONSTRAINT backup_entries_storage_check
|
||||
CHECK (storage IN ('local', 's3', 'both'));
|
||||
END IF;
|
||||
END $$;
|
||||
@@ -0,0 +1,19 @@
|
||||
-- GeoLite2 mmdb (страна/ASN для netflow): настройки автообновления зеркала P3TERX
|
||||
|
||||
CREATE TABLE IF NOT EXISTS geoip_settings (
|
||||
id BIGINT PRIMARY KEY CHECK (id = 1),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
update_interval_sec INTEGER NOT NULL DEFAULT 604800,
|
||||
last_check_at TIMESTAMPTZ,
|
||||
last_success_at TIMESTAMPTZ,
|
||||
last_error TEXT,
|
||||
country_build_at TIMESTAMPTZ,
|
||||
asn_build_at TIMESTAMPTZ,
|
||||
etags_json JSONB NOT NULL DEFAULT '{}'::jsonb,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
|
||||
updated_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
);
|
||||
|
||||
INSERT INTO geoip_settings (id)
|
||||
VALUES (1)
|
||||
ON CONFLICT (id) DO NOTHING;
|
||||
@@ -0,0 +1,28 @@
|
||||
-- Statistics cube: hour + daily facts (server × iface × country × service × ASN).
|
||||
-- Compact types. FILLFACTOR/autovacuum нельзя на partitioned parent (PG 42809) —
|
||||
-- задаются на листовых партициях в ensurePartitionFor.
|
||||
-- Retention: DROP partitions only (see PARTITION_SPECS).
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_hour_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
bucket_at TIMESTAMPTZ NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, bucket_at, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (bucket_at);
|
||||
|
||||
CREATE TABLE IF NOT EXISTS flow_daily_facts (
|
||||
server_id BIGINT NOT NULL REFERENCES servers(id) ON DELETE CASCADE,
|
||||
day DATE NOT NULL,
|
||||
iface TEXT NOT NULL,
|
||||
country CHAR(2) NOT NULL,
|
||||
service TEXT NOT NULL,
|
||||
asn INTEGER NOT NULL,
|
||||
bytes BIGINT NOT NULL DEFAULT 0,
|
||||
packets BIGINT NOT NULL DEFAULT 0,
|
||||
PRIMARY KEY (server_id, day, iface, country, service, asn)
|
||||
) PARTITION BY RANGE (day);
|
||||
@@ -12,15 +12,19 @@
|
||||
"db:migrate": "drizzle-kit migrate",
|
||||
"db:studio": "drizzle-kit studio",
|
||||
"db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts",
|
||||
"facts:rebuild": "tsx src/scripts/rebuild-flow-facts.ts",
|
||||
"test:auth": "tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts",
|
||||
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts",
|
||||
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts",
|
||||
"test:traffic-flow": "tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-dest.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/traffic-flow-facts-rebuild.test.ts && tsx src/services/statistics-aggregate.test.ts",
|
||||
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts",
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/pg-schema.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg"
|
||||
"test:pg": "tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts",
|
||||
"test:backups": "tsx src/services/s3-backup-client.test.ts",
|
||||
"test": "npm run test:alert-engine && npm run test:auth && npm run test:wireguard && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups",
|
||||
"test:geoip": "tsx src/services/traffic-flow-geoip.test.ts"
|
||||
},
|
||||
"dependencies": {
|
||||
"@aws-sdk/client-s3": "^3.888.0",
|
||||
"@fastify/cors": "^11.2.0",
|
||||
"@fastify/jwt": "^10.2.2",
|
||||
"@fastify/type-provider-zod": "^1.0.0",
|
||||
@@ -31,6 +35,7 @@
|
||||
"drizzle-orm": "^0.45.2",
|
||||
"fastify": "^5.8.5",
|
||||
"fastify-plugin": "^5.1.0",
|
||||
"maxmind": "^5.0.7",
|
||||
"pg": "^8.23.0",
|
||||
"undici": "^8.1.0",
|
||||
"zod": "^4.4.1"
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { pool } from "./index.js"
|
||||
import { applySqlMigrations } from "./migrate.js"
|
||||
import { dropExpiredPartitions, ensurePartitionsAround } from "./partitions.js"
|
||||
import { importSqliteToPostgres, shouldImportSqlite } from "./sqlite-import.js"
|
||||
import { importSqliteToPostgres, shouldImportSqlite, sqliteFileLooksPresent } from "./sqlite-import.js"
|
||||
import { env } from "../config.js"
|
||||
|
||||
const ETL_LOCK = 8723101
|
||||
@@ -19,6 +19,15 @@ export async function initDatabase(): Promise<void> {
|
||||
console.log(
|
||||
`SQLite → PostgreSQL: готово за ${report.durationMs}ms, таблиц ${Object.keys(report.tables).length}`,
|
||||
)
|
||||
} else {
|
||||
const marker = await pool.query<{ sqlite_imported_at: string | null }>(
|
||||
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
|
||||
)
|
||||
if (!marker.rows[0]?.sqlite_imported_at && !sqliteFileLooksPresent(env.DATABASE_PATH)) {
|
||||
console.warn(
|
||||
`SQLite → PostgreSQL: файл ${env.DATABASE_PATH} не найден, база после wipe остаётся пустой (defaults settings)`,
|
||||
)
|
||||
}
|
||||
}
|
||||
} finally {
|
||||
try {
|
||||
|
||||
+23
-12
@@ -1,9 +1,9 @@
|
||||
import { readFileSync } from "node:fs"
|
||||
import { readdirSync, readFileSync } from "node:fs"
|
||||
import { dirname, join } from "node:path"
|
||||
import { fileURLToPath } from "node:url"
|
||||
import type { Pool } from "pg"
|
||||
|
||||
const MIGRATION_ID = "0000_postgresql"
|
||||
const FIRST_MIGRATION = "0000_postgresql.sql"
|
||||
|
||||
function migrationsDir(): string {
|
||||
const here = dirname(fileURLToPath(import.meta.url))
|
||||
@@ -14,7 +14,7 @@ function migrationsDir(): string {
|
||||
]
|
||||
for (const dir of candidates) {
|
||||
try {
|
||||
readFileSync(join(dir, `${MIGRATION_ID}.sql`), "utf8")
|
||||
readFileSync(join(dir, FIRST_MIGRATION), "utf8")
|
||||
return dir
|
||||
} catch {
|
||||
/* try next */
|
||||
@@ -23,6 +23,10 @@ function migrationsDir(): string {
|
||||
throw new Error("Не найден backend/drizzle/0000_postgresql.sql")
|
||||
}
|
||||
|
||||
function migrationId(file: string): string {
|
||||
return file.replace(/\.sql$/i, "")
|
||||
}
|
||||
|
||||
export async function applySqlMigrations(pool: Pool): Promise<void> {
|
||||
await pool.query(`
|
||||
CREATE TABLE IF NOT EXISTS schema_migrations (
|
||||
@@ -30,14 +34,21 @@ export async function applySqlMigrations(pool: Pool): Promise<void> {
|
||||
applied_at TIMESTAMPTZ NOT NULL DEFAULT now()
|
||||
)
|
||||
`)
|
||||
const { rows } = await pool.query<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = $1`,
|
||||
[MIGRATION_ID],
|
||||
const dir = migrationsDir()
|
||||
const files = readdirSync(dir)
|
||||
.filter((f) => /^\d{4}_.+\.sql$/i.test(f))
|
||||
.sort((a, b) => a.localeCompare(b))
|
||||
const applied = new Set(
|
||||
(await pool.query<{ id: string }>(`SELECT id FROM schema_migrations`)).rows.map((r) => r.id),
|
||||
)
|
||||
if (rows.length > 0) return
|
||||
const sql = readFileSync(join(migrationsDir(), `${MIGRATION_ID}.sql`), "utf8")
|
||||
await pool.query(sql)
|
||||
await pool.query(`INSERT INTO schema_migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`, [
|
||||
MIGRATION_ID,
|
||||
])
|
||||
for (const file of files) {
|
||||
const id = migrationId(file)
|
||||
if (applied.has(id)) continue
|
||||
const sql = readFileSync(join(dir, file), "utf8")
|
||||
await pool.query(sql)
|
||||
await pool.query(
|
||||
`INSERT INTO schema_migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`,
|
||||
[id],
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -8,11 +8,19 @@ export interface PartitionSpec {
|
||||
keepDays: number
|
||||
}
|
||||
|
||||
/** Leaf-only: PG forbids storage params on partitioned parents (SQLSTATE 42809). */
|
||||
const FACT_LEAF_STORAGE =
|
||||
"fillfactor = 70, autovacuum_vacuum_scale_factor = 0.05, autovacuum_vacuum_cost_limit = 2000"
|
||||
|
||||
const FACT_PARENTS = new Set(["flow_hour_facts", "flow_daily_facts"])
|
||||
|
||||
export const PARTITION_SPECS: PartitionSpec[] = [
|
||||
{ parent: "flow_buckets", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_stats", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_minute_dims", kind: "day", keepDays: 4 },
|
||||
{ parent: "flow_daily_dims", kind: "month", keepDays: 420 },
|
||||
{ parent: "flow_hour_facts", kind: "day", keepDays: 3 },
|
||||
{ parent: "flow_daily_facts", kind: "month", keepDays: 420 },
|
||||
{ parent: "traffic_samples", kind: "week", keepDays: 21 },
|
||||
{ parent: "servers_rest_ping_samples", kind: "week", keepDays: 35 },
|
||||
{ parent: "uptime_probe_samples", kind: "week", keepDays: 21 },
|
||||
@@ -21,6 +29,37 @@ export const PARTITION_SPECS: PartitionSpec[] = [
|
||||
{ parent: "internet_path_snapshots", kind: "week", keepDays: 21 },
|
||||
]
|
||||
|
||||
const WEEK_SLACK_DAYS = 7
|
||||
|
||||
async function resolveKeepDays(pool: Pool): Promise<Map<string, number>> {
|
||||
const map = new Map(PARTITION_SPECS.map((s) => [s.parent, s.keepDays]))
|
||||
try {
|
||||
const traffic = await pool.query<{ retention_days: number }>(
|
||||
`SELECT retention_days FROM traffic_settings WHERE id = 1`,
|
||||
)
|
||||
const td = Number(traffic.rows[0]?.retention_days)
|
||||
if (Number.isFinite(td) && td > 0) map.set("traffic_samples", td + WEEK_SLACK_DAYS)
|
||||
|
||||
const uptime = await pool.query<{ retention_days: number }>(
|
||||
`SELECT retention_days FROM uptime_settings WHERE id = 1`,
|
||||
)
|
||||
const ud = Number(uptime.rows[0]?.retention_days)
|
||||
if (Number.isFinite(ud) && ud > 0) {
|
||||
map.set("uptime_probe_samples", ud + WEEK_SLACK_DAYS)
|
||||
map.set("uptime_resource_samples", ud + WEEK_SLACK_DAYS)
|
||||
}
|
||||
|
||||
const path = await pool.query<{ retention_days: number }>(
|
||||
`SELECT retention_days FROM internet_path_settings WHERE id = 1`,
|
||||
)
|
||||
const pd = Number(path.rows[0]?.retention_days)
|
||||
if (Number.isFinite(pd) && pd > 0) map.set("internet_path_snapshots", pd + WEEK_SLACK_DAYS)
|
||||
} catch {
|
||||
/* settings may be absent mid-migration */
|
||||
}
|
||||
return map
|
||||
}
|
||||
|
||||
function utcDate(d: Date): Date {
|
||||
return new Date(Date.UTC(d.getUTCFullYear(), d.getUTCMonth(), d.getUTCDate()))
|
||||
}
|
||||
@@ -79,13 +118,35 @@ export async function ensurePartitionFor(
|
||||
await pool.query(
|
||||
`CREATE TABLE IF NOT EXISTS ${name} PARTITION OF ${parent} FOR VALUES FROM ('${from}') TO ('${to}')`,
|
||||
)
|
||||
if (FACT_PARENTS.has(parent)) {
|
||||
await pool.query(`ALTER TABLE ${name} SET (${FACT_LEAF_STORAGE})`)
|
||||
}
|
||||
return name
|
||||
}
|
||||
|
||||
export async function ensurePartitionsBetween(
|
||||
pool: Pool,
|
||||
parent: string,
|
||||
kind: PartitionKind,
|
||||
from: Date,
|
||||
to: Date,
|
||||
): Promise<void> {
|
||||
const start = from.getTime() <= to.getTime() ? from : to
|
||||
const end = from.getTime() <= to.getTime() ? to : from
|
||||
for (let t = new Date(start.getTime()); t <= end; ) {
|
||||
await ensurePartitionFor(pool, parent, kind, t)
|
||||
if (kind === "day") t = addUtcDays(t, 1)
|
||||
else if (kind === "week") t = addUtcDays(t, 7)
|
||||
else t = new Date(Date.UTC(t.getUTCFullYear(), t.getUTCMonth() + 1, 1))
|
||||
}
|
||||
}
|
||||
|
||||
export async function ensurePartitionsAround(pool: Pool, around = new Date()): Promise<void> {
|
||||
const keepDays = await resolveKeepDays(pool)
|
||||
for (const spec of PARTITION_SPECS) {
|
||||
const keep = keepDays.get(spec.parent) ?? spec.keepDays
|
||||
const daysAhead = spec.kind === "month" ? 40 : spec.kind === "week" ? 21 : 8
|
||||
const start = addUtcDays(around, -spec.keepDays)
|
||||
const start = addUtcDays(around, -keep)
|
||||
const end = addUtcDays(around, daysAhead)
|
||||
for (let t = new Date(start.getTime()); t < end; ) {
|
||||
await ensurePartitionFor(pool, spec.parent, spec.kind, t)
|
||||
@@ -97,8 +158,10 @@ export async function ensurePartitionsAround(pool: Pool, around = new Date()): P
|
||||
}
|
||||
|
||||
export async function dropExpiredPartitions(pool: Pool, around = new Date()): Promise<void> {
|
||||
const keepDays = await resolveKeepDays(pool)
|
||||
for (const spec of PARTITION_SPECS) {
|
||||
const cutoff = addUtcDays(around, -spec.keepDays)
|
||||
const keep = keepDays.get(spec.parent) ?? spec.keepDays
|
||||
const cutoff = addUtcDays(around, -keep)
|
||||
const { rows } = await pool.query<{ relname: string }>(
|
||||
`SELECT c.relname
|
||||
FROM pg_inherits i
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery, pool } from "./index.js"
|
||||
import { applySqlMigrations } from "./migrate.js"
|
||||
import { ensurePartitionFor } from "./partitions.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
@@ -80,4 +81,93 @@ if (!(await withPgOrSkip())) {
|
||||
assert.equal(arrayAsPgArrayFailed, true, "JS array must not be bound as jsonb without stringify")
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ attname: string }>(`
|
||||
SELECT a.attname
|
||||
FROM pg_index i
|
||||
JOIN unnest(i.indkey) WITH ORDINALITY AS k(attnum, ord) ON true
|
||||
JOIN pg_attribute a ON a.attrelid = i.indrelid AND a.attnum = k.attnum
|
||||
WHERE i.indrelid = 'traffic_samples'::regclass AND i.indisprimary
|
||||
ORDER BY k.ord
|
||||
`)
|
||||
assert.deepEqual(
|
||||
rows.map((r) => r.attname),
|
||||
["server_id", "sampled_at", "interface_name", "peer_public_key"],
|
||||
"traffic_samples PK без id",
|
||||
)
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ column_name: string }>(`
|
||||
SELECT column_name FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'traffic_samples'
|
||||
`)
|
||||
const cols = new Set(rows.map((r) => r.column_name))
|
||||
assert.equal(cols.has("id"), false)
|
||||
assert.equal(cols.has("running"), false)
|
||||
assert.equal(cols.has("disabled"), false)
|
||||
assert.equal(cols.has("flags"), true)
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ column_name: string; udt_name: string }>(`
|
||||
SELECT column_name, udt_name
|
||||
FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'flow_buckets'
|
||||
AND column_name IN ('src', 'dst', 'next_hop', 'proto')
|
||||
`)
|
||||
const by = Object.fromEntries(rows.map((r) => [r.column_name, r.udt_name]))
|
||||
assert.equal(by.src, "inet")
|
||||
assert.equal(by.dst, "inet")
|
||||
assert.equal(by.next_hop, "inet")
|
||||
assert.equal(by.proto, "int2")
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ indexdef: string }>(`
|
||||
SELECT indexdef FROM pg_indexes
|
||||
WHERE schemaname = 'public' AND tablename = 'traffic_samples'
|
||||
`)
|
||||
const defs = rows.map((r) => r.indexdef.toLowerCase())
|
||||
assert.equal(defs.some((d) => d.includes("using brin")), false, "нет BRIN на traffic_samples")
|
||||
assert.equal(
|
||||
defs.filter((d) => d.includes("idx_traffic_samples_server_iface_time")).length,
|
||||
1,
|
||||
"один btree (server_id, interface_name, sampled_at)",
|
||||
)
|
||||
assert.equal(defs.some((d) => d.includes("idx_traffic_samples_server_time")), false)
|
||||
}
|
||||
|
||||
{
|
||||
const { rows } = await dbQuery<{ column_name: string }>(`
|
||||
SELECT column_name FROM information_schema.columns
|
||||
WHERE table_schema = 'public' AND table_name = 'uptime_resource_samples'
|
||||
`)
|
||||
const cols = new Set(rows.map((r) => r.column_name))
|
||||
assert.equal(cols.has("board_name"), false)
|
||||
assert.equal(cols.has("ros_version"), false)
|
||||
}
|
||||
|
||||
{
|
||||
const mig = await dbQuery<{ id: string }>(
|
||||
`SELECT id FROM schema_migrations WHERE id = '0002_compact_schema'`,
|
||||
)
|
||||
assert.equal(mig.rows.length, 1, "0002 применена")
|
||||
|
||||
await dbQuery(`INSERT INTO servers (name, host) VALUES ('pg-wipe-idempotent', '127.0.0.1')`)
|
||||
await applySqlMigrations(pool)
|
||||
const still = await dbQuery<{ n: string }>(
|
||||
`SELECT COUNT(*)::text AS n FROM servers WHERE name = 'pg-wipe-idempotent'`,
|
||||
)
|
||||
assert.equal(still.rows[0]?.n, "1", "повторный applySqlMigrations не wipe")
|
||||
await dbQuery(`DELETE FROM servers WHERE name = 'pg-wipe-idempotent'`)
|
||||
}
|
||||
|
||||
{
|
||||
const marker = await dbQuery<{ sqlite_imported_at: string | null }>(
|
||||
`SELECT sqlite_imported_at FROM data_migration WHERE id = 1`,
|
||||
)
|
||||
assert.ok(marker.rows[0], "data_migration singleton после wipe")
|
||||
}
|
||||
|
||||
console.log("pg-schema.test.ts: ok")
|
||||
|
||||
+75
-20
@@ -5,10 +5,12 @@ import {
|
||||
date,
|
||||
doublePrecision,
|
||||
index,
|
||||
inet,
|
||||
integer,
|
||||
jsonb,
|
||||
pgTable,
|
||||
primaryKey,
|
||||
smallint,
|
||||
text,
|
||||
timestamp,
|
||||
uniqueIndex,
|
||||
@@ -135,15 +137,13 @@ export const serversApiPingSettings = pgTable("servers_api_ping_settings", {
|
||||
})
|
||||
|
||||
export const serversRestPingSamples = pgTable("servers_rest_ping_samples", {
|
||||
id: idIdentity(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
sampledAt: ts("sampled_at").notNull(),
|
||||
ok: boolean("ok").notNull(),
|
||||
latencyMs: integer("latency_ms"),
|
||||
error: text("error"),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_servers_rest_ping_samples_server_id").on(t.serverId, t.sampledAt),
|
||||
primaryKey({ columns: [t.serverId, t.sampledAt] }),
|
||||
])
|
||||
|
||||
export const trafficFlowSettings = pgTable("traffic_flow_settings", {
|
||||
@@ -208,19 +208,49 @@ export const flowDailyDims = pgTable("flow_daily_dims", {
|
||||
index("idx_flow_daily_dims_day").on(t.day, t.dim),
|
||||
])
|
||||
|
||||
/** Hour-grain traffic cube for statistics (≤48h). No secondary indexes. */
|
||||
export const flowHourFacts = pgTable("flow_hour_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.bucketAt, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
/** Daily-grain traffic cube for statistics (long window). No secondary indexes. */
|
||||
export const flowDailyFacts = pgTable("flow_daily_facts", {
|
||||
serverId: bigint("server_id", { mode: "number" }).notNull()
|
||||
.references(() => servers.id, { onDelete: "cascade" }),
|
||||
day: date("day", { mode: "string" }).notNull(),
|
||||
iface: text("iface").notNull(),
|
||||
country: text("country").notNull(),
|
||||
service: text("service").notNull(),
|
||||
asn: integer("asn").notNull(),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.serverId, t.day, t.iface, t.country, t.service, t.asn] }),
|
||||
])
|
||||
|
||||
export const flowBuckets = pgTable("flow_buckets", {
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
bucketAt: ts("bucket_at").notNull(),
|
||||
src: text("src").notNull(),
|
||||
dst: text("dst").notNull(),
|
||||
proto: integer("proto").notNull().default(0),
|
||||
src: inet("src").notNull(),
|
||||
dst: inet("dst").notNull(),
|
||||
proto: smallint("proto").notNull().default(0),
|
||||
srcPort: integer("src_port").notNull().default(0),
|
||||
dstPort: integer("dst_port").notNull().default(0),
|
||||
bytes: bigint("bytes", { mode: "number" }).notNull().default(0),
|
||||
packets: bigint("packets", { mode: "number" }).notNull().default(0),
|
||||
inIface: text("in_iface").notNull().default(""),
|
||||
outIface: text("out_iface").notNull().default(""),
|
||||
nextHop: text("next_hop").notNull().default(""),
|
||||
nextHop: inet("next_hop"),
|
||||
flowStartMs: bigint("flow_start_ms", { mode: "number" }).notNull().default(0),
|
||||
flowEndMs: bigint("flow_end_ms", { mode: "number" }).notNull().default(0),
|
||||
}, (t) => [
|
||||
@@ -231,6 +261,20 @@ export const flowBuckets = pgTable("flow_buckets", {
|
||||
index("idx_flow_buckets_server_time").on(t.serverId, t.bucketAt),
|
||||
])
|
||||
|
||||
export const geoipSettings = pgTable("geoip_settings", {
|
||||
id: idSingleton(),
|
||||
enabled: boolean("enabled").notNull().default(true),
|
||||
updateIntervalSec: integer("update_interval_sec").notNull().default(604800),
|
||||
lastCheckAt: ts("last_check_at"),
|
||||
lastSuccessAt: ts("last_success_at"),
|
||||
lastError: text("last_error"),
|
||||
countryBuildAt: ts("country_build_at"),
|
||||
asnBuildAt: ts("asn_build_at"),
|
||||
etagsJson: jsonb("etags_json").notNull().default(sql`'{}'::jsonb`),
|
||||
createdAt: ts("created_at").notNull().defaultNow(),
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const flowIpMeta = pgTable("flow_ip_meta", {
|
||||
prefix: text("prefix").primaryKey(),
|
||||
asn: integer("asn").notNull().default(0),
|
||||
@@ -249,7 +293,6 @@ export const flowAsnMeta = pgTable("flow_asn_meta", {
|
||||
})
|
||||
|
||||
export const trafficSamples = pgTable("traffic_samples", {
|
||||
id: idIdentity(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
interfaceName: text("interface_name").notNull(),
|
||||
peerPublicKey: text("peer_public_key").notNull().default(""),
|
||||
@@ -258,11 +301,9 @@ export const trafficSamples = pgTable("traffic_samples", {
|
||||
txBytes: bigint("tx_bytes", { mode: "number" }).notNull().default(0),
|
||||
rxBps: bigint("rx_bps", { mode: "number" }).notNull().default(0),
|
||||
txBps: bigint("tx_bps", { mode: "number" }).notNull().default(0),
|
||||
running: boolean("running").notNull().default(false),
|
||||
disabled: boolean("disabled").notNull().default(false),
|
||||
flags: smallint("flags").notNull().default(0),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_traffic_samples_server_time").on(t.serverId, t.sampledAt),
|
||||
primaryKey({ columns: [t.serverId, t.sampledAt, t.interfaceName, t.peerPublicKey] }),
|
||||
index("idx_traffic_samples_server_iface_time").on(t.serverId, t.interfaceName, t.sampledAt),
|
||||
])
|
||||
|
||||
@@ -302,19 +343,16 @@ export const uptimeProbes = pgTable("uptime_probes", {
|
||||
])
|
||||
|
||||
export const uptimeProbeSamples = pgTable("uptime_probe_samples", {
|
||||
id: idIdentity(),
|
||||
probeId: text("probe_id").notNull().references(() => uptimeProbes.id, { onDelete: "cascade" }),
|
||||
sampledAt: ts("sampled_at").notNull(),
|
||||
rttMs: integer("rtt_ms"),
|
||||
lossPct: integer("loss_pct").notNull().default(0),
|
||||
status: text("status", { enum: ["up", "warn", "down"] }).notNull().default("down"),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_uptime_probe_samples_probe_time").on(t.probeId, t.sampledAt),
|
||||
primaryKey({ columns: [t.probeId, t.sampledAt] }),
|
||||
])
|
||||
|
||||
export const uptimeResourceSamples = pgTable("uptime_resource_samples", {
|
||||
id: idIdentity(),
|
||||
serverId: intPkRef().references(() => servers.id, { onDelete: "cascade" }),
|
||||
sampledAt: ts("sampled_at").notNull(),
|
||||
status: text("status", { enum: ["online", "offline"] }).notNull().default("offline"),
|
||||
@@ -324,11 +362,8 @@ export const uptimeResourceSamples = pgTable("uptime_resource_samples", {
|
||||
freeHddSpace: bigint("free_hdd_space", { mode: "number" }).notNull().default(0),
|
||||
totalHddSpace: bigint("total_hdd_space", { mode: "number" }).notNull().default(0),
|
||||
uptimeSeconds: bigint("uptime_seconds", { mode: "number" }).notNull().default(0),
|
||||
boardName: text("board_name").notNull().default(""),
|
||||
rosVersion: text("ros_version").notNull().default(""),
|
||||
}, (t) => [
|
||||
primaryKey({ columns: [t.id, t.sampledAt] }),
|
||||
index("idx_uptime_resource_samples_server_time").on(t.serverId, t.sampledAt),
|
||||
primaryKey({ columns: [t.serverId, t.sampledAt] }),
|
||||
])
|
||||
|
||||
export const uptimeSpeedProbes = pgTable("uptime_speed_probes", {
|
||||
@@ -429,6 +464,22 @@ export const backupScheduleSettings = pgTable("backup_schedule_settings", {
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const backupStorageSettings = pgTable("backup_storage_settings", {
|
||||
id: idSingleton(),
|
||||
provider: text("provider", { enum: ["local", "s3"] }).notNull().default("local"),
|
||||
s3Endpoint: text("s3_endpoint").notNull().default(""),
|
||||
s3Region: text("s3_region").notNull().default("us-east-1"),
|
||||
s3Bucket: text("s3_bucket").notNull().default(""),
|
||||
s3Prefix: text("s3_prefix").notNull().default("mikrotik"),
|
||||
s3AccessKeyId: text("s3_access_key_id").notNull().default(""),
|
||||
s3SecretAccessKey: text("s3_secret_access_key").notNull().default(""),
|
||||
s3ForcePathStyle: boolean("s3_force_path_style").notNull().default(true),
|
||||
keepLocalCopy: boolean("keep_local_copy").notNull().default(true),
|
||||
lastTestAt: ts("last_test_at"),
|
||||
lastTestError: text("last_test_error"),
|
||||
updatedAt: ts("updated_at").notNull().defaultNow(),
|
||||
})
|
||||
|
||||
export const backupEntries = pgTable("backup_entries", {
|
||||
id: text("id").primaryKey(),
|
||||
serverId: bigint("server_id", { mode: "number" })
|
||||
@@ -438,6 +489,10 @@ export const backupEntries = pgTable("backup_entries", {
|
||||
sizeBytes: bigint("size_bytes", { mode: "number" }).notNull(),
|
||||
kind: text("kind", { enum: ["manual", "auto"] }).notNull().default("manual"),
|
||||
notes: text("notes"),
|
||||
storage: text("storage", { enum: ["local", "s3", "both"] }).notNull().default("local"),
|
||||
s3Key: text("s3_key"),
|
||||
s3Etag: text("s3_etag"),
|
||||
uploadError: text("upload_error"),
|
||||
createdAt: ts("created_at").notNull(),
|
||||
}, (t) => [
|
||||
uniqueIndex("idx_backup_entries_filename").on(t.filename),
|
||||
|
||||
@@ -3,7 +3,8 @@ import { existsSync, readFileSync } from "node:fs"
|
||||
import Database from "better-sqlite3"
|
||||
import type { Pool } from "pg"
|
||||
import { env } from "../config.js"
|
||||
import { ensurePartitionFor, specForParent } from "./partitions.js"
|
||||
import { ensurePartitionsBetween, specForParent } from "./partitions.js"
|
||||
import { encodeTrafficFlags } from "./traffic-flags.js"
|
||||
|
||||
export interface ImportReport {
|
||||
sqlitePath: string
|
||||
@@ -15,7 +16,9 @@ export interface ImportReport {
|
||||
|
||||
const SNAPSHOT_RETENTION_DAYS = 14
|
||||
|
||||
type ColKind = "ts" | "date" | "bool" | "json" | "json-null" | "bigint-id" | "int" | "text" | "num"
|
||||
type ColKind = "ts" | "date" | "bool" | "json" | "json-null" | "bigint-id" | "int" | "text" | "num" | "flags" | "inet"
|
||||
|
||||
const INSERT_CHUNK = 1000
|
||||
|
||||
interface TableCopy {
|
||||
table: string
|
||||
@@ -78,6 +81,12 @@ const TABLES: TableCopy[] = [
|
||||
["server_ids_json", "json"], ["last_run_at", "ts"], ["last_duration_ms", "int"],
|
||||
["last_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "backup_storage_settings", upsert: true, columns: [
|
||||
["id", "int"], ["provider", "text"], ["s3_endpoint", "text"], ["s3_region", "text"],
|
||||
["s3_bucket", "text"], ["s3_prefix", "text"], ["s3_access_key_id", "text"],
|
||||
["s3_secret_access_key", "text"], ["s3_force_path_style", "bool"], ["keep_local_copy", "bool"],
|
||||
["last_test_at", "ts"], ["last_test_error", "text"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "internet_path_settings", upsert: true, columns: [
|
||||
["id", "int"], ["enabled", "bool"], ["interval_sec", "int"], ["retention_days", "int"],
|
||||
["last_collected_at", "ts"], ["last_duration_ms", "int"], ["last_error", "text"],
|
||||
@@ -103,18 +112,18 @@ const TABLES: TableCopy[] = [
|
||||
["free_memory", "int"], ["total_memory", "int"], ["identity_name", "text"],
|
||||
["raw_interfaces", "json-null"], ["raw_ip_addresses", "json-null"],
|
||||
]},
|
||||
{ table: "traffic_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["interface_name", "text"], ["peer_public_key", "text"],
|
||||
{ table: "traffic_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["server_id", "int"], ["interface_name", "text"], ["peer_public_key", "text"],
|
||||
["sampled_at", "ts"], ["rx_bytes", "int"], ["tx_bytes", "int"], ["rx_bps", "int"], ["tx_bps", "int"],
|
||||
["running", "bool"], ["disabled", "bool"],
|
||||
["flags", "flags"],
|
||||
]},
|
||||
{ table: "servers_rest_ping_samples", identity: true, timeCol: "sampled_at", retentionDays: 30, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["ok", "bool"], ["latency_ms", "int"], ["error", "text"],
|
||||
{ table: "servers_rest_ping_samples", timeCol: "sampled_at", retentionDays: 30, columns: [
|
||||
["server_id", "int"], ["sampled_at", "ts"], ["ok", "bool"], ["latency_ms", "int"], ["error", "text"],
|
||||
]},
|
||||
{ table: "flow_buckets", timeCol: "bucket_at", retentionDays: 2, columns: [
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["src", "text"], ["dst", "text"], ["proto", "int"],
|
||||
["server_id", "int"], ["bucket_at", "ts"], ["src", "inet"], ["dst", "inet"], ["proto", "int"],
|
||||
["src_port", "int"], ["dst_port", "int"], ["bytes", "int"], ["packets", "int"],
|
||||
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "text"],
|
||||
["in_iface", "text"], ["out_iface", "text"], ["next_hop", "inet"],
|
||||
["flow_start_ms", "int"], ["flow_end_ms", "int"],
|
||||
]},
|
||||
{ table: "flow_minute_stats", timeCol: "bucket_at", retentionDays: 3, columns: [
|
||||
@@ -139,13 +148,13 @@ const TABLES: TableCopy[] = [
|
||||
["target", "text"], ["probe_filter", "text"], ["enabled", "bool"], ["interval_sec", "int"],
|
||||
["show_on_dashboard", "bool"], ["sort_order", "int"], ["created_at", "ts"], ["updated_at", "ts"],
|
||||
]},
|
||||
{ table: "uptime_probe_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["probe_id", "text"], ["sampled_at", "ts"], ["rtt_ms", "int"], ["loss_pct", "int"], ["status", "text"],
|
||||
{ table: "uptime_probe_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["probe_id", "text"], ["sampled_at", "ts"], ["rtt_ms", "int"], ["loss_pct", "int"], ["status", "text"],
|
||||
]},
|
||||
{ table: "uptime_resource_samples", identity: true, timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["id", "int"], ["server_id", "int"], ["sampled_at", "ts"], ["status", "text"], ["cpu_load", "int"],
|
||||
{ table: "uptime_resource_samples", timeCol: "sampled_at", retentionDays: 14, columns: [
|
||||
["server_id", "int"], ["sampled_at", "ts"], ["status", "text"], ["cpu_load", "int"],
|
||||
["free_memory", "int"], ["total_memory", "int"], ["free_hdd_space", "int"], ["total_hdd_space", "int"],
|
||||
["uptime_seconds", "int"], ["board_name", "text"], ["ros_version", "text"],
|
||||
["uptime_seconds", "int"],
|
||||
]},
|
||||
{ table: "uptime_speed_probes", columns: [
|
||||
["id", "text"], ["src_server_id", "int"], ["dst_server_id", "int"], ["src_interface", "text"],
|
||||
@@ -269,7 +278,20 @@ function parseJson(value: unknown, fallback: unknown): unknown {
|
||||
}
|
||||
}
|
||||
|
||||
function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[], ctx: string): unknown {
|
||||
function parseInet(value: unknown): string | null {
|
||||
const s = String(value ?? "").trim()
|
||||
if (!s) return null
|
||||
return s
|
||||
}
|
||||
|
||||
function coerce(
|
||||
kind: ColKind,
|
||||
value: unknown,
|
||||
strict: boolean,
|
||||
rejects: string[],
|
||||
ctx: string,
|
||||
row?: Record<string, unknown>,
|
||||
): unknown {
|
||||
switch (kind) {
|
||||
case "ts":
|
||||
return parseTs(value, strict, rejects, ctx)
|
||||
@@ -303,6 +325,10 @@ function coerce(kind: ColKind, value: unknown, strict: boolean, rejects: string[
|
||||
return Number(value)
|
||||
case "text":
|
||||
return value == null ? "" : String(value)
|
||||
case "flags":
|
||||
return encodeTrafficFlags(parseBool(row?.running), parseBool(row?.disabled))
|
||||
case "inet":
|
||||
return parseInet(value)
|
||||
default:
|
||||
return value == null ? null : String(value)
|
||||
}
|
||||
@@ -326,6 +352,35 @@ async function setval(pool: Pool, table: string): Promise<void> {
|
||||
)
|
||||
}
|
||||
|
||||
function placeholderFor(kind: ColKind, index: number): string {
|
||||
if (kind === "json" || kind === "json-null") return `$${index}::jsonb`
|
||||
if (kind === "inet") return `$${index}::inet`
|
||||
return `$${index}`
|
||||
}
|
||||
|
||||
async function precreatePartitions(
|
||||
sqlite: Database.Database,
|
||||
pool: Pool,
|
||||
spec: TableCopy,
|
||||
where: string,
|
||||
): Promise<void> {
|
||||
const part = specForParent(spec.table)
|
||||
if (!part || !spec.timeCol) return
|
||||
const bounds = sqlite.prepare(
|
||||
`SELECT MIN(${spec.timeCol}) AS a, MAX(${spec.timeCol}) AS b FROM ${spec.table}${where}`,
|
||||
).get() as { a?: unknown; b?: unknown }
|
||||
if (bounds?.a == null || bounds?.b == null) return
|
||||
const isDate = spec.columns.find((c) => c[0] === spec.timeCol)?.[1] === "date"
|
||||
const fromIso = isDate
|
||||
? `${String(bounds.a).slice(0, 10)}T00:00:00Z`
|
||||
: parseTs(bounds.a, false, [], spec.table)
|
||||
const toIso = isDate
|
||||
? `${String(bounds.b).slice(0, 10)}T00:00:00Z`
|
||||
: parseTs(bounds.b, false, [], spec.table)
|
||||
if (!fromIso || !toIso) return
|
||||
await ensurePartitionsBetween(pool, spec.table, part.kind, new Date(fromIso), new Date(toIso))
|
||||
}
|
||||
|
||||
async function copyTable(
|
||||
sqlite: Database.Database,
|
||||
pool: Pool,
|
||||
@@ -341,28 +396,27 @@ async function copyTable(
|
||||
where = ` WHERE ${spec.timeCol} >= '${cutoff.replace("T", " ").slice(0, 19)}' OR ${spec.timeCol} >= '${cutoff}'`
|
||||
}
|
||||
const total = (sqlite.prepare(`SELECT COUNT(*) AS c FROM ${spec.table}${where}`).get() as { c: number }).c
|
||||
const part = specForParent(spec.table)
|
||||
await precreatePartitions(sqlite, pool, spec, where)
|
||||
const cols = spec.columns.map(([c]) => c)
|
||||
const placeholders = spec.columns
|
||||
.map(([, kind], i) => (kind === "json" || kind === "json-null" ? `$${i + 1}::jsonb` : `$${i + 1}`))
|
||||
.join(", ")
|
||||
const conflictSql = spec.upsert
|
||||
? `ON CONFLICT (id) DO UPDATE SET ${cols.filter((c) => c !== "id").map((c) => `${c} = EXCLUDED.${c}`).join(", ")}`
|
||||
: `ON CONFLICT DO NOTHING`
|
||||
const insertSql = `INSERT INTO ${spec.table} (${cols.join(", ")}) VALUES (${placeholders}) ${conflictSql}`
|
||||
let copied = 0
|
||||
let skipped = 0
|
||||
const stmt = sqlite.prepare(`SELECT * FROM ${spec.table}${where}`)
|
||||
const batch: unknown[][] = []
|
||||
const flush = async () => {
|
||||
if (batch.length === 0) return
|
||||
const n = spec.columns.length
|
||||
const valuesSql = batch.map((_, i) =>
|
||||
`(${spec.columns.map(([, kind], j) => placeholderFor(kind, i * n + j + 1)).join(", ")})`,
|
||||
).join(", ")
|
||||
const insertSql = `INSERT INTO ${spec.table} (${cols.join(", ")}) VALUES ${valuesSql} ${conflictSql}`
|
||||
const client = await pool.connect()
|
||||
try {
|
||||
await client.query("BEGIN")
|
||||
for (const values of batch) {
|
||||
await client.query(insertSql, values)
|
||||
copied += 1
|
||||
}
|
||||
await client.query(insertSql, batch.flat())
|
||||
copied += batch.length
|
||||
await client.query("COMMIT")
|
||||
} catch (err) {
|
||||
await client.query("ROLLBACK")
|
||||
@@ -375,22 +429,15 @@ async function copyTable(
|
||||
}
|
||||
for (const row of stmt.iterate() as Iterable<Record<string, unknown>>) {
|
||||
try {
|
||||
if (part && spec.timeCol) {
|
||||
const raw = row[spec.timeCol]
|
||||
const ts = spec.columns.find((c) => c[0] === spec.timeCol)?.[1] === "date"
|
||||
? `${String(raw).slice(0, 10)}T00:00:00Z`
|
||||
: parseTs(raw, false, opts.rejects, spec.table)
|
||||
if (ts) await ensurePartitionFor(pool, spec.table, part.kind, new Date(ts))
|
||||
}
|
||||
const values = spec.columns.map(([col, kind]) =>
|
||||
coerce(kind, row[col], opts.strict, opts.rejects, `${spec.table}.${col}`),
|
||||
coerce(kind, row[col], opts.strict, opts.rejects, `${spec.table}.${col}`, row),
|
||||
)
|
||||
if (spec.table === "certificate_issue_jobs" && values[4] == null) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
batch.push(values)
|
||||
if (batch.length >= 200) await flush()
|
||||
if (batch.length >= INSERT_CHUNK) await flush()
|
||||
} catch (err) {
|
||||
skipped += 1
|
||||
const msg = `${spec.table}: ${err instanceof Error ? err.message : String(err)}`
|
||||
|
||||
@@ -0,0 +1,19 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
decodeTrafficSampleFlags,
|
||||
encodeTrafficFlags,
|
||||
trafficFlagDisabled,
|
||||
trafficFlagRunning,
|
||||
} from "./traffic-flags.js"
|
||||
|
||||
assert.equal(encodeTrafficFlags(true, false), 1)
|
||||
assert.equal(encodeTrafficFlags(false, true), 2)
|
||||
assert.equal(encodeTrafficFlags(true, true), 3)
|
||||
assert.equal(encodeTrafficFlags(false, false), 0)
|
||||
assert.equal(trafficFlagRunning(1), true)
|
||||
assert.equal(trafficFlagDisabled(1), false)
|
||||
assert.deepEqual(decodeTrafficSampleFlags(1), { running: true, disabled: false })
|
||||
assert.deepEqual(decodeTrafficSampleFlags(0), { running: false, disabled: false })
|
||||
assert.deepEqual(decodeTrafficSampleFlags(undefined), { running: false, disabled: false })
|
||||
|
||||
console.log("traffic-flags.test.ts: ok")
|
||||
@@ -0,0 +1,24 @@
|
||||
export const TRAFFIC_FLAG_RUNNING = 1
|
||||
export const TRAFFIC_FLAG_DISABLED = 2
|
||||
|
||||
export function encodeTrafficFlags(running: boolean, disabled: boolean): number {
|
||||
return (running ? TRAFFIC_FLAG_RUNNING : 0) | (disabled ? TRAFFIC_FLAG_DISABLED : 0)
|
||||
}
|
||||
|
||||
export function trafficFlagRunning(flags: number | null | undefined): boolean {
|
||||
return ((Number(flags) || 0) & TRAFFIC_FLAG_RUNNING) !== 0
|
||||
}
|
||||
|
||||
export function trafficFlagDisabled(flags: number | null | undefined): boolean {
|
||||
return ((Number(flags) || 0) & TRAFFIC_FLAG_DISABLED) !== 0
|
||||
}
|
||||
|
||||
export function decodeTrafficSampleFlags(flags: number | null | undefined): {
|
||||
running: boolean
|
||||
disabled: boolean
|
||||
} {
|
||||
return {
|
||||
running: trafficFlagRunning(flags),
|
||||
disabled: trafficFlagDisabled(flags),
|
||||
}
|
||||
}
|
||||
@@ -14,6 +14,7 @@ import filtersRoutes from "./routes/filters.js"
|
||||
import recursiveRoutes from "./routes/recursive-routes.js"
|
||||
import trafficRoutes from "./routes/traffic.js"
|
||||
import trafficFlowRoutes from "./routes/traffic-flow.js"
|
||||
import geoipRoutes from "./routes/geoip.js"
|
||||
import serversApiPingRoutes from "./routes/servers-api-ping.js"
|
||||
import uptimeRoutes from "./routes/uptime.js"
|
||||
import networkRoutes from "./routes/network.js"
|
||||
@@ -30,8 +31,10 @@ import eventsRoutes from "./routes/events.js"
|
||||
import wireguardRoutes from "./routes/wireguard.js"
|
||||
import firewallRoutes from "./routes/firewall.js"
|
||||
import usersRoutes from "./routes/users.js"
|
||||
import statisticsRoutes from "./routes/statistics.js"
|
||||
import { refreshScheduler, stopScheduler } from "./services/scheduler.js"
|
||||
import { getFlowWorkerHealth, startTrafficFlowListener, stopTrafficFlowListener } from "./services/traffic-flow-ingest.js"
|
||||
import { initGeoip } from "./services/traffic-flow-geoip.js"
|
||||
|
||||
const eventLoopDelay = monitorEventLoopDelay({ resolution: 20 })
|
||||
eventLoopDelay.enable()
|
||||
@@ -116,6 +119,7 @@ export async function buildApp(opts?: {
|
||||
await app.register(recursiveRoutes, { prefix: "/api" })
|
||||
await app.register(trafficRoutes, { prefix: "/api" })
|
||||
await app.register(trafficFlowRoutes, { prefix: "/api" })
|
||||
await app.register(geoipRoutes, { prefix: "/api" })
|
||||
await app.register(serversApiPingRoutes, { prefix: "/api" })
|
||||
await app.register(uptimeRoutes, { prefix: "/api" })
|
||||
await app.register(networkRoutes, { prefix: "/api" })
|
||||
@@ -132,9 +136,11 @@ export async function buildApp(opts?: {
|
||||
await app.register(wireguardRoutes, { prefix: "/api" })
|
||||
await app.register(firewallRoutes, { prefix: "/api" })
|
||||
await app.register(usersRoutes, { prefix: "/api" })
|
||||
await app.register(statisticsRoutes, { prefix: "/api" })
|
||||
|
||||
if (opts?.startScheduler !== false) {
|
||||
await refreshScheduler()
|
||||
await initGeoip()
|
||||
await startTrafficFlowListener()
|
||||
app.addHook("onClose", async () => {
|
||||
stopScheduler()
|
||||
|
||||
@@ -18,7 +18,7 @@ assert.equal(
|
||||
"mm:settings:admin",
|
||||
)
|
||||
assert.equal(
|
||||
permissionForRequest("GET", "/api/traffic/servers/1/live"),
|
||||
permissionForRequest("GET", "/api/statistics"),
|
||||
"mm:traffic:read",
|
||||
)
|
||||
assert.equal(
|
||||
|
||||
@@ -107,7 +107,7 @@ const RULES: Rule[] = [
|
||||
},
|
||||
{
|
||||
methods: ["GET"],
|
||||
match: (p) => p.startsWith("/api/traffic"),
|
||||
match: (p) => p.startsWith("/api/traffic") || p.startsWith("/api/statistics"),
|
||||
permission: "mm:traffic:read",
|
||||
},
|
||||
{
|
||||
|
||||
@@ -13,6 +13,7 @@ import type {
|
||||
} from "@mmapp/contracts/users"
|
||||
import { db } from "../../../db/index.js"
|
||||
import { parseJsonArray } from "../../../db/json.js"
|
||||
import { decodeTrafficSampleFlags } from "../../../db/traffic-flags.js"
|
||||
import { servers, trafficSamples } from "../../../db/schema.js"
|
||||
import {
|
||||
createBindingRow,
|
||||
@@ -270,8 +271,7 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
|
||||
.select({
|
||||
interfaceName: trafficSamples.interfaceName,
|
||||
peerPublicKey: trafficSamples.peerPublicKey,
|
||||
running: trafficSamples.running,
|
||||
disabled: trafficSamples.disabled,
|
||||
flags: trafficSamples.flags,
|
||||
})
|
||||
.from(trafficSamples)
|
||||
.where(eq(trafficSamples.serverId, serverId)))
|
||||
@@ -281,11 +281,12 @@ export async function listInterfaceCatalog(serverId: number): Promise<CatalogInt
|
||||
for (const r of rows) {
|
||||
if (seen.has(r.interfaceName)) continue
|
||||
seen.add(r.interfaceName)
|
||||
const decoded = decodeTrafficSampleFlags(r.flags)
|
||||
ifaces.push({
|
||||
name: r.interfaceName,
|
||||
type: mapRosInterfaceType("", r.interfaceName),
|
||||
running: Boolean(r.running),
|
||||
disabled: Boolean(r.disabled),
|
||||
running: decoded.running,
|
||||
disabled: decoded.disabled,
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,20 +1,26 @@
|
||||
import { randomUUID } from "node:crypto"
|
||||
import { readFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { z } from "zod"
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { putBackupScheduleSettingsSchema } from "@mmapp/contracts/backups"
|
||||
import {
|
||||
putBackupScheduleSettingsSchema,
|
||||
putBackupStorageSettingsSchema,
|
||||
} from "@mmapp/contracts/backups"
|
||||
import { listServersRead } from "../modules/servers/service/servers-service.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
import { refreshScheduler } from "../services/scheduler.js"
|
||||
import {
|
||||
deleteBackupRecord,
|
||||
getBackupById,
|
||||
getBackupsDir,
|
||||
getBackupScheduleSettings,
|
||||
getBackupStorageSettings,
|
||||
listBackups,
|
||||
readBackupContent,
|
||||
restoreBackupToDevice,
|
||||
runBackupForServer,
|
||||
syncBackupsFromS3,
|
||||
testBackupStorageConnection,
|
||||
updateBackupScheduleSettings,
|
||||
updateBackupStorageSettings,
|
||||
type BackupMeta,
|
||||
} from "../services/backup-service.js"
|
||||
|
||||
@@ -97,6 +103,39 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
return reply.send(result)
|
||||
})
|
||||
|
||||
app.get("/backups/storage", async (_req, reply) => {
|
||||
return reply.send(await getBackupStorageSettings())
|
||||
})
|
||||
|
||||
app.put("/backups/storage", async (req, reply) => {
|
||||
const parsed = putBackupStorageSettingsSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(await updateBackupStorageSettings(parsed.data))
|
||||
})
|
||||
|
||||
app.post("/backups/storage/test", async (_req, reply) => {
|
||||
try {
|
||||
return reply.send(await testBackupStorageConnection())
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка проверки S3",
|
||||
settings: await getBackupStorageSettings(),
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/storage/sync", async (_req, reply) => {
|
||||
try {
|
||||
return reply.send(await syncBackupsFromS3())
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Ошибка синхронизации S3",
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/create", { schema: { body: CreateBackupBodySchema } }, async (req, reply) => {
|
||||
const inputIds = req.body.serverIds.map((x) => String(x))
|
||||
const notes = req.body.notes?.trim() || undefined
|
||||
@@ -169,12 +208,34 @@ const backupsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/backups/:id/download", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
const hit = await getBackupById(req.params.id)
|
||||
if (!hit) return reply.status(404).send({ error: "Бэкап не найден" })
|
||||
const filePath = path.join(getBackupsDir(), hit.filename)
|
||||
const content = await readFile(filePath, "utf8").catch(() => null)
|
||||
if (content == null) return reply.status(404).send({ error: "Файл бэкапа не найден" })
|
||||
reply.header("Content-Type", "text/plain; charset=utf-8")
|
||||
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
|
||||
return reply.send(content)
|
||||
try {
|
||||
const content = await readBackupContent(hit)
|
||||
reply.header("Content-Type", "text/plain; charset=utf-8")
|
||||
reply.header("Content-Disposition", `attachment; filename="${hit.filename}"`)
|
||||
return reply.send(content)
|
||||
} catch {
|
||||
return reply.status(404).send({ error: "Файл бэкапа не найден" })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/backups/:id/restore", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
try {
|
||||
const result = await restoreBackupToDevice(req.params.id)
|
||||
await appendEvent({
|
||||
level: "warning",
|
||||
eventType: "backups.restore",
|
||||
sourceModule: "backups",
|
||||
title: "Восстановление бэкапа",
|
||||
message: `${result.filename} → ${result.serverName}`,
|
||||
entityType: "backup",
|
||||
entityId: req.params.id,
|
||||
})
|
||||
return reply.send(result)
|
||||
} catch (err) {
|
||||
return reply.status(400).send({
|
||||
error: err instanceof Error ? err.message : "Не удалось восстановить бэкап",
|
||||
})
|
||||
}
|
||||
})
|
||||
|
||||
app.delete("/backups/:id", { schema: { params: BackupIdParamSchema } }, async (req, reply) => {
|
||||
|
||||
@@ -0,0 +1,57 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { geoipSettingsPatchSchema } from "@mmapp/contracts/geoip"
|
||||
import { refreshScheduler } from "../services/scheduler.js"
|
||||
import { getGeoipSettings, updateGeoipSettings } from "../services/geoip-settings.js"
|
||||
import {
|
||||
GEOIP_ASN_FILE,
|
||||
GEOIP_COUNTRY_FILE,
|
||||
geoipReadersStatus,
|
||||
initGeoip,
|
||||
} from "../services/traffic-flow-geoip.js"
|
||||
import { collectGeoipUpdateOnce, getGeoipUpdateState } from "../services/geoip-update-collector.js"
|
||||
|
||||
async function buildGeoipStatus() {
|
||||
await initGeoip()
|
||||
const readers = geoipReadersStatus()
|
||||
return {
|
||||
ready: readers.countryLoaded && readers.asnLoaded,
|
||||
countryLoaded: readers.countryLoaded,
|
||||
asnLoaded: readers.asnLoaded,
|
||||
countryFile: GEOIP_COUNTRY_FILE,
|
||||
asnFile: GEOIP_ASN_FILE,
|
||||
dir: readers.dir,
|
||||
running: getGeoipUpdateState().running,
|
||||
settings: await getGeoipSettings(),
|
||||
}
|
||||
}
|
||||
|
||||
const geoipRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/geoip", async (_req, reply) => {
|
||||
return reply.send(await buildGeoipStatus())
|
||||
})
|
||||
|
||||
app.put("/geoip", async (req, reply) => {
|
||||
const parsed = geoipSettingsPatchSchema.safeParse(req.body ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply
|
||||
.status(400)
|
||||
.send({ error: "Некорректное тело запроса", details: parsed.error.flatten() })
|
||||
}
|
||||
await updateGeoipSettings(parsed.data)
|
||||
await refreshScheduler()
|
||||
return reply.send({ ok: true, status: await buildGeoipStatus() })
|
||||
})
|
||||
|
||||
app.post("/geoip/update", async (_req, reply) => {
|
||||
try {
|
||||
const snapshot = await collectGeoipUpdateOnce({ force: true })
|
||||
return reply.send({ ok: !snapshot.fatalError && snapshot.errors.length === 0, snapshot })
|
||||
} catch (e) {
|
||||
const status = (e as { statusCode?: number }).statusCode ?? 502
|
||||
const msg = e instanceof Error ? e.message : String(e)
|
||||
return reply.status(status).send({ error: msg })
|
||||
}
|
||||
})
|
||||
}
|
||||
|
||||
export default geoipRoutes
|
||||
@@ -217,7 +217,7 @@ async function getLatestSnapshotLatencyMs(serverId: number): Promise<number> {
|
||||
return latest.length > 0 ? Math.max(1, Math.round(latest[0].latencyMs ?? 100)) : 100
|
||||
}
|
||||
|
||||
async function latestTrafficByInterface(serverId: number): Promise<Map<string, { id: number; serverId: number; disabled: boolean; sampledAt: string; interfaceName: string; peerPublicKey: string; rxBytes: number; txBytes: number; rxBps: number; txBps: number; running: boolean; }>> {
|
||||
async function latestTrafficByInterface(serverId: number): Promise<Map<string, TrafficSampleRow>> {
|
||||
const rows = await db
|
||||
.select()
|
||||
.from(trafficSamples)
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { statisticsPivotQuerySchema, statisticsQuerySchema } from "@mmapp/contracts/statistics"
|
||||
import { getStatistics, getStatisticsPivot, pivotDimsConflict } from "../services/statistics-aggregate.js"
|
||||
|
||||
const statisticsRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
app.get("/statistics", async (req, reply) => {
|
||||
const parsed = statisticsQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или фильтры", details: parsed.error.flatten() })
|
||||
}
|
||||
return reply.send(await getStatistics(parsed.data))
|
||||
})
|
||||
|
||||
app.get("/statistics/pivot", async (req, reply) => {
|
||||
const parsed = statisticsPivotQuerySchema.safeParse(req.query ?? {})
|
||||
if (!parsed.success) {
|
||||
return reply.status(400).send({ error: "Некорректный период или измерения", details: parsed.error.flatten() })
|
||||
}
|
||||
if (pivotDimsConflict(parsed.data.row, parsed.data.col)) {
|
||||
return reply.status(400).send({ error: "Строки и колонки должны отличаться" })
|
||||
}
|
||||
return reply.send(await getStatisticsPivot(parsed.data))
|
||||
})
|
||||
}
|
||||
|
||||
export default statisticsRoutes
|
||||
@@ -27,6 +27,7 @@ import {
|
||||
import { buildFlowMapHops } from "../services/traffic-flow-map-hops.js"
|
||||
import { applyFlowOverlay } from "../services/traffic-flow-overlay.js"
|
||||
import { listTrafficFlowHostFiles } from "../services/traffic-flow-host-files.js"
|
||||
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
|
||||
import { appendEvent } from "../modules/events/service/events-service.js"
|
||||
|
||||
const LIVE_TICK_MS = 2000
|
||||
@@ -203,6 +204,27 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/rebuild-facts", async (_req, reply) => {
|
||||
try {
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
await appendEvent({
|
||||
level: "info",
|
||||
eventType: "traffic.flow.rebuild_facts",
|
||||
sourceModule: "traffic",
|
||||
title: "Пересчитан куб NetFlow",
|
||||
message: `Факты ${result.facts} из ${result.buckets} сессий, дней ${result.days.length}`,
|
||||
entityType: "traffic_flow",
|
||||
entityId: "rebuild-facts",
|
||||
payload: { buckets: result.buckets, facts: result.facts, days: result.days },
|
||||
})
|
||||
return reply.send(result)
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
const status = message.includes("уже выполняется") ? 409 : 500
|
||||
return reply.status(status).send({ error: message })
|
||||
}
|
||||
})
|
||||
|
||||
app.post("/traffic/flow/overlay", applyOverlayHandler)
|
||||
app.post("/traffic/flow-overlay", applyOverlayHandler)
|
||||
|
||||
@@ -275,7 +297,7 @@ const trafficFlowRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
|
||||
try {
|
||||
while (!abort.signal.aborted) {
|
||||
const payload = safeBuildLiveFlowSample(liveQuery)
|
||||
const payload = await safeBuildLiveFlowSample(liveQuery)
|
||||
writeSse(reply.raw, payload.event, payload.data)
|
||||
await sleep(LIVE_TICK_MS, abort.signal)
|
||||
}
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import type { FastifyPluginAsyncZod } from "@fastify/type-provider-zod"
|
||||
import { asc, desc, eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers, uptimeSpeedProbes, uptimeSpeedTestRuns } from "../db/schema.js"
|
||||
import { servers, serverSnapshots, uptimeSpeedProbes, uptimeSpeedTestRuns } from "../db/schema.js"
|
||||
import { MikrotikClient } from "../services/mikrotik.js"
|
||||
import { scheduleAlertEngineAfterDataCollectors } from "../services/alert-collector-hooks.js"
|
||||
import { refreshScheduler, getSchedulerStatus } from "../services/scheduler.js"
|
||||
@@ -502,7 +502,14 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
})
|
||||
|
||||
const resources = await Promise.all(allServers.map(async (s) => {
|
||||
const rows = await readResourceSamplesSince(sinceIso, s.id)
|
||||
const [rows, snap] = await Promise.all([
|
||||
readResourceSamplesSince(sinceIso, s.id),
|
||||
db.select({ boardName: serverSnapshots.boardName })
|
||||
.from(serverSnapshots)
|
||||
.where(eq(serverSnapshots.serverId, s.id))
|
||||
.orderBy(desc(serverSnapshots.polledAt))
|
||||
.limit(1),
|
||||
])
|
||||
const { row: pick, hasData } = pickResourceDisplayRow(rows, resourceFallbackMaxGapMs)
|
||||
const cpuHistory = toSeries(rows.map((r) => r.cpuLoad), 40)
|
||||
return {
|
||||
@@ -515,7 +522,7 @@ const uptimeRoutes: FastifyPluginAsyncZod = async (app) => {
|
||||
hddUsed: Math.max(0, ((pick?.totalHddSpace ?? 0) - (pick?.freeHddSpace ?? 0)) / (1024 * 1024)),
|
||||
hddTotal: Math.max(0, (pick?.totalHddSpace ?? 0) / (1024 * 1024)),
|
||||
uptimeSeconds: pick?.uptimeSeconds ?? 0,
|
||||
boardName: pick?.boardName || "RouterBOARD",
|
||||
boardName: snap[0]?.boardName || "RouterBOARD",
|
||||
temp: undefined as number | undefined,
|
||||
}
|
||||
}))
|
||||
|
||||
@@ -0,0 +1,8 @@
|
||||
import { initDatabase } from "../db/bootstrap.js"
|
||||
import { closePool } from "../db/index.js"
|
||||
import { rebuildFlowFactsFromBuckets } from "../services/traffic-flow-facts-rebuild.js"
|
||||
|
||||
await initDatabase()
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
console.log(JSON.stringify(result, null, 2))
|
||||
await closePool()
|
||||
@@ -1,14 +1,30 @@
|
||||
import { randomUUID } from "node:crypto"
|
||||
import { mkdir, rm, stat, writeFile } from "node:fs/promises"
|
||||
import { mkdir, rm, stat, writeFile, readFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { desc, eq } from "drizzle-orm"
|
||||
import type { BackupScheduleSettingsDto } from "@mmapp/contracts/backups"
|
||||
import type {
|
||||
BackupScheduleSettingsDto,
|
||||
BackupStorageSettingsDto,
|
||||
PutBackupStorageSettings,
|
||||
} from "@mmapp/contracts/backups"
|
||||
import { db } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { backupEntries, backupScheduleSettings } from "../db/schema.js"
|
||||
import { backupEntries, backupScheduleSettings, backupStorageSettings } from "../db/schema.js"
|
||||
import { getServerRowById } from "../modules/servers/repository/servers-repository.js"
|
||||
import { listServersRead } from "../modules/servers/service/servers-service.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
import {
|
||||
buildS3ObjectKey,
|
||||
createS3ClientFromConfig,
|
||||
parseS3ObjectKey,
|
||||
sanitizeServerName,
|
||||
s3DeleteObject,
|
||||
s3GetObject,
|
||||
s3ListObjects,
|
||||
s3PutObject,
|
||||
s3TestConnection,
|
||||
type S3BackupConfig,
|
||||
} from "./s3-backup-client.js"
|
||||
|
||||
const SETTINGS_ID = 1
|
||||
const BACKUPS_DIR = path.resolve(process.cwd(), "storage", "backups")
|
||||
@@ -22,9 +38,14 @@ export type BackupMeta = {
|
||||
createdAt: string
|
||||
kind: "manual" | "auto"
|
||||
notes?: string
|
||||
storage: "local" | "s3" | "both"
|
||||
s3Key?: string | null
|
||||
uploadError?: string | null
|
||||
}
|
||||
|
||||
function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
|
||||
type BackupRow = typeof backupEntries.$inferSelect
|
||||
|
||||
function rowToMeta(row: BackupRow): BackupMeta {
|
||||
return {
|
||||
id: row.id,
|
||||
serverId: row.serverId,
|
||||
@@ -34,6 +55,9 @@ function rowToMeta(row: typeof backupEntries.$inferSelect): BackupMeta {
|
||||
createdAt: row.createdAt,
|
||||
kind: row.kind,
|
||||
notes: row.notes ?? undefined,
|
||||
storage: row.storage ?? "local",
|
||||
s3Key: row.s3Key,
|
||||
uploadError: row.uploadError,
|
||||
}
|
||||
}
|
||||
|
||||
@@ -60,16 +84,36 @@ export async function insertBackup(meta: BackupMeta): Promise<void> {
|
||||
sizeBytes: meta.sizeBytes,
|
||||
kind: meta.kind,
|
||||
notes: meta.notes ?? null,
|
||||
storage: meta.storage,
|
||||
s3Key: meta.s3Key ?? null,
|
||||
s3Etag: null,
|
||||
uploadError: meta.uploadError ?? null,
|
||||
createdAt: meta.createdAt,
|
||||
})
|
||||
}
|
||||
|
||||
async function getBackupRow(id: string): Promise<BackupRow | undefined> {
|
||||
return (await db.select().from(backupEntries).where(eq(backupEntries.id, id)).limit(1))[0]
|
||||
}
|
||||
|
||||
export async function deleteBackupRecord(id: string): Promise<BackupMeta | null> {
|
||||
const hit = await getBackupById(id)
|
||||
if (!hit) return null
|
||||
const row = await getBackupRow(id)
|
||||
if (!row) return null
|
||||
const meta = rowToMeta(row)
|
||||
if (row.s3Key) {
|
||||
try {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (cfg) {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
await s3DeleteObject(client, cfg.bucket, row.s3Key)
|
||||
}
|
||||
} catch {
|
||||
/* объект мог уже отсутствовать */
|
||||
}
|
||||
}
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
return hit
|
||||
await rm(path.join(BACKUPS_DIR, row.filename), { force: true })
|
||||
return meta
|
||||
}
|
||||
|
||||
function fmtTs(d = new Date()): string {
|
||||
@@ -167,6 +211,127 @@ export async function touchBackupScheduleRunMeta(patch: {
|
||||
}).where(eq(backupScheduleSettings.id, SETTINGS_ID))
|
||||
}
|
||||
|
||||
function defaultStorageRow() {
|
||||
return {
|
||||
id: SETTINGS_ID,
|
||||
provider: "local" as const,
|
||||
s3Endpoint: "",
|
||||
s3Region: "us-east-1",
|
||||
s3Bucket: "",
|
||||
s3Prefix: "mikrotik",
|
||||
s3AccessKeyId: "",
|
||||
s3SecretAccessKey: "",
|
||||
s3ForcePathStyle: true,
|
||||
keepLocalCopy: true,
|
||||
lastTestAt: null as string | null,
|
||||
lastTestError: null as string | null,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
async function getBackupStorageSettingsRow() {
|
||||
return (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
?? defaultStorageRow()
|
||||
}
|
||||
|
||||
function toStorageDto(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): BackupStorageSettingsDto {
|
||||
return {
|
||||
provider: row.provider,
|
||||
s3Endpoint: row.s3Endpoint,
|
||||
s3Region: row.s3Region,
|
||||
s3Bucket: row.s3Bucket,
|
||||
s3Prefix: row.s3Prefix,
|
||||
s3AccessKeyId: row.s3AccessKeyId,
|
||||
secretConfigured: Boolean(row.s3SecretAccessKey),
|
||||
s3ForcePathStyle: row.s3ForcePathStyle,
|
||||
keepLocalCopy: row.keepLocalCopy,
|
||||
lastTestAt: row.lastTestAt ?? null,
|
||||
lastTestError: row.lastTestError ?? null,
|
||||
updatedAt: row.updatedAt,
|
||||
}
|
||||
}
|
||||
|
||||
export async function getBackupStorageSettings(): Promise<BackupStorageSettingsDto> {
|
||||
return toStorageDto(await getBackupStorageSettingsRow())
|
||||
}
|
||||
|
||||
export async function updateBackupStorageSettings(
|
||||
patch: PutBackupStorageSettings,
|
||||
): Promise<BackupStorageSettingsDto> {
|
||||
const prev = await getBackupStorageSettingsRow()
|
||||
const now = new Date().toISOString()
|
||||
const secret = patch.s3SecretAccessKey
|
||||
const next = {
|
||||
provider: patch.provider ?? prev.provider,
|
||||
s3Endpoint: patch.s3Endpoint ?? prev.s3Endpoint,
|
||||
s3Region: patch.s3Region ?? prev.s3Region,
|
||||
s3Bucket: patch.s3Bucket ?? prev.s3Bucket,
|
||||
s3Prefix: patch.s3Prefix ?? prev.s3Prefix,
|
||||
s3AccessKeyId: patch.s3AccessKeyId ?? prev.s3AccessKeyId,
|
||||
s3SecretAccessKey: secret && secret.length > 0 ? secret : prev.s3SecretAccessKey,
|
||||
s3ForcePathStyle: patch.s3ForcePathStyle ?? prev.s3ForcePathStyle,
|
||||
keepLocalCopy: patch.keepLocalCopy ?? prev.keepLocalCopy,
|
||||
updatedAt: now,
|
||||
}
|
||||
if ((await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]) {
|
||||
await db.update(backupStorageSettings).set(next).where(eq(backupStorageSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return await getBackupStorageSettings()
|
||||
}
|
||||
|
||||
function rowToS3Config(row: Awaited<ReturnType<typeof getBackupStorageSettingsRow>>): S3BackupConfig | null {
|
||||
if (row.provider !== "s3") return null
|
||||
if (!row.s3Bucket.trim() || !row.s3AccessKeyId.trim() || !row.s3SecretAccessKey) return null
|
||||
return {
|
||||
endpoint: row.s3Endpoint,
|
||||
region: row.s3Region,
|
||||
bucket: row.s3Bucket.trim(),
|
||||
prefix: row.s3Prefix,
|
||||
accessKeyId: row.s3AccessKeyId,
|
||||
secretAccessKey: row.s3SecretAccessKey,
|
||||
forcePathStyle: row.s3ForcePathStyle,
|
||||
}
|
||||
}
|
||||
|
||||
async function getS3ConfigIfEnabled(): Promise<S3BackupConfig | null> {
|
||||
return rowToS3Config(await getBackupStorageSettingsRow())
|
||||
}
|
||||
|
||||
export async function testBackupStorageConnection(): Promise<BackupStorageSettingsDto> {
|
||||
const row = await getBackupStorageSettingsRow()
|
||||
const cfg = rowToS3Config(row)
|
||||
const now = new Date().toISOString()
|
||||
if (!cfg) {
|
||||
const error = row.provider === "s3"
|
||||
? "Заполните bucket, ключ доступа и секрет"
|
||||
: "S3 не выбран"
|
||||
await persistStorageTest(now, error)
|
||||
throw new Error(error)
|
||||
}
|
||||
try {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
await s3TestConnection(client, cfg.bucket)
|
||||
await persistStorageTest(now, null)
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
await persistStorageTest(now, message)
|
||||
throw new Error(message)
|
||||
}
|
||||
return await getBackupStorageSettings()
|
||||
}
|
||||
|
||||
async function persistStorageTest(at: string, error: string | null) {
|
||||
const exists = (await db.select().from(backupStorageSettings).where(eq(backupStorageSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
const patch = { lastTestAt: at, lastTestError: error, updatedAt: at }
|
||||
if (exists) {
|
||||
await db.update(backupStorageSettings).set(patch).where(eq(backupStorageSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(backupStorageSettings).values({ id: SETTINGS_ID, ...patch })
|
||||
}
|
||||
}
|
||||
|
||||
export async function resolveBackupServerIds(settings: BackupScheduleSettingsDto): Promise<string[]> {
|
||||
const enabled = new Set((await listServersRead()).map((s) => String(s.id)))
|
||||
const requested = settings.serverIds.length > 0 ? settings.serverIds : [...enabled]
|
||||
@@ -214,6 +379,23 @@ export function isBackupDue(now: Date, settings: BackupScheduleSettingsDto, last
|
||||
return true
|
||||
}
|
||||
|
||||
async function uploadBackupToS3(params: {
|
||||
serverName: string
|
||||
filename: string
|
||||
body: string
|
||||
}): Promise<{ key: string; etag?: string } | { error: string }> {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (!cfg) return { error: "S3 не настроен" }
|
||||
try {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
const key = buildS3ObjectKey(cfg.prefix, sanitizeServerName(params.serverName), params.filename)
|
||||
const put = await s3PutObject(client, cfg.bucket, key, params.body)
|
||||
return { key, etag: put.etag }
|
||||
} catch (err) {
|
||||
return { error: err instanceof Error ? err.message : String(err) }
|
||||
}
|
||||
}
|
||||
|
||||
export async function runBackupForServer(
|
||||
id: string,
|
||||
kind: BackupMeta["kind"],
|
||||
@@ -230,12 +412,33 @@ export async function runBackupForServer(
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const script = await client.exportConfigScript()
|
||||
const ts = fmtTs()
|
||||
const safeServer = row.name.replace(/[^a-zA-Z0-9._-]+/g, "_")
|
||||
const safeServer = sanitizeServerName(row.name)
|
||||
const filename = `${safeServer}_${ts}.rsc`
|
||||
const filePath = path.join(BACKUPS_DIR, filename)
|
||||
await ensureBackupStorage()
|
||||
await writeFile(filePath, script, "utf8")
|
||||
const st = await stat(filePath)
|
||||
const storageRow = await getBackupStorageSettingsRow()
|
||||
let storage: BackupMeta["storage"] = "local"
|
||||
let s3Key: string | null = null
|
||||
let s3Etag: string | null = null
|
||||
let uploadError: string | null = null
|
||||
|
||||
if (storageRow.provider === "s3") {
|
||||
const uploaded = await uploadBackupToS3({ serverName: row.name, filename, body: script })
|
||||
if ("key" in uploaded) {
|
||||
s3Key = uploaded.key
|
||||
s3Etag = uploaded.etag ?? null
|
||||
storage = storageRow.keepLocalCopy ? "both" : "s3"
|
||||
if (!storageRow.keepLocalCopy) {
|
||||
await rm(filePath, { force: true })
|
||||
}
|
||||
} else {
|
||||
uploadError = uploaded.error
|
||||
storage = "local"
|
||||
}
|
||||
}
|
||||
|
||||
const meta: BackupMeta = {
|
||||
id: randomUUID(),
|
||||
serverId: row.id,
|
||||
@@ -245,8 +448,24 @@ export async function runBackupForServer(
|
||||
createdAt: new Date().toISOString(),
|
||||
kind,
|
||||
notes,
|
||||
storage,
|
||||
s3Key,
|
||||
uploadError,
|
||||
}
|
||||
await insertBackup(meta)
|
||||
await db.insert(backupEntries).values({
|
||||
id: meta.id,
|
||||
serverId: meta.serverId,
|
||||
serverName: meta.serverName,
|
||||
filename: meta.filename,
|
||||
sizeBytes: meta.sizeBytes,
|
||||
kind: meta.kind,
|
||||
notes: meta.notes ?? null,
|
||||
storage: meta.storage,
|
||||
s3Key: meta.s3Key ?? null,
|
||||
s3Etag,
|
||||
uploadError: meta.uploadError ?? null,
|
||||
createdAt: meta.createdAt,
|
||||
})
|
||||
return meta
|
||||
}
|
||||
|
||||
@@ -257,12 +476,82 @@ export async function pruneBackupsForServer(serverId: string, keepCount: number)
|
||||
if (rows.length <= keepCount) return 0
|
||||
const toDelete = rows.slice(keepCount)
|
||||
for (const hit of toDelete) {
|
||||
await db.delete(backupEntries).where(eq(backupEntries.id, hit.id))
|
||||
await rm(path.join(BACKUPS_DIR, hit.filename), { force: true })
|
||||
await deleteBackupRecord(hit.id)
|
||||
}
|
||||
return toDelete.length
|
||||
}
|
||||
|
||||
export async function readBackupContent(meta: BackupMeta): Promise<Buffer> {
|
||||
if ((meta.storage === "s3" || meta.storage === "both") && meta.s3Key) {
|
||||
try {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (cfg) {
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
return await s3GetObject(client, cfg.bucket, meta.s3Key)
|
||||
}
|
||||
} catch {
|
||||
/* fallback: локальная копия, если есть */
|
||||
}
|
||||
}
|
||||
return await readFile(path.join(BACKUPS_DIR, meta.filename))
|
||||
}
|
||||
|
||||
export async function restoreBackupToDevice(id: string): Promise<{ filename: string; serverName: string }> {
|
||||
const meta = await getBackupById(id)
|
||||
if (!meta) throw new Error("Бэкап не найден")
|
||||
if (!meta.serverId) throw new Error("Сервер бэкапа удалён — восстановить нельзя")
|
||||
const row = await getServerRowById(meta.serverId)
|
||||
if (!row) throw new Error("Сервер не найден")
|
||||
const content = await readBackupContent(meta)
|
||||
const client = MikrotikClient.fromServer(row)
|
||||
const uploaded = await client.uploadTextFile(meta.filename, content.toString("utf8"), 60_000)
|
||||
await client.importUploadedFile(uploaded)
|
||||
return { filename: meta.filename, serverName: row.name }
|
||||
}
|
||||
|
||||
export async function syncBackupsFromS3(): Promise<{ imported: number; skipped: number }> {
|
||||
const cfg = await getS3ConfigIfEnabled()
|
||||
if (!cfg) throw new Error("S3 не настроен")
|
||||
const client = createS3ClientFromConfig(cfg)
|
||||
const objects = await s3ListObjects(client, cfg.bucket, cfg.prefix)
|
||||
const existing = new Set(
|
||||
(await db.select({ filename: backupEntries.filename, s3Key: backupEntries.s3Key }).from(backupEntries))
|
||||
.flatMap((row) => [row.filename, row.s3Key].filter((v): v is string => Boolean(v))),
|
||||
)
|
||||
let imported = 0
|
||||
let skipped = 0
|
||||
for (const obj of objects) {
|
||||
if (!obj.key.toLowerCase().endsWith(".rsc")) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
const parsed = parseS3ObjectKey(obj.key)
|
||||
if (existing.has(obj.key) || existing.has(parsed.filename)) {
|
||||
skipped += 1
|
||||
continue
|
||||
}
|
||||
const createdAt = obj.lastModified ?? new Date().toISOString()
|
||||
await db.insert(backupEntries).values({
|
||||
id: randomUUID(),
|
||||
serverId: null,
|
||||
serverName: parsed.serverName,
|
||||
filename: parsed.filename,
|
||||
sizeBytes: obj.size,
|
||||
kind: "auto",
|
||||
notes: "Импорт из S3",
|
||||
storage: "s3",
|
||||
s3Key: obj.key,
|
||||
s3Etag: null,
|
||||
uploadError: null,
|
||||
createdAt,
|
||||
})
|
||||
existing.add(obj.key)
|
||||
existing.add(parsed.filename)
|
||||
imported += 1
|
||||
}
|
||||
return { imported, skipped }
|
||||
}
|
||||
|
||||
export function getBackupsDir(): string {
|
||||
return BACKUPS_DIR
|
||||
}
|
||||
|
||||
@@ -113,6 +113,15 @@ export async function collectCertificatesRenewOnce(): Promise<CertificatesRenewR
|
||||
continue
|
||||
}
|
||||
|
||||
const stillOn = await getCertificateRenewSettings()
|
||||
if (!stillOn.enabled) {
|
||||
item.action = "skipped"
|
||||
item.message = "Автообновление выключено"
|
||||
snapshot.skippedTargets += 1
|
||||
snapshot.targets?.push(item)
|
||||
continue
|
||||
}
|
||||
|
||||
const jobId = randomUUID()
|
||||
await createIssueJobRecord({
|
||||
id: jobId,
|
||||
|
||||
@@ -0,0 +1,100 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { geoipSettings } from "../db/schema.js"
|
||||
import type { GeoipSettingsDto, GeoipSettingsPatch } from "@mmapp/contracts/geoip"
|
||||
|
||||
const SETTINGS_ID = 1
|
||||
const DEFAULT_INTERVAL_SEC = 604800
|
||||
|
||||
let dbEnabled = true
|
||||
|
||||
/** Тесты без PostgreSQL: геттеры отдают дефолты, touch/update — no-op. */
|
||||
export function disableGeoipDbForTests(): void {
|
||||
dbEnabled = false
|
||||
}
|
||||
|
||||
export function resetGeoipSettingsForTests(): void {
|
||||
dbEnabled = true
|
||||
}
|
||||
|
||||
type GeoipSettingsRow = typeof geoipSettings.$inferSelect
|
||||
|
||||
async function getGeoipSettingsRow(): Promise<GeoipSettingsRow | undefined> {
|
||||
if (!dbEnabled) return undefined
|
||||
return (
|
||||
(await db.select().from(geoipSettings).where(eq(geoipSettings.id, SETTINGS_ID)).limit(1))[0]
|
||||
)
|
||||
}
|
||||
|
||||
function toDto(row: GeoipSettingsRow | undefined): GeoipSettingsDto {
|
||||
return {
|
||||
enabled: row?.enabled ?? true,
|
||||
updateIntervalSec: row?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
|
||||
lastCheckAt: row?.lastCheckAt ?? null,
|
||||
lastSuccessAt: row?.lastSuccessAt ?? null,
|
||||
lastError: row?.lastError ?? null,
|
||||
countryBuildAt: row?.countryBuildAt ?? null,
|
||||
asnBuildAt: row?.asnBuildAt ?? null,
|
||||
updatedAt: row?.updatedAt ?? new Date().toISOString(),
|
||||
}
|
||||
}
|
||||
|
||||
export async function getGeoipSettings(): Promise<GeoipSettingsDto> {
|
||||
return toDto(await getGeoipSettingsRow())
|
||||
}
|
||||
|
||||
/** ETag'и зеркала для conditional GET (ключ — имя файла базы). */
|
||||
export async function getGeoipEtags(): Promise<Record<string, string>> {
|
||||
const row = await getGeoipSettingsRow()
|
||||
if (!row) return {}
|
||||
const raw = row?.etagsJson
|
||||
if (!raw || typeof raw !== "object") return {}
|
||||
return Object.fromEntries(
|
||||
Object.entries(raw as Record<string, unknown>).filter(
|
||||
(entry): entry is [string, string] => typeof entry[1] === "string",
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
export async function updateGeoipSettings(patch: GeoipSettingsPatch): Promise<GeoipSettingsDto> {
|
||||
const prev = await getGeoipSettingsRow()
|
||||
const next = {
|
||||
enabled: patch.enabled ?? prev?.enabled ?? true,
|
||||
updateIntervalSec: patch.updateIntervalSec ?? prev?.updateIntervalSec ?? DEFAULT_INTERVAL_SEC,
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
if (prev) {
|
||||
await db.update(geoipSettings).set(next).where(eq(geoipSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...next })
|
||||
}
|
||||
return getGeoipSettings()
|
||||
}
|
||||
|
||||
export async function touchGeoipRunMeta(patch: {
|
||||
lastCheckAt?: string
|
||||
lastSuccessAt?: string | null
|
||||
lastError?: string | null
|
||||
countryBuildAt?: string | null
|
||||
asnBuildAt?: string | null
|
||||
etags?: Record<string, string>
|
||||
}): Promise<void> {
|
||||
const prev = await getGeoipSettingsRow()
|
||||
const set: Partial<typeof geoipSettings.$inferInsert> = {
|
||||
updatedAt: new Date().toISOString(),
|
||||
}
|
||||
if (patch.lastCheckAt !== undefined) set.lastCheckAt = patch.lastCheckAt
|
||||
if (patch.lastSuccessAt !== undefined) set.lastSuccessAt = patch.lastSuccessAt
|
||||
if (patch.lastError !== undefined) set.lastError = patch.lastError
|
||||
if (patch.countryBuildAt !== undefined) set.countryBuildAt = patch.countryBuildAt
|
||||
if (patch.asnBuildAt !== undefined) set.asnBuildAt = patch.asnBuildAt
|
||||
if (patch.etags !== undefined) {
|
||||
const prevEtags = (prev?.etagsJson as Record<string, string> | null) ?? {}
|
||||
set.etagsJson = { ...prevEtags, ...patch.etags }
|
||||
}
|
||||
if (prev) {
|
||||
await db.update(geoipSettings).set(set).where(eq(geoipSettings.id, SETTINGS_ID))
|
||||
} else {
|
||||
await db.insert(geoipSettings).values({ id: SETTINGS_ID, ...set })
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,198 @@
|
||||
import { rename, rm, mkdir, writeFile } from "node:fs/promises"
|
||||
import path from "node:path"
|
||||
import { open, type AsnResponse, type CountryResponse } from "maxmind"
|
||||
import type { GeoipUpdateRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
|
||||
import { getGeoipEtags, getGeoipSettings, touchGeoipRunMeta } from "./geoip-settings.js"
|
||||
import {
|
||||
GEOIP_ASN_FILE,
|
||||
GEOIP_COUNTRY_FILE,
|
||||
geoipDir,
|
||||
reloadGeoipReaders,
|
||||
} from "./traffic-flow-geoip.js"
|
||||
|
||||
/** Зеркало GeoLite2 без регистрации и ключей (см. README: GeoIP). */
|
||||
const MIRROR_BASE = "https://github.com/P3TERX/GeoLite.mmdb/raw/download"
|
||||
const DOWNLOAD_TIMEOUT_MS = 120_000
|
||||
/** Пробный IP для валидации скачанной базы: Google DNS. */
|
||||
const PROBE_IP = "8.8.8.8"
|
||||
|
||||
type GeoipDbKind = "country" | "asn"
|
||||
|
||||
let updating = false
|
||||
let fetchImpl: typeof fetch = globalThis.fetch.bind(globalThis)
|
||||
|
||||
export function getGeoipUpdateState(): { running: boolean } {
|
||||
return { running: updating }
|
||||
}
|
||||
|
||||
async function validateCountryFile(filePath: string): Promise<string> {
|
||||
const reader = await open<CountryResponse>(filePath)
|
||||
const rec = reader.get(PROBE_IP)
|
||||
const iso = rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
|
||||
if (iso !== "US") {
|
||||
throw new Error(`база Country не распознала ${PROBE_IP} как US (${iso || "нет записи"})`)
|
||||
}
|
||||
return reader.metadata.buildEpoch.toISOString()
|
||||
}
|
||||
|
||||
async function validateAsnFile(filePath: string): Promise<string> {
|
||||
const reader = await open<AsnResponse>(filePath)
|
||||
const rec = reader.get(PROBE_IP)
|
||||
const asn = rec?.autonomous_system_number ?? 0
|
||||
if (asn !== 15169) {
|
||||
throw new Error(`база ASN не распознала ${PROBE_IP} как AS15169 (${asn ? `AS${asn}` : "нет записи"})`)
|
||||
}
|
||||
return reader.metadata.buildEpoch.toISOString()
|
||||
}
|
||||
|
||||
let validateCountry = validateCountryFile
|
||||
let validateAsn = validateAsnFile
|
||||
|
||||
export function setGeoipFetchForTests(fn: typeof fetch): void {
|
||||
fetchImpl = fn
|
||||
}
|
||||
|
||||
export function setGeoipValidateForTests(opts: {
|
||||
country?: (filePath: string) => Promise<string>
|
||||
asn?: (filePath: string) => Promise<string>
|
||||
}): void {
|
||||
validateCountry = opts.country ?? validateCountryFile
|
||||
validateAsn = opts.asn ?? validateAsnFile
|
||||
}
|
||||
|
||||
export function resetGeoipUpdateForTests(): void {
|
||||
updating = false
|
||||
fetchImpl = globalThis.fetch.bind(globalThis)
|
||||
validateCountry = validateCountryFile
|
||||
validateAsn = validateAsnFile
|
||||
}
|
||||
|
||||
/**
|
||||
* Разовая проверка/доставка баз с зеркала P3TERX. Conditional GET по ETag
|
||||
* (304 = не меняем файл), валидация пробоем 8.8.8.8, атомарная подмена через rename.
|
||||
*/
|
||||
export async function collectGeoipUpdateOnce(
|
||||
opts: { force?: boolean } = {},
|
||||
): Promise<GeoipUpdateRunSnapshot> {
|
||||
const sampledAt = new Date().toISOString()
|
||||
if (updating) {
|
||||
if (opts.force) {
|
||||
throw Object.assign(new Error("Обновление GeoIP уже выполняется"), { statusCode: 409 })
|
||||
}
|
||||
return emptySnapshot(sampledAt, true)
|
||||
}
|
||||
|
||||
const settings = await getGeoipSettings()
|
||||
if (!settings.enabled && !opts.force) {
|
||||
return emptySnapshot(sampledAt, true)
|
||||
}
|
||||
|
||||
updating = true
|
||||
const snapshot: GeoipUpdateRunSnapshot = {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "geoip_update",
|
||||
sampledAt,
|
||||
checked: 0,
|
||||
downloaded: 0,
|
||||
skippedUnchanged: 0,
|
||||
bytes: 0,
|
||||
errors: [],
|
||||
}
|
||||
|
||||
try {
|
||||
const dir = geoipDir()
|
||||
await mkdir(dir, { recursive: true })
|
||||
const storedEtags = await getGeoipEtags()
|
||||
const etags: Record<string, string> = {}
|
||||
const buildAt: Partial<Record<GeoipDbKind, string>> = {}
|
||||
|
||||
for (const kind of ["country", "asn"] as const) {
|
||||
snapshot.checked += 1
|
||||
const file = kind === "country" ? GEOIP_COUNTRY_FILE : GEOIP_ASN_FILE
|
||||
const target = path.join(dir, file)
|
||||
const tmp = `${target}.tmp`
|
||||
const prevEtag = storedEtags[file]
|
||||
try {
|
||||
const ac = new AbortController()
|
||||
const timer = setTimeout(() => ac.abort(), DOWNLOAD_TIMEOUT_MS)
|
||||
let res: Response
|
||||
try {
|
||||
res = await fetchImpl(`${MIRROR_BASE}/${file}`, {
|
||||
headers: prevEtag ? { "If-None-Match": prevEtag } : {},
|
||||
signal: ac.signal,
|
||||
})
|
||||
} finally {
|
||||
clearTimeout(timer)
|
||||
}
|
||||
if (res.status === 304) {
|
||||
snapshot.skippedUnchanged += 1
|
||||
if (prevEtag) etags[file] = prevEtag
|
||||
continue
|
||||
}
|
||||
if (!res.ok) throw new Error(`HTTP ${res.status}`)
|
||||
const etag = res.headers.get("etag") ?? ""
|
||||
const body = Buffer.from(await res.arrayBuffer())
|
||||
snapshot.bytes += body.byteLength
|
||||
await writeFile(tmp, body)
|
||||
buildAt[kind] =
|
||||
kind === "country" ? await validateCountry(tmp) : await validateAsn(tmp)
|
||||
|
||||
const prevFile = `${target}.prev`
|
||||
await rm(prevFile, { force: true })
|
||||
await rename(target, prevFile).catch(() => {
|
||||
/* текущего файла могло ещё не быть */
|
||||
})
|
||||
await rename(tmp, target)
|
||||
snapshot.downloaded += 1
|
||||
if (etag) etags[file] = etag
|
||||
} catch (e) {
|
||||
await rm(tmp, { force: true }).catch(() => {
|
||||
/* best-effort */
|
||||
})
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.errors.push(`${file}: ${message}`)
|
||||
}
|
||||
}
|
||||
|
||||
if (snapshot.downloaded > 0) {
|
||||
await reloadGeoipReaders()
|
||||
}
|
||||
|
||||
await touchGeoipRunMeta({
|
||||
lastCheckAt: sampledAt,
|
||||
lastSuccessAt: snapshot.errors.length ? null : sampledAt,
|
||||
lastError: snapshot.errors.length ? snapshot.errors.join("; ") : null,
|
||||
countryBuildAt: buildAt.country,
|
||||
asnBuildAt: buildAt.asn,
|
||||
etags,
|
||||
})
|
||||
return snapshot
|
||||
} catch (e) {
|
||||
const message = e instanceof Error ? e.message : String(e)
|
||||
snapshot.fatalError = message
|
||||
await touchGeoipRunMeta({
|
||||
lastCheckAt: sampledAt,
|
||||
lastError: message,
|
||||
}).catch(() => {
|
||||
/* best-effort */
|
||||
})
|
||||
return snapshot
|
||||
} finally {
|
||||
updating = false
|
||||
}
|
||||
}
|
||||
|
||||
function emptySnapshot(sampledAt: string, skipped: boolean): GeoipUpdateRunSnapshot {
|
||||
return {
|
||||
v: SCHEDULER_RUN_SNAPSHOT_VERSION,
|
||||
job: "geoip_update",
|
||||
sampledAt,
|
||||
skipped,
|
||||
checked: 0,
|
||||
downloaded: 0,
|
||||
skippedUnchanged: 0,
|
||||
bytes: 0,
|
||||
errors: [],
|
||||
}
|
||||
}
|
||||
@@ -1,4 +1,4 @@
|
||||
import { and, asc, desc, eq, lt } from "drizzle-orm"
|
||||
import { and, asc, desc, eq } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import {
|
||||
filterRules,
|
||||
@@ -45,11 +45,6 @@ async function getSettingsRow() {
|
||||
return (await db.select().from(internetPathSettings).where(eq(internetPathSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
async function cleanupSnapshots(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
await db.delete(internetPathSnapshots).where(lt(internetPathSnapshots.sampledAt, cutoff))
|
||||
}
|
||||
|
||||
async function buildRulesets() {
|
||||
const enabled = await db.select().from(servers).where(eq(servers.enabled, true))
|
||||
const rules = await db.select().from(filterRules).orderBy(asc(filterRules.serverId), asc(filterRules.sortOrder))
|
||||
@@ -273,7 +268,6 @@ export async function collectInternetPathSnapshotOnce(): Promise<InternetPathRun
|
||||
sampledAt,
|
||||
payloadJson: payload,
|
||||
})
|
||||
await cleanupSnapshots(Math.max(1, settings.retentionDays))
|
||||
await db.update(internetPathSettings).set({
|
||||
lastCollectedAt: sampledAt,
|
||||
lastDurationMs: Date.now() - started,
|
||||
|
||||
@@ -586,6 +586,14 @@ export class MikrotikClient {
|
||||
: new Error(`Не удалось загрузить файл ${normalized} на RouterOS`)
|
||||
}
|
||||
|
||||
async importUploadedFile(fileName: string): Promise<unknown> {
|
||||
try {
|
||||
return await this.post("/import", { "file-name": fileName }, 120_000)
|
||||
} catch {
|
||||
return await this.post("/execute", { script: `/import file-name="${fileName}"` }, 120_000)
|
||||
}
|
||||
}
|
||||
|
||||
async importCertificate(params: {
|
||||
fileName: string
|
||||
name: string
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import { eq, lt } from "drizzle-orm"
|
||||
import { db, pool } from "../db/index.js"
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db, dbQuery, pool } from "../db/index.js"
|
||||
import { dropExpiredPartitions, ensurePartitionsAround } from "../db/partitions.js"
|
||||
import { servers, serverSnapshots } from "../db/schema.js"
|
||||
import type { SnapshotInsert } from "../db/schema.js"
|
||||
@@ -82,11 +82,18 @@ export async function pollServer(serverId: number): Promise<SnapshotRead> {
|
||||
.values(partialSnap as SnapshotInsert)
|
||||
.returning()
|
||||
|
||||
const cutoffIso = new Date(Date.now() - 14 * 24 * 3600_000).toISOString()
|
||||
await db.delete(serverSnapshots).where(lt(serverSnapshots.polledAt, cutoffIso))
|
||||
if (inserted) {
|
||||
await dbQuery(
|
||||
`UPDATE server_snapshots
|
||||
SET raw_interfaces = NULL, raw_ip_addresses = NULL
|
||||
WHERE server_id = $1 AND polled_at < $2
|
||||
AND (raw_interfaces IS NOT NULL OR raw_ip_addresses IS NOT NULL)`,
|
||||
[serverId, inserted.polledAt],
|
||||
)
|
||||
}
|
||||
void dropExpiredPartitions(pool).then(() => ensurePartitionsAround(pool))
|
||||
|
||||
return toSnapshotRead(inserted)
|
||||
return toSnapshotRead(inserted!)
|
||||
}
|
||||
|
||||
// ── helper ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
@@ -0,0 +1,89 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadBucketCommand,
|
||||
ListObjectsV2Command,
|
||||
PutObjectCommand,
|
||||
type S3Client,
|
||||
} from "@aws-sdk/client-s3"
|
||||
import {
|
||||
buildS3ObjectKey,
|
||||
normalizeS3Prefix,
|
||||
parseS3ObjectKey,
|
||||
sanitizeServerName,
|
||||
s3DeleteObject,
|
||||
s3GetObject,
|
||||
s3ListObjects,
|
||||
s3PutObject,
|
||||
s3TestConnection,
|
||||
} from "./s3-backup-client.js"
|
||||
|
||||
assert.equal(normalizeS3Prefix("/mikrotik/backups/"), "mikrotik/backups")
|
||||
assert.equal(sanitizeServerName("MSK CHR 01"), "MSK_CHR_01")
|
||||
assert.equal(
|
||||
buildS3ObjectKey("mikrotik", "msk-chr01", "chr_2026-09-08_03-00-00.rsc"),
|
||||
"mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc",
|
||||
)
|
||||
assert.deepEqual(
|
||||
parseS3ObjectKey("mikrotik/msk-chr01/chr_2026-09-08_03-00-00.rsc"),
|
||||
{ filename: "chr_2026-09-08_03-00-00.rsc", serverName: "msk-chr01" },
|
||||
)
|
||||
|
||||
const store = new Map<string, Buffer>()
|
||||
let lastCommand = ""
|
||||
|
||||
const fake = {
|
||||
send: async (command: { input?: Record<string, unknown> }) => {
|
||||
const name = command.constructor.name
|
||||
lastCommand = name
|
||||
const input = command.input ?? {}
|
||||
if (command instanceof HeadBucketCommand || name === "HeadBucketCommand") {
|
||||
if (input.Bucket !== "backups") throw new Error("no bucket")
|
||||
return {}
|
||||
}
|
||||
if (command instanceof PutObjectCommand || name === "PutObjectCommand") {
|
||||
const key = String(input.Key)
|
||||
const body = input.Body
|
||||
store.set(key, Buffer.isBuffer(body) ? body : Buffer.from(String(body)))
|
||||
return { ETag: '"etag-1"' }
|
||||
}
|
||||
if (command instanceof GetObjectCommand || name === "GetObjectCommand") {
|
||||
const key = String(input.Key)
|
||||
const body = store.get(key)
|
||||
if (!body) throw new Error("not found")
|
||||
return { Body: { transformToByteArray: async () => new Uint8Array(body) } }
|
||||
}
|
||||
if (command instanceof DeleteObjectCommand || name === "DeleteObjectCommand") {
|
||||
store.delete(String(input.Key))
|
||||
return {}
|
||||
}
|
||||
if (command instanceof ListObjectsV2Command || name === "ListObjectsV2Command") {
|
||||
const prefix = String(input.Prefix ?? "")
|
||||
const contents = [...store.entries()]
|
||||
.filter(([key]) => !prefix || key.startsWith(prefix))
|
||||
.map(([key, buf]) => ({ Key: key, Size: buf.length, LastModified: new Date("2026-09-08T00:00:00Z") }))
|
||||
return { Contents: contents, IsTruncated: false }
|
||||
}
|
||||
throw new Error(`unexpected command ${name}`)
|
||||
},
|
||||
} as unknown as S3Client
|
||||
|
||||
await s3TestConnection(fake, "backups")
|
||||
assert.equal(lastCommand === "HeadBucketCommand" || lastCommand.includes("Head"), true)
|
||||
|
||||
const put = await s3PutObject(fake, "backups", "mikrotik/a/file.rsc", "hello")
|
||||
assert.equal(put.etag, '"etag-1"')
|
||||
|
||||
const got = await s3GetObject(fake, "backups", "mikrotik/a/file.rsc")
|
||||
assert.equal(got.toString("utf8"), "hello")
|
||||
|
||||
const listed = await s3ListObjects(fake, "backups", "mikrotik")
|
||||
assert.equal(listed.length, 1)
|
||||
assert.equal(listed[0]?.key, "mikrotik/a/file.rsc")
|
||||
|
||||
await s3DeleteObject(fake, "backups", "mikrotik/a/file.rsc")
|
||||
const after = await s3ListObjects(fake, "backups", "mikrotik")
|
||||
assert.equal(after.length, 0)
|
||||
|
||||
console.log("s3-backup-client.test.ts: ok")
|
||||
@@ -0,0 +1,128 @@
|
||||
import {
|
||||
DeleteObjectCommand,
|
||||
GetObjectCommand,
|
||||
HeadBucketCommand,
|
||||
ListObjectsV2Command,
|
||||
PutObjectCommand,
|
||||
S3Client,
|
||||
type S3ClientConfig,
|
||||
} from "@aws-sdk/client-s3"
|
||||
|
||||
export type S3BackupConfig = {
|
||||
endpoint: string
|
||||
region: string
|
||||
bucket: string
|
||||
prefix: string
|
||||
accessKeyId: string
|
||||
secretAccessKey: string
|
||||
forcePathStyle: boolean
|
||||
}
|
||||
|
||||
export type S3ListedObject = {
|
||||
key: string
|
||||
size: number
|
||||
lastModified?: string
|
||||
}
|
||||
|
||||
export function normalizeS3Prefix(prefix: string): string {
|
||||
return prefix.trim().replace(/^\/+|\/+$/g, "")
|
||||
}
|
||||
|
||||
export function sanitizeServerName(name: string): string {
|
||||
const safe = name.replace(/[^a-zA-Z0-9._-]+/g, "_").replace(/^_+|_+$/g, "")
|
||||
return safe || "server"
|
||||
}
|
||||
|
||||
export function buildS3ObjectKey(prefix: string, serverSafe: string, filename: string): string {
|
||||
const parts = [normalizeS3Prefix(prefix), sanitizeServerName(serverSafe), filename]
|
||||
.filter((part) => part.length > 0)
|
||||
return parts.join("/")
|
||||
}
|
||||
|
||||
export function parseS3ObjectKey(key: string): { filename: string; serverName: string } {
|
||||
const parts = key.split("/").filter(Boolean)
|
||||
const filename = parts.pop() ?? key
|
||||
const folder = parts.pop() ?? ""
|
||||
const base = filename.replace(/\.(rsc|backup)$/i, "")
|
||||
const fromFilename = base.replace(/_\d{4}-\d{2}-\d{2}_\d{2}-\d{2}-\d{2}$/, "")
|
||||
return { filename, serverName: folder || fromFilename || filename }
|
||||
}
|
||||
|
||||
export function createS3ClientFromConfig(cfg: S3BackupConfig): S3Client {
|
||||
const options: S3ClientConfig = {
|
||||
region: cfg.region.trim() || "us-east-1",
|
||||
credentials: {
|
||||
accessKeyId: cfg.accessKeyId,
|
||||
secretAccessKey: cfg.secretAccessKey,
|
||||
},
|
||||
forcePathStyle: cfg.forcePathStyle,
|
||||
}
|
||||
const endpoint = cfg.endpoint.trim()
|
||||
if (endpoint) options.endpoint = endpoint
|
||||
return new S3Client(options)
|
||||
}
|
||||
|
||||
export async function s3TestConnection(client: S3Client, bucket: string): Promise<void> {
|
||||
try {
|
||||
await client.send(new HeadBucketCommand({ Bucket: bucket }))
|
||||
} catch {
|
||||
await client.send(new ListObjectsV2Command({ Bucket: bucket, MaxKeys: 1 }))
|
||||
}
|
||||
}
|
||||
|
||||
export async function s3PutObject(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
key: string,
|
||||
body: Buffer | string,
|
||||
): Promise<{ etag?: string }> {
|
||||
const out = await client.send(new PutObjectCommand({
|
||||
Bucket: bucket,
|
||||
Key: key,
|
||||
Body: body,
|
||||
ContentType: "text/plain; charset=utf-8",
|
||||
}))
|
||||
return { etag: out.ETag }
|
||||
}
|
||||
|
||||
export async function s3GetObject(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
key: string,
|
||||
): Promise<Buffer> {
|
||||
const out = await client.send(new GetObjectCommand({ Bucket: bucket, Key: key }))
|
||||
const bytes = await out.Body?.transformToByteArray()
|
||||
if (!bytes) throw new Error("Пустой объект S3")
|
||||
return Buffer.from(bytes)
|
||||
}
|
||||
|
||||
export async function s3DeleteObject(client: S3Client, bucket: string, key: string): Promise<void> {
|
||||
await client.send(new DeleteObjectCommand({ Bucket: bucket, Key: key }))
|
||||
}
|
||||
|
||||
export async function s3ListObjects(
|
||||
client: S3Client,
|
||||
bucket: string,
|
||||
prefix: string,
|
||||
): Promise<S3ListedObject[]> {
|
||||
const items: S3ListedObject[] = []
|
||||
let token: string | undefined
|
||||
const normalized = normalizeS3Prefix(prefix)
|
||||
do {
|
||||
const out = await client.send(new ListObjectsV2Command({
|
||||
Bucket: bucket,
|
||||
Prefix: normalized ? `${normalized}/` : undefined,
|
||||
ContinuationToken: token,
|
||||
}))
|
||||
for (const obj of out.Contents ?? []) {
|
||||
if (!obj.Key) continue
|
||||
items.push({
|
||||
key: obj.Key,
|
||||
size: obj.Size ?? 0,
|
||||
lastModified: obj.LastModified?.toISOString(),
|
||||
})
|
||||
}
|
||||
token = out.IsTruncated ? out.NextContinuationToken : undefined
|
||||
} while (token)
|
||||
return items
|
||||
}
|
||||
@@ -46,6 +46,8 @@ import { collectCertificatesRenewOnce } from "./certificate-renew-collector.js"
|
||||
import { getCertificateRenewSettings } from "./certificates-service.js"
|
||||
import { collectScheduledBackupsOnce } from "./backup-scheduler-collector.js"
|
||||
import { getBackupScheduleSettings } from "./backup-service.js"
|
||||
import { collectGeoipUpdateOnce } from "./geoip-update-collector.js"
|
||||
import { getGeoipSettings } from "./geoip-settings.js"
|
||||
import {
|
||||
endSchedulerJob,
|
||||
isSchedulerJobRunning,
|
||||
@@ -63,6 +65,7 @@ export const JOB_KEYS = [
|
||||
"gre_bgp",
|
||||
"certificates_renew",
|
||||
"backups",
|
||||
"geoip_update",
|
||||
"alert_engine",
|
||||
] as const
|
||||
export type SchedulerJobKey = (typeof JOB_KEYS)[number]
|
||||
@@ -148,6 +151,9 @@ async function runSchedulerJobBody(jobKey: SchedulerJobKey): Promise<void> {
|
||||
case "backups":
|
||||
snapshot = await collectScheduledBackupsOnce()
|
||||
break
|
||||
case "geoip_update":
|
||||
snapshot = await collectGeoipUpdateOnce()
|
||||
break
|
||||
case "alert_engine": {
|
||||
const r = await runAlertEngineOnce()
|
||||
snapshot = {
|
||||
@@ -377,6 +383,18 @@ export async function refreshScheduler(): Promise<void> {
|
||||
)
|
||||
}
|
||||
|
||||
const geoip = await getGeoipSettings()
|
||||
if (geoip.enabled) {
|
||||
const geoipMs = Math.max(6 * 3600_000, geoip.updateIntervalSec * 1000)
|
||||
void executeSchedulerJob("geoip_update").catch(() => {})
|
||||
timers.set(
|
||||
"geoip_update",
|
||||
setInterval(() => {
|
||||
void executeSchedulerJob("geoip_update").catch(() => {})
|
||||
}, geoipMs),
|
||||
)
|
||||
}
|
||||
|
||||
const alertMs = 20_000
|
||||
void executeSchedulerJob("alert_engine").catch(() => {})
|
||||
timers.set(
|
||||
@@ -409,6 +427,7 @@ export async function getSchedulerStatus() {
|
||||
const internetPath = await getInternetPathSettings()
|
||||
const certRenew = await getCertificateRenewSettings()
|
||||
const backupSchedule = await getBackupScheduleSettings()
|
||||
const geoip = await getGeoipSettings()
|
||||
|
||||
const resOn = uptime.resourcesEnabled ?? uptime.enabled
|
||||
const pingOn = uptime.pingEnabled ?? uptime.enabled
|
||||
@@ -424,6 +443,7 @@ export async function getSchedulerStatus() {
|
||||
gre_bgp: { enabled: true, intervalSec: 30 },
|
||||
certificates_renew: { enabled: certRenew.enabled, intervalSec: certRenew.intervalSec },
|
||||
backups: { enabled: backupSchedule.enabled, intervalSec: 60 },
|
||||
geoip_update: { enabled: geoip.enabled, intervalSec: geoip.updateIntervalSec },
|
||||
alert_engine: { enabled: true, intervalSec: 20 },
|
||||
}
|
||||
|
||||
|
||||
@@ -162,8 +162,6 @@ export async function collectServersRestPingOnce(): Promise<ServersRestPingRunSn
|
||||
}
|
||||
})
|
||||
|
||||
await dbQuery(`DELETE FROM servers_rest_ping_samples WHERE sampled_at < now() - interval '30 days'`)
|
||||
|
||||
await db.update(serversApiPingSettings)
|
||||
.set({
|
||||
lastCollectedAt: sampledAt,
|
||||
|
||||
@@ -0,0 +1,285 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { getStatistics, getStatisticsPivot, parseStatisticsPeriod, pivotDimsConflict } from "./statistics-aggregate.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
import { setRefreshIfacesForTests } from "./traffic-flow-ifaces.js"
|
||||
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { ensurePartitionFor } from "../db/partitions.js"
|
||||
import { pool } from "../db/index.js"
|
||||
import { STATISTICS_UNBOUND_USER_ID } from "@mmapp/contracts/statistics"
|
||||
|
||||
{
|
||||
const sameDay = parseStatisticsPeriod("2026-09-10", "2026-09-10")
|
||||
assert.ok(sameDay)
|
||||
assert.equal(sameDay.fromDay, "2026-09-10")
|
||||
assert.equal(sameDay.toDayExclusive, "2026-09-11")
|
||||
assert.equal(sameDay.grain, "hour")
|
||||
const month = parseStatisticsPeriod("2026-08-01", "2026-08-31")
|
||||
assert.ok(month)
|
||||
assert.equal(month.grain, "day")
|
||||
assert.equal(month.toDayExclusive, "2026-09-01")
|
||||
assert.equal(parseStatisticsPeriod("2026-09-10", "2026-09-09"), null)
|
||||
assert.equal(pivotDimsConflict("country", "country"), true)
|
||||
assert.equal(pivotDimsConflict("country", "service"), false)
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("statistics-aggregate.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const inserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type, wan_uplinks)
|
||||
VALUES ('stats-cube', '127.0.0.1', 'jump-host', '[{"iface":"wan1"}]'::jsonb)
|
||||
RETURNING id
|
||||
`)
|
||||
const serverId = inserted.rows[0]?.id
|
||||
if (serverId == null) throw new Error("no server")
|
||||
|
||||
const enInserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type)
|
||||
VALUES ('stats-en', '198.51.100.1', 'exit-node')
|
||||
RETURNING id
|
||||
`)
|
||||
const enId = enInserted.rows[0]?.id
|
||||
if (enId == null) throw new Error("no en server")
|
||||
|
||||
await ensurePartitionFor(pool, "flow_daily_facts", "month", new Date("2026-09-01T00:00:00Z"))
|
||||
await ensurePartitionFor(pool, "flow_hour_facts", "day", new Date("2026-09-10T00:00:00Z"))
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-stats-1'`)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login, role, active)
|
||||
VALUES ('u-stats-1', 'Клиент', 'stats-user', 'viewer', TRUE)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-stats-1', 'u-stats-1', $1, 'gre-client', 'gre')
|
||||
`, [serverId])
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
|
||||
VALUES
|
||||
($1, '2026-09-10T12:00:00Z', 'online', $3::jsonb),
|
||||
($2, '2026-09-10T12:00:00Z', 'online', $4::jsonb)
|
||||
`, [
|
||||
serverId,
|
||||
enId,
|
||||
JSON.stringify([
|
||||
{ name: "gre-client", type: "gre-tunnel" },
|
||||
{ name: "wan1", type: "ether" },
|
||||
{ name: "gre-en", type: "gre-tunnel" },
|
||||
{ name: "NSK-SERVHOST-RTK", type: "gre-tunnel" },
|
||||
{ name: "wg-mesh", type: "wg" },
|
||||
{ name: "wg-server", type: "wg" },
|
||||
{ name: "wg-flow", type: "wg" },
|
||||
]),
|
||||
JSON.stringify([
|
||||
{ name: "ether1", type: "ether" },
|
||||
{ name: "gre-jh", type: "gre-tunnel" },
|
||||
]),
|
||||
])
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
rememberServerIfaces(serverId, [
|
||||
{ name: "gre-client", ifindex: "2" },
|
||||
{ name: "wan1", ifindex: "8" },
|
||||
{ name: "gre-en", ifindex: "9" },
|
||||
{ name: "NSK-SERVHOST-RTK" },
|
||||
{ name: "wg-mesh" },
|
||||
{ name: "wg-server" },
|
||||
{ name: "wg-flow" },
|
||||
])
|
||||
rememberServerIfaces(enId, [
|
||||
{ name: "ether1", ifindex: "2" },
|
||||
{ name: "gre-jh", ifindex: "5" },
|
||||
])
|
||||
setRefreshIfacesForTests(async () => {})
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
|
||||
VALUES
|
||||
($1, '2026-09-10', '2', 'US', 'https', 15169, 800, 10),
|
||||
($1, '2026-09-10', '2', 'DE', 'dns', 15133, 200, 4),
|
||||
($1, '2026-09-10', 'wan1', 'NL', 'other', 0, 70, 1),
|
||||
($1, '2026-09-10', '0', 'US', 'https', 0, 999, 3),
|
||||
($1, '2026-09-10', 'gre-en', 'US', 'https', 15169, 400, 2),
|
||||
($1, '2026-09-10', 'NSK-SERVHOST-RTK', 'US', 'https', 15169, 300, 2),
|
||||
($1, '2026-09-10', 'wg-mesh', 'US', 'https', 0, 250, 2),
|
||||
($1, '2026-09-10', 'wg-flow', 'US', 'https', 0, 80, 1),
|
||||
($2, '2026-09-10', 'gre-jh', 'US', 'https', 15169, 500, 5),
|
||||
($2, '2026-09-10', 'ether1', 'US', 'https', 15169, 200, 2)
|
||||
`, [serverId, enId])
|
||||
|
||||
try {
|
||||
const unique = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
|
||||
assert.equal(unique.grain, "day")
|
||||
assert.equal(unique.kpis.bytes, 1000)
|
||||
const uniqueAsnSum = unique.asns.reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(uniqueAsnSum, unique.kpis.bytes, "unique KPI = SUM dest ASN")
|
||||
assert.equal(unique.kpis.users, 1)
|
||||
assert.ok(unique.countries.some((r) => r.id === "US"))
|
||||
assert.ok(unique.users.some((r) => r.id === "u-stats-1"))
|
||||
assert.equal(unique.users.find((r) => r.id === STATISTICS_UNBOUND_USER_ID), undefined)
|
||||
assert.ok(unique.servers.some((r) => r.id === String(serverId)))
|
||||
assert.ok(!unique.servers.some((r) => r.id === String(enId)), "EN-транзит не в сетевом KPI")
|
||||
const greIface = unique.interfaces.find((r) => r.label.includes("gre-client"))
|
||||
assert.ok(greIface)
|
||||
assert.equal(greIface.bytes, 1000)
|
||||
assert.equal(greIface.id, `${serverId}:gre-client`)
|
||||
assert.ok(!unique.interfaces.some((r) => /· (?:#)?\d+$/.test(r.label)))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes(" · —") || r.label.endsWith("· —")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("gre-en")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("NSK-SERVHOST-RTK")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-mesh")))
|
||||
assert.ok(!unique.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
assert.equal(unique.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined, "unique без WAN")
|
||||
|
||||
const allPlanes = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "all" })
|
||||
assert.equal(allPlanes.kpis.bytes, 1000, "KPI unique и all одинаковый")
|
||||
const wanRow = allPlanes.interfaces.find((r) => r.id === `${serverId}:wan1`)
|
||||
assert.ok(wanRow)
|
||||
assert.ok(wanRow.label.includes("WAN · интернет"))
|
||||
assert.equal(wanRow.bytes, 70)
|
||||
assert.equal(wanRow.percent, 0)
|
||||
const overlayGre = allPlanes.interfaces.find((r) => r.id === `${serverId}:gre-en`)
|
||||
assert.ok(overlayGre)
|
||||
assert.ok(overlayGre.label.includes("дубль"))
|
||||
assert.equal(overlayGre.percent, 0)
|
||||
const overlayCustom = allPlanes.interfaces.find((r) => r.label.includes("NSK-SERVHOST-RTK"))
|
||||
assert.ok(overlayCustom)
|
||||
assert.ok(overlayCustom.label.includes("дубль"))
|
||||
const overlayWg = allPlanes.interfaces.find((r) => r.label.includes("wg-mesh"))
|
||||
assert.ok(overlayWg)
|
||||
assert.ok(overlayWg.label.includes("дубль"))
|
||||
assert.ok(!allPlanes.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
|
||||
const wanSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
iface: "wan1",
|
||||
})
|
||||
assert.equal(wanSlice.kpis.bytes, 70)
|
||||
|
||||
const nodeSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
planes: "unique",
|
||||
})
|
||||
assert.equal(nodeSlice.kpis.bytes, 1000)
|
||||
assert.equal(nodeSlice.interfaces.find((r) => r.id === `${serverId}:wan1`), undefined)
|
||||
assert.ok(!nodeSlice.users.some((r) => r.id === STATISTICS_UNBOUND_USER_ID))
|
||||
|
||||
const nodeAll = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId,
|
||||
planes: "all",
|
||||
})
|
||||
assert.equal(nodeAll.kpis.bytes, 1000)
|
||||
const nodeWan = nodeAll.interfaces.find((r) => r.id === `${serverId}:wan1`)
|
||||
assert.ok(nodeWan)
|
||||
assert.equal(nodeWan.percent, 0)
|
||||
assert.ok(nodeWan.label.includes("WAN · интернет"))
|
||||
|
||||
const enSlice = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId: enId,
|
||||
planes: "unique",
|
||||
})
|
||||
assert.equal(enSlice.kpis.bytes, 0)
|
||||
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("gre-jh")))
|
||||
assert.ok(!enSlice.interfaces.some((r) => r.label.includes("WAN · интернет")))
|
||||
|
||||
const enAll = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
serverId: enId,
|
||||
planes: "all",
|
||||
})
|
||||
assert.equal(enAll.kpis.bytes, 0)
|
||||
assert.ok(enAll.interfaces.some((r) => r.label.includes("WAN · интернет") && r.label.includes("ether1") && r.percent === 0))
|
||||
assert.ok(enAll.interfaces.some((r) => r.label.includes("gre-jh") && r.label.includes("дубль")))
|
||||
|
||||
const sliced = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
})
|
||||
assert.equal(sliced.kpis.bytes, 800)
|
||||
assert.equal(sliced.countries.length, 1)
|
||||
assert.equal(sliced.countries[0]?.id, "US")
|
||||
assert.ok(sliced.users.some((r) => r.id === "u-stats-1"))
|
||||
|
||||
const byUser = await getStatistics({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
userId: "u-stats-1",
|
||||
})
|
||||
assert.equal(byUser.kpis.bytes, 1000)
|
||||
|
||||
const pivot = await getStatisticsPivot({
|
||||
from: "2026-09-01",
|
||||
to: "2026-09-30",
|
||||
row: "country",
|
||||
col: "service",
|
||||
metric: "bytes",
|
||||
})
|
||||
const us = pivot.rows.find((r) => r.id === "US")
|
||||
const de = pivot.rows.find((r) => r.id === "DE")
|
||||
assert.ok(us)
|
||||
assert.ok(de)
|
||||
assert.equal(us.cells.https, 800)
|
||||
assert.equal(de.cells.dns, 200)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-stats-wg', 'u-stats-1', $1, 'wg-server', 'wg')
|
||||
`, [serverId])
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_daily_facts (server_id, day, iface, country, service, asn, bytes, packets)
|
||||
VALUES ($1, '2026-09-10', 'wg-server', 'US', 'https', 15169, 150, 2)
|
||||
`, [serverId])
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
const withWg = await getStatistics({ from: "2026-09-01", to: "2026-09-30", planes: "unique" })
|
||||
assert.equal(withWg.kpis.bytes, 1150)
|
||||
assert.ok(withWg.interfaces.some((r) => r.label.includes("wg-server") && r.bytes === 150))
|
||||
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-flow")))
|
||||
assert.ok(!withWg.interfaces.some((r) => r.label.includes("wg-mesh")))
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_hour_facts (server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
VALUES ($1, '2026-09-10T10:00:00Z', '2', 'US', 'https', 15169, 40, 2)
|
||||
`, [serverId])
|
||||
const hourly = await getStatistics({
|
||||
from: "2026-09-10T00:00:00.000Z",
|
||||
to: "2026-09-10T23:00:00.000Z",
|
||||
})
|
||||
assert.equal(hourly.grain, "hour")
|
||||
assert.equal(hourly.kpis.bytes, 40)
|
||||
assert.ok(hourly.users.some((r) => r.id === "u-stats-1"))
|
||||
} finally {
|
||||
setRefreshIfacesForTests(null)
|
||||
resetIfaceCacheForTests()
|
||||
invalidateFlowCatalogCache()
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id IN ($1, $2)`, [serverId, enId])
|
||||
await dbQuery(`DELETE FROM servers WHERE id IN ($1, $2)`, [serverId, enId])
|
||||
}
|
||||
|
||||
console.log("statistics-aggregate.test.ts: ok")
|
||||
@@ -0,0 +1,885 @@
|
||||
import { eq } from "drizzle-orm"
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { appUsers, flowAsnMeta, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import {
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
type StatisticsBreakdownRow,
|
||||
type StatisticsDto,
|
||||
type StatisticsPivotDim,
|
||||
type StatisticsPivotDto,
|
||||
type StatisticsPivotQuery,
|
||||
type StatisticsQuery,
|
||||
} from "@mmapp/contracts/statistics"
|
||||
import {
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
listCachedIfaceNames,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
import { refreshServerIfaces } from "./traffic-flow-ifaces.js"
|
||||
import {
|
||||
isDashDisplayIface,
|
||||
isJunkFactIface,
|
||||
isOverlayTunnelIface,
|
||||
isWanFactIface,
|
||||
overlayDupLabel,
|
||||
wanIfaceLabel,
|
||||
} from "./traffic-flow-facts-filter.js"
|
||||
import { getServerCatalog, loadFlowTopology, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
const TOP_N = 200
|
||||
const HOUR_WINDOW_MS = 48 * 3600_000
|
||||
const PIVOT_ROW_CAP = 50
|
||||
const PIVOT_COL_CAP = 15
|
||||
const PIVOT_OTHER_ID = "__other__"
|
||||
|
||||
export interface ParsedPeriod {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
grain: "hour" | "day"
|
||||
windowSec: number
|
||||
}
|
||||
|
||||
function pad2(n: number): string {
|
||||
return String(n).padStart(2, "0")
|
||||
}
|
||||
|
||||
function toUtcDay(d: Date): string {
|
||||
return `${d.getUTCFullYear()}-${pad2(d.getUTCMonth() + 1)}-${pad2(d.getUTCDate())}`
|
||||
}
|
||||
|
||||
function addUtcDays(day: string, n: number): string {
|
||||
const d = new Date(`${day}T00:00:00Z`)
|
||||
d.setUTCDate(d.getUTCDate() + n)
|
||||
return toUtcDay(d)
|
||||
}
|
||||
|
||||
/** Parse from/to. Date-only `to` is inclusive (end of that UTC day). */
|
||||
export function parseStatisticsPeriod(fromRaw: string, toRaw: string): ParsedPeriod | null {
|
||||
const from = Date.parse(fromRaw.includes("T") ? fromRaw : `${fromRaw}T00:00:00Z`)
|
||||
const toHasTime = toRaw.includes("T")
|
||||
const to = Date.parse(toHasTime ? toRaw : `${toRaw}T00:00:00Z`)
|
||||
if (!Number.isFinite(from) || !Number.isFinite(to)) return null
|
||||
const fromDate = new Date(from)
|
||||
let toDate = new Date(to)
|
||||
let toDayExclusive: string
|
||||
if (toHasTime) {
|
||||
toDayExclusive = toUtcDay(toDate)
|
||||
if (toDate.getUTCHours() !== 0 || toDate.getUTCMinutes() !== 0 || toDate.getUTCSeconds() !== 0) {
|
||||
toDayExclusive = addUtcDays(toDayExclusive, 1)
|
||||
}
|
||||
} else {
|
||||
toDayExclusive = addUtcDays(toUtcDay(toDate), 1)
|
||||
toDate = new Date(`${toDayExclusive}T00:00:00Z`)
|
||||
}
|
||||
if (toDate.getTime() <= from) return null
|
||||
const windowSec = Math.max(1, Math.round((toDate.getTime() - from) / 1000))
|
||||
const grain: "hour" | "day" = toDate.getTime() - from <= HOUR_WINDOW_MS ? "hour" : "day"
|
||||
return {
|
||||
fromIso: fromDate.toISOString(),
|
||||
toIso: toDate.toISOString(),
|
||||
fromDay: toUtcDay(fromDate),
|
||||
toDayExclusive,
|
||||
grain,
|
||||
windowSec,
|
||||
}
|
||||
}
|
||||
|
||||
type FactScope = "unique" | "wan" | "overlay"
|
||||
|
||||
interface FilterCtx {
|
||||
fromIso: string
|
||||
toIso: string
|
||||
fromDay: string
|
||||
toDayExclusive: string
|
||||
serverId?: number
|
||||
iface?: string
|
||||
country?: string
|
||||
service?: string
|
||||
asn?: number
|
||||
planes: "unique" | "all"
|
||||
userIfaces: Array<{ serverId: number; iface: string }> | null
|
||||
unboundOnly: boolean
|
||||
boundIfaces: Array<{ serverId: number; iface: string }>
|
||||
overlayIfaces: Array<{ serverId: number; iface: string }>
|
||||
wanIfaces: Array<{ serverId: number; iface: string }>
|
||||
excludeServerIds: number[]
|
||||
topo: FlowTopology | null
|
||||
}
|
||||
|
||||
function ifaceFilterAliases(iface: string, serverId?: number): string[] {
|
||||
return factIfaceAliases(iface.trim(), serverId)
|
||||
}
|
||||
|
||||
function looksLikeIfIndex(iface: string): boolean {
|
||||
const raw = iface.trim()
|
||||
return /^\d+$/.test(raw) || /^#\d+$/.test(raw)
|
||||
}
|
||||
|
||||
async function warmIfaceCache(ids: Iterable<number>): Promise<void> {
|
||||
const uniq = [...new Set(ids)].filter((id) => Number.isFinite(id) && id > 0)
|
||||
if (!uniq.length) return
|
||||
await Promise.all(uniq.map((id) => refreshServerIfaces(id)))
|
||||
}
|
||||
|
||||
async function warmBindingIfaceCache(): Promise<void> {
|
||||
const rows = await db.select({ serverId: userInterfaceBindings.serverId }).from(userInterfaceBindings)
|
||||
await warmIfaceCache(rows.map((r) => r.serverId))
|
||||
}
|
||||
|
||||
function canonicalIfaceDimId(id: string): string {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (!Number.isFinite(sid)) return id
|
||||
return `${sid}:${displayFactIface(sid, id.slice(colon + 1))}`
|
||||
}
|
||||
|
||||
function pushIfaceTuples(
|
||||
parts: string[],
|
||||
params: unknown[],
|
||||
alias: string,
|
||||
tuples: Array<{ serverId: number; iface: string }>,
|
||||
op: "IN" | "NOT IN",
|
||||
): void {
|
||||
if (!tuples.length) {
|
||||
if (op === "IN") parts.push("FALSE")
|
||||
return
|
||||
}
|
||||
const sql = tuples.map(() => "(?, ?)").join(", ")
|
||||
parts.push(`(${alias}.server_id, ${alias}.iface) ${op} (${sql})`)
|
||||
for (const t of tuples) {
|
||||
params.push(t.serverId, t.iface)
|
||||
}
|
||||
}
|
||||
|
||||
function factWhere(
|
||||
alias: string,
|
||||
grain: "hour" | "day",
|
||||
ctx: FilterCtx,
|
||||
scope: FactScope = "unique",
|
||||
): { sql: string; params: unknown[] } {
|
||||
const params: unknown[] = []
|
||||
const parts: string[] = []
|
||||
if (grain === "hour") {
|
||||
params.push(ctx.fromIso, ctx.toIso)
|
||||
parts.push(`${alias}.bucket_at >= ? AND ${alias}.bucket_at < ?`)
|
||||
} else {
|
||||
params.push(ctx.fromDay, ctx.toDayExclusive)
|
||||
parts.push(`${alias}.day >= ? AND ${alias}.day < ?`)
|
||||
}
|
||||
if (ctx.serverId != null) {
|
||||
parts.push(`${alias}.server_id = ?`)
|
||||
params.push(ctx.serverId)
|
||||
}
|
||||
if (ctx.country) {
|
||||
parts.push(`${alias}.country = ?`)
|
||||
params.push(ctx.country.toUpperCase())
|
||||
}
|
||||
if (ctx.service) {
|
||||
parts.push(`${alias}.service = ?`)
|
||||
params.push(ctx.service)
|
||||
}
|
||||
if (ctx.asn != null) {
|
||||
parts.push(`${alias}.asn = ?`)
|
||||
params.push(ctx.asn)
|
||||
}
|
||||
parts.push(`${alias}.iface NOT IN ('0', '—', '__unknown__', 'wg-flow', '')`)
|
||||
|
||||
if (scope === "wan") {
|
||||
pushIfaceTuples(parts, params, alias, ctx.wanIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (scope === "overlay") {
|
||||
pushIfaceTuples(parts, params, alias, ctx.overlayIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
|
||||
if (ctx.iface) {
|
||||
const aliases = ifaceFilterAliases(ctx.iface, ctx.serverId)
|
||||
if (aliases.length <= 1) {
|
||||
parts.push(`${alias}.iface = ?`)
|
||||
params.push(aliases[0] ?? ctx.iface)
|
||||
} else {
|
||||
parts.push(`${alias}.iface IN (${aliases.map(() => "?").join(", ")})`)
|
||||
params.push(...aliases)
|
||||
}
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (ctx.userIfaces) {
|
||||
pushIfaceTuples(parts, params, alias, ctx.userIfaces, "IN")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
if (ctx.unboundOnly) {
|
||||
parts.push("FALSE")
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
pushIfaceTuples(parts, params, alias, ctx.boundIfaces, "IN")
|
||||
if (ctx.excludeServerIds.length) {
|
||||
parts.push(`${alias}.server_id NOT IN (${ctx.excludeServerIds.map(() => "?").join(", ")})`)
|
||||
params.push(...ctx.excludeServerIds)
|
||||
}
|
||||
return { sql: parts.join(" AND "), params }
|
||||
}
|
||||
|
||||
function emptyDto(period: ParsedPeriod): StatisticsDto {
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes: 0,
|
||||
packets: 0,
|
||||
avgBps: 0,
|
||||
users: 0,
|
||||
servers: 0,
|
||||
ifaces: 0,
|
||||
topCountry: "—",
|
||||
topService: "—",
|
||||
},
|
||||
series: [],
|
||||
users: [],
|
||||
servers: [],
|
||||
interfaces: [],
|
||||
countries: [],
|
||||
services: [],
|
||||
asns: [],
|
||||
}
|
||||
}
|
||||
|
||||
function toBreakdown(
|
||||
rows: Array<{ id: string; label: string; bytes: number; packets: number }>,
|
||||
totalBytes: number,
|
||||
windowSec: number,
|
||||
): StatisticsBreakdownRow[] {
|
||||
const denom = totalBytes || 1
|
||||
return rows
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
.slice(0, TOP_N)
|
||||
.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.label,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
bps: (r.bytes * 8) / windowSec,
|
||||
percent: (r.bytes / denom) * 100,
|
||||
}))
|
||||
}
|
||||
|
||||
interface UserBindTuple {
|
||||
userId: string
|
||||
serverId: number
|
||||
iface: string
|
||||
}
|
||||
|
||||
async function loadBindUserTuples(): Promise<UserBindTuple[]> {
|
||||
const binds = await db.select().from(userInterfaceBindings)
|
||||
const seen = new Set<string>()
|
||||
const out: UserBindTuple[] = []
|
||||
for (const b of binds) {
|
||||
for (const iface of factIfaceAliases(b.interfaceName, b.serverId)) {
|
||||
const k = `${b.userId}\0${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ userId: b.userId, serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
function uniqueBoundIfaces(tuples: UserBindTuple[]): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const t of tuples) {
|
||||
const k = `${t.serverId}\0${t.iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: t.serverId, iface: t.iface })
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function resolveUserIfaces(userId?: string): Promise<Array<{ serverId: number; iface: string }> | null> {
|
||||
if (!userId || userId === STATISTICS_UNBOUND_USER_ID) return null
|
||||
const binds = await db.select().from(userInterfaceBindings).where(eq(userInterfaceBindings.userId, userId))
|
||||
return expandBindingIfaces(binds.map((b) => ({ serverId: b.serverId, iface: b.interfaceName })))
|
||||
}
|
||||
|
||||
function userBindJoinSql(tuples: UserBindTuple[]): { sql: string; params: unknown[] } {
|
||||
const values = tuples.map(() => "(?::text, ?::int, ?::text)").join(", ")
|
||||
const params = tuples.flatMap((t) => [t.userId, t.serverId, t.iface])
|
||||
return {
|
||||
sql: `JOIN (VALUES ${values}) AS b(user_id, server_id, iface) ON b.server_id = f.server_id AND b.iface = f.iface`,
|
||||
params,
|
||||
}
|
||||
}
|
||||
|
||||
function expandIfaceTuples(
|
||||
items: Array<{ serverId: number; iface: string }>,
|
||||
): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const t of items) {
|
||||
for (const iface of factIfaceAliases(t.iface, t.serverId)) {
|
||||
const k = `${t.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: t.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
async function loadPayloadScope(serverId?: number): Promise<{
|
||||
overlayIfaces: Array<{ serverId: number; iface: string }>
|
||||
wanIfaces: Array<{ serverId: number; iface: string }>
|
||||
excludeServerIds: number[]
|
||||
topo: FlowTopology
|
||||
}> {
|
||||
const topo = await loadFlowTopology()
|
||||
const catalog = await getServerCatalog()
|
||||
await warmIfaceCache(catalog.list.map((s) => s.id))
|
||||
const overlayRaw: Array<{ serverId: number; iface: string }> = []
|
||||
const wanRaw: Array<{ serverId: number; iface: string }> = []
|
||||
for (const s of catalog.list) {
|
||||
if (serverId != null && s.id !== serverId) continue
|
||||
const wanSet = topo.wanIfaces.get(s.id)
|
||||
const wanNames = wanSet && wanSet.size > 0
|
||||
? [...wanSet]
|
||||
: s.type === "home-router" ? [] : ["ether1"]
|
||||
for (const name of wanNames) wanRaw.push({ serverId: s.id, iface: name })
|
||||
const names = new Set(listCachedIfaceNames(s.id))
|
||||
for (const name of topo.tunnelIfaces?.get(s.id) ?? []) names.add(name)
|
||||
for (const name of names) {
|
||||
if (isOverlayTunnelIface(topo, s.id, name)) overlayRaw.push({ serverId: s.id, iface: name })
|
||||
}
|
||||
}
|
||||
return {
|
||||
overlayIfaces: expandIfaceTuples(overlayRaw),
|
||||
wanIfaces: expandIfaceTuples(wanRaw),
|
||||
excludeServerIds: serverId != null
|
||||
? []
|
||||
: catalog.list.filter((s) => s.type === "exit-node").map((s) => s.id),
|
||||
topo,
|
||||
}
|
||||
}
|
||||
|
||||
async function buildFilterCtx(query: StatisticsQuery, period: ParsedPeriod): Promise<FilterCtx | null> {
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const boundIfaces = uniqueBoundIfaces(bindTuples)
|
||||
const unboundOnly = query.userId === STATISTICS_UNBOUND_USER_ID
|
||||
const userIfaces = unboundOnly ? null : await resolveUserIfaces(query.userId)
|
||||
if (userIfaces && userIfaces.length === 0) return null
|
||||
if (unboundOnly) return null
|
||||
const scope = await loadPayloadScope(query.serverId)
|
||||
return {
|
||||
...period,
|
||||
serverId: query.serverId,
|
||||
iface: query.iface,
|
||||
country: query.country,
|
||||
service: query.service,
|
||||
asn: query.asn,
|
||||
planes: query.planes ?? "unique",
|
||||
userIfaces,
|
||||
unboundOnly,
|
||||
boundIfaces,
|
||||
overlayIfaces: scope.overlayIfaces,
|
||||
wanIfaces: scope.wanIfaces,
|
||||
excludeServerIds: scope.excludeServerIds,
|
||||
topo: scope.topo,
|
||||
}
|
||||
}
|
||||
|
||||
export async function getStatistics(query: StatisticsQuery): Promise<StatisticsDto> {
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyDto({
|
||||
fromIso: query.from,
|
||||
toIso: query.to,
|
||||
fromDay: query.from.slice(0, 10),
|
||||
toDayExclusive: query.to.slice(0, 10),
|
||||
grain: "day",
|
||||
windowSec: 1,
|
||||
})
|
||||
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyDto(period)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const timeCol = period.grain === "hour" ? "bucket_at" : "day"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
|
||||
const totals = await dbAll<{ bytes: number; packets: number; servers: number }>(`
|
||||
SELECT
|
||||
COALESCE(SUM(f.bytes), 0) AS bytes,
|
||||
COALESCE(SUM(f.packets), 0) AS packets,
|
||||
COUNT(DISTINCT f.server_id)::int AS servers
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
`, where.params)
|
||||
|
||||
const bytes = Number(totals[0]?.bytes) || 0
|
||||
const packets = Number(totals[0]?.packets) || 0
|
||||
const serverCount = Number(totals[0]?.servers) || 0
|
||||
|
||||
const seriesRows = await dbAll<{ t: string; bytes: number }>(`
|
||||
SELECT ${timeCol}::text AS t, SUM(f.bytes) AS bytes
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY ${timeCol}
|
||||
ORDER BY ${timeCol}
|
||||
`, where.params)
|
||||
|
||||
const countryRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.country AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.country
|
||||
`, where.params)
|
||||
|
||||
const serviceRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT f.service AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.service
|
||||
`, where.params)
|
||||
|
||||
const asnRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.asn AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.asn
|
||||
`, where.params)
|
||||
|
||||
const serverRows = await dbAll<{ id: number; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id
|
||||
`, where.params)
|
||||
|
||||
const ifaceRowsRaw = await dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${where.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, where.params)
|
||||
await warmIfaceCache(ifaceRowsRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId))
|
||||
const ifaceRows = collapseServerIfaceRows(ifaceRowsRaw).filter((r) => {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) return false
|
||||
if (ctx.iface) return true
|
||||
if (ctx.topo && isOverlayTunnelIface(ctx.topo, r.serverId, r.iface)) return false
|
||||
if (ctx.topo && isWanFactIface(ctx.topo, r.serverId, r.iface)) return false
|
||||
return true
|
||||
})
|
||||
const ifaceCount = ifaceRows.length
|
||||
|
||||
let dupeIfaceRows: Array<{ serverId: number; iface: string; bytes: number; packets: number; kind: "wan" | "overlay" }> = []
|
||||
if (ctx.planes === "all" && !ctx.iface) {
|
||||
const wanWhere = factWhere("f", period.grain, ctx, "wan")
|
||||
const overlayWhere = factWhere("f", period.grain, ctx, "overlay")
|
||||
const [wanRaw, overlayRaw] = await Promise.all([
|
||||
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${wanWhere.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, wanWhere.params),
|
||||
dbAll<{ serverId: number; iface: string; bytes: number; packets: number }>(`
|
||||
SELECT f.server_id AS "serverId", f.iface AS iface, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
WHERE ${overlayWhere.sql}
|
||||
GROUP BY f.server_id, f.iface
|
||||
`, overlayWhere.params),
|
||||
])
|
||||
await warmIfaceCache([
|
||||
...wanRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
|
||||
...overlayRaw.filter((r) => looksLikeIfIndex(r.iface)).map((r) => r.serverId),
|
||||
])
|
||||
const seen = new Set(ifaceRows.map((r) => `${r.serverId}:${r.iface}`))
|
||||
for (const r of collapseServerIfaceRows(wanRaw)) {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
|
||||
const key = `${r.serverId}:${r.iface}`
|
||||
if (seen.has(key)) continue
|
||||
seen.add(key)
|
||||
dupeIfaceRows.push({ ...r, kind: "wan" })
|
||||
}
|
||||
for (const r of collapseServerIfaceRows(overlayRaw)) {
|
||||
if (isJunkFactIface(r.iface) || isDashDisplayIface(r.iface)) continue
|
||||
const key = `${r.serverId}:${r.iface}`
|
||||
if (seen.has(key)) continue
|
||||
seen.add(key)
|
||||
dupeIfaceRows.push({ ...r, kind: "overlay" })
|
||||
}
|
||||
}
|
||||
|
||||
let userRows: Array<{ id: string; bytes: number; packets: number }> = []
|
||||
if (bindTuples.length && !ctx.unboundOnly) {
|
||||
const join = userBindJoinSql(bindTuples)
|
||||
userRows = await dbAll<{ id: string; bytes: number; packets: number }>(`
|
||||
SELECT b.user_id AS id, SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY b.user_id
|
||||
`, [...join.params, ...where.params])
|
||||
}
|
||||
|
||||
const serverNames = new Map<number, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(s.id, s.name || s.host)
|
||||
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
|
||||
const asnHolders = new Map<number, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(a.asn, a.holder)
|
||||
|
||||
const countries = toBreakdown(
|
||||
countryRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id === "XX" ? "Неизвестно" : r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const services = toBreakdown(
|
||||
serviceRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const asns = toBreakdown(
|
||||
asnRows.map((r) => {
|
||||
const id = Number(r.id) || 0
|
||||
const holder = asnHolders.get(id)
|
||||
return {
|
||||
id: String(id),
|
||||
label: id === 0 ? "other" : holder ? `AS${id} · ${holder}` : `AS${id}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
}
|
||||
}),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const serverBreakdown = toBreakdown(
|
||||
serverRows.map((r) => ({
|
||||
id: String(r.id),
|
||||
label: serverNames.get(r.id) || String(r.id),
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const uniqueInterfaces = toBreakdown(
|
||||
ifaceRows.map((r) => {
|
||||
const serverName = serverNames.get(r.serverId) || String(r.serverId)
|
||||
const wan = ctx.topo ? isWanFactIface(ctx.topo, r.serverId, r.iface) : false
|
||||
return {
|
||||
id: `${r.serverId}:${r.iface}`,
|
||||
label: wan ? wanIfaceLabel(serverName, r.iface) : `${serverName} · ${r.iface}`,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
}
|
||||
}),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
const dupeInterfaces: StatisticsBreakdownRow[] = dupeIfaceRows.map((r) => {
|
||||
const serverName = serverNames.get(r.serverId) || String(r.serverId)
|
||||
const rowBytes = Number(r.bytes) || 0
|
||||
const rowPackets = Number(r.packets) || 0
|
||||
return {
|
||||
id: `${r.serverId}:${r.iface}`,
|
||||
label: r.kind === "wan" ? wanIfaceLabel(serverName, r.iface) : overlayDupLabel(serverName, r.iface),
|
||||
bytes: rowBytes,
|
||||
packets: rowPackets,
|
||||
bps: (rowBytes * 8) / period.windowSec,
|
||||
percent: 0,
|
||||
}
|
||||
})
|
||||
const interfaces = [...uniqueInterfaces, ...dupeInterfaces]
|
||||
const matchedUsers = toBreakdown(
|
||||
userRows.map((r) => ({
|
||||
id: r.id,
|
||||
label: userNames.get(r.id) || r.id,
|
||||
bytes: Number(r.bytes) || 0,
|
||||
packets: Number(r.packets) || 0,
|
||||
})),
|
||||
bytes,
|
||||
period.windowSec,
|
||||
)
|
||||
|
||||
const users = [...matchedUsers]
|
||||
|
||||
return {
|
||||
from: period.fromIso,
|
||||
to: period.toIso,
|
||||
grain: period.grain,
|
||||
kpis: {
|
||||
bytes,
|
||||
packets,
|
||||
avgBps: (bytes * 8) / period.windowSec,
|
||||
users: matchedUsers.length,
|
||||
servers: serverCount,
|
||||
ifaces: ifaceCount,
|
||||
topCountry: countries[0]?.label || "—",
|
||||
topService: services[0]?.label || "—",
|
||||
},
|
||||
series: seriesRows.map((r) => ({ t: r.t, bytes: Number(r.bytes) || 0 })),
|
||||
users,
|
||||
servers: serverBreakdown,
|
||||
interfaces,
|
||||
countries,
|
||||
services,
|
||||
asns,
|
||||
}
|
||||
}
|
||||
|
||||
function dimSql(dim: StatisticsPivotDim, factAlias: string, bindAlias: string): string {
|
||||
if (dim === "country") return `${factAlias}.country`
|
||||
if (dim === "service") return `${factAlias}.service`
|
||||
if (dim === "asn") return `${factAlias}.asn::text`
|
||||
if (dim === "server") return `${factAlias}.server_id::text`
|
||||
if (dim === "iface") return `(${factAlias}.server_id::text || ':' || ${factAlias}.iface)`
|
||||
return `${bindAlias}.user_id`
|
||||
}
|
||||
|
||||
function emptyPivot(query: StatisticsPivotQuery): StatisticsPivotDto {
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric: query.metric,
|
||||
columns: [],
|
||||
rows: [],
|
||||
otherBytes: 0,
|
||||
}
|
||||
}
|
||||
|
||||
export function pivotDimsConflict(row: StatisticsPivotDim, col: StatisticsPivotDim): boolean {
|
||||
return row === col
|
||||
}
|
||||
|
||||
export async function getStatisticsPivot(query: StatisticsPivotQuery): Promise<StatisticsPivotDto> {
|
||||
if (pivotDimsConflict(query.row, query.col)) return emptyPivot(query)
|
||||
const period = parseStatisticsPeriod(query.from, query.to)
|
||||
if (!period) return emptyPivot(query)
|
||||
await warmBindingIfaceCache()
|
||||
if (query.serverId) await warmIfaceCache([query.serverId])
|
||||
const bindTuples = await loadBindUserTuples()
|
||||
const ctx = await buildFilterCtx(query, period)
|
||||
if (!ctx) return emptyPivot(query)
|
||||
const needsUser = query.row === "user" || query.col === "user"
|
||||
if (needsUser && bindTuples.length === 0) return emptyPivot(query)
|
||||
|
||||
const table = period.grain === "hour" ? "flow_hour_facts" : "flow_daily_facts"
|
||||
const where = factWhere("f", period.grain, ctx)
|
||||
const rowExpr = dimSql(query.row, "f", "b")
|
||||
const colExpr = dimSql(query.col, "f", "b")
|
||||
const join = needsUser ? userBindJoinSql(bindTuples) : { sql: "", params: [] as unknown[] }
|
||||
|
||||
const raw = await dbAll<{ row_id: string; col_id: string; bytes: number; packets: number }>(`
|
||||
SELECT ${rowExpr} AS row_id, ${colExpr} AS col_id,
|
||||
SUM(f.bytes) AS bytes, SUM(f.packets) AS packets
|
||||
FROM ${table} f
|
||||
${join.sql}
|
||||
WHERE ${where.sql}
|
||||
GROUP BY 1, 2
|
||||
`, [...join.params, ...where.params])
|
||||
|
||||
if (query.row === "iface" || query.col === "iface") {
|
||||
const ifaceServerIds: number[] = []
|
||||
for (const r of raw) {
|
||||
for (const dim of [query.row, query.col] as const) {
|
||||
if (dim !== "iface") continue
|
||||
const id = dim === query.row ? String(r.row_id ?? "") : String(r.col_id ?? "")
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) continue
|
||||
const sid = Number(id.slice(0, colon))
|
||||
if (looksLikeIfIndex(id.slice(colon + 1)) && Number.isFinite(sid)) ifaceServerIds.push(sid)
|
||||
}
|
||||
}
|
||||
await warmIfaceCache(ifaceServerIds)
|
||||
for (const r of raw) {
|
||||
if (query.row === "iface") r.row_id = canonicalIfaceDimId(String(r.row_id ?? ""))
|
||||
if (query.col === "iface") r.col_id = canonicalIfaceDimId(String(r.col_id ?? ""))
|
||||
}
|
||||
}
|
||||
|
||||
const metric = query.metric
|
||||
type Acc = { bytes: number; packets: number }
|
||||
const cell = new Map<string, Map<string, Acc>>()
|
||||
const colTotals = new Map<string, number>()
|
||||
for (const r of raw) {
|
||||
const rid = String(r.row_id ?? "")
|
||||
const cid = String(r.col_id ?? "")
|
||||
const acc: Acc = { bytes: Number(r.bytes) || 0, packets: Number(r.packets) || 0 }
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
let rowMap = cell.get(rid)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
cell.set(rid, rowMap)
|
||||
}
|
||||
const prev = rowMap.get(cid)
|
||||
if (prev) {
|
||||
prev.bytes += acc.bytes
|
||||
prev.packets += acc.packets
|
||||
} else {
|
||||
rowMap.set(cid, acc)
|
||||
}
|
||||
colTotals.set(cid, (colTotals.get(cid) ?? 0) + val)
|
||||
}
|
||||
|
||||
const topCols = [...colTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_COL_CAP)
|
||||
.map(([id]) => id)
|
||||
const topColSet = new Set(topCols)
|
||||
const folded = new Map<string, Map<string, number>>()
|
||||
const foldedColTotals = new Map<string, number>()
|
||||
let otherBytes = 0
|
||||
for (const [rid, cols] of cell) {
|
||||
const rowMap = new Map<string, number>()
|
||||
for (const [cid, acc] of cols) {
|
||||
const val = metric === "packets" ? acc.packets : acc.bytes
|
||||
const dest = topColSet.has(cid) ? cid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) otherBytes += val
|
||||
rowMap.set(dest, (rowMap.get(dest) ?? 0) + val)
|
||||
foldedColTotals.set(dest, (foldedColTotals.get(dest) ?? 0) + val)
|
||||
}
|
||||
folded.set(rid, rowMap)
|
||||
}
|
||||
|
||||
const rowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
rowTotals.set(rid, t)
|
||||
}
|
||||
const topRows = [...rowTotals.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, PIVOT_ROW_CAP)
|
||||
.map(([id]) => id)
|
||||
const topRowSet = new Set(topRows)
|
||||
const finalRows = new Map<string, Map<string, number>>()
|
||||
const finalRowTotals = new Map<string, number>()
|
||||
for (const [rid, cols] of folded) {
|
||||
const dest = topRowSet.has(rid) ? rid : PIVOT_OTHER_ID
|
||||
if (dest === PIVOT_OTHER_ID) {
|
||||
for (const [cid, v] of cols) {
|
||||
if (cid !== PIVOT_OTHER_ID) otherBytes += v
|
||||
}
|
||||
}
|
||||
let rowMap = finalRows.get(dest)
|
||||
if (!rowMap) {
|
||||
rowMap = new Map()
|
||||
finalRows.set(dest, rowMap)
|
||||
}
|
||||
for (const [cid, v] of cols) {
|
||||
rowMap.set(cid, (rowMap.get(cid) ?? 0) + v)
|
||||
}
|
||||
}
|
||||
for (const [rid, cols] of finalRows) {
|
||||
let t = 0
|
||||
for (const v of cols.values()) t += v
|
||||
finalRowTotals.set(rid, t)
|
||||
}
|
||||
|
||||
const colIds = [...topCols]
|
||||
if (foldedColTotals.has(PIVOT_OTHER_ID)) colIds.push(PIVOT_OTHER_ID)
|
||||
const rowIds = [...topRows]
|
||||
if (finalRows.has(PIVOT_OTHER_ID) && !topRowSet.has(PIVOT_OTHER_ID)) rowIds.push(PIVOT_OTHER_ID)
|
||||
|
||||
const labels = await loadPivotLabels(query.row, query.col, rowIds, colIds)
|
||||
|
||||
return {
|
||||
rowDim: query.row,
|
||||
colDim: query.col,
|
||||
metric,
|
||||
columns: colIds.map((id) => ({
|
||||
id,
|
||||
label: labels.col.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: foldedColTotals.get(id) ?? 0,
|
||||
})),
|
||||
rows: rowIds.map((id) => {
|
||||
const cols = finalRows.get(id) ?? new Map()
|
||||
const cells: Record<string, number> = {}
|
||||
for (const cid of colIds) cells[cid] = cols.get(cid) ?? 0
|
||||
return {
|
||||
id,
|
||||
label: labels.row.get(id) ?? (id === PIVOT_OTHER_ID ? "Прочие" : id),
|
||||
total: finalRowTotals.get(id) ?? 0,
|
||||
cells,
|
||||
}
|
||||
}),
|
||||
otherBytes,
|
||||
}
|
||||
}
|
||||
|
||||
async function loadPivotLabels(
|
||||
rowDim: StatisticsPivotDim,
|
||||
colDim: StatisticsPivotDim,
|
||||
rowIds: string[],
|
||||
colIds: string[],
|
||||
): Promise<{ row: Map<string, string>; col: Map<string, string> }> {
|
||||
const topo = await loadFlowTopology()
|
||||
const serverNames = new Map<string, string>()
|
||||
const allServers = await db.select({ id: servers.id, name: servers.name, host: servers.host }).from(servers)
|
||||
for (const s of allServers) serverNames.set(String(s.id), s.name || s.host)
|
||||
const userNames = new Map<string, string>()
|
||||
const allUsers = await db.select({ id: appUsers.id, name: appUsers.name, login: appUsers.login }).from(appUsers)
|
||||
for (const u of allUsers) userNames.set(u.id, u.name || u.login)
|
||||
const asnHolders = new Map<string, string>()
|
||||
const asnMeta = await db.select({ asn: flowAsnMeta.asn, holder: flowAsnMeta.holder }).from(flowAsnMeta)
|
||||
for (const a of asnMeta) asnHolders.set(String(a.asn), a.holder)
|
||||
|
||||
function label(dim: StatisticsPivotDim, id: string): string {
|
||||
if (id === PIVOT_OTHER_ID) return "Прочие"
|
||||
if (dim === "country") return id === "XX" ? "Неизвестно" : id
|
||||
if (dim === "server") return serverNames.get(id) || id
|
||||
if (dim === "user") return userNames.get(id) || id
|
||||
if (dim === "asn") {
|
||||
if (id === "0") return "other"
|
||||
const holder = asnHolders.get(id)
|
||||
return holder ? `AS${id} · ${holder}` : `AS${id}`
|
||||
}
|
||||
if (dim === "iface") {
|
||||
const colon = id.indexOf(":")
|
||||
if (colon < 0) return id
|
||||
const sid = id.slice(0, colon)
|
||||
const iface = id.slice(colon + 1)
|
||||
const sidNum = Number(sid)
|
||||
const name = Number.isFinite(sidNum) ? displayFactIface(sidNum, iface) : iface
|
||||
const serverName = serverNames.get(sid) || sid
|
||||
if (Number.isFinite(sidNum) && isWanFactIface(topo, sidNum, name)) {
|
||||
return wanIfaceLabel(serverName, name)
|
||||
}
|
||||
if (Number.isFinite(sidNum) && isOverlayTunnelIface(topo, sidNum, name)) {
|
||||
return overlayDupLabel(serverName, name)
|
||||
}
|
||||
return `${serverName} · ${name}`
|
||||
}
|
||||
return id
|
||||
}
|
||||
|
||||
const row = new Map<string, string>()
|
||||
const col = new Map<string, string>()
|
||||
for (const id of rowIds) row.set(id, label(rowDim, id))
|
||||
for (const id of colIds) col.set(id, label(colDim, id))
|
||||
return { row, col }
|
||||
}
|
||||
@@ -1,6 +1,7 @@
|
||||
import { and, asc, desc, eq, gte, lt } from "drizzle-orm"
|
||||
import { and, asc, desc, eq, gte } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import { servers, trafficSamples, trafficSettings } from "../db/schema.js"
|
||||
import { decodeTrafficSampleFlags, encodeTrafficFlags } from "../db/traffic-flags.js"
|
||||
import type { TrafficRunSnapshot } from "../types/scheduler-run-snapshot.js"
|
||||
import { SCHEDULER_RUN_SNAPSHOT_VERSION } from "../types/scheduler-run-snapshot.js"
|
||||
import { MikrotikClient } from "./mikrotik.js"
|
||||
@@ -79,12 +80,6 @@ async function getSettingsRow() {
|
||||
return (await db.select().from(trafficSettings).where(eq(trafficSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
async function cleanupOldSamples(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
await db.delete(trafficSamples)
|
||||
.where(lt(trafficSamples.sampledAt, cutoff))
|
||||
}
|
||||
|
||||
async function readPreviousWave(serverId: number): Promise<Map<string, { rxBytes: number; txBytes: number; sampledAt: string }>> {
|
||||
const last = (await db
|
||||
.select({ sampledAt: trafficSamples.sampledAt })
|
||||
@@ -164,8 +159,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
txBytes,
|
||||
rxBps,
|
||||
txBps,
|
||||
running,
|
||||
disabled,
|
||||
flags: encodeTrafficFlags(running, disabled),
|
||||
}
|
||||
})
|
||||
try {
|
||||
@@ -194,8 +188,7 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
txBytes,
|
||||
rxBps,
|
||||
txBps,
|
||||
running,
|
||||
disabled,
|
||||
flags: encodeTrafficFlags(running, disabled),
|
||||
})
|
||||
}
|
||||
} catch {
|
||||
@@ -224,7 +217,6 @@ export async function collectTrafficOnce(): Promise<TrafficRunSnapshot> {
|
||||
}
|
||||
}
|
||||
|
||||
await cleanupOldSamples(Math.max(1, settings.retentionDays))
|
||||
await db.update(trafficSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - startedAt,
|
||||
@@ -286,11 +278,12 @@ export async function updateTrafficSettings(patch: {
|
||||
}
|
||||
|
||||
export async function readServerSamplesInRange(serverId: number, sinceIso: string) {
|
||||
return await db.select()
|
||||
const rows = await db.select()
|
||||
.from(trafficSamples)
|
||||
.where(and(
|
||||
eq(trafficSamples.serverId, serverId),
|
||||
gte(trafficSamples.sampledAt, sinceIso),
|
||||
))
|
||||
.orderBy(asc(trafficSamples.sampledAt))
|
||||
return rows.map((r) => ({ ...r, ...decodeTrafficSampleFlags(r.flags) }))
|
||||
}
|
||||
|
||||
@@ -16,6 +16,29 @@ import {
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
|
||||
{
|
||||
const liveErr = await formatLiveSseFromBuilder(() => {
|
||||
throw new Error("SQLITE_BUSY")
|
||||
})
|
||||
assert.equal(liveErr.event, "error")
|
||||
assert.equal((liveErr.data as { error: string }).error, "SQLITE_BUSY")
|
||||
const liveOk = await formatLiveSseFromBuilder(() => ({ ok: true }))
|
||||
assert.equal(liveOk.event, "sample")
|
||||
const liveAsync = await formatLiveSseFromBuilder(async () => ({
|
||||
uniqueSrc: 3,
|
||||
destinations: [{ id: "8.8.8.8", label: "8.8.8.8", bytes: 1, packets: 1, bps: 1, percent: 100 }],
|
||||
}))
|
||||
assert.equal(liveAsync.event, "sample")
|
||||
assert.notEqual(JSON.stringify(liveAsync.data), "{}")
|
||||
assert.equal((liveAsync.data as { uniqueSrc: number }).uniqueSrc, 3)
|
||||
assert.ok(Array.isArray((liveAsync.data as { destinations: unknown[] }).destinations))
|
||||
const liveReject = await formatLiveSseFromBuilder(async () => {
|
||||
throw new Error("pg down")
|
||||
})
|
||||
assert.equal(liveReject.event, "error")
|
||||
assert.equal((liveReject.data as { error: string }).error, "pg down")
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-analytics.test.ts: skip")
|
||||
process.exit(0)
|
||||
@@ -248,13 +271,6 @@ try {
|
||||
assert.equal(degraded.degraded, true)
|
||||
assert.equal(degraded.conversationsList.length, 0)
|
||||
assert.ok((degraded.bytes ?? 0) >= 12_000)
|
||||
const liveErr = formatLiveSseFromBuilder(() => {
|
||||
throw new Error("SQLITE_BUSY")
|
||||
})
|
||||
assert.equal(liveErr.event, "error")
|
||||
assert.equal((liveErr.data as { error: string }).error, "SQLITE_BUSY")
|
||||
const liveOk = formatLiveSseFromBuilder(() => ({ ok: true }))
|
||||
assert.equal(liveOk.event, "sample")
|
||||
const exporters = await listFlowExporters(5)
|
||||
const clients = await listFlowClients(5)
|
||||
assert.ok(Array.isArray(exporters.exporters))
|
||||
|
||||
@@ -26,11 +26,10 @@ import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { getTrafficFlowSettingsRow, listHostPeers } from "./traffic-flow-settings.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached } from "./traffic-flow-ripe.js"
|
||||
import { classifyFlowDst, refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { enqueueRipeMisses } from "./traffic-flow-ripe.js"
|
||||
import { classifyFlowPlane, flowBps, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import { refreshFlowCatalogInBackground } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
enGreIfaceNames,
|
||||
getServerCatalog,
|
||||
@@ -252,11 +251,20 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
totalPackets += r.packets
|
||||
srcs.add(r.src)
|
||||
dsts.add(r.dst)
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
peers.add(peer)
|
||||
const destMeta = resolveInternetDest({
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
serverId: r.serverId,
|
||||
inIface: resolved.name,
|
||||
topo,
|
||||
})
|
||||
if (destMeta.dest) peers.add(destMeta.dest)
|
||||
const app = applicationName(r.proto, r.dstPort, r.srcPort)
|
||||
const ripe = lookupRipeCached(peer)
|
||||
const classified = classifyFlowDst(peer, r.proto, r.dstPort, r.srcPort, ripe)
|
||||
const ripe = destMeta.ripe
|
||||
const classified = destMeta.classified
|
||||
bump(applications, app, r.bytes, r.packets)
|
||||
bump(protocols, protoName(r.proto), r.bytes, r.packets)
|
||||
bump(sources, r.src, r.bytes, r.packets)
|
||||
@@ -268,7 +276,7 @@ async function buildFlowAnalyticsUncached(q: FlowAnalyticsQuery): Promise<FlowAn
|
||||
const asnLabel = ripe.holder ? `AS${ripe.asn} ${ripe.holder}` : `AS${ripe.asn}`
|
||||
bump(asns, asnId, r.bytes, r.packets, asnLabel)
|
||||
}
|
||||
const dstCountry = ripe?.ok && isIsoCountry(ripe.country) ? ripe.country : ""
|
||||
const dstCountry = destMeta.country && destMeta.country !== "unknown" ? destMeta.country : ""
|
||||
if (dstCountry) {
|
||||
bump(countries, dstCountry, r.bytes, r.packets)
|
||||
}
|
||||
@@ -599,9 +607,12 @@ export async function listFlowClients(minutes: number): Promise<{ clients: { id:
|
||||
return { clients }
|
||||
}
|
||||
|
||||
export function formatLiveSseFromBuilder(build: () => unknown): { event: "sample" | "error"; data: unknown } {
|
||||
export async function formatLiveSseFromBuilder(
|
||||
build: () => unknown | Promise<unknown>,
|
||||
): Promise<{ event: "sample" | "error"; data: unknown }> {
|
||||
try {
|
||||
return { event: "sample", data: build() }
|
||||
const data = await build()
|
||||
return { event: "sample", data }
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
return { event: "error", data: { error: message } }
|
||||
@@ -613,10 +624,10 @@ export function isFlowAnalyticsDegraded(): boolean {
|
||||
return health.pendingSize >= LIVE_DEGRADED_PENDING
|
||||
}
|
||||
|
||||
export function safeBuildLiveFlowSample(q: Omit<FlowAnalyticsQuery, "minutes" | "skipHeavy">): {
|
||||
export async function safeBuildLiveFlowSample(q: Omit<FlowAnalyticsQuery, "minutes" | "skipHeavy">): Promise<{
|
||||
event: "sample" | "error"
|
||||
data: unknown
|
||||
} {
|
||||
}> {
|
||||
return formatLiveSseFromBuilder(async () => {
|
||||
const skipHeavy = isFlowAnalyticsDegraded()
|
||||
return await buildFlowAnalytics({ ...q, minutes: LIVE_ANALYTICS_MINUTES, skipHeavy })
|
||||
|
||||
@@ -1,11 +1,14 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
brandByAsn,
|
||||
brandByHolder,
|
||||
countryFromHolder,
|
||||
isSteamGamePort,
|
||||
lookupBrand,
|
||||
OTHER_SERVICE,
|
||||
isNamedInternetService,
|
||||
mapServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
resolveRipeCountry,
|
||||
} from "./traffic-flow-brands.js"
|
||||
|
||||
@@ -39,4 +42,35 @@ assert.equal(isNamedInternetService("DNS", "DNS"), false)
|
||||
assert.equal(mapServiceNodeId("AWS"), "svc:aws")
|
||||
assert.equal(mapServiceNodeId("Cloudflare"), "svc:cloudflare")
|
||||
|
||||
assert.equal(brandByAsn(714)?.service, "Apple")
|
||||
assert.equal(brandByAsn(714)?.category, "CDN")
|
||||
assert.equal(brandByAsn(36459)?.service, "GitHub")
|
||||
assert.equal(brandByAsn(395701)?.service, "Epic")
|
||||
assert.equal(brandByAsn(6507)?.service, "Riot")
|
||||
assert.equal(brandByAsn(33353)?.service, "PlayStation")
|
||||
assert.equal(brandByAsn(14061)?.service, "DigitalOcean")
|
||||
assert.equal(brandByAsn(24940)?.service, "Hetzner")
|
||||
assert.equal(brandByAsn(8403)?.service, "Spotify")
|
||||
assert.equal(brandByAsn(13414)?.service, "X")
|
||||
assert.equal(brandByAsn(47541)?.service, "VK")
|
||||
assert.equal(brandByAsn(47764)?.service, "VK")
|
||||
assert.equal(brandByAsn(30103)?.service, "Zoom")
|
||||
assert.equal(brandByAsn(19281)?.service, "Quad9")
|
||||
assert.equal(brandByAsn(9059)?.service, "AWS")
|
||||
assert.equal(brandByAsn(396982)?.service, "Google")
|
||||
assert.equal(brandByAsn(400645)?.service, "ChatGPT")
|
||||
|
||||
assert.equal(brandByHolder("VALVE-CORPORATION")?.service, "Steam")
|
||||
assert.equal(brandByHolder("OpenAI, LLC")?.service, "ChatGPT")
|
||||
assert.equal(brandByHolder("YouTube LLC")?.service, "YouTube")
|
||||
assert.equal(brandByHolder("AMAZON-AES - Amazon.com, Inc."), null)
|
||||
|
||||
assert.equal(isSteamGamePort(17, 27015, 50000), true)
|
||||
assert.equal(isSteamGamePort(6, 443, 50000), false)
|
||||
|
||||
assert.equal(resolveFlowBrand("104.18.35.51", 32590, "VALVE-CORPORATION", 6, 443, 1)?.service, "Cloudflare")
|
||||
assert.equal(resolveFlowBrand("203.0.113.9", 32590, "", 17, 27015, 50000)?.service, "Steam")
|
||||
assert.equal(resolveRipeCountry("", 9059, ""), "IE")
|
||||
assert.equal(resolveRipeCountry("", 24940, ""), "DE")
|
||||
|
||||
console.log("traffic-flow-brands.test.ts: ok")
|
||||
|
||||
@@ -7,59 +7,162 @@ export interface BrandHit {
|
||||
category: string
|
||||
}
|
||||
|
||||
const CDN = { category: "CDN" } as const
|
||||
const WEB = { category: "Веб" } as const
|
||||
const VIDEO = { category: "Видео / стриминг" } as const
|
||||
const GAMES = { category: "Игры" } as const
|
||||
const VOICE = { category: "Голос" } as const
|
||||
const AI = { category: "ИИ" } as const
|
||||
const DNS = { category: "DNS" } as const
|
||||
|
||||
const CLOUDFLARE: BrandHit = { service: "Cloudflare", ...CDN }
|
||||
const FASTLY: BrandHit = { service: "Fastly", ...CDN }
|
||||
const AKAMAI: BrandHit = { service: "Akamai", ...CDN }
|
||||
const AWS: BrandHit = { service: "AWS", ...CDN }
|
||||
const MICROSOFT: BrandHit = { service: "Microsoft", ...CDN }
|
||||
const YANDEX: BrandHit = { service: "Yandex", ...CDN }
|
||||
const APPLE: BrandHit = { service: "Apple", ...CDN }
|
||||
const DIGITALOCEAN: BrandHit = { service: "DigitalOcean", ...CDN }
|
||||
const HETZNER: BrandHit = { service: "Hetzner", ...CDN }
|
||||
const OVH: BrandHit = { service: "OVH", ...CDN }
|
||||
const ORACLE: BrandHit = { service: "Oracle", ...CDN }
|
||||
const LINODE: BrandHit = { service: "Linode", ...CDN }
|
||||
const VULTR: BrandHit = { service: "Vultr", ...CDN }
|
||||
const SCALEWAY: BrandHit = { service: "Scaleway", ...CDN }
|
||||
const IBM_CLOUD: BrandHit = { service: "IBM Cloud", ...CDN }
|
||||
const ALIBABA: BrandHit = { service: "Alibaba", ...CDN }
|
||||
const TENCENT: BrandHit = { service: "Tencent", ...CDN }
|
||||
const GCORE: BrandHit = { service: "G-Core", ...CDN }
|
||||
const CDN77: BrandHit = { service: "CDN77", ...CDN }
|
||||
const SELECTEL: BrandHit = { service: "Selectel", ...CDN }
|
||||
const TIMEWEB: BrandHit = { service: "Timeweb", ...CDN }
|
||||
const BEGET: BrandHit = { service: "Beget", ...CDN }
|
||||
const DDOS_GUARD: BrandHit = { service: "DDoS-Guard", ...CDN }
|
||||
const META: BrandHit = { service: "Meta", ...CDN }
|
||||
|
||||
const GOOGLE: BrandHit = { service: "Google", ...WEB }
|
||||
const GITHUB: BrandHit = { service: "GitHub", ...WEB }
|
||||
const GITLAB: BrandHit = { service: "GitLab", ...WEB }
|
||||
const X: BrandHit = { service: "X", ...WEB }
|
||||
const LINKEDIN: BrandHit = { service: "LinkedIn", ...WEB }
|
||||
const VK: BrandHit = { service: "VK", ...WEB }
|
||||
const REDDIT: BrandHit = { service: "Reddit", ...WEB }
|
||||
const DROPBOX: BrandHit = { service: "Dropbox", ...WEB }
|
||||
const SNAP: BrandHit = { service: "Snap", ...WEB }
|
||||
const WIKIPEDIA: BrandHit = { service: "Wikipedia", ...WEB }
|
||||
const PAYPAL: BrandHit = { service: "PayPal", ...WEB }
|
||||
const SALESFORCE: BrandHit = { service: "Salesforce", ...WEB }
|
||||
|
||||
const YOUTUBE: BrandHit = { service: "YouTube", ...VIDEO }
|
||||
const NETFLIX: BrandHit = { service: "Netflix", ...VIDEO }
|
||||
const TWITCH: BrandHit = { service: "Twitch", ...VIDEO }
|
||||
const TIKTOK: BrandHit = { service: "TikTok", ...VIDEO }
|
||||
const SPOTIFY: BrandHit = { service: "Spotify", ...VIDEO }
|
||||
|
||||
const STEAM: BrandHit = { service: "Steam", ...GAMES }
|
||||
const BLIZZARD: BrandHit = { service: "Blizzard", ...GAMES }
|
||||
const EPIC: BrandHit = { service: "Epic", ...GAMES }
|
||||
const RIOT: BrandHit = { service: "Riot", ...GAMES }
|
||||
const PLAYSTATION: BrandHit = { service: "PlayStation", ...GAMES }
|
||||
const ROBLOX: BrandHit = { service: "Roblox", ...GAMES }
|
||||
const UBISOFT: BrandHit = { service: "Ubisoft", ...GAMES }
|
||||
|
||||
const DISCORD: BrandHit = { service: "Discord", ...VOICE }
|
||||
const TELEGRAM: BrandHit = { service: "Telegram", ...VOICE }
|
||||
const ZOOM: BrandHit = { service: "Zoom", ...VOICE }
|
||||
|
||||
const CHATGPT: BrandHit = { service: "ChatGPT", ...AI }
|
||||
|
||||
const QUAD9: BrandHit = { service: "Quad9", ...DNS }
|
||||
const OPENDNS: BrandHit = { service: "OpenDNS", ...DNS }
|
||||
|
||||
function brandEntries(hit: BrandHit, asns: number[]): Array<[number, BrandHit]> {
|
||||
return asns.map((asn) => [asn, hit])
|
||||
}
|
||||
|
||||
function hqEntries(cc: string, asns: number[]): Array<[number, string]> {
|
||||
return asns.map((asn) => [asn, cc])
|
||||
}
|
||||
|
||||
const ASN_BRANDS = new Map<number, BrandHit>([
|
||||
[13335, { service: "Cloudflare", category: "CDN" }],
|
||||
[209242, { service: "Cloudflare", category: "CDN" }],
|
||||
[54113, { service: "Fastly", category: "CDN" }],
|
||||
[20940, { service: "Akamai", category: "CDN" }],
|
||||
[16509, { service: "AWS", category: "CDN" }],
|
||||
[14618, { service: "AWS", category: "CDN" }],
|
||||
[8075, { service: "Microsoft", category: "CDN" }],
|
||||
[13238, { service: "Yandex", category: "CDN" }],
|
||||
[32590, { service: "Steam", category: "Игры" }],
|
||||
[57976, { service: "Blizzard", category: "Игры" }],
|
||||
[2906, { service: "Netflix", category: "Видео / стриминг" }],
|
||||
[40027, { service: "Netflix", category: "Видео / стриминг" }],
|
||||
[15169, { service: "Google", category: "Веб" }],
|
||||
[36040, { service: "YouTube", category: "Видео / стриминг" }],
|
||||
[46489, { service: "Twitch", category: "Видео / стриминг" }],
|
||||
[401115, { service: "ChatGPT", category: "ИИ" }],
|
||||
[49544, { service: "Discord", category: "Голос" }],
|
||||
[62041, { service: "Telegram", category: "Голос" }],
|
||||
[59930, { service: "Telegram", category: "Голос" }],
|
||||
[211157, { service: "Telegram", category: "Голос" }],
|
||||
[32934, { service: "Meta", category: "CDN" }],
|
||||
[396986, { service: "TikTok", category: "Видео / стриминг" }],
|
||||
...brandEntries(CLOUDFLARE, [13335, 209242]),
|
||||
...brandEntries(FASTLY, [54113]),
|
||||
...brandEntries(AKAMAI, [20940, 16625, 32787, 35994, 16702, 24319]),
|
||||
...brandEntries(AWS, [16509, 14618, 8987, 7224, 9059]),
|
||||
...brandEntries(MICROSOFT, [8075, 8068, 8069, 8070]),
|
||||
...brandEntries(YANDEX, [13238]),
|
||||
...brandEntries(APPLE, [714, 6185]),
|
||||
...brandEntries(DIGITALOCEAN, [14061]),
|
||||
...brandEntries(HETZNER, [24940, 213230]),
|
||||
...brandEntries(OVH, [16276]),
|
||||
...brandEntries(ORACLE, [31898]),
|
||||
...brandEntries(LINODE, [63949]),
|
||||
...brandEntries(VULTR, [20473]),
|
||||
...brandEntries(SCALEWAY, [12876]),
|
||||
...brandEntries(IBM_CLOUD, [36351]),
|
||||
...brandEntries(ALIBABA, [45102]),
|
||||
...brandEntries(TENCENT, [132203]),
|
||||
...brandEntries(GCORE, [199524]),
|
||||
...brandEntries(CDN77, [60068]),
|
||||
...brandEntries(SELECTEL, [50340, 49505]),
|
||||
...brandEntries(TIMEWEB, [9123]),
|
||||
...brandEntries(BEGET, [198610]),
|
||||
...brandEntries(DDOS_GUARD, [57724]),
|
||||
...brandEntries(META, [32934, 63293, 54115]),
|
||||
...brandEntries(GOOGLE, [15169, 396982]),
|
||||
...brandEntries(GITHUB, [36459]),
|
||||
...brandEntries(GITLAB, [54876]),
|
||||
...brandEntries(X, [13414]),
|
||||
...brandEntries(LINKEDIN, [14413, 40793]),
|
||||
...brandEntries(VK, [47541, 47764]),
|
||||
...brandEntries(REDDIT, [394706]),
|
||||
...brandEntries(DROPBOX, [19679]),
|
||||
...brandEntries(SNAP, [19750]),
|
||||
...brandEntries(WIKIPEDIA, [14907]),
|
||||
...brandEntries(PAYPAL, [17012, 26101]),
|
||||
...brandEntries(SALESFORCE, [14340]),
|
||||
...brandEntries(YOUTUBE, [36040, 43515]),
|
||||
...brandEntries(NETFLIX, [2906, 40027]),
|
||||
...brandEntries(TWITCH, [46489]),
|
||||
...brandEntries(TIKTOK, [396986, 138699]),
|
||||
...brandEntries(SPOTIFY, [8403, 34081]),
|
||||
...brandEntries(STEAM, [32590]),
|
||||
...brandEntries(BLIZZARD, [57976]),
|
||||
...brandEntries(EPIC, [395701]),
|
||||
...brandEntries(RIOT, [6507, 62830]),
|
||||
...brandEntries(PLAYSTATION, [33353]),
|
||||
...brandEntries(ROBLOX, [22697]),
|
||||
...brandEntries(UBISOFT, [197922]),
|
||||
...brandEntries(DISCORD, [49544, 394141]),
|
||||
...brandEntries(TELEGRAM, [62041, 59930, 211157]),
|
||||
...brandEntries(ZOOM, [30103]),
|
||||
...brandEntries(CHATGPT, [401115, 400645]),
|
||||
...brandEntries(QUAD9, [19281]),
|
||||
...brandEntries(OPENDNS, [36692]),
|
||||
])
|
||||
|
||||
const ASN_HQ_COUNTRY = new Map<number, string>([
|
||||
[13335, "US"],
|
||||
[209242, "US"],
|
||||
[54113, "US"],
|
||||
[20940, "US"],
|
||||
[16509, "US"],
|
||||
[14618, "US"],
|
||||
[8075, "US"],
|
||||
[15169, "US"],
|
||||
[32590, "US"],
|
||||
[57976, "US"],
|
||||
[2906, "US"],
|
||||
[40027, "US"],
|
||||
[36040, "US"],
|
||||
[46489, "US"],
|
||||
[401115, "US"],
|
||||
[49544, "US"],
|
||||
[32934, "US"],
|
||||
[13238, "RU"],
|
||||
[62041, "NL"],
|
||||
[59930, "NL"],
|
||||
[211157, "NL"],
|
||||
...hqEntries("US", [
|
||||
13335, 209242, 54113, 20940, 16625, 32787, 35994, 16702, 24319,
|
||||
16509, 14618, 8987, 7224, 8075, 8068, 8069, 8070, 15169, 396982,
|
||||
32590, 57976, 2906, 40027, 36040, 43515, 46489, 401115, 400645,
|
||||
49544, 394141, 32934, 63293, 54115, 714, 6185, 36459, 54876, 14061,
|
||||
31898, 63949, 20473, 36351, 13414, 14413, 40793, 394706, 19679, 19750,
|
||||
14907, 17012, 26101, 14340, 30103, 36692, 395701, 6507, 62830, 33353, 22697,
|
||||
]),
|
||||
...hqEntries("IE", [9059]),
|
||||
...hqEntries("SG", [138699]),
|
||||
...hqEntries("DE", [24940, 213230]),
|
||||
...hqEntries("FR", [16276, 12876, 197922]),
|
||||
...hqEntries("CN", [45102, 132203]),
|
||||
...hqEntries("LU", [199524]),
|
||||
...hqEntries("CZ", [60068]),
|
||||
...hqEntries("RU", [13238, 50340, 49505, 9123, 198610, 57724, 47541, 47764]),
|
||||
...hqEntries("SE", [8403, 34081]),
|
||||
...hqEntries("NL", [62041, 59930, 211157]),
|
||||
...hqEntries("CH", [19281]),
|
||||
])
|
||||
|
||||
const GOOGLE: BrandHit = { service: "Google", category: "Веб" }
|
||||
const CLOUDFLARE: BrandHit = { service: "Cloudflare", category: "CDN" }
|
||||
const YOUTUBE: BrandHit = { service: "YouTube", category: "Видео / стриминг" }
|
||||
|
||||
const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
|
||||
{ cidr: "104.16.0.0/13", prefixLen: 13, hit: CLOUDFLARE },
|
||||
{ cidr: "104.24.0.0/14", prefixLen: 14, hit: CLOUDFLARE },
|
||||
@@ -75,8 +178,25 @@ const CIDR_BRANDS: Array<{ cidr: string; prefixLen: number; hit: BrandHit }> = [
|
||||
{ cidr: "208.117.224.0/19", prefixLen: 19, hit: YOUTUBE },
|
||||
].sort((a, b) => b.prefixLen - a.prefixLen)
|
||||
|
||||
const HOLDER_BRANDS: Array<{ re: RegExp; hit: BrandHit }> = [
|
||||
{ re: /youtube/i, hit: YOUTUBE },
|
||||
{ re: /valve|\bsteam\b/i, hit: STEAM },
|
||||
{ re: /blizzard|battle.?net/i, hit: BLIZZARD },
|
||||
{ re: /openai/i, hit: CHATGPT },
|
||||
{ re: /riot games/i, hit: RIOT },
|
||||
{ re: /epic games/i, hit: EPIC },
|
||||
{ re: /\bapple\b/i, hit: APPLE },
|
||||
{ re: /github/i, hit: GITHUB },
|
||||
{ re: /spotify/i, hit: SPOTIFY },
|
||||
{ re: /twitter|\bx corp\b/i, hit: X },
|
||||
{ re: /dropbox/i, hit: DROPBOX },
|
||||
{ re: /akamai/i, hit: AKAMAI },
|
||||
]
|
||||
|
||||
const NON_ISO = new Set(["EU", "AP", "ZZ", "XX", "A1", "A2", "O1"])
|
||||
|
||||
const STEAM_ASN = 32590
|
||||
|
||||
export function isIsoCountry(code: string): boolean {
|
||||
const c = String(code ?? "").trim().toUpperCase()
|
||||
return /^[A-Z]{2}$/.test(c) && !NON_ISO.has(c)
|
||||
@@ -114,10 +234,51 @@ export function brandByCidr(ip: string): BrandHit | null {
|
||||
return null
|
||||
}
|
||||
|
||||
export function brandByHolder(holder: string): BrandHit | null {
|
||||
const h = String(holder ?? "").trim()
|
||||
if (!h) return null
|
||||
for (const row of HOLDER_BRANDS) {
|
||||
if (row.re.test(h)) return row.hit
|
||||
}
|
||||
return null
|
||||
}
|
||||
|
||||
/** Игровые порты Steam — только вместе с AS32590, никогда :80/:443. */
|
||||
export function isSteamGamePort(proto: number, dstPort: number, srcPort: number): boolean {
|
||||
if (proto !== 6 && proto !== 17) return false
|
||||
const port = dstPort || srcPort
|
||||
if (!port || port === 80 || port === 443) return false
|
||||
if (port === 4380 || port === 3478) return true
|
||||
return port >= 27000 && port <= 27100
|
||||
}
|
||||
|
||||
export function lookupBrand(ip: string, asn: number): BrandHit | null {
|
||||
return brandByCidr(ip) || brandByAsn(asn)
|
||||
}
|
||||
|
||||
/**
|
||||
* Cloudflare CIDR бьёт holder (витрина на CF не становится Steam).
|
||||
* Holder (YouTube и др.) бьёт остальные CIDR/ASN.
|
||||
* Порты Steam — только AS32590 и не выше Cloudflare CIDR.
|
||||
*/
|
||||
export function resolveFlowBrand(
|
||||
ip: string,
|
||||
asn: number,
|
||||
holder: string,
|
||||
proto = 0,
|
||||
dstPort = 0,
|
||||
srcPort = 0,
|
||||
): BrandHit | null {
|
||||
const cidrBrand = brandByCidr(ip)
|
||||
if (cidrBrand?.service === "Cloudflare") return cidrBrand
|
||||
const holderBrand = brandByHolder(holder)
|
||||
if (holderBrand) return holderBrand
|
||||
const fromLookup = cidrBrand || brandByAsn(asn)
|
||||
if (fromLookup) return fromLookup
|
||||
if (asn === STEAM_ASN && isSteamGamePort(proto, dstPort, srcPort)) return STEAM
|
||||
return null
|
||||
}
|
||||
|
||||
const SKIP_MAP_SERVICES = new Set([
|
||||
OTHER_SERVICE,
|
||||
"GRE",
|
||||
|
||||
@@ -53,6 +53,71 @@ const youtube = classifyFlowDst("173.194.160.163", 6, 443, 1, {
|
||||
assert.equal(youtube.service, "YouTube")
|
||||
assert.equal(youtube.category, "Видео / стриминг")
|
||||
|
||||
const valve = classifyFlowDst("203.0.113.40", 17, 27015, 50000, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 64501,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "VALVE-CORPORATION",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(valve.service, "Steam")
|
||||
assert.equal(valve.category, "Игры")
|
||||
|
||||
const openaiHolder = classifyFlowDst("203.0.113.41", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 64502,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "OPENAI, US",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(openaiHolder.service, "ChatGPT")
|
||||
assert.equal(openaiHolder.category, "ИИ")
|
||||
|
||||
const cfNotSteam = classifyFlowDst("104.18.35.51", 6, 443, 1, {
|
||||
prefix: "104.18.0.0/16",
|
||||
asn: 32590,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "VALVE-CORPORATION",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(cfNotSteam.service, "Cloudflare")
|
||||
assert.notEqual(cfNotSteam.service, "Steam")
|
||||
|
||||
const awsIeu = classifyFlowDst("203.0.113.42", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 9059,
|
||||
country: "IE",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "AMAZON-02",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(awsIeu.service, "AWS")
|
||||
assert.equal(awsIeu.category, "CDN")
|
||||
|
||||
const googleCloud = classifyFlowDst("203.0.113.43", 6, 443, 1, {
|
||||
prefix: "203.0.113.0/24",
|
||||
asn: 396982,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE-CLOUD",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(googleCloud.service, "Google")
|
||||
assert.equal(googleCloud.category, "Веб")
|
||||
|
||||
const gre = classifyFlowDst("198.51.100.1", 47, 0, 0, null)
|
||||
assert.equal(gre.service, "GRE")
|
||||
assert.equal(gre.category, "Туннель")
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { lookupBrand, OTHER_SERVICE } from "./traffic-flow-brands.js"
|
||||
import { OTHER_SERVICE, resolveFlowBrand } from "./traffic-flow-brands.js"
|
||||
import { db } from "../db/index.js"
|
||||
import { evobgpSettings } from "../db/schema.js"
|
||||
import { ipInCidrV4, parseCidrV4 } from "./traffic-flow-ip.js"
|
||||
@@ -47,12 +47,13 @@ export function seedFlowCatalogForTests(input: {
|
||||
|
||||
export function categoryFromPurpose(purpose: string, proto: number, dstPort: number, srcPort: number): string {
|
||||
const p = purpose.toLowerCase()
|
||||
if (/gaming|steam|epic|riot/.test(p)) return "Игры"
|
||||
if (/streaming|youtube|netflix|twitch|video/.test(p)) return "Видео / стриминг"
|
||||
if (/cdn|cloudflare|akamai|fastly/.test(p)) return "CDN"
|
||||
if (/gaming|steam|epic|riot|playstation|roblox|ubisoft/.test(p)) return "Игры"
|
||||
if (/streaming|youtube|netflix|twitch|video|spotify/.test(p)) return "Видео / стриминг"
|
||||
if (/cdn|cloudflare|akamai|fastly|hetzner|ovh|apple/.test(p)) return "CDN"
|
||||
if (/voip|discord|zoom/.test(p)) return "Голос"
|
||||
if (/openai|chatgpt|\bai\b/.test(p)) return "ИИ"
|
||||
if (/веб|web|google/.test(p)) return "Веб"
|
||||
if (/quad9|opendns/.test(p)) return "DNS"
|
||||
if (/веб|web|google|github|paypal|vk|linkedin/.test(p)) return "Веб"
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "DNS" || app === "SSH" || app === "BGP") return app
|
||||
if (app === "GRE" || app === "ESP" || app === "WireGuard") return "Туннель"
|
||||
@@ -79,10 +80,7 @@ export function classifyFlowDst(
|
||||
if (app === "WireGuard") return { service: "WireGuard", category: "Туннель" }
|
||||
const hit = matchCidr(dst)
|
||||
const holder = ripe?.holder ?? ""
|
||||
const youtubeHolder = /youtube/i.test(holder)
|
||||
const brand = youtubeHolder
|
||||
? { service: "YouTube", category: "Видео / стриминг" }
|
||||
: lookupBrand(dst, ripe?.asn ?? 0)
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, holder, proto, dstPort, srcPort)
|
||||
const asnName = ripe?.asn ? asnPurpose.get(ripe.asn) : undefined
|
||||
const service = (hit?.purpose || brand?.service || asnName || OTHER_SERVICE).trim() || OTHER_SERVICE
|
||||
const category = hit
|
||||
|
||||
@@ -60,7 +60,7 @@ function stopListener(): void {
|
||||
clearInterval(flushTimer)
|
||||
flushTimer = null
|
||||
}
|
||||
void flushPending().catch((e) => {
|
||||
void flushPending({ force: true }).catch((e) => {
|
||||
setEngineError(e instanceof Error ? e.message : String(e))
|
||||
})
|
||||
if (socket) {
|
||||
|
||||
@@ -1,5 +1,10 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { dedupFlowRowsMaxBytes, flowTupleKey } from "./traffic-flow-dedup.js"
|
||||
import {
|
||||
dedupFlowRowsAcrossExporters,
|
||||
dedupFlowRowsMaxBytes,
|
||||
flowConversationKey,
|
||||
flowTupleKey,
|
||||
} from "./traffic-flow-dedup.js"
|
||||
|
||||
const a = {
|
||||
serverId: 7,
|
||||
@@ -22,4 +27,13 @@ assert.equal(flowTupleKey(a), flowTupleKey(b))
|
||||
const sameIface = dedupFlowRowsMaxBytes([a, { ...a, bytes: 3_000, packets: 2 }])
|
||||
assert.equal(sameIface[0]?.bytes, 15_000)
|
||||
|
||||
const jh = { ...a, serverId: 7, bytes: 9_000 }
|
||||
const en = { ...a, serverId: 9, bytes: 11_000, inIface: "1" }
|
||||
assert.equal(flowConversationKey(jh), flowConversationKey(en))
|
||||
assert.notEqual(flowTupleKey(jh), flowTupleKey(en))
|
||||
const across = dedupFlowRowsAcrossExporters([en, jh], (x, y) => (x.serverId === 7 ? x : y))
|
||||
assert.equal(across.length, 1)
|
||||
assert.equal(across[0]?.serverId, 7)
|
||||
assert.equal(across[0]?.bytes, 9_000)
|
||||
|
||||
console.log("traffic-flow-dedup.test.ts: ok")
|
||||
|
||||
@@ -14,6 +14,32 @@ export function flowTupleKey(r: Pick<FlowTupleRow, "serverId" | "src" | "dst" |
|
||||
return `${r.serverId}|${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/** Один разговор на всех экспортёрах (JH+EN), без serverId. */
|
||||
export function flowConversationKey(r: Pick<FlowTupleRow, "src" | "dst" | "proto" | "srcPort" | "dstPort">): string {
|
||||
return `${r.src}|${r.dst}|${r.proto}|${r.srcPort}|${r.dstPort}`
|
||||
}
|
||||
|
||||
/**
|
||||
* Схлопнуть копии одного 5-tuple с разных серверов.
|
||||
* `prefer` выбирает ряд (клиент на JH важнее голого EN).
|
||||
*/
|
||||
export function dedupFlowRowsAcrossExporters<T extends FlowTupleRow>(
|
||||
rows: T[],
|
||||
prefer: (a: T, b: T) => T,
|
||||
): T[] {
|
||||
const byConv = new Map<string, T>()
|
||||
for (const row of rows) {
|
||||
const key = flowConversationKey(row)
|
||||
const prev = byConv.get(key)
|
||||
if (!prev) {
|
||||
byConv.set(key, row)
|
||||
continue
|
||||
}
|
||||
byConv.set(key, prefer(prev, row))
|
||||
}
|
||||
return [...byConv.values()]
|
||||
}
|
||||
|
||||
function ifaceKey(r: FlowTupleRow): string {
|
||||
return `${flowTupleKey(r)}|${r.inIface}`
|
||||
}
|
||||
|
||||
@@ -0,0 +1,109 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
ingestParsedFlowsForServerForTests,
|
||||
resetEngineForTests,
|
||||
} from "./traffic-flow-engine.js"
|
||||
import { factsSnapshotForTests } from "./traffic-flow-facts.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetEngineForTests()
|
||||
resetIfaceCacheForTests()
|
||||
|
||||
seedRipeCacheForTests({
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "95.167.0.0/16",
|
||||
asn: 12389,
|
||||
country: "RU",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "ROSTELECOM-AS",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
|
||||
const topo: FlowTopology = {
|
||||
clientIfaces: new Map([[1, new Set(["gre-client"])]]),
|
||||
clientByIface: new Map([["1|gre-client", {
|
||||
userId: "u-rost",
|
||||
login: "alice",
|
||||
name: "Alice",
|
||||
serverId: 1,
|
||||
interfaceName: "gre-client",
|
||||
}]]),
|
||||
enNodes: [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
wanIfaces: new Map([[1, new Set(["ether1"])]]),
|
||||
plane: {
|
||||
clientIfaceNames: new Set(["gre-client"]),
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
},
|
||||
}
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
|
||||
|
||||
ingestParsedFlowsForServerForTests(1, [
|
||||
{
|
||||
src: "95.167.1.10",
|
||||
dst: "10.200.100.53",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 100,
|
||||
packets: 2,
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
},
|
||||
{
|
||||
src: "95.167.1.10",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 50,
|
||||
packets: 1,
|
||||
inIface: "gre-client",
|
||||
outIface: "ether1",
|
||||
},
|
||||
])
|
||||
|
||||
const facts = factsSnapshotForTests()
|
||||
const total = facts.reduce((s, r) => s + r.bytes, 0)
|
||||
const asnBytes = facts.reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(total, 150)
|
||||
assert.equal(asnBytes, 150, "unique bytes = SUM dest ASN")
|
||||
assert.equal(facts.some((r) => r.asn === 12389), false, "ASN клиента не в кубе")
|
||||
const google = facts.find((r) => r.asn === 15169)
|
||||
assert.ok(google)
|
||||
assert.equal(google.bytes, 50)
|
||||
const other = facts.filter((r) => r.asn === 0).reduce((s, r) => s + r.bytes, 0)
|
||||
assert.equal(other, 100)
|
||||
|
||||
resetEngineForTests()
|
||||
seedFlowTopologyForTests(null)
|
||||
resetRipeCacheForTests()
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-dest.test.ts: ok")
|
||||
@@ -0,0 +1,60 @@
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { classifyFlowDst, type FlowClassification } from "./traffic-flow-classify.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { pickInternetDest, type InternetDestCtx } from "./traffic-flow-ip.js"
|
||||
import type { FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import {
|
||||
flowOursHosts,
|
||||
resolveClient,
|
||||
type FlowTopology,
|
||||
} from "./traffic-flow-topology.js"
|
||||
|
||||
export interface InternetDestMeta {
|
||||
dest: string
|
||||
ripe: FlowIpMeta | null
|
||||
classified: FlowClassification
|
||||
country: string
|
||||
asn: number
|
||||
}
|
||||
|
||||
export function destCtxForIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
inIface: string,
|
||||
): InternetDestCtx {
|
||||
const name = canonicalFactIface(serverId, inIface) || String(inIface ?? "").trim()
|
||||
return {
|
||||
ours: flowOursHosts(topo),
|
||||
boundClient: Boolean(topo && name && resolveClient(topo, serverId, name)),
|
||||
}
|
||||
}
|
||||
|
||||
export function resolveInternetDest(opts: {
|
||||
src: string
|
||||
dst: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
serverId: number
|
||||
inIface: string
|
||||
topo?: FlowTopology | null
|
||||
}): InternetDestMeta {
|
||||
const dest = pickInternetDest(
|
||||
opts.src,
|
||||
opts.dst,
|
||||
opts.srcPort,
|
||||
opts.dstPort,
|
||||
destCtxForIface(opts.topo, opts.serverId, opts.inIface),
|
||||
)
|
||||
const ripe = dest ? resolveFlowIp(dest) : null
|
||||
const classified = classifyFlowDst(dest || opts.dst, opts.proto, opts.dstPort, opts.srcPort, ripe)
|
||||
if (!dest) {
|
||||
return { dest: "", ripe: null, classified, country: "", asn: 0 }
|
||||
}
|
||||
const country = ripe?.ok && isIsoCountry(ripe.country)
|
||||
? ripe.country
|
||||
: (ripe?.ok ? "" : "unknown")
|
||||
const asn = ripe?.ok && ripe.asn ? ripe.asn : 0
|
||||
return { dest, ripe, classified, country, asn }
|
||||
}
|
||||
@@ -4,14 +4,29 @@ import { normalizeParsedFlow, parseFlowPacket, protoName, type ParsedFlow, type
|
||||
import { classifyFlowPlaneLite } from "./traffic-flow-planes.js"
|
||||
import { pickServerIdForExporter, type OverlayPeerRef } from "./traffic-flow-map-exporter.js"
|
||||
import { applicationName } from "./traffic-flow-apps.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { enqueueRipeMisses, lookupRipeCached, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { enqueueRipeMisses, pruneRipeSqlite } from "./traffic-flow-ripe.js"
|
||||
import { invalidateTrafficFlowSettingsCache } from "./traffic-flow-settings.js"
|
||||
import { isIsoCountry } from "./traffic-flow-brands.js"
|
||||
import { maybeRefreshIfaces } from "./traffic-flow-ifaces.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import {
|
||||
getServerCatalog,
|
||||
loadFlowTopology,
|
||||
peekFlowTopology,
|
||||
peekServerCatalog,
|
||||
} from "./traffic-flow-topology.js"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
factsPendingSize,
|
||||
flushFlowFacts,
|
||||
hourBucketIso,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
export const TICK_MS = 2_000
|
||||
export const PERSIST_MS = 10_000
|
||||
export const STATS_PERSIST_MS = 15_000
|
||||
export const RING_LEN = 60
|
||||
export const MAX_PENDING = 50_000
|
||||
export const DAILY_ASN_TOP = 500
|
||||
@@ -48,6 +63,66 @@ export interface PendingFlowRow {
|
||||
flowEndMs: number
|
||||
}
|
||||
|
||||
function inetOrNull(value: string | null | undefined): string | null {
|
||||
const s = String(value ?? "").trim()
|
||||
return s.length > 0 ? s : null
|
||||
}
|
||||
|
||||
export function isValidFlowInet(value: string): boolean {
|
||||
const s = value.trim()
|
||||
if (!s) return false
|
||||
const v4 = /^(\d{1,3})\.(\d{1,3})\.(\d{1,3})\.(\d{1,3})$/.exec(s)
|
||||
if (v4) {
|
||||
return v4.slice(1).every((octet) => {
|
||||
const n = Number(octet)
|
||||
return Number.isInteger(n) && n >= 0 && n <= 255
|
||||
})
|
||||
}
|
||||
if (!s.includes(":")) return false
|
||||
if (!/^[0-9a-fA-F:]+$/.test(s)) return false
|
||||
const parts = s.split(":")
|
||||
if (parts.length < 3 || parts.length > 8) return false
|
||||
return parts.every((p) => p.length <= 4)
|
||||
}
|
||||
|
||||
function clampProto(n: number): number {
|
||||
if (!Number.isFinite(n)) return 0
|
||||
return Math.max(0, Math.min(255, Math.trunc(n)))
|
||||
}
|
||||
|
||||
function sanitizeFlowRow(r: PendingFlowRow): PendingFlowRow | null {
|
||||
const src = (r.src || "").trim() || "0.0.0.0"
|
||||
const dst = (r.dst || "").trim() || "0.0.0.0"
|
||||
if (!isValidFlowInet(src) || !isValidFlowInet(dst)) return null
|
||||
const next = inetOrNull(r.nextHop)
|
||||
return {
|
||||
...r,
|
||||
src,
|
||||
dst,
|
||||
nextHop: next && isValidFlowInet(next) ? next : "",
|
||||
proto: clampProto(r.proto),
|
||||
}
|
||||
}
|
||||
|
||||
function flowUpsertParams(r: PendingFlowRow) {
|
||||
return {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: inetOrNull(r.nextHop),
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
}
|
||||
}
|
||||
|
||||
export interface EngineStats {
|
||||
packetsReceived: number
|
||||
lastExporterIp: string | null
|
||||
@@ -105,6 +180,9 @@ let rowsStored = 0
|
||||
let lastFlushUsedTransaction = false
|
||||
let lastPruneAt = 0
|
||||
let lastPassiveCheckpointAt = 0
|
||||
let lastPersistAt = 0
|
||||
let lastFlushedMinute = ""
|
||||
let lastStatsPersistAt = 0
|
||||
let dataEpoch = 0
|
||||
let lastPersistedStats: {
|
||||
packetsReceived: number
|
||||
@@ -262,20 +340,33 @@ export function getEngineStats(): EngineStats {
|
||||
export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): void {
|
||||
if (flows.length) bumpDataEpoch()
|
||||
const bucketAt = minuteBucketIso()
|
||||
const hourAt = hourBucketIso()
|
||||
const ripeMisses: string[] = []
|
||||
const topo = peekFlowTopology()
|
||||
const catalog = peekServerCatalog()
|
||||
if (!topo) void loadFlowTopology().catch(() => {})
|
||||
if (!catalog) void getServerCatalog().catch(() => {})
|
||||
const serverType = catalog?.byId.get(serverId)?.type
|
||||
for (const raw of flows) {
|
||||
const flow = normalizeParsedFlow(raw)
|
||||
addToTick(serverId, flow, flow.bytes)
|
||||
bumpRollup(serverId, bucketAt, flow, flow.bytes, flow.packets)
|
||||
const peer = pickInternetPeer(flow.src, flow.dst, flow.srcPort, flow.dstPort)
|
||||
const ripe = lookupRipeCached(peer)
|
||||
if (peer && !ripe) ripeMisses.push(peer)
|
||||
const classified = classifyFlowDst(peer, flow.proto, flow.dstPort, flow.srcPort, ripe)
|
||||
const destMeta = resolveInternetDest({
|
||||
src: flow.src,
|
||||
dst: flow.dst,
|
||||
proto: flow.proto,
|
||||
srcPort: flow.srcPort,
|
||||
dstPort: flow.dstPort,
|
||||
serverId,
|
||||
inIface: flow.inIface,
|
||||
topo,
|
||||
})
|
||||
const ripe = destMeta.ripe
|
||||
if (destMeta.dest && !ripe) ripeMisses.push(destMeta.dest)
|
||||
const classified = destMeta.classified
|
||||
const app = applicationName(flow.proto, flow.dstPort, flow.srcPort)
|
||||
const country = ripe?.ok && isIsoCountry(ripe.country)
|
||||
? ripe.country
|
||||
: (ripe?.ok ? "" : "unknown")
|
||||
const asnKey = ripe?.ok && ripe.asn ? String(ripe.asn) : "unknown"
|
||||
const country = destMeta.country
|
||||
const asnKey = destMeta.asn ? String(destMeta.asn) : "unknown"
|
||||
bumpDim(serverId, bucketAt, "proto", protoName(flow.proto), flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "app", app, flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "iface", flow.inIface || "__unknown__", flow.bytes, flow.packets)
|
||||
@@ -283,6 +374,29 @@ export function queueParsedFlows(serverId: number, flows: ParsedFlowInput[]): vo
|
||||
bumpDim(serverId, bucketAt, "service", classified.service, flow.bytes, flow.packets)
|
||||
if (country) bumpDim(serverId, bucketAt, "country", country, flow.bytes, flow.packets)
|
||||
bumpDim(serverId, bucketAt, "asn", asnKey, flow.bytes, flow.packets)
|
||||
if (shouldWriteFlowFact({
|
||||
serverId,
|
||||
serverType,
|
||||
inIface: flow.inIface,
|
||||
outIface: flow.outIface,
|
||||
proto: flow.proto,
|
||||
srcPort: flow.srcPort,
|
||||
dstPort: flow.dstPort,
|
||||
src: flow.src,
|
||||
dst: flow.dst,
|
||||
topo,
|
||||
})) {
|
||||
bumpFlowFact({
|
||||
serverId,
|
||||
bucketAt: hourAt,
|
||||
iface: canonicalFactIface(serverId, flow.inIface),
|
||||
country: country || "XX",
|
||||
service: classified.service,
|
||||
asn: destMeta.asn,
|
||||
bytes: flow.bytes,
|
||||
packets: flow.packets,
|
||||
})
|
||||
}
|
||||
|
||||
const key = pendingKey(serverId, bucketAt, flow)
|
||||
const prev = pending.get(key)
|
||||
@@ -449,7 +563,7 @@ export function applyRingSnapshot(rows: Array<{ key: string; inBps: number[]; ou
|
||||
}
|
||||
}
|
||||
|
||||
async function persistListenerStats(): Promise<boolean> {
|
||||
async function persistListenerStats(force = false): Promise<boolean> {
|
||||
if (
|
||||
lastPersistedStats
|
||||
&& lastPersistedStats.packetsReceived === packetsReceived
|
||||
@@ -459,6 +573,12 @@ async function persistListenerStats(): Promise<boolean> {
|
||||
) {
|
||||
return false
|
||||
}
|
||||
const errorChanged = lastPersistedStats?.lastError !== lastError
|
||||
const exporterChanged = lastPersistedStats?.lastExporterIp !== lastExporterIp
|
||||
const now = Date.now()
|
||||
if (!force && !errorChanged && !exporterChanged && now - lastStatsPersistAt < STATS_PERSIST_MS) {
|
||||
return false
|
||||
}
|
||||
await dbQuery(`
|
||||
UPDATE traffic_flow_settings
|
||||
SET packets_received = @packetsReceived,
|
||||
@@ -480,6 +600,7 @@ async function persistListenerStats(): Promise<boolean> {
|
||||
lastExporterIp,
|
||||
lastError,
|
||||
}
|
||||
lastStatsPersistAt = now
|
||||
invalidateTrafficFlowSettingsCache()
|
||||
return true
|
||||
}
|
||||
@@ -644,20 +765,70 @@ function topNPending(rows: PendingFlowRow[]): PendingFlowRow[] {
|
||||
return out
|
||||
}
|
||||
|
||||
export async function flushPending(): Promise<void> {
|
||||
pruneRecent()
|
||||
rollFlowRings()
|
||||
await persistListenerStats()
|
||||
if (pending.size === 0 && minuteRollup.size === 0 && minuteDims.size === 0) {
|
||||
await pruneStored()
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
const rows = topNPending([...pending.values()].map(toPendingRow))
|
||||
pending.clear()
|
||||
for (const row of rows) mergeInto(recent, row)
|
||||
function persistDue(force: boolean, hasWork: boolean): boolean {
|
||||
if (force) return true
|
||||
if (!hasWork) return false
|
||||
if (minuteBucketIso() !== lastFlushedMinute) return true
|
||||
return Date.now() - lastPersistAt >= PERSIST_MS
|
||||
}
|
||||
|
||||
const upsertSql = `
|
||||
async function upsertFlowBucketsBatch(rows: PendingFlowRow[]): Promise<void> {
|
||||
if (rows.length === 0) return
|
||||
const days = new Set(rows.map((r) => r.bucketAt))
|
||||
for (const bucketAt of days) await ensureParentPartition("flow_buckets", bucketAt)
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::timestamptz[],
|
||||
$3::inet[],
|
||||
$4::inet[],
|
||||
$5::smallint[],
|
||||
$6::int[],
|
||||
$7::int[],
|
||||
$8::bigint[],
|
||||
$9::bigint[],
|
||||
$10::text[],
|
||||
$11::text[],
|
||||
$12::inet[],
|
||||
$13::bigint[],
|
||||
$14::bigint[]
|
||||
) AS t(server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms)
|
||||
ON CONFLICT (server_id, bucket_at, src, dst, proto, src_port, dst_port, in_iface)
|
||||
DO UPDATE SET
|
||||
bytes = flow_buckets.bytes + excluded.bytes,
|
||||
packets = flow_buckets.packets + excluded.packets,
|
||||
out_iface = CASE WHEN excluded.out_iface != '' THEN excluded.out_iface ELSE flow_buckets.out_iface END,
|
||||
next_hop = COALESCE(excluded.next_hop, flow_buckets.next_hop),
|
||||
flow_start_ms = CASE
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_buckets.flow_start_ms = 0 OR excluded.flow_start_ms < flow_buckets.flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_buckets.flow_start_ms END,
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms)
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => r.bucketAt),
|
||||
rows.map((r) => r.src),
|
||||
rows.map((r) => r.dst),
|
||||
rows.map((r) => r.proto),
|
||||
rows.map((r) => r.srcPort),
|
||||
rows.map((r) => r.dstPort),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
rows.map((r) => r.inIface),
|
||||
rows.map((r) => r.outIface),
|
||||
rows.map((r) => inetOrNull(r.nextHop)),
|
||||
rows.map((r) => r.flowStartMs),
|
||||
rows.map((r) => r.flowEndMs),
|
||||
],
|
||||
})
|
||||
}
|
||||
|
||||
const FLOW_UPSERT_SQL = `
|
||||
INSERT INTO flow_buckets (
|
||||
server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface, next_hop, flow_start_ms, flow_end_ms
|
||||
) VALUES (
|
||||
@@ -668,61 +839,94 @@ export async function flushPending(): Promise<void> {
|
||||
bytes = flow_buckets.bytes + excluded.bytes,
|
||||
packets = flow_buckets.packets + excluded.packets,
|
||||
out_iface = CASE WHEN excluded.out_iface != '' THEN excluded.out_iface ELSE flow_buckets.out_iface END,
|
||||
next_hop = CASE WHEN excluded.next_hop != '' THEN excluded.next_hop ELSE flow_buckets.next_hop END,
|
||||
next_hop = COALESCE(excluded.next_hop, flow_buckets.next_hop),
|
||||
flow_start_ms = CASE
|
||||
WHEN excluded.flow_start_ms > 0 AND (flow_buckets.flow_start_ms = 0 OR excluded.flow_start_ms < flow_buckets.flow_start_ms)
|
||||
THEN excluded.flow_start_ms ELSE flow_buckets.flow_start_ms END,
|
||||
flow_end_ms = GREATEST(flow_buckets.flow_end_ms, excluded.flow_end_ms)
|
||||
`
|
||||
lastFlushUsedTransaction = false
|
||||
|
||||
async function upsertFlowBuckets(rows: PendingFlowRow[]): Promise<number> {
|
||||
if (rows.length === 0) return 0
|
||||
try {
|
||||
for (const r of rows) {
|
||||
await ensureParentPartition("flow_buckets", r.bucketAt)
|
||||
await dbQuery(upsertSql, {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: r.nextHop,
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
})
|
||||
}
|
||||
lastFlushUsedTransaction = true
|
||||
rowsStored += rows.length
|
||||
bumpDataEpoch()
|
||||
} catch {
|
||||
await upsertFlowBucketsBatch(rows)
|
||||
return rows.length
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
setEngineError(`flow_buckets: ${message}`)
|
||||
let stored = 0
|
||||
for (const r of rows) {
|
||||
try {
|
||||
await ensureParentPartition("flow_buckets", r.bucketAt)
|
||||
await dbQuery(upsertSql, {
|
||||
serverId: r.serverId,
|
||||
bucketAt: r.bucketAt,
|
||||
src: r.src,
|
||||
dst: r.dst,
|
||||
proto: r.proto,
|
||||
srcPort: r.srcPort,
|
||||
dstPort: r.dstPort,
|
||||
bytes: r.bytes,
|
||||
packets: r.packets,
|
||||
inIface: r.inIface,
|
||||
outIface: r.outIface,
|
||||
nextHop: r.nextHop,
|
||||
flowStartMs: r.flowStartMs,
|
||||
flowEndMs: r.flowEndMs,
|
||||
})
|
||||
rowsStored += 1
|
||||
} catch {
|
||||
/* ignore single-row failures */
|
||||
await dbQuery(FLOW_UPSERT_SQL, flowUpsertParams(r))
|
||||
stored += 1
|
||||
} catch (rowErr) {
|
||||
if (stored === 0 && lastError.startsWith("flow_buckets:")) {
|
||||
const rowMsg = rowErr instanceof Error ? rowErr.message : String(rowErr)
|
||||
setEngineError(`flow_buckets: ${rowMsg}`)
|
||||
}
|
||||
}
|
||||
}
|
||||
return stored
|
||||
}
|
||||
}
|
||||
|
||||
export async function flushPending(opts?: { force?: boolean; prune?: boolean }): Promise<void> {
|
||||
pruneRecent()
|
||||
rollFlowRings()
|
||||
const force = Boolean(opts?.force)
|
||||
const doPrune = opts?.prune !== false
|
||||
const hasWork = pending.size > 0 || minuteRollup.size > 0 || minuteDims.size > 0 || factsPendingSize() > 0
|
||||
const due = persistDue(force, hasWork)
|
||||
try {
|
||||
await persistListenerStats(force)
|
||||
} catch {
|
||||
/* settings row may be absent in unit tests */
|
||||
}
|
||||
|
||||
if (!hasWork) {
|
||||
if (force && doPrune) {
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
}
|
||||
}
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
if (!due) {
|
||||
lastFlushUsedTransaction = false
|
||||
return
|
||||
}
|
||||
|
||||
const sanitized: PendingFlowRow[] = []
|
||||
let skippedInet = 0
|
||||
for (const row of topNPending([...pending.values()].map(toPendingRow))) {
|
||||
const clean = sanitizeFlowRow(row)
|
||||
if (!clean) {
|
||||
skippedInet += 1
|
||||
continue
|
||||
}
|
||||
sanitized.push(clean)
|
||||
}
|
||||
pending.clear()
|
||||
for (const row of sanitized) mergeInto(recent, row)
|
||||
if (skippedInet > 0) {
|
||||
setEngineError(`flow_buckets: пропуск ${skippedInet} строк с невалидным IP`)
|
||||
}
|
||||
|
||||
lastFlushUsedTransaction = false
|
||||
lastPersistAt = Date.now()
|
||||
lastFlushedMinute = minuteBucketIso()
|
||||
try {
|
||||
const stored = await upsertFlowBuckets(sanitized)
|
||||
lastFlushUsedTransaction = stored === sanitized.length
|
||||
rowsStored += stored
|
||||
if (stored > 0) bumpDataEpoch()
|
||||
} catch (err) {
|
||||
const message = err instanceof Error ? err.message : String(err)
|
||||
setEngineError(`flow_buckets: ${message}`)
|
||||
}
|
||||
try {
|
||||
await upsertMinuteAndDaily()
|
||||
@@ -730,15 +934,30 @@ export async function flushPending(): Promise<void> {
|
||||
} catch {
|
||||
/* rollup best-effort */
|
||||
}
|
||||
await pruneStored()
|
||||
try {
|
||||
await flushFlowFacts()
|
||||
} catch {
|
||||
/* statistics cube best-effort */
|
||||
}
|
||||
if (doPrune) {
|
||||
try {
|
||||
await pruneStored()
|
||||
} catch {
|
||||
/* prune best-effort */
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
export function lastFlushUsedTransactionForTests(): boolean {
|
||||
return lastFlushUsedTransaction
|
||||
}
|
||||
|
||||
export async function flushEngineNow(opts?: { prune?: boolean }): Promise<void> {
|
||||
await flushPending({ force: true, prune: opts?.prune })
|
||||
}
|
||||
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending()
|
||||
await flushEngineNow()
|
||||
}
|
||||
|
||||
export function onEngineTick(): void {
|
||||
@@ -757,6 +976,7 @@ export function resetEngineForTests(): void {
|
||||
rings.clear()
|
||||
minuteRollup.clear()
|
||||
minuteDims.clear()
|
||||
resetFactsForTests()
|
||||
packetsReceived = 0
|
||||
lastExporterIp = null
|
||||
lastError = ""
|
||||
@@ -766,6 +986,9 @@ export function resetEngineForTests(): void {
|
||||
lastFlushUsedTransaction = false
|
||||
lastPruneAt = 0
|
||||
lastPassiveCheckpointAt = Date.now()
|
||||
lastPersistAt = 0
|
||||
lastFlushedMinute = ""
|
||||
lastStatsPersistAt = 0
|
||||
lastPersistedStats = null
|
||||
bumpDataEpoch()
|
||||
pendingCap = MAX_PENDING
|
||||
@@ -778,3 +1001,25 @@ export function pendingSizeForTests(): number {
|
||||
export function droppedForTests(): number {
|
||||
return dropped
|
||||
}
|
||||
|
||||
/** Снимок минутных dims (dim → key → bytes) для тестов обогащения потоков. */
|
||||
export function minuteDimsSnapshotForTests(): Map<string, Map<string, { bytes: number; packets: number }>> {
|
||||
const out = new Map<string, Map<string, { bytes: number; packets: number }>>()
|
||||
for (const [k, acc] of minuteDims) {
|
||||
// dimKey: serverId\0bucketAt\0dim\0key
|
||||
const parts = k.split("\0")
|
||||
const dim = parts[2] ?? ""
|
||||
const key = parts.slice(3).join("\0")
|
||||
let byKey = out.get(dim)
|
||||
if (!byKey) {
|
||||
byKey = new Map()
|
||||
out.set(dim, byKey)
|
||||
}
|
||||
const prev = byKey.get(key)
|
||||
byKey.set(key, {
|
||||
bytes: (prev?.bytes ?? 0) + acc.bytes,
|
||||
packets: (prev?.packets ?? 0) + acc.packets,
|
||||
})
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
@@ -0,0 +1,150 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
isJunkFactIface,
|
||||
isOverlayGreIface,
|
||||
isOverlayTunnelIface,
|
||||
isWanFactIface,
|
||||
shouldWriteFlowFact,
|
||||
} from "./traffic-flow-facts-filter.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
|
||||
function topo(partial: Partial<FlowTopology> = {}): FlowTopology {
|
||||
const wanIfaces = partial.wanIfaces ?? new Map([[1, new Set(["ether1"])]])
|
||||
const clientIfaces = partial.clientIfaces ?? new Map([[1, new Set(["gre-client"])]])
|
||||
const clientByIface = partial.clientByIface ?? new Map()
|
||||
const enHosts = partial.enHosts ?? new Set(["198.51.100.1"])
|
||||
const jhHosts = partial.jhHosts ?? new Set(["203.0.113.10"])
|
||||
return {
|
||||
clientIfaces,
|
||||
clientByIface,
|
||||
enNodes: partial.enNodes ?? [{ id: 2, name: "en", hosts: ["198.51.100.1"] }],
|
||||
enHosts,
|
||||
jhHosts,
|
||||
wanIfaces,
|
||||
tunnelIfaces: partial.tunnelIfaces,
|
||||
plane: partial.plane ?? {
|
||||
clientIfaceNames: new Set(["gre-client"]),
|
||||
enHosts,
|
||||
jhHosts,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
rememberServerIfaces(1, [{ name: "ether1", ifindex: "2" }])
|
||||
seedFlowTopologyForTests(topo())
|
||||
|
||||
assert.equal(isJunkFactIface("0"), true)
|
||||
assert.equal(isJunkFactIface(""), true)
|
||||
assert.equal(isJunkFactIface("wg-flow"), true)
|
||||
assert.equal(isJunkFactIface("ether1"), false)
|
||||
assert.equal(isWanFactIface(topo(), 1, "ether1"), true)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "gre-en"), true)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "gre-client"), false)
|
||||
assert.equal(isOverlayGreIface(topo(), 1, "ether1"), false)
|
||||
|
||||
const typed = topo({
|
||||
clientIfaces: new Map([[1, new Set(["gre-client", "wg-server"])]]),
|
||||
tunnelIfaces: new Map([[1, new Set(["gre-en", "NSK-SERVHOST-RTK", "wg-jh-en", "wg-server"])]]),
|
||||
plane: {
|
||||
clientIfaceNames: new Set(["gre-client", "wg-server"]),
|
||||
enHosts: new Set(["198.51.100.1"]),
|
||||
jhHosts: new Set(["203.0.113.10"]),
|
||||
},
|
||||
})
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "NSK-SERVHOST-RTK"), true, "кастомное GRE overlay по type")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-jh-en"), true, "WG overlay по type")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-server"), false, "клиентский WG с binding")
|
||||
assert.equal(isOverlayTunnelIface(typed, 1, "wg-flow"), false, "wg-flow не overlay")
|
||||
assert.equal(isOverlayTunnelIface(topo(), 1, "NSK-SERVHOST-RTK"), false, "без type в снимке — не overlay")
|
||||
|
||||
const overlayOuter = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "ether1",
|
||||
outIface: "gre-en",
|
||||
proto: 47,
|
||||
srcPort: 0,
|
||||
dstPort: 0,
|
||||
src: "203.0.113.10",
|
||||
dst: "198.51.100.1",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(overlayOuter, false, "overlay proto 47 на ether1 не в facts")
|
||||
|
||||
const payloadGre = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "gre-client",
|
||||
outIface: "gre-en",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(payloadGre, true, "payload на GRE — да")
|
||||
|
||||
const payloadWan = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "ether1",
|
||||
outIface: "gre-client",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 51234,
|
||||
src: "8.8.8.8",
|
||||
dst: "10.100.1.17",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(payloadWan, true, "payload на ether1 WAN — да")
|
||||
|
||||
const junkZero = shouldWriteFlowFact({
|
||||
serverId: 1,
|
||||
serverType: "jump-host",
|
||||
inIface: "0",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 80,
|
||||
src: "1.1.1.1",
|
||||
dst: "8.8.8.8",
|
||||
topo: topo(),
|
||||
})
|
||||
assert.equal(junkZero, false)
|
||||
|
||||
const enTopo = topo({
|
||||
clientIfaces: new Map([[2, new Set()]]),
|
||||
wanIfaces: new Map([[2, new Set(["ether1"])]]),
|
||||
})
|
||||
const enTransit = shouldWriteFlowFact({
|
||||
serverId: 2,
|
||||
serverType: "exit-node",
|
||||
inIface: "gre-jh",
|
||||
outIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
topo: enTopo,
|
||||
})
|
||||
assert.equal(enTransit, false, "EN-транзит без клиента — нет")
|
||||
|
||||
const enWan = shouldWriteFlowFact({
|
||||
serverId: 2,
|
||||
serverType: "exit-node",
|
||||
inIface: "ether1",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 80,
|
||||
src: "8.8.8.8",
|
||||
dst: "198.51.100.1",
|
||||
topo: enTopo,
|
||||
})
|
||||
assert.equal(enWan, true, "WAN payload на EN — да")
|
||||
|
||||
seedFlowTopologyForTests(null)
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-facts-filter.test.ts: ok")
|
||||
@@ -0,0 +1,119 @@
|
||||
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
|
||||
import { STATISTICS_DUP_MARK, STATISTICS_WAN_MARK } from "@mmapp/contracts/statistics"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { classifyFlowPlane } from "./traffic-flow-planes.js"
|
||||
import { resolveClient, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
const JUNK_IFACE = new Set(["", "0", "—", "__unknown__", "wg-flow"])
|
||||
|
||||
export { STATISTICS_WAN_MARK, STATISTICS_DUP_MARK }
|
||||
|
||||
export function isJunkFactIface(iface: string | null | undefined): boolean {
|
||||
const n = String(iface ?? "").trim()
|
||||
if (JUNK_IFACE.has(n)) return true
|
||||
return /^#?0$/.test(n)
|
||||
}
|
||||
|
||||
export function isDashDisplayIface(iface: string): boolean {
|
||||
return String(iface ?? "").trim() === "—"
|
||||
}
|
||||
|
||||
export function isMgmtIface(name: string): boolean {
|
||||
const n = String(name ?? "").trim().toLowerCase()
|
||||
return n === "wg-flow" || n.endsWith("/wg-flow") || n.includes("wg-flow")
|
||||
}
|
||||
|
||||
/** GRE или WG по снимку RouterOS, иначе по имени. */
|
||||
export function isTunnelIfaceName(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isJunkFactIface(name) || isMgmtIface(name)) return false
|
||||
const typed = topo?.tunnelIfaces?.get(serverId)
|
||||
if (typed && typed.size > 0) return typed.has(name)
|
||||
const t = mapRosInterfaceType("", name)
|
||||
return t === "gre" || t === "wg"
|
||||
}
|
||||
|
||||
/** WAN uplink: `wanIfaces` топологии, иначе ether1 у JH/EN без wan_uplinks. */
|
||||
export function isWanFactIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isJunkFactIface(name) || isDashDisplayIface(name)) return false
|
||||
const wan = topo?.wanIfaces.get(serverId)
|
||||
if (wan && wan.size > 0) return wan.has(name)
|
||||
return /^ether1$/i.test(name)
|
||||
}
|
||||
|
||||
/** Overlay JH↔EN: GRE/WG не клиент, не WAN, не wg-flow. */
|
||||
export function isOverlayTunnelIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
const name = String(iface ?? "").trim()
|
||||
if (!name || isWanFactIface(topo, serverId, name) || isMgmtIface(name)) return false
|
||||
if (topo?.clientIfaces.get(serverId)?.has(name)) return false
|
||||
return isTunnelIfaceName(topo, serverId, name)
|
||||
}
|
||||
|
||||
/** @deprecated используйте isOverlayTunnelIface (GRE и WG). */
|
||||
export function isOverlayGreIface(
|
||||
topo: FlowTopology | null | undefined,
|
||||
serverId: number,
|
||||
iface: string,
|
||||
): boolean {
|
||||
return isOverlayTunnelIface(topo, serverId, iface)
|
||||
}
|
||||
|
||||
export function shouldWriteFlowFact(opts: {
|
||||
serverId: number
|
||||
serverType?: string
|
||||
inIface: string
|
||||
outIface?: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
src: string
|
||||
dst: string
|
||||
topo?: FlowTopology | null
|
||||
}): boolean {
|
||||
const inName = canonicalFactIface(opts.serverId, opts.inIface) || String(opts.inIface ?? "").trim()
|
||||
if (isJunkFactIface(inName) || isJunkFactIface(opts.inIface)) return false
|
||||
const outRaw = String(opts.outIface ?? "").trim()
|
||||
const outName = outRaw ? (canonicalFactIface(opts.serverId, outRaw) || outRaw) : ""
|
||||
const plane = classifyFlowPlane({
|
||||
src: opts.src,
|
||||
dst: opts.dst,
|
||||
proto: opts.proto,
|
||||
srcPort: opts.srcPort,
|
||||
dstPort: opts.dstPort,
|
||||
inIface: inName,
|
||||
outIface: outName || undefined,
|
||||
}, opts.topo?.plane)
|
||||
if (plane !== "payload") return false
|
||||
if (opts.serverType === "exit-node" && opts.topo) {
|
||||
const client =
|
||||
resolveClient(opts.topo, opts.serverId, inName)
|
||||
?? (outName ? resolveClient(opts.topo, opts.serverId, outName) : null)
|
||||
if (!client && isOverlayTunnelIface(opts.topo, opts.serverId, inName)) return false
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
export function wanIfaceLabel(serverName: string, iface: string): string {
|
||||
return `${serverName} · ${iface} · ${STATISTICS_WAN_MARK}`
|
||||
}
|
||||
|
||||
export function overlayDupLabel(serverName: string, iface: string): string {
|
||||
return `${serverName} · ${iface} · ${STATISTICS_DUP_MARK}`
|
||||
}
|
||||
|
||||
export function isNonUniqueShareLabel(label: string): boolean {
|
||||
return label.includes(STATISTICS_WAN_MARK) || label.includes(`· ${STATISTICS_DUP_MARK}`)
|
||||
}
|
||||
@@ -0,0 +1,138 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { ensurePartitionFor } from "../db/partitions.js"
|
||||
import { pool } from "../db/index.js"
|
||||
import { invalidateFlowCatalogCache } from "./traffic-flow-topology.js"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import { rememberServerIfaces, resetIfaceCacheForTests } from "./traffic-flow-ifindex.js"
|
||||
import { resetEngineForTests } from "./traffic-flow-engine.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-facts-rebuild.test.ts: skip")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
const nServers = (await dbQuery<{ n: number }>(`SELECT COUNT(*)::int AS n FROM servers`)).rows[0]?.n ?? 0
|
||||
if (nServers > 10) {
|
||||
console.warn("traffic-flow-facts-rebuild.test.ts: skip (не пустая БД)")
|
||||
process.exit(0)
|
||||
}
|
||||
|
||||
disableCatalogFetchForTests()
|
||||
resetFlowCatalogForTests()
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetEngineForTests()
|
||||
resetIfaceCacheForTests()
|
||||
|
||||
seedRipeCacheForTests({
|
||||
prefix: "8.8.8.0/24",
|
||||
asn: 15169,
|
||||
country: "US",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "GOOGLE",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
seedRipeCacheForTests({
|
||||
prefix: "95.167.0.0/16",
|
||||
asn: 12389,
|
||||
country: "RU",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "ROSTELECOM-AS",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
|
||||
const inserted = await dbQuery<{ id: number }>(`
|
||||
INSERT INTO servers (name, host, type, wan_uplinks)
|
||||
VALUES ('rebuild-facts-jh', '203.0.113.10', 'jump-host', '[{"iface":"ether1"}]'::jsonb)
|
||||
RETURNING id
|
||||
`)
|
||||
const serverId = inserted.rows[0]?.id
|
||||
if (serverId == null) throw new Error("no server")
|
||||
|
||||
const ts = new Date()
|
||||
await ensurePartitionFor(pool, "flow_buckets", "day", ts)
|
||||
await ensurePartitionFor(pool, "flow_hour_facts", "day", ts)
|
||||
await ensurePartitionFor(pool, "flow_daily_facts", "month", ts)
|
||||
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO app_users (id, name, login, role, active)
|
||||
VALUES ('u-rebuild-1', 'Клиент', 'rebuild-user', 'viewer', TRUE)
|
||||
ON CONFLICT (id) DO NOTHING
|
||||
`)
|
||||
await dbQuery(`
|
||||
INSERT INTO user_interface_bindings (id, user_id, server_id, interface_name, interface_type)
|
||||
VALUES ('bind-rebuild-1', 'u-rebuild-1', $1, 'gre-client', 'gre')
|
||||
`, [serverId])
|
||||
await dbQuery(`
|
||||
INSERT INTO server_snapshots (server_id, polled_at, status, raw_interfaces)
|
||||
VALUES ($1, now(), 'online', $2::jsonb)
|
||||
`, [serverId, JSON.stringify([{ name: "gre-client", type: "gre-tunnel" }, { name: "ether1", type: "ether" }])])
|
||||
|
||||
rememberServerIfaces(serverId, [{ name: "gre-client", ifindex: "2" }])
|
||||
invalidateFlowCatalogCache()
|
||||
|
||||
const bucketAt = new Date(Date.UTC(
|
||||
ts.getUTCFullYear(),
|
||||
ts.getUTCMonth(),
|
||||
ts.getUTCDate(),
|
||||
ts.getUTCHours(),
|
||||
0, 0, 0,
|
||||
)).toISOString()
|
||||
|
||||
await dbQuery(`
|
||||
INSERT INTO flow_buckets (server_id, bucket_at, src, dst, proto, src_port, dst_port, bytes, packets, in_iface, out_iface)
|
||||
VALUES
|
||||
($1, $2, '95.167.1.10', '10.200.100.53', 6, 51234, 443, 100, 2, 'gre-client', 'ether1'),
|
||||
($1, $2, '95.167.1.10', '8.8.8.8', 6, 51234, 443, 50, 1, 'gre-client', 'ether1')
|
||||
`, [serverId, bucketAt])
|
||||
|
||||
try {
|
||||
const { rebuildFlowFactsFromBuckets } = await import("./traffic-flow-facts-rebuild.js")
|
||||
const result = await rebuildFlowFactsFromBuckets()
|
||||
assert.equal(result.ok, true)
|
||||
assert.ok(result.buckets >= 2)
|
||||
|
||||
const rows = await dbQuery<{ asn: number; bytes: number }>(`
|
||||
SELECT asn, SUM(bytes)::bigint AS bytes
|
||||
FROM flow_hour_facts
|
||||
WHERE server_id = $1
|
||||
GROUP BY asn
|
||||
`, [serverId])
|
||||
const byAsn = new Map(rows.rows.map((r) => [Number(r.asn), Number(r.bytes)]))
|
||||
const total = [...byAsn.values()].reduce((s, n) => s + n, 0)
|
||||
assert.equal(total, 150)
|
||||
assert.equal(byAsn.get(12389), undefined, "ASN клиента не в hour facts")
|
||||
assert.equal(byAsn.get(15169), 50)
|
||||
assert.equal(byAsn.get(0), 100)
|
||||
} finally {
|
||||
await dbQuery(`DELETE FROM flow_hour_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_daily_facts WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM flow_buckets WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM user_interface_bindings WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM server_snapshots WHERE server_id = $1`, [serverId])
|
||||
await dbQuery(`DELETE FROM app_users WHERE id = 'u-rebuild-1'`)
|
||||
await dbQuery(`DELETE FROM servers WHERE id = $1`, [serverId])
|
||||
resetEngineForTests()
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-facts-rebuild.test.ts: ok")
|
||||
@@ -0,0 +1,130 @@
|
||||
import { dbAll, dbGet, dbQuery, withAdvisoryLock } from "../db/index.js"
|
||||
import { flushEngineNow } from "./traffic-flow-engine.js"
|
||||
import { resolveInternetDest } from "./traffic-flow-dest.js"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
discardPendingFacts,
|
||||
flushFlowFacts,
|
||||
hourBucketIso,
|
||||
} from "./traffic-flow-facts.js"
|
||||
import { shouldWriteFlowFact } from "./traffic-flow-facts-filter.js"
|
||||
import { canonicalFactIface } from "./traffic-flow-ifindex.js"
|
||||
import { getServerCatalog, loadFlowTopology } from "./traffic-flow-topology.js"
|
||||
|
||||
export const FACT_REBUILD_LOCK_KEY = 8_723_104
|
||||
const BATCH = 4_000
|
||||
|
||||
export interface FlowFactsRebuildResult {
|
||||
ok: true
|
||||
buckets: number
|
||||
facts: number
|
||||
days: string[]
|
||||
}
|
||||
|
||||
function hourFromBucket(raw: Date | string): string {
|
||||
const iso = raw instanceof Date ? raw.toISOString() : String(raw)
|
||||
const ms = new Date(iso).getTime()
|
||||
return hourBucketIso(Number.isFinite(ms) ? ms : Date.now())
|
||||
}
|
||||
|
||||
export async function rebuildFlowFactsFromBuckets(): Promise<FlowFactsRebuildResult> {
|
||||
return await withAdvisoryLock(FACT_REBUILD_LOCK_KEY, async () => {
|
||||
await flushEngineNow({ prune: false })
|
||||
discardPendingFacts()
|
||||
|
||||
const days = await dbAll<{ day: string }>(`
|
||||
SELECT DISTINCT (bucket_at AT TIME ZONE 'UTC')::date::text AS day
|
||||
FROM flow_buckets
|
||||
ORDER BY 1
|
||||
`)
|
||||
const dayList = days.map((r) => r.day).filter(Boolean)
|
||||
if (dayList.length === 0) {
|
||||
return { ok: true as const, buckets: 0, facts: 0, days: [] }
|
||||
}
|
||||
|
||||
await dbQuery(
|
||||
`DELETE FROM flow_hour_facts WHERE (bucket_at AT TIME ZONE 'UTC')::date = ANY(?::date[])`,
|
||||
[dayList],
|
||||
)
|
||||
await dbQuery(
|
||||
`DELETE FROM flow_daily_facts WHERE day = ANY(?::date[])`,
|
||||
[dayList],
|
||||
)
|
||||
|
||||
const topo = await loadFlowTopology()
|
||||
const catalog = await getServerCatalog()
|
||||
let offset = 0
|
||||
let buckets = 0
|
||||
|
||||
for (;;) {
|
||||
const rows = await dbAll<{
|
||||
serverId: number
|
||||
bucketAt: Date | string
|
||||
src: string
|
||||
dst: string
|
||||
proto: number
|
||||
srcPort: number
|
||||
dstPort: number
|
||||
bytes: number
|
||||
packets: number
|
||||
inIface: string
|
||||
outIface: string
|
||||
}>(`
|
||||
SELECT server_id AS "serverId", bucket_at AS "bucketAt",
|
||||
host(src) AS src, host(dst) AS dst, proto, src_port AS "srcPort", dst_port AS "dstPort",
|
||||
bytes, packets, in_iface AS "inIface", COALESCE(out_iface, '') AS "outIface"
|
||||
FROM flow_buckets
|
||||
ORDER BY bucket_at, server_id
|
||||
LIMIT ? OFFSET ?
|
||||
`, [BATCH, offset])
|
||||
if (rows.length === 0) break
|
||||
for (const row of rows) {
|
||||
buckets += 1
|
||||
const serverType = catalog.byId.get(row.serverId)?.type
|
||||
if (!shouldWriteFlowFact({
|
||||
serverId: row.serverId,
|
||||
serverType,
|
||||
inIface: row.inIface,
|
||||
outIface: row.outIface,
|
||||
proto: Number(row.proto) || 0,
|
||||
srcPort: Number(row.srcPort) || 0,
|
||||
dstPort: Number(row.dstPort) || 0,
|
||||
src: row.src,
|
||||
dst: row.dst,
|
||||
topo,
|
||||
})) continue
|
||||
const destMeta = resolveInternetDest({
|
||||
src: row.src,
|
||||
dst: row.dst,
|
||||
proto: Number(row.proto) || 0,
|
||||
srcPort: Number(row.srcPort) || 0,
|
||||
dstPort: Number(row.dstPort) || 0,
|
||||
serverId: row.serverId,
|
||||
inIface: row.inIface,
|
||||
topo,
|
||||
})
|
||||
bumpFlowFact({
|
||||
serverId: row.serverId,
|
||||
bucketAt: hourFromBucket(row.bucketAt),
|
||||
iface: canonicalFactIface(row.serverId, row.inIface),
|
||||
country: destMeta.country || "XX",
|
||||
service: destMeta.classified.service,
|
||||
asn: destMeta.asn,
|
||||
bytes: Number(row.bytes) || 0,
|
||||
packets: Number(row.packets) || 0,
|
||||
})
|
||||
}
|
||||
offset += rows.length
|
||||
if (rows.length < BATCH) break
|
||||
}
|
||||
|
||||
const facts = await flushFlowFacts()
|
||||
const range = await dbGet<{ n: number }>(`SELECT COUNT(*)::int AS n FROM flow_buckets`)
|
||||
return {
|
||||
ok: true as const,
|
||||
buckets: Number(range?.n) || buckets,
|
||||
facts,
|
||||
days: dayList,
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -0,0 +1,71 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bumpFlowFact,
|
||||
collectCappedFacts,
|
||||
FACT_ASN_TOP,
|
||||
FACT_TUPLE_CAP,
|
||||
resetFactsForTests,
|
||||
} from "./traffic-flow-facts.js"
|
||||
|
||||
resetFactsForTests()
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "US",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 100,
|
||||
packets: 2,
|
||||
})
|
||||
bumpFlowFact({
|
||||
serverId: 1,
|
||||
bucketAt: "2026-09-10T10:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "us",
|
||||
service: "https",
|
||||
asn: 15169,
|
||||
bytes: 50,
|
||||
packets: 1,
|
||||
})
|
||||
const merged = collectCappedFacts()
|
||||
assert.equal(merged.length, 1)
|
||||
assert.equal(merged[0]?.bytes, 150)
|
||||
assert.equal(merged[0]?.country, "US")
|
||||
assert.equal(merged[0]?.asn, 15169)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 1; i <= FACT_ASN_TOP + 20; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 2,
|
||||
bucketAt: "2026-09-10T11:00:00.000Z",
|
||||
iface: "ether1",
|
||||
country: "DE",
|
||||
service: "https",
|
||||
asn: i,
|
||||
bytes: FACT_ASN_TOP + 21 - i,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedAsn = collectCappedFacts()
|
||||
const asns = new Set(cappedAsn.map((r) => r.asn))
|
||||
assert.ok(asns.has(0))
|
||||
assert.ok(asns.size <= FACT_ASN_TOP + 1)
|
||||
|
||||
resetFactsForTests()
|
||||
for (let i = 0; i < FACT_TUPLE_CAP + 30; i++) {
|
||||
bumpFlowFact({
|
||||
serverId: 3,
|
||||
bucketAt: "2026-09-10T12:00:00.000Z",
|
||||
iface: `ether${i % 3}`,
|
||||
country: "NL",
|
||||
service: `svc-${i}`,
|
||||
asn: 1,
|
||||
bytes: 10,
|
||||
packets: 1,
|
||||
})
|
||||
}
|
||||
const cappedTuples = collectCappedFacts()
|
||||
assert.ok(cappedTuples.length <= FACT_TUPLE_CAP + 3)
|
||||
|
||||
console.log("traffic-flow-facts.test.ts: ok")
|
||||
@@ -0,0 +1,289 @@
|
||||
import { pool } from "../db/index.js"
|
||||
import { ensurePartitionFor, specForParent } from "../db/partitions.js"
|
||||
|
||||
export const FACT_ASN_TOP = 200
|
||||
export const FACT_TUPLE_CAP = 8000
|
||||
export const FACT_SERVICE_MAX_LEN = 64
|
||||
export const UNKNOWN_COUNTRY = "XX"
|
||||
export const OTHER_SERVICE = "other"
|
||||
export const UNKNOWN_IFACE = "__unknown__"
|
||||
|
||||
export interface FactAcc {
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
export interface FactRow {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}
|
||||
|
||||
const hourFacts = new Map<string, FactAcc>()
|
||||
const ensuredParts = new Set<string>()
|
||||
|
||||
export function hourBucketIso(at = Date.now()): string {
|
||||
const d = new Date(at)
|
||||
d.setMinutes(0, 0, 0)
|
||||
return d.toISOString()
|
||||
}
|
||||
|
||||
export function normalizeFactCountry(raw: string): string {
|
||||
const iso = raw.trim().toUpperCase()
|
||||
if (/^[A-Z]{2}$/.test(iso)) return iso
|
||||
return UNKNOWN_COUNTRY
|
||||
}
|
||||
|
||||
export function normalizeFactService(raw: string): string {
|
||||
const s = raw.trim().slice(0, FACT_SERVICE_MAX_LEN)
|
||||
return s || OTHER_SERVICE
|
||||
}
|
||||
|
||||
export function normalizeFactIface(raw: string): string {
|
||||
return raw.trim() || UNKNOWN_IFACE
|
||||
}
|
||||
|
||||
export function normalizeFactAsn(raw: number): number {
|
||||
if (!Number.isFinite(raw) || raw <= 0) return 0
|
||||
return Math.trunc(raw)
|
||||
}
|
||||
|
||||
function factKey(
|
||||
serverId: number,
|
||||
bucketAt: string,
|
||||
iface: string,
|
||||
country: string,
|
||||
service: string,
|
||||
asn: number,
|
||||
): string {
|
||||
return `${serverId}\0${bucketAt}\0${iface}\0${country}\0${service}\0${asn}`
|
||||
}
|
||||
|
||||
function parseFactKey(k: string, acc: FactAcc): FactRow | null {
|
||||
const parts = k.split("\0")
|
||||
if (parts.length !== 6) return null
|
||||
const serverId = Number(parts[0])
|
||||
const asn = Number(parts[5])
|
||||
if (!Number.isFinite(serverId) || !Number.isFinite(asn)) return null
|
||||
return {
|
||||
serverId,
|
||||
bucketAt: parts[1] ?? "",
|
||||
iface: parts[2] ?? UNKNOWN_IFACE,
|
||||
country: parts[3] ?? UNKNOWN_COUNTRY,
|
||||
service: parts[4] ?? OTHER_SERVICE,
|
||||
asn,
|
||||
bytes: acc.bytes,
|
||||
packets: acc.packets,
|
||||
}
|
||||
}
|
||||
|
||||
export function bumpFlowFact(row: {
|
||||
serverId: number
|
||||
bucketAt: string
|
||||
iface: string
|
||||
country: string
|
||||
service: string
|
||||
asn: number
|
||||
bytes: number
|
||||
packets: number
|
||||
}): void {
|
||||
const iface = normalizeFactIface(row.iface)
|
||||
const country = normalizeFactCountry(row.country)
|
||||
const service = normalizeFactService(row.service)
|
||||
const asn = normalizeFactAsn(row.asn)
|
||||
const k = factKey(row.serverId, row.bucketAt, iface, country, service, asn)
|
||||
const prev = hourFacts.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
hourFacts.set(k, { bytes: row.bytes, packets: row.packets })
|
||||
}
|
||||
|
||||
function groupKey(row: FactRow): string {
|
||||
return `${row.serverId}\0${row.bucketAt}`
|
||||
}
|
||||
|
||||
function mergeRow(map: Map<string, FactRow>, row: FactRow): void {
|
||||
const k = factKey(row.serverId, row.bucketAt, row.iface, row.country, row.service, row.asn)
|
||||
const prev = map.get(k)
|
||||
if (prev) {
|
||||
prev.bytes += row.bytes
|
||||
prev.packets += row.packets
|
||||
return
|
||||
}
|
||||
map.set(k, { ...row })
|
||||
}
|
||||
|
||||
/** Cap ASN tail and tuple count per server×hour before persist. */
|
||||
export function collectCappedFacts(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
hourFacts.clear()
|
||||
|
||||
const groups = new Map<string, FactRow[]>()
|
||||
for (const row of parsed) {
|
||||
const g = groupKey(row)
|
||||
const list = groups.get(g) ?? []
|
||||
list.push(row)
|
||||
groups.set(g, list)
|
||||
}
|
||||
|
||||
const out = new Map<string, FactRow>()
|
||||
for (const list of groups.values()) {
|
||||
const byAsn = new Map<number, number>()
|
||||
for (const row of list) {
|
||||
byAsn.set(row.asn, (byAsn.get(row.asn) ?? 0) + row.bytes)
|
||||
}
|
||||
const asnKeep = new Set(
|
||||
[...byAsn.entries()]
|
||||
.sort((a, b) => b[1] - a[1])
|
||||
.slice(0, FACT_ASN_TOP)
|
||||
.map(([asn]) => asn),
|
||||
)
|
||||
const afterAsn: FactRow[] = []
|
||||
for (const row of list) {
|
||||
if (asnKeep.has(row.asn) || row.asn === 0) {
|
||||
afterAsn.push(row)
|
||||
continue
|
||||
}
|
||||
afterAsn.push({ ...row, asn: 0 })
|
||||
}
|
||||
const collapsed = new Map<string, FactRow>()
|
||||
for (const row of afterAsn) mergeRow(collapsed, row)
|
||||
const tuples = [...collapsed.values()].sort((a, b) => b.bytes - a.bytes)
|
||||
const keep = tuples.slice(0, FACT_TUPLE_CAP)
|
||||
const tail = tuples.slice(FACT_TUPLE_CAP)
|
||||
for (const row of keep) mergeRow(out, row)
|
||||
for (const row of tail) {
|
||||
mergeRow(out, {
|
||||
...row,
|
||||
country: UNKNOWN_COUNTRY,
|
||||
service: OTHER_SERVICE,
|
||||
asn: 0,
|
||||
})
|
||||
}
|
||||
}
|
||||
return [...out.values()]
|
||||
}
|
||||
|
||||
async function ensureParentPartition(parent: string, ts: string): Promise<void> {
|
||||
const spec = specForParent(parent)
|
||||
if (!spec) return
|
||||
const iso = ts.length === 10 ? `${ts}T00:00:00Z` : ts
|
||||
const key = `${parent}:${iso.slice(0, 10)}`
|
||||
if (ensuredParts.has(key)) return
|
||||
await ensurePartitionFor(pool, parent, spec.kind, new Date(iso))
|
||||
ensuredParts.add(key)
|
||||
}
|
||||
|
||||
function dayKey(bucketAt: string): string {
|
||||
return bucketAt.slice(0, 10)
|
||||
}
|
||||
|
||||
export async function flushFlowFacts(): Promise<number> {
|
||||
const rows = collectCappedFacts()
|
||||
if (rows.length === 0) return 0
|
||||
const hours = new Set(rows.map((r) => r.bucketAt))
|
||||
const days = new Set(rows.map((r) => dayKey(r.bucketAt)))
|
||||
for (const h of hours) await ensureParentPartition("flow_hour_facts", h)
|
||||
for (const d of days) await ensureParentPartition("flow_daily_facts", d)
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_hour_facts (
|
||||
server_id, bucket_at, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::timestamptz[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, bucket_at, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, bucket_at, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_hour_facts.bytes + excluded.bytes,
|
||||
packets = flow_hour_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => r.bucketAt),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
|
||||
await pool.query({
|
||||
text: `
|
||||
INSERT INTO flow_daily_facts (
|
||||
server_id, day, iface, country, service, asn, bytes, packets
|
||||
)
|
||||
SELECT *
|
||||
FROM UNNEST(
|
||||
$1::bigint[],
|
||||
$2::date[],
|
||||
$3::text[],
|
||||
$4::char(2)[],
|
||||
$5::text[],
|
||||
$6::int[],
|
||||
$7::bigint[],
|
||||
$8::bigint[]
|
||||
) AS t(server_id, day, iface, country, service, asn, bytes, packets)
|
||||
ON CONFLICT (server_id, day, iface, country, service, asn)
|
||||
DO UPDATE SET
|
||||
bytes = flow_daily_facts.bytes + excluded.bytes,
|
||||
packets = flow_daily_facts.packets + excluded.packets
|
||||
`,
|
||||
values: [
|
||||
rows.map((r) => r.serverId),
|
||||
rows.map((r) => dayKey(r.bucketAt)),
|
||||
rows.map((r) => r.iface),
|
||||
rows.map((r) => r.country),
|
||||
rows.map((r) => r.service),
|
||||
rows.map((r) => r.asn),
|
||||
rows.map((r) => r.bytes),
|
||||
rows.map((r) => r.packets),
|
||||
],
|
||||
})
|
||||
return rows.length
|
||||
}
|
||||
|
||||
export function factsPendingSize(): number {
|
||||
return hourFacts.size
|
||||
}
|
||||
|
||||
export function resetFactsForTests(): void {
|
||||
hourFacts.clear()
|
||||
ensuredParts.clear()
|
||||
}
|
||||
|
||||
export function discardPendingFacts(): void {
|
||||
hourFacts.clear()
|
||||
}
|
||||
|
||||
export function factsSnapshotForTests(): FactRow[] {
|
||||
const parsed: FactRow[] = []
|
||||
for (const [k, acc] of hourFacts) {
|
||||
const row = parseFactKey(k, acc)
|
||||
if (row) parsed.push(row)
|
||||
}
|
||||
return parsed
|
||||
}
|
||||
@@ -0,0 +1,186 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"
|
||||
import { tmpdir } from "node:os"
|
||||
import path from "node:path"
|
||||
import type { AsnResponse, CountryResponse, Reader } from "maxmind"
|
||||
import {
|
||||
disableRipeEnqueueForTests,
|
||||
disableRipePersistForTests,
|
||||
resetRipeCacheForTests,
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
import {
|
||||
lookupGeoip,
|
||||
resetGeoipForTests,
|
||||
resolveFlowIp,
|
||||
setGeoipReadersForTests,
|
||||
} from "./traffic-flow-geoip.js"
|
||||
import {
|
||||
resetEngineForTests,
|
||||
ingestParsedFlowsForServerForTests,
|
||||
minuteDimsSnapshotForTests,
|
||||
} from "./traffic-flow-engine.js"
|
||||
import { classifyFlowDst } from "./traffic-flow-classify.js"
|
||||
import { disableGeoipDbForTests } from "./geoip-settings.js"
|
||||
import {
|
||||
collectGeoipUpdateOnce,
|
||||
resetGeoipUpdateForTests,
|
||||
setGeoipFetchForTests,
|
||||
setGeoipValidateForTests,
|
||||
} from "./geoip-update-collector.js"
|
||||
|
||||
disableRipePersistForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetGeoipForTests()
|
||||
|
||||
// ── lookupGeoip: приватные IP → negative без ридеров ──────────────────────────
|
||||
assert.equal(lookupGeoip("10.1.1.8")?.ok, false)
|
||||
assert.equal(lookupGeoip("192.168.0.1")?.prefix, "192.168.0.1/32")
|
||||
assert.equal(lookupGeoip("100.64.1.2")?.ok, false)
|
||||
assert.equal(lookupGeoip("fe80::1")?.prefix, "fe80::1/128")
|
||||
|
||||
// ── без ридеров публичный IP → null, resolveFlowIp уходит в RIPE-кэш ─────────
|
||||
assert.equal(lookupGeoip("1.2.3.10"), null)
|
||||
seedRipeCacheForTests({
|
||||
prefix: "1.2.3.0/24",
|
||||
asn: 64500,
|
||||
country: "NL",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "TEST",
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.country, "NL")
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
|
||||
|
||||
// ── fake-ридеры: geoip приоритетнее RIPE ──────────────────────────────────────
|
||||
function fakeCountryReader(byIp: Record<string, string>): Reader<CountryResponse> {
|
||||
return {
|
||||
get(ip: string) {
|
||||
const iso = byIp[ip]
|
||||
return iso ? ({ country: { iso_code: iso } } as CountryResponse) : null
|
||||
},
|
||||
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
|
||||
} as unknown as Reader<CountryResponse>
|
||||
}
|
||||
|
||||
function fakeAsnReader(byIp: Record<string, { asn: number; org: string }>): Reader<AsnResponse> {
|
||||
return {
|
||||
get(ip: string) {
|
||||
const hit = byIp[ip]
|
||||
return hit
|
||||
? ({ autonomous_system_number: hit.asn, autonomous_system_organization: hit.org } as AsnResponse)
|
||||
: null
|
||||
},
|
||||
metadata: { buildEpoch: new Date("2026-09-02T00:00:00Z") },
|
||||
} as unknown as Reader<AsnResponse>
|
||||
}
|
||||
|
||||
setGeoipReadersForTests({
|
||||
country: fakeCountryReader({ "8.8.8.8": "US", "6.6.6.6": "EU" }),
|
||||
asn: fakeAsnReader({
|
||||
"8.8.8.8": { asn: 15169, org: "GOOGLE" },
|
||||
"6.6.6.6": { asn: 15169, org: "GOOGLE" },
|
||||
}),
|
||||
})
|
||||
|
||||
const hit = resolveFlowIp("8.8.8.8")
|
||||
assert.equal(hit?.country, "US")
|
||||
assert.equal(hit?.asn, 15169)
|
||||
assert.equal(hit?.holder, "GOOGLE")
|
||||
assert.equal(hit?.ok, true)
|
||||
|
||||
// 1.2.3.10 в fake-ридерах нет — по-прежнему из RIPE-кэша
|
||||
assert.equal(resolveFlowIp("1.2.3.10")?.asn, 64500)
|
||||
|
||||
// EU не ISO-страна: отфильтрована, страна выведена из ASN (HQ Google → US)
|
||||
assert.equal(lookupGeoip("6.6.6.6")?.country, "US")
|
||||
|
||||
// geoip-мета совместима с classifyFlowDst (бренд по ASN 15169)
|
||||
const classified = classifyFlowDst("8.8.8.8", 6, 443, 51504, hit)
|
||||
assert.equal(classified.service, "Google")
|
||||
|
||||
// ── движок: dims country/asn наполняются из geoip-ридеров ────────────────────
|
||||
resetEngineForTests()
|
||||
ingestParsedFlowsForServerForTests(1, [{
|
||||
src: "192.168.88.10",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51504,
|
||||
dstPort: 443,
|
||||
bytes: 1000,
|
||||
packets: 10,
|
||||
inIface: "wg-flow",
|
||||
outIface: "",
|
||||
nextHop: "",
|
||||
flowStartMs: 0,
|
||||
flowEndMs: 0,
|
||||
natSrc: "",
|
||||
natDst: "",
|
||||
}])
|
||||
const dims = minuteDimsSnapshotForTests()
|
||||
assert.equal(dims.get("country")?.get("US")?.bytes, 1000)
|
||||
assert.equal(dims.get("asn")?.get("15169")?.bytes, 1000)
|
||||
|
||||
// ── коллектор: 304 → обе базы без изменений ───────────────────────────────────
|
||||
disableGeoipDbForTests()
|
||||
resetGeoipUpdateForTests()
|
||||
const geoipDir = mkdtempSync(path.join(tmpdir(), "mm-geoip-test-"))
|
||||
process.env.GEOIP_DIR = geoipDir
|
||||
|
||||
setGeoipFetchForTests(async () => new Response(null, { status: 304 }))
|
||||
let snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.skippedUnchanged, 2)
|
||||
assert.equal(snap.downloaded, 0)
|
||||
assert.equal(existsSync(path.join(geoipDir, "GeoLite2-Country.mmdb")), false)
|
||||
|
||||
// ── коллектор: 200 + валидация ok → подмена, старый файл в .prev ─────────────
|
||||
const countryPath = path.join(geoipDir, "GeoLite2-Country.mmdb")
|
||||
const asnPath = path.join(geoipDir, "GeoLite2-ASN.mmdb")
|
||||
writeFileSync(countryPath, "old-country")
|
||||
|
||||
setGeoipFetchForTests(async () =>
|
||||
new Response(new Uint8Array([1, 2, 3]), { status: 200, headers: { etag: '"v1"' } }))
|
||||
setGeoipValidateForTests({
|
||||
country: async (p) => {
|
||||
assert.ok(p.endsWith(".tmp"), "валидация должна идти по tmp-файлу")
|
||||
return "2026-09-08T00:00:00.000Z"
|
||||
},
|
||||
asn: async () => "2026-09-08T00:00:00.000Z",
|
||||
})
|
||||
snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.downloaded, 2)
|
||||
assert.equal(snap.errors.length, 0)
|
||||
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]))
|
||||
assert.equal(readFileSync(`${countryPath}.prev`, "utf8"), "old-country")
|
||||
assert.equal(existsSync(`${asnPath}.prev`), false, "prev у asn не бывает при первой загрузке")
|
||||
assert.equal(existsSync(`${countryPath}.tmp`), false)
|
||||
|
||||
// ── коллектор: битая база → подмены нет, старый файл цел, tmp удалён ─────────
|
||||
writeFileSync(asnPath, "good-asn")
|
||||
setGeoipFetchForTests(async () =>
|
||||
new Response(new Uint8Array([9, 9]), { status: 200 }))
|
||||
setGeoipValidateForTests({
|
||||
country: async () => {
|
||||
throw new Error("битая база")
|
||||
},
|
||||
asn: async () => {
|
||||
throw new Error("битая база")
|
||||
},
|
||||
})
|
||||
snap = await collectGeoipUpdateOnce({ force: true })
|
||||
assert.equal(snap.downloaded, 0)
|
||||
assert.equal(snap.errors.length, 2)
|
||||
assert.deepEqual(readFileSync(countryPath), Buffer.from([1, 2, 3]), "country не тронута")
|
||||
assert.equal(readFileSync(asnPath, "utf8"), "good-asn", "asn не тронут")
|
||||
assert.equal(existsSync(`${countryPath}.tmp`), false)
|
||||
assert.equal(existsSync(`${asnPath}.tmp`), false)
|
||||
|
||||
rmSync(geoipDir, { recursive: true, force: true })
|
||||
delete process.env.GEOIP_DIR
|
||||
resetGeoipUpdateForTests()
|
||||
resetGeoipForTests()
|
||||
|
||||
console.log("traffic-flow-geoip.test.ts: ok")
|
||||
@@ -0,0 +1,162 @@
|
||||
import { existsSync } from "node:fs"
|
||||
import path from "node:path"
|
||||
import { open, type AsnResponse, type CountryResponse, type Reader } from "maxmind"
|
||||
import { isNonPublicIp } from "./traffic-flow-ip.js"
|
||||
import { isIsoCountry, resolveRipeCountry } from "./traffic-flow-brands.js"
|
||||
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
|
||||
export const GEOIP_COUNTRY_FILE = "GeoLite2-Country.mmdb"
|
||||
export const GEOIP_ASN_FILE = "GeoLite2-ASN.mmdb"
|
||||
|
||||
/** Каталог баз: `storage/geoip` рядом со storage/backups; переопределяется GEOIP_DIR. */
|
||||
export function geoipDir(): string {
|
||||
return path.resolve(process.env.GEOIP_DIR ?? path.join(process.cwd(), "storage", "geoip"))
|
||||
}
|
||||
|
||||
export function geoipCountryPath(): string {
|
||||
return path.join(geoipDir(), GEOIP_COUNTRY_FILE)
|
||||
}
|
||||
|
||||
export function geoipAsnPath(): string {
|
||||
return path.join(geoipDir(), GEOIP_ASN_FILE)
|
||||
}
|
||||
|
||||
export interface GeoipReaders {
|
||||
country: Reader<CountryResponse> | null
|
||||
asn: Reader<AsnResponse> | null
|
||||
}
|
||||
|
||||
let readers: GeoipReaders = { country: null, asn: null }
|
||||
let initPromise: Promise<GeoipReaders> | null = null
|
||||
|
||||
/** Открывает оба файла best-effort: отсутствующий/битый файл не мешает второму. */
|
||||
export async function openGeoipReaders(dir = geoipDir()): Promise<GeoipReaders> {
|
||||
const next: GeoipReaders = { country: null, asn: null }
|
||||
if (existsSync(path.join(dir, GEOIP_COUNTRY_FILE))) {
|
||||
try {
|
||||
next.country = await open<CountryResponse>(path.join(dir, GEOIP_COUNTRY_FILE))
|
||||
} catch {
|
||||
/* битый файл — работаем без country */
|
||||
}
|
||||
}
|
||||
if (existsSync(path.join(dir, GEOIP_ASN_FILE))) {
|
||||
try {
|
||||
next.asn = await open<AsnResponse>(path.join(dir, GEOIP_ASN_FILE))
|
||||
} catch {
|
||||
/* битый файл — работаем без ASN */
|
||||
}
|
||||
}
|
||||
return next
|
||||
}
|
||||
|
||||
/** Открывает ридеры при старте; файлы есть — работают, нет — lookup уходит в RIPE-fallback. */
|
||||
export async function initGeoip(): Promise<GeoipReaders> {
|
||||
if (!initPromise) {
|
||||
initPromise = openGeoipReaders().then((next) => {
|
||||
readers = next
|
||||
return next
|
||||
})
|
||||
}
|
||||
return initPromise
|
||||
}
|
||||
|
||||
/** Переоткрывает ридеры после обновления файлов (атомарная замена ссылок). */
|
||||
export async function reloadGeoipReaders(): Promise<GeoipReaders> {
|
||||
const next = await openGeoipReaders()
|
||||
readers = next
|
||||
initPromise = Promise.resolve(next)
|
||||
return next
|
||||
}
|
||||
|
||||
export function setGeoipReadersForTests(next: Partial<GeoipReaders>): void {
|
||||
readers = { country: next.country ?? null, asn: next.asn ?? null }
|
||||
}
|
||||
|
||||
export function resetGeoipForTests(): void {
|
||||
readers = { country: null, asn: null }
|
||||
initPromise = null
|
||||
}
|
||||
|
||||
function negativeMeta(ip: string): FlowIpMeta {
|
||||
const v6 = ip.includes(":")
|
||||
return {
|
||||
prefix: `${ip}/${v6 ? 128 : 32}`,
|
||||
asn: 0,
|
||||
country: "—",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder: "",
|
||||
ok: false,
|
||||
fetchedAt: Date.now(),
|
||||
}
|
||||
}
|
||||
|
||||
function safeCountryIso(reader: Reader<CountryResponse>, ip: string): string {
|
||||
try {
|
||||
const rec = reader.get(ip)
|
||||
return rec?.country?.iso_code ?? rec?.registered_country?.iso_code ?? ""
|
||||
} catch {
|
||||
return ""
|
||||
}
|
||||
}
|
||||
|
||||
function safeAsn(reader: Reader<AsnResponse>, ip: string): { asn: number; holder: string } {
|
||||
try {
|
||||
const rec = reader.get(ip)
|
||||
return {
|
||||
asn: rec?.autonomous_system_number ?? 0,
|
||||
holder: rec?.autonomous_system_organization ?? "",
|
||||
}
|
||||
} catch {
|
||||
return { asn: 0, holder: "" }
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Синхронный lookup по локальным GeoLite2. Возвращает FlowIpMeta в семантике RIPE-кэша
|
||||
* (ok=true когда есть страна или ASN; null — данных нет, пусть пробует RIPE).
|
||||
*/
|
||||
export function lookupGeoip(ip: string): FlowIpMeta | null {
|
||||
const trimmed = String(ip ?? "").trim()
|
||||
if (!trimmed) return null
|
||||
if (isNonPublicIp(trimmed)) return negativeMeta(trimmed)
|
||||
const { country: countryReader, asn: asnReader } = readers
|
||||
if (!countryReader && !asnReader) return null
|
||||
const iso = countryReader ? safeCountryIso(countryReader, trimmed) : ""
|
||||
const country = iso && isIsoCountry(iso) ? iso : ""
|
||||
const { asn, holder } = asnReader ? safeAsn(asnReader, trimmed) : { asn: 0, holder: "" }
|
||||
if (!asn && !country) return null
|
||||
return {
|
||||
prefix: `${trimmed}/${trimmed.includes(":") ? 128 : 32}`,
|
||||
asn,
|
||||
country: resolveRipeCountry(country, asn, holder) || "—",
|
||||
lat: null,
|
||||
lng: null,
|
||||
holder,
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
}
|
||||
}
|
||||
|
||||
/** Главный вход для потребителей пайплайна: локальные базы первыми, RIPE-кэш fallback. */
|
||||
export function resolveFlowIp(ip: string): FlowIpMeta | null {
|
||||
return lookupGeoip(ip) ?? lookupRipeCached(ip)
|
||||
}
|
||||
|
||||
export interface GeoipReadersStatus {
|
||||
countryLoaded: boolean
|
||||
asnLoaded: boolean
|
||||
countryBuildAt: string | null
|
||||
asnBuildAt: string | null
|
||||
dir: string
|
||||
}
|
||||
|
||||
export function geoipReadersStatus(): GeoipReadersStatus {
|
||||
return {
|
||||
countryLoaded: Boolean(readers.country),
|
||||
asnLoaded: Boolean(readers.asn),
|
||||
countryBuildAt: readers.country?.metadata.buildEpoch.toISOString() ?? null,
|
||||
asnBuildAt: readers.asn?.metadata.buildEpoch.toISOString() ?? null,
|
||||
dir: geoipDir(),
|
||||
}
|
||||
}
|
||||
@@ -22,8 +22,9 @@ rememberServerIfaces(7, [
|
||||
{ ".id": "*A", name: "wg-flow" },
|
||||
{ ".id": "*D", name: "bridge" },
|
||||
])
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "#2").name, "ether1")
|
||||
assert.equal(resolveIfaceName(7, "10").name, "wg-flow")
|
||||
assert.equal(resolveIfaceName(7, "13").name, "bridge")
|
||||
assert.equal(resolveIfaceName(7, "0").name, "—")
|
||||
assert.equal(resolveIfaceName(7, "ether1").name, "ether1")
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
import assert from "node:assert/strict"
|
||||
import {
|
||||
bindingIfaceAliases,
|
||||
bindingIfaceAliasesAllServers,
|
||||
canonicalFactIface,
|
||||
collapseServerIfaceRows,
|
||||
displayFactIface,
|
||||
expandBindingIfaces,
|
||||
factIfaceAliases,
|
||||
rememberServerIfaces,
|
||||
resetIfaceCacheForTests,
|
||||
resolveIfaceName,
|
||||
} from "./traffic-flow-ifindex.js"
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
assert.equal(canonicalFactIface(1, "2"), "2")
|
||||
assert.deepEqual(bindingIfaceAliases(1, "gre-client"), ["gre-client"])
|
||||
|
||||
rememberServerIfaces(1, [{ name: "gre-client", ifindex: "2" }])
|
||||
assert.equal(canonicalFactIface(1, "2"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "gre-client"), "gre-client")
|
||||
assert.equal(canonicalFactIface(1, "9"), "9")
|
||||
assert.equal(resolveIfaceName(1, "9").name, "#9")
|
||||
assert.equal(resolveIfaceName(1, "2").name, "gre-client")
|
||||
assert.equal(resolveIfaceName(1, "#2").name, "gre-client")
|
||||
assert.equal(displayFactIface(1, "2"), "gre-client")
|
||||
|
||||
const aliases = bindingIfaceAliases(1, "gre-client")
|
||||
assert.ok(aliases.includes("gre-client"))
|
||||
assert.ok(aliases.includes("2"))
|
||||
assert.ok(aliases.includes("#2"))
|
||||
|
||||
const fromIndex = factIfaceAliases("2", 1)
|
||||
assert.ok(fromIndex.includes("gre-client"))
|
||||
assert.ok(fromIndex.includes("2"))
|
||||
assert.ok(fromIndex.includes("#2"))
|
||||
|
||||
const all = bindingIfaceAliasesAllServers("gre-client")
|
||||
assert.ok(all.includes("2"))
|
||||
|
||||
const expanded = expandBindingIfaces([{ serverId: 1, iface: "gre-client" }])
|
||||
assert.ok(expanded.some((x) => x.iface === "2"))
|
||||
assert.ok(expanded.some((x) => x.iface === "gre-client"))
|
||||
|
||||
const collapsed = collapseServerIfaceRows([
|
||||
{ serverId: 1, iface: "2", bytes: 10, packets: 1 },
|
||||
{ serverId: 1, iface: "gre-client", bytes: 5, packets: 2 },
|
||||
{ serverId: 1, iface: "wan1", bytes: 3, packets: 1 },
|
||||
])
|
||||
assert.equal(collapsed.length, 2)
|
||||
const gre = collapsed.find((r) => r.iface === "gre-client")
|
||||
assert.ok(gre)
|
||||
assert.equal(gre.bytes, 15)
|
||||
assert.equal(gre.packets, 3)
|
||||
assert.ok(collapsed.some((r) => r.iface === "wan1"))
|
||||
|
||||
resetIfaceCacheForTests()
|
||||
console.log("traffic-flow-ifindex.test.ts: ok")
|
||||
@@ -36,12 +36,135 @@ export function rememberServerIfaces(serverId: number, rows: RosIfaceIndexRow[])
|
||||
|
||||
export function resolveIfaceName(serverId: number, indexOrName: string): { name: string; index: string } {
|
||||
const trimmed = String(indexOrName ?? "").trim()
|
||||
if (!trimmed || trimmed === "0") return { name: "—", index: trimmed }
|
||||
if (!/^\d+$/.test(trimmed)) return { name: trimmed, index: "" }
|
||||
const idx = Number(trimmed)
|
||||
const name = cache.get(serverId)?.get(idx)
|
||||
if (name) return { name, index: trimmed }
|
||||
return { name: `#${trimmed}`, index: trimmed }
|
||||
const asIndex = trimmed.startsWith("#") && /^\d+$/.test(trimmed.slice(1)) ? trimmed.slice(1) : trimmed
|
||||
if (!asIndex || asIndex === "0") return { name: "—", index: asIndex }
|
||||
if (!/^\d+$/.test(asIndex)) return { name: trimmed, index: "" }
|
||||
const name = cache.get(serverId)?.get(Number(asIndex))
|
||||
if (name) return { name, index: asIndex }
|
||||
return { name: `#${asIndex}`, index: asIndex }
|
||||
}
|
||||
|
||||
/** Имя iface для факта куба: ifIndex→имя, без `#13` при пустом кэше. */
|
||||
export function canonicalFactIface(serverId: number, inIface: string): string {
|
||||
const trimmed = String(inIface ?? "").trim()
|
||||
if (!trimmed) return trimmed
|
||||
if (!/^\d+$/.test(trimmed)) return trimmed
|
||||
const name = cache.get(serverId)?.get(Number(trimmed))
|
||||
return name || trimmed
|
||||
}
|
||||
|
||||
function numericIfaceIndex(iface: string): string | null {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (/^\d+$/.test(raw)) return raw
|
||||
if (raw.startsWith("#") && /^\d+$/.test(raw.slice(1))) return raw.slice(1)
|
||||
return null
|
||||
}
|
||||
|
||||
/** Имя для UI: ifIndex → RouterOS name; `0` → «—»; miss → `#n`. */
|
||||
export function displayFactIface(serverId: number, iface: string): string {
|
||||
return resolveIfaceName(serverId, iface).name
|
||||
}
|
||||
|
||||
/** Склеить факты `2` + `ether1` в одну строку после резолва ifIndex. */
|
||||
export function collapseServerIfaceRows(
|
||||
rows: Array<{ serverId: number; iface: string; bytes: number; packets: number }>,
|
||||
): Array<{ serverId: number; iface: string; bytes: number; packets: number }> {
|
||||
const acc = new Map<string, { serverId: number; iface: string; bytes: number; packets: number }>()
|
||||
for (const r of rows) {
|
||||
const name = displayFactIface(r.serverId, r.iface)
|
||||
const k = `${r.serverId}\0${name}`
|
||||
const prev = acc.get(k)
|
||||
const bytes = Number(r.bytes) || 0
|
||||
const packets = Number(r.packets) || 0
|
||||
if (prev) {
|
||||
prev.bytes += bytes
|
||||
prev.packets += packets
|
||||
} else {
|
||||
acc.set(k, { serverId: r.serverId, iface: name, bytes, packets })
|
||||
}
|
||||
}
|
||||
return [...acc.values()]
|
||||
}
|
||||
|
||||
/** Ключи факта для фильтра: имя, ifIndex и `#n`. */
|
||||
export function factIfaceAliases(iface: string, serverId?: number): string[] {
|
||||
const raw = String(iface ?? "").trim()
|
||||
if (!raw) return []
|
||||
const out = new Set<string>([raw])
|
||||
const idx = numericIfaceIndex(raw)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
const n = Number(idx)
|
||||
if (serverId != null) {
|
||||
const name = cache.get(serverId)?.get(n)
|
||||
if (name) out.add(name)
|
||||
} else {
|
||||
for (const map of cache.values()) {
|
||||
const name = map.get(n)
|
||||
if (name) out.add(name)
|
||||
}
|
||||
}
|
||||
}
|
||||
if (serverId != null) {
|
||||
for (const a of bindingIfaceAliases(serverId, raw)) out.add(a)
|
||||
} else {
|
||||
for (const a of bindingIfaceAliasesAllServers(raw)) out.add(a)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
/** Имя + ifIndex + `#n` — тот же матч, что карта `/traffic`. */
|
||||
export function bindingIfaceAliases(serverId: number, interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
if (!name) return []
|
||||
const out = new Set<string>([name])
|
||||
const map = cache.get(serverId)
|
||||
const idx = numericIfaceIndex(name)
|
||||
const canonical = (idx && map?.get(Number(idx))) || name
|
||||
out.add(canonical)
|
||||
if (idx) {
|
||||
out.add(idx)
|
||||
out.add(`#${idx}`)
|
||||
}
|
||||
if (!map) return [...out]
|
||||
for (const [i, n] of map) {
|
||||
if (n !== canonical && n !== name) continue
|
||||
out.add(String(i))
|
||||
out.add(`#${i}`)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function bindingIfaceAliasesAllServers(interfaceName: string): string[] {
|
||||
const name = String(interfaceName ?? "").trim()
|
||||
const out = new Set<string>(name ? [name] : [])
|
||||
for (const serverId of cache.keys()) {
|
||||
for (const alias of bindingIfaceAliases(serverId, name)) out.add(alias)
|
||||
}
|
||||
return [...out]
|
||||
}
|
||||
|
||||
export function expandBindingIfaces(
|
||||
binds: Array<{ serverId: number; iface: string }>,
|
||||
): Array<{ serverId: number; iface: string }> {
|
||||
const seen = new Set<string>()
|
||||
const out: Array<{ serverId: number; iface: string }> = []
|
||||
for (const b of binds) {
|
||||
for (const iface of bindingIfaceAliases(b.serverId, b.iface)) {
|
||||
const k = `${b.serverId}\0${iface}`
|
||||
if (seen.has(k)) continue
|
||||
seen.add(k)
|
||||
out.push({ serverId: b.serverId, iface })
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
|
||||
export function listCachedIfaceNames(serverId: number): string[] {
|
||||
const map = cache.get(serverId)
|
||||
if (!map) return []
|
||||
return [...new Set(map.values())]
|
||||
}
|
||||
|
||||
export function ifaceCacheHas(serverId: number): boolean {
|
||||
|
||||
@@ -21,7 +21,7 @@ import {
|
||||
setWantListenForTests,
|
||||
simulateWorkerExitForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { configureEngine, droppedForTests, pendingSizeForTests } from "./traffic-flow-engine.js"
|
||||
import { configureEngine, droppedForTests, getEngineStats, isValidFlowInet, pendingSizeForTests } from "./traffic-flow-engine.js"
|
||||
import { dbQuery } from "../db/index.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
|
||||
@@ -69,6 +69,27 @@ assert.equal(droppedForTests(), 3)
|
||||
assert.equal(peekPendingFlows().length, 3)
|
||||
setPendingCapForTests(null)
|
||||
|
||||
assert.equal(isValidFlowInet("10.0.0.1"), true)
|
||||
assert.equal(isValidFlowInet("8.8.8.8"), true)
|
||||
assert.equal(isValidFlowInet("0:0:0:0:0:0:0:1"), true)
|
||||
assert.equal(isValidFlowInet("not-an-ip"), false)
|
||||
assert.equal(isValidFlowInet("999.1.1.1"), false)
|
||||
|
||||
resetFlowRingsForTests()
|
||||
ingestParsedFlowsForServerForTests(1, [{
|
||||
src: "not-an-ip",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 1,
|
||||
dstPort: 443,
|
||||
bytes: 10,
|
||||
packets: 1,
|
||||
inIface: "2",
|
||||
outIface: "",
|
||||
}])
|
||||
await flushPendingForTests()
|
||||
assert.match(getEngineStats().lastError, /невалидн/)
|
||||
|
||||
resetFlowRingsForTests()
|
||||
configureEngine({ topN: 20 })
|
||||
const talkers = Array.from({ length: 25 }, (_, i) => ({
|
||||
|
||||
@@ -267,7 +267,7 @@ export async function startTrafficFlowListener() {
|
||||
export function stopTrafficFlowListener() {
|
||||
wantListen = false
|
||||
stopWorkerProcess()
|
||||
void flushPending().catch(() => { /* ignore */ })
|
||||
void flushPending({ force: true }).catch(() => { /* ignore */ })
|
||||
state = { bound: false, address: null }
|
||||
}
|
||||
|
||||
@@ -425,7 +425,7 @@ export async function ingestParsedFlowsForTests(exporterIp: string, flows: Parse
|
||||
if (serverId == null) return
|
||||
queueParsedFlows(serverId, flows)
|
||||
rollFlowRings()
|
||||
await flushPending()
|
||||
await flushPending({ force: true })
|
||||
}
|
||||
|
||||
export function ingestParsedFlowsForServerForTests(serverId: number, flows: ParsedFlowInput[]) {
|
||||
@@ -444,7 +444,7 @@ export function lastFlushUsedTransactionForTests(): boolean {
|
||||
}
|
||||
|
||||
export async function flushPendingForTests(): Promise<void> {
|
||||
await flushPending()
|
||||
await flushPending({ force: true })
|
||||
}
|
||||
|
||||
async function tableCount(name: string): Promise<number> {
|
||||
@@ -467,11 +467,15 @@ export async function purgeTrafficFlowStore(): Promise<FlowPurgeDto> {
|
||||
minuteStats: await tableCount("flow_minute_stats"),
|
||||
minuteDims: await tableCount("flow_minute_dims"),
|
||||
dailyDims: await tableCount("flow_daily_dims"),
|
||||
hourFacts: await tableCount("flow_hour_facts"),
|
||||
dailyFacts: await tableCount("flow_daily_facts"),
|
||||
}
|
||||
await dbQuery(`DELETE FROM flow_buckets`)
|
||||
await dbQuery(`DELETE FROM flow_minute_stats`)
|
||||
await dbQuery(`DELETE FROM flow_minute_dims`)
|
||||
await dbQuery(`DELETE FROM flow_daily_dims`)
|
||||
await dbQuery(`DELETE FROM flow_hour_facts`)
|
||||
await dbQuery(`DELETE FROM flow_daily_facts`)
|
||||
await resetFlowIngestCounters()
|
||||
await dropExpiredPartitions(pool)
|
||||
try {
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
import assert from "node:assert/strict"
|
||||
import { isNonPublicIp, pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { isNonPublicIp, pickInternetDest, pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
|
||||
assert.equal(isNonPublicIp("10.200.100.53"), true)
|
||||
assert.equal(isNonPublicIp("173.194.151.65"), false)
|
||||
@@ -22,4 +22,39 @@ assert.equal(
|
||||
)
|
||||
assert.equal(pickInternetPeer("10.1.1.1", "10.2.2.2", 443, 80), "10.2.2.2")
|
||||
|
||||
const rost = "95.167.1.10"
|
||||
const ours = new Set(["198.51.100.1", "203.0.113.10"])
|
||||
const client = { ours, boundClient: true }
|
||||
|
||||
assert.equal(
|
||||
pickInternetDest("10.200.100.53", "104.18.35.51", 53880, 443, client),
|
||||
"104.18.35.51",
|
||||
"RFC1918 → CF на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("173.194.151.65", "10.200.100.53", 443, 57182, client),
|
||||
"173.194.151.65",
|
||||
"Google:443 → RFC1918 на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "10.200.100.53", 51234, 443, client),
|
||||
"",
|
||||
"Rostelecom → overlay 10.x: не dest ASN клиента",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "8.8.8.8", 51234, 443, client),
|
||||
"8.8.8.8",
|
||||
"Rostelecom → Google:443 на client GRE",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest(rost, "1.1.1.1", 51234, 40000, client),
|
||||
"1.1.1.1",
|
||||
"оба публичные без well-known на client GRE → dst",
|
||||
)
|
||||
assert.equal(
|
||||
pickInternetDest("8.8.8.8", "198.51.100.1", 443, 51234, { ours }),
|
||||
"8.8.8.8",
|
||||
"ours как dst: dest = публичный src",
|
||||
)
|
||||
|
||||
console.log("traffic-flow-ip.test.ts: ok")
|
||||
|
||||
@@ -55,13 +55,46 @@ export function isNonPublicIp(ip: string): boolean {
|
||||
|
||||
const PEER_WELL_KNOWN_PORTS = new Set([80, 443, 53, 853])
|
||||
|
||||
export interface InternetDestCtx {
|
||||
/** WAN IP узлов сети (EN/JH) — не интернет-назначение. */
|
||||
ours?: ReadonlySet<string>
|
||||
/** Ingress с bound GRE/WG клиента: dest = нелокальный IP, не ASN клиента. */
|
||||
boundClient?: boolean
|
||||
}
|
||||
|
||||
export function isLocalIp(ip: string, ours?: ReadonlySet<string>): boolean {
|
||||
if (isNonPublicIp(ip)) return true
|
||||
return Boolean(ours?.has(String(ip ?? "").trim()))
|
||||
}
|
||||
|
||||
/**
|
||||
* Интернет-сторона потока: у IPFIX сервис часто в src (Google:443 → RFC1918:ephemeral).
|
||||
* Классифицировать этот IP, не слепой dst.
|
||||
* Интернет-назначение потока для ASN/страны/сервиса.
|
||||
* Пустая строка — dest нет (не GeoIP IP клиента).
|
||||
*/
|
||||
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
|
||||
const srcPub = !isNonPublicIp(src)
|
||||
const dstPub = !isNonPublicIp(dst)
|
||||
export function pickInternetDest(
|
||||
src: string,
|
||||
dst: string,
|
||||
srcPort: number,
|
||||
dstPort: number,
|
||||
ctx?: InternetDestCtx,
|
||||
): string {
|
||||
const ours = ctx?.ours
|
||||
const srcLocal = isLocalIp(src, ours)
|
||||
const dstLocal = isLocalIp(dst, ours)
|
||||
const srcPub = !srcLocal
|
||||
const dstPub = !dstLocal
|
||||
|
||||
if (ctx?.boundClient) {
|
||||
if (dstPub) return dst
|
||||
if (srcPub && dstLocal) {
|
||||
const srcWk = PEER_WELL_KNOWN_PORTS.has(srcPort)
|
||||
const dstWk = PEER_WELL_KNOWN_PORTS.has(dstPort)
|
||||
if (srcWk && !dstWk) return src
|
||||
return ""
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
if (srcPub && !dstPub) return src
|
||||
if (dstPub && !srcPub) return dst
|
||||
if (srcPub && dstPub) {
|
||||
@@ -72,3 +105,11 @@ export function pickInternetPeer(src: string, dst: string, srcPort: number, dstP
|
||||
}
|
||||
return dst
|
||||
}
|
||||
|
||||
/**
|
||||
* Интернет-сторона потока без топологии: у IPFIX сервис часто в src (Google:443 → RFC1918).
|
||||
* Для куба статистики используйте pickInternetDest.
|
||||
*/
|
||||
export function pickInternetPeer(src: string, dst: string, srcPort: number, dstPort: number): string {
|
||||
return pickInternetDest(src, dst, srcPort, dstPort) || dst
|
||||
}
|
||||
|
||||
@@ -4,7 +4,13 @@ import {
|
||||
ingestParsedFlowsForServerForTests,
|
||||
resetFlowRingsForTests,
|
||||
} from "./traffic-flow-ingest.js"
|
||||
import { buildFlowMapHops, resetFlowMapHopsCacheForTests } from "./traffic-flow-map-hops.js"
|
||||
import {
|
||||
buildFlowMapHops,
|
||||
MAP_SERVICE_MIN_NODES,
|
||||
MAP_SERVICE_NODE_CAP,
|
||||
pickMapServices,
|
||||
resetFlowMapHopsCacheForTests,
|
||||
} from "./traffic-flow-map-hops.js"
|
||||
import { withPgOrSkip } from "../test/pg.js"
|
||||
import { seedFlowTopologyForTests, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { disableCatalogFetchForTests, resetFlowCatalogForTests } from "./traffic-flow-classify.js"
|
||||
@@ -15,6 +21,55 @@ import {
|
||||
seedRipeCacheForTests,
|
||||
} from "./traffic-flow-ripe.js"
|
||||
|
||||
{
|
||||
const googleOnly = pickMapServices(
|
||||
[{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 1 }],
|
||||
5,
|
||||
)
|
||||
assert.equal(googleOnly.length, 1)
|
||||
assert.equal(googleOnly[0]?.share, 1)
|
||||
|
||||
const twoNamed = pickMapServices(
|
||||
[
|
||||
{ id: "svc:google", label: "Google", category: "Веб", bytes: 400, bps: 0, share: 0.5 },
|
||||
{ id: "svc:cloudflare", label: "Cloudflare", category: "CDN", bytes: 400, bps: 0, share: 0.5 },
|
||||
],
|
||||
5,
|
||||
)
|
||||
assert.equal(twoNamed.length, 2)
|
||||
|
||||
const tinyTail = pickMapServices(
|
||||
[
|
||||
{ id: "svc:google", label: "Google", category: "Веб", bytes: 9000, bps: 0, share: 0.9 },
|
||||
...Array.from({ length: 9 }, (_, i) => ({
|
||||
id: `svc:t${i}`,
|
||||
label: `T${i}`,
|
||||
category: "Веб",
|
||||
bytes: 100,
|
||||
bps: 0,
|
||||
share: 0.01,
|
||||
})),
|
||||
],
|
||||
5,
|
||||
)
|
||||
assert.equal(tinyTail.length, MAP_SERVICE_MIN_NODES)
|
||||
assert.equal(tinyTail.at(-1)?.id, "svc:t6")
|
||||
|
||||
const allOff = pickMapServices(
|
||||
Array.from({ length: 25 }, (_, i) => ({
|
||||
id: `svc:n${i}`,
|
||||
label: `N${i}`,
|
||||
category: "Веб",
|
||||
bytes: 25 - i,
|
||||
bps: 0,
|
||||
share: 0.04,
|
||||
})),
|
||||
0,
|
||||
)
|
||||
assert.equal(allOff.length, MAP_SERVICE_NODE_CAP)
|
||||
console.log("traffic-flow-map-hops.test.ts: pickMapServices ok")
|
||||
}
|
||||
|
||||
if (!(await withPgOrSkip())) {
|
||||
console.log("traffic-flow-map-hops.test.ts: skip")
|
||||
process.exit(0)
|
||||
@@ -178,6 +233,19 @@ function googleRipe() {
|
||||
})
|
||||
}
|
||||
|
||||
function seedRipeAsn(ip: string, asn: number, holder: string) {
|
||||
seedRipeCacheForTests({
|
||||
prefix: `${ip}/32`,
|
||||
asn,
|
||||
country: "US",
|
||||
lat: 37.4,
|
||||
lng: -122.1,
|
||||
holder,
|
||||
ok: true,
|
||||
fetchedAt: Date.now(),
|
||||
})
|
||||
}
|
||||
|
||||
function payloadFlow(dst: string, bytes: number) {
|
||||
return {
|
||||
src: "10.100.1.17",
|
||||
@@ -241,10 +309,106 @@ try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const four = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.equal(four.totalBytes, 10_000)
|
||||
assert.ok(!(four.services ?? []).some((s) => s.id === "svc:google"), "Google < 5% hidden")
|
||||
const googleFour = four.services?.find((s) => s.id === "svc:google")
|
||||
assert.ok(googleFour, "единственный бренд виден при 4% от окна")
|
||||
assert.ok(googleFour.share >= 0.99, "доля среди брендов ≈ 1")
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const off = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google 4%")
|
||||
assert.ok(off.services?.some((s) => s.id === "svc:google"), "порог 0 показывает Google")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 400),
|
||||
payloadFlow("104.18.35.51", 400),
|
||||
payloadFlow("203.0.113.50", 9200),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const two = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const googleTwo = two.services?.find((s) => s.id === "svc:google")
|
||||
const cfTwo = two.services?.find((s) => s.id === "svc:cloudflare")
|
||||
assert.ok(googleTwo, "Google среди брендов")
|
||||
assert.ok(cfTwo, "Cloudflare среди брендов")
|
||||
assert.ok(googleTwo.share >= 0.05)
|
||||
assert.ok(cfTwo.share >= 0.05)
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
seedRipeAsn("162.254.192.71", 32590, "VALVE-CORP")
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("162.254.192.71", 2000),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const steam = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
assert.ok(steam.services?.some((s) => s.id === "svc:steam"), "Steam AS32590 на карте")
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
}
|
||||
|
||||
const smallBrands: Array<{ ip: string; asn: number; holder: string; bytes: number; id: string }> = [
|
||||
{ ip: "203.0.113.1", asn: 714, holder: "APPLE-ENGINEERING", bytes: 400, id: "svc:apple" },
|
||||
{ ip: "203.0.113.2", asn: 36459, holder: "GITHUB", bytes: 390, id: "svc:github" },
|
||||
{ ip: "203.0.113.3", asn: 54876, holder: "GITLAB", bytes: 380, id: "svc:gitlab" },
|
||||
{ ip: "203.0.113.4", asn: 8403, holder: "SPOTIFY", bytes: 370, id: "svc:spotify" },
|
||||
{ ip: "203.0.113.5", asn: 13414, holder: "TWITTER", bytes: 360, id: "svc:x" },
|
||||
{ ip: "203.0.113.6", asn: 47541, holder: "VKONTAKTE", bytes: 350, id: "svc:vk" },
|
||||
{ ip: "203.0.113.7", asn: 30103, holder: "ZOOM", bytes: 340, id: "svc:zoom" },
|
||||
{ ip: "203.0.113.8", asn: 395701, holder: "EPIC-GAMES", bytes: 330, id: "svc:epic" },
|
||||
{ ip: "203.0.113.9", asn: 6507, holder: "RIOT-GAMES", bytes: 320, id: "svc:riot" },
|
||||
]
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
for (const b of smallBrands) seedRipeAsn(b.ip, b.asn, b.holder)
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 5000),
|
||||
...smallBrands.map((b) => payloadFlow(b.ip, b.bytes)),
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const top = await buildFlowMapHops({ minutes: 5, minSharePct: 5 })
|
||||
const ids = new Set((top.services ?? []).map((s) => s.id))
|
||||
assert.equal(top.services?.length, MAP_SERVICE_MIN_NODES, "топ-8 брендов на карте")
|
||||
assert.ok(ids.has("svc:google"))
|
||||
for (const b of smallBrands.slice(0, 7)) assert.ok(ids.has(b.id), b.id)
|
||||
assert.ok(!ids.has("svc:epic"), "хвост ниже ранга 8 скрыт")
|
||||
assert.ok(!ids.has("svc:riot"))
|
||||
} finally {
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
@@ -420,4 +584,98 @@ try {
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
rememberServerIfaces(9, [
|
||||
{ ".id": "*1", name: "ether1" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [payloadFlow("8.8.8.8", 12_000)])
|
||||
ingestParsedFlowsForServerForTests(9, [{
|
||||
src: "10.100.1.17",
|
||||
dst: "8.8.8.8",
|
||||
proto: 6,
|
||||
srcPort: 51234,
|
||||
dstPort: 443,
|
||||
bytes: 12_000,
|
||||
packets: 10,
|
||||
inIface: "1",
|
||||
outIface: "1",
|
||||
nextHop: "",
|
||||
}])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const dual = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = dual.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE JH→EN сохранён")
|
||||
assert.equal(gre.bytes, 12_000)
|
||||
const googlePaths = (dual.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "один путь без копии EN")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.equal(googlePaths[0]?.viaId, "7")
|
||||
const googleEdge = dual.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
disableRipeEnqueueForTests()
|
||||
seedFlowTopologyForTests(topo)
|
||||
rememberServerIfaces(7, [
|
||||
{ ".id": "*2", name: "gre-client" },
|
||||
{ ".id": "*3", name: "gre-jh-en" },
|
||||
])
|
||||
googleRipe()
|
||||
ingestParsedFlowsForServerForTests(7, [
|
||||
payloadFlow("8.8.8.8", 8_000),
|
||||
{
|
||||
src: "8.8.8.8",
|
||||
dst: "10.100.1.17",
|
||||
proto: 6,
|
||||
srcPort: 443,
|
||||
dstPort: 51234,
|
||||
bytes: 4_000,
|
||||
packets: 8,
|
||||
inIface: "3",
|
||||
outIface: "2",
|
||||
nextHop: "",
|
||||
},
|
||||
])
|
||||
try {
|
||||
resetFlowMapHopsCacheForTests()
|
||||
const bothDir = await buildFlowMapHops({ minutes: 5, minSharePct: 0 })
|
||||
const gre = bothDir.hops.find((h) => h.kind === "gre" && h.fromId === "7" && h.toId === "9")
|
||||
assert.ok(gre, "GRE-hop при fwd/rev")
|
||||
const googlePaths = (bothDir.servicePaths ?? []).filter((p) => p.serviceId === "svc:google")
|
||||
assert.equal(googlePaths.length, 1, "fwd+rev — один клиент")
|
||||
assert.equal(googlePaths[0]?.clientId, "u1")
|
||||
assert.ok(!(bothDir.servicePaths ?? []).some((p) => p.serviceId === "svc:google" && p.clientId === "—"))
|
||||
const googleEdge = bothDir.serviceEdges?.find((e) => e.toId === "svc:google" && e.fromId === "9")
|
||||
assert.ok(googleEdge)
|
||||
assert.equal(googleEdge.bytes, 12_000)
|
||||
assert.equal(googleEdge.bps, googlePaths[0]?.bps)
|
||||
} finally {
|
||||
seedFlowTopologyForTests(null)
|
||||
resetFlowRingsForTests()
|
||||
resetIfaceCacheForTests()
|
||||
resetRipeCacheForTests()
|
||||
resetFlowCatalogForTests()
|
||||
}
|
||||
|
||||
console.log("traffic-flow-map-hops.test.ts: ok")
|
||||
|
||||
@@ -5,21 +5,25 @@ import { userInterfaceBindings } from "../db/schema.js"
|
||||
import { applicationName, flowRowMatchesFilter } from "./traffic-flow-apps.js"
|
||||
import {
|
||||
isNamedInternetService,
|
||||
lookupBrand,
|
||||
mapServiceNodeId,
|
||||
resolveFlowBrand,
|
||||
} from "./traffic-flow-brands.js"
|
||||
import { dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { dedupFlowRowsAcrossExporters, dedupFlowRowsMaxBytes } from "./traffic-flow-dedup.js"
|
||||
import { getFlowListenerState, listFlowRowsForWindow } from "./traffic-flow-ingest.js"
|
||||
import { resolveIfaceName } from "./traffic-flow-ifaces.js"
|
||||
import { classifyFlowPlane, shouldKeepPlane } from "./traffic-flow-planes.js"
|
||||
import { pickInternetPeer } from "./traffic-flow-ip.js"
|
||||
import { lookupRipeCached, type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { destCtxForIface } from "./traffic-flow-dest.js"
|
||||
import { pickInternetDest } from "./traffic-flow-ip.js"
|
||||
import { type FlowIpMeta } from "./traffic-flow-ripe.js"
|
||||
import { resolveFlowIp } from "./traffic-flow-geoip.js"
|
||||
import { getTrafficFlowSettingsRow } from "./traffic-flow-settings.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog } from "./traffic-flow-topology.js"
|
||||
import { loadFlowTopology, resolveClient, resolveEn, getServerCatalog, type FlowTopology } from "./traffic-flow-topology.js"
|
||||
import { flowDataEpoch } from "./traffic-flow-engine.js"
|
||||
|
||||
export const DEFAULT_MAP_SERVICE_MIN_SHARE_PCT = 5
|
||||
export const MAP_SERVICE_NODE_CAP = 20
|
||||
/** Минимум узлов-брендов на карте, даже если доля ниже порога. */
|
||||
export const MAP_SERVICE_MIN_NODES = 8
|
||||
const HOPS_CACHE_TTL_MS = 2000
|
||||
|
||||
export interface FlowMapHopsQuery {
|
||||
@@ -99,6 +103,15 @@ export function clampMapServiceMinSharePct(n: unknown): number {
|
||||
return Math.min(100, Math.max(0, v))
|
||||
}
|
||||
|
||||
/** Доля среди именованных брендов; порог ИЛИ топ-N, затем cap. */
|
||||
export function pickMapServices(ranked: FlowMapService[], minSharePct: number): FlowMapService[] {
|
||||
if (minSharePct <= 0) return ranked.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
const minShare = minSharePct / 100
|
||||
return ranked
|
||||
.filter((s, i) => s.share >= minShare || i < MAP_SERVICE_MIN_NODES)
|
||||
.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
}
|
||||
|
||||
function hopsQueryKey(q: FlowMapHopsQuery, minSharePct: number): string {
|
||||
return JSON.stringify({
|
||||
epoch: flowDataEpoch(),
|
||||
@@ -129,6 +142,16 @@ function ifaceUsable(name: string): boolean {
|
||||
return Boolean(name) && name !== "—"
|
||||
}
|
||||
|
||||
function resolveMapClient(
|
||||
topo: FlowTopology,
|
||||
serverId: number,
|
||||
inName: string,
|
||||
outName: string,
|
||||
) {
|
||||
return resolveClient(topo, serverId, inName)
|
||||
?? (ifaceUsable(outName) ? resolveClient(topo, serverId, outName) : null)
|
||||
}
|
||||
|
||||
function bump(acc: Map<string, HopAcc>, key: string, seed: Omit<HopAcc, "bytes" | "bytesFwd" | "bytesRev">, bytes: number, dir: "fwd" | "rev" | "both"): void {
|
||||
const prev = acc.get(key)
|
||||
const addFwd = dir === "fwd" || dir === "both" ? bytes : 0
|
||||
@@ -174,10 +197,7 @@ function classifyMapDstLite(
|
||||
if (proto === 47 || proto === 50) return null
|
||||
const app = applicationName(proto, dstPort, srcPort)
|
||||
if (app === "WireGuard" || app === "DNS" || app === "SSH" || app === "BGP") return null
|
||||
if (/youtube/i.test(ripe?.holder ?? "")) {
|
||||
return { service: "YouTube", category: "Видео / стриминг" }
|
||||
}
|
||||
const brand = lookupBrand(dst, ripe?.asn ?? 0)
|
||||
const brand = resolveFlowBrand(dst, ripe?.asn ?? 0, ripe?.holder ?? "", proto, dstPort, srcPort)
|
||||
if (!brand || !isNamedInternetService(brand.service, brand.category)) return null
|
||||
return brand
|
||||
}
|
||||
@@ -229,6 +249,21 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const enIds = new Set(topo.enNodes.map((n) => n.id))
|
||||
let totalBytes = 0
|
||||
|
||||
function rowClient(r: (typeof working)[number]) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
return resolveMapClient(topo, r.serverId, inName, outName)
|
||||
}
|
||||
|
||||
const payloadRows = wantDedup
|
||||
? dedupFlowRowsAcrossExporters(working, (a, b) => {
|
||||
const aCli = Boolean(rowClient(a))
|
||||
const bCli = Boolean(rowClient(b))
|
||||
if (aCli !== bCli) return aCli ? a : b
|
||||
return a.bytes >= b.bytes ? a : b
|
||||
})
|
||||
: working
|
||||
|
||||
for (const r of working) {
|
||||
const inRes = resolveIfaceName(r.serverId, r.inIface)
|
||||
const outRes = resolveIfaceName(r.serverId, r.outIface)
|
||||
@@ -314,11 +349,22 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}, r.bytes, "fwd")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
for (const r of payloadRows) {
|
||||
const inName = resolveIfaceName(r.serverId, r.inIface).name
|
||||
const outName = resolveIfaceName(r.serverId, r.outIface).name
|
||||
totalBytes += r.bytes
|
||||
const peer = pickInternetPeer(r.src, r.dst, r.srcPort, r.dstPort)
|
||||
const client = resolveClient(topo, r.serverId, inName)
|
||||
const prevDst = dstAcc.get(peer)
|
||||
const dest = pickInternetDest(
|
||||
r.src,
|
||||
r.dst,
|
||||
r.srcPort,
|
||||
r.dstPort,
|
||||
destCtxForIface(topo, r.serverId, inName),
|
||||
)
|
||||
if (!dest) continue
|
||||
const client = resolveMapClient(topo, r.serverId, inName, outName)
|
||||
const prevDst = dstAcc.get(dest)
|
||||
if (prevDst) {
|
||||
prevDst.bytes += r.bytes
|
||||
bumpFrom(prevDst, String(r.serverId), r.bytes, client)
|
||||
@@ -331,7 +377,7 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
fromBytes: new Map(),
|
||||
}
|
||||
bumpFrom(acc, String(r.serverId), r.bytes, client)
|
||||
dstAcc.set(peer, acc)
|
||||
dstAcc.set(dest, acc)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -386,7 +432,7 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}
|
||||
|
||||
for (const [dst, acc] of dstAcc) {
|
||||
const ripe = lookupRipeCached(dst)
|
||||
const ripe = resolveFlowIp(dst)
|
||||
const classified = classifyMapDstLite(dst, acc.proto, acc.dstPort, acc.srcPort, ripe)
|
||||
if (!classified) continue
|
||||
const toId = mapServiceNodeId(classified.service)
|
||||
@@ -419,10 +465,15 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
const enName = nodeName(fromId)
|
||||
const viaName = nodeName(exporterId)
|
||||
for (const [clientId, c] of from.clients) {
|
||||
const pathKey = `${clientId}|${exporterId}|${fromId}|${toId}`
|
||||
const pathKey = `${clientId}|${fromId}|${toId}`
|
||||
const prevPath = svcPaths.get(pathKey)
|
||||
if (prevPath) {
|
||||
prevPath.bytes += c.bytes
|
||||
if (exporterId !== fromId && prevPath.viaId === fromId) {
|
||||
prevPath.viaId = exporterId
|
||||
prevPath.viaName = viaName
|
||||
}
|
||||
if (prevPath.clientName === "—" && c.name !== "—") prevPath.clientName = c.name
|
||||
} else {
|
||||
svcPaths.set(pathKey, {
|
||||
clientId,
|
||||
@@ -439,21 +490,20 @@ async function buildFlowMapHopsUncached(q: FlowMapHopsQuery, minSharePct: number
|
||||
}
|
||||
}
|
||||
|
||||
const minShare = minSharePct / 100
|
||||
let services: FlowMapService[] = [...svcTotals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: totalBytes > 0 ? s.bytes / totalBytes : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes)
|
||||
if (minSharePct > 0) {
|
||||
services = services.filter((s) => s.share >= minShare)
|
||||
}
|
||||
services = services.slice(0, MAP_SERVICE_NODE_CAP)
|
||||
const namedBytes = [...svcTotals.values()].reduce((n, s) => n + s.bytes, 0)
|
||||
const services = pickMapServices(
|
||||
[...svcTotals.entries()]
|
||||
.map(([id, s]) => ({
|
||||
id,
|
||||
label: s.label,
|
||||
category: s.category,
|
||||
bytes: s.bytes,
|
||||
bps: (s.bytes * 8) / windowSec,
|
||||
share: namedBytes > 0 ? s.bytes / namedBytes : 0,
|
||||
}))
|
||||
.sort((a, b) => b.bytes - a.bytes),
|
||||
minSharePct,
|
||||
)
|
||||
const keepSvc = new Set(services.map((s) => s.id))
|
||||
const serviceEdges: FlowMapServiceEdge[] = [...svcEdges.values()]
|
||||
.filter((e) => keepSvc.has(e.toId))
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import { db, dbAll } from "../db/index.js"
|
||||
import { parseJsonArray } from "../db/json.js"
|
||||
import { appUsers, servers, userInterfaceBindings } from "../db/schema.js"
|
||||
import { mapRosInterfaceType } from "../modules/users/iface-type.js"
|
||||
import { mapRosInterfaceType, parseRawInterfaces } from "../modules/users/iface-type.js"
|
||||
import type { PlaneTopology } from "./traffic-flow-planes.js"
|
||||
|
||||
export interface FlowClientBinding {
|
||||
@@ -25,6 +25,8 @@ export interface FlowTopology {
|
||||
enHosts: Set<string>
|
||||
jhHosts: Set<string>
|
||||
wanIfaces: Map<number, Set<string>>
|
||||
/** GRE/WG из последнего снимка RouterOS (`type`), без mgmt. */
|
||||
tunnelIfaces?: Map<number, Set<string>>
|
||||
plane: PlaneTopology
|
||||
}
|
||||
|
||||
@@ -48,6 +50,15 @@ export function invalidateFlowCatalogCache(): void {
|
||||
serverCatalogCache = null
|
||||
}
|
||||
|
||||
export function peekFlowTopology(): FlowTopology | null {
|
||||
if (seeded) return seeded
|
||||
return topologyCache?.topo ?? null
|
||||
}
|
||||
|
||||
export function peekServerCatalog(): { list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> } | null {
|
||||
return serverCatalogCache
|
||||
}
|
||||
|
||||
export async function getServerCatalog(): Promise<{ list: ServerCatalogEntry[]; byId: Map<number, ServerCatalogEntry> }> {
|
||||
const now = Date.now()
|
||||
if (serverCatalogCache && now - serverCatalogCache.at < CATALOG_TTL_MS) {
|
||||
@@ -76,6 +87,25 @@ function ifaceKey(serverId: number, name: string): string {
|
||||
return `${serverId}|${name}`
|
||||
}
|
||||
|
||||
async function loadTunnelIfacesFromSnapshots(): Promise<Map<number, Set<string>>> {
|
||||
const rows = await dbAll<{ serverId: number; rawInterfaces: unknown }>(`
|
||||
SELECT DISTINCT ON (server_id) server_id AS "serverId", raw_interfaces AS "rawInterfaces"
|
||||
FROM server_snapshots
|
||||
ORDER BY server_id, polled_at DESC
|
||||
`)
|
||||
const map = new Map<number, Set<string>>()
|
||||
for (const r of rows) {
|
||||
const set = new Set<string>()
|
||||
for (const iface of parseRawInterfaces(r.rawInterfaces)) {
|
||||
if (iface.type !== "gre" && iface.type !== "wg") continue
|
||||
if (iface.name.toLowerCase() === "wg-flow") continue
|
||||
set.add(iface.name)
|
||||
}
|
||||
if (set.size) map.set(r.serverId, set)
|
||||
}
|
||||
return map
|
||||
}
|
||||
|
||||
export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
if (seeded) return seeded
|
||||
const now = Date.now()
|
||||
@@ -118,6 +148,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
for (const h of hosts) jhHosts.add(h)
|
||||
}
|
||||
}
|
||||
const tunnelIfaces = await loadTunnelIfacesFromSnapshots()
|
||||
const topo: FlowTopology = {
|
||||
clientIfaces,
|
||||
clientByIface,
|
||||
@@ -125,6 +156,7 @@ export async function loadFlowTopology(): Promise<FlowTopology> {
|
||||
enHosts,
|
||||
jhHosts,
|
||||
wanIfaces,
|
||||
tunnelIfaces,
|
||||
plane: {
|
||||
clientIfaceNames: allClientNames,
|
||||
enHosts,
|
||||
@@ -140,6 +172,18 @@ export function seedFlowTopologyForTests(topo: FlowTopology | null): void {
|
||||
invalidateFlowCatalogCache()
|
||||
}
|
||||
|
||||
export function flowOursHosts(topo: FlowTopology | null | undefined): Set<string> {
|
||||
const ours = new Set<string>()
|
||||
if (!topo) return ours
|
||||
for (const h of topo.enHosts) {
|
||||
if (h) ours.add(h)
|
||||
}
|
||||
for (const h of topo.jhHosts) {
|
||||
if (h) ours.add(h)
|
||||
}
|
||||
return ours
|
||||
}
|
||||
|
||||
export function resolveClient(
|
||||
topo: FlowTopology,
|
||||
serverId: number,
|
||||
@@ -168,10 +212,14 @@ export function resolveEn(
|
||||
|
||||
export function enGreIfaceNames(topo: FlowTopology, serverId: number, ifaceNames: string[]): string[] {
|
||||
const client = topo.clientIfaces.get(serverId) ?? new Set<string>()
|
||||
const wan = topo.wanIfaces.get(serverId) ?? new Set<string>()
|
||||
const typed = topo.tunnelIfaces?.get(serverId)
|
||||
return ifaceNames.filter((name) => {
|
||||
if (client.has(name)) return false
|
||||
if (client.has(name) || wan.has(name)) return false
|
||||
if (name === "wg-flow") return false
|
||||
return mapRosInterfaceType("", name) === "gre"
|
||||
if (typed && typed.size > 0) return typed.has(name)
|
||||
const t = mapRosInterfaceType("", name)
|
||||
return t === "gre" || t === "wg"
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { and, asc, eq, gte, inArray, lt, max, or } from "drizzle-orm"
|
||||
import { and, asc, eq, gte, inArray, max, or } from "drizzle-orm"
|
||||
import { db } from "../db/index.js"
|
||||
import {
|
||||
servers,
|
||||
@@ -63,12 +63,6 @@ export async function getSettings() {
|
||||
return (await db.select().from(uptimeSettings).where(eq(uptimeSettings.id, 1)).limit(1))[0]
|
||||
}
|
||||
|
||||
async function cleanup(retentionDays: number) {
|
||||
const cutoff = new Date(Date.now() - retentionDays * 24 * 60 * 60 * 1000).toISOString()
|
||||
await db.delete(uptimeProbeSamples).where(lt(uptimeProbeSamples.sampledAt, cutoff))
|
||||
await db.delete(uptimeResourceSamples).where(lt(uptimeResourceSamples.sampledAt, cutoff))
|
||||
}
|
||||
|
||||
export function parsePing(results: Array<{ time?: string; status?: string; sent?: string; received?: string; "packet-loss"?: string; "avg-rtt"?: string }>) {
|
||||
const sum = [...results].reverse().find((r) =>
|
||||
r.sent != null || r.received != null || r["packet-loss"] != null || r["avg-rtt"] != null,
|
||||
@@ -166,8 +160,6 @@ export async function collectResourceSamplesOnce(): Promise<ResourcesRunSnapshot
|
||||
freeHddSpace: freeHdd,
|
||||
totalHddSpace: totalHdd,
|
||||
uptimeSeconds,
|
||||
boardName: String(resource["board-name"] ?? ""),
|
||||
rosVersion: String(resource["version"] ?? ""),
|
||||
})
|
||||
const memUsedMb = totalMem > 0 ? Math.round((totalMem - freeMem) / (1024 * 1024)) : 0
|
||||
const memTotalMb = totalMem > 0 ? Math.round(totalMem / (1024 * 1024)) : 0
|
||||
@@ -198,8 +190,6 @@ export async function collectResourceSamplesOnce(): Promise<ResourcesRunSnapshot
|
||||
freeHddSpace: 0,
|
||||
totalHddSpace: 0,
|
||||
uptimeSeconds: 0,
|
||||
boardName: "",
|
||||
rosVersion: "",
|
||||
})
|
||||
snapshot.servers.push({
|
||||
serverId: s.id,
|
||||
@@ -211,7 +201,6 @@ export async function collectResourceSamplesOnce(): Promise<ResourcesRunSnapshot
|
||||
}
|
||||
}
|
||||
|
||||
await cleanup(Math.max(1, settings.retentionDays))
|
||||
await db.update(uptimeSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - started,
|
||||
@@ -324,7 +313,6 @@ export async function collectPingProbesOnce(): Promise<PingRunSnapshot> {
|
||||
}
|
||||
}
|
||||
|
||||
await cleanup(Math.max(1, settings.retentionDays))
|
||||
await db.update(uptimeSettings).set({
|
||||
lastCollectedAt: now,
|
||||
lastDurationMs: Date.now() - started,
|
||||
@@ -399,7 +387,6 @@ export async function collectPingForProbeIds(probeIds: string[]): Promise<{ poll
|
||||
polled += 1
|
||||
}
|
||||
|
||||
await cleanup(Math.max(1, settings.retentionDays))
|
||||
return { polled }
|
||||
} finally {
|
||||
collectingPing = false
|
||||
|
||||
@@ -226,6 +226,19 @@ export interface BackupsRunSnapshot {
|
||||
fatalError?: string
|
||||
}
|
||||
|
||||
export interface GeoipUpdateRunSnapshot {
|
||||
v: typeof SCHEDULER_RUN_SNAPSHOT_VERSION
|
||||
job: "geoip_update"
|
||||
sampledAt: string
|
||||
skipped?: boolean
|
||||
fatalError?: string
|
||||
checked: number
|
||||
downloaded: number
|
||||
skippedUnchanged: number
|
||||
bytes: number
|
||||
errors: string[]
|
||||
}
|
||||
|
||||
export type SchedulerRunSnapshot =
|
||||
| TrafficRunSnapshot
|
||||
| ResourcesRunSnapshot
|
||||
@@ -236,4 +249,5 @@ export type SchedulerRunSnapshot =
|
||||
| InternetPathRunSnapshot
|
||||
| CertificatesRenewRunSnapshot
|
||||
| BackupsRunSnapshot
|
||||
| GeoipUpdateRunSnapshot
|
||||
| AlertEngineRunSnapshot
|
||||
|
||||
@@ -13,6 +13,7 @@ import {
|
||||
import {
|
||||
LayoutDashboardIcon,
|
||||
ActivityIcon,
|
||||
ChartColumnIcon,
|
||||
MapIcon,
|
||||
HeartPulseIcon,
|
||||
GlobeIcon,
|
||||
@@ -54,6 +55,7 @@ const navStructure: { label: string; items: NavItemBase[] }[] = [
|
||||
items: [
|
||||
{ title: "Дашборд", url: "/dashboard", icon: <LayoutDashboardIcon /> },
|
||||
{ title: "Трафик", url: "/traffic", icon: <ActivityIcon /> },
|
||||
{ title: "Статистика", url: "/statistics", icon: <ChartColumnIcon /> },
|
||||
{ title: "Карта сети", url: "/network-map", icon: <MapIcon /> },
|
||||
{ title: "Мониторинг", url: "/uptime", icon: <HeartPulseIcon /> },
|
||||
],
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"use client"
|
||||
|
||||
import {
|
||||
AlertDialog,
|
||||
AlertDialogAction,
|
||||
AlertDialogCancel,
|
||||
AlertDialogContent,
|
||||
AlertDialogDescription,
|
||||
AlertDialogFooter,
|
||||
AlertDialogHeader,
|
||||
AlertDialogMedia,
|
||||
AlertDialogTitle,
|
||||
} from "@/components/ui/alert-dialog"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import type { Backup } from "@/lib/data"
|
||||
import { AlertCircleIcon, LoaderCircleIcon, TriangleAlertIcon } from "lucide-react"
|
||||
|
||||
export function BackupDeleteDialog({
|
||||
backup,
|
||||
busy,
|
||||
onConfirm,
|
||||
onCancel,
|
||||
}: {
|
||||
backup: Backup | null
|
||||
busy: boolean
|
||||
onConfirm: () => void
|
||||
onCancel: () => void
|
||||
}) {
|
||||
return (
|
||||
<AlertDialog open={Boolean(backup)} onOpenChange={(v) => { if (!v && !busy) onCancel() }}>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogMedia className="bg-destructive/10 text-destructive">
|
||||
<AlertCircleIcon />
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Удалить бэкап?</AlertDialogTitle>
|
||||
<AlertDialogDescription>
|
||||
Файл <span className="font-mono text-foreground">{backup?.filename}</span> будет удалён
|
||||
{backup?.storage === "s3" || backup?.storage === "both" ? " локально и из S3" : " с диска"}.
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel disabled={busy} onClick={onCancel}>Отмена</AlertDialogCancel>
|
||||
<AlertDialogAction variant="destructive" disabled={busy} onClick={onConfirm}>
|
||||
{busy ? <LoaderCircleIcon className="size-4 animate-spin" /> : null}
|
||||
Удалить
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
)
|
||||
}
|
||||
|
||||
export function BackupRestoreDialog({
|
||||
backup,
|
||||
busy,
|
||||
onConfirm,
|
||||
onCancel,
|
||||
}: {
|
||||
backup: Backup | null
|
||||
busy: boolean
|
||||
onConfirm: () => void
|
||||
onCancel: () => void
|
||||
}) {
|
||||
return (
|
||||
<AlertDialog open={Boolean(backup)} onOpenChange={(v) => { if (!v && !busy) onCancel() }}>
|
||||
<AlertDialogContent>
|
||||
<AlertDialogHeader>
|
||||
<AlertDialogMedia className="bg-warning/10 text-warning">
|
||||
<TriangleAlertIcon />
|
||||
</AlertDialogMedia>
|
||||
<AlertDialogTitle>Восстановить конфигурацию?</AlertDialogTitle>
|
||||
<AlertDialogDescription className="flex flex-col gap-3">
|
||||
<span>
|
||||
Файл <span className="font-mono text-foreground">{backup?.filename}</span> будет загружен
|
||||
на <span className="text-foreground">{backup?.server}</span> и импортирован.
|
||||
</span>
|
||||
<Alert variant="warning">
|
||||
<TriangleAlertIcon />
|
||||
<AlertTitle>Это изменит рабочую конфигурацию роутера</AlertTitle>
|
||||
<AlertDescription>
|
||||
Сессия может оборваться. Убедитесь, что выбран именно этот сервер и этот снимок.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
</AlertDialogDescription>
|
||||
</AlertDialogHeader>
|
||||
<AlertDialogFooter>
|
||||
<AlertDialogCancel disabled={busy} onClick={onCancel}>Отмена</AlertDialogCancel>
|
||||
<AlertDialogAction disabled={busy} onClick={onConfirm}>
|
||||
{busy ? <LoaderCircleIcon className="size-4 animate-spin" /> : null}
|
||||
Восстановить
|
||||
</AlertDialogAction>
|
||||
</AlertDialogFooter>
|
||||
</AlertDialogContent>
|
||||
</AlertDialog>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,191 @@
|
||||
"use client"
|
||||
|
||||
import type { Server } from "@/lib/data"
|
||||
import { FormField } from "@/components/form-kit"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Checkbox } from "@/components/ui/checkbox"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import {
|
||||
Sheet,
|
||||
SheetContent,
|
||||
SheetHeader,
|
||||
SheetTitle,
|
||||
SheetDescription,
|
||||
SheetFooter,
|
||||
SheetClose,
|
||||
} from "@/components/ui/sheet"
|
||||
import {
|
||||
Stepper,
|
||||
StepperContent,
|
||||
StepperIndicator,
|
||||
StepperItem,
|
||||
StepperNav,
|
||||
StepperPanel,
|
||||
StepperSeparator,
|
||||
StepperTitle,
|
||||
StepperTrigger,
|
||||
} from "@/components/reui/stepper"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
export function BackupCreateSheet({
|
||||
open,
|
||||
onOpenChange,
|
||||
step,
|
||||
onStepChange,
|
||||
servers,
|
||||
selected,
|
||||
onToggle,
|
||||
onSelectAll,
|
||||
onClear,
|
||||
notes,
|
||||
onNotesChange,
|
||||
destinationLabel,
|
||||
busy,
|
||||
onSubmit,
|
||||
}: {
|
||||
open: boolean
|
||||
onOpenChange: (open: boolean) => void
|
||||
step: number
|
||||
onStepChange: (step: number) => void
|
||||
servers: Server[]
|
||||
selected: Set<string>
|
||||
onToggle: (id: string) => void
|
||||
onSelectAll: () => void
|
||||
onClear: () => void
|
||||
notes: string
|
||||
onNotesChange: (value: string) => void
|
||||
destinationLabel: string
|
||||
busy: boolean
|
||||
onSubmit: () => void
|
||||
}) {
|
||||
return (
|
||||
<Sheet open={open} onOpenChange={(v) => { onOpenChange(v); if (!v) onStepChange(1) }}>
|
||||
<SheetContent side="right" className="w-full sm:max-w-md flex flex-col gap-0 p-0">
|
||||
<SheetHeader className="px-6 pt-6 pb-4 border-b shrink-0">
|
||||
<SheetTitle>Новый бэкап</SheetTitle>
|
||||
<SheetDescription>Снять конфигурацию вручную с выбранных серверов</SheetDescription>
|
||||
</SheetHeader>
|
||||
|
||||
<Stepper value={step} onValueChange={onStepChange} className="flex-1 flex flex-col min-h-0 px-6 py-5">
|
||||
<StepperNav className="mb-5">
|
||||
<StepperItem step={1}>
|
||||
<StepperTrigger>
|
||||
<StepperIndicator>1</StepperIndicator>
|
||||
<StepperTitle className="sr-only">Серверы</StepperTitle>
|
||||
</StepperTrigger>
|
||||
<StepperSeparator />
|
||||
</StepperItem>
|
||||
<StepperItem step={2}>
|
||||
<StepperTrigger>
|
||||
<StepperIndicator>2</StepperIndicator>
|
||||
<StepperTitle className="sr-only">Заметка</StepperTitle>
|
||||
</StepperTrigger>
|
||||
<StepperSeparator />
|
||||
</StepperItem>
|
||||
<StepperItem step={3}>
|
||||
<StepperTrigger>
|
||||
<StepperIndicator>3</StepperIndicator>
|
||||
<StepperTitle className="sr-only">Подтверждение</StepperTitle>
|
||||
</StepperTrigger>
|
||||
</StepperItem>
|
||||
</StepperNav>
|
||||
<StepperPanel className="flex-1 overflow-y-auto">
|
||||
<StepperContent value={1} className="flex flex-col gap-3">
|
||||
<div className="flex items-center justify-between mb-1">
|
||||
<p className="text-sm font-medium">Выберите серверы</p>
|
||||
<div className="flex items-center gap-2">
|
||||
<button type="button" onClick={onSelectAll} className="text-xs text-primary hover:underline">
|
||||
Все
|
||||
</button>
|
||||
<span className="text-border">·</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClear}
|
||||
className="text-xs text-muted-foreground hover:text-foreground hover:underline"
|
||||
>
|
||||
Сбросить
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
{servers.map((s) => {
|
||||
const checked = selected.has(s.id)
|
||||
return (
|
||||
<label
|
||||
key={s.id}
|
||||
className={cn(
|
||||
"flex cursor-pointer items-center gap-3 rounded-lg border p-3 text-left transition-colors",
|
||||
checked ? "border-primary/40 bg-primary/5" : "border-border hover:bg-muted/40",
|
||||
)}
|
||||
>
|
||||
<Checkbox
|
||||
checked={checked}
|
||||
onCheckedChange={() => onToggle(s.id)}
|
||||
aria-label={`Выбрать ${s.name}`}
|
||||
/>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="text-sm font-medium">{s.name}</p>
|
||||
<div className="flex items-center gap-1.5 mt-0.5">
|
||||
<span className="text-xs font-mono text-muted-foreground">{s.host}</span>
|
||||
<StatusBadge status={s.status} />
|
||||
</div>
|
||||
</div>
|
||||
{s.status === "offline" && (
|
||||
<span className="text-xs text-muted-foreground">недоступен</span>
|
||||
)}
|
||||
</label>
|
||||
)
|
||||
})}
|
||||
</StepperContent>
|
||||
<StepperContent value={2} className="flex flex-col gap-4">
|
||||
<FormField label="Заметка">
|
||||
<Input
|
||||
placeholder="Например: перед обновлением BGP"
|
||||
value={notes}
|
||||
onChange={(e) => onNotesChange(e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
</StepperContent>
|
||||
<StepperContent value={3} className="flex flex-col gap-3 text-sm">
|
||||
<p className="text-muted-foreground">
|
||||
Будет создан бэкап для <strong className="text-foreground">{selected.size}</strong> серверов.
|
||||
</p>
|
||||
<p className="text-muted-foreground">
|
||||
Куда сохранится: <strong className="text-foreground">{destinationLabel}</strong>
|
||||
</p>
|
||||
{notes ? <p className="text-muted-foreground">Заметка: {notes}</p> : null}
|
||||
</StepperContent>
|
||||
</StepperPanel>
|
||||
</Stepper>
|
||||
|
||||
<SheetFooter className="px-6 py-4 border-t shrink-0 flex-row gap-2">
|
||||
<SheetClose render={<Button variant="outline" className="flex-1" />}>Отмена</SheetClose>
|
||||
{step > 1 && (
|
||||
<Button type="button" variant="outline" className="flex-1" onClick={() => onStepChange(step - 1)}>
|
||||
Назад
|
||||
</Button>
|
||||
)}
|
||||
{step < 3 ? (
|
||||
<Button
|
||||
type="button"
|
||||
className="flex-1"
|
||||
disabled={step === 1 && selected.size === 0}
|
||||
onClick={() => onStepChange(step + 1)}
|
||||
>
|
||||
Далее
|
||||
</Button>
|
||||
) : (
|
||||
<Button
|
||||
type="button"
|
||||
className="flex-1"
|
||||
disabled={selected.size === 0 || busy}
|
||||
onClick={onSubmit}
|
||||
>
|
||||
Снять бэкап ({selected.size})
|
||||
</Button>
|
||||
)}
|
||||
</SheetFooter>
|
||||
</SheetContent>
|
||||
</Sheet>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,97 @@
|
||||
"use client"
|
||||
|
||||
import type { Filter } from "@/components/reui/filters"
|
||||
import type { Backup } from "@/lib/data"
|
||||
import { applyReuiFilters } from "@/lib/data-filters/apply-reui-filters"
|
||||
import {
|
||||
BACKUP_FILTER_ACCESSORS,
|
||||
BACKUP_FILTER_FIELDS,
|
||||
} from "@/lib/data-filters/backup-filter-fields"
|
||||
import { DataPageCard } from "@/components/data-page-card"
|
||||
import { DataPageToolbar } from "@/components/data-page-toolbar"
|
||||
import { BackupsDataGrid } from "@/components/data-grids/backups-data-grid"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { PlusIcon } from "lucide-react"
|
||||
|
||||
type KindFilter = "all" | "auto" | "manual"
|
||||
|
||||
export function BackupsHistory({
|
||||
backups,
|
||||
kindFilter,
|
||||
onKindFilterChange,
|
||||
search,
|
||||
onSearchChange,
|
||||
filters,
|
||||
onFiltersChange,
|
||||
onDownload,
|
||||
onRestore,
|
||||
onDelete,
|
||||
onCreate,
|
||||
}: {
|
||||
backups: Backup[]
|
||||
kindFilter: KindFilter
|
||||
onKindFilterChange: (value: KindFilter) => void
|
||||
search: string
|
||||
onSearchChange: (value: string) => void
|
||||
filters: Filter[]
|
||||
onFiltersChange: (filters: Filter[]) => void
|
||||
onDownload: (id: string, filename: string) => void
|
||||
onRestore: (backup: Backup) => void
|
||||
onDelete: (backup: Backup) => void
|
||||
onCreate: () => void
|
||||
}) {
|
||||
const autoCount = backups.filter((b) => b.kind === "auto").length
|
||||
const manualCount = backups.filter((b) => b.kind === "manual").length
|
||||
const byKind = kindFilter === "all" ? backups : backups.filter((b) => b.kind === kindFilter)
|
||||
const q = search.trim().toLowerCase()
|
||||
const searched = q
|
||||
? byKind.filter((b) =>
|
||||
[b.filename, b.server, b.notes].some((v) => v.toLowerCase().includes(q)),
|
||||
)
|
||||
: byKind
|
||||
const filtered = applyReuiFilters(searched, filters, BACKUP_FILTER_ACCESSORS)
|
||||
const isEmptyAll = backups.length === 0
|
||||
|
||||
return (
|
||||
<DataPageCard>
|
||||
<DataPageToolbar
|
||||
segmented={{
|
||||
value: kindFilter,
|
||||
onChange: onKindFilterChange,
|
||||
options: [
|
||||
{ value: "all", label: "Все", count: backups.length },
|
||||
{ value: "auto", label: "Авто", count: autoCount },
|
||||
{ value: "manual", label: "Вручную", count: manualCount },
|
||||
],
|
||||
}}
|
||||
filters={filters}
|
||||
onFiltersChange={onFiltersChange}
|
||||
filterFields={BACKUP_FILTER_FIELDS}
|
||||
search={search}
|
||||
onSearchChange={onSearchChange}
|
||||
searchPlaceholder="Поиск по файлу, серверу, заметке…"
|
||||
countLabel={`${filtered.length} бэкапов`}
|
||||
/>
|
||||
<BackupsDataGrid
|
||||
backups={filtered}
|
||||
onDownload={onDownload}
|
||||
onRestore={onRestore}
|
||||
onDelete={onDelete}
|
||||
emptyTitle={isEmptyAll ? "Нет бэкапов" : "Ничего не найдено"}
|
||||
emptyDescription={
|
||||
isEmptyAll
|
||||
? "Создайте первый бэкап вручную или настройте расписание"
|
||||
: "Измените фильтры или поисковый запрос"
|
||||
}
|
||||
emptyAction={
|
||||
isEmptyAll ? (
|
||||
<Button type="button" size="sm" onClick={onCreate}>
|
||||
<PlusIcon className="size-4" />
|
||||
Новый бэкап
|
||||
</Button>
|
||||
) : null
|
||||
}
|
||||
/>
|
||||
</DataPageCard>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,419 @@
|
||||
"use client"
|
||||
|
||||
import type { Server } from "@/lib/data"
|
||||
import type { BackupStorageSettingsDto } from "@mmapp/contracts/backups"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { FormField, FormToggle, SegmentedControl } from "@/components/form-kit"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Checkbox } from "@/components/ui/checkbox"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { LoaderCircleIcon } from "lucide-react"
|
||||
|
||||
export const WEEK_DAYS = ["Пн", "Вт", "Ср", "Чт", "Пт", "Сб", "Вс"]
|
||||
|
||||
export type BackupFreq = "daily" | "weekly" | "monthly"
|
||||
export type StorageProvider = "local" | "s3"
|
||||
|
||||
export type BackupScheduleForm = {
|
||||
enabled: boolean
|
||||
frequency: BackupFreq
|
||||
hour: number
|
||||
minute: number
|
||||
weekDay: number
|
||||
monthDay: number
|
||||
keepCount: number
|
||||
format: "rsc" | "backup"
|
||||
}
|
||||
|
||||
export type BackupStorageForm = {
|
||||
provider: StorageProvider
|
||||
s3Endpoint: string
|
||||
s3Region: string
|
||||
s3Bucket: string
|
||||
s3Prefix: string
|
||||
s3AccessKeyId: string
|
||||
s3SecretAccessKey: string
|
||||
s3ForcePathStyle: boolean
|
||||
keepLocalCopy: boolean
|
||||
showPassword: boolean
|
||||
}
|
||||
|
||||
export const defaultSchedule: BackupScheduleForm = {
|
||||
enabled: true,
|
||||
frequency: "daily",
|
||||
hour: 3,
|
||||
minute: 0,
|
||||
weekDay: 0,
|
||||
monthDay: 1,
|
||||
keepCount: 7,
|
||||
format: "rsc",
|
||||
}
|
||||
|
||||
export const defaultStorageForm: BackupStorageForm = {
|
||||
provider: "local",
|
||||
s3Endpoint: "",
|
||||
s3Region: "us-east-1",
|
||||
s3Bucket: "",
|
||||
s3Prefix: "mikrotik",
|
||||
s3AccessKeyId: "",
|
||||
s3SecretAccessKey: "",
|
||||
s3ForcePathStyle: true,
|
||||
keepLocalCopy: true,
|
||||
showPassword: false,
|
||||
}
|
||||
|
||||
function storageStatus(saved: BackupStorageSettingsDto | null, form: BackupStorageForm) {
|
||||
if (form.provider === "local") {
|
||||
return { label: "Локально", variant: "secondary" as const }
|
||||
}
|
||||
if (saved?.lastTestError) {
|
||||
return { label: "Ошибка", variant: "destructive-light" as const }
|
||||
}
|
||||
if (saved?.lastTestAt && !saved.lastTestError) {
|
||||
return { label: "Connected", variant: "success-light" as const }
|
||||
}
|
||||
if (saved?.secretConfigured && saved.s3Bucket) {
|
||||
return { label: "Не проверено", variant: "warning-light" as const }
|
||||
}
|
||||
return { label: "Не настроено", variant: "secondary" as const }
|
||||
}
|
||||
|
||||
export function BackupsSettings({
|
||||
schedule,
|
||||
onScheduleChange,
|
||||
storage,
|
||||
onStorageChange,
|
||||
savedStorage,
|
||||
servers,
|
||||
selectedServers,
|
||||
onToggleServer,
|
||||
onSelectAll,
|
||||
onClearServers,
|
||||
onSave,
|
||||
onTest,
|
||||
onSync,
|
||||
saveBusy,
|
||||
testBusy,
|
||||
syncBusy,
|
||||
}: {
|
||||
schedule: BackupScheduleForm
|
||||
onScheduleChange: <K extends keyof BackupScheduleForm>(k: K, v: BackupScheduleForm[K]) => void
|
||||
storage: BackupStorageForm
|
||||
onStorageChange: <K extends keyof BackupStorageForm>(k: K, v: BackupStorageForm[K]) => void
|
||||
savedStorage: BackupStorageSettingsDto | null
|
||||
servers: Server[]
|
||||
selectedServers: Set<string>
|
||||
onToggleServer: (id: string) => void
|
||||
onSelectAll: () => void
|
||||
onClearServers: () => void
|
||||
onSave: () => void
|
||||
onTest: () => void
|
||||
onSync: () => void
|
||||
saveBusy: boolean
|
||||
testBusy: boolean
|
||||
syncBusy: boolean
|
||||
}) {
|
||||
const status = storageStatus(savedStorage, storage)
|
||||
const secretPlaceholder = savedStorage?.secretConfigured ? "•••••••• (сохранён)" : "••••••••"
|
||||
|
||||
return (
|
||||
<div className="grid grid-cols-1 lg:grid-cols-2 gap-5">
|
||||
<OpsPanel title="Расписание" description="Автоматический съём конфигурации" contentClassName="px-5 py-5 flex flex-col gap-5">
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Автоматический бэкап</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">Создавать бэкапы по расписанию</p>
|
||||
</div>
|
||||
<FormToggle checked={schedule.enabled} onChange={(v) => onScheduleChange("enabled", v)} />
|
||||
</div>
|
||||
|
||||
<div className={cn("flex flex-col gap-4", !schedule.enabled && "opacity-40 pointer-events-none")}>
|
||||
<FormField label="Частота">
|
||||
<SegmentedControl
|
||||
value={schedule.frequency}
|
||||
onChange={(v) => onScheduleChange("frequency", v)}
|
||||
options={[
|
||||
{ value: "daily", label: "Ежедневно" },
|
||||
{ value: "weekly", label: "Еженедельно" },
|
||||
{ value: "monthly", label: "Ежемесячно" },
|
||||
]}
|
||||
/>
|
||||
</FormField>
|
||||
|
||||
{schedule.frequency === "weekly" && (
|
||||
<FormField label="День недели">
|
||||
<div className="flex gap-1">
|
||||
{WEEK_DAYS.map((d, i) => (
|
||||
<button
|
||||
key={d}
|
||||
type="button"
|
||||
onClick={() => onScheduleChange("weekDay", i)}
|
||||
className={cn(
|
||||
"w-9 h-9 rounded text-sm font-medium border transition-colors",
|
||||
schedule.weekDay === i
|
||||
? "bg-primary text-primary-foreground border-primary"
|
||||
: "border-border text-muted-foreground hover:text-foreground",
|
||||
)}
|
||||
>
|
||||
{d}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</FormField>
|
||||
)}
|
||||
|
||||
{schedule.frequency === "monthly" && (
|
||||
<FormField label="День месяца" hint="1–28">
|
||||
<Input
|
||||
type="number"
|
||||
min={1}
|
||||
max={28}
|
||||
className="font-mono w-24"
|
||||
value={schedule.monthDay}
|
||||
onChange={(e) => onScheduleChange("monthDay", Math.min(28, Math.max(1, Number(e.target.value))))}
|
||||
/>
|
||||
</FormField>
|
||||
)}
|
||||
|
||||
<FormField label="Время запуска">
|
||||
<div className="flex items-center gap-2">
|
||||
<Input
|
||||
type="number"
|
||||
min={0}
|
||||
max={23}
|
||||
className="font-mono w-20 text-center"
|
||||
value={String(schedule.hour).padStart(2, "0")}
|
||||
onChange={(e) => onScheduleChange("hour", Math.min(23, Math.max(0, Number(e.target.value))))}
|
||||
/>
|
||||
<span className="text-muted-foreground font-mono text-lg">:</span>
|
||||
<div className="flex gap-1">
|
||||
{[0, 15, 30, 45].map((m) => (
|
||||
<button
|
||||
key={m}
|
||||
type="button"
|
||||
onClick={() => onScheduleChange("minute", m)}
|
||||
className={cn(
|
||||
"px-2.5 py-1.5 rounded text-xs font-mono border transition-colors",
|
||||
schedule.minute === m
|
||||
? "bg-primary text-primary-foreground border-primary"
|
||||
: "border-border text-muted-foreground hover:text-foreground",
|
||||
)}
|
||||
>
|
||||
{String(m).padStart(2, "0")}
|
||||
</button>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</FormField>
|
||||
|
||||
<FormField label="Хранить бэкапов" hint="На каждый сервер">
|
||||
<Input
|
||||
type="number"
|
||||
min={1}
|
||||
max={90}
|
||||
className="font-mono w-24"
|
||||
value={schedule.keepCount}
|
||||
onChange={(e) => onScheduleChange("keepCount", Math.max(1, Number(e.target.value)))}
|
||||
/>
|
||||
</FormField>
|
||||
|
||||
<div className="flex items-center justify-between gap-3 rounded-lg border border-border px-3 py-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Формат файла</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">Снимается текстовый экспорт RouterOS</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-2">
|
||||
<Badge variant="secondary" size="sm">.rsc</Badge>
|
||||
<Badge variant="warning-light" size="sm">.backup скоро</Badge>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</OpsPanel>
|
||||
|
||||
<OpsPanel
|
||||
title="Хранилище"
|
||||
description="Локальный диск приложения или S3-compatible бакет"
|
||||
headerRight={
|
||||
<Badge variant={status.variant} size="sm" radius="full">
|
||||
{status.label}
|
||||
</Badge>
|
||||
}
|
||||
contentClassName="px-5 py-5 flex flex-col gap-5"
|
||||
>
|
||||
<FormField label="Тип хранилища">
|
||||
<SegmentedControl
|
||||
value={storage.provider}
|
||||
onChange={(v) => onStorageChange("provider", v)}
|
||||
options={[
|
||||
{ value: "local", label: "Локально" },
|
||||
{ value: "s3", label: "S3" },
|
||||
]}
|
||||
/>
|
||||
</FormField>
|
||||
|
||||
{storage.provider === "local" ? (
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Файлы пишутся в каталог приложения <span className="font-mono text-foreground">storage/backups</span>.
|
||||
</p>
|
||||
) : (
|
||||
<div className="flex flex-col gap-4">
|
||||
<FormField label="Endpoint" hint="Пусто для AWS. Для R2/MinIO/Selectel — полный URL">
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="https://s3.amazonaws.com"
|
||||
value={storage.s3Endpoint}
|
||||
onChange={(e) => onStorageChange("s3Endpoint", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<FormField label="Region">
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="us-east-1"
|
||||
value={storage.s3Region}
|
||||
onChange={(e) => onStorageChange("s3Region", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="Bucket" required>
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="mikrotik-backups"
|
||||
value={storage.s3Bucket}
|
||||
onChange={(e) => onStorageChange("s3Bucket", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
</div>
|
||||
<FormField label="Prefix">
|
||||
<Input
|
||||
className="font-mono"
|
||||
placeholder="mikrotik"
|
||||
value={storage.s3Prefix}
|
||||
onChange={(e) => onStorageChange("s3Prefix", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<div className="grid grid-cols-2 gap-3">
|
||||
<FormField label="Access key" required>
|
||||
<Input
|
||||
className="font-mono"
|
||||
autoComplete="off"
|
||||
value={storage.s3AccessKeyId}
|
||||
onChange={(e) => onStorageChange("s3AccessKeyId", e.target.value)}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="Secret key">
|
||||
<div className="relative">
|
||||
<Input
|
||||
type={storage.showPassword ? "text" : "password"}
|
||||
className="font-mono pr-14"
|
||||
autoComplete="new-password"
|
||||
placeholder={secretPlaceholder}
|
||||
value={storage.s3SecretAccessKey}
|
||||
onChange={(e) => onStorageChange("s3SecretAccessKey", e.target.value)}
|
||||
/>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => onStorageChange("showPassword", !storage.showPassword)}
|
||||
className="absolute right-2.5 top-1/2 -translate-y-1/2 text-xs text-muted-foreground hover:text-foreground"
|
||||
>
|
||||
{storage.showPassword ? "скрыть" : "показ"}
|
||||
</button>
|
||||
</div>
|
||||
</FormField>
|
||||
</div>
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Path-style</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">Нужен для MinIO и части совместимых API</p>
|
||||
</div>
|
||||
<FormToggle checked={storage.s3ForcePathStyle} onChange={(v) => onStorageChange("s3ForcePathStyle", v)} />
|
||||
</div>
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Оставлять локальную копию</p>
|
||||
<p className="text-xs text-muted-foreground mt-0.5">После успешной загрузки в S3</p>
|
||||
</div>
|
||||
<FormToggle checked={storage.keepLocalCopy} onChange={(v) => onStorageChange("keepLocalCopy", v)} />
|
||||
</div>
|
||||
{savedStorage?.lastTestError ? (
|
||||
<p className="text-xs text-destructive">{savedStorage.lastTestError}</p>
|
||||
) : null}
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<Button type="button" variant="outline" size="sm" onClick={onTest} disabled={testBusy}>
|
||||
{testBusy ? <LoaderCircleIcon className="size-3.5 animate-spin" /> : null}
|
||||
Проверить
|
||||
</Button>
|
||||
<Button type="button" variant="outline" size="sm" onClick={onSync} disabled={syncBusy}>
|
||||
{syncBusy ? <LoaderCircleIcon className="size-3.5 animate-spin" /> : null}
|
||||
Синхронизировать из бакета
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</OpsPanel>
|
||||
|
||||
<OpsPanel
|
||||
className="lg:col-span-2"
|
||||
title="Серверы для бэкапа"
|
||||
headerRight={
|
||||
<div className="flex items-center gap-2 shrink-0">
|
||||
<button type="button" onClick={onSelectAll} className="text-xs text-primary hover:underline">
|
||||
Выбрать все
|
||||
</button>
|
||||
<span className="text-border">·</span>
|
||||
<button
|
||||
type="button"
|
||||
onClick={onClearServers}
|
||||
className="text-xs text-muted-foreground hover:text-foreground hover:underline"
|
||||
>
|
||||
Сбросить
|
||||
</button>
|
||||
</div>
|
||||
}
|
||||
contentClassName="px-5 py-5 flex flex-col gap-4"
|
||||
>
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-3 xl:grid-cols-4 gap-2">
|
||||
{servers.map((s) => {
|
||||
const checked = selectedServers.has(s.id)
|
||||
return (
|
||||
<label
|
||||
key={s.id}
|
||||
className={cn(
|
||||
"flex cursor-pointer items-center gap-3 rounded-lg border p-3 text-left transition-colors",
|
||||
checked
|
||||
? "border-primary/40 bg-primary/5"
|
||||
: "border-border hover:border-border/80 hover:bg-muted/40",
|
||||
)}
|
||||
>
|
||||
<Checkbox
|
||||
checked={checked}
|
||||
onCheckedChange={() => onToggleServer(s.id)}
|
||||
aria-label={`Выбрать ${s.name}`}
|
||||
/>
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="text-sm font-medium truncate">{s.name}</p>
|
||||
<div className="flex items-center gap-1.5 mt-0.5">
|
||||
<span className="text-xs text-muted-foreground">{s.site}</span>
|
||||
<StatusBadge status={s.status} />
|
||||
</div>
|
||||
</div>
|
||||
</label>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Выбрано {selectedServers.size} из {servers.length} серверов
|
||||
</p>
|
||||
</OpsPanel>
|
||||
|
||||
<div className="lg:col-span-2 flex items-center gap-3">
|
||||
<Button type="button" onClick={onSave} className="gap-2" disabled={saveBusy}>
|
||||
{saveBusy ? <LoaderCircleIcon className="size-4 animate-spin" /> : null}
|
||||
Сохранить настройки
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,184 @@
|
||||
"use client"
|
||||
|
||||
import { useCallback, useEffect, useState } from "react"
|
||||
import Link from "next/link"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { FormField, FormToggle } from "@/components/form-kit"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { Button, buttonVariants } from "@/components/ui/button"
|
||||
import { Input } from "@/components/ui/input"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
getCertificateRenewSettings,
|
||||
putCertificateRenewSettings,
|
||||
} from "@/shared/api/certificates"
|
||||
import { toast } from "sonner"
|
||||
|
||||
/**
|
||||
* Автообновление сертификатов через MM (ACME DNS-01).
|
||||
* Preview: https://reui.io/preview/base/settings-16 · https://reui.io/preview/base/settings-3
|
||||
* Docs: https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/alert · https://reui.io/docs/components/base/badge
|
||||
*/
|
||||
export function CertificateRenewSettingsPanel({
|
||||
backendUrl,
|
||||
liveReady,
|
||||
}: {
|
||||
backendUrl: string
|
||||
liveReady: boolean
|
||||
}) {
|
||||
const [enabled, setEnabled] = useState(true)
|
||||
const [intervalDraft, setIntervalDraft] = useState("21600")
|
||||
const [daysDraft, setDaysDraft] = useState("30")
|
||||
const [lastCollectedAt, setLastCollectedAt] = useState<string | null>(null)
|
||||
const [lastError, setLastError] = useState<string | null>(null)
|
||||
const [loaded, setLoaded] = useState(false)
|
||||
const [toggleBusy, setToggleBusy] = useState(false)
|
||||
const [saveBusy, setSaveBusy] = useState(false)
|
||||
|
||||
const load = useCallback(async () => {
|
||||
if (!liveReady) return
|
||||
try {
|
||||
const s = await getCertificateRenewSettings(backendUrl)
|
||||
setEnabled(s.enabled)
|
||||
setIntervalDraft(String(s.intervalSec))
|
||||
setDaysDraft(String(s.renewBeforeDays))
|
||||
setLastCollectedAt(s.lastCollectedAt ?? null)
|
||||
setLastError(s.lastError ?? null)
|
||||
setLoaded(true)
|
||||
} catch (e) {
|
||||
setLoaded(true)
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось загрузить настройки автообновления")
|
||||
}
|
||||
}, [backendUrl, liveReady])
|
||||
|
||||
useEffect(() => {
|
||||
queueMicrotask(() => {
|
||||
void load()
|
||||
})
|
||||
}, [load])
|
||||
|
||||
async function handleEnabledChange(next: boolean) {
|
||||
if (!liveReady || toggleBusy) return
|
||||
const prev = enabled
|
||||
setEnabled(next)
|
||||
setToggleBusy(true)
|
||||
try {
|
||||
const saved = await putCertificateRenewSettings(backendUrl, { enabled: next })
|
||||
setEnabled(saved.enabled)
|
||||
toast.success(next ? "Автообновление через MikrotikManager включено" : "Автообновление через MikrotikManager выключено")
|
||||
} catch (e) {
|
||||
setEnabled(prev)
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось сохранить")
|
||||
} finally {
|
||||
setToggleBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
async function handleSaveSchedule() {
|
||||
if (!liveReady || saveBusy) return
|
||||
const intervalSec = Math.max(300, Number.parseInt(intervalDraft, 10) || 21600)
|
||||
const renewBeforeDays = Math.max(1, Math.min(90, Number.parseInt(daysDraft, 10) || 30))
|
||||
setSaveBusy(true)
|
||||
try {
|
||||
const saved = await putCertificateRenewSettings(backendUrl, { intervalSec, renewBeforeDays })
|
||||
setIntervalDraft(String(saved.intervalSec))
|
||||
setDaysDraft(String(saved.renewBeforeDays))
|
||||
toast.success("Расписание автообновления сохранено")
|
||||
} catch (e) {
|
||||
toast.error(e instanceof Error ? e.message : "Не удалось сохранить расписание")
|
||||
} finally {
|
||||
setSaveBusy(false)
|
||||
}
|
||||
}
|
||||
|
||||
const interactionsOff = !liveReady || toggleBusy || (liveReady && !loaded)
|
||||
|
||||
return (
|
||||
<OpsPanel
|
||||
title="Автообновление через MikrotikManager"
|
||||
description="Фоновый выпуск Let's Encrypt (Cloudflare DNS-01) для сертификатов, выпущенных из этой панели. Ручной выпуск не зависит от переключателя."
|
||||
headerRight={
|
||||
<Badge
|
||||
size="sm"
|
||||
variant={!liveReady ? "warning-light" : enabled ? "success-light" : "secondary"}
|
||||
>
|
||||
{!liveReady ? "нет backend" : enabled ? "Включено" : "Выключено"}
|
||||
</Badge>
|
||||
}
|
||||
contentClassName="px-5 py-4 flex flex-col gap-4"
|
||||
>
|
||||
<div className="flex items-center justify-between gap-3">
|
||||
<div>
|
||||
<p className="text-sm font-medium">Обновлять сертификаты из MM</p>
|
||||
<p className="text-muted-foreground mt-0.5 text-xs">
|
||||
Если ACME уже крутит RouterOS — выключите, чтобы не было двойного перевыпуска.
|
||||
</p>
|
||||
</div>
|
||||
<FormToggle checked={enabled} onChange={handleEnabledChange} disabled={interactionsOff} />
|
||||
</div>
|
||||
|
||||
{!liveReady ? (
|
||||
<Alert variant="warning">
|
||||
<AlertTitle>Нет подключения к API</AlertTitle>
|
||||
<AlertDescription>
|
||||
Переключатель станет активен, когда backend доступен. Планировщик читает тот же флаг, что и страница «Сбор данных».
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : enabled ? (
|
||||
<Alert variant="warning">
|
||||
<AlertTitle>Не смешивайте с ACME RouterOS</AlertTitle>
|
||||
<AlertDescription>
|
||||
MM обновляет только сертификаты, выпущенные через эту страницу. Встроенный Let's Encrypt на
|
||||
устройстве для тех же имён лучше не включать одновременно.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
) : (
|
||||
<Alert variant="info">
|
||||
<AlertTitle>Обновление отдано RouterOS</AlertTitle>
|
||||
<AlertDescription>
|
||||
Планировщик MM больше не проверяет срок и не перевыпускает сертификаты. Ручной выпуск и импорт
|
||||
остаются доступны.
|
||||
</AlertDescription>
|
||||
</Alert>
|
||||
)}
|
||||
|
||||
<div className={cn("flex flex-col gap-4", !enabled && "pointer-events-none opacity-40")}>
|
||||
<div className="grid grid-cols-1 gap-3 sm:grid-cols-2">
|
||||
<FormField label="Интервал проверки" hint="Секунды, минимум 300">
|
||||
<Input
|
||||
inputMode="numeric"
|
||||
value={intervalDraft}
|
||||
onChange={(e) => setIntervalDraft(e.target.value)}
|
||||
disabled={!liveReady || saveBusy}
|
||||
/>
|
||||
</FormField>
|
||||
<FormField label="Обновлять за" hint="Дней до истечения, 1–90">
|
||||
<Input
|
||||
inputMode="numeric"
|
||||
value={daysDraft}
|
||||
onChange={(e) => setDaysDraft(e.target.value)}
|
||||
disabled={!liveReady || saveBusy}
|
||||
/>
|
||||
</FormField>
|
||||
</div>
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<Button variant="outline" size="sm" disabled={!liveReady || saveBusy || !enabled} onClick={() => void handleSaveSchedule()}>
|
||||
Сохранить расписание
|
||||
</Button>
|
||||
<Link href="/data-collection" className={cn(buttonVariants({ variant: "ghost", size: "sm" }), "h-8 text-xs")}>
|
||||
Журнал планировщика →
|
||||
</Link>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{lastCollectedAt || lastError ? (
|
||||
<p className="text-muted-foreground text-xs">
|
||||
Последний прогон:{" "}
|
||||
{lastCollectedAt ? new Date(lastCollectedAt).toLocaleString("ru-RU") : "ещё не было"}
|
||||
{lastError ? ` · ошибка: ${lastError}` : ""}
|
||||
</p>
|
||||
) : null}
|
||||
</OpsPanel>
|
||||
)
|
||||
}
|
||||
@@ -4,7 +4,7 @@ import { useEffect, useState, useRef, useMemo } from "react"
|
||||
import { useRouter, usePathname } from "next/navigation"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
SearchIcon, LayoutDashboardIcon, ActivityIcon, MapIcon, HeartPulseIcon,
|
||||
SearchIcon, LayoutDashboardIcon, ActivityIcon, ChartColumnIcon, MapIcon, HeartPulseIcon,
|
||||
GlobeIcon, NetworkIcon, LayersIcon, TagIcon, ServerIcon, FilterIcon,
|
||||
ShieldIcon, ShieldCheckIcon, CableIcon, BoxIcon, BadgeCheckIcon,
|
||||
HardDriveIcon, RouteIcon, GitForkIcon, GitMergeIcon, ScanLineIcon,
|
||||
@@ -27,6 +27,7 @@ const ALL_ITEMS: CommandItem[] = [
|
||||
// Обзор
|
||||
{ id: "dashboard", title: "Дашборд", group: "Обзор", url: "/dashboard", icon: <LayoutDashboardIcon />, keywords: ["главная","home","overview"] },
|
||||
{ id: "traffic", title: "Трафик", group: "Обзор", url: "/traffic", icon: <ActivityIcon />, keywords: ["bandwidth","traffic","клиенты","интерфейсы"] },
|
||||
{ id: "statistics", title: "Статистика", group: "Обзор", url: "/statistics", icon: <ChartColumnIcon />, keywords: ["stats","отчёт","куб","страны","asn","ipfix"] },
|
||||
{ id: "network-map", title: "Карта сети", group: "Обзор", url: "/network-map", icon: <MapIcon />, keywords: ["topology","топология","map"] },
|
||||
{ id: "uptime", title: "Мониторинг / Uptime", group: "Обзор", url: "/uptime", icon: <HeartPulseIcon />, keywords: ["ping","uptime","мониторинг","проверка"] },
|
||||
// Данные
|
||||
|
||||
@@ -0,0 +1,90 @@
|
||||
"use client"
|
||||
|
||||
import Link from "next/link"
|
||||
import {
|
||||
Timeline,
|
||||
TimelineContent,
|
||||
TimelineDate,
|
||||
TimelineHeader,
|
||||
TimelineIndicator,
|
||||
TimelineItem,
|
||||
TimelineSeparator,
|
||||
TimelineTitle,
|
||||
} from "@/components/reui/timeline"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { EventItem } from "@mmapp/contracts/events"
|
||||
|
||||
function formatEventAge(iso: string): string {
|
||||
const ts = Date.parse(iso)
|
||||
if (!Number.isFinite(ts)) return "—"
|
||||
const diffMs = Math.max(0, Date.now() - ts)
|
||||
const minutes = Math.floor(diffMs / 60_000)
|
||||
if (minutes < 1) return "сейчас"
|
||||
if (minutes < 60) return `${minutes}м`
|
||||
const hours = Math.floor(minutes / 60)
|
||||
if (hours < 24) return `${hours}ч`
|
||||
const days = Math.floor(hours / 24)
|
||||
return `${days}д`
|
||||
}
|
||||
|
||||
const LEVEL_DOT: Record<EventItem["level"], string> = {
|
||||
critical: "border-destructive bg-destructive/20 group-data-completed/timeline-item:border-destructive",
|
||||
warning: "border-warning bg-warning/20 group-data-completed/timeline-item:border-warning",
|
||||
info: "border-info bg-info/20 group-data-completed/timeline-item:border-info",
|
||||
}
|
||||
|
||||
/**
|
||||
* Compact activity timeline.
|
||||
* Preview: https://reui.io/preview/base/timeline-3
|
||||
* Docs: https://reui.io/docs/components/base/timeline
|
||||
*/
|
||||
export function DashboardEventsTimeline({
|
||||
events,
|
||||
loading,
|
||||
error,
|
||||
}: {
|
||||
events: EventItem[]
|
||||
loading?: boolean
|
||||
error?: string | null
|
||||
}) {
|
||||
if (loading && events.length === 0) {
|
||||
return <p className="text-muted-foreground px-5 py-6 text-sm">Загрузка событий…</p>
|
||||
}
|
||||
if (error && events.length === 0) {
|
||||
return (
|
||||
<div className="px-5 py-4">
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>События недоступны</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
if (events.length === 0) {
|
||||
return (
|
||||
<p className="text-muted-foreground px-5 py-6 text-sm">
|
||||
Событий пока нет.{" "}
|
||||
<Link href="/alerts" className="underline underline-offset-2">
|
||||
Оповещения
|
||||
</Link>
|
||||
</p>
|
||||
)
|
||||
}
|
||||
|
||||
return (
|
||||
<Timeline defaultValue={events.length} className="px-5 py-4">
|
||||
{events.map((event, index) => (
|
||||
<TimelineItem key={event.id} step={index + 1}>
|
||||
<TimelineHeader>
|
||||
<TimelineSeparator />
|
||||
<TimelineDate>{formatEventAge(event.createdAt)}</TimelineDate>
|
||||
<TimelineTitle className="text-[13px] leading-tight">{event.title}</TimelineTitle>
|
||||
<TimelineIndicator className={cn(LEVEL_DOT[event.level])} />
|
||||
</TimelineHeader>
|
||||
<TimelineContent className="text-xs leading-snug">{event.message}</TimelineContent>
|
||||
</TimelineItem>
|
||||
))}
|
||||
</Timeline>
|
||||
)
|
||||
}
|
||||
@@ -2,8 +2,6 @@
|
||||
|
||||
import { useMemo, useState } from "react"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { InternetPathViewModel } from "@/lib/dashboard-internet-path"
|
||||
import { Maximize2Icon, ZoomInIcon, ZoomOutIcon } from "lucide-react"
|
||||
@@ -167,7 +165,7 @@ function ServerNode({
|
||||
)
|
||||
}
|
||||
|
||||
export function InternetPathMapCard({ model }: { model: InternetPathViewModel | null }) {
|
||||
export function InternetPathMapCanvas({ model }: { model: InternetPathViewModel | null }) {
|
||||
const [zoom, setZoom] = useState(1)
|
||||
const [pan, setPan] = useState({ x: 0, y: 0 })
|
||||
const [isDragging, setIsDragging] = useState(false)
|
||||
@@ -226,22 +224,7 @@ export function InternetPathMapCard({ model }: { model: InternetPathViewModel |
|
||||
}
|
||||
|
||||
return (
|
||||
<OpsPanel
|
||||
title="Internet path map"
|
||||
description="Основной и текущий путь трафика HomeRouter → Internet"
|
||||
headerRight={
|
||||
<StatusBadge
|
||||
status={
|
||||
model?.pathState === "healthy"
|
||||
? "online"
|
||||
: model?.pathState === "failover"
|
||||
? "degraded"
|
||||
: "offline"
|
||||
}
|
||||
/>
|
||||
}
|
||||
contentClassName="px-5 pb-4"
|
||||
>
|
||||
<div className="flex flex-col gap-3">
|
||||
{!model && (
|
||||
<div className="h-[240px] rounded-md border border-dashed border-border grid place-items-center text-sm text-muted-foreground">
|
||||
Недостаточно данных для построения маршрута
|
||||
@@ -412,6 +395,6 @@ export function InternetPathMapCard({ model }: { model: InternetPathViewModel |
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</OpsPanel>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"use client"
|
||||
|
||||
import { useEffect, useState } from "react"
|
||||
import Link from "next/link"
|
||||
import { ChevronDownIcon } from "lucide-react"
|
||||
import { OpsPanel } from "@/components/ops-panel"
|
||||
import { Alert, AlertDescription, AlertTitle } from "@/components/reui/alert"
|
||||
import { buttonVariants } from "@/components/ui/button"
|
||||
import {
|
||||
Collapsible,
|
||||
CollapsibleContent,
|
||||
CollapsibleTrigger,
|
||||
} from "@/components/ui/collapsible"
|
||||
import { cn } from "@/lib/utils"
|
||||
import type { InternetPathViewModel } from "@/lib/dashboard-internet-path"
|
||||
import { InternetPathMapCanvas } from "@/components/dashboard/internet-path-map"
|
||||
import { InternetPathSummary } from "@/components/dashboard/internet-path-summary"
|
||||
|
||||
const PATH_MAP_OPEN_LS = "mm:dashboard-path-map-open"
|
||||
|
||||
function readMapOpen(): boolean {
|
||||
if (typeof window === "undefined") return true
|
||||
try {
|
||||
const raw = localStorage.getItem(PATH_MAP_OPEN_LS)
|
||||
if (raw === "0") return false
|
||||
if (raw === "1") return true
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
return true
|
||||
}
|
||||
|
||||
export function InternetPathPanel({
|
||||
model,
|
||||
loading,
|
||||
error,
|
||||
}: {
|
||||
model: InternetPathViewModel | null
|
||||
loading?: boolean
|
||||
error?: string | null
|
||||
}) {
|
||||
const [open, setOpen] = useState(true)
|
||||
const [hydrated, setHydrated] = useState(false)
|
||||
|
||||
useEffect(() => {
|
||||
queueMicrotask(() => {
|
||||
setOpen(readMapOpen())
|
||||
setHydrated(true)
|
||||
})
|
||||
}, [])
|
||||
|
||||
function handleOpenChange(next: boolean) {
|
||||
setOpen(next)
|
||||
try {
|
||||
localStorage.setItem(PATH_MAP_OPEN_LS, next ? "1" : "0")
|
||||
} catch {
|
||||
/* ignore */
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<OpsPanel
|
||||
title="Internet path"
|
||||
description="Home → WAN → JH → Exit"
|
||||
headerRight={
|
||||
<Link href="/network-map" className={cn(buttonVariants({ variant: "outline", size: "sm" }), "h-7 text-xs")}>
|
||||
Карта сети →
|
||||
</Link>
|
||||
}
|
||||
contentClassName="flex flex-col gap-3 px-5 pb-4"
|
||||
>
|
||||
{error ? (
|
||||
<Alert variant="destructive">
|
||||
<AlertTitle>Не удалось загрузить путь</AlertTitle>
|
||||
<AlertDescription>{error}</AlertDescription>
|
||||
</Alert>
|
||||
) : null}
|
||||
{loading && !model ? (
|
||||
<div className="h-20 animate-pulse rounded-md bg-muted/40" />
|
||||
) : (
|
||||
<InternetPathSummary model={model} />
|
||||
)}
|
||||
|
||||
<Collapsible open={hydrated ? open : true} onOpenChange={handleOpenChange}>
|
||||
<CollapsibleTrigger
|
||||
className={cn(buttonVariants({ variant: "ghost", size: "sm" }), "h-7 w-fit gap-1.5 text-xs")}
|
||||
>
|
||||
<ChevronDownIcon className={cn("size-3.5 transition-transform", open && "rotate-180")} />
|
||||
{open ? "Скрыть карту" : "Показать карту"}
|
||||
</CollapsibleTrigger>
|
||||
<CollapsibleContent>
|
||||
<InternetPathMapCanvas model={model} />
|
||||
</CollapsibleContent>
|
||||
</Collapsible>
|
||||
</OpsPanel>
|
||||
)
|
||||
}
|
||||
@@ -0,0 +1,130 @@
|
||||
"use client"
|
||||
|
||||
import type { ReactNode } from "react"
|
||||
import Link from "next/link"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { StatusBadge } from "@/components/status-badge"
|
||||
import { Flag } from "@/components/flag"
|
||||
import type { InternetPathViewModel } from "@/lib/dashboard-internet-path"
|
||||
import type { ServerStatus } from "@/lib/data"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { ArrowRightIcon, HomeIcon, RadioIcon, ServerIcon, GlobeIcon } from "lucide-react"
|
||||
|
||||
function pathStateToServerStatus(state: InternetPathViewModel["pathState"]): ServerStatus {
|
||||
if (state === "healthy") return "online"
|
||||
if (state === "failover" || state === "degraded") return "degraded"
|
||||
return "offline"
|
||||
}
|
||||
|
||||
function HopChip({
|
||||
label,
|
||||
name,
|
||||
country,
|
||||
icon,
|
||||
iconClassName,
|
||||
}: {
|
||||
label: string
|
||||
name: string
|
||||
country?: string
|
||||
icon: ReactNode
|
||||
iconClassName?: string
|
||||
}) {
|
||||
return (
|
||||
<div className="flex min-w-0 items-center gap-2 rounded-md border border-border/60 px-2 py-1.5">
|
||||
<IconTile variant="elevated" size="sm" className={cn("shrink-0", iconClassName)} aria-hidden="true">
|
||||
{icon}
|
||||
</IconTile>
|
||||
<div className="min-w-0">
|
||||
<p className="text-muted-foreground text-[10px] font-medium uppercase tracking-wide">{label}</p>
|
||||
<p className="flex items-center gap-1 truncate text-sm font-medium">
|
||||
{country ? <Flag code={country} className="shrink-0" /> : null}
|
||||
<span className="truncate">{name}</span>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* Compact live path strip (Frame-friendly). Canvas lives separately.
|
||||
* Preview: https://reui.io/preview/base/stats-12
|
||||
* Docs: https://reui.io/docs/components/base/icon-tile
|
||||
*/
|
||||
export function InternetPathSummary({ model }: { model: InternetPathViewModel | null }) {
|
||||
if (!model) {
|
||||
return (
|
||||
<p className="text-muted-foreground text-sm">
|
||||
Недостаточно данных для пути. Добавьте home-router и проверьте{" "}
|
||||
<Link href="/network-map" className="underline underline-offset-2">
|
||||
карту сети
|
||||
</Link>
|
||||
.
|
||||
</p>
|
||||
)
|
||||
}
|
||||
|
||||
const hop = model.currentHop ?? model.primaryHop
|
||||
const wanName = hop?.wan.name ?? model.activeWanUplink?.name ?? "WAN"
|
||||
const wanIsp = hop?.wan.isp ?? model.activeWanUplink?.isp ?? "—"
|
||||
const ping = hop?.wanJhMetrics.pingMs
|
||||
const dl = hop?.wanJhMetrics.dlMbps
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-3">
|
||||
<div className="flex flex-wrap items-center gap-2">
|
||||
<StatusBadge status={pathStateToServerStatus(model.pathState)} />
|
||||
{model.pathState === "failover" ? (
|
||||
<Badge variant="warning-light" size="sm">failover</Badge>
|
||||
) : null}
|
||||
{ping != null ? (
|
||||
<Badge variant="outline" size="sm" className="tabular-nums">
|
||||
{ping} мс
|
||||
</Badge>
|
||||
) : null}
|
||||
{dl != null ? (
|
||||
<Badge variant="outline" size="sm" className="tabular-nums">
|
||||
{Math.round(dl)} ↓ Мбит/с
|
||||
</Badge>
|
||||
) : null}
|
||||
</div>
|
||||
|
||||
<div className="flex flex-col gap-2 @3xl:flex-row @3xl:items-center">
|
||||
<HopChip
|
||||
label="Home"
|
||||
name={model.homeRouter.name}
|
||||
country={model.homeRouter.country}
|
||||
icon={<HomeIcon />}
|
||||
iconClassName="text-success"
|
||||
/>
|
||||
<ArrowRightIcon className="text-muted-foreground hidden size-4 shrink-0 @3xl:block" aria-hidden="true" />
|
||||
<HopChip
|
||||
label="WAN"
|
||||
name={`${wanName} · ${wanIsp}`}
|
||||
icon={<RadioIcon />}
|
||||
iconClassName="text-info"
|
||||
/>
|
||||
<ArrowRightIcon className="text-muted-foreground hidden size-4 shrink-0 @3xl:block" aria-hidden="true" />
|
||||
<HopChip
|
||||
label="JH"
|
||||
name={hop?.jumpHost.name ?? model.fallbackJumpHost?.name ?? "—"}
|
||||
country={hop?.jumpHost.country ?? model.fallbackJumpHost?.country}
|
||||
icon={<ServerIcon />}
|
||||
iconClassName="text-primary"
|
||||
/>
|
||||
<ArrowRightIcon className="text-muted-foreground hidden size-4 shrink-0 @3xl:block" aria-hidden="true" />
|
||||
<HopChip
|
||||
label="Exit"
|
||||
name={hop?.exitNode.name ?? model.fallbackExitNode?.name ?? "—"}
|
||||
country={hop?.exitNode.country ?? model.fallbackExitNode?.country}
|
||||
icon={<GlobeIcon />}
|
||||
iconClassName="text-muted-foreground"
|
||||
/>
|
||||
</div>
|
||||
|
||||
<p className="text-muted-foreground text-xs leading-relaxed">
|
||||
{model.currentPath?.reason ?? model.primaryPath?.reason ?? "Текущий путь не определён"}
|
||||
</p>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -1,6 +1,6 @@
|
||||
"use client"
|
||||
|
||||
import { useMemo } from "react"
|
||||
import { useMemo, type ReactNode } from "react"
|
||||
import {
|
||||
type ColumnDef,
|
||||
getCoreRowModel,
|
||||
@@ -8,8 +8,9 @@ import {
|
||||
useReactTable,
|
||||
} from "@tanstack/react-table"
|
||||
import type { Backup } from "@/lib/data"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { DataGridShell } from "@/components/data-grids/shared/data-grid-shell"
|
||||
import {
|
||||
DATA_GRID_CELL_PAD,
|
||||
@@ -18,13 +19,28 @@ import {
|
||||
} from "@/components/data-grids/shared/data-grid-layout"
|
||||
import { DataGridSortHeader } from "@/components/data-grids/shared/data-grid-sort-header"
|
||||
import { EmptyState } from "@/components/empty-state"
|
||||
import { DownloadIcon, HardDriveIcon, RefreshCwIcon, Trash2Icon } from "lucide-react"
|
||||
import {
|
||||
CloudIcon,
|
||||
DownloadIcon,
|
||||
HardDriveIcon,
|
||||
RefreshCwIcon,
|
||||
Trash2Icon,
|
||||
} from "lucide-react"
|
||||
|
||||
interface BackupsDataGridProps {
|
||||
backups: Backup[]
|
||||
onDownload: (id: string, filename: string) => void
|
||||
onRestore: (backup: Backup) => void
|
||||
onDelete: (id: string) => void
|
||||
onDelete: (backup: Backup) => void
|
||||
emptyAction?: ReactNode
|
||||
emptyTitle?: string
|
||||
emptyDescription?: string
|
||||
}
|
||||
|
||||
function storageBadge(storage: Backup["storage"]) {
|
||||
if (storage === "s3") return { label: "S3", variant: "info-light" as const }
|
||||
if (storage === "both") return { label: "Локально + S3", variant: "success-light" as const }
|
||||
return { label: "Локально", variant: "secondary" as const }
|
||||
}
|
||||
|
||||
function BackupsDataGrid({
|
||||
@@ -32,6 +48,9 @@ function BackupsDataGrid({
|
||||
onDownload,
|
||||
onRestore,
|
||||
onDelete,
|
||||
emptyAction,
|
||||
emptyTitle = "Нет бэкапов",
|
||||
emptyDescription = "Создайте первый бэкап вручную или настройте расписание",
|
||||
}: BackupsDataGridProps) {
|
||||
const columns = useMemo<ColumnDef<Backup>[]>(
|
||||
() => [
|
||||
@@ -39,9 +58,27 @@ function BackupsDataGrid({
|
||||
id: "filename",
|
||||
accessorKey: "filename",
|
||||
header: ({ column }) => <DataGridSortHeader column={column} title="Файл" />,
|
||||
cell: ({ row }) => (
|
||||
<span className="font-mono text-xs font-medium">{row.original.filename}</span>
|
||||
),
|
||||
cell: ({ row }) => {
|
||||
const b = row.original
|
||||
const inS3 = b.storage === "s3" || b.storage === "both"
|
||||
return (
|
||||
<div className="flex items-center gap-3 min-w-0">
|
||||
<IconTile
|
||||
variant="elevated"
|
||||
className={inS3 ? "size-10.5 shrink-0 text-info" : "size-10.5 shrink-0 text-muted-foreground"}
|
||||
aria-hidden="true"
|
||||
>
|
||||
{inS3 ? <CloudIcon /> : <HardDriveIcon />}
|
||||
</IconTile>
|
||||
<div className="min-w-0">
|
||||
<span className="font-mono text-xs font-medium block truncate">{b.filename}</span>
|
||||
{b.uploadError ? (
|
||||
<span className="text-[11px] text-destructive truncate block">{b.uploadError}</span>
|
||||
) : null}
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
},
|
||||
meta: {
|
||||
headerTitle: "Файл",
|
||||
headerClassName: DATA_GRID_CELL_PAD_FIRST,
|
||||
@@ -78,23 +115,35 @@ function BackupsDataGrid({
|
||||
id: "kind",
|
||||
accessorKey: "kind",
|
||||
header: ({ column }) => <DataGridSortHeader column={column} title="Тип" />,
|
||||
cell: ({ row }) => (
|
||||
<Badge
|
||||
variant={row.original.kind === "manual" ? "info-light" : "secondary"}
|
||||
size="sm"
|
||||
radius="full"
|
||||
>
|
||||
{row.original.kind === "auto" ? "авто" : "вручную"}
|
||||
</Badge>
|
||||
),
|
||||
meta: {
|
||||
headerTitle: "Тип",
|
||||
headerClassName: DATA_GRID_CELL_PAD,
|
||||
cellClassName: DATA_GRID_CELL_PAD,
|
||||
},
|
||||
},
|
||||
{
|
||||
id: "storage",
|
||||
accessorKey: "storage",
|
||||
header: ({ column }) => <DataGridSortHeader column={column} title="Хранилище" />,
|
||||
cell: ({ row }) => {
|
||||
const kind = row.original.kind
|
||||
const badge = storageBadge(row.original.storage)
|
||||
return (
|
||||
<span
|
||||
className={cn(
|
||||
"text-xs px-2 py-0.5 rounded border font-medium",
|
||||
kind === "manual"
|
||||
? "bg-blue-500/10 text-blue-400 border-blue-500/20"
|
||||
: "bg-muted text-muted-foreground border-border",
|
||||
)}
|
||||
>
|
||||
{kind === "auto" ? "авто" : "вручную"}
|
||||
</span>
|
||||
<Badge variant={badge.variant} size="sm" radius="full">
|
||||
{badge.label}
|
||||
</Badge>
|
||||
)
|
||||
},
|
||||
meta: {
|
||||
headerTitle: "Тип",
|
||||
headerTitle: "Хранилище",
|
||||
headerClassName: DATA_GRID_CELL_PAD,
|
||||
cellClassName: DATA_GRID_CELL_PAD,
|
||||
},
|
||||
@@ -135,29 +184,35 @@ function BackupsDataGrid({
|
||||
return (
|
||||
<div className="flex items-center gap-1 justify-end opacity-0 transition-opacity group-hover/row:opacity-100 focus-within:opacity-100">
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
className="size-7"
|
||||
title="Скачать"
|
||||
aria-label={`Скачать ${b.filename}`}
|
||||
onClick={() => onDownload(b.id, b.filename)}
|
||||
>
|
||||
<DownloadIcon className="size-3.5" />
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
className="size-7"
|
||||
title="Восстановить"
|
||||
aria-label={`Восстановить ${b.filename}`}
|
||||
onClick={() => onRestore(b)}
|
||||
>
|
||||
<RefreshCwIcon className="size-3.5" />
|
||||
</Button>
|
||||
<Button
|
||||
type="button"
|
||||
variant="ghost"
|
||||
size="icon"
|
||||
className="size-7 text-destructive hover:text-destructive"
|
||||
title="Удалить"
|
||||
onClick={() => onDelete(b.id)}
|
||||
aria-label={`Удалить ${b.filename}`}
|
||||
onClick={() => onDelete(b)}
|
||||
>
|
||||
<Trash2Icon className="size-3.5" />
|
||||
</Button>
|
||||
@@ -186,9 +241,10 @@ function BackupsDataGrid({
|
||||
if (backups.length === 0) {
|
||||
return (
|
||||
<EmptyState
|
||||
icon={<HardDriveIcon className="size-4" />}
|
||||
title="Нет бэкапов"
|
||||
description="Создайте первый бэкап вручную или настройте расписание"
|
||||
icon={<HardDriveIcon className="size-5" />}
|
||||
title={emptyTitle}
|
||||
description={emptyDescription}
|
||||
action={emptyAction}
|
||||
className="border-0 py-10"
|
||||
/>
|
||||
)
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
"use client"
|
||||
|
||||
import { CompactDataGrid, type CompactDataGridColumn } from "@/components/data-grids/compact-data-grid"
|
||||
import { Flag } from "@/components/flag"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { fmtBps, formatBytes } from "@/lib/fmt-rate"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
STATISTICS_DUP_MARK,
|
||||
STATISTICS_UNBOUND_USER_ID,
|
||||
STATISTICS_WAN_MARK,
|
||||
type StatisticsBreakdownRow,
|
||||
} from "@mmapp/contracts/statistics"
|
||||
|
||||
export type StatisticsSliceKind = "users" | "servers" | "interfaces" | "countries" | "services" | "asns"
|
||||
|
||||
export function StatisticsBreakdownDataGrid({
|
||||
rows,
|
||||
kind,
|
||||
selectedId,
|
||||
onRowClick,
|
||||
isLoading,
|
||||
density = "full",
|
||||
}: {
|
||||
rows: StatisticsBreakdownRow[]
|
||||
kind: StatisticsSliceKind
|
||||
selectedId?: string
|
||||
onRowClick?: (row: StatisticsBreakdownRow) => void
|
||||
isLoading?: boolean
|
||||
density?: "full" | "mini"
|
||||
}) {
|
||||
const mini = density === "mini"
|
||||
const columns: CompactDataGridColumn<StatisticsBreakdownRow>[] = [
|
||||
{
|
||||
id: "label",
|
||||
header: "Имя",
|
||||
accessorKey: "label",
|
||||
cell: (row) => (
|
||||
<span className={cn("flex items-center gap-2", selectedId === row.id && "font-medium")}>
|
||||
{kind === "countries" && row.id !== "XX" && row.id !== STATISTICS_UNBOUND_USER_ID ? (
|
||||
<Flag code={row.id} size={16} />
|
||||
) : null}
|
||||
<span className="truncate">{row.label || row.id}</span>
|
||||
{selectedId === row.id ? (
|
||||
<Badge variant="outline" size="sm">
|
||||
слайс
|
||||
</Badge>
|
||||
) : null}
|
||||
</span>
|
||||
),
|
||||
},
|
||||
{
|
||||
id: "bytes",
|
||||
header: "Байты",
|
||||
accessorKey: "bytes",
|
||||
cell: (row) => <span className="tabular-nums">{formatBytes(row.bytes)}</span>,
|
||||
},
|
||||
...(!mini
|
||||
? [
|
||||
{
|
||||
id: "packets",
|
||||
header: "Пакеты",
|
||||
accessorKey: "packets" as const,
|
||||
cell: (row: StatisticsBreakdownRow) => (
|
||||
<span className="tabular-nums">{row.packets.toLocaleString("ru-RU")}</span>
|
||||
),
|
||||
},
|
||||
{
|
||||
id: "bps",
|
||||
header: "Средний bitrate",
|
||||
accessorKey: "bps" as const,
|
||||
cell: (row: StatisticsBreakdownRow) => <span className="tabular-nums">{fmtBps(row.bps)}</span>,
|
||||
},
|
||||
]
|
||||
: []),
|
||||
{
|
||||
id: "percent",
|
||||
header: "Доля",
|
||||
accessorKey: "percent",
|
||||
cell: (row) => (
|
||||
<span className="tabular-nums">
|
||||
{row.percent === 0
|
||||
&& (row.label.includes(STATISTICS_WAN_MARK) || row.label.includes(`· ${STATISTICS_DUP_MARK}`))
|
||||
? "—"
|
||||
: `${row.percent.toFixed(1)}%`}
|
||||
</span>
|
||||
),
|
||||
},
|
||||
]
|
||||
|
||||
return (
|
||||
<CompactDataGrid
|
||||
data={rows}
|
||||
columns={columns}
|
||||
isLoading={isLoading}
|
||||
emptyTitle="Нет трафика"
|
||||
emptyDescription="За выбранный период и слайсы нет данных куба."
|
||||
onRowClick={onRowClick}
|
||||
/>
|
||||
)
|
||||
}
|
||||
@@ -1,13 +1,14 @@
|
||||
"use client"
|
||||
|
||||
import { ReactNode } from "react"
|
||||
import { SearchIcon } from "lucide-react"
|
||||
import { ListFilterIcon, SearchIcon } from "lucide-react"
|
||||
import { cn } from "@/lib/utils"
|
||||
import {
|
||||
InputGroup,
|
||||
InputGroupAddon,
|
||||
InputGroupInput,
|
||||
} from "@/components/ui/input-group"
|
||||
import { Button } from "@/components/ui/button"
|
||||
import {
|
||||
Filters,
|
||||
type Filter,
|
||||
@@ -77,6 +78,12 @@ function DataPageToolbar<T extends string = string>({
|
||||
fields={filterFields}
|
||||
onChange={onFiltersChange}
|
||||
size="sm"
|
||||
trigger={
|
||||
<Button type="button" variant="outline" size="sm">
|
||||
<ListFilterIcon className="size-3.5" />
|
||||
Фильтры
|
||||
</Button>
|
||||
}
|
||||
/>
|
||||
)}
|
||||
{onSearchChange != null && (
|
||||
|
||||
@@ -125,6 +125,117 @@ export function ServiceBrandIcon({ label, size = 22 }: { label: string; size?: n
|
||||
<path d="M13.4 4v9.1a3.3 3.3 0 1 1-2.8-3.3V7.2c1.6.9 3.2 1.4 5 1.5V5.4c-1.4-.1-2.7-.6-3.8-1.4H13.4Z" fill="#FE2C55" transform="translate(1.2 1)" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "apple":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<path d="M14.7 6.2c.8-.9 1.3-2.2 1.2-3.5-1.2.1-2.6.8-3.4 1.8-.8.9-1.5 2.2-1.3 3.5 1.3 0 2.6-.8 3.5-1.8Z" fill="#111" />
|
||||
<path d="M16.8 12.2c0-2.2 1.8-3.3 1.9-3.4-1.1-1.6-2.7-1.8-3.3-1.8-1.4-.1-2.7.8-3.4.8s-1.8-.8-3-.8c-1.5 0-3 .9-3.8 2.3-1.6 2.8-.4 7 1.2 9.3.8 1.1 1.7 2.3 2.9 2.3 1.2 0 1.6-.7 3-.7s1.8.7 3 .7 2-.1 2.9-2.2c1.1-1.5 1.5-3 1.5-3.1-.1 0-2.9-1.1-2.9-4.4Z" fill="#111" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "github":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#181717" />
|
||||
<path d="M12 6.4c-3.1 0-5.6 2.5-5.6 5.6 0 2.5 1.6 4.6 3.8 5.3.3.1.4-.1.4-.3v-1.1c-1.6.3-1.9-.7-1.9-.7-.3-.6-.6-.8-.6-.8-.5-.4 0-.4 0-.4.6 0 .9.6.9.6.5.9 1.4.6 1.7.5.1-.4.2-.6.4-.8-1.2-.1-2.5-.6-2.5-2.8 0-.6.2-1.1.6-1.5-.1-.1-.3-.7 0-1.4 0 0 .5-.2 1.6.6.5-.1 1-.2 1.5-.2s1 .1 1.5.2c1.1-.8 1.6-.6 1.6-.6.3.7.1 1.3 0 1.4.4.4.6.9.6 1.5 0 2.2-1.3 2.6-2.5 2.8.2.2.4.5.4 1.1v1.6c0 .2.1.4.4.3 2.2-.7 3.8-2.8 3.8-5.3 0-3.1-2.5-5.6-5.6-5.6Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "gitlab":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<path d="M12 19.2 8.4 8.4h7.2L12 19.2Z" fill="#E24329" />
|
||||
<path d="M12 19.2 8.4 8.4 5.2 16.2 12 19.2Z" fill="#FC6D26" />
|
||||
<path d="M12 19.2 15.6 8.4 18.8 16.2 12 19.2Z" fill="#FC6D26" />
|
||||
<path d="M5.2 16.2 3 8.4h5.4L5.2 16.2Z" fill="#FCA326" />
|
||||
<path d="M18.8 16.2 21 8.4h-5.4l3.2 7.8Z" fill="#FCA326" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "spotify":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#1DB954" />
|
||||
<path d="M7.2 10.4c3.2-1 6.8-.8 9.6.8M7.6 13c2.6-.8 5.6-.6 8 .6M8 15.4c2-.6 4.4-.4 6.2.4" fill="none" stroke="#fff" strokeWidth="1.5" strokeLinecap="round" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "x":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#111" />
|
||||
<path d="M6.2 5.6h3.2l3 4.2 3.6-4.2H18l-5.2 6.1 5.4 6.7h-3.2l-3.4-4.4-4 4.4H6.4l5.6-6.4Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "vk":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#0077FF" />
|
||||
<path d="M4.8 7.8h2.6c.1 4.2 1.9 6.7 5.4 6.7V7.8h2.4v3.9c1.5-.2 2.9-1.7 3.4-3.9h2.4c-.6 3.3-2.6 5.4-4.4 6.2 1.8.6 4.1 2.4 5 5.2h-2.8c-.7-1.9-2.2-3.4-4-3.6v3.6h-2.4v-3.6c-3.5.1-6.1-2.4-6.6-6.8Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "zoom":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#2D8CFF" />
|
||||
<path d="M5.2 9.2h7.2a2.2 2.2 0 0 1 2.2 2.2v5.2H7.4A2.2 2.2 0 0 1 5.2 14.4Z" fill="#fff" />
|
||||
<path d="M16.2 11.2 20 9.4v7.4l-3.8-1.8Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "epic":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#111" />
|
||||
<path d="M8.2 7.4h7.6v2H10.6v2h4.6v2h-4.6v3.2H8.2Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "riot":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<path d="M5 19.2 12 4.2 19 19.2h-3.2L12 10.6 8.2 19.2Z" fill="#D32936" />
|
||||
<path d="M9.4 19.2h5.2l-2.6-5.2Z" fill="#EB0029" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "playstation":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#003087" />
|
||||
<path d="M8.2 14.6c-1 .4-1.8.2-2-.4s.4-1.2 1.6-1.6l2-.7v1.6l-1.2.4c-.6.2-.8.4-.7.6.1.2.4.2.9 0l1-.4v1.5Zm3-6.4v8.2c-1.1.4-2.1.5-2.8.2-.9-.4-.9-1.3 0-1.7.5-.2 1.2-.3 2-.2V9.4c0-1.2.5-1.8 1.4-1.5.4.2.7.6.8 1.2Zm5.2 7.6c-1.1.4-2.2.4-3 0-.8-.4-.8-1.2 0-1.6.5-.2 1.2-.3 2-.2v-2.2l-2.4.8V11l4.2-1.5v6.3Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "roblox":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="5" fill="#111" />
|
||||
<path d="M8.4 6.2 17.6 8.8 15.6 17.8 6.4 15.2Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "digitalocean":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#0080FF" />
|
||||
<path d="M12.4 6.2A5.8 5.8 0 0 0 7.8 16l1.6-1.5A3.6 3.6 0 1 1 16 12h-3.6Z" fill="#fff" />
|
||||
<path d="M12.4 16.2h-1.6v1.6h1.6zm-1.6-2h-1.4v1.4h1.4z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "hetzner":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="4" fill="#D50C2D" />
|
||||
<path d="M7.2 6.4h2.6v4.4h4.4V6.4h2.6v11.2h-2.6v-4.4H9.8v4.4H7.2Z" fill="#fff" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "ovh":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<rect width="24" height="24" rx="4" fill="#123F6D" />
|
||||
<path d="M4.6 15.6 8.4 8.4h3.2L7.8 15.6Zm6.4 0 3.8-7.2h3.2l-3.8 7.2Zm2.2 0h3.4l1.8-3.4h-3.4Z" fill="#00A2E2" />
|
||||
</BrandSvg>
|
||||
)
|
||||
case "chatgpt":
|
||||
return (
|
||||
<BrandSvg size={size}>
|
||||
<circle cx="12" cy="12" r="10" fill="#10A37F" />
|
||||
<path d="M12.2 6.2c.9-.5 2-.5 2.9 0l2.2 1.3c.9.5 1.4 1.4 1.4 2.4v2.6c0 1-.5 1.9-1.4 2.4l-2.2 1.3c-.9.5-2 .5-2.9 0l-.4-.2c.6-.4 1-1 1.1-1.7l.5.3c.4.2.9.2 1.3 0l2.2-1.3c.4-.2.6-.6.6-1.1V10c0-.4-.2-.8-.6-1.1l-2.2-1.3c-.4-.2-.9-.2-1.3 0L10.2 9c-.4.2-.6.6-.6 1.1v.4h-2V10c0-1 .5-1.9 1.4-2.4Z" fill="#fff" />
|
||||
<path d="M8.8 9.6c.6-.4 1.3-.5 2-.3v2.1c0 .4.2.8.6 1.1l2.2 1.3c.4.2.9.2 1.3 0l.5-.3c.2.7.6 1.3 1.1 1.7l-.4.2c-.9.5-2 .5-2.9 0l-2.2-1.3c-.9-.5-1.4-1.4-1.4-2.4Z" fill="#fff" opacity="0.85" />
|
||||
</BrandSvg>
|
||||
)
|
||||
default:
|
||||
return <GenericCloud size={size} />
|
||||
}
|
||||
|
||||
@@ -0,0 +1,93 @@
|
||||
"use client"
|
||||
|
||||
import type { ReactNode } from "react"
|
||||
import type { LucideIcon } from "lucide-react"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import {
|
||||
Frame,
|
||||
FrameHeader,
|
||||
FramePanel,
|
||||
FrameTitle,
|
||||
} from "@/components/reui/frame"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { cn } from "@/lib/utils"
|
||||
|
||||
/**
|
||||
* Sibling Frame columns for dashboard attention queue.
|
||||
* Preview: https://reui.io/preview/base/dashboard-1 · https://reui.io/preview/base/stats-12
|
||||
* Docs: https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/icon-tile · https://reui.io/docs/components/base/badge
|
||||
*/
|
||||
export interface AttentionQueueColumn {
|
||||
id: string
|
||||
title: string
|
||||
icon: LucideIcon
|
||||
iconClassName?: string
|
||||
count: number
|
||||
countVariant?: "destructive" | "warning" | "secondary" | "destructive-light" | "warning-light"
|
||||
emptyTitle: string
|
||||
emptyDescription: string
|
||||
emptyAction?: ReactNode
|
||||
children: ReactNode
|
||||
}
|
||||
|
||||
interface AttentionQueueProps {
|
||||
columns: AttentionQueueColumn[]
|
||||
className?: string
|
||||
}
|
||||
|
||||
const DEFAULT_ICON_CLASS = "text-muted-foreground [&_svg]:text-current"
|
||||
|
||||
export function AttentionQueue({ columns, className }: AttentionQueueProps) {
|
||||
return (
|
||||
<div
|
||||
className={cn(
|
||||
"@container grid min-w-0 items-start gap-2 @3xl:grid-cols-3",
|
||||
className,
|
||||
)}
|
||||
>
|
||||
{columns.map((column) => {
|
||||
const Icon = column.icon
|
||||
const isEmpty = column.count === 0
|
||||
return (
|
||||
<Frame key={column.id} dense spacing="sm" className="min-w-0 w-full">
|
||||
<FrameHeader>
|
||||
<div className="flex min-w-0 items-center gap-2">
|
||||
<IconTile
|
||||
variant="elevated"
|
||||
size="sm"
|
||||
className={cn(DEFAULT_ICON_CLASS, column.iconClassName)}
|
||||
aria-hidden="true"
|
||||
>
|
||||
<Icon />
|
||||
</IconTile>
|
||||
<FrameTitle className="min-w-0 truncate">{column.title}</FrameTitle>
|
||||
{column.count > 0 ? (
|
||||
<Badge
|
||||
size="sm"
|
||||
variant={column.countVariant ?? "secondary"}
|
||||
className="tabular-nums"
|
||||
>
|
||||
{column.count}
|
||||
</Badge>
|
||||
) : null}
|
||||
</div>
|
||||
</FrameHeader>
|
||||
<FramePanel className="min-w-0">
|
||||
{isEmpty ? (
|
||||
<div className="flex min-h-24 flex-col items-start justify-center gap-1 py-3">
|
||||
<p className="text-sm font-medium">{column.emptyTitle}</p>
|
||||
<p className="text-muted-foreground text-xs leading-relaxed">
|
||||
{column.emptyDescription}
|
||||
</p>
|
||||
{column.emptyAction ? <div className="pt-1">{column.emptyAction}</div> : null}
|
||||
</div>
|
||||
) : (
|
||||
column.children
|
||||
)}
|
||||
</FramePanel>
|
||||
</Frame>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@@ -3,3 +3,7 @@ export type { CodeExportFormat, CodeExportSheetProps } from "./code-export-sheet
|
||||
export { KpiStatGrid, KpiStatCardTile, kpiStatItemKey } from "./kpi-stat-grid"
|
||||
export type { KpiStatItem, KpiStatCardData, KpiStatVariant } from "./kpi-stat-grid"
|
||||
export { kpiCols } from "./kpi-cols"
|
||||
export { QuickActionGrid } from "./quick-action-grid"
|
||||
export type { QuickActionItem } from "./quick-action-grid"
|
||||
export { AttentionQueue } from "./attention-queue"
|
||||
export type { AttentionQueueColumn } from "./attention-queue"
|
||||
|
||||
@@ -0,0 +1,161 @@
|
||||
"use client"
|
||||
|
||||
import type { KeyboardEvent, ReactNode } from "react"
|
||||
import Link from "next/link"
|
||||
import {
|
||||
Frame,
|
||||
FrameDescription,
|
||||
FrameHeader,
|
||||
FramePanel,
|
||||
FrameTitle,
|
||||
} from "@/components/reui/frame"
|
||||
import { Badge } from "@/components/reui/badge"
|
||||
import { IconTile } from "@/components/reui/icon-tile"
|
||||
import { cn } from "@/lib/utils"
|
||||
import { kpiCols } from "./kpi-cols"
|
||||
|
||||
type QuickActionBase = {
|
||||
id: string
|
||||
title: string
|
||||
description: string
|
||||
icon?: ReactNode
|
||||
iconClassName?: string
|
||||
badge?: string
|
||||
disabled?: boolean
|
||||
}
|
||||
|
||||
export type QuickActionItem = QuickActionBase &
|
||||
(
|
||||
| { href: string; onClick?: never }
|
||||
| { onClick: () => void; href?: never }
|
||||
)
|
||||
|
||||
interface QuickActionGridProps {
|
||||
actions: QuickActionItem[]
|
||||
title?: string
|
||||
description?: string
|
||||
className?: string
|
||||
}
|
||||
|
||||
const DEFAULT_ICON_CLASS = "text-muted-foreground [&_svg]:text-current"
|
||||
|
||||
function resolveBadge(action: QuickActionItem): string {
|
||||
if (action.badge) return action.badge
|
||||
return action.onClick ? "Выполнить" : "Перейти"
|
||||
}
|
||||
|
||||
function handleActionKeyDown(onActivate: () => void, event: KeyboardEvent<HTMLDivElement>) {
|
||||
if (event.key === "Enter" || event.key === " ") {
|
||||
event.preventDefault()
|
||||
onActivate()
|
||||
}
|
||||
}
|
||||
|
||||
function QuickActionBody({ action }: { action: QuickActionItem }) {
|
||||
return (
|
||||
<div className="relative z-10 flex h-full items-start gap-3">
|
||||
{action.icon ? (
|
||||
<IconTile
|
||||
variant="elevated"
|
||||
aria-hidden="true"
|
||||
className={cn("size-10.5", action.iconClassName ?? DEFAULT_ICON_CLASS)}
|
||||
>
|
||||
{action.icon}
|
||||
</IconTile>
|
||||
) : null}
|
||||
|
||||
<div className="flex min-w-0 flex-1 flex-col gap-0.5">
|
||||
<div className="flex items-start justify-between gap-2">
|
||||
<span className="text-foreground text-sm font-medium">{action.title}</span>
|
||||
<Badge variant="outline" size="sm" className="shrink-0">
|
||||
{resolveBadge(action)}
|
||||
</Badge>
|
||||
</div>
|
||||
<p className="text-muted-foreground line-clamp-2 text-xs leading-relaxed">
|
||||
{action.description}
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
function panelClassName(disabled?: boolean) {
|
||||
return cn(
|
||||
"relative isolate flex h-full flex-col transition-colors",
|
||||
disabled
|
||||
? "cursor-not-allowed opacity-60"
|
||||
: "hover:bg-muted/40 focus-within:ring-ring cursor-pointer focus-within:ring-2",
|
||||
)
|
||||
}
|
||||
|
||||
/**
|
||||
* KPI-like quick actions strip (horizontal Frame tiles).
|
||||
* Preview: https://reui.io/preview/base/stats-12 · https://reui.io/preview/base/card-12
|
||||
* Docs: https://reui.io/docs/components/base/frame · https://reui.io/docs/components/base/icon-tile
|
||||
*/
|
||||
export function QuickActionGrid({
|
||||
actions,
|
||||
title = "Быстрые действия",
|
||||
description,
|
||||
className,
|
||||
}: QuickActionGridProps) {
|
||||
if (actions.length === 0) return null
|
||||
|
||||
return (
|
||||
<Frame dense spacing="sm" className={cn("@container w-full", className)}>
|
||||
{(title || description) && (
|
||||
<FrameHeader>
|
||||
{title ? <FrameTitle>{title}</FrameTitle> : null}
|
||||
{description ? <FrameDescription>{description}</FrameDescription> : null}
|
||||
</FrameHeader>
|
||||
)}
|
||||
<div className={cn("grid gap-2", kpiCols(actions.length))}>
|
||||
{actions.map((action) => {
|
||||
const label = `${action.title}: ${action.description}`
|
||||
|
||||
if ("href" in action && action.href) {
|
||||
return (
|
||||
<FramePanel key={action.id} className={panelClassName(action.disabled)}>
|
||||
{action.disabled ? (
|
||||
<div aria-disabled aria-label={label}>
|
||||
<QuickActionBody action={action} />
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
<QuickActionBody action={action} />
|
||||
<Link
|
||||
href={action.href}
|
||||
className="absolute inset-0 z-20 focus-visible:outline-none"
|
||||
aria-label={label}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</FramePanel>
|
||||
)
|
||||
}
|
||||
|
||||
const onClick = action.onClick
|
||||
const onActivate = () => {
|
||||
if (action.disabled || !onClick) return
|
||||
onClick()
|
||||
}
|
||||
|
||||
return (
|
||||
<FramePanel
|
||||
key={action.id}
|
||||
className={panelClassName(action.disabled)}
|
||||
role="button"
|
||||
tabIndex={action.disabled ? -1 : 0}
|
||||
aria-disabled={action.disabled || undefined}
|
||||
aria-label={label}
|
||||
onClick={onActivate}
|
||||
onKeyDown={(e) => handleActionKeyDown(onActivate, e)}
|
||||
>
|
||||
<QuickActionBody action={action} />
|
||||
</FramePanel>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
</Frame>
|
||||
)
|
||||
}
|
||||
@@ -70,63 +70,72 @@ export function TrafficRxTxChart({
|
||||
rx,
|
||||
tx,
|
||||
range = "1h",
|
||||
embedded = false,
|
||||
}: {
|
||||
rx: number[]
|
||||
tx: number[]
|
||||
range?: string
|
||||
/** Skip outer Frame when already inside OpsPanel / Frame. */
|
||||
embedded?: boolean
|
||||
}) {
|
||||
const rangeMinutes = TRAFFIC_RANGE_MINUTES[range] ?? 60
|
||||
const data = toChartData(rx, tx, rangeMinutes)
|
||||
const tickEvery = Math.max(1, Math.ceil(data.length / 6))
|
||||
|
||||
const chart = (
|
||||
<div className="flex flex-col gap-4">
|
||||
<ChartContainer config={chartConfig} className="-ms-4 aspect-auto h-[220px] w-full">
|
||||
<LineChart data={data} margin={{ top: 5, right: 5, left: 5, bottom: 5 }}>
|
||||
<CartesianGrid
|
||||
strokeDasharray="4 8"
|
||||
vertical={false}
|
||||
stroke="var(--border)"
|
||||
/>
|
||||
<XAxis
|
||||
dataKey="time"
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickMargin={10}
|
||||
interval={tickEvery - 1}
|
||||
/>
|
||||
<YAxis
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickFormatter={(v: number) => fmtRate(Number(v))}
|
||||
tickMargin={8}
|
||||
width={72}
|
||||
/>
|
||||
<ChartTooltip content={<CustomTooltip />} />
|
||||
<Line
|
||||
dataKey="rx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-rx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
<Line
|
||||
dataKey="tx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-tx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
</LineChart>
|
||||
</ChartContainer>
|
||||
<div className="mb-1 flex items-center justify-center gap-6">
|
||||
<ChartLegendItem label="RX (входящий)" color="var(--chart-rx)" />
|
||||
<ChartLegendItem label="TX (исходящий)" color="var(--chart-tx)" />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
|
||||
if (embedded) return chart
|
||||
|
||||
return (
|
||||
<Frame className="w-full">
|
||||
<FramePanel className="flex flex-col gap-6">
|
||||
<ChartContainer config={chartConfig} className="-ms-4 aspect-auto h-[220px] w-full">
|
||||
<LineChart data={data} margin={{ top: 5, right: 5, left: 5, bottom: 5 }}>
|
||||
<CartesianGrid
|
||||
strokeDasharray="4 8"
|
||||
vertical={false}
|
||||
stroke="var(--border)"
|
||||
/>
|
||||
<XAxis
|
||||
dataKey="time"
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickMargin={10}
|
||||
interval={tickEvery - 1}
|
||||
/>
|
||||
<YAxis
|
||||
axisLine={false}
|
||||
tickLine={false}
|
||||
tick={{ fontSize: 11 }}
|
||||
tickFormatter={(v: number) => fmtRate(Number(v))}
|
||||
tickMargin={8}
|
||||
width={72}
|
||||
/>
|
||||
<ChartTooltip content={<CustomTooltip />} />
|
||||
<Line
|
||||
dataKey="rx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-rx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
<Line
|
||||
dataKey="tx"
|
||||
type="monotone"
|
||||
stroke="var(--chart-tx)"
|
||||
strokeWidth={2}
|
||||
dot={false}
|
||||
/>
|
||||
</LineChart>
|
||||
</ChartContainer>
|
||||
<div className="mb-1 flex items-center justify-center gap-6">
|
||||
<ChartLegendItem label="RX (входящий)" color="var(--chart-rx)" />
|
||||
<ChartLegendItem label="TX (исходящий)" color="var(--chart-tx)" />
|
||||
</div>
|
||||
</FramePanel>
|
||||
<FramePanel className="flex flex-col gap-6">{chart}</FramePanel>
|
||||
</Frame>
|
||||
)
|
||||
}
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user