Compare commits

...
14 Commits
Author SHA1 Message Date
DenozordecandCursor a228febc27 fix(services): сохранить доп. FQDN у IP при пуле из одного адреса
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 9s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m10s
CD / quality (push) Successful in 1m23s
CD / publish (push) Successful in 1m41s
При одном IP общий и доп. FQDN неотличимы по target_ips; после сохранения гидратация относила оба к общим доменам.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-31 12:19:53 +07:00
DenozordecandCursor 5cf39880f8 fix(services): сразу возвращать IP в DNS после первой успешной пробы
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 8s
quality / changes (push) Successful in 8s
quality / web (push) Skipped
quality / docker-check (push) Skipped
quality / api (push) Successful in 1m20s
CD / quality (push) Successful in 1m31s
CD / publish (push) Successful in 2m4s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 19:51:23 +07:00
DenozordecandCursor f1443f1db5 fix(services): не балансировать сервис с одним IP
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / changes (push) Successful in 8s
quality / docker-check (push) Skipped
quality / web (push) Successful in 55s
quality / api (push) Successful in 52s
CD / quality (push) Successful in 2m4s
CD / publish (push) Successful in 1m51s
KPI и карточка показывают живой OK, а не гистерезис unknown; DNS по-прежнему ждёт повторные успехи.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 18:13:06 +07:00
DenozordecandCursor a9a84fabac fix(services): передать режим балансировки в панель пула
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 7s
quality / changes (push) Successful in 10s
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m23s
quality / api (push) Successful in 59s
CD / quality (push) Successful in 2m36s
CD / publish (push) Successful in 1m58s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:52:21 +07:00
DenozordecandCursor 2b150fa3a6 fix(services): снимать Down только с общего FQDN
Персональные A не крутить и не трогать при падении. Панель называется по режиму балансировки.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:51:49 +07:00
DenozordecandCursor 3d0ea33baf feat(services): крутить weighted как доли времени на одном IP
quality / changes (push) Successful in 9s
quality / commitlint (push) Skipped
quality / docker-check (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / web (push) Successful in 55s
quality / api (push) Successful in 45s
CD / quality (push) Successful in 1m57s
CD / publish (push) Successful in 1m36s
Веса задают долю слотов на общем FQDN (1 к 3 = ¼ и ¾ цикла), TTL 60.
В селекте режима показывать текстовое имя, не ID.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:35:13 +07:00
DenozordecandCursor 8a13888db7 fix(services): вернуть вес и приоритет в режиме балансировки
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 8s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 54s
CD / quality (push) Successful in 1m5s
CD / publish (push) Successful in 1m43s
После объединения адресов в форме пропали per-IP поля. Weighted снова задаёт вес, failover — основной и запасной.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:06:29 +07:00
DenozordecandCursor f5d97e463a fix(services): показывать живой health и историю выхода из пула
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 8s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m2s
CD / quality (push) Successful in 1m14s
CD / publish (push) Successful in 2m3s
Таблица IP брала hysteresis unknown вместо последней пробы. Failover теперь показывает выход и возврат в пул, а не только DNS add.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 16:43:22 +07:00
DenozordecandCursor be8f94143f fix(health): не затирать график uptime пробой down в том же бакете
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / changes (push) Successful in 8s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 55s
CD / quality (push) Successful in 1m7s
CD / publish (push) Successful in 1m38s
Если в минутном бакете есть живой IP, линия строится по его пингу, а не по null от Down.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 16:07:24 +07:00
DenozordecandCursor fa7d2b6df3 feat(services): считать сервис живым при одном OK и писать историю failover
quality / commitlint (push) Skipped
quality / changes (push) Successful in 11s
quality / docker-check (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / web (push) Successful in 1m11s
quality / api (push) Successful in 1m1s
CD / quality (push) Successful in 2m30s
CD / publish (push) Successful in 2m4s
Сервис up, если жив хотя бы один IP. Failover показывает Down по FQDN и журнал add/remove A-записей.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 15:49:17 +07:00
DenozordecandCursor 7938d2f707 fix(services): считать failover по IP Health а не по строке ноды
quality / commitlint (push) Skipped
quality / changes (push) Successful in 9s
quality / docker-check (push) Skipped
CD / update-wiki (push) Successful in 4s
quality / web (push) Successful in 52s
quality / api (push) Successful in 42s
CD / quality (push) Successful in 1m50s
CD / publish (push) Successful in 1m43s
Панель брала service_nodes, которую затирал group apply.
Теперь тот же binding ip_health, что таблица активов; group не пишет в ноду.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 15:08:10 +07:00
DenozordecandCursor 994e79e118 fix(services): считать failover по DNS-пулу а не по статусу ноды
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 7s
quality / changes (push) Successful in 9s
quality / api (push) Skipped
quality / docker-check (push) Skipped
CD / quality (push) Canceled after 34s
CD / publish (push) Canceled after 0s
quality / web (push) Canceled after 22s
Красный инцидент только если адреса нет в active_addresses, даже при unhealthy.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 14:53:44 +07:00
DenozordecandCursor 87b0f1a894 fix(services): оформить failover timeline в стиле страницы сервиса
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / changes (push) Successful in 9s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 53s
CD / quality (push) Successful in 1m6s
CD / publish (push) Successful in 1m39s
Frame stacked как у мониторинга, Alert и semantic timeline вместо плоского степпера.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 14:31:30 +07:00
DenozordecandCursor ba4e04a224 feat(services): заменить один общий FQDN списком и убрать Другие FQDN
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 9s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m6s
CD / quality (push) Successful in 1m18s
CD / publish (push) Successful in 2m1s
Каждый общий домен смотрит на весь пул IP; CNAME без UI сохраняются.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 14:21:15 +07:00
45 changed files with 3243 additions and 524 deletions
+5
View File
@@ -38,6 +38,10 @@ import {
healthEngineFallbacksFromConfig,
scheduleHealthCheckJob,
} from "./services/health-check-scheduler.js";
import {
createWeightedDnsTask,
scheduleWeightedDnsJob,
} from "./services/weighted-dns-scheduler.js";
import { fireEnsureHealthWorker } from "./services/health/health-worker-deploy.js";
import { AsyncTask, CronJob } from "toad-scheduler";
@@ -133,6 +137,7 @@ export async function buildApp(opts: BuildAppOptions = {}) {
app.decorate("reloadHealthCheckJob", () => {
scheduleHealthCheckJob(app, config, healthTask);
});
scheduleWeightedDnsJob(app, createWeightedDnsTask(app));
if (config.cloudflareApiToken) {
fireEnsureHealthWorker(
app.db,
+12
View File
@@ -78,6 +78,18 @@ export async function serviceRoutes(app: FastifyInstance) {
};
});
app.get("/services/:id/failover-log", async (request) => {
const { id } = request.params as { id: string };
repos.getService(request.server.db, Number(id));
return {
items: repos.listFailoverLogForService(
request.server.db,
Number(id),
200,
),
};
});
app.get("/services/:id/certificates", async (request) => {
const { id } = request.params as { id: string };
return certificateService.listServiceCertificates(
@@ -290,7 +290,7 @@ export interface RunAllChecksOptions {
target: HealthCheckTarget,
prevState: IpHealthState | null,
nextState: IpHealthState,
) => void;
) => void | Promise<void>;
}
function sleep(ms: number): Promise<void> {
@@ -326,7 +326,7 @@ function logSourceResult(
});
}
function applyAggregatedStatus(
async function applyAggregatedStatus(
db: Db,
target: HealthCheckTarget,
sources: Array<{ provider: HealthCheckProvider; result: ProbeResult }>,
@@ -386,7 +386,8 @@ function applyAggregatedStatus(
{ colo, provider: statusProvider },
);
const matchedNode = repos.findNodeByIp(db, target.ip);
if (matchedNode && matchedNode.enabled) {
// Binding-scope only: group apply must not clobber node with its own fetch failed.
if (matchedNode && matchedNode.enabled && target.scope === "binding") {
repos.updateNode(db, matchedNode.id, {
health_status: node,
consecutive_failures: failures,
@@ -396,7 +397,7 @@ function applyAggregatedStatus(
});
}
if (prevState !== state) {
options.onStatusChange?.(target, prevState, state);
await options.onStatusChange?.(target, prevState, state);
}
}
@@ -512,7 +513,7 @@ export async function runAllChecks(
for (const source of sources) {
logSourceResult(db, target, source.provider, source.result);
}
applyAggregatedStatus(db, target, sources, options);
await applyAggregatedStatus(db, target, sources, options);
}
}
+19 -6
View File
@@ -9,7 +9,10 @@ import type {
import { AppError } from "../errors.js";
import { isValidIpv4 } from "../lib/validators.js";
import { getView } from "./service-config-service.js";
import { selectActiveIpsByMode } from "./routing/index.js";
import {
isSharedPool,
resolveDesiredAIps,
} from "./routing/index.js";
function assertAddress(address: string): void {
if (!isValidIpv4(address)) {
@@ -89,8 +92,12 @@ export async function getOverview(
: null;
const active = new Set<string>();
const serviceIps = service.ips.filter(
(ip) => service.ip_enabled[ip] !== false,
);
for (const binding of bindings) {
const metas = repos.listBindingIpsWithMeta(db, binding.id);
const targetIps = metas.map((entry) => entry.ip);
const rows = metas.map((entry) => {
const status = repos.getIpHealthStatusRow(db, "binding", binding.id, entry.ip);
return {
@@ -100,12 +107,15 @@ export async function getOverview(
health: status ? status.status : ("unknown" as const),
};
});
for (const ip of selectActiveIpsByMode(
for (const ip of resolveDesiredAIps(
{
lb_mode: binding.lb_mode,
health_check_enabled: binding.health_check_enabled,
},
rows,
targetIps,
Date.now(),
serviceIps,
)) {
active.add(ip);
}
@@ -134,10 +144,13 @@ export function opsSummary(db: Db) {
if (binding.lb_mode !== "failover" || !binding.health_check_enabled) {
return false;
}
return binding.target_ips.some((ip) => {
const row = repos.getIpHealthStatusRow(db, "binding", binding.id, ip);
return row?.status === "down";
});
return (
isSharedPool(binding.target_ips) &&
binding.target_ips.some((ip) => {
const row = repos.getIpHealthStatusRow(db, "binding", binding.id, ip);
return row?.status === "down";
})
);
}).length;
return {
domains: domains.length,
+4 -4
View File
@@ -1,16 +1,16 @@
import type { LbIpRow } from "./types.js";
import { isHealthy } from "./health.js";
import { isPoolMember } from "./health.js";
export function failoverDesired(rows: LbIpRow[]): string[] {
if (rows.length === 0) return [];
const healthy = rows.filter((r) => isHealthy(r.health));
const pool = healthy.length > 0 ? healthy : rows;
const live = rows.filter((r) => isPoolMember(r.health));
const pool = live.length > 0 ? live : rows;
const sorted = [...pool].sort(
(a, b) => a.priority - b.priority || a.weight - b.weight,
);
const minPriority = sorted[0]!.priority;
const primaries = sorted.filter((r) => r.priority === minPriority);
if (healthy.length > 0) {
if (live.length > 0) {
return primaries.map((r) => r.ip);
}
return [sorted[0]!.ip];
+9
View File
@@ -3,3 +3,12 @@ import type { IpHealthState, NodeHealthState } from "@cfdm/shared";
export function isHealthy(state: IpHealthState | NodeHealthState | string): boolean {
return state === "up" || state === "healthy";
}
export function isDown(state: IpHealthState | NodeHealthState | string): boolean {
return state === "down" || state === "unhealthy";
}
/** A-pool membership: only Down is drained. Recovering (unknown/checking) and Slow return immediately. */
export function isPoolMember(state: IpHealthState | NodeHealthState | string): boolean {
return !isDown(state);
}
+32 -3
View File
@@ -1,26 +1,55 @@
import type { LbMode } from "@cfdm/shared";
import { failoverDesired } from "./failover.js";
import { canApplyLb, isSharedPool } from "./pool.js";
import { roundRobinDesired } from "./round-robin.js";
import type { LbIpRow, LbTargetConfig } from "./types.js";
import { weightedDesired } from "./weighted.js";
export type { LbIpRow, LbTargetConfig } from "./types.js";
export { isHealthy } from "./health.js";
export { isDown, isHealthy, isPoolMember } from "./health.js";
export { withBindingLock } from "./binding-lock.js";
export {
canApplyLb,
isSharedPool,
shouldRecordFailoverDnsDiff,
uniqueIpCount,
} from "./pool.js";
export { WEIGHTED_DNS_TTL, WEIGHTED_SLOT_MS, weightedDesired } from "./weighted.js";
export function selectActiveIpsByMode(
config: LbTargetConfig,
rows: LbIpRow[],
nowMs = Date.now(),
): string[] {
if (rows.length === 0) return [];
if (config.lb_mode === "failover") {
return failoverDesired(rows);
}
// weighted = round_robin on DNS (one A per IP)
if (config.lb_mode === "weighted") {
return weightedDesired(rows, nowMs);
}
return roundRobinDesired(rows);
}
export function resolveDesiredAIps(
config: LbTargetConfig,
rows: LbIpRow[],
fallbackIps: readonly string[],
nowMs = Date.now(),
serviceIps: readonly string[] = fallbackIps,
): string[] {
const fallback = [...fallbackIps];
if (!canApplyLb(serviceIps, fallback)) return fallback;
if (!isSharedPool(rows.map((row) => row.ip))) return fallback;
if (config.lb_mode === "weighted" || config.health_check_enabled) {
const activeIps = selectActiveIpsByMode(config, rows, nowMs);
if (activeIps.length > 0) return activeIps;
}
return fallback;
}
export function strategyLabel(mode: LbMode): string {
if (mode === "failover") return "Failover";
if (mode === "weighted") return "Round Robin (weighted alias)";
if (mode === "weighted") return "Weighted";
return "Round Robin";
}
+30
View File
@@ -0,0 +1,30 @@
import type { LbMode } from "@cfdm/shared";
export function uniqueIpCount(ips: readonly string[]): number {
return new Set(ips.filter(Boolean)).size;
}
/** Shared pool — two or more unique IPs. One IP (even duplicated) is not a pool. */
export function isSharedPool(ips: readonly string[]): boolean {
return uniqueIpCount(ips) >= 2;
}
/** LB / drain only when the service itself has a pool AND this FQDN is shared. */
export function canApplyLb(
serviceIps: readonly string[],
bindingIps: readonly string[],
): boolean {
return isSharedPool(serviceIps) && isSharedPool(bindingIps);
}
export function shouldRecordFailoverDnsDiff(input: {
configuredIps: readonly string[];
lbMode: LbMode;
added: readonly string[];
removed: readonly string[];
downIps: ReadonlySet<string>;
}): boolean {
if (!isSharedPool(input.configuredIps)) return false;
if (input.lbMode !== "weighted") return true;
return [...input.added, ...input.removed].some((ip) => input.downIps.has(ip));
}
+3 -3
View File
@@ -1,8 +1,8 @@
import type { LbIpRow } from "./types.js";
import { isHealthy } from "./health.js";
import { isPoolMember } from "./health.js";
export function roundRobinDesired(rows: LbIpRow[]): string[] {
const healthy = rows.filter((r) => isHealthy(r.health));
const pool = healthy.length > 0 ? healthy : rows;
const live = rows.filter((r) => isPoolMember(r.health));
const pool = live.length > 0 ? live : rows;
return pool.map((r) => r.ip);
}
+24
View File
@@ -0,0 +1,24 @@
import type { LbIpRow } from "./types.js";
import { isPoolMember } from "./health.js";
/** Slot length for time-sliced weighted DNS (one A at a time). */
export const WEIGHTED_SLOT_MS = 60_000;
/** Cloudflare DNS-only minimum TTL; Auto (1) is ~300s and would smear ratios. */
export const WEIGHTED_DNS_TTL = 60;
export function weightedDesired(rows: LbIpRow[], nowMs = Date.now()): string[] {
if (rows.length === 0) return [];
const live = rows.filter((r) => isPoolMember(r.health));
const pool = live.length > 0 ? live : rows;
if (pool.length === 1) return [pool[0]!.ip];
const sorted = [...pool].sort((a, b) => a.ip.localeCompare(b.ip));
const cycle: string[] = [];
for (const row of sorted) {
const weight = Math.max(1, Math.round(row.weight));
for (let i = 0; i < weight; i++) cycle.push(row.ip);
}
const slot = Math.floor(nowMs / WEIGHTED_SLOT_MS) % cycle.length;
return [cycle[slot]!];
}
+269 -47
View File
@@ -29,15 +29,91 @@ import * as domainService from "./domain-service.js";
import { syncServiceToVpsTracker } from "./vps-tracker-sync.js";
import { fireEnsureHealthWorker, DEFAULT_HEALTH_FALLBACKS } from "./health/health-worker-deploy.js";
import {
isHealthy,
canApplyLb,
isPoolMember,
isSharedPool,
resolveDesiredAIps,
selectActiveIpsByMode,
shouldRecordFailoverDnsDiff,
withBindingLock,
WEIGHTED_DNS_TTL,
type LbIpRow,
type LbTargetConfig,
} from "./routing/index.js";
export type { LbIpRow, LbTargetConfig };
export { selectActiveIpsByMode };
export {
canApplyLb,
resolveDesiredAIps,
selectActiveIpsByMode,
shouldRecordFailoverDnsDiff,
};
const AUTO_DNS_TTL = 1;
function ttlForBinding(mode: LbMode, ips: readonly string[]): number {
return mode === "weighted" && isSharedPool(ips) ? WEIGHTED_DNS_TTL : AUTO_DNS_TTL;
}
function enabledServiceIps(db: Db, serviceId: number): string[] {
return repos
.listServiceIpRows(db, serviceId)
.filter((row) => row.enabled)
.map((row) => row.ip);
}
export function failoverARecordDiff(
existingA: readonly string[],
desiredIps: readonly string[],
): { added: string[]; removed: string[] } {
const before = new Set(existingA);
const after = new Set(desiredIps);
return {
added: desiredIps.filter((ip) => !before.has(ip)),
removed: existingA.filter((ip) => !after.has(ip)),
};
}
function recordFailoverDnsDiff(
db: Db,
bindingId: number,
hostname: string,
zoneName: string,
existingRecords: DnsRecord[],
desiredIps: string[],
): void {
const existingA = existingRecords
.filter((record) => record.record_type.toUpperCase() === "A")
.map((record) => record.content);
const { added, removed } = failoverARecordDiff(existingA, desiredIps);
if (added.length === 0 && removed.length === 0) return;
const binding = repos.getBinding(db, bindingId);
const configuredIps = repos.listBindingIps(db, bindingId);
const { config, rows } = getBindingLbState(db, bindingId);
const downIps = new Set(
rows.filter((row) => row.health === "down").map((row) => row.ip),
);
if (
!shouldRecordFailoverDnsDiff({
configuredIps,
lbMode: config.lb_mode,
added,
removed,
downIps,
})
) {
return;
}
repos.insertFailoverLog(db, {
serviceId: binding.service_id,
bindingId,
fqdn: fqdnToDisplay(hostname, zoneName),
entries: [
...added.map((ip) => ({ ip, action: "added" as const })),
...removed.map((ip) => ({ ip, action: "removed" as const })),
],
});
}
export interface ServiceDomainInput {
fqdn: string;
@@ -211,7 +287,7 @@ function getGroupLbState(
} else {
existing.weight += weight;
existing.priority = Math.min(existing.priority, priority);
if (isHealthy(existing.health) && status && !isHealthy(status.status as IpHealthState)) {
if (isPoolMember(existing.health) && status && !isPoolMember(status.status as IpHealthState)) {
existing.health = status.status as IpHealthState;
}
}
@@ -227,16 +303,27 @@ function getGroupLbState(
};
}
function computeActiveIps(
function desiredAIps(
db: Db,
scope: HealthCheckScope,
refId: number,
fallbackIps: string[],
): string[] {
const state =
scope === "binding"
? getBindingLbState(db, refId)
: getGroupLbState(db, refId);
return selectActiveIpsByMode(state.config, state.rows);
const serviceIps =
scope === "binding"
? enabledServiceIps(db, repos.getBinding(db, refId).service_id)
: fallbackIps;
return resolveDesiredAIps(
state.config,
state.rows,
fallbackIps,
Date.now(),
serviceIps,
);
}
async function collectKnownZones(
@@ -285,6 +372,11 @@ async function buildView(db: Db, serviceId: number): Promise<ServiceView> {
if (target_ip_priorities[ip] === undefined) target_ip_priorities[ip] = 1;
}
const { config, rows } = getBindingLbState(db, binding.id);
const bindingActiveIps = targetCname
? []
: resolveDesiredAIps(config, rows, targetIps, Date.now(), ips);
return {
binding_id: binding.id,
domain_id: binding.domain_id,
@@ -312,13 +404,13 @@ async function buildView(db: Db, serviceId: number): Promise<ServiceView> {
health_check_aggregate: binding.health_check_aggregate ?? "majority",
cert_monitoring: binding.cert_monitoring ?? "auto",
sync_status: aggregateSyncStatus(statuses),
active_ips: bindingActiveIps,
};
});
const activeIps = new Set<string>();
for (const binding of bindings) {
const { config, rows } = getBindingLbState(db, binding.id);
for (const ip of selectActiveIpsByMode(config, rows)) {
for (const domain of domainViews) {
for (const ip of domain.active_ips) {
activeIps.add(ip);
}
}
@@ -404,6 +496,21 @@ function fallbackCnameHealth(
);
}
function overlayLiveHealth(
stored: IpHealthState | undefined,
live: IpHealthState | undefined,
): IpHealthState {
if (live && live !== "unknown") return live;
return stored ?? live ?? "unknown";
}
function bestAliveDisplayStatus(statuses: readonly string[]): IpHealthState {
if (statuses.some((status) => status === "up")) return "up";
if (statuses.some((status) => status === "degraded")) return "degraded";
if (statuses.some((status) => status === "down")) return "down";
return "unknown";
}
function attachServiceHealth(
db: Db,
views: ServiceView[],
@@ -411,10 +518,13 @@ function attachServiceHealth(
const ids = views.map((v) => v.id);
const healthByService = repos.aggregateIpHealthByServiceIds(db, ids);
const ipHealthByService = repos.listIpHealthByServiceIds(db, ids);
const liveByService = repos.listLatestLiveHealthByServiceIds(db, ids);
return views.map((view) => {
const health = healthByService.get(view.id);
const rows = ipHealthByService.get(view.id) ?? [];
const liveRows = liveByService.get(view.id) ?? [];
const byIp = new Map(rows.map((row) => [row.ip, row]));
const liveByIp = new Map(liveRows.map((row) => [row.ip, row]));
const cnameFallback = fallbackCnameHealth(rows, view);
const aRecordIps = new Set(
(view.domains ?? []).flatMap((domain) =>
@@ -423,20 +533,33 @@ function attachServiceHealth(
);
const ip_health = (view.ips ?? []).map((ip) => {
const row = byIp.get(ip) ?? (aRecordIps.has(ip) ? undefined : cnameFallback);
const live = liveByIp.get(ip);
const status = overlayLiveHealth(row?.status, live?.status);
const extras = live && live.status !== "unknown" ? live : row;
return {
ip,
status: row?.status ?? ("unknown" as const),
latency_ms: row?.latency_ms ?? null,
last_checked_at: row?.last_checked_at ?? null,
last_error: row?.last_error ?? null,
provider: row?.provider ?? "local",
colo: row?.colo ?? null,
status,
latency_ms: extras?.latency_ms ?? null,
last_checked_at: extras?.last_checked_at ?? null,
last_error:
live && live.status !== "unknown"
? live.last_error
: (row?.last_error ?? null),
provider: extras?.provider ?? "local",
colo: extras?.colo ?? null,
};
});
const displayStatus = bestAliveDisplayStatus(ip_health.map((row) => row.status));
const latencyRow =
ip_health.find((row) => row.status === displayStatus && row.latency_ms != null) ??
ip_health.find((row) => row.latency_ms != null);
return {
...view,
health_status: health?.health_status ?? "unknown",
health_latency_ms: health?.health_latency_ms ?? null,
health_status: overlayLiveHealth(health?.health_status, displayStatus),
health_latency_ms:
displayStatus !== "unknown"
? (latencyRow?.latency_ms ?? null)
: (health?.health_latency_ms ?? null),
ip_health,
};
});
@@ -571,7 +694,17 @@ async function syncBindingDns(
return;
}
await syncBindingADns(db, cf, bindingId, domainId, hostname, desiredIps);
const binding = repos.getBinding(db, bindingId);
const configuredIps = repos.listBindingIps(db, bindingId);
await syncBindingADns(
db,
cf,
bindingId,
domainId,
hostname,
desiredIps,
ttlForBinding(binding.lb_mode, configuredIps),
);
}
async function syncBindingCnameDns(
@@ -658,6 +791,7 @@ async function syncBindingADns(
domainId: number,
hostname: string,
desiredIps: string[],
ttl: number,
): Promise<void> {
const domain = repos.getDomain(db, domainId);
const zoneName = domain.zone_name;
@@ -677,6 +811,14 @@ async function syncBindingADns(
if (desiredIps.length === 0) {
repos.setBindingDnsRecordId(db, bindingId, null);
recordFailoverDnsDiff(
db,
bindingId,
hostname,
zoneName,
existingRecords,
desiredIps,
);
return;
}
@@ -685,13 +827,15 @@ async function syncBindingADns(
for (const ip of desiredIps) {
const existing = refreshed.find((r) => r.content === ip);
const recordName = dnsNameForBinding(hostname, zoneName);
let recordId: number;
if (existing) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName)) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName) || existing.ttl !== ttl) {
await dnsService.update(db, cf, domainId, existing.id, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
@@ -709,12 +853,24 @@ async function syncBindingADns(
if (adopted) {
repos.linkBindingRecord(db, bindingId, adopted.id);
recordId = adopted.id;
if (
!dnsRecordNamesMatch(adopted.name, hostname, zoneName) ||
adopted.ttl !== ttl
) {
await dnsService.update(db, cf, domainId, adopted.id, {
record_type: "A",
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
} else {
const record = await dnsService.create(db, cf, domainId, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl: 1,
ttl,
proxied: false,
});
repos.linkBindingRecord(db, bindingId, record.id);
@@ -725,6 +881,14 @@ async function syncBindingADns(
}
repos.setBindingDnsRecordId(db, bindingId, primaryId);
recordFailoverDnsDiff(
db,
bindingId,
hostname,
zoneName,
existingRecords,
desiredIps,
);
}
async function cleanupBindingDns(
@@ -943,12 +1107,7 @@ async function syncServiceBindingsToDns(
}
validateTargetIpsInPool(targetIps, ips);
if (binding.health_check_enabled) {
const activeIps = computeActiveIps(db, "binding", binding.id);
if (activeIps.length > 0) {
targetIps = activeIps;
}
}
const desiredIps = desiredAIps(db, "binding", binding.id, targetIps);
await syncBindingDns(
db,
@@ -956,7 +1115,7 @@ async function syncServiceBindingsToDns(
binding.id,
binding.domain_id,
binding.hostname,
targetIps,
desiredIps,
null,
);
}
@@ -988,6 +1147,7 @@ async function syncGroupDomainDnsRecords(
domainId: number,
hostname: string,
desiredIps: string[],
ttl: number = AUTO_DNS_TTL,
): Promise<void> {
const domain = repos.getDomain(db, domainId);
const zoneName = domain.zone_name;
@@ -1003,14 +1163,16 @@ async function syncGroupDomainDnsRecords(
if (desiredIps.length === 0) return;
const refreshed = repos.listGroupDnsRecords(db, groupId);
const recordName = dnsNameForBinding(hostname, zoneName);
for (const ip of desiredIps) {
const existing = refreshed.find((r) => r.content === ip);
if (existing) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName)) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName) || existing.ttl !== ttl) {
await dnsService.update(db, cf, domainId, existing.id, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
@@ -1026,13 +1188,25 @@ async function syncGroupDomainDnsRecords(
);
if (adopted) {
repos.linkGroupDnsRecord(db, groupId, adopted.id);
if (
!dnsRecordNamesMatch(adopted.name, hostname, zoneName) ||
adopted.ttl !== ttl
) {
await dnsService.update(db, cf, domainId, adopted.id, {
record_type: "A",
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
continue;
}
const record = await dnsService.create(db, cf, domainId, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl: 1,
ttl,
proxied: false,
});
repos.linkGroupDnsRecord(db, groupId, record.id);
@@ -1083,9 +1257,8 @@ async function syncGroupDomainDns(
const knownZones = await collectKnownZones(db, cf);
const { zoneName, hostname } = parseFqdn(domainValue, knownZones);
const domainId = await resolveDomainId(db, cf, zoneName);
const desiredIps = group.health_check_enabled
? computeActiveIps(db, "group", groupId)
: await collectGroupDnsIps(db, groupId);
const fallbackIps = await collectGroupDnsIps(db, groupId);
const desiredIps = desiredAIps(db, "group", groupId, fallbackIps);
await syncGroupDomainDnsRecords(
db,
cf,
@@ -1093,6 +1266,7 @@ async function syncGroupDomainDns(
domainId,
hostname,
desiredIps,
ttlForBinding(group.lb_mode, fallbackIps),
);
}
@@ -1273,14 +1447,7 @@ export async function updateConfig(
}
if (pushDns) {
let effectiveIps = targetIps;
const refreshedBinding = repos.getBinding(db, binding.id);
if (refreshedBinding.health_check_enabled) {
const activeIps = computeActiveIps(db, "binding", binding.id);
if (activeIps.length > 0) {
effectiveIps = activeIps;
}
}
const effectiveIps = desiredAIps(db, "binding", binding.id, targetIps);
await syncBindingDns(
db,
cf,
@@ -1586,16 +1753,18 @@ export async function reconcileDnsForTarget(
if (scope === "binding") {
await withBindingLock(refId, async () => {
const binding = repos.getBinding(db, refId);
if (!binding.health_check_enabled) return;
if (!binding.health_check_enabled && binding.lb_mode !== "weighted") return;
const service = repos.getService(db, binding.service_id);
if (!shouldPushDns(db, service)) return;
const cnameTarget = binding.cname_target?.trim() || null;
if (cnameTarget) return;
const ips = repos.listServiceIps(db, service.id);
const poolIps = enabledServiceIps(db, service.id);
const targetIps = repos.listBindingIps(db, binding.id);
validateTargetIpsInPool(targetIps, ips);
const activeIps = computeActiveIps(db, "binding", refId);
const desiredIps = activeIps.length > 0 ? activeIps : targetIps;
const desiredIps = canApplyLb(poolIps, targetIps)
? desiredAIps(db, "binding", refId, targetIps)
: targetIps;
await syncBindingDns(
db,
cf,
@@ -1610,8 +1779,61 @@ export async function reconcileDnsForTarget(
}
const group = repos.getServiceGroup(db, refId);
if (!group.enabled || !group.domain?.trim() || !group.health_check_enabled) {
if (!group.enabled || !group.domain?.trim()) {
return;
}
if (!group.health_check_enabled && group.lb_mode !== "weighted") {
return;
}
await syncGroupDomainDns(db, cf, refId);
}
export async function reconcileWeightedDns(
db: Db,
cf: CloudflareClient,
): Promise<number> {
let n = 0;
for (const binding of repos.listAllBindings(db)) {
if (binding.lb_mode !== "weighted") continue;
if (binding.cname_target?.trim()) continue;
if (!isSharedPool(binding.target_ips ?? [])) continue;
try {
await withBindingLock(binding.id, async () => {
const latest = repos.getBinding(db, binding.id);
if (latest.lb_mode !== "weighted") return;
if (latest.cname_target?.trim()) return;
const service = repos.getService(db, latest.service_id);
if (!shouldPushDns(db, service)) return;
const targetIps = repos.listBindingIps(db, latest.id);
const poolIps = enabledServiceIps(db, service.id);
if (!canApplyLb(poolIps, targetIps)) return;
const ips = repos.listServiceIps(db, service.id);
validateTargetIpsInPool(targetIps, ips);
const desiredIps = desiredAIps(db, "binding", latest.id, targetIps);
await syncBindingDns(
db,
cf,
latest.id,
latest.domain_id,
latest.hostname,
desiredIps,
null,
);
n += 1;
});
} catch {
continue;
}
}
for (const group of repos.listServiceGroups(db)) {
if (group.lb_mode !== "weighted") continue;
if (!group.enabled || !group.domain?.trim()) continue;
try {
await syncGroupDomainDns(db, cf, group.id);
n += 1;
} catch {
continue;
}
}
return n;
}
@@ -0,0 +1,39 @@
import type { FastifyInstance } from "fastify";
import { AsyncTask, SimpleIntervalJob } from "toad-scheduler";
import * as serviceConfigService from "./service-config-service.js";
import { WEIGHTED_SLOT_MS } from "./routing/weighted.js";
export const WEIGHTED_DNS_JOB_ID = "weighted-dns";
export function createWeightedDnsTask(app: FastifyInstance): AsyncTask {
return new AsyncTask(
WEIGHTED_DNS_JOB_ID,
async () => {
const n = await serviceConfigService.reconcileWeightedDns(app.db, app.cf);
if (n > 0) {
app.log.info({ reconciled: n }, "weighted dns rotated");
}
},
(err) => {
app.log.warn({ err }, "weighted dns rotate failed");
},
);
}
export function scheduleWeightedDnsJob(
app: FastifyInstance,
task: AsyncTask,
): void {
const scheduler = app.scheduler;
if (!scheduler) return;
if (scheduler.existsById(WEIGHTED_DNS_JOB_ID)) {
scheduler.removeById(WEIGHTED_DNS_JOB_ID);
}
scheduler.addSimpleIntervalJob(
new SimpleIntervalJob(
{ seconds: WEIGHTED_SLOT_MS / 1000, runImmediately: true },
task,
{ id: WEIGHTED_DNS_JOB_ID, preventOverrun: true },
),
);
}
+112
View File
@@ -0,0 +1,112 @@
import { describe, expect, it } from "vitest";
import { createMemoryDb, repos, runMigrations } from "@cfdm/db";
import { buildApp } from "../src/app.js";
import { loadConfig } from "../src/config.js";
import { failoverARecordDiff } from "../src/services/service-config-service.js";
async function authHeaders(app: Awaited<ReturnType<typeof buildApp>>) {
const config = loadConfig();
const res = await app.inject({
method: "POST",
url: "/api/v1/auth/login",
payload: { username: config.adminUsername, password: "admin" },
});
expect(res.statusCode).toBe(200);
const { token } = res.json() as { token: string };
return { authorization: `Bearer ${token}` };
}
describe("failoverARecordDiff", () => {
it("diffs added and removed A contents", () => {
expect(
failoverARecordDiff(
["130.49.213.153", "93.115.203.183"],
["93.115.203.183"],
),
).toEqual({
added: [],
removed: ["130.49.213.153"],
});
expect(failoverARecordDiff(["10.0.0.1"], ["10.0.0.1", "10.0.0.2"])).toEqual({
added: ["10.0.0.2"],
removed: [],
});
});
});
describe("GET /services/:id/failover-log", () => {
it("returns add/remove rows for the service", async () => {
const app = await buildApp({
config: { ...loadConfig(), staticDir: null },
memory: true,
});
const headers = await authHeaders(app);
const domain = repos.createDomain(app.db, null, "rkns.top", "zone-1");
const service = repos.createService(app.db, "MSK Hip", "msk-hip");
const binding = repos.insertBinding(app.db, domain.id, service.id, "gt", null);
repos.insertFailoverLog(app.db, {
serviceId: service.id,
bindingId: binding.id,
fqdn: "gt.rkns.top",
entries: [
{ ip: "130.49.213.153", action: "removed" },
{ ip: "93.115.203.183", action: "added" },
],
});
const res = await app.inject({
method: "GET",
url: `/api/v1/services/${service.id}/failover-log`,
headers,
});
expect(res.statusCode).toBe(200);
const body = res.json() as {
items: Array<{ ip: string; fqdn: string; action: string }>;
};
expect(body.items).toEqual(
expect.arrayContaining([
expect.objectContaining({
ip: "130.49.213.153",
fqdn: "gt.rkns.top",
action: "removed",
}),
expect.objectContaining({
ip: "93.115.203.183",
fqdn: "gt.rkns.top",
action: "added",
}),
]),
);
await app.close();
});
});
describe("failover_log table", () => {
it("lists newest first", () => {
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "example.com", "zone-1");
const service = repos.createService(db, "Panel", "panel");
const binding = repos.insertBinding(db, domain.id, service.id, "panel", null);
repos.insertFailoverLog(db, {
serviceId: service.id,
bindingId: binding.id,
fqdn: "panel.example.com",
entries: [{ ip: "1.1.1.1", action: "removed" }],
});
repos.insertFailoverLog(db, {
serviceId: service.id,
bindingId: binding.id,
fqdn: "panel.example.com",
entries: [{ ip: "1.1.1.1", action: "added" }],
});
const rows = repos.listFailoverLogForService(db, service.id);
expect(rows.map((row) => row.action)).toEqual(["added", "removed"]);
});
});
+152
View File
@@ -291,6 +291,77 @@ describe("health-check state derivation via runAllChecks", () => {
expect(targets[0]?.ip).toBe("2.59.161.102");
expect(targets[0]?.hostname).toBe("s.rkns.top");
});
it("does not mark node unhealthy when binding majority is OK and group local fails", async () => {
const { createMemoryDb, repos, runMigrations } = await import("@cfdm/db");
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const tcp = await startTcpServer();
try {
const domain = repos.createDomain(db, null, "example.com", "zone-id");
const group = repos.createServiceGroup(
db,
"VPN",
"vpn",
null,
"vpn.example.com",
{
health_check_enabled: true,
health_check_type: "http",
health_check_port: 1,
health_check_timeout_ms: 200,
health_check_path: "/",
},
);
const service = repos.createService(db, "Svc", "svc");
repos.setServiceGroup(db, service.id, group.id);
repos.setServiceEnabled(db, service.id, true);
const binding = repos.insertBinding(db, domain.id, service.id, "@", null);
repos.updateBindingLbConfig(db, binding.id, {
health_check_enabled: true,
health_check_type: "tcp",
health_check_port: tcp.port,
health_check_timeout_ms: 500,
});
repos.replaceBindingIpsWithMeta(db, binding.id, [
{ ip: "127.0.0.1", weight: 1, priority: 1 },
]);
const node = repos.findNodeByIp(db, "127.0.0.1");
expect(node).not.toBeNull();
await healthCheckService.runAllChecks(db, {
probeGapMs: 0,
thresholds: {
degradedFailures: 1,
downFailures: 1,
latencyWarnMs: 1000,
},
});
const bindingHealth = repos.getIpHealthStatusRow(
db,
"binding",
binding.id,
"127.0.0.1",
);
const groupHealth = repos.getIpHealthStatusRow(
db,
"group",
group.id,
"127.0.0.1",
);
const after = repos.getNode(db, node!.id);
expect(bindingHealth?.status).toBe("up");
expect(groupHealth?.status).toBe("down");
expect(after.health_status).toBe("healthy");
expect(after.consecutive_failures).toBe(0);
expect(after.last_failure_reason).toBeNull();
} finally {
await new Promise<void>((resolve) => tcp.server.close(() => resolve()));
}
});
});
describe("CNAME health mapped onto service IPs", () => {
@@ -326,4 +397,85 @@ describe("CNAME health mapped onto service IPs", () => {
}),
]);
});
it("getView is up when any binding IP is up", async () => {
const { createMemoryDb, repos, runMigrations } = await import("@cfdm/db");
const { getView } = await import("../src/services/service-config-service.js");
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "rkns.top", "zone-id");
const service = repos.createService(db, "MSK Hip", "msk-hip");
repos.replaceServiceIps(db, service.id, ["10.0.0.1", "10.0.0.2"]);
const binding = repos.insertBinding(db, domain.id, service.id, "gt", null);
repos.replaceBindingIpsWithMeta(db, binding.id, [
{ ip: "10.0.0.1", weight: 1, priority: 1 },
{ ip: "10.0.0.2", weight: 1, priority: 1 },
]);
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"10.0.0.1",
"up",
12,
0,
null,
);
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"10.0.0.2",
"down",
null,
5,
"timeout",
);
const view = await getView(db, service.id);
expect(view.health_status).toBe("up");
});
it("getView shows live OK over hysteresis unknown", async () => {
const { createMemoryDb, repos, runMigrations } = await import("@cfdm/db");
const { getView } = await import("../src/services/service-config-service.js");
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "rkns.top", "zone-id");
const service = repos.createService(db, "RW Panel", "rw-panel");
repos.replaceServiceIps(db, service.id, ["2.59.161.102"]);
const binding = repos.insertBinding(db, domain.id, service.id, "c", null);
repos.replaceBindingIpsWithMeta(db, binding.id, [
{ ip: "2.59.161.102", weight: 1, priority: 1 },
]);
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"2.59.161.102",
"unknown",
63,
0,
null,
1,
);
repos.insertHealthProbeLog(db, {
scope: "binding",
refId: binding.id,
ip: "2.59.161.102",
provider: "local",
status: "up",
ok: true,
latencyMs: 63,
colo: null,
error: null,
});
const view = await getView(db, service.id);
expect(view.health_status).toBe("up");
expect(view.ip_health[0]?.status).toBe("up");
expect(view.ip_health[0]?.latency_ms).toBe(63);
});
});
+200 -12
View File
@@ -1,9 +1,12 @@
import { describe, expect, it } from "vitest";
import {
resolveDesiredAIps,
selectActiveIpsByMode,
shouldRecordFailoverDnsDiff,
type LbIpRow,
type LbTargetConfig,
} from "../src/services/service-config-service.js";
import { WEIGHTED_SLOT_MS } from "../src/services/routing/weighted.js";
function row(
ip: string,
@@ -17,6 +20,11 @@ function row(
};
}
const weightedConfig: LbTargetConfig = {
lb_mode: "weighted",
health_check_enabled: true,
};
describe("selectActiveIpsByMode", () => {
it("round_robin returns all healthy ips, falls back to all if none healthy", () => {
const config: LbTargetConfig = {
@@ -62,6 +70,18 @@ describe("selectActiveIpsByMode", () => {
expect(selectActiveIpsByMode(config, rows)).toEqual(["1.1.1.1"]);
});
it("failover returns recovering unknown primary immediately", () => {
const config: LbTargetConfig = {
lb_mode: "failover",
health_check_enabled: true,
};
const rows = [
row("1.1.1.1", { priority: 1, health: "unknown" }),
row("2.2.2.2", { priority: 2, health: "up" }),
];
expect(selectActiveIpsByMode(config, rows)).toEqual(["1.1.1.1"]);
});
it("failover falls back to min-priority ip among all when none healthy", () => {
const config: LbTargetConfig = {
lb_mode: "failover",
@@ -75,23 +95,61 @@ describe("selectActiveIpsByMode", () => {
expect(selectActiveIpsByMode(config, rows)).toEqual(["2.2.2.2"]);
});
it("weighted returns all healthy ips (one A per ip; weights stored for display)", () => {
const config: LbTargetConfig = {
lb_mode: "weighted",
health_check_enabled: true,
};
it("weighted 1:3 picks the lighter ip on slot 0 and the heavier on slot 1", () => {
const rows = [
row("1.1.1.1", { weight: 3, health: "up" }),
row("2.2.2.2", { weight: 1, health: "up" }),
row("3.3.3.3", { weight: 2, health: "down" }),
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "up" }),
];
expect(selectActiveIpsByMode(config, rows).sort()).toEqual([
"1.1.1.1",
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS)).toEqual([
"2.2.2.2",
]);
});
it("round_robin excludes unknown when another ip is up", () => {
it("weighted excludes down ips from the cycle", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "down" }),
];
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(
selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS),
).toEqual(["1.1.1.1"]);
});
it("weighted includes recovering unknown in the cycle", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "unknown" }),
];
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(
selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS),
).toEqual(["2.2.2.2"]);
});
it("weighted with one ip always returns that ip", () => {
expect(
selectActiveIpsByMode(weightedConfig, [row("1.1.1.1", { weight: 5 })], 0),
).toEqual(["1.1.1.1"]);
});
it("weighted with all unknown rotates across every ip", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "unknown" }),
row("2.2.2.2", { weight: 3, health: "unknown" }),
];
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS)).toEqual([
"2.2.2.2",
]);
});
it("weighted returns empty array for no rows", () => {
expect(selectActiveIpsByMode(weightedConfig, [], 0)).toEqual([]);
});
it("round_robin puts recovering unknown back with live ips", () => {
const config: LbTargetConfig = {
lb_mode: "round_robin",
health_check_enabled: true,
@@ -100,7 +158,25 @@ describe("selectActiveIpsByMode", () => {
row("1.1.1.1", { health: "up" }),
row("2.2.2.2", { health: "unknown" }),
];
expect(selectActiveIpsByMode(config, rows)).toEqual(["1.1.1.1"]);
expect(selectActiveIpsByMode(config, rows).sort()).toEqual([
"1.1.1.1",
"2.2.2.2",
]);
});
it("round_robin keeps degraded in the pool with live ips", () => {
const config: LbTargetConfig = {
lb_mode: "round_robin",
health_check_enabled: true,
};
const rows = [
row("1.1.1.1", { health: "up" }),
row("2.2.2.2", { health: "degraded" }),
];
expect(selectActiveIpsByMode(config, rows).sort()).toEqual([
"1.1.1.1",
"2.2.2.2",
]);
});
it("returns empty array for no rows", () => {
@@ -111,3 +187,115 @@ describe("selectActiveIpsByMode", () => {
expect(selectActiveIpsByMode(config, [])).toEqual([]);
});
});
describe("resolveDesiredAIps", () => {
it("keeps a dedicated single IP even when down", () => {
expect(
resolveDesiredAIps(
weightedConfig,
[row("1.1.1.1", { health: "down" })],
["1.1.1.1"],
),
).toEqual(["1.1.1.1"]);
});
it("does not drain when the service has only one unique IP", () => {
expect(
resolveDesiredAIps(
weightedConfig,
[row("1.1.1.1", { health: "down" })],
["1.1.1.1", "1.1.1.1"],
0,
["1.1.1.1"],
),
).toEqual(["1.1.1.1", "1.1.1.1"]);
});
it("does not apply overlay when service pool is a single IP", () => {
const rows = [
row("1.1.1.1", { health: "up" }),
row("2.2.2.2", { health: "down" }),
];
expect(
resolveDesiredAIps(
weightedConfig,
rows,
["1.1.1.1", "2.2.2.2"],
0,
["1.1.1.1"],
),
).toEqual(["1.1.1.1", "2.2.2.2"]);
});
it("applies weighted overlay on a shared pool", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "up" }),
];
expect(
resolveDesiredAIps(weightedConfig, rows, ["1.1.1.1", "2.2.2.2"], 0),
).toEqual(selectActiveIpsByMode(weightedConfig, rows, 0));
});
});
describe("shouldRecordFailoverDnsDiff", () => {
it("skips duplicate listings of the same IP", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "1.1.1.1"],
lbMode: "failover",
added: [],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(false);
});
it("skips dedicated extra-FQDN", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1"],
lbMode: "failover",
added: [],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(false);
});
it("skips weighted live-to-live slot swap", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "2.2.2.2"],
lbMode: "weighted",
added: ["2.2.2.2"],
removed: ["1.1.1.1"],
downIps: new Set(),
}),
).toBe(false);
});
it("logs weighted swap when a down ip leaves the pool", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "2.2.2.2"],
lbMode: "weighted",
added: ["2.2.2.2"],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(true);
});
it("logs failover diffs on a shared pool", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "2.2.2.2"],
lbMode: "failover",
added: ["2.2.2.2"],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(true);
});
});
+142 -21
View File
@@ -1,40 +1,161 @@
import { ShieldCheckIcon } from 'lucide-react'
import {
Timeline,
TimelineContent,
TimelineDate,
TimelineHeader,
TimelineIndicator,
TimelineItem,
TimelineSeparator,
TimelineTitle,
} from '@/components/reui/timeline'
import { HealthCheckBadge } from '@/components/health-check-badge'
import { EmptyState } from '@/components/empty-state'
import { formatDate, formatRelative, sqliteUtcToIso } from '@/lib/format'
import {
failoverEventCopy,
type FailoverEvent,
type FailoverHistoryItem,
} from '@/lib/failover-events'
import { cn } from '@cfdm/ui/lib/utils'
import type { ComponentProps } from 'react'
export interface FailoverEvent {
id: string
title: string
detail: string
type HealthBadgeStatus = ComponentProps<typeof HealthCheckBadge>['status']
function failStreakLabel(count: number): string {
const mod10 = count % 10
const mod100 = count % 100
if (mod10 === 1 && mod100 !== 11) return `${count} ошибка подряд`
if (mod10 >= 2 && mod10 <= 4 && (mod100 < 12 || mod100 > 14)) {
return `${count} ошибки подряд`
}
return `${count} ошибок подряд`
}
export function FailoverTimeline({ events }: { events: FailoverEvent[] }) {
if (events.length === 0) {
function indicatorClass(tone: 'down' | 'added' | 'removed'): string {
if (tone === 'added') {
return 'border-success bg-success/15 group-data-completed/timeline-item:border-success'
}
return 'border-destructive bg-destructive/15 group-data-completed/timeline-item:border-destructive'
}
function separatorClass(tone: 'down' | 'added' | 'removed'): string {
if (tone === 'added') return 'bg-success/25'
return 'bg-destructive/25'
}
/**
* Failover как sibling «Смены статуса»: ReUI Timeline + Badge.
* Preview: https://reui.io/preview/base/components/c-timeline-10
* Preview: https://reui.io/preview/base/empty-state-12
* Docs: https://reui.io/docs/components/base/timeline
* Docs: https://reui.io/docs/components/base/badge
*/
export function FailoverTimeline({
events,
history = [],
}: {
events: FailoverEvent[]
history?: readonly FailoverHistoryItem[]
}) {
if (events.length === 0 && history.length === 0) {
return (
<p className="text-muted-foreground text-sm">
Событий failover пока нет.
</p>
<EmptyState
icon={ShieldCheckIcon}
title="Нет инцидентов"
description="Нет Down и нет выходов из общего пула"
stackedIcon={false}
centered={false}
/>
)
}
return (
<Timeline defaultValue={events.length} className="w-full">
{events.map((event, index) => (
<TimelineItem key={event.id} step={index + 1}>
<TimelineSeparator />
<TimelineIndicator />
<TimelineHeader>
<TimelineTitle>{event.title}</TimelineTitle>
</TimelineHeader>
<TimelineContent>{event.detail}</TimelineContent>
</TimelineItem>
))}
</Timeline>
<div className="flex flex-col gap-4">
{events.length > 0 ? (
<Timeline defaultValue={0} className="gap-0">
{events.map((event, index) => {
const checkedIso = event.lastCheckAt
? (sqliteUtcToIso(event.lastCheckAt) ?? event.lastCheckAt)
: null
return (
<TimelineItem key={event.id} step={index + 1}>
<TimelineSeparator className={separatorClass('down')} />
<TimelineIndicator className={indicatorClass('down')} />
<TimelineHeader>
<TimelineTitle className="flex flex-wrap items-center gap-2">
<span className="font-mono text-sm">{event.address}</span>
<HealthCheckBadge
status={event.status as HealthBadgeStatus}
lastError={event.lastFailureReason}
lastCheckedAt={checkedIso}
size="xs"
/>
</TimelineTitle>
<TimelineDate>
{event.consecutiveFailures > 0
? failStreakLabel(event.consecutiveFailures)
: null}
{checkedIso
? `${event.consecutiveFailures > 0 ? ' · ' : ''}${formatRelative(checkedIso)} · ${formatDate(checkedIso)}`
: null}
</TimelineDate>
</TimelineHeader>
<TimelineContent className="flex flex-col gap-2">
<p className="text-foreground text-sm">
{failoverEventCopy(event)}
</p>
{event.lastFailureReason ? (
<code
className={cn(
'bg-muted block overflow-x-auto rounded-md px-2 py-1.5 font-mono text-xs',
)}
>
{event.lastFailureReason}
</code>
) : null}
</TimelineContent>
</TimelineItem>
)
})}
</Timeline>
) : null}
{history.length > 0 ? (
<Timeline defaultValue={0} className="gap-0">
{history.map((item, index) => {
const checkedIso = sqliteUtcToIso(item.created_at) ?? item.created_at
const tone = item.action === 'added' ? 'added' : 'removed'
return (
<TimelineItem key={item.id} step={index + 1}>
<TimelineSeparator className={separatorClass(tone)} />
<TimelineIndicator className={indicatorClass(tone)} />
<TimelineHeader>
<TimelineTitle className="flex flex-wrap items-center gap-2">
<span className="font-mono text-sm">{item.ip}</span>
<HealthCheckBadge
status={item.action === 'added' ? 'up' : 'down'}
size="xs"
/>
<span className="text-muted-foreground text-xs">
{item.fqdn}
</span>
</TimelineTitle>
<TimelineDate>
{formatRelative(checkedIso)} · {formatDate(checkedIso)}
</TimelineDate>
</TimelineHeader>
<TimelineContent>
<p className="text-foreground text-sm">{item.copy}</p>
</TimelineContent>
</TimelineItem>
)
})}
</Timeline>
) : null}
</div>
)
}
@@ -1,6 +1,7 @@
import { FormFieldSimple } from '@/components/form-field'
import { AppInput } from '@/components/app-input'
import { SettingRow } from '@/components/setting-row'
import { SelectField } from '@/components/select-field'
import { Badge } from '@/components/reui/badge'
import {
NumberField,
@@ -9,15 +10,8 @@ import {
NumberFieldIncrement,
NumberFieldInput,
} from '@/components/reui/number-field'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@cfdm/ui/components/select'
import { Switch } from '@cfdm/ui/components/switch'
import { FieldGroup } from '@cfdm/ui/components/field'
import { Field, FieldGroup, FieldLabel } from '@cfdm/ui/components/field'
import { Button } from '@cfdm/ui/components/button'
import { ButtonGroup } from '@cfdm/ui/components/button-group'
import { CableIcon, GlobeIcon } from 'lucide-react'
@@ -59,9 +53,14 @@ export interface LbAndHealthConfig extends HealthCheckConfig {
const defaultLbModeOptions = [
{ value: 'round_robin', label: 'Round Robin' },
{ value: 'failover', label: 'Failover (приоритет)' },
{ value: 'weighted', label: 'Weighted (веса)' },
{ value: 'weighted', label: 'Веса (подмена IP)' },
]
export interface LbPoolMetaChange {
weight?: number
priority?: number
}
function CompactNumberField({
id,
value,
@@ -95,6 +94,89 @@ function CompactNumberField({
)
}
function PoolLbMetaFields({
idPrefix,
mode,
ips,
weights,
priorities,
onMetaChange,
}: {
idPrefix: string
mode: Exclude<LbMode, 'round_robin'>
ips: readonly string[]
weights: Record<string, number>
priorities: Record<string, number>
onMetaChange?: (ip: string, meta: LbPoolMetaChange) => void
}) {
const isWeighted = mode === 'weighted'
const minPriority =
ips.length === 0
? 1
: Math.min(...ips.map((ip) => priorities[ip] ?? 1))
return (
<SettingRow
title={isWeighted ? 'Вес IP' : 'Приоритет IP'}
description={
isWeighted
? 'Доля времени на общем FQDN: 1 и 3 = ¼ и ¾ цикла (слот 60 с)'
: '1 — основной, больше — запасной'
}
compact
stacked
className="gap-3 px-0 py-3"
contentClassName="min-w-0"
>
{ips.length === 0 ? (
<p className="text-muted-foreground text-sm">Сначала добавьте IP выше</p>
) : (
<div className="flex w-full flex-col gap-2">
{ips.map((ip) => {
const isPrimary = (priorities[ip] ?? 1) === minPriority
const fieldId = isWeighted
? `${idPrefix}-weight-${ip}`
: `${idPrefix}-priority-${ip}`
return (
<div key={ip} className="flex items-center gap-3">
<span className="min-w-0 flex-1 truncate font-mono text-sm">{ip}</span>
{!isWeighted ? (
<Badge
variant={isPrimary ? 'success-light' : 'outline'}
size="xs"
radius="full"
>
{isPrimary ? 'Основной' : 'Запасной'}
</Badge>
) : null}
<Field className="w-28 gap-0">
<FieldLabel htmlFor={fieldId} className="sr-only">
{isWeighted ? `Вес ${ip}` : `Приоритет ${ip}`}
</FieldLabel>
<CompactNumberField
id={fieldId}
value={isWeighted ? (weights[ip] ?? 1) : (priorities[ip] ?? 1)}
min={1}
max={100}
onValueChange={(next) =>
onMetaChange?.(
ip,
isWeighted
? { weight: next ?? 1 }
: { priority: next ?? 1 },
)
}
/>
</Field>
</div>
)
})}
</div>
)}
</SettingRow>
)
}
export function HealthCheckConfigFields({
value,
onChange,
@@ -102,6 +184,10 @@ export function HealthCheckConfigFields({
lbModeOptions = defaultLbModeOptions,
idPrefix = 'health',
showLbMode = true,
ips = [],
weights = {},
priorities = {},
onMetaChange,
className,
}: {
value: LbAndHealthConfig
@@ -110,6 +196,10 @@ export function HealthCheckConfigFields({
lbModeOptions?: { value: string; label: string }[]
idPrefix?: string
showLbMode?: boolean
ips?: readonly string[]
weights?: Record<string, number>
priorities?: Record<string, number>
onMetaChange?: (ip: string, meta: LbPoolMetaChange) => void
className?: string
}) {
function patch(next: Partial<LbAndHealthConfig>) {
@@ -134,25 +224,28 @@ export function HealthCheckConfigFields({
compact
className={rowClass}
>
<Select
<SelectField
modal={false}
value={value.lb_mode}
onValueChange={(v) => patch({ lb_mode: (v ?? 'round_robin') as LbMode })}
>
<SelectTrigger id={`${idPrefix}-lb-mode`} className="w-full">
<SelectValue placeholder="Выберите режим" />
</SelectTrigger>
<SelectContent>
{lbModeOptions.map((item) => (
<SelectItem key={item.value} value={item.value}>
{item.label}
</SelectItem>
))}
</SelectContent>
</Select>
triggerId={`${idPrefix}-lb-mode`}
placeholder="Выберите режим"
options={lbModeOptions}
/>
</SettingRow>
) : null}
{showLbMode && value.lb_mode !== 'round_robin' ? (
<PoolLbMetaFields
idPrefix={idPrefix}
mode={value.lb_mode}
ips={ips}
weights={weights}
priorities={priorities}
onMetaChange={onMetaChange}
/>
) : null}
<SettingRow
title="Провайдер health-check"
description="Кто пробирует цель. Можно выбрать несколько источников."
@@ -1,5 +1,6 @@
export { UptimeChart, type UptimeProbe, type UptimePeriodKey, probeUptimePercent, lastProbeLatency } from './uptime-chart'
export { ServiceHealthMonitor } from './service-health-monitor'
export { ServiceFailoverPanel } from './service-failover-panel'
export { applyFiltersToData, getActiveFilters, renderSingleSelectedLabel } from './filter-utils'
export { CertStatusChart, GroupDomainsChart } from './dashboard-analytics'
export { ResourcePage, type ResourcePageProps, type ResourcePageTab } from './resource-page'
@@ -1,8 +1,7 @@
import { useState, type KeyboardEvent } from 'react'
import { PlusIcon, ServerIcon, Trash2Icon } from 'lucide-react'
import { useState, type KeyboardEvent, type ReactNode } from 'react'
import { ServerIcon, Trash2Icon } from 'lucide-react'
import { EmptyState } from '@/components/empty-state'
import { ServiceBindingIpInput } from '@/components/service-binding-ip-input'
import { isValidIpv4 } from '@/components/tagged-input'
import { Badge } from '@/components/reui/badge'
import {
@@ -16,15 +15,15 @@ import { IconTile } from '@/components/reui/icon-tile'
import { parseFqdn } from '@/lib/parse-fqdn'
import {
addAddressNode,
emptyBindingDraft,
addCommonFqdn,
addressHasFqdn,
removeAddressNode,
withPoolIps,
removeCommonFqdn,
updateCommonFqdn,
type AddressBlockState,
type ServiceBindingDraft,
} from '@/lib/service-address'
import { Button } from '@cfdm/ui/components/button'
import { Field, FieldLabel } from '@cfdm/ui/components/field'
import { Input } from '@cfdm/ui/components/input'
import {
InputGroup,
InputGroupAddon,
@@ -39,16 +38,36 @@ import {
ItemMedia,
ItemTitle,
} from '@cfdm/ui/components/item'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@cfdm/ui/components/select'
function ZoneAddon({
fqdn,
zoneHints,
trailing,
}: {
fqdn: string
zoneHints: string[]
trailing?: ReactNode
}) {
const parsed = parseFqdn(fqdn, zoneHints)
if (!parsed && !fqdn.trim() && !trailing) return null
return (
<InputGroupAddon align="inline-end">
{parsed ? (
<Badge variant="outline" size="xs" className="font-mono">
{parsed.zoneName}
</Badge>
) : fqdn.trim() ? (
<Badge variant="warning-light" size="xs">
зона не найдена
</Badge>
) : null}
{trailing}
</InputGroupAddon>
)
}
/**
* Единый блок адресов сервиса: общий FQDN + пул IP с опциональным доп. доменом.
* Единый блок адресов сервиса: список общих FQDN на весь пул + IP с доп. доменом.
* Preview: https://reui.io/preview/base/settings-3
* Preview: https://reui.io/preview/base/list-9
* Preview: https://reui.io/preview/base/form-7
@@ -67,17 +86,30 @@ export function ServiceAddressBlock({
}) {
const [pendingIp, setPendingIp] = useState('')
const [ipInvalid, setIpInvalid] = useState(false)
const [otherOpen, setOtherOpen] = useState(value.otherBindings.length > 0)
const [pendingFqdn, setPendingFqdn] = useState('')
const [fqdnInvalid, setFqdnInvalid] = useState(false)
const pool = value.nodes.map((node) => node.ip)
const parsedCommon = parseFqdn(value.commonFqdn, zoneHints)
const showOthers = otherOpen || value.otherBindings.length > 0
const pendingTrimmed = pendingIp.trim()
const pendingInvalid =
ipInvalid && pendingTrimmed.length > 0 && !isValidIpv4(pendingTrimmed)
const pendingIpTrimmed = pendingIp.trim()
const pendingFqdnTrimmed = pendingFqdn.trim()
const pendingIpInvalid =
ipInvalid && pendingIpTrimmed.length > 0 && !isValidIpv4(pendingIpTrimmed)
const pendingFqdnInvalid =
fqdnInvalid && pendingFqdnTrimmed.length > 0
function handleCommonFqdn(next: string) {
onChange({ ...value, commonFqdn: next })
function tryAddFqdn(raw: string) {
const trimmed = raw.trim()
if (!trimmed) {
setFqdnInvalid(false)
return
}
if (addressHasFqdn(value, trimmed)) {
setFqdnInvalid(true)
return
}
onChange(addCommonFqdn(value, trimmed))
setPendingFqdn('')
setFqdnInvalid(false)
}
function tryAddIp(raw: string) {
@@ -95,7 +127,14 @@ export function ServiceAddressBlock({
setIpInvalid(false)
}
function handlePendingKeyDown(event: KeyboardEvent<HTMLInputElement>) {
function handleFqdnKeyDown(event: KeyboardEvent<HTMLInputElement>) {
if (event.key === 'Enter') {
event.preventDefault()
tryAddFqdn(pendingFqdn)
}
}
function handleIpKeyDown(event: KeyboardEvent<HTMLInputElement>) {
if (event.key === 'Enter') {
event.preventDefault()
tryAddIp(pendingIp)
@@ -111,64 +150,69 @@ export function ServiceAddressBlock({
})
}
function handleRemoveIp(ip: string) {
onChange(removeAddressNode(value, ip))
}
function handleAddOther() {
setOtherOpen(true)
onChange({
...value,
otherBindings: [...value.otherBindings, withPoolIps(emptyBindingDraft(), pool)],
})
}
function handleOtherChange(index: number, next: ServiceBindingDraft) {
onChange({
...value,
otherBindings: value.otherBindings.map((item, i) => (i === index ? next : item)),
})
}
function handleRemoveOther(index: number) {
const otherBindings = value.otherBindings.filter((_, i) => i !== index)
onChange({ ...value, otherBindings })
if (otherBindings.length === 0) setOtherOpen(false)
}
return (
<Frame stacked dense spacing="sm" className="w-full min-w-0">
<FramePanel fit className="flex flex-col gap-3">
<FrameHeader className="px-0 pt-0">
<FrameTitle>Адреса</FrameTitle>
<FrameDescription>
Общий FQDN на весь пул · у каждого IP свой доп. домен
Общие FQDN на весь пул · у IP свой доп. домен
</FrameDescription>
</FrameHeader>
<Field>
<FieldLabel htmlFor="service-common-fqdn">Общий домен (FQDN)</FieldLabel>
<InputGroup>
<InputGroupInput
id="service-common-fqdn"
className="font-mono"
value={value.commonFqdn}
placeholder={zoneHints[0] ? `gw.${zoneHints[0]}` : 'gw.ivx.su'}
onChange={(event) => handleCommonFqdn(event.target.value)}
/>
{parsedCommon ? (
<InputGroupAddon align="inline-end">
<Badge variant="outline" size="xs" className="font-mono">
{parsedCommon.zoneName}
</Badge>
</InputGroupAddon>
) : value.commonFqdn.trim() ? (
<InputGroupAddon align="inline-end">
<Badge variant="warning-light" size="xs">
зона не найдена
</Badge>
</InputGroupAddon>
) : null}
</InputGroup>
<FieldLabel htmlFor="service-common-fqdn-add">Общие домены (FQDN)</FieldLabel>
<div className="flex w-full flex-col gap-2">
{value.commonFqdns.map((fqdn, index) => (
<InputGroup key={`common-fqdn-${index}`}>
<InputGroupInput
id={`service-common-fqdn-${index}`}
className="font-mono"
value={fqdn}
placeholder={zoneHints[0] ? `gw.${zoneHints[0]}` : 'gw.ivx.su'}
onChange={(event) =>
onChange(updateCommonFqdn(value, index, event.target.value))
}
/>
<ZoneAddon
fqdn={fqdn}
zoneHints={zoneHints}
trailing={
<InputGroupButton
size="icon-xs"
aria-label={`Удалить ${fqdn || 'FQDN'}`}
onClick={() => onChange(removeCommonFqdn(value, index))}
>
<Trash2Icon />
</InputGroupButton>
}
/>
</InputGroup>
))}
<InputGroup>
<InputGroupInput
id="service-common-fqdn-add"
className="font-mono"
value={pendingFqdn}
placeholder={zoneHints[0] ? `gw.${zoneHints[0]}` : 'gw.ivx.su'}
aria-invalid={pendingFqdnInvalid || undefined}
onChange={(event) => {
setPendingFqdn(event.target.value)
setFqdnInvalid(false)
}}
onKeyDown={handleFqdnKeyDown}
onBlur={() => tryAddFqdn(pendingFqdn)}
/>
<ZoneAddon
fqdn={pendingFqdn}
zoneHints={zoneHints}
trailing={
<InputGroupButton size="sm" onClick={() => tryAddFqdn(pendingFqdn)}>
Добавить
</InputGroupButton>
}
/>
</InputGroup>
</div>
</Field>
</FramePanel>
@@ -187,7 +231,6 @@ export function ServiceAddressBlock({
) : (
<ItemGroup className="gap-2">
{value.nodes.map((node) => {
const parsedExtra = parseFqdn(node.extraFqdn, zoneHints)
return (
<Item
key={node.ip}
@@ -214,7 +257,7 @@ export function ServiceAddressBlock({
variant="ghost"
size="icon-sm"
aria-label={`Удалить ${node.ip}`}
onClick={() => handleRemoveIp(node.ip)}
onClick={() => onChange(removeAddressNode(value, node.ip))}
>
<Trash2Icon />
</Button>
@@ -241,19 +284,7 @@ export function ServiceAddressBlock({
handleNodeFqdn(node.ip, event.target.value)
}
/>
{parsedExtra ? (
<InputGroupAddon align="inline-end">
<Badge variant="outline" size="xs" className="font-mono">
{parsedExtra.zoneName}
</Badge>
</InputGroupAddon>
) : node.extraFqdn.trim() ? (
<InputGroupAddon align="inline-end">
<Badge variant="warning-light" size="xs">
зона не найдена
</Badge>
</InputGroupAddon>
) : null}
<ZoneAddon fqdn={node.extraFqdn} zoneHints={zoneHints} />
</InputGroup>
</Field>
</ItemContent>
@@ -268,12 +299,12 @@ export function ServiceAddressBlock({
className="font-mono"
value={pendingIp}
placeholder="192.168.1.1"
aria-invalid={pendingInvalid || undefined}
aria-invalid={pendingIpInvalid || undefined}
onChange={(event) => {
setPendingIp(event.target.value)
setIpInvalid(false)
}}
onKeyDown={handlePendingKeyDown}
onKeyDown={handleIpKeyDown}
onBlur={() => tryAddIp(pendingIp)}
/>
<InputGroupAddon align="inline-end">
@@ -282,156 +313,7 @@ export function ServiceAddressBlock({
</InputGroupButton>
</InputGroupAddon>
</InputGroup>
{showOthers ? null : (
<div className="flex justify-end">
<Button
type="button"
variant="ghost"
size="sm"
onClick={handleAddOther}
>
<PlusIcon data-icon="inline-start" />
Другой FQDN
</Button>
</div>
)}
</FramePanel>
{showOthers ? (
<FramePanel fit className="flex flex-col gap-3">
<FrameHeader className="flex flex-row items-start justify-between gap-2 px-0 pt-0">
<div className="flex min-w-0 flex-col gap-1">
<FrameTitle>Другие FQDN</FrameTitle>
<FrameDescription>CNAME и A не 1:1 с IP пула</FrameDescription>
</div>
<Button type="button" variant="outline" size="sm" onClick={handleAddOther}>
<PlusIcon data-icon="inline-start" />
Добавить
</Button>
</FrameHeader>
{value.otherBindings.length === 0 ? (
<p className="text-muted-foreground text-sm">Нет дополнительных FQDN</p>
) : (
<ItemGroup className="gap-2">
{value.otherBindings.map((binding, index) => {
const parsedZone = parseFqdn(binding.fqdn, zoneHints)
return (
<Item
key={`other-binding-${index}`}
variant="outline"
size="sm"
className="items-stretch"
>
<ItemContent className="flex w-full min-w-0 flex-col gap-2">
<div className="flex items-center gap-2">
{parsedZone ? (
<Badge variant="outline" size="xs" className="font-mono">
{parsedZone.zoneName}
</Badge>
) : binding.fqdn.trim() ? (
<Badge variant="warning-light" size="xs">
зона не найдена
</Badge>
) : (
<span className="text-muted-foreground text-xs">FQDN</span>
)}
<Button
type="button"
variant="ghost"
size="icon-sm"
className="ml-auto shrink-0"
aria-label="Удалить FQDN"
onClick={() => handleRemoveOther(index)}
>
<Trash2Icon />
</Button>
</div>
<div className="grid grid-cols-1 gap-2 sm:grid-cols-[minmax(0,1fr)_7.5rem]">
<Input
id={`other-fqdn-${index}`}
className="font-mono"
value={binding.fqdn}
onChange={(event) =>
handleOtherChange(index, {
...binding,
fqdn: event.target.value,
})
}
placeholder={
zoneHints[0] ? `api.${zoneHints[0]}` : 'api.ivx.su'
}
/>
<Select
items={[
{ label: 'A (IP)', value: 'A' },
{ label: 'CNAME', value: 'CNAME' },
]}
value={binding.record_type}
onValueChange={(next) => {
const recordType = (next ?? 'A') as 'A' | 'CNAME'
handleOtherChange(index, {
...binding,
record_type: recordType,
target_ips: recordType === 'A' ? binding.target_ips : [],
target_cname:
recordType === 'CNAME' ? binding.target_cname : '',
})
}}
>
<SelectTrigger id={`other-type-${index}`} className="w-full">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="A">A (IP)</SelectItem>
<SelectItem value="CNAME">CNAME</SelectItem>
</SelectContent>
</Select>
</div>
{binding.record_type === 'CNAME' ? (
<Input
id={`other-cname-${index}`}
value={binding.target_cname}
placeholder="mmsk.rkns.top"
onChange={(event) =>
handleOtherChange(index, {
...binding,
target_cname: event.target.value,
})
}
/>
) : (
<ServiceBindingIpInput
id={`other-ip-${index}`}
value={binding.target_ips}
pool={pool}
onChange={(targetIps) =>
handleOtherChange(index, {
...binding,
target_ips: targetIps,
target_ip_weights: Object.fromEntries(
targetIps.map((ip) => [
ip,
binding.target_ip_weights[ip] ?? 1,
]),
),
target_ip_priorities: Object.fromEntries(
targetIps.map((ip) => [
ip,
binding.target_ip_priorities[ip] ?? 1,
]),
),
})
}
/>
)}
</ItemContent>
</Item>
)
})}
</ItemGroup>
)}
</FramePanel>
) : null}
</Frame>
)
}
@@ -0,0 +1,141 @@
import { UnplugIcon } from 'lucide-react'
import { FailoverTimeline } from '@/components/failover-timeline'
import {
mergeFailoverHistory,
toFailoverEvents,
type FailoverBindingPool,
type FailoverHealthInput,
} from '@/lib/failover-events'
import {
latestHealthByIp,
resolveIpDisplayHealth,
type HealthLogProbe,
type HealthLogStatus,
} from '@/lib/health-log'
import type { FailoverLogEntry, ServiceView } from '@/lib/schemas'
import { Badge } from '@/components/reui/badge'
import {
Frame,
FrameDescription,
FrameHeader,
FramePanel,
FrameTitle,
} from '@/components/reui/frame'
import {
Alert,
AlertDescription,
AlertTitle,
} from '@/components/reui/alert'
type LbMode = ServiceView['lb_mode']
const PANEL_COPY: Record<LbMode, { title: string; description: string }> = {
failover: {
title: 'Failover (приоритет)',
description: 'Down снимается с общего FQDN. История: кто вышел из пула и кто вернулся',
},
weighted: {
title: 'Веса (подмена IP)',
description: 'На общем FQDN один IP по весам. Down выводится из цикла',
},
round_robin: {
title: 'Round Robin',
description: 'На общем FQDN все живые A. Down снимается с пула',
},
}
function failoverCountLabel(count: number): string {
const mod10 = count % 10
const mod100 = count % 100
if (mod10 === 1 && mod100 !== 11) return `${count} адрес Down`
if (mod10 >= 2 && mod10 <= 4 && (mod100 < 12 || mod100 > 14)) {
return `${count} адреса Down`
}
return `${count} адресов Down`
}
function alertDescription(events: { kind: string; fqdns: string[] }[]): string {
const removedCount = events.filter((event) => event.kind === 'removed').length
if (removedCount > 0) return 'Сняты с общего FQDN'
if (events.some((event) => event.fqdns.length > 0)) {
return 'Остались в A-записях общего FQDN как last-resort'
}
return 'Down: персональные FQDN не меняются'
}
/**
* Балансировка — текущие Down + кто вышел из общего пула.
* Preview: https://reui.io/preview/base/components/c-timeline-10
* Preview: https://reui.io/preview/base/empty-state-12
* Docs: https://reui.io/docs/components/base/frame
* Docs: https://reui.io/docs/components/base/timeline
* Docs: https://reui.io/docs/components/base/badge
* Docs: https://reui.io/docs/components/base/alert
*/
export function ServiceFailoverPanel({
lbMode = 'round_robin',
ipHealth,
bindings,
history,
probes = [],
}: {
lbMode?: LbMode
ipHealth: readonly FailoverHealthInput[]
bindings: readonly FailoverBindingPool[]
history: readonly FailoverLogEntry[]
probes?: readonly HealthLogProbe[]
}) {
const copy = PANEL_COPY[lbMode] ?? PANEL_COPY.round_robin
const liveByIp = latestHealthByIp(probes)
const overlayHealth = ipHealth.map((row) => {
const live = liveByIp.get(row.ip)
return {
...row,
status: resolveIpDisplayHealth(
row.status as HealthLogStatus,
live?.status,
),
last_error:
live && live.status !== 'unknown' ? live.last_error : row.last_error,
last_checked_at:
live && live.status !== 'unknown'
? live.last_checked_at
: row.last_checked_at,
}
})
const events = toFailoverEvents(overlayHealth, bindings)
const mergedHistory = mergeFailoverHistory(history, probes, bindings)
return (
<Frame stacked spacing="sm" className="min-w-0 w-full">
<FramePanel className="flex flex-col gap-3">
<FrameHeader className="gap-1 px-0 py-0">
<FrameTitle className="flex flex-wrap items-center gap-2">
{copy.title}
{events.length > 0 ? (
<Badge variant="destructive-light" size="xs" radius="full">
{events.length}
</Badge>
) : (
<Badge variant="success-light" size="xs" radius="full">
OK
</Badge>
)}
</FrameTitle>
<FrameDescription>{copy.description}</FrameDescription>
</FrameHeader>
{events.length > 0 ? (
<Alert variant="destructive">
<UnplugIcon aria-hidden="true" />
<AlertTitle>{failoverCountLabel(events.length)}</AlertTitle>
<AlertDescription>{alertDescription(events)}</AlertDescription>
</Alert>
) : null}
<FailoverTimeline events={events} history={mergedHistory} />
</FramePanel>
</Frame>
)
}
@@ -45,6 +45,28 @@ describe('toAlignedSeries', () => {
expect(keys).toEqual(['local', 'cloudflare', 'globalping'])
})
it('keeps live latency when another IP in the same bucket is down', () => {
const { points } = toAlignedSeries([
probe({
id: 1,
latency_ms: 18,
checked_at: '2026-01-01T00:00:10.000Z',
}),
probe({
id: 2,
status: 'down',
ok: false,
latency_ms: null,
checked_at: '2026-01-01T00:00:12.000Z',
}),
])
expect(points).toHaveLength(1)
expect(points[0]?.local).toBe(18)
expect(points[0]?.localOk).toBe(true)
expect(points[0]?.ok).toBe(true)
})
it('does not plot down probes as latency 0', () => {
const { points } = toAlignedSeries([
probe({
@@ -124,8 +124,13 @@ export function toAlignedSeries(items: UptimeProbe[]): {
}
const ok = probeOk(item)
row[`${key}Ok`] = ok
row[key] = ok ? item.latency_ms : null
const prevOk = row[`${key}Ok`]
row[`${key}Ok`] = prevOk === true || ok
if (ok && item.latency_ms != null) {
row[key] = item.latency_ms
} else if (row[key] === undefined) {
row[key] = null
}
}
const points = [...buckets.entries()]
@@ -17,6 +17,7 @@ import {
DEFAULT_BINDING_HEALTH,
emptyAddressBlock,
hydrateAddressBlock,
patchAddressIpMeta,
toBindingDrafts,
toDomainsPayload,
type AddressBlockState,
@@ -218,8 +219,8 @@ export function ServiceEditSheet({
<SheetHeader className="shrink-0 border-b pb-4">
<SheetTitle>{isCreate ? 'Новый сервис' : 'Редактирование сервиса'}</SheetTitle>
<SheetDescription>
Общий домен и пул IP в одном блоке. У каждого адреса можно указать
свой доп. FQDN.
Общие FQDN на весь пул IP. У каждого адреса можно указать свой доп.
FQDN.
</SheetDescription>
</SheetHeader>
@@ -282,6 +283,12 @@ export function ServiceEditSheet({
idPrefix="service-health"
value={primaryHealthValue}
onChange={handlePrimaryHealthChange}
ips={address.nodes.map((node) => node.ip)}
weights={address.target_ip_weights}
priorities={address.target_ip_priorities}
onMetaChange={(ip, meta) =>
setAddress((current) => patchAddressIpMeta(current, ip, meta))
}
/>
</section>
</div>
@@ -18,6 +18,11 @@ import { DataGridColumnHeader } from '@/components/reui/data-grid/data-grid-colu
import { IconTile } from '@/components/reui/icon-tile'
import { createFilter, type Filter, type FilterFieldConfig } from '@/components/reui/filters'
import { ResourcePage } from '@/components/reui-kit'
import {
latestHealthByIp,
resolveIpDisplayHealth,
type HealthLogProbe,
} from '@/lib/health-log'
import { certRelativeBadge } from '@/components/columns/certificates-columns'
import { certMonitoringOptions } from '@/lib/cert-monitoring'
import { formatDate } from '@/lib/format'
@@ -99,8 +104,12 @@ function mapNodeHealth(status: string): HealthStatus {
return 'unknown'
}
function buildIpRows(service: ServiceView): ServiceIpRow[] {
function buildIpRows(
service: ServiceView,
probes: readonly HealthLogProbe[] = [],
): ServiceIpRow[] {
const healthByIp = new Map(service.ip_health.map((row) => [row.ip, row]))
const liveByIp = latestHealthByIp(probes)
const weights = Object.assign(
{},
...service.domains.map((domain) => domain.target_ip_weights ?? {}),
@@ -113,19 +122,25 @@ function buildIpRows(service: ServiceView): ServiceIpRow[] {
return service.ips.map((ip) => {
const health = healthByIp.get(ip)
const live = liveByIp.get(ip)
const status = resolveIpDisplayHealth(health?.status, live?.status)
const extras = live && live.status !== 'unknown' ? live : health
return {
id: ip,
ip,
status: health?.status ?? 'unknown',
status,
enabled: service.ip_enabled[ip] !== false,
active: activeSet.has(ip),
weight: weights[ip] ?? 1,
priority: priorities[ip] ?? 1,
latency_ms: health?.latency_ms ?? null,
last_checked_at: health?.last_checked_at ?? null,
last_error: health?.last_error ?? null,
colo: health?.colo ?? null,
provider: health?.provider ?? null,
latency_ms: extras?.latency_ms ?? null,
last_checked_at: extras?.last_checked_at ?? null,
last_error:
live && live.status !== 'unknown'
? live.last_error
: (health?.last_error ?? null),
colo: extras?.colo ?? null,
provider: extras?.provider ?? null,
}
})
}
@@ -151,17 +166,23 @@ function buildNodeRows(
weight: number
priority: number
}>,
probes: readonly HealthLogProbe[] = [],
): ServiceNodeRow[] {
return nodes.map((node) => ({
id: String(node.id),
nodeId: node.id,
address: node.address,
protocol: node.protocol,
port: node.port,
health_status: mapNodeHealth(node.health_status),
weight: node.weight,
priority: node.priority,
}))
const liveByIp = latestHealthByIp(probes)
return nodes.map((node) => {
const stored = mapNodeHealth(node.health_status)
const live = liveByIp.get(node.address)
return {
id: String(node.id),
nodeId: node.id,
address: node.address,
protocol: node.protocol,
port: node.port,
health_status: resolveIpDisplayHealth(stored, live?.status),
weight: node.weight,
priority: node.priority,
}
})
}
function NameCell({
@@ -199,6 +220,7 @@ interface ServiceDetailGridProps {
weight: number
priority: number
}>
probes?: readonly HealthLogProbe[]
togglingIp: string | null
onToggleIp: (ip: string, enabled: boolean) => void
onChangeIp: (row: ServiceFqdnRow) => void
@@ -211,6 +233,7 @@ interface ServiceDetailGridProps {
export function ServiceDetailGrid({
service,
nodes,
probes = [],
togglingIp,
onToggleIp,
onChangeIp,
@@ -283,9 +306,9 @@ export function ServiceDetailGrid({
},
})
const ipRows = useMemo(() => buildIpRows(service), [service])
const ipRows = useMemo(() => buildIpRows(service, probes), [service, probes])
const fqdnRows = useMemo(() => buildFqdnRows(service), [service])
const nodeRows = useMemo(() => buildNodeRows(nodes), [nodes])
const nodeRows = useMemo(() => buildNodeRows(nodes, probes), [nodes, probes])
const markActive = service.lb_mode === 'failover' || service.lb_mode === 'weighted'
const tabs = TABS.map((entry) => ({
@@ -71,7 +71,7 @@ const LB_MODE_META: Record<
weighted: {
icon: ScaleIcon,
className: 'text-info',
label: 'Weighted (веса)',
label: 'Веса (подмена IP)',
},
}
+336
View File
@@ -0,0 +1,336 @@
import { describe, expect, it } from 'vitest'
import {
failoverEventCopy,
isFailoverEventStatus,
mergeFailoverHistory,
toFailoverEvents,
toIpAliveTransitions,
type FailoverBindingPool,
type FailoverHealthInput,
} from '@/lib/failover-events'
import type { FailoverLogEntry } from '@/lib/schemas'
import type { HealthLogProbe } from '@/lib/health-log'
function row(
overrides: Partial<FailoverHealthInput> & Pick<FailoverHealthInput, 'ip'>,
): FailoverHealthInput {
return {
status: 'up',
consecutive_failures: 0,
last_error: null,
last_checked_at: null,
...overrides,
}
}
const mskHip: FailoverBindingPool[] = [
{
fqdn: 'gt.rkns.top',
configured: ['130.49.213.153', '93.115.203.183'],
active: ['93.115.203.183'],
},
{
fqdn: 'nsgt.rkns.top',
configured: ['130.49.213.153'],
active: ['130.49.213.153'],
},
{
fqdn: 'rutg.rkns.top',
configured: ['93.115.203.183'],
active: ['93.115.203.183'],
},
]
describe('toFailoverEvents', () => {
it('инцидент только при binding down, не при unhealthy ноды', () => {
expect(isFailoverEventStatus('down')).toBe(true)
expect(isFailoverEventStatus('up')).toBe(false)
expect(isFailoverEventStatus('degraded')).toBe(false)
expect(isFailoverEventStatus('unknown')).toBe(false)
expect(isFailoverEventStatus('unhealthy')).toBe(false)
})
it('один IP у сервиса — не инцидент пула', () => {
const events = toFailoverEvents(
[
row({
ip: '10.0.0.1',
status: 'down',
consecutive_failures: 9,
last_error: 'fetch failed',
}),
],
[
{
fqdn: 'solo.example.com',
configured: ['10.0.0.1'],
active: [],
},
],
)
expect(events).toEqual([])
})
it('Down без shared pool не событие балансировки', () => {
const events = toFailoverEvents([
row({
ip: '130.49.213.153',
status: 'down',
consecutive_failures: 9,
last_error: 'fetch failed',
}),
])
expect(events).toEqual([])
})
it('OK вне пула не инцидент (standby)', () => {
const events = toFailoverEvents(
[
row({ ip: '10.0.0.1', status: 'up' }),
row({ ip: '130.49.213.153', status: 'up' }),
],
[
{
fqdn: 'pool.example.com',
configured: ['10.0.0.1', '130.49.213.153'],
active: ['10.0.0.1'],
},
],
)
expect(events).toEqual([])
})
it('unknown и degraded вне пула не инцидент', () => {
const events = toFailoverEvents(
[
row({ ip: '10.0.0.1', status: 'up' }),
row({ ip: '10.0.0.2', status: 'unknown' }),
row({ ip: '10.0.0.3', status: 'degraded' }),
],
[
{
fqdn: 'pool.example.com',
configured: ['10.0.0.1', '10.0.0.2', '10.0.0.3'],
active: ['10.0.0.1'],
},
],
)
expect(events).toEqual([])
})
it('Down на доп. FQDN last-resort, снятая с общего пула', () => {
const events = toFailoverEvents(
[
row({
ip: '130.49.213.153',
status: 'down',
consecutive_failures: 9,
last_error: 'fetch failed',
last_checked_at: '2026-08-20 07:00:00',
}),
row({ ip: '93.115.203.183', status: 'up' }),
],
mskHip,
)
expect(events).toEqual([
{
id: '130.49.213.153',
address: '130.49.213.153',
status: 'down',
kind: 'removed',
fqdns: ['gt.rkns.top'],
consecutiveFailures: 9,
lastFailureReason: 'fetch failed',
lastCheckAt: '2026-08-20 07:00:00',
},
])
expect(failoverEventCopy(events[0]!)).toBe('Снята с gt.rkns.top')
expect(events[0]?.fqdns).not.toContain('nsgt.rkns.top')
})
it('Down только last-resort на своём FQDN', () => {
const events = toFailoverEvents(
[row({ ip: '130.49.213.153', status: 'down' })],
[
{
fqdn: 'nsgt.rkns.top',
configured: ['130.49.213.153'],
active: ['130.49.213.153'],
},
],
)
expect(events).toEqual([])
})
it('down без A-записи на configured FQDN — снятие', () => {
const events = toFailoverEvents(
[
row({
ip: '130.49.213.153',
status: 'down',
consecutive_failures: 9,
last_error: 'fetch failed',
last_checked_at: '2026-08-20 07:00:00',
}),
],
[
{
fqdn: 'pool.example.com',
configured: ['10.0.0.1', '130.49.213.153'],
active: ['10.0.0.1'],
},
],
)
expect(events).toEqual([
{
id: '130.49.213.153',
address: '130.49.213.153',
status: 'down',
kind: 'removed',
fqdns: ['pool.example.com'],
consecutiveFailures: 9,
lastFailureReason: 'fetch failed',
lastCheckAt: '2026-08-20 07:00:00',
},
])
})
})
function probe(
overrides: Partial<HealthLogProbe> & Pick<HealthLogProbe, 'id' | 'status' | 'checked_at'>,
): HealthLogProbe {
return {
ip: '130.49.213.153',
provider: 'local',
ok: overrides.status === 'up',
latency_ms: 12,
colo: null,
error: null,
...overrides,
}
}
function dns(
overrides: Partial<FailoverLogEntry> & Pick<FailoverLogEntry, 'id' | 'action' | 'created_at'>,
): FailoverLogEntry {
return {
service_id: 13,
binding_id: 1,
fqdn: 'gt.rkns.top',
ip: '130.49.213.153',
...overrides,
}
}
describe('toIpAliveTransitions', () => {
it('emits leave then return, not the initial up', () => {
const transitions = toIpAliveTransitions([
probe({ id: 1, status: 'up', checked_at: '2026-08-20T09:00:00Z' }),
probe({ id: 2, status: 'up', checked_at: '2026-08-20T09:01:00Z' }),
probe({ id: 3, status: 'down', checked_at: '2026-08-20T09:02:00Z' }),
probe({ id: 4, status: 'down', checked_at: '2026-08-20T09:03:00Z' }),
probe({ id: 5, status: 'up', checked_at: '2026-08-20T09:04:00Z' }),
])
expect(transitions).toEqual([
{ id: 'probe:3', ip: '130.49.213.153', alive: false, at: '2026-08-20T09:02:00Z' },
{ id: 'probe:5', ip: '130.49.213.153', alive: true, at: '2026-08-20T09:04:00Z' },
])
})
it('any-up across providers: leave only when every source is down', () => {
const transitions = toIpAliveTransitions([
probe({
id: 1,
provider: 'local',
status: 'up',
checked_at: '2026-08-20T09:00:00Z',
}),
probe({
id: 2,
provider: 'cloudflare',
status: 'down',
checked_at: '2026-08-20T09:01:00Z',
}),
probe({
id: 3,
provider: 'local',
status: 'down',
checked_at: '2026-08-20T09:02:00Z',
}),
probe({
id: 4,
provider: 'local',
status: 'up',
checked_at: '2026-08-20T09:03:00Z',
}),
])
expect(transitions.map((item) => item.id)).toEqual(['probe:3', 'probe:4'])
})
})
describe('mergeFailoverHistory', () => {
it('fills leave/return from probes when DNS has only the add', () => {
const history = mergeFailoverHistory(
[dns({ id: 10, action: 'added', created_at: '2026-08-20 09:26:00' })],
[
probe({ id: 1, status: 'up', checked_at: '2026-08-20T09:00:00Z' }),
probe({ id: 2, status: 'down', checked_at: '2026-08-20T09:10:00Z' }),
probe({ id: 3, status: 'up', checked_at: '2026-08-20T09:26:30Z' }),
],
mskHip,
)
expect(history.map((item) => `${item.action}:${item.source}`)).toEqual([
'added:dns',
'removed:probe',
])
expect(history[0]?.copy).toBe('130.49.213.153 добавлена на gt.rkns.top')
expect(history[1]?.copy).toBe(
'130.49.213.153 вышла из пула (gt.rkns.top)',
)
})
it('keeps DNS over a probe return in the same 2-minute window', () => {
const history = mergeFailoverHistory(
[dns({ id: 10, action: 'added', created_at: '2026-08-20 09:26:00' })],
[probe({ id: 3, status: 'up', checked_at: '2026-08-20T09:26:30Z' })],
mskHip,
)
expect(history).toHaveLength(1)
expect(history[0]?.source).toBe('dns')
})
it('drops probe leave/return when the service has no shared pool', () => {
const history = mergeFailoverHistory(
[],
[
probe({ id: 1, status: 'up', checked_at: '2026-08-20T09:00:00Z' }),
probe({ id: 2, status: 'down', checked_at: '2026-08-20T09:10:00Z' }),
],
[
{
fqdn: 'solo.example.com',
configured: ['130.49.213.153'],
active: ['130.49.213.153'],
},
],
)
expect(history).toEqual([])
})
it('drops dedicated extra-FQDN DNS rows', () => {
const history = mergeFailoverHistory(
[
dns({
id: 11,
fqdn: 'nsgt.rkns.top',
action: 'added',
created_at: '2026-08-20 09:26:00',
}),
],
[],
mskHip,
)
expect(history).toEqual([])
})
})
+262
View File
@@ -0,0 +1,262 @@
import {
bestAliveHealthStatus,
probeTime,
type HealthLogProbe,
type HealthLogStatus,
} from '@/lib/health-log'
import type { FailoverLogEntry } from '@/lib/schemas'
export type FailoverEventKind = 'removed' | 'last-resort'
export interface FailoverHistoryItem {
id: string
ip: string
fqdn: string
action: 'added' | 'removed'
created_at: string
copy: string
source: 'dns' | 'probe'
}
export interface FailoverEvent {
id: string
address: string
status: string
kind: FailoverEventKind
fqdns: string[]
consecutiveFailures: number
lastFailureReason: string | null
lastCheckAt?: string | null
}
/** Binding IP health — тот же контур, что таблица активов. */
export interface FailoverHealthInput {
ip: string
status: string
consecutive_failures?: number
last_error?: string | null
last_checked_at?: string | null
}
export interface FailoverBindingPool {
fqdn: string
configured: readonly string[]
active: readonly string[]
}
/** Unique A targets. Duplicates of the same IP are not a pool. */
export function uniqueIpCount(ips: readonly string[]): number {
return new Set(ips.filter(Boolean)).size
}
/** Shared pool FQDN — two or more unique A targets. */
export function isSharedPoolBinding(binding: FailoverBindingPool): boolean {
return uniqueIpCount(binding.configured) >= 2
}
export function hasSharedPool(
bindings: readonly FailoverBindingPool[],
): boolean {
return bindings.some(isSharedPoolBinding)
}
/** Инцидент только при down. up / degraded / unknown — не вывод из пула. */
export function isFailoverEventStatus(status: string): boolean {
return status === 'down'
}
export function failoverEventCopy(event: FailoverEvent): string {
if (event.kind === 'removed') {
return event.fqdns.length > 0
? `Снята с ${event.fqdns.join(', ')}`
: 'Снята с DNS'
}
return event.fqdns.length > 0
? `Down, в A-записях last-resort на ${event.fqdns.join(', ')}`
: 'Down, в A-записях last-resort'
}
/**
* Инциденты пула только если у сервиса есть shared FQDN (2+ уникальных IP).
* Один IP — не балансировка и не вывод из пула; Down смотрит health-монитор.
*/
export function toFailoverEvents(
ipHealth: readonly FailoverHealthInput[],
bindings: readonly FailoverBindingPool[] = [],
): FailoverEvent[] {
if (!hasSharedPool(bindings)) return []
return ipHealth.filter((row) => isFailoverEventStatus(row.status)).map((row) => {
const removedFqdns: string[] = []
const lastResortFqdns: string[] = []
for (const binding of bindings) {
if (!isSharedPoolBinding(binding)) continue
const configured = binding.configured.includes(row.ip)
const active = binding.active.includes(row.ip)
if (configured && !active) removedFqdns.push(binding.fqdn)
else if (configured && active) lastResortFqdns.push(binding.fqdn)
}
const kind: FailoverEventKind =
removedFqdns.length > 0 ? 'removed' : 'last-resort'
return {
id: row.ip,
address: row.ip,
status: row.status,
kind,
fqdns: kind === 'removed' ? removedFqdns : lastResortFqdns,
consecutiveFailures: row.consecutive_failures ?? 0,
lastFailureReason: row.last_error ?? null,
lastCheckAt: row.last_checked_at ?? null,
}
})
}
const DEDUPE_WINDOW_MS = 2 * 60 * 1000
function fqdnsForIp(
ip: string,
bindings: readonly FailoverBindingPool[],
): string[] {
return bindings
.filter((binding) => isSharedPoolBinding(binding) && binding.configured.includes(ip))
.map((binding) => binding.fqdn)
}
function isPoolFqdn(
fqdn: string,
bindings: readonly FailoverBindingPool[],
): boolean {
const binding = bindings.find((item) => item.fqdn === fqdn)
if (!binding) return true
return isSharedPoolBinding(binding)
}
function fqdnLabel(fqdns: string[]): string {
return fqdns.join(', ') || 'пул'
}
function isAliveStatus(status: HealthLogStatus): boolean {
return status === 'up' || status === 'degraded'
}
/**
* Per-IP any-up flips: down → вышла из пула, up после down → вернулась.
* Initial state is not an event.
*/
export function toIpAliveTransitions(
probes: readonly HealthLogProbe[],
): Array<{ id: string; ip: string; alive: boolean; at: string }> {
const byIp = new Map<string, HealthLogProbe[]>()
for (const item of probes) {
const list = byIp.get(item.ip)
if (list) list.push(item)
else byIp.set(item.ip, [item])
}
const out: Array<{ id: string; ip: string; alive: boolean; at: string }> = []
for (const [ip, list] of byIp) {
list.sort(
(a, b) => probeTime(a.checked_at) - probeTime(b.checked_at) || a.id - b.id,
)
const latestByProvider = new Map<string, HealthLogProbe>()
let prevAlive: boolean | undefined
for (const probe of list) {
latestByProvider.set(probe.provider, probe)
const status = bestAliveHealthStatus(
[...latestByProvider.values()].map((item) => item.status),
)
if (status === 'unknown') continue
const alive = isAliveStatus(status)
if (prevAlive !== undefined && alive !== prevAlive) {
out.push({
id: `probe:${probe.id}`,
ip,
alive,
at: probe.checked_at,
})
}
prevAlive = alive
}
}
return out
}
function dnsHistoryItem(item: FailoverLogEntry): FailoverHistoryItem {
return {
id: `dns:${item.id}`,
ip: item.ip,
fqdn: item.fqdn,
action: item.action,
created_at: item.created_at,
copy:
item.action === 'removed'
? `${item.ip} убрана с ${item.fqdn}`
: `${item.ip} добавлена на ${item.fqdn}`,
source: 'dns',
}
}
function probeHistoryItem(
transition: { id: string; ip: string; alive: boolean; at: string },
bindings: readonly FailoverBindingPool[],
): FailoverHistoryItem {
const fqdns = fqdnsForIp(transition.ip, bindings)
const fqdn = fqdnLabel(fqdns)
const action = transition.alive ? 'added' : 'removed'
return {
id: transition.id,
ip: transition.ip,
fqdn,
action,
created_at: transition.at,
copy:
action === 'removed'
? `${transition.ip} вышла из пула (${fqdn})`
: `${transition.ip} вернулась в пул (${fqdn})`,
source: 'probe',
}
}
function eventTime(value: string): number {
return probeTime(value)
}
/**
* DNS add/remove + health leave/return, newest first.
* Same IP+action within 2 minutes: keep the DNS row (it has a concrete FQDN).
*/
export function mergeFailoverHistory(
dns: readonly FailoverLogEntry[],
probes: readonly HealthLogProbe[],
bindings: readonly FailoverBindingPool[] = [],
): FailoverHistoryItem[] {
const merged = [
...dns
.filter((item) => isPoolFqdn(item.fqdn, bindings))
.map(dnsHistoryItem),
...toIpAliveTransitions(probes)
.filter((transition) => fqdnsForIp(transition.ip, bindings).length > 0)
.map((transition) => probeHistoryItem(transition, bindings)),
]
merged.sort(
(a, b) => eventTime(b.created_at) - eventTime(a.created_at) || a.id.localeCompare(b.id),
)
const kept: FailoverHistoryItem[] = []
for (const item of merged) {
const duplicate = kept.find(
(other) =>
other.ip === item.ip &&
other.action === item.action &&
Math.abs(eventTime(other.created_at) - eventTime(item.created_at)) <=
DEDUPE_WINDOW_MS,
)
if (!duplicate) {
kept.push(item)
continue
}
if (duplicate.source === 'probe' && item.source === 'dns') {
kept[kept.indexOf(duplicate)] = item
}
}
return kept
}
+73 -2
View File
@@ -1,9 +1,13 @@
import { describe, expect, it } from 'vitest'
import {
bestAliveHealthStatus,
collapseStatusChanges,
enabledHealthProviders,
latestHealthByIp,
providerHealthStatuses,
resolveIpDisplayHealth,
resolveServiceDisplayHealth,
worstHealthStatus,
type HealthLogProbe,
} from '@/lib/health-log'
@@ -73,7 +77,7 @@ describe('enabledHealthProviders', () => {
})
describe('providerHealthStatuses', () => {
it('uses worst latest-per-ip status', () => {
it('uses any-up among latest-per-ip statuses', () => {
const items = [
probe({ id: 1, ip: '1.1.1.1', status: 'up', checked_at: '2026-01-01T00:02:00Z' }),
probe({ id: 2, ip: '2.2.2.2', status: 'down', checked_at: '2026-01-01T00:01:00Z' }),
@@ -84,7 +88,7 @@ describe('providerHealthStatuses', () => {
checked_at: '2026-01-01T00:00:00Z',
}),
]
expect(providerHealthStatuses(items, ['local']).local).toBe('down')
expect(providerHealthStatuses(items, ['local']).local).toBe('up')
})
})
@@ -95,3 +99,70 @@ describe('worstHealthStatus', () => {
expect(worstHealthStatus([])).toBe('unknown')
})
})
describe('bestAliveHealthStatus', () => {
it('is up if any IP is up', () => {
expect(bestAliveHealthStatus(['up', 'down'])).toBe('up')
expect(bestAliveHealthStatus(['degraded', 'down'])).toBe('degraded')
expect(bestAliveHealthStatus(['down'])).toBe('down')
expect(bestAliveHealthStatus([])).toBe('unknown')
})
})
describe('latestHealthByIp', () => {
it('any-up among latest-per-provider probes', () => {
const items = [
probe({
id: 1,
ip: '130.49.213.153',
provider: 'local',
status: 'up',
checked_at: '2026-08-20T09:00:00Z',
}),
probe({
id: 2,
ip: '130.49.213.153',
provider: 'cloudflare',
status: 'down',
checked_at: '2026-08-20T09:00:01Z',
}),
probe({
id: 3,
ip: '93.115.203.183',
status: 'unknown',
checked_at: '2026-08-20T08:59:00Z',
}),
]
const byIp = latestHealthByIp(items)
expect(byIp.get('130.49.213.153')?.status).toBe('up')
expect(byIp.get('93.115.203.183')?.status).toBe('unknown')
})
})
describe('resolveIpDisplayHealth', () => {
it('prefers a live probe over stored unknown', () => {
expect(resolveIpDisplayHealth('unknown', 'up')).toBe('up')
expect(resolveIpDisplayHealth('down', 'up')).toBe('up')
expect(resolveIpDisplayHealth('up', 'unknown')).toBe('up')
expect(resolveIpDisplayHealth('unknown', undefined)).toBe('unknown')
})
})
describe('resolveServiceDisplayHealth', () => {
it('shows OK when live probes recovered and stored is still unknown', () => {
expect(
resolveServiceDisplayHealth(
'unknown',
[{ ip: '2.59.161.102', status: 'unknown' }],
[
probe({
id: 1,
ip: '2.59.161.102',
status: 'up',
checked_at: '2026-08-20T18:00:00Z',
}),
],
),
).toBe('up')
})
})
+90 -2
View File
@@ -98,7 +98,18 @@ export function worstHealthStatus(statuses: readonly HealthLogStatus[]): HealthL
)
}
/** Latest probe per IP for a provider, then worst among those IPs. */
/** Any up → up; else degraded, then down, then unknown. */
export function bestAliveHealthStatus(
statuses: readonly HealthLogStatus[],
): HealthLogStatus {
if (statuses.length === 0) return 'unknown'
if (statuses.some((status) => status === 'up')) return 'up'
if (statuses.some((status) => status === 'degraded')) return 'degraded'
if (statuses.some((status) => status === 'down')) return 'down'
return 'unknown'
}
/** Latest probe per IP for a provider, then any-up among those IPs. */
export function providerHealthStatuses(
items: readonly HealthLogProbe[],
providers: readonly HealthCheckProvider[],
@@ -120,8 +131,85 @@ export function providerHealthStatuses(
const statuses = [...latestByIp.values()]
.filter((item) => item.provider === provider)
.map((item) => item.status)
result[provider] = worstHealthStatus(statuses)
result[provider] = bestAliveHealthStatus(statuses)
}
return result
}
export interface IpDisplayHealth {
status: HealthLogStatus
latency_ms: number | null
last_checked_at: string
last_error: string | null
colo: string | null
provider: HealthCheckProvider
}
/**
* Latest probe per provider+IP, then any-up among those providers.
* Used by the IP table so hysteresis `unknown` in ip_health does not hide a live OK.
*/
export function latestHealthByIp(
items: readonly HealthLogProbe[],
): Map<string, IpDisplayHealth> {
const latest = new Map<string, HealthLogProbe>()
const sorted = [...items].sort(
(a, b) => probeTime(b.checked_at) - probeTime(a.checked_at) || b.id - a.id,
)
for (const item of sorted) {
const key = `${item.provider}\0${item.ip}`
if (!latest.has(key)) latest.set(key, item)
}
const byIp = new Map<string, HealthLogProbe[]>()
for (const item of latest.values()) {
const list = byIp.get(item.ip)
if (list) list.push(item)
else byIp.set(item.ip, [item])
}
const result = new Map<string, IpDisplayHealth>()
for (const [ip, probes] of byIp) {
const status = bestAliveHealthStatus(probes.map((probe) => probe.status))
const preferred =
probes.find((probe) => probe.status === status) ?? probes[0]!
result.set(ip, {
status,
latency_ms: preferred.latency_ms,
last_checked_at: preferred.checked_at,
last_error: preferred.error,
colo: preferred.colo,
provider: preferred.provider,
})
}
return result
}
/** Prefer a concrete live probe over stored hysteresis `unknown`. */
export function resolveIpDisplayHealth(
stored: HealthLogStatus | undefined,
live: HealthLogStatus | undefined,
): HealthLogStatus {
if (live && live !== 'unknown') return live
return stored ?? live ?? 'unknown'
}
/** Service KPI / card: any-up of per-IP overlay (live probes beat hysteresis). */
export function resolveServiceDisplayHealth(
stored: HealthLogStatus | undefined,
ipHealth: readonly { ip: string; status: string }[],
probes: readonly HealthLogProbe[] = [],
): HealthLogStatus {
const liveByIp = latestHealthByIp(probes)
const statuses =
ipHealth.length > 0
? ipHealth.map((row) =>
resolveIpDisplayHealth(
row.status as HealthLogStatus,
liveByIp.get(row.ip)?.status,
),
)
: [...liveByIp.values()].map((row) => row.status)
return resolveIpDisplayHealth(stored, bestAliveHealthStatus(statuses))
}
+13
View File
@@ -84,6 +84,7 @@ export const serviceDomainBindingSchema = z
health_check_aggregate: z.enum(['any', 'all', 'majority']).catch('majority'),
cert_monitoring: z.enum(['auto', 'required', 'skipped']).default('auto'),
sync_status: z.string().nullable().default(null),
active_ips: z.array(z.string()).default([]),
})
.transform((binding) => ({
...binding,
@@ -125,6 +126,18 @@ export const healthProbeLogSchema = z.object({
checked_at: z.string(),
})
export const failoverLogSchema = z.object({
id: z.number(),
service_id: z.number(),
binding_id: z.number(),
fqdn: z.string(),
ip: z.string(),
action: z.enum(['added', 'removed']),
created_at: z.string(),
})
export type FailoverLogEntry = z.infer<typeof failoverLogSchema>
export const serviceViewSchema = serviceSchema.extend({
subdomain: z.string().default(''),
enabled: z.coerce.boolean().default(false),
+119 -26
View File
@@ -3,9 +3,11 @@ import { describe, expect, it } from 'vitest'
import {
DEFAULT_BINDING_HEALTH,
addAddressNode,
addCommonFqdn,
emptyAddressBlock,
emptyBindingDraft,
hydrateAddressBlock,
patchAddressIpMeta,
removeAddressNode,
toAddressBindings,
toDomainsPayload,
@@ -41,15 +43,15 @@ describe('hydrateAddressBlock', () => {
const state = hydrateAddressBlock(drafts, ['93.115.203.183', '185.244.181.61'])
expect(state.commonFqdn).toBe('rutg.rkns.top')
expect(state.commonFqdns).toEqual(['rutg.rkns.top'])
expect(state.nodes).toEqual([
{ ip: '93.115.203.183', extraFqdn: 'msk.rutg.rkns.top' },
{ ip: '185.244.181.61', extraFqdn: '' },
])
expect(state.otherBindings).toEqual([])
expect(state.preservedBindings).toEqual([])
})
it('не схлопывает CNAME и A на несколько IP', () => {
it('кладёт A на весь пул в commonFqdns, CNAME — в preserved', () => {
const cname: ServiceBindingDraft = {
...emptyBindingDraft('alias.rkns.top'),
record_type: 'CNAME',
@@ -63,14 +65,12 @@ describe('hydrateAddressBlock', () => {
const state = hydrateAddressBlock(drafts, ['1.1.1.1', '2.2.2.2'])
expect(state.commonFqdns).toEqual(['rutg.rkns.top', 'both.rkns.top'])
expect(state.nodes.every((node) => node.extraFqdn === '')).toBe(true)
expect(state.otherBindings.map((item) => item.fqdn)).toEqual([
'both.rkns.top',
'alias.rkns.top',
])
expect(state.preservedBindings.map((item) => item.fqdn)).toEqual(['alias.rkns.top'])
})
it('кладёт extra A с IP вне пула в otherBindings', () => {
it('кладёт extra A с IP вне пула в preservedBindings', () => {
const drafts = [
aRecord('gw.example.com', ['10.0.0.1']),
aRecord('edge.example.com', ['8.8.8.8']),
@@ -79,13 +79,49 @@ describe('hydrateAddressBlock', () => {
const state = hydrateAddressBlock(drafts, ['10.0.0.1'])
expect(state.nodes).toEqual([{ ip: '10.0.0.1', extraFqdn: '' }])
expect(state.otherBindings).toHaveLength(1)
expect(state.otherBindings[0]?.fqdn).toBe('edge.example.com')
expect(state.commonFqdns).toEqual(['gw.example.com'])
expect(state.preservedBindings).toHaveLength(1)
expect(state.preservedBindings[0]?.fqdn).toBe('edge.example.com')
})
it('при одном IP пула отделяет второй A в extraFqdn узла', () => {
const drafts = [
aRecord('dns.shnt.top', ['130.49.213.176']),
aRecord('ndns.shnt.top', ['130.49.213.176']),
]
const state = hydrateAddressBlock(drafts, ['130.49.213.176'])
expect(state.commonFqdns).toEqual(['dns.shnt.top'])
expect(state.nodes).toEqual([
{ ip: '130.49.213.176', extraFqdn: 'ndns.shnt.top' },
])
expect(state.preservedBindings).toEqual([])
})
it('поднимает веса и приоритеты с общего FQDN', () => {
const drafts = [
aRecord('gt.rkns.top', ['130.49.213.153', '93.115.203.183'], {
target_ip_weights: { '130.49.213.153': 3, '93.115.203.183': 1 },
target_ip_priorities: { '130.49.213.153': 2, '93.115.203.183': 1 },
}),
aRecord('nsgt.rkns.top', ['130.49.213.153']),
]
const state = hydrateAddressBlock(drafts, ['130.49.213.153', '93.115.203.183'])
expect(state.target_ip_weights).toEqual({
'130.49.213.153': 3,
'93.115.203.183': 1,
})
expect(state.target_ip_priorities).toEqual({
'130.49.213.153': 2,
'93.115.203.183': 1,
})
expect(state.nodes[0]?.extraFqdn).toBe('nsgt.rkns.top')
})
})
describe('toDomainsPayload', () => {
it('собирает primary на весь пул и extra binding на один IP', () => {
it('собирает каждый common на весь пул и extra binding на один IP', () => {
const drafts = [
aRecord('rutg.rkns.top', ['93.115.203.183', '185.244.181.61']),
aRecord('msk.rutg.rkns.top', ['93.115.203.183']),
@@ -107,28 +143,48 @@ describe('toDomainsPayload', () => {
])
})
it('круг hydrate → payload → hydrate сохраняет extra FQDN', () => {
it('круг hydrate → payload → hydrate сохраняет два common и extra FQDN', () => {
const drafts = [
aRecord('rutg.rkns.top', ['93.115.203.183', '185.244.181.61']),
aRecord('msk.rutg.rkns.top', ['93.115.203.183']),
aRecord('gt.rkns.top', ['93.115.203.183', '185.244.181.61']),
aRecord('msk.rkns.top', ['93.115.203.183', '185.244.181.61']),
aRecord('nsgt.rkns.top', ['93.115.203.183']),
]
const first = hydrateAddressBlock(drafts, ['93.115.203.183', '185.244.181.61'])
expect(first.commonFqdns).toEqual(['gt.rkns.top', 'msk.rkns.top'])
const rebound = toAddressBindings(first, primaryMeta)
const second = hydrateAddressBlock(rebound, rebound[0]?.target_ips ?? [])
expect(second.commonFqdn).toBe(first.commonFqdn)
expect(second.commonFqdns).toEqual(first.commonFqdns)
expect(second.nodes).toEqual(first.nodes)
expect(second.otherBindings).toEqual([])
expect(second.preservedBindings).toEqual([])
})
it('круг hydrate → payload → hydrate сохраняет extra FQDN при одном IP', () => {
const drafts = [
aRecord('dns.shnt.top', ['130.49.213.176']),
aRecord('ndns.shnt.top', ['130.49.213.176']),
]
const first = hydrateAddressBlock(drafts, ['130.49.213.176'])
expect(first.commonFqdns).toEqual(['dns.shnt.top'])
expect(first.nodes).toEqual([
{ ip: '130.49.213.176', extraFqdn: 'ndns.shnt.top' },
])
const rebound = toAddressBindings(first, primaryMeta)
const second = hydrateAddressBlock(rebound, ['130.49.213.176'])
expect(second.commonFqdns).toEqual(first.commonFqdns)
expect(second.nodes).toEqual(first.nodes)
expect(second.preservedBindings).toEqual([])
})
})
describe('removeAddressNode', () => {
it('удаляет extra FQDN узла и IP из other A-bindings', () => {
it('удаляет extra FQDN узла и IP из preserved A-bindings', () => {
const state = hydrateAddressBlock(
[
aRecord('gw.example.com', ['10.0.0.1', '10.0.0.2']),
aRecord('msk.example.com', ['10.0.0.1']),
aRecord('pair.example.com', ['10.0.0.1', '10.0.0.2']),
aRecord('edge.example.com', ['9.9.9.9', '10.0.0.1']),
],
['10.0.0.1', '10.0.0.2'],
)
@@ -136,12 +192,12 @@ describe('removeAddressNode', () => {
const next = removeAddressNode(state, '10.0.0.1')
expect(next.nodes).toEqual([{ ip: '10.0.0.2', extraFqdn: '' }])
expect(next.otherBindings).toHaveLength(1)
expect(next.otherBindings[0]?.target_ips).toEqual(['10.0.0.2'])
expect(next.preservedBindings).toHaveLength(1)
expect(next.preservedBindings[0]?.target_ips).toEqual(['9.9.9.9'])
})
})
describe('addAddressNode', () => {
describe('addAddressNode / addCommonFqdn', () => {
it('не добавляет дубликат IP', () => {
const withIp = addAddressNode(
{ ...emptyAddressBlock(), nodes: [{ ip: '1.1.1.1', extraFqdn: '' }] },
@@ -149,23 +205,60 @@ describe('addAddressNode', () => {
)
expect(withIp.nodes).toHaveLength(1)
})
it('не добавляет дубликат common FQDN', () => {
const state = addCommonFqdn(
{ ...emptyAddressBlock(), commonFqdns: ['gt.rkns.top'] },
'GT.rkns.top',
)
expect(state.commonFqdns).toEqual(['gt.rkns.top'])
})
})
describe('CNAME / otherBindings', () => {
it('сохраняет CNAME в otherBindings при круге hydrate → payload', () => {
describe('patchAddressIpMeta', () => {
it('меняет вес одного IP и не трогает extraFqdn', () => {
const state = {
...addAddressNode(addAddressNode(emptyAddressBlock(), '1.1.1.1'), '2.2.2.2'),
nodes: [
{ ip: '1.1.1.1', extraFqdn: 'msk.example.com' },
{ ip: '2.2.2.2', extraFqdn: '' },
],
}
const next = patchAddressIpMeta(state, '1.1.1.1', { weight: 7 })
expect(next.target_ip_weights).toEqual({ '1.1.1.1': 7, '2.2.2.2': 1 })
expect(next.target_ip_priorities).toEqual(state.target_ip_priorities)
expect(next.nodes).toEqual(state.nodes)
})
it('clamp веса и приоритета в 1–100', () => {
const state = addAddressNode(emptyAddressBlock(), '10.0.0.1')
expect(patchAddressIpMeta(state, '10.0.0.1', { weight: 0 }).target_ip_weights['10.0.0.1']).toBe(1)
expect(
patchAddressIpMeta(state, '10.0.0.1', { priority: 999 }).target_ip_priorities['10.0.0.1'],
).toBe(100)
})
it('игнорирует IP вне пула', () => {
const state = addAddressNode(emptyAddressBlock(), '10.0.0.1')
expect(patchAddressIpMeta(state, '8.8.8.8', { weight: 5 })).toBe(state)
})
})
describe('CNAME / preservedBindings', () => {
it('сохраняет CNAME в preserved при круге hydrate → payload', () => {
const cname: ServiceBindingDraft = {
...emptyBindingDraft('alias.rkns.top'),
record_type: 'CNAME',
target_cname: 'rutg.rkns.top',
}
const drafts = [
aRecord('rutg.rkns.top', ['1.1.1.1']),
aRecord('rutg.rkns.top', ['1.1.1.1', '2.2.2.2']),
aRecord('msk.rkns.top', ['1.1.1.1']),
cname,
]
const state = hydrateAddressBlock(drafts, ['1.1.1.1'])
const state = hydrateAddressBlock(drafts, ['1.1.1.1', '2.2.2.2'])
expect(state.nodes[0]?.extraFqdn).toBe('msk.rkns.top')
expect(state.otherBindings).toHaveLength(1)
expect(state.preservedBindings).toHaveLength(1)
const payload = toDomainsPayload(state, primaryMeta)
expect(payload.map((item) => item.fqdn)).toEqual([
+155 -50
View File
@@ -37,9 +37,9 @@ export interface AddressNode {
}
export interface AddressBlockState {
commonFqdn: string
commonFqdns: string[]
nodes: AddressNode[]
otherBindings: ServiceBindingDraft[]
preservedBindings: ServiceBindingDraft[]
target_ip_weights: Record<string, number>
target_ip_priorities: Record<string, number>
}
@@ -78,31 +78,14 @@ export function emptyBindingDraft(fqdn = ''): ServiceBindingDraft {
export function emptyAddressBlock(): AddressBlockState {
return {
commonFqdn: '',
commonFqdns: [],
nodes: [],
otherBindings: [],
preservedBindings: [],
target_ip_weights: {},
target_ip_priorities: {},
}
}
export function withPoolIps(
draft: ServiceBindingDraft,
pool: string[],
): ServiceBindingDraft {
if (draft.record_type !== 'A' || draft.target_ips.length > 0 || pool.length === 0) {
return draft
}
return {
...draft,
target_ips: pool,
target_ip_weights: Object.fromEntries(pool.map((ip) => [ip, draft.target_ip_weights[ip] ?? 1])),
target_ip_priorities: Object.fromEntries(
pool.map((ip) => [ip, draft.target_ip_priorities[ip] ?? 1]),
),
}
}
function uniqueIps(...lists: string[][]): string[] {
const seen = new Set<string>()
const out: string[] = []
@@ -123,6 +106,13 @@ function omitKey(record: Record<string, number>, key: string): Record<string, nu
return next
}
function sameIpSet(left: string[], right: string[]): boolean {
if (left.length === 0 || left.length !== right.length) return false
const set = new Set(left.map((ip) => ip.trim()).filter(Boolean))
if (set.size !== left.length) return false
return right.every((ip) => set.has(ip.trim()))
}
export function toBindingDrafts(service: ServiceView): ServiceBindingDraft[] {
return (service.domains ?? []).map((binding) => ({
fqdn: bindingToFqdn(binding),
@@ -151,49 +141,100 @@ export function toBindingDrafts(service: ServiceView): ServiceBindingDraft[] {
}))
}
function canCollapseToNode(
extra: ServiceBindingDraft,
pool: Set<string>,
claimed: Set<string>,
): string | null {
if (extra.record_type !== 'A') return null
if (extra.target_ips.length !== 1) return null
const ip = extra.target_ips[0]?.trim() ?? ''
if (!ip || !pool.has(ip) || claimed.has(ip)) return null
if (!extra.fqdn.trim()) return null
return ip
function isFullPoolA(draft: ServiceBindingDraft, pool: string[]): boolean {
return draft.record_type === 'A' && sameIpSet(draft.target_ips, pool)
}
function takeAsCommon(
draft: ServiceBindingDraft,
fqdn: string,
commonFqdns: string[],
weights: Record<string, number>,
priorities: Record<string, number>,
): { weights: Record<string, number>; priorities: Record<string, number> } {
if (fqdn) commonFqdns.push(draft.fqdn)
if (Object.keys(weights).length === 0) {
return {
weights: { ...draft.target_ip_weights },
priorities: { ...draft.target_ip_priorities },
}
}
return { weights, priorities }
}
export function hydrateAddressBlock(
drafts: ServiceBindingDraft[],
pool: string[] = [],
): AddressBlockState {
const primary = drafts[0]
const ips = uniqueIps(pool, primary?.target_ips ?? [])
const multiIpTargets = drafts
.filter((draft) => draft.record_type === 'A' && draft.target_ips.length > 1)
.map((draft) => draft.target_ips)
const allAIps = drafts
.filter((draft) => draft.record_type === 'A')
.map((draft) => draft.target_ips)
const ips =
pool.length > 0
? uniqueIps(pool)
: uniqueIps(...(multiIpTargets.length > 0 ? multiIpTargets : allAIps))
const poolSet = new Set(ips)
const commonFqdns: string[] = []
const claimed = new Set<string>()
const extraByIp = new Map<string, string>()
const otherBindings: ServiceBindingDraft[] = []
const preservedBindings: ServiceBindingDraft[] = []
let weights: Record<string, number> = {}
let priorities: Record<string, number> = {}
const splitSinglePool =
ips.length === 1 &&
drafts.filter((draft) => isFullPoolA(draft, ips)).length > 1
let assignedFirstSinglePoolCommon = false
for (const extra of drafts.slice(1)) {
const ip = canCollapseToNode(extra, poolSet, claimed)
if (ip) {
claimed.add(ip)
extraByIp.set(ip, extra.fqdn)
for (const draft of drafts) {
const fqdn = draft.fqdn.trim()
if (splitSinglePool && isFullPoolA(draft, ips)) {
if (!assignedFirstSinglePoolCommon) {
assignedFirstSinglePoolCommon = true
const next = takeAsCommon(draft, fqdn, commonFqdns, weights, priorities)
weights = next.weights
priorities = next.priorities
continue
}
const ip = draft.target_ips[0]?.trim() ?? ''
if (ip && poolSet.has(ip) && fqdn && !claimed.has(ip)) {
claimed.add(ip)
extraByIp.set(ip, draft.fqdn)
continue
}
const overflow = takeAsCommon(draft, fqdn, commonFqdns, weights, priorities)
weights = overflow.weights
priorities = overflow.priorities
continue
}
otherBindings.push(extra)
if (isFullPoolA(draft, ips)) {
const next = takeAsCommon(draft, fqdn, commonFqdns, weights, priorities)
weights = next.weights
priorities = next.priorities
continue
}
if (draft.record_type === 'A' && draft.target_ips.length === 1) {
const ip = draft.target_ips[0]?.trim() ?? ''
if (ip && poolSet.has(ip) && fqdn && !claimed.has(ip)) {
claimed.add(ip)
extraByIp.set(ip, draft.fqdn)
continue
}
}
preservedBindings.push(draft)
}
return {
commonFqdn: primary?.fqdn ?? '',
commonFqdns,
nodes: ips.map((ip) => ({
ip,
extraFqdn: extraByIp.get(ip) ?? '',
})),
otherBindings,
target_ip_weights: { ...(primary?.target_ip_weights ?? {}) },
target_ip_priorities: { ...(primary?.target_ip_priorities ?? {}) },
preservedBindings,
target_ip_weights: weights,
target_ip_priorities: priorities,
}
}
@@ -221,7 +262,7 @@ export function removeAddressNode(state: AddressBlockState, ip: string): Address
return {
...state,
nodes: state.nodes.filter((node) => node.ip !== ip),
otherBindings: pruneIpFromBindings(state.otherBindings, ip),
preservedBindings: pruneIpFromBindings(state.preservedBindings, ip),
target_ip_weights: omitKey(state.target_ip_weights, ip),
target_ip_priorities: omitKey(state.target_ip_priorities, ip),
}
@@ -240,6 +281,69 @@ export function addAddressNode(state: AddressBlockState, ip: string): AddressBlo
}
}
const LB_META_MIN = 1
const LB_META_MAX = 100
function clampLbMeta(value: number): number {
if (!Number.isFinite(value)) return LB_META_MIN
return Math.min(LB_META_MAX, Math.max(LB_META_MIN, Math.round(value)))
}
export function patchAddressIpMeta(
state: AddressBlockState,
ip: string,
meta: { weight?: number; priority?: number },
): AddressBlockState {
if (!state.nodes.some((node) => node.ip === ip)) return state
return {
...state,
target_ip_weights:
meta.weight === undefined
? state.target_ip_weights
: { ...state.target_ip_weights, [ip]: clampLbMeta(meta.weight) },
target_ip_priorities:
meta.priority === undefined
? state.target_ip_priorities
: { ...state.target_ip_priorities, [ip]: clampLbMeta(meta.priority) },
}
}
function fqdnKey(value: string): string {
return value.trim().toLowerCase()
}
export function addressHasFqdn(state: AddressBlockState, fqdn: string): boolean {
const key = fqdnKey(fqdn)
if (!key) return false
if (state.commonFqdns.some((item) => fqdnKey(item) === key)) return true
if (state.nodes.some((node) => fqdnKey(node.extraFqdn) === key)) return true
return false
}
export function addCommonFqdn(state: AddressBlockState, fqdn: string): AddressBlockState {
const trimmed = fqdn.trim()
if (!trimmed || addressHasFqdn(state, trimmed)) return state
return { ...state, commonFqdns: [...state.commonFqdns, trimmed] }
}
export function removeCommonFqdn(state: AddressBlockState, index: number): AddressBlockState {
return {
...state,
commonFqdns: state.commonFqdns.filter((_, i) => i !== index),
}
}
export function updateCommonFqdn(
state: AddressBlockState,
index: number,
fqdn: string,
): AddressBlockState {
return {
...state,
commonFqdns: state.commonFqdns.map((item, i) => (i === index ? fqdn : item)),
}
}
export function toAddressBindings(
state: AddressBlockState,
primary: AddressPrimaryMeta,
@@ -253,10 +357,11 @@ export function toAddressBindings(
)
const drafts: ServiceBindingDraft[] = []
const hasPrimary = Boolean(state.commonFqdn.trim()) || ips.length > 0
if (hasPrimary) {
for (const raw of state.commonFqdns) {
const fqdn = raw.trim()
if (!fqdn || ips.length === 0) continue
drafts.push({
fqdn: state.commonFqdn,
fqdn,
record_type: 'A',
target_ips: ips,
target_cname: '',
@@ -282,7 +387,7 @@ export function toAddressBindings(
})
}
drafts.push(...state.otherBindings)
drafts.push(...state.preservedBindings)
return drafts
}
+17
View File
@@ -1,6 +1,7 @@
import { queryOptions } from '@tanstack/react-query'
import { api } from '@/lib/api-client'
import {
failoverLogSchema,
healthProbeLogSchema,
serviceBindingSchema,
serviceGroupsResponseSchema,
@@ -35,6 +36,8 @@ export const serviceGroupsQueryOptions = () =>
}
return parsed.data
},
refetchInterval: 10_000,
staleTime: 5_000,
})
export const servicesQueryOptions = () =>
@@ -89,6 +92,7 @@ export const serviceDetailKeys = {
overview: (id: number) => [...serviceKeys.all, id, 'overview'] as const,
nodes: (id: number) => [...serviceKeys.all, id, 'nodes'] as const,
healthLog: (id: number) => [...serviceKeys.all, id, 'health-log'] as const,
failoverLog: (id: number) => [...serviceKeys.all, id, 'failover-log'] as const,
view: (id: number) => [...serviceKeys.all, id, 'view'] as const,
}
@@ -99,6 +103,8 @@ export const serviceViewQueryOptions = (id: number) =>
const data = await api.get<unknown>(`/api/v1/services/${id}`)
return serviceViewSchema.parse(data)
},
refetchInterval: 10_000,
staleTime: 5_000,
})
export const serviceHealthLogQueryOptions = (id: number) =>
@@ -108,6 +114,17 @@ export const serviceHealthLogQueryOptions = (id: number) =>
const data = await api.get<unknown>(`/api/v1/services/${id}/health-log`)
return z.object({ items: z.array(healthProbeLogSchema) }).parse(data)
},
refetchInterval: 10_000,
staleTime: 5_000,
})
export const serviceFailoverLogQueryOptions = (id: number) =>
queryOptions({
queryKey: serviceDetailKeys.failoverLog(id),
queryFn: async () => {
const data = await api.get<unknown>(`/api/v1/services/${id}/failover-log`)
return z.object({ items: z.array(failoverLogSchema) }).parse(data)
},
})
export const serviceOverviewQueryOptions = (id: number) =>
@@ -14,7 +14,6 @@ import {
import { ChangeDomainSheet } from '@/components/change-domain-sheet'
import { ChangeIpSheet } from '@/components/change-ip-sheet'
import { EmptyState } from '@/components/empty-state'
import { FailoverTimeline } from '@/components/failover-timeline'
import { FormFieldSimple } from '@/components/form-field'
import { FormSheet } from '@/components/form-sheet'
import { HealthCheckBadge } from '@/components/health-check-badge'
@@ -29,19 +28,15 @@ import {
import { LbModeTile } from '@/components/services/service-unit-card'
import {
KpiStatGrid,
ServiceFailoverPanel,
ServiceHealthMonitor,
} from '@/components/reui-kit'
import {
Frame,
FrameDescription,
FrameHeader,
FramePanel,
FrameTitle,
} from '@/components/reui/frame'
import { api } from '@/lib/api-client'
import { hasSharedPool } from '@/lib/failover-events'
import {
enabledHealthProviders,
providerHealthStatuses,
resolveServiceDisplayHealth,
} from '@/lib/health-log'
import type { ServiceView, UpdateServiceConfigInput } from '@/lib/schemas'
import {
@@ -51,6 +46,7 @@ import {
domainsListQueryOptions,
serviceBindingKeys,
serviceDetailKeys,
serviceFailoverLogQueryOptions,
serviceGroupKeys,
serviceGroupsQueryOptions,
serviceHealthLogQueryOptions,
@@ -84,6 +80,7 @@ interface OverviewPayload {
priority: number
consecutive_failures: number
last_failure_reason: string | null
last_check_at?: string | null
}>
}
@@ -96,6 +93,7 @@ function ServiceDetailPage() {
const viewQuery = useQuery(serviceViewQueryOptions(id))
const overviewQuery = useQuery(serviceOverviewQueryOptions(id))
const logQuery = useQuery(serviceHealthLogQueryOptions(id))
const failoverLogQuery = useQuery(serviceFailoverLogQueryOptions(id))
const nodesQuery = useQuery(serviceNodesQueryOptions(id))
const groupsQuery = useQuery(serviceGroupsQueryOptions())
const domainsQuery = useQuery(domainsListQueryOptions())
@@ -106,6 +104,35 @@ function ServiceDetailPage() {
() => logQuery.data?.items ?? [],
[logQuery.data?.items],
)
const failoverHistory = useMemo(
() => failoverLogQuery.data?.items ?? [],
[failoverLogQuery.data?.items],
)
const failoverBindings = useMemo(
() =>
(service?.domains ?? []).map((domain) => ({
fqdn: domain.fqdn,
configured: domain.target_ips,
active: domain.active_ips ?? [],
})),
[service?.domains],
)
const uniqueEnabledIps = useMemo(
() =>
(service?.ips ?? []).filter((ip) => service?.ip_enabled[ip] !== false),
[service?.ips, service?.ip_enabled],
)
const displayHealth = useMemo(
() =>
resolveServiceDisplayHealth(
service?.health_status,
service?.ip_health ?? [],
logItems,
),
[service?.health_status, service?.ip_health, logItems],
)
const showPoolPanel =
uniqueEnabledIps.length >= 2 && hasSharedPool(failoverBindings)
const nodes = (nodesQuery.data as OverviewPayload['nodes']) ?? []
const [editOpen, setEditOpen] = useState(false)
@@ -135,6 +162,7 @@ function ServiceDetailPage() {
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.overview(id) }),
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.nodes(id) }),
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.healthLog(id) }),
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.failoverLog(id) }),
])
}
@@ -209,22 +237,6 @@ function ServiceDetailPage() {
const isError = viewQuery.isError || overviewQuery.isError
const error = viewQuery.error ?? overviewQuery.error
const failoverEvents =
(nodes.length > 0 ? nodes : (overview?.nodes ?? []))
.filter(
(node) =>
node.health_status === 'unhealthy' ||
node.health_status === 'down' ||
node.health_status === 'checking',
)
.map((node) => ({
id: node.address,
title: `${node.address}: ${node.health_status}`,
detail: node.last_failure_reason
? `${node.last_failure_reason} · fail ${node.consecutive_failures}`
: `fail ${node.consecutive_failures}`,
}))
const enabledProviders = useMemo(
() => enabledHealthProviders(service?.domains ?? []),
[service],
@@ -257,7 +269,7 @@ function ServiceDetailPage() {
actions={
<>
<LbModeTile mode={service.lb_mode} />
<HealthCheckBadge status={service.health_status} />
<HealthCheckBadge status={displayHealth} />
<Tooltip>
<TooltipTrigger
render={
@@ -284,20 +296,20 @@ function ServiceDetailPage() {
id: 'status',
icon: <ActivityIcon />,
label: 'Статус',
value: service.health_status === 'up' ? 'OK' : service.health_status,
value: displayHealth === 'up' ? 'OK' : displayHealth,
variant:
service.health_status === 'down'
displayHealth === 'down'
? 'destructive'
: service.health_status === 'degraded'
: displayHealth === 'degraded'
? 'warning'
: 'default',
iconClassName:
service.health_status === 'down'
displayHealth === 'down'
? 'text-destructive'
: service.health_status === 'degraded'
: displayHealth === 'degraded'
? 'text-warning'
: 'text-success',
hint: <HealthCheckBadge status={service.health_status} size="xs" />,
hint: <HealthCheckBadge status={displayHealth} size="xs" />,
},
{
id: 'fqdn',
@@ -311,21 +323,33 @@ function ServiceDetailPage() {
icon: <NetworkIcon />,
label: 'IP',
value: String(service.ips.length),
hint: `${service.active_ips.length} в пуле`,
hint: showPoolPanel
? `${service.active_ips.length} в пуле`
: uniqueEnabledIps.length <= 1
? 'без балансировки'
: service.ips.join(', ') || 'нет',
},
{
id: 'pool',
icon: <ServerIcon />,
label: 'Активный пул',
value: String((overview?.active_addresses ?? service.active_ips).length),
hint: (overview?.active_addresses ?? service.active_ips).join(', ') || 'нет',
label: showPoolPanel ? 'Активный пул' : 'Адреса',
value: String(
(overview?.active_addresses ?? service.active_ips).length,
),
hint:
(overview?.active_addresses ?? service.active_ips).join(', ') ||
'нет',
},
]}
/>
<section
aria-label="Мониторинг"
className="grid min-w-0 items-start gap-2 @3xl:grid-cols-2"
className={
showPoolPanel
? 'grid min-w-0 items-start gap-2 @3xl:grid-cols-2'
: 'grid min-w-0 items-start gap-2'
}
>
<ServiceHealthMonitor
items={logItems}
@@ -333,17 +357,15 @@ function ServiceDetailPage() {
statuses={providerStatuses}
isLoading={logQuery.isLoading}
/>
<Frame dense spacing="sm" className="min-w-0 w-full">
<FrameHeader>
<FrameTitle>Failover</FrameTitle>
<FrameDescription>
Нездоровые ноды и причины последней ошибки
</FrameDescription>
</FrameHeader>
<FramePanel>
<FailoverTimeline events={failoverEvents} />
</FramePanel>
</Frame>
{showPoolPanel ? (
<ServiceFailoverPanel
lbMode={service.lb_mode}
ipHealth={service.ip_health}
bindings={failoverBindings}
history={failoverHistory}
probes={logItems}
/>
) : null}
</section>
{service.ips.length === 0 && service.domains.length === 0 ? (
@@ -356,6 +378,7 @@ function ServiceDetailPage() {
<ServiceDetailGrid
service={service}
nodes={nodes}
probes={logItems}
togglingIp={togglingIp}
onToggleIp={(ip, enabled) => {
setTogglingIp(ip)
@@ -1,6 +1,7 @@
import { createFileRoute, Outlet } from '@tanstack/react-router'
import { PageShell } from '@/components/page-shell'
import {
serviceFailoverLogQueryOptions,
serviceHealthLogQueryOptions,
serviceNodesQueryOptions,
serviceOverviewQueryOptions,
@@ -14,6 +15,7 @@ export const Route = createFileRoute('/_auth/services/$serviceId')({
queryClient.ensureQueryData(serviceViewQueryOptions(id)),
queryClient.ensureQueryData(serviceOverviewQueryOptions(id)),
queryClient.ensureQueryData(serviceHealthLogQueryOptions(id)),
queryClient.ensureQueryData(serviceFailoverLogQueryOptions(id)),
queryClient.ensureQueryData(serviceNodesQueryOptions(id)),
])
return { breadcrumb: view.name }
+4 -3
View File
@@ -44,9 +44,10 @@ health-check работают на двух уровнях:
- **Change Domain** — перенос привязок между зонами `POST /api/v1/services/:id/change-domain`.
Режимы LB: `round_robin`, `failover`, `weighted`. В Cloudflare free `weighted`
работает как `round_robin` (одна A на IP). `unknown` **не** считается healthy и
не попадает в пул, пока нет успешных проб; восстановление — `UNHEALTHY → CHECKING → HEALTHY`
после `HEALTH_SUCCESS_RECOVERIES` (default 2). Пороги и cron движка задаются в
работает как `round_robin` (одна A на IP). В A-пул попадает всё, кроме **Down**
(`unknown` / checking / Slow возвращаются в DNS на первой успешной пробе).
Бейдж Healthy — `UNHEALTHY → CHECKING → HEALTHY` после `HEALTH_SUCCESS_RECOVERIES`
(default 2). Пороги и cron движка задаются в
**Настройки → Health-check** (env — fallback, пока значения не сохранены в UI).
### Источники проб: Local, Cloudflare Worker, Globalping
+295 -3
View File
File diff suppressed because one or more lines are too long
+40 -1
View File
@@ -355,6 +355,15 @@ var notificationLog = sqliteTable("notification_log", {
message: text("message").notNull(),
created_at: text("created_at").notNull().default(sql`datetime('now')`)
});
var failoverLog = sqliteTable("failover_log", {
id: integer("id").primaryKey({ autoIncrement: true }),
service_id: integer("service_id").notNull().references(() => services.id, { onDelete: "cascade" }),
binding_id: integer("binding_id").notNull().references(() => serviceBindings.id, { onDelete: "cascade" }),
fqdn: text("fqdn").notNull(),
ip: text("ip").notNull(),
action: text("action").notNull(),
created_at: text("created_at").notNull().default(sql`datetime('now')`)
});
var auditLog = sqliteTable("audit_log", {
id: text("id").primaryKey(),
event_id: text("event_id"),
@@ -395,6 +404,7 @@ var schema = {
domainMonitorResults,
healthProbeLog,
notificationLog,
failoverLog,
auditLog
};
@@ -715,6 +725,7 @@ __export(repos_exports, {
getSyncJob: () => getSyncJob,
insertBinding: () => insertBinding,
insertDnsRecord: () => insertDnsRecord,
insertFailoverLog: () => insertFailoverLog,
insertHealthProbeLog: () => insertHealthProbeLog,
insertNotificationLog: () => insertNotificationLog,
linkBindingRecord: () => linkBindingRecord,
@@ -738,6 +749,7 @@ __export(repos_exports, {
listDomains: () => listDomains,
listDomainsEnriched: () => listDomainsEnriched,
listEnabledDomainMonitors: () => listEnabledDomainMonitors,
listFailoverLogForService: () => listFailoverLogForService,
listGroupDnsRecords: () => listGroupDnsRecords,
listGroups: () => listGroups,
listHealthCheckTargets: () => listHealthCheckTargets,
@@ -1995,6 +2007,12 @@ var WORST_HEALTH_SQL = sql2.raw(`CASE
END) = 1 THEN 'up'
ELSE 'unknown'
END`);
var BEST_ALIVE_HEALTH_SQL = sql2.raw(`CASE
WHEN MAX(CASE WHEN status = 'up' THEN 1 ELSE 0 END) = 1 THEN 'up'
WHEN MAX(CASE WHEN status = 'degraded' THEN 1 ELSE 0 END) = 1 THEN 'degraded'
WHEN MAX(CASE WHEN status = 'down' THEN 1 ELSE 0 END) = 1 THEN 'down'
ELSE 'unknown'
END`);
function aggregateIpHealthByRefs(db, scope, refIds) {
const result = /* @__PURE__ */ new Map();
if (refIds.length === 0) return result;
@@ -2055,7 +2073,7 @@ function aggregateIpHealthByServiceIds(db, serviceIds) {
);
const rows = db.all(sql2`
SELECT sb.service_id AS service_id,
${WORST_HEALTH_SQL} AS health_status,
${BEST_ALIVE_HEALTH_SQL} AS health_status,
MAX(ihs.latency_ms) AS health_latency_ms
FROM ip_health_status ihs
INNER JOIN service_bindings sb
@@ -2545,6 +2563,26 @@ function listNotificationLog(db, limit = 50) {
LIMIT ${limit}
`);
}
function insertFailoverLog(db, input) {
for (const entry of input.entries) {
db.insert(failoverLog).values({
service_id: input.serviceId,
binding_id: input.bindingId,
fqdn: input.fqdn,
ip: entry.ip,
action: entry.action
}).run();
}
}
function listFailoverLogForService(db, serviceId, limit = 100) {
return db.all(sql2`
SELECT id, service_id, binding_id, fqdn, ip, action, created_at
FROM failover_log
WHERE service_id = ${serviceId}
ORDER BY created_at DESC, id DESC
LIMIT ${limit}
`);
}
export {
ConflictError,
NotFoundError,
@@ -2560,6 +2598,7 @@ export {
domainMonitors,
domainTags,
domains,
failoverLog,
getAppSettings,
getAppSettingsSecrets,
groups,
@@ -0,0 +1,11 @@
CREATE TABLE failover_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
service_id INTEGER NOT NULL REFERENCES services(id) ON DELETE CASCADE,
binding_id INTEGER NOT NULL REFERENCES service_bindings(id) ON DELETE CASCADE,
fqdn TEXT NOT NULL,
ip TEXT NOT NULL,
action TEXT NOT NULL CHECK (action IN ('added', 'removed')),
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX idx_failover_log_service ON failover_log(service_id, created_at DESC);
+166 -2
View File
@@ -49,6 +49,7 @@ import {
nodes,
bindingNodes,
notificationLog,
failoverLog,
serviceBindingIps,
serviceBindingRecords,
serviceBindings,
@@ -2163,6 +2164,14 @@ const WORST_HEALTH_SQL = sql.raw(`CASE
ELSE 'unknown'
END`);
/** Service is up if any binding IP is up. */
const BEST_ALIVE_HEALTH_SQL = sql.raw(`CASE
WHEN MAX(CASE WHEN status = 'up' THEN 1 ELSE 0 END) = 1 THEN 'up'
WHEN MAX(CASE WHEN status = 'degraded' THEN 1 ELSE 0 END) = 1 THEN 'degraded'
WHEN MAX(CASE WHEN status = 'down' THEN 1 ELSE 0 END) = 1 THEN 'down'
ELSE 'unknown'
END`);
/** Worst status across ip_health_status rows for each ref_id in a scope. */
export function aggregateIpHealthByRefs(
db: Db,
@@ -2236,7 +2245,7 @@ export function aggregateGroupScopeHealthByIds(
return result;
}
/** Worst binding-scope health rolled up per service_id. */
/** Binding-scope health rolled up per service_id: any up → service up. */
export function aggregateIpHealthByServiceIds(
db: Db,
serviceIds: number[],
@@ -2253,7 +2262,7 @@ export function aggregateIpHealthByServiceIds(
health_latency_ms: number | null;
}>(sql`
SELECT sb.service_id AS service_id,
${WORST_HEALTH_SQL} AS health_status,
${BEST_ALIVE_HEALTH_SQL} AS health_status,
MAX(ihs.latency_ms) AS health_latency_ms
FROM ip_health_status ihs
INNER JOIN service_bindings sb
@@ -2332,6 +2341,113 @@ export function listIpHealthByServiceIds(
return result;
}
function parseIpHealthState(status: string | null): IpHealthState | undefined {
if (
status === "up" ||
status === "down" ||
status === "degraded" ||
status === "unknown"
) {
return status;
}
return undefined;
}
function bestAliveIpState(statuses: readonly IpHealthState[]): IpHealthState {
if (statuses.some((status) => status === "up")) return "up";
if (statuses.some((status) => status === "degraded")) return "degraded";
if (statuses.some((status) => status === "down")) return "down";
return "unknown";
}
/**
* Latest probe per service+IP+provider from health_probe_log, then any-up per IP.
* Display overlay — hysteresis in ip_health_status is unchanged (DNS).
*/
export function listLatestLiveHealthByServiceIds(
db: Db,
serviceIds: number[],
): Map<number, ServiceIpHealthRow[]> {
const result = new Map<number, ServiceIpHealthRow[]>();
if (serviceIds.length === 0) return result;
const idList = sql.join(
serviceIds.map((id) => sql`${id}`),
sql`, `,
);
const rows = db.all<{
service_id: number;
ip: string;
provider: string | null;
status: string | null;
latency_ms: number | null;
last_error: string | null;
colo: string | null;
checked_at: string | null;
}>(sql`
SELECT sb.service_id AS service_id,
l.ip AS ip,
l.provider AS provider,
l.status AS status,
l.latency_ms AS latency_ms,
l.error AS last_error,
l.colo AS colo,
l.checked_at AS checked_at
FROM health_probe_log l
INNER JOIN service_bindings sb
ON l.scope = 'binding' AND l.ref_id = sb.id
INNER JOIN (
SELECT sb2.service_id AS service_id,
l2.ip AS ip,
l2.provider AS provider,
MAX(l2.id) AS max_id
FROM health_probe_log l2
INNER JOIN service_bindings sb2
ON l2.scope = 'binding' AND l2.ref_id = sb2.id
WHERE sb2.service_id IN (${idList})
GROUP BY sb2.service_id, l2.ip, l2.provider
) latest
ON latest.max_id = l.id
`);
const byServiceIp = new Map<
string,
{ serviceId: number; ip: string; probes: ServiceIpHealthRow[] }
>();
for (const row of rows) {
const status = parseIpHealthState(row.status);
if (!status) continue;
const key = `${row.service_id}\0${row.ip}`;
const probe: ServiceIpHealthRow = {
ip: row.ip,
status,
latency_ms: row.latency_ms,
last_checked_at: row.checked_at,
last_error: row.last_error,
provider: normalizeStatusProvider(row.provider),
colo: row.colo,
};
const bucket = byServiceIp.get(key);
if (bucket) bucket.probes.push(probe);
else {
byServiceIp.set(key, {
serviceId: row.service_id,
ip: row.ip,
probes: [probe],
});
}
}
for (const { serviceId, ip, probes } of byServiceIp.values()) {
const status = bestAliveIpState(probes.map((probe) => probe.status));
const preferred =
probes.find((probe) => probe.status === status) ?? probes[0]!;
const list = result.get(serviceId) ?? [];
list.push({ ...preferred, ip, status });
result.set(serviceId, list);
}
return result;
}
export function mergeHealthAggregates(
parts: Array<HealthAggregate | undefined | null>,
): HealthAggregate {
@@ -3055,3 +3171,51 @@ export function listNotificationLog(
`);
}
export type FailoverLogAction = "added" | "removed";
export type FailoverLogRow = {
id: number;
service_id: number;
binding_id: number;
fqdn: string;
ip: string;
action: FailoverLogAction;
created_at: string;
};
export function insertFailoverLog(
db: Db,
input: {
serviceId: number;
bindingId: number;
fqdn: string;
entries: Array<{ ip: string; action: FailoverLogAction }>;
},
): void {
for (const entry of input.entries) {
db.insert(failoverLog)
.values({
service_id: input.serviceId,
binding_id: input.bindingId,
fqdn: input.fqdn,
ip: entry.ip,
action: entry.action,
})
.run();
}
}
export function listFailoverLogForService(
db: Db,
serviceId: number,
limit = 100,
): FailoverLogRow[] {
return db.all<FailoverLogRow>(sql`
SELECT id, service_id, binding_id, fqdn, ip, action, created_at
FROM failover_log
WHERE service_id = ${serviceId}
ORDER BY created_at DESC, id DESC
LIMIT ${limit}
`);
}
+17
View File
@@ -497,6 +497,22 @@ export const notificationLog = sqliteTable("notification_log", {
.default(sql`datetime('now')`),
});
export const failoverLog = sqliteTable("failover_log", {
id: integer("id").primaryKey({ autoIncrement: true }),
service_id: integer("service_id")
.notNull()
.references(() => services.id, { onDelete: "cascade" }),
binding_id: integer("binding_id")
.notNull()
.references(() => serviceBindings.id, { onDelete: "cascade" }),
fqdn: text("fqdn").notNull(),
ip: text("ip").notNull(),
action: text("action").notNull(),
created_at: text("created_at")
.notNull()
.default(sql`datetime('now')`),
});
export const auditLog = sqliteTable("audit_log", {
id: text("id").primaryKey(),
event_id: text("event_id"),
@@ -540,5 +556,6 @@ export const schema = {
domainMonitorResults,
healthProbeLog,
notificationLog,
failoverLog,
auditLog,
};
+30 -1
View File
@@ -143,6 +143,7 @@ interface ServiceDomainBindingView {
fqdn: string;
record_type: "A" | "CNAME";
target_ips: string[];
active_ips: string[];
target_ip_weights: Record<string, number>;
target_ip_priorities: Record<string, number>;
target_cname: string | null;
@@ -505,6 +506,19 @@ declare const healthProbeLogSchema: z.ZodObject<{
checked_at: z.ZodString;
}, z.core.$strip>;
type HealthProbeLog = z.infer<typeof healthProbeLogSchema>;
declare const failoverLogSchema: z.ZodObject<{
id: z.ZodNumber;
service_id: z.ZodNumber;
binding_id: z.ZodNumber;
fqdn: z.ZodString;
ip: z.ZodString;
action: z.ZodEnum<{
added: "added";
removed: "removed";
}>;
created_at: z.ZodString;
}, z.core.$strip>;
type FailoverLogEntry = z.infer<typeof failoverLogSchema>;
declare const groupSchema: z.ZodObject<{
id: z.ZodNumber;
name: z.ZodString;
@@ -643,6 +657,7 @@ declare const serviceDomainBindingSchema: z.ZodPipe<z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -668,6 +683,7 @@ declare const serviceDomainBindingSchema: z.ZodPipe<z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -690,6 +706,7 @@ declare const serviceDomainBindingSchema: z.ZodPipe<z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -763,6 +780,7 @@ declare const serviceViewSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -788,6 +806,7 @@ declare const serviceViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -810,6 +829,7 @@ declare const serviceViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -965,6 +985,7 @@ declare const serviceGroupViewSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -990,6 +1011,7 @@ declare const serviceGroupViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -1012,6 +1034,7 @@ declare const serviceGroupViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -1176,6 +1199,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -1201,6 +1225,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -1223,6 +1248,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -1338,6 +1364,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -1363,6 +1390,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -1385,6 +1413,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -2412,4 +2441,4 @@ declare const ingestAuditEventSchema: z.ZodObject<{
}, z.core.$strip>;
type IngestAuditEvent = z.infer<typeof ingestAuditEventSchema>;
export { AUDIT_SEVERITIES, AUDIT_SOURCE_APPS, AUDIT_TARGET_TYPES, type AppSettingsPatch, type AppSwitcherConfig, type AppSwitcherEntry, type AuditListQuery, type AuditLogEntry, type AuditSeverity, type AuditSourceApp, type AuditTargetType, type BulkUpdateDomainsInput, CERT_ERROR, CERT_EXPIRED, CERT_MONITORING_VALUES, CERT_MONITOR_AUTO, CERT_MONITOR_REQUIRED, CERT_MONITOR_SKIPPED, CERT_OK, CERT_UNKNOWN, CERT_WARNING, type CertMonitoring, type Certificate, type CfDnsRecord, type CfHealthCheck, type CfZone, type CfdmBindingSyncItem, type ChangeDomainInput, type ChangeIpInput, type CreateDnsRecordInput, type CreateDnsRecordPayload, type CreateDomainInput, type CreateDomainMonitorInput, type CreateGroupInput, type CreateOriginHealthCheckInput, type CreateServiceBindingInput, type CreateServiceGroupInput, type CreateServiceInput, type CreateServiceNodeInput, type CreateServiceWithConfigInput, type CreateSubdomainInput, type DnsRecord, type Domain, type DomainEnvironment, type DomainListItem, type DomainMonitor, type DomainMonitorResult, type DomainMonitorType, type Group, type GroupWithStats, HEALTH_CHECK_AGGREGATES, HEALTH_CHECK_PROVIDERS, HEALTH_KV_CURSOR_KEY, HEALTH_KV_RESULTS_KEY, HEALTH_KV_TARGETS_KEY, HEALTH_PROBE_BATCH, HEALTH_PROBE_CONCURRENCY, HEALTH_PROBE_KV_TITLE, HEALTH_PROBE_SCRIPT_NAME, HEALTH_STATUS_PROVIDERS, type HealthCheckAggregate, type HealthCheckConfig, type HealthCheckProvider, type HealthCheckScope, type HealthCheckTarget, type HealthCheckType, type HealthProbeLog, type HealthProbeResultItem, type HealthProbeResultsDoc, type HealthProbeTargetItem, type HealthProbeTargetsDoc, type HealthStatusProvider, type HealthStatusQuery, type HealthWorkerStatus, type IngestAuditEvent, type IpHealthState, type IpHealthStatus, type JwtClaims, type LbMode, type LoginInput, type LoginRequest, type LoginResponse, type NodeHealthState, type NotificationLog, type OriginHealthCheck, type OriginHealthCheckRecord, type ParsedFqdn, type PatchDnsRecordPayload, type ReorderServicesInput, SYNC_CONFLICT, SYNC_ERROR, SYNC_PENDING_DELETE, SYNC_PENDING_PUSH, SYNC_SYNCED, type Service, type ServiceBinding, type ServiceBindingView, type ServiceCertificateRow, type ServiceDomainBinding, type ServiceDomainBindingView, type ServiceGroup, type ServiceGroupView, type ServiceGroupsResponse, type ServiceIpHealth, type ServiceNode, type ServiceNodeRecord, type ServiceOverview, type ServiceView, type Subdomain, type SubdomainRecord, type SyncJob, type ToggleEnabledInput, type ToggleServiceIpInput, type UpdateDomainInput, type UpdateServiceConfigInput, type UpdateServiceGroupInput, type UpdateServiceNodeInput, type UpdateSubdomainInput, ValidationError, type VpsTrackerEvent, aggregateHealthOk, appSettingsPatchSchema, appSwitcherConfigSchema, appSwitcherEntrySchema, appSwitcherIconSchema, auditListQuerySchema, auditLogEntrySchema, auditSeveritySchema, auditSourceAppSchema, auditTargetTypeSchema, bindingToFqdn, bulkUpdateDomainsSchema, certMonitoringSchema, certStatusFromExpiry, certificateSchema, cfdmBindingSyncItemSchema, cfdmSyncBindingsBodySchema, changeDomainSchema, changeIpSchema, clampGlobalpingLimit, createDnsRecordSchema, createDomainMonitorSchema, createDomainSchema, createGroupSchema, createOriginHealthCheckSchema, createServiceBindingSchema, createServiceGroupSchema, createServiceNodeSchema, createServiceSchema, createServiceWithConfigSchema, createSubdomainSchema, derivePrimaryProvider, dnsNameToSubdomainLabel, dnsRecordNamesMatch, dnsRecordSchema, domainEnvironmentSchema, domainListItemSchema, domainMonitorResultSchema, domainMonitorSchema, domainMonitorTypeSchema, domainSchema, fqdnToDisplay, groupSchema, groupWithStatsSchema, healthCheckAggregateSchema, healthCheckConfigSchema, healthCheckProviderSchema, healthCheckProvidersSchema, healthCheckScopeSchema, healthCheckTypeSchema, healthProbeLogSchema, healthStatusProviderSchema, healthStatusQuerySchema, ingestAuditEventSchema, ipHealthStateSchema, ipHealthStatusSchema, isIpLiteral, isValidIpv4, lbModeSchema, loginSchema, nodeHealthStateSchema, normalizeDnsRecordName, normalizeProbeProvider, normalizeStatusProvider, notificationLogSchema, originHealthCheckSchema, parseFqdn, parseGlobalpingLocations, parseHealthAggregate, parseHealthProviders, reorderServicesSchema, serializeHealthProviders, serviceBindingSchema, serviceCertificateRowSchema, serviceDomainBindingSchema, serviceGroupSchema, serviceGroupTypeSchema, serviceGroupViewSchema, serviceGroupsResponseSchema, serviceIpHealthSchema, serviceNodeSchema, serviceSchema, serviceViewSchema, shouldMonitorService, subdomainLabelToFqdn, subdomainSchema, targetHasProvider, targetProviders, toggleEnabledSchema, toggleServiceIpSchema, uniqueHealthProviders, updateDomainGroupSchema, updateDomainSchema, updateServiceConfigSchema, updateServiceGroupSchema, updateServiceNodeSchema, updateSubdomainSchema, validateDnsRecord, vpsTrackerEventSchema };
export { AUDIT_SEVERITIES, AUDIT_SOURCE_APPS, AUDIT_TARGET_TYPES, type AppSettingsPatch, type AppSwitcherConfig, type AppSwitcherEntry, type AuditListQuery, type AuditLogEntry, type AuditSeverity, type AuditSourceApp, type AuditTargetType, type BulkUpdateDomainsInput, CERT_ERROR, CERT_EXPIRED, CERT_MONITORING_VALUES, CERT_MONITOR_AUTO, CERT_MONITOR_REQUIRED, CERT_MONITOR_SKIPPED, CERT_OK, CERT_UNKNOWN, CERT_WARNING, type CertMonitoring, type Certificate, type CfDnsRecord, type CfHealthCheck, type CfZone, type CfdmBindingSyncItem, type ChangeDomainInput, type ChangeIpInput, type CreateDnsRecordInput, type CreateDnsRecordPayload, type CreateDomainInput, type CreateDomainMonitorInput, type CreateGroupInput, type CreateOriginHealthCheckInput, type CreateServiceBindingInput, type CreateServiceGroupInput, type CreateServiceInput, type CreateServiceNodeInput, type CreateServiceWithConfigInput, type CreateSubdomainInput, type DnsRecord, type Domain, type DomainEnvironment, type DomainListItem, type DomainMonitor, type DomainMonitorResult, type DomainMonitorType, type FailoverLogEntry, type Group, type GroupWithStats, HEALTH_CHECK_AGGREGATES, HEALTH_CHECK_PROVIDERS, HEALTH_KV_CURSOR_KEY, HEALTH_KV_RESULTS_KEY, HEALTH_KV_TARGETS_KEY, HEALTH_PROBE_BATCH, HEALTH_PROBE_CONCURRENCY, HEALTH_PROBE_KV_TITLE, HEALTH_PROBE_SCRIPT_NAME, HEALTH_STATUS_PROVIDERS, type HealthCheckAggregate, type HealthCheckConfig, type HealthCheckProvider, type HealthCheckScope, type HealthCheckTarget, type HealthCheckType, type HealthProbeLog, type HealthProbeResultItem, type HealthProbeResultsDoc, type HealthProbeTargetItem, type HealthProbeTargetsDoc, type HealthStatusProvider, type HealthStatusQuery, type HealthWorkerStatus, type IngestAuditEvent, type IpHealthState, type IpHealthStatus, type JwtClaims, type LbMode, type LoginInput, type LoginRequest, type LoginResponse, type NodeHealthState, type NotificationLog, type OriginHealthCheck, type OriginHealthCheckRecord, type ParsedFqdn, type PatchDnsRecordPayload, type ReorderServicesInput, SYNC_CONFLICT, SYNC_ERROR, SYNC_PENDING_DELETE, SYNC_PENDING_PUSH, SYNC_SYNCED, type Service, type ServiceBinding, type ServiceBindingView, type ServiceCertificateRow, type ServiceDomainBinding, type ServiceDomainBindingView, type ServiceGroup, type ServiceGroupView, type ServiceGroupsResponse, type ServiceIpHealth, type ServiceNode, type ServiceNodeRecord, type ServiceOverview, type ServiceView, type Subdomain, type SubdomainRecord, type SyncJob, type ToggleEnabledInput, type ToggleServiceIpInput, type UpdateDomainInput, type UpdateServiceConfigInput, type UpdateServiceGroupInput, type UpdateServiceNodeInput, type UpdateSubdomainInput, ValidationError, type VpsTrackerEvent, aggregateHealthOk, appSettingsPatchSchema, appSwitcherConfigSchema, appSwitcherEntrySchema, appSwitcherIconSchema, auditListQuerySchema, auditLogEntrySchema, auditSeveritySchema, auditSourceAppSchema, auditTargetTypeSchema, bindingToFqdn, bulkUpdateDomainsSchema, certMonitoringSchema, certStatusFromExpiry, certificateSchema, cfdmBindingSyncItemSchema, cfdmSyncBindingsBodySchema, changeDomainSchema, changeIpSchema, clampGlobalpingLimit, createDnsRecordSchema, createDomainMonitorSchema, createDomainSchema, createGroupSchema, createOriginHealthCheckSchema, createServiceBindingSchema, createServiceGroupSchema, createServiceNodeSchema, createServiceSchema, createServiceWithConfigSchema, createSubdomainSchema, derivePrimaryProvider, dnsNameToSubdomainLabel, dnsRecordNamesMatch, dnsRecordSchema, domainEnvironmentSchema, domainListItemSchema, domainMonitorResultSchema, domainMonitorSchema, domainMonitorTypeSchema, domainSchema, failoverLogSchema, fqdnToDisplay, groupSchema, groupWithStatsSchema, healthCheckAggregateSchema, healthCheckConfigSchema, healthCheckProviderSchema, healthCheckProvidersSchema, healthCheckScopeSchema, healthCheckTypeSchema, healthProbeLogSchema, healthStatusProviderSchema, healthStatusQuerySchema, ingestAuditEventSchema, ipHealthStateSchema, ipHealthStatusSchema, isIpLiteral, isValidIpv4, lbModeSchema, loginSchema, nodeHealthStateSchema, normalizeDnsRecordName, normalizeProbeProvider, normalizeStatusProvider, notificationLogSchema, originHealthCheckSchema, parseFqdn, parseGlobalpingLocations, parseHealthAggregate, parseHealthProviders, reorderServicesSchema, serializeHealthProviders, serviceBindingSchema, serviceCertificateRowSchema, serviceDomainBindingSchema, serviceGroupSchema, serviceGroupTypeSchema, serviceGroupViewSchema, serviceGroupsResponseSchema, serviceIpHealthSchema, serviceNodeSchema, serviceSchema, serviceViewSchema, shouldMonitorService, subdomainLabelToFqdn, subdomainSchema, targetHasProvider, targetProviders, toggleEnabledSchema, toggleServiceIpSchema, uniqueHealthProviders, updateDomainGroupSchema, updateDomainSchema, updateServiceConfigSchema, updateServiceGroupSchema, updateServiceNodeSchema, updateSubdomainSchema, validateDnsRecord, vpsTrackerEventSchema };
+12 -1
View File
@@ -322,6 +322,15 @@ var healthProbeLogSchema = z.object({
error: z.string().nullable(),
checked_at: z.string()
});
var failoverLogSchema = z.object({
id: z.number(),
service_id: z.number(),
binding_id: z.number(),
fqdn: z.string(),
ip: z.string(),
action: z.enum(["added", "removed"]),
created_at: z.string()
});
var groupSchema = z.object({
id: z.number(),
name: z.string(),
@@ -399,7 +408,8 @@ var serviceDomainBindingSchema = z.object({
health_check_providers: healthCheckProvidersSchema.catch(["local"]),
health_check_aggregate: healthCheckAggregateSchema.catch("majority"),
cert_monitoring: certMonitoringSchema.default("auto"),
sync_status: z.string().nullable().default(null)
sync_status: z.string().nullable().default(null),
active_ips: z.array(z.string()).default([])
}).transform((binding) => ({
...binding,
target_ips: binding.target_ips && binding.target_ips.length > 0 ? binding.target_ips : binding.target_ip ? [binding.target_ip] : [],
@@ -1019,6 +1029,7 @@ export {
domainMonitorSchema,
domainMonitorTypeSchema,
domainSchema,
failoverLogSchema,
fqdnToDisplay,
groupSchema,
groupWithStatsSchema,
+13
View File
@@ -100,6 +100,18 @@ export const healthProbeLogSchema = z.object({
export type HealthProbeLog = z.infer<typeof healthProbeLogSchema>
export const failoverLogSchema = z.object({
id: z.number(),
service_id: z.number(),
binding_id: z.number(),
fqdn: z.string(),
ip: z.string(),
action: z.enum(['added', 'removed']),
created_at: z.string(),
})
export type FailoverLogEntry = z.infer<typeof failoverLogSchema>
export const groupSchema = z.object({
id: z.number(),
name: z.string(),
@@ -184,6 +196,7 @@ export const serviceDomainBindingSchema = z
health_check_aggregate: healthCheckAggregateSchema.catch('majority'),
cert_monitoring: certMonitoringSchema.default('auto'),
sync_status: z.string().nullable().default(null),
active_ips: z.array(z.string()).default([]),
})
.transform((binding) => ({
...binding,
+1
View File
@@ -190,6 +190,7 @@ export interface ServiceDomainBindingView {
fqdn: string;
record_type: "A" | "CNAME";
target_ips: string[];
active_ips: string[];
target_ip_weights: Record<string, number>;
target_ip_priorities: Record<string, number>;
target_cname: string | null;