Compare commits

..
19 Commits
Author SHA1 Message Date
DenozordecandCursor b078fa0a03 fix(health): не показывать Down при выключенном health-check
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 4s
quality / changes (push) Successful in 8s
quality / docker-check (push) Skipped
quality / web (push) Successful in 54s
quality / api (push) Successful in 49s
CD / quality (push) Successful in 1m58s
CD / publish (push) Successful in 1m25s
Маскируем устаревший down, если HC выкл (binding/группа), IP или сервис отключены.
Бейдж Disabled — нейтральный, не красный.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 20:46:37 +07:00
DenozordecandCursor 75575a3243 fix(dns): искать записи в Cloudflare по имени и содержимому
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / changes (push) Successful in 7s
quality / web (push) Skipped
quality / docker-check (push) Skipped
quality / api (push) Successful in 48s
CD / quality (push) Successful in 58s
CD / publish (push) Successful in 1m35s
cf_record_id только как кэш-подсказка: при удалении/обновлении
сначала list + match type/name/content, иначе create или no-op.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 20:19:16 +07:00
DenozordecandCursor 5e2c301442 fix(dns): выравнивать доп. FQDN при конфликте с локальной/CF записью
quality / changes (push) Successful in 9s
quality / web (push) Skipped
quality / commitlint (push) Skipped
quality / docker-check (push) Skipped
CD / update-wiki (push) Successful in 4s
quality / api (push) Successful in 48s
CD / quality (push) Successful in 1m2s
CD / publish (push) Successful in 1m26s
Больше не авто-подхватывать чужой CNAME в A-режиме (это стирало IP). Перед публикацией A удаляем конфликтующий CNAME и устаревшие A на том же имени.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 20:05:35 +07:00
DenozordecandCursor 56cddedefe fix(services): восстанавливать урезанные общие FQDN без пересоздания сервиса
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 8s
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m16s
quality / api (push) Successful in 58s
CD / quality (push) Successful in 2m26s
CD / publish (push) Successful in 1m40s
Legacy toggle оставлял multi-IP привязки с неполным пулом в невидимом preserved — UI терял домены и падал на дубликатах. Чиним hydrate и чиним БД при buildView.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-03 19:26:46 +07:00
Denozordec df5d5ef4ab feat(dns): add functions to handle missing DNS records and mark records as synced
CD / quality (push) Successful in 1m16s
quality / changes (push) Successful in 9s
quality / web (push) Skipped
quality / api (push) Successful in 1m3s
quality / commitlint (push) Skipped
quality / docker-check (push) Skipped
CD / update-wiki (push) Successful in 5s
CD / publish (push) Successful in 1m24s
- Introduced `isMissingCfDnsRecord` to identify missing Cloudflare DNS records based on error messages.
- Added `markSynced` function to update DNS record fields in the database and return the updated record.
- Refactored `pushRecord` to utilize `markSynced` for better code organization and clarity.
- Enhanced error handling for cases where DNS records need to be recreated after manual edits.
2026-09-03 18:52:39 +07:00
DenozordecandCursor de3dbe8521 fix(sync): не считать сервис с одним IP резервированием
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / changes (push) Successful in 9s
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m20s
quality / api (push) Successful in 1m4s
CD / quality (push) Successful in 2m38s
CD / publish (push) Successful in 56s
В payload для VPS Tracker lbMode не отдаём без пула уникальных IP; в UI показываем без резервирования вместо Round robin.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 15:09:19 +07:00
DenozordecandCursor 7eb195c5d7 feat(sync): передавать lbMode в payload для VPS Tracker
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 4s
quality / changes (push) Successful in 8s
quality / docker-check (push) Skipped
quality / web (push) Successful in 53s
quality / api (push) Successful in 49s
CD / quality (push) Successful in 1m54s
CD / publish (push) Successful in 1m29s
Тип HA уходит в bindings, чтобы схема могла показать резервирование.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-09-01 11:30:41 +07:00
DenozordecandCursor 7f06466058 fix(dns): разрешить вложенный wildcard в имени DNS-записи
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 8s
quality / changes (push) Successful in 12s
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m9s
quality / api (push) Successful in 1m3s
CD / quality (push) Successful in 2m29s
CD / publish (push) Successful in 1m49s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-31 16:02:20 +07:00
DenozordecandCursor 0d567379fa fix(services): разрешить несколько доп. FQDN на IP включая wildcard
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 10s
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m16s
quality / api (push) Successful in 1m14s
CD / quality (push) Successful in 2m44s
CD / publish (push) Successful in 1m50s
У IP был один extra FQDN; wildcard вида *.mdns.shnt.top не матчился с именем из Cloudflare.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-31 12:40:27 +07:00
DenozordecandCursor a228febc27 fix(services): сохранить доп. FQDN у IP при пуле из одного адреса
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 9s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m10s
CD / quality (push) Successful in 1m23s
CD / publish (push) Successful in 1m41s
При одном IP общий и доп. FQDN неотличимы по target_ips; после сохранения гидратация относила оба к общим доменам.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-31 12:19:53 +07:00
DenozordecandCursor 5cf39880f8 fix(services): сразу возвращать IP в DNS после первой успешной пробы
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 8s
quality / changes (push) Successful in 8s
quality / web (push) Skipped
quality / docker-check (push) Skipped
quality / api (push) Successful in 1m20s
CD / quality (push) Successful in 1m31s
CD / publish (push) Successful in 2m4s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 19:51:23 +07:00
DenozordecandCursor f1443f1db5 fix(services): не балансировать сервис с одним IP
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / changes (push) Successful in 8s
quality / docker-check (push) Skipped
quality / web (push) Successful in 55s
quality / api (push) Successful in 52s
CD / quality (push) Successful in 2m4s
CD / publish (push) Successful in 1m51s
KPI и карточка показывают живой OK, а не гистерезис unknown; DNS по-прежнему ждёт повторные успехи.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 18:13:06 +07:00
DenozordecandCursor a9a84fabac fix(services): передать режим балансировки в панель пула
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 7s
quality / changes (push) Successful in 10s
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m23s
quality / api (push) Successful in 59s
CD / quality (push) Successful in 2m36s
CD / publish (push) Successful in 1m58s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:52:21 +07:00
DenozordecandCursor 2b150fa3a6 fix(services): снимать Down только с общего FQDN
Персональные A не крутить и не трогать при падении. Панель называется по режиму балансировки.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:51:49 +07:00
DenozordecandCursor 3d0ea33baf feat(services): крутить weighted как доли времени на одном IP
quality / changes (push) Successful in 9s
quality / commitlint (push) Skipped
quality / docker-check (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / web (push) Successful in 55s
quality / api (push) Successful in 45s
CD / quality (push) Successful in 1m57s
CD / publish (push) Successful in 1m36s
Веса задают долю слотов на общем FQDN (1 к 3 = ¼ и ¾ цикла), TTL 60.
В селекте режима показывать текстовое имя, не ID.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:35:13 +07:00
DenozordecandCursor 8a13888db7 fix(services): вернуть вес и приоритет в режиме балансировки
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 8s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 54s
CD / quality (push) Successful in 1m5s
CD / publish (push) Successful in 1m43s
После объединения адресов в форме пропали per-IP поля. Weighted снова задаёт вес, failover — основной и запасной.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 17:06:29 +07:00
DenozordecandCursor f5d97e463a fix(services): показывать живой health и историю выхода из пула
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 6s
quality / changes (push) Successful in 8s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 1m2s
CD / quality (push) Successful in 1m14s
CD / publish (push) Successful in 2m3s
Таблица IP брала hysteresis unknown вместо последней пробы. Failover теперь показывает выход и возврат в пул, а не только DNS add.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 16:43:22 +07:00
DenozordecandCursor be8f94143f fix(health): не затирать график uptime пробой down в том же бакете
quality / commitlint (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / changes (push) Successful in 8s
quality / api (push) Skipped
quality / docker-check (push) Skipped
quality / web (push) Successful in 55s
CD / quality (push) Successful in 1m7s
CD / publish (push) Successful in 1m38s
Если в минутном бакете есть живой IP, линия строится по его пингу, а не по null от Down.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 16:07:24 +07:00
DenozordecandCursor fa7d2b6df3 feat(services): считать сервис живым при одном OK и писать историю failover
quality / commitlint (push) Skipped
quality / changes (push) Successful in 11s
quality / docker-check (push) Skipped
CD / update-wiki (push) Successful in 5s
quality / web (push) Successful in 1m11s
quality / api (push) Successful in 1m1s
CD / quality (push) Successful in 2m30s
CD / publish (push) Successful in 2m4s
Сервис up, если жив хотя бы один IP. Failover показывает Down по FQDN и журнал add/remove A-записей.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-08-20 15:49:17 +07:00
57 changed files with 4377 additions and 451 deletions
+5
View File
@@ -38,6 +38,10 @@ import {
healthEngineFallbacksFromConfig,
scheduleHealthCheckJob,
} from "./services/health-check-scheduler.js";
import {
createWeightedDnsTask,
scheduleWeightedDnsJob,
} from "./services/weighted-dns-scheduler.js";
import { fireEnsureHealthWorker } from "./services/health/health-worker-deploy.js";
import { AsyncTask, CronJob } from "toad-scheduler";
@@ -133,6 +137,7 @@ export async function buildApp(opts: BuildAppOptions = {}) {
app.decorate("reloadHealthCheckJob", () => {
scheduleHealthCheckJob(app, config, healthTask);
});
scheduleWeightedDnsJob(app, createWeightedDnsTask(app));
if (config.cloudflareApiToken) {
fireEnsureHealthWorker(
app.db,
+12
View File
@@ -78,6 +78,18 @@ export async function serviceRoutes(app: FastifyInstance) {
};
});
app.get("/services/:id/failover-log", async (request) => {
const { id } = request.params as { id: string };
repos.getService(request.server.db, Number(id));
return {
items: repos.listFailoverLogForService(
request.server.db,
Number(id),
200,
),
};
});
app.get("/services/:id/certificates", async (request) => {
const { id } = request.params as { id: string };
return certificateService.listServiceCertificates(
+189 -54
View File
@@ -1,11 +1,17 @@
import type { Db } from "@cfdm/db";
import { repos, type DnsListFilter } from "@cfdm/db";
import type { CreateDnsRecordPayload, DnsRecord, PatchDnsRecordPayload } from "@cfdm/shared";
import type {
CfDnsRecord,
CreateDnsRecordPayload,
DnsRecord,
PatchDnsRecordPayload,
} from "@cfdm/shared";
import {
SYNC_CONFLICT,
SYNC_ERROR,
SYNC_PENDING_PUSH,
SYNC_SYNCED,
dnsRecordNamesMatch,
normalizeDnsRecordName,
} from "@cfdm/shared";
import type { CloudflareClient } from "../lib/cf-client.js";
@@ -64,6 +70,86 @@ function toCfPayload(
};
}
function isMissingCfDnsRecord(error: unknown): boolean {
const message = error instanceof Error ? error.message : String(error);
return /record does not exist|81044/i.test(message);
}
function dnsContentMatches(
recordType: string,
left: string,
right: string,
): boolean {
if (recordType.toUpperCase() === "CNAME") {
return (
left.trim().replace(/\.+$/, "").toLowerCase() ===
right.trim().replace(/\.+$/, "").toLowerCase()
);
}
return left === right;
}
/** Resolve live Cloudflare record by name + type + content (IP / CNAME target). */
function findRemoteByIdentity(
remote: readonly CfDnsRecord[],
zoneName: string,
recordType: string,
name: string,
content: string,
): CfDnsRecord | undefined {
const type = recordType.toUpperCase();
return remote.find(
(record) =>
Boolean(record.id) &&
(record.type ?? "").toUpperCase() === type &&
dnsRecordNamesMatch(record.name, name, zoneName) &&
dnsContentMatches(type, record.content, content),
);
}
function findRemoteByCfId(
remote: readonly CfDnsRecord[],
cfRecordId: string | null | undefined,
): CfDnsRecord | undefined {
if (!cfRecordId) return undefined;
return remote.find((record) => record.id === cfRecordId);
}
async function markSynced(
db: Db,
domainId: number,
record: DnsRecord,
cfRec: {
id?: string | null;
type?: string;
name: string;
content: string;
ttl: number;
proxied?: boolean | null;
priority?: number | null;
},
): Promise<DnsRecord> {
repos.updateDnsFields(
db,
record.id,
cfRec.type ?? record.record_type,
cfRec.name,
cfRec.content,
cfRec.ttl,
cfRec.proxied ?? false,
cfRec.priority ?? null,
SYNC_SYNCED,
cfRec.id ?? null,
null,
);
return repos.getDnsRecord(db, domainId, record.id);
}
/**
* Push local desired state to Cloudflare.
* Identity is name + type + content; cf_record_id is only a cache hint
* (records may be deleted/recreated outside CFDM).
*/
async function pushRecord(
db: Db,
cf: CloudflareClient,
@@ -71,6 +157,7 @@ async function pushRecord(
cfZoneId: string,
record: DnsRecord,
): Promise<DnsRecord> {
const domain = repos.getDomain(db, domainId);
const payload = toCfPayload(
record.record_type,
record.name,
@@ -81,25 +168,38 @@ async function pushRecord(
);
try {
const cfRec = record.cf_record_id
? await cf.updateDnsRecord(cfZoneId, record.cf_record_id, payload)
: await cf.createDnsRecord(cfZoneId, payload);
repos.updateDnsFields(
db,
record.id,
cfRec.type ?? record.record_type,
cfRec.name,
cfRec.content,
cfRec.ttl,
cfRec.proxied ?? false,
cfRec.priority ?? null,
SYNC_SYNCED,
cfRec.id ?? null,
null,
const remote = await cf.listDnsRecords(cfZoneId);
const byIdentity = findRemoteByIdentity(
remote,
domain.zone_name,
record.record_type,
record.name,
record.content,
);
return repos.getDnsRecord(db, domainId, record.id);
const byCachedId = findRemoteByCfId(remote, record.cf_record_id);
const targetId = byIdentity?.id ?? byCachedId?.id ?? null;
const cfRec = targetId
? await cf.updateDnsRecord(cfZoneId, targetId, payload)
: await cf.createDnsRecord(cfZoneId, payload);
return markSynced(db, domainId, record, cfRec);
} catch (e) {
// Race: cached id vanished mid-flight — recreate by identity.
if (isMissingCfDnsRecord(e)) {
try {
const created = await cf.createDnsRecord(cfZoneId, payload);
return markSynced(db, domainId, record, created);
} catch (createErr) {
repos.setDnsSyncStatus(
db,
record.id,
SYNC_ERROR,
null,
createErr instanceof Error ? createErr.message : String(createErr),
);
throw createErr;
}
}
repos.setDnsSyncStatus(
db,
record.id,
@@ -188,15 +288,23 @@ export async function patchContent(
): Promise<DnsRecord> {
const domain = repos.getDomain(db, domainId);
const existing = repos.getDnsRecord(db, domainId, recordId);
if (!existing.cf_record_id) {
throw AppError.dnsUpdateFailed("у DNS-записи нет идентификатора Cloudflare");
}
try {
const cfRec = await cf.patchDnsRecord(
domain.cf_zone_id,
existing.cf_record_id,
payload,
const remote = await cf.listDnsRecords(domain.cf_zone_id);
const byIdentity = findRemoteByIdentity(
remote,
domain.zone_name,
existing.record_type,
existing.name,
existing.content,
);
const byCachedId = findRemoteByCfId(remote, existing.cf_record_id);
const targetId = byIdentity?.id ?? byCachedId?.id ?? null;
if (!targetId) {
throw AppError.dnsUpdateFailed(
"DNS-запись не найдена в Cloudflare по имени и содержимому",
);
}
const cfRec = await cf.patchDnsRecord(domain.cf_zone_id, targetId, payload);
repos.updateDnsFields(
db,
existing.id,
@@ -207,7 +315,7 @@ export async function patchContent(
cfRec.proxied ?? existing.proxied,
cfRec.priority ?? existing.priority,
SYNC_SYNCED,
cfRec.id ?? existing.cf_record_id,
cfRec.id ?? targetId,
null,
);
return repos.getDnsRecord(db, domainId, existing.id);
@@ -235,20 +343,41 @@ export async function deleteRecord(
const record = repos.getDnsRecord(db, domainId, recordId);
repos.markDnsPendingDelete(db, recordId);
if (record.cf_record_id) {
let targetId: string | null = null;
try {
const remote = await cf.listDnsRecords(domain.cf_zone_id);
const byIdentity = findRemoteByIdentity(
remote,
domain.zone_name,
record.record_type,
record.name,
record.content,
);
const byCachedId = findRemoteByCfId(remote, record.cf_record_id);
targetId = byIdentity?.id ?? byCachedId?.id ?? null;
} catch {
// Zone list failed — fall back to cached id only.
targetId = record.cf_record_id;
}
if (targetId) {
try {
await cf.deleteDnsRecord(domain.cf_zone_id, record.cf_record_id);
await cf.deleteDnsRecord(domain.cf_zone_id, targetId);
} catch (e) {
repos.setDnsSyncStatus(
db,
recordId,
SYNC_ERROR,
record.cf_record_id,
e instanceof Error ? e.message : String(e),
);
throw e;
// Already gone in Cloudflare (manual delete) — drop local row.
if (!isMissingCfDnsRecord(e)) {
repos.setDnsSyncStatus(
db,
recordId,
SYNC_ERROR,
targetId,
e instanceof Error ? e.message : String(e),
);
throw e;
}
}
}
repos.deleteDnsRecord(db, recordId);
}
@@ -326,24 +455,30 @@ export async function resolveConflict(
}
if (req.source === "cloudflare") {
if (record.cf_record_id) {
const remote = await cf.listDnsRecords(domain.cf_zone_id);
const r = remote.find((x) => x.id === record.cf_record_id);
if (r) {
repos.updateDnsFields(
db,
recordId,
r.type,
r.name,
r.content,
r.ttl,
r.proxied ?? false,
r.priority ?? null,
SYNC_SYNCED,
r.id ?? null,
null,
);
}
const remote = await cf.listDnsRecords(domain.cf_zone_id);
const byIdentity = findRemoteByIdentity(
remote,
domain.zone_name,
record.record_type,
record.name,
record.content,
);
const byCachedId = findRemoteByCfId(remote, record.cf_record_id);
const r = byIdentity ?? byCachedId;
if (r) {
repos.updateDnsFields(
db,
recordId,
r.type,
r.name,
r.content,
r.ttl,
r.proxied ?? false,
r.priority ?? null,
SYNC_SYNCED,
r.id ?? null,
null,
);
}
return repos.getDnsRecord(db, domainId, recordId);
}
@@ -290,7 +290,7 @@ export interface RunAllChecksOptions {
target: HealthCheckTarget,
prevState: IpHealthState | null,
nextState: IpHealthState,
) => void;
) => void | Promise<void>;
}
function sleep(ms: number): Promise<void> {
@@ -326,7 +326,7 @@ function logSourceResult(
});
}
function applyAggregatedStatus(
async function applyAggregatedStatus(
db: Db,
target: HealthCheckTarget,
sources: Array<{ provider: HealthCheckProvider; result: ProbeResult }>,
@@ -397,7 +397,7 @@ function applyAggregatedStatus(
});
}
if (prevState !== state) {
options.onStatusChange?.(target, prevState, state);
await options.onStatusChange?.(target, prevState, state);
}
}
@@ -513,7 +513,7 @@ export async function runAllChecks(
for (const source of sources) {
logSourceResult(db, target, source.provider, source.result);
}
applyAggregatedStatus(db, target, sources, options);
await applyAggregatedStatus(db, target, sources, options);
}
}
+19 -6
View File
@@ -9,7 +9,10 @@ import type {
import { AppError } from "../errors.js";
import { isValidIpv4 } from "../lib/validators.js";
import { getView } from "./service-config-service.js";
import { selectActiveIpsByMode } from "./routing/index.js";
import {
isSharedPool,
resolveDesiredAIps,
} from "./routing/index.js";
function assertAddress(address: string): void {
if (!isValidIpv4(address)) {
@@ -89,8 +92,12 @@ export async function getOverview(
: null;
const active = new Set<string>();
const serviceIps = service.ips.filter(
(ip) => service.ip_enabled[ip] !== false,
);
for (const binding of bindings) {
const metas = repos.listBindingIpsWithMeta(db, binding.id);
const targetIps = metas.map((entry) => entry.ip);
const rows = metas.map((entry) => {
const status = repos.getIpHealthStatusRow(db, "binding", binding.id, entry.ip);
return {
@@ -100,12 +107,15 @@ export async function getOverview(
health: status ? status.status : ("unknown" as const),
};
});
for (const ip of selectActiveIpsByMode(
for (const ip of resolveDesiredAIps(
{
lb_mode: binding.lb_mode,
health_check_enabled: binding.health_check_enabled,
},
rows,
targetIps,
Date.now(),
serviceIps,
)) {
active.add(ip);
}
@@ -134,10 +144,13 @@ export function opsSummary(db: Db) {
if (binding.lb_mode !== "failover" || !binding.health_check_enabled) {
return false;
}
return binding.target_ips.some((ip) => {
const row = repos.getIpHealthStatusRow(db, "binding", binding.id, ip);
return row?.status === "down";
});
return (
isSharedPool(binding.target_ips) &&
binding.target_ips.some((ip) => {
const row = repos.getIpHealthStatusRow(db, "binding", binding.id, ip);
return row?.status === "down";
})
);
}).length;
return {
domains: domains.length,
+4 -4
View File
@@ -1,16 +1,16 @@
import type { LbIpRow } from "./types.js";
import { isHealthy } from "./health.js";
import { isPoolMember } from "./health.js";
export function failoverDesired(rows: LbIpRow[]): string[] {
if (rows.length === 0) return [];
const healthy = rows.filter((r) => isHealthy(r.health));
const pool = healthy.length > 0 ? healthy : rows;
const live = rows.filter((r) => isPoolMember(r.health));
const pool = live.length > 0 ? live : rows;
const sorted = [...pool].sort(
(a, b) => a.priority - b.priority || a.weight - b.weight,
);
const minPriority = sorted[0]!.priority;
const primaries = sorted.filter((r) => r.priority === minPriority);
if (healthy.length > 0) {
if (live.length > 0) {
return primaries.map((r) => r.ip);
}
return [sorted[0]!.ip];
+9
View File
@@ -3,3 +3,12 @@ import type { IpHealthState, NodeHealthState } from "@cfdm/shared";
export function isHealthy(state: IpHealthState | NodeHealthState | string): boolean {
return state === "up" || state === "healthy";
}
export function isDown(state: IpHealthState | NodeHealthState | string): boolean {
return state === "down" || state === "unhealthy";
}
/** A-pool membership: only Down is drained. Recovering (unknown/checking) and Slow return immediately. */
export function isPoolMember(state: IpHealthState | NodeHealthState | string): boolean {
return !isDown(state);
}
+32 -3
View File
@@ -1,26 +1,55 @@
import type { LbMode } from "@cfdm/shared";
import { failoverDesired } from "./failover.js";
import { canApplyLb, isSharedPool } from "./pool.js";
import { roundRobinDesired } from "./round-robin.js";
import type { LbIpRow, LbTargetConfig } from "./types.js";
import { weightedDesired } from "./weighted.js";
export type { LbIpRow, LbTargetConfig } from "./types.js";
export { isHealthy } from "./health.js";
export { isDown, isHealthy, isPoolMember } from "./health.js";
export { withBindingLock } from "./binding-lock.js";
export {
canApplyLb,
isSharedPool,
shouldRecordFailoverDnsDiff,
uniqueIpCount,
} from "./pool.js";
export { WEIGHTED_DNS_TTL, WEIGHTED_SLOT_MS, weightedDesired } from "./weighted.js";
export function selectActiveIpsByMode(
config: LbTargetConfig,
rows: LbIpRow[],
nowMs = Date.now(),
): string[] {
if (rows.length === 0) return [];
if (config.lb_mode === "failover") {
return failoverDesired(rows);
}
// weighted = round_robin on DNS (one A per IP)
if (config.lb_mode === "weighted") {
return weightedDesired(rows, nowMs);
}
return roundRobinDesired(rows);
}
export function resolveDesiredAIps(
config: LbTargetConfig,
rows: LbIpRow[],
fallbackIps: readonly string[],
nowMs = Date.now(),
serviceIps: readonly string[] = fallbackIps,
): string[] {
const fallback = [...fallbackIps];
if (!canApplyLb(serviceIps, fallback)) return fallback;
if (!isSharedPool(rows.map((row) => row.ip))) return fallback;
if (config.lb_mode === "weighted" || config.health_check_enabled) {
const activeIps = selectActiveIpsByMode(config, rows, nowMs);
if (activeIps.length > 0) return activeIps;
}
return fallback;
}
export function strategyLabel(mode: LbMode): string {
if (mode === "failover") return "Failover";
if (mode === "weighted") return "Round Robin (weighted alias)";
if (mode === "weighted") return "Weighted";
return "Round Robin";
}
+30
View File
@@ -0,0 +1,30 @@
import type { LbMode } from "@cfdm/shared";
export function uniqueIpCount(ips: readonly string[]): number {
return new Set(ips.filter(Boolean)).size;
}
/** Shared pool — two or more unique IPs. One IP (even duplicated) is not a pool. */
export function isSharedPool(ips: readonly string[]): boolean {
return uniqueIpCount(ips) >= 2;
}
/** LB / drain only when the service itself has a pool AND this FQDN is shared. */
export function canApplyLb(
serviceIps: readonly string[],
bindingIps: readonly string[],
): boolean {
return isSharedPool(serviceIps) && isSharedPool(bindingIps);
}
export function shouldRecordFailoverDnsDiff(input: {
configuredIps: readonly string[];
lbMode: LbMode;
added: readonly string[];
removed: readonly string[];
downIps: ReadonlySet<string>;
}): boolean {
if (!isSharedPool(input.configuredIps)) return false;
if (input.lbMode !== "weighted") return true;
return [...input.added, ...input.removed].some((ip) => input.downIps.has(ip));
}
+3 -3
View File
@@ -1,8 +1,8 @@
import type { LbIpRow } from "./types.js";
import { isHealthy } from "./health.js";
import { isPoolMember } from "./health.js";
export function roundRobinDesired(rows: LbIpRow[]): string[] {
const healthy = rows.filter((r) => isHealthy(r.health));
const pool = healthy.length > 0 ? healthy : rows;
const live = rows.filter((r) => isPoolMember(r.health));
const pool = live.length > 0 ? live : rows;
return pool.map((r) => r.ip);
}
+24
View File
@@ -0,0 +1,24 @@
import type { LbIpRow } from "./types.js";
import { isPoolMember } from "./health.js";
/** Slot length for time-sliced weighted DNS (one A at a time). */
export const WEIGHTED_SLOT_MS = 60_000;
/** Cloudflare DNS-only minimum TTL; Auto (1) is ~300s and would smear ratios. */
export const WEIGHTED_DNS_TTL = 60;
export function weightedDesired(rows: LbIpRow[], nowMs = Date.now()): string[] {
if (rows.length === 0) return [];
const live = rows.filter((r) => isPoolMember(r.health));
const pool = live.length > 0 ? live : rows;
if (pool.length === 1) return [pool[0]!.ip];
const sorted = [...pool].sort((a, b) => a.ip.localeCompare(b.ip));
const cycle: string[] = [];
for (const row of sorted) {
const weight = Math.max(1, Math.round(row.weight));
for (let i = 0; i < weight; i++) cycle.push(row.ip);
}
const slot = Math.floor(nowMs / WEIGHTED_SLOT_MS) % cycle.length;
return [cycle[slot]!];
}
+438 -87
View File
@@ -29,15 +29,91 @@ import * as domainService from "./domain-service.js";
import { syncServiceToVpsTracker } from "./vps-tracker-sync.js";
import { fireEnsureHealthWorker, DEFAULT_HEALTH_FALLBACKS } from "./health/health-worker-deploy.js";
import {
isHealthy,
canApplyLb,
isPoolMember,
isSharedPool,
resolveDesiredAIps,
selectActiveIpsByMode,
shouldRecordFailoverDnsDiff,
withBindingLock,
WEIGHTED_DNS_TTL,
type LbIpRow,
type LbTargetConfig,
} from "./routing/index.js";
export type { LbIpRow, LbTargetConfig };
export { selectActiveIpsByMode };
export {
canApplyLb,
resolveDesiredAIps,
selectActiveIpsByMode,
shouldRecordFailoverDnsDiff,
};
const AUTO_DNS_TTL = 1;
function ttlForBinding(mode: LbMode, ips: readonly string[]): number {
return mode === "weighted" && isSharedPool(ips) ? WEIGHTED_DNS_TTL : AUTO_DNS_TTL;
}
function enabledServiceIps(db: Db, serviceId: number): string[] {
return repos
.listServiceIpRows(db, serviceId)
.filter((row) => row.enabled)
.map((row) => row.ip);
}
export function failoverARecordDiff(
existingA: readonly string[],
desiredIps: readonly string[],
): { added: string[]; removed: string[] } {
const before = new Set(existingA);
const after = new Set(desiredIps);
return {
added: desiredIps.filter((ip) => !before.has(ip)),
removed: existingA.filter((ip) => !after.has(ip)),
};
}
function recordFailoverDnsDiff(
db: Db,
bindingId: number,
hostname: string,
zoneName: string,
existingRecords: DnsRecord[],
desiredIps: string[],
): void {
const existingA = existingRecords
.filter((record) => record.record_type.toUpperCase() === "A")
.map((record) => record.content);
const { added, removed } = failoverARecordDiff(existingA, desiredIps);
if (added.length === 0 && removed.length === 0) return;
const binding = repos.getBinding(db, bindingId);
const configuredIps = repos.listBindingIps(db, bindingId);
const { config, rows } = getBindingLbState(db, bindingId);
const downIps = new Set(
rows.filter((row) => row.health === "down").map((row) => row.ip),
);
if (
!shouldRecordFailoverDnsDiff({
configuredIps,
lbMode: config.lb_mode,
added,
removed,
downIps,
})
) {
return;
}
repos.insertFailoverLog(db, {
serviceId: binding.service_id,
bindingId,
fqdn: fqdnToDisplay(hostname, zoneName),
entries: [
...added.map((ip) => ({ ip, action: "added" as const })),
...removed.map((ip) => ({ ip, action: "removed" as const })),
],
});
}
export interface ServiceDomainInput {
fqdn: string;
@@ -211,7 +287,7 @@ function getGroupLbState(
} else {
existing.weight += weight;
existing.priority = Math.min(existing.priority, priority);
if (isHealthy(existing.health) && status && !isHealthy(status.status as IpHealthState)) {
if (isPoolMember(existing.health) && status && !isPoolMember(status.status as IpHealthState)) {
existing.health = status.status as IpHealthState;
}
}
@@ -227,16 +303,31 @@ function getGroupLbState(
};
}
function computeActiveIps(
function desiredAIps(
db: Db,
scope: HealthCheckScope,
refId: number,
fallbackIps: string[],
): string[] {
const state =
scope === "binding"
? getBindingLbState(db, refId)
: getGroupLbState(db, refId);
return selectActiveIpsByMode(state.config, state.rows);
// Configured binding/group IPs stay intact; DNS publishes only enabled ones.
const enabledIps =
scope === "binding"
? enabledServiceIps(db, repos.getBinding(db, refId).service_id)
: fallbackIps;
const enabledSet = new Set(enabledIps);
const activeFallback = fallbackIps.filter((ip) => enabledSet.has(ip));
const activeRows = state.rows.filter((row) => enabledSet.has(row.ip));
return resolveDesiredAIps(
state.config,
activeRows,
activeFallback,
Date.now(),
enabledIps,
);
}
async function collectKnownZones(
@@ -254,7 +345,38 @@ async function collectKnownZones(
return zones;
}
/** Restore common A-bindings whose IPs were shrunk by legacy IP toggles. */
function repairPoolSubsetBindings(db: Db, serviceId: number): void {
const pool = repos.listServiceIps(db, serviceId);
if (pool.length < 2) return;
const poolSet = new Set(pool);
for (const binding of repos.listBindingsByService(db, serviceId)) {
if (binding.cname_target?.trim()) continue;
const current = repos.listBindingIpsWithMeta(db, binding.id);
if (current.length <= 1) continue;
if (!current.every((entry) => poolSet.has(entry.ip))) continue;
const currentSet = new Set(current.map((entry) => entry.ip));
if (currentSet.size === pool.length && pool.every((ip) => currentSet.has(ip))) {
continue;
}
const byIp = new Map(current.map((entry) => [entry.ip, entry]));
repos.replaceBindingIpsWithMeta(
db,
binding.id,
pool.map((ip) => ({
ip,
weight: byIp.get(ip)?.weight ?? 1,
priority: byIp.get(ip)?.priority ?? 1,
})),
);
}
}
async function buildView(db: Db, serviceId: number): Promise<ServiceView> {
repairPoolSubsetBindings(db, serviceId);
const service = repos.getService(db, serviceId);
const ipRows = repos.listServiceIpRows(db, serviceId);
const ips = ipRows.map((row) => row.ip);
@@ -285,6 +407,11 @@ async function buildView(db: Db, serviceId: number): Promise<ServiceView> {
if (target_ip_priorities[ip] === undefined) target_ip_priorities[ip] = 1;
}
const { config, rows } = getBindingLbState(db, binding.id);
const bindingActiveIps = targetCname
? []
: resolveDesiredAIps(config, rows, targetIps, Date.now(), ips);
return {
binding_id: binding.id,
domain_id: binding.domain_id,
@@ -312,13 +439,13 @@ async function buildView(db: Db, serviceId: number): Promise<ServiceView> {
health_check_aggregate: binding.health_check_aggregate ?? "majority",
cert_monitoring: binding.cert_monitoring ?? "auto",
sync_status: aggregateSyncStatus(statuses),
active_ips: bindingActiveIps,
};
});
const activeIps = new Set<string>();
for (const binding of bindings) {
const { config, rows } = getBindingLbState(db, binding.id);
for (const ip of selectActiveIpsByMode(config, rows)) {
for (const domain of domainViews) {
for (const ip of domain.active_ips) {
activeIps.add(ip);
}
}
@@ -404,6 +531,49 @@ function fallbackCnameHealth(
);
}
function overlayLiveHealth(
stored: IpHealthState | undefined,
live: IpHealthState | undefined,
): IpHealthState {
if (live && live !== "unknown") return live;
return stored ?? live ?? "unknown";
}
function bestAliveDisplayStatus(statuses: readonly string[]): IpHealthState {
if (statuses.some((status) => status === "up")) return "up";
if (statuses.some((status) => status === "degraded")) return "degraded";
if (statuses.some((status) => status === "down")) return "down";
return "unknown";
}
/** Health badge applies only when the service, IP and HC (binding or group) are active. */
function isServiceHealthCheckActive(db: Db, view: ServiceView): boolean {
if ((view.domains ?? []).some((domain) => domain.health_check_enabled)) {
return true;
}
if (!view.service_group_id) return false;
const group = repos.getServiceGroup(db, view.service_group_id);
return Boolean(group.health_check_enabled);
}
function isIpHealthMonitored(db: Db, view: ServiceView, ip: string): boolean {
if (!view.enabled) return false;
if (view.ip_enabled[ip] === false) return false;
return isServiceHealthCheckActive(db, view);
}
function inactiveIpHealthRow(ip: string): ServiceHealthRow {
return {
ip,
status: "unknown",
latency_ms: null,
last_checked_at: null,
last_error: null,
provider: "local",
colo: null,
};
}
function attachServiceHealth(
db: Db,
views: ServiceView[],
@@ -411,10 +581,13 @@ function attachServiceHealth(
const ids = views.map((v) => v.id);
const healthByService = repos.aggregateIpHealthByServiceIds(db, ids);
const ipHealthByService = repos.listIpHealthByServiceIds(db, ids);
const liveByService = repos.listLatestLiveHealthByServiceIds(db, ids);
return views.map((view) => {
const health = healthByService.get(view.id);
const rows = ipHealthByService.get(view.id) ?? [];
const liveRows = liveByService.get(view.id) ?? [];
const byIp = new Map(rows.map((row) => [row.ip, row]));
const liveByIp = new Map(liveRows.map((row) => [row.ip, row]));
const cnameFallback = fallbackCnameHealth(rows, view);
const aRecordIps = new Set(
(view.domains ?? []).flatMap((domain) =>
@@ -422,21 +595,46 @@ function attachServiceHealth(
),
);
const ip_health = (view.ips ?? []).map((ip) => {
if (!isIpHealthMonitored(db, view, ip)) {
return inactiveIpHealthRow(ip);
}
const row = byIp.get(ip) ?? (aRecordIps.has(ip) ? undefined : cnameFallback);
const live = liveByIp.get(ip);
const status = overlayLiveHealth(row?.status, live?.status);
const extras = live && live.status !== "unknown" ? live : row;
return {
ip,
status: row?.status ?? ("unknown" as const),
latency_ms: row?.latency_ms ?? null,
last_checked_at: row?.last_checked_at ?? null,
last_error: row?.last_error ?? null,
provider: row?.provider ?? "local",
colo: row?.colo ?? null,
status,
latency_ms: extras?.latency_ms ?? null,
last_checked_at: extras?.last_checked_at ?? null,
last_error:
live && live.status !== "unknown"
? live.last_error
: (row?.last_error ?? null),
provider: extras?.provider ?? "local",
colo: extras?.colo ?? null,
};
});
const monitoredStatuses = ip_health
.filter((row) => isIpHealthMonitored(db, view, row.ip))
.map((row) => row.status);
const displayStatus =
monitoredStatuses.length > 0
? bestAliveDisplayStatus(monitoredStatuses)
: ("unknown" as const);
const latencyRow =
ip_health.find((row) => row.status === displayStatus && row.latency_ms != null) ??
ip_health.find((row) => row.latency_ms != null);
return {
...view,
health_status: health?.health_status ?? "unknown",
health_latency_ms: health?.health_latency_ms ?? null,
health_status:
monitoredStatuses.length > 0
? overlayLiveHealth(health?.health_status, displayStatus)
: "unknown",
health_latency_ms:
monitoredStatuses.length > 0 && displayStatus !== "unknown"
? (latencyRow?.latency_ms ?? null)
: null,
ip_health,
};
});
@@ -539,26 +737,11 @@ async function syncBindingDns(
desiredIps: string[],
cnameTarget: string | null,
): Promise<void> {
const domain = repos.getDomain(db, domainId);
const zoneName = domain.zone_name;
let effectiveCname = cnameTarget?.trim() || null;
if (!effectiveCname) {
const existingCname = await findOrImportDnsRecord(
db,
cf,
domainId,
zoneName,
hostname,
"CNAME",
);
if (existingCname) {
effectiveCname = existingCname.content;
repos.setBindingCnameTarget(db, bindingId, effectiveCname);
repos.replaceBindingIps(db, bindingId, []);
}
}
const effectiveCname = cnameTarget?.trim() || null;
// Desired config wins. Do NOT auto-adopt leftover CNAME from local/CF when the
// binding is A-mode — that wiped IPs and blocked extra FQDN publishes.
// Docs: https://developers.cloudflare.com/dns/manage-dns-records/troubleshooting/records-with-same-name/
if (effectiveCname) {
await syncBindingCnameDns(
db,
@@ -571,7 +754,83 @@ async function syncBindingDns(
return;
}
await syncBindingADns(db, cf, bindingId, domainId, hostname, desiredIps);
const binding = repos.getBinding(db, bindingId);
const configuredIps = repos.listBindingIps(db, bindingId);
await syncBindingADns(
db,
cf,
bindingId,
domainId,
hostname,
desiredIps,
ttlForBinding(binding.lb_mode, configuredIps),
);
}
/** Delete every local+CF record for hostname that must not remain in A mode. */
async function reconcileHostnameForA(
db: Db,
cf: CloudflareClient,
domainId: number,
zoneName: string,
hostname: string,
desiredIps: readonly string[],
): Promise<void> {
const desired = new Set(desiredIps);
// CNAME on the same name blocks creating A records in Cloudflare.
for (;;) {
const cname = await findOrImportDnsRecord(
db,
cf,
domainId,
zoneName,
hostname,
"CNAME",
);
if (!cname) break;
await dnsService.deleteRecord(db, cf, domainId, cname.id);
}
const domain = repos.getDomain(db, domainId);
const staleLocal = repos
.listDnsByDomain(db, domainId)
.filter(
(record) =>
record.record_type.toUpperCase() === "A" &&
dnsRecordNamesMatch(record.name, hostname, zoneName) &&
!desired.has(record.content),
);
for (const record of staleLocal) {
await dnsService.deleteRecord(db, cf, domainId, record.id);
}
const remote = await cf.listDnsRecords(domain.cf_zone_id);
for (const cfRec of remote) {
if ((cfRec.type ?? "").toUpperCase() !== "A" || !cfRec.id) continue;
if (!dnsRecordNamesMatch(cfRec.name, hostname, zoneName)) continue;
if (desired.has(cfRec.content)) continue;
const existing = repos.findDnsByCfId(db, domainId, cfRec.id);
if (existing) {
await dnsService.deleteRecord(db, cf, domainId, existing.id);
continue;
}
const imported = repos.insertDnsRecord(
db,
domainId,
cfRec.type,
cfRec.name,
cfRec.content,
cfRec.ttl,
cfRec.proxied ?? false,
cfRec.priority ?? null,
SYNC_SYNCED,
"cloudflare",
cfRec.id,
);
await dnsService.deleteRecord(db, cf, domainId, imported.id);
}
}
async function syncBindingCnameDns(
@@ -585,6 +844,21 @@ async function syncBindingCnameDns(
const domain = repos.getDomain(db, domainId);
const zoneName = domain.zone_name;
const normalized = normalizeCnameTarget(cnameTarget, zoneName);
// A/AAAA on the same name blocks CNAME create in Cloudflare.
for (;;) {
const conflictingA = await findOrImportDnsRecord(
db,
cf,
domainId,
zoneName,
hostname,
"A",
);
if (!conflictingA) break;
await dnsService.deleteRecord(db, cf, domainId, conflictingA.id);
}
const existingRecords = repos.listRecordsForBinding(db, bindingId);
for (const record of existingRecords) {
@@ -658,9 +932,14 @@ async function syncBindingADns(
domainId: number,
hostname: string,
desiredIps: string[],
ttl: number,
): Promise<void> {
const domain = repos.getDomain(db, domainId);
const zoneName = domain.zone_name;
// Align zone/local leftovers with desired A set (CNAME conflicts, stale A IPs).
await reconcileHostnameForA(db, cf, domainId, zoneName, hostname, desiredIps);
const existingRecords = repos.listRecordsForBinding(db, bindingId);
for (const record of existingRecords) {
@@ -677,6 +956,14 @@ async function syncBindingADns(
if (desiredIps.length === 0) {
repos.setBindingDnsRecordId(db, bindingId, null);
recordFailoverDnsDiff(
db,
bindingId,
hostname,
zoneName,
existingRecords,
desiredIps,
);
return;
}
@@ -685,13 +972,15 @@ async function syncBindingADns(
for (const ip of desiredIps) {
const existing = refreshed.find((r) => r.content === ip);
const recordName = dnsNameForBinding(hostname, zoneName);
let recordId: number;
if (existing) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName)) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName) || existing.ttl !== ttl) {
await dnsService.update(db, cf, domainId, existing.id, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
@@ -709,12 +998,24 @@ async function syncBindingADns(
if (adopted) {
repos.linkBindingRecord(db, bindingId, adopted.id);
recordId = adopted.id;
if (
!dnsRecordNamesMatch(adopted.name, hostname, zoneName) ||
adopted.ttl !== ttl
) {
await dnsService.update(db, cf, domainId, adopted.id, {
record_type: "A",
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
} else {
const record = await dnsService.create(db, cf, domainId, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl: 1,
ttl,
proxied: false,
});
repos.linkBindingRecord(db, bindingId, record.id);
@@ -725,6 +1026,14 @@ async function syncBindingADns(
}
repos.setBindingDnsRecordId(db, bindingId, primaryId);
recordFailoverDnsDiff(
db,
bindingId,
hostname,
zoneName,
existingRecords,
desiredIps,
);
}
async function cleanupBindingDns(
@@ -943,12 +1252,7 @@ async function syncServiceBindingsToDns(
}
validateTargetIpsInPool(targetIps, ips);
if (binding.health_check_enabled) {
const activeIps = computeActiveIps(db, "binding", binding.id);
if (activeIps.length > 0) {
targetIps = activeIps;
}
}
const desiredIps = desiredAIps(db, "binding", binding.id, targetIps);
await syncBindingDns(
db,
@@ -956,7 +1260,7 @@ async function syncServiceBindingsToDns(
binding.id,
binding.domain_id,
binding.hostname,
targetIps,
desiredIps,
null,
);
}
@@ -970,10 +1274,12 @@ async function collectGroupDnsIps(
const ips: string[] = [];
for (const service of services) {
if (!service.enabled) continue;
const enabled = new Set(enabledServiceIps(db, service.id));
const bindings = repos.listBindingsByService(db, service.id);
for (const binding of bindings) {
for (const ip of repos.listBindingIps(db, binding.id)) {
if (!ips.includes(ip)) ips.push(ip);
if (!enabled.has(ip) || ips.includes(ip)) continue;
ips.push(ip);
}
}
}
@@ -988,6 +1294,7 @@ async function syncGroupDomainDnsRecords(
domainId: number,
hostname: string,
desiredIps: string[],
ttl: number = AUTO_DNS_TTL,
): Promise<void> {
const domain = repos.getDomain(db, domainId);
const zoneName = domain.zone_name;
@@ -1003,14 +1310,16 @@ async function syncGroupDomainDnsRecords(
if (desiredIps.length === 0) return;
const refreshed = repos.listGroupDnsRecords(db, groupId);
const recordName = dnsNameForBinding(hostname, zoneName);
for (const ip of desiredIps) {
const existing = refreshed.find((r) => r.content === ip);
if (existing) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName)) {
if (!dnsRecordNamesMatch(existing.name, hostname, zoneName) || existing.ttl !== ttl) {
await dnsService.update(db, cf, domainId, existing.id, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
@@ -1026,13 +1335,25 @@ async function syncGroupDomainDnsRecords(
);
if (adopted) {
repos.linkGroupDnsRecord(db, groupId, adopted.id);
if (
!dnsRecordNamesMatch(adopted.name, hostname, zoneName) ||
adopted.ttl !== ttl
) {
await dnsService.update(db, cf, domainId, adopted.id, {
record_type: "A",
name: recordName,
content: ip,
ttl,
proxied: false,
});
}
continue;
}
const record = await dnsService.create(db, cf, domainId, {
record_type: "A",
name: dnsNameForBinding(hostname, zoneName),
name: recordName,
content: ip,
ttl: 1,
ttl,
proxied: false,
});
repos.linkGroupDnsRecord(db, groupId, record.id);
@@ -1083,9 +1404,8 @@ async function syncGroupDomainDns(
const knownZones = await collectKnownZones(db, cf);
const { zoneName, hostname } = parseFqdn(domainValue, knownZones);
const domainId = await resolveDomainId(db, cf, zoneName);
const desiredIps = group.health_check_enabled
? computeActiveIps(db, "group", groupId)
: await collectGroupDnsIps(db, groupId);
const fallbackIps = await collectGroupDnsIps(db, groupId);
const desiredIps = desiredAIps(db, "group", groupId, fallbackIps);
await syncGroupDomainDnsRecords(
db,
cf,
@@ -1093,6 +1413,7 @@ async function syncGroupDomainDns(
domainId,
hostname,
desiredIps,
ttlForBinding(group.lb_mode, fallbackIps),
);
}
@@ -1273,14 +1594,7 @@ export async function updateConfig(
}
if (pushDns) {
let effectiveIps = targetIps;
const refreshedBinding = repos.getBinding(db, binding.id);
if (refreshedBinding.health_check_enabled) {
const activeIps = computeActiveIps(db, "binding", binding.id);
if (activeIps.length > 0) {
effectiveIps = activeIps;
}
}
const effectiveIps = desiredAIps(db, "binding", binding.id, targetIps);
await syncBindingDns(
db,
cf,
@@ -1486,26 +1800,8 @@ export async function toggleServiceIp(
repos.updateNode(db, node.id, { enabled });
}
const bindings = repos.listBindingsByService(db, serviceId);
for (const binding of bindings) {
if (binding.cname_target?.trim()) continue;
const current = repos.listBindingIpsWithMeta(db, binding.id);
const hasIp = current.some((entry) => entry.ip === ip);
if (enabled && !hasIp) {
repos.replaceBindingIpsWithMeta(db, binding.id, [
...current,
{ ip, weight: 1, priority: 1 },
]);
continue;
}
if (!enabled && hasIp) {
repos.replaceBindingIpsWithMeta(
db,
binding.id,
current.filter((entry) => entry.ip !== ip),
);
}
}
// Keep binding IP membership stable (common FQDN = full pool). DNS sync
// filters by enabledServiceIps via desiredAIps — do not reshuffle bindings.
const service = repos.getService(db, serviceId);
if (shouldPushDns(db, service)) {
@@ -1586,16 +1882,18 @@ export async function reconcileDnsForTarget(
if (scope === "binding") {
await withBindingLock(refId, async () => {
const binding = repos.getBinding(db, refId);
if (!binding.health_check_enabled) return;
if (!binding.health_check_enabled && binding.lb_mode !== "weighted") return;
const service = repos.getService(db, binding.service_id);
if (!shouldPushDns(db, service)) return;
const cnameTarget = binding.cname_target?.trim() || null;
if (cnameTarget) return;
const ips = repos.listServiceIps(db, service.id);
const poolIps = enabledServiceIps(db, service.id);
const targetIps = repos.listBindingIps(db, binding.id);
validateTargetIpsInPool(targetIps, ips);
const activeIps = computeActiveIps(db, "binding", refId);
const desiredIps = activeIps.length > 0 ? activeIps : targetIps;
const desiredIps = canApplyLb(poolIps, targetIps)
? desiredAIps(db, "binding", refId, targetIps)
: targetIps;
await syncBindingDns(
db,
cf,
@@ -1610,8 +1908,61 @@ export async function reconcileDnsForTarget(
}
const group = repos.getServiceGroup(db, refId);
if (!group.enabled || !group.domain?.trim() || !group.health_check_enabled) {
if (!group.enabled || !group.domain?.trim()) {
return;
}
if (!group.health_check_enabled && group.lb_mode !== "weighted") {
return;
}
await syncGroupDomainDns(db, cf, refId);
}
export async function reconcileWeightedDns(
db: Db,
cf: CloudflareClient,
): Promise<number> {
let n = 0;
for (const binding of repos.listAllBindings(db)) {
if (binding.lb_mode !== "weighted") continue;
if (binding.cname_target?.trim()) continue;
if (!isSharedPool(binding.target_ips ?? [])) continue;
try {
await withBindingLock(binding.id, async () => {
const latest = repos.getBinding(db, binding.id);
if (latest.lb_mode !== "weighted") return;
if (latest.cname_target?.trim()) return;
const service = repos.getService(db, latest.service_id);
if (!shouldPushDns(db, service)) return;
const targetIps = repos.listBindingIps(db, latest.id);
const poolIps = enabledServiceIps(db, service.id);
if (!canApplyLb(poolIps, targetIps)) return;
const ips = repos.listServiceIps(db, service.id);
validateTargetIpsInPool(targetIps, ips);
const desiredIps = desiredAIps(db, "binding", latest.id, targetIps);
await syncBindingDns(
db,
cf,
latest.id,
latest.domain_id,
latest.hostname,
desiredIps,
null,
);
n += 1;
});
} catch {
continue;
}
}
for (const group of repos.listServiceGroups(db)) {
if (group.lb_mode !== "weighted") continue;
if (!group.enabled || !group.domain?.trim()) continue;
try {
await syncGroupDomainDns(db, cf, group.id);
n += 1;
} catch {
continue;
}
}
return n;
}
+54 -1
View File
@@ -1,8 +1,53 @@
import { resolve4 } from "node:dns/promises";
import type { CfdmBindingSyncItem, ServiceBindingView } from "@cfdm/shared";
import type { CfdmBindingSyncItem, LbMode, ServiceBindingView } from "@cfdm/shared";
import { isIpLiteral } from "@cfdm/shared";
import type { Db } from "@cfdm/db";
import { repos, getAppSettingsSecrets, touchVpsTrackerSync } from "@cfdm/db";
import { isSharedPool } from "./routing/pool.js";
export function isLbMode(value: unknown): value is LbMode {
return value === "round_robin" || value === "failover" || value === "weighted";
}
/** lb_mode binding, иначе service group. */
export function resolveLbModeForSync(
bindingLbMode: string | undefined | null,
groupLbMode?: string | null,
): LbMode | undefined {
if (isLbMode(bindingLbMode)) return bindingLbMode;
if (isLbMode(groupLbMode)) return groupLbMode;
return undefined;
}
/** Effective HA only when the service has two or more unique origin IPs. */
export function effectiveLbModeForSync(
bindingLbMode: string | undefined | null,
groupLbMode: string | undefined | null,
serviceIps: readonly string[],
): LbMode | undefined {
if (!isSharedPool(serviceIps)) return undefined;
return resolveLbModeForSync(bindingLbMode, groupLbMode);
}
function groupLbModeForService(
db: Db,
serviceId: number,
cache: Map<number, LbMode | undefined>,
): LbMode | undefined {
if (cache.has(serviceId)) return cache.get(serviceId);
let mode: LbMode | undefined;
try {
const service = repos.getService(db, serviceId);
if (service.service_group_id != null) {
const group = repos.getServiceGroup(db, service.service_group_id);
mode = resolveLbModeForSync(undefined, group.lb_mode);
}
} catch {
mode = undefined;
}
cache.set(serviceId, mode);
return mode;
}
function fqdnToDisplay(hostname: string, zoneName: string): string {
if (hostname === "@" || !hostname.trim()) return zoneName;
@@ -126,10 +171,13 @@ export async function buildServiceSyncBindingsAsync(
const allBindings = repos.listAllBindings(db);
const index = buildBindingIndex(allBindings);
const bindings = allBindings.filter((row) => row.service_id === serviceId);
const groupLbCache = new Map<number, LbMode | undefined>();
const groupLb = groupLbModeForService(db, serviceId, groupLbCache);
const items: CfdmBindingSyncItem[] = [];
for (const binding of bindings) {
const ips = await resolveBindingIpsForSync(binding, serviceIps, index, db);
const lbMode = effectiveLbModeForSync(binding.lb_mode, groupLb, serviceIps);
items.push({
bindingId: binding.id,
serviceId: service.id,
@@ -140,6 +188,7 @@ export async function buildServiceSyncBindingsAsync(
hostname: binding.hostname,
ips,
cnameTarget: cnameTargetForSync(binding),
...(lbMode ? { lbMode } : {}),
});
}
@@ -166,6 +215,7 @@ export async function buildAllSyncBindings(
const bindings = repos.listAllBindings(db);
const index = buildBindingIndex(bindings);
const serviceIpCache = new Map<number, string[]>();
const groupLbCache = new Map<number, LbMode | undefined>();
const items: CfdmBindingSyncItem[] = [];
for (const binding of bindings) {
@@ -175,6 +225,8 @@ export async function buildAllSyncBindings(
serviceIpCache.set(binding.service_id, serviceIps);
}
const ips = await resolveBindingIpsForSync(binding, serviceIps, index, db);
const groupLb = groupLbModeForService(db, binding.service_id, groupLbCache);
const lbMode = effectiveLbModeForSync(binding.lb_mode, groupLb, serviceIps);
items.push({
bindingId: binding.id,
serviceId: binding.service_id,
@@ -185,6 +237,7 @@ export async function buildAllSyncBindings(
hostname: binding.hostname,
ips,
cnameTarget: cnameTargetForSync(binding),
...(lbMode ? { lbMode } : {}),
});
}
return items;
@@ -0,0 +1,39 @@
import type { FastifyInstance } from "fastify";
import { AsyncTask, SimpleIntervalJob } from "toad-scheduler";
import * as serviceConfigService from "./service-config-service.js";
import { WEIGHTED_SLOT_MS } from "./routing/weighted.js";
export const WEIGHTED_DNS_JOB_ID = "weighted-dns";
export function createWeightedDnsTask(app: FastifyInstance): AsyncTask {
return new AsyncTask(
WEIGHTED_DNS_JOB_ID,
async () => {
const n = await serviceConfigService.reconcileWeightedDns(app.db, app.cf);
if (n > 0) {
app.log.info({ reconciled: n }, "weighted dns rotated");
}
},
(err) => {
app.log.warn({ err }, "weighted dns rotate failed");
},
);
}
export function scheduleWeightedDnsJob(
app: FastifyInstance,
task: AsyncTask,
): void {
const scheduler = app.scheduler;
if (!scheduler) return;
if (scheduler.existsById(WEIGHTED_DNS_JOB_ID)) {
scheduler.removeById(WEIGHTED_DNS_JOB_ID);
}
scheduler.addSimpleIntervalJob(
new SimpleIntervalJob(
{ seconds: WEIGHTED_SLOT_MS / 1000, runImmediately: true },
task,
{ id: WEIGHTED_DNS_JOB_ID, preventOverrun: true },
),
);
}
+220
View File
@@ -0,0 +1,220 @@
import { describe, expect, it } from "vitest";
import { createMemoryDb, repos, runMigrations } from "@cfdm/db";
import { SYNC_SYNCED } from "@cfdm/shared";
import type { CloudflareClient } from "../src/lib/cf-client.js";
import * as dnsService from "../src/services/dns-service.js";
import { updateConfig } from "../src/services/service-config-service.js";
type CfRec = {
id: string;
type: string;
name: string;
content: string;
ttl: number;
proxied: boolean;
};
function setupDb() {
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
return db;
}
describe("DNS identity by name + content", () => {
it("deletes by name+IP when cached cf_record_id is stale/missing in CF", async () => {
const db = setupDb();
const remote: CfRec[] = [
{
id: "cf-live",
type: "A",
name: "nsgt.example.com",
content: "130.49.213.176",
ttl: 1,
proxied: false,
},
];
const deleted: string[] = [];
const cf = {
listDnsRecords: async () => [...remote],
createDnsRecord: async () => {
throw new Error("create should not run");
},
updateDnsRecord: async () => {
throw new Error("update should not run");
},
deleteDnsRecord: async (_zoneId: string, id: string) => {
deleted.push(id);
const idx = remote.findIndex((r) => r.id === id);
if (idx >= 0) remote.splice(idx, 1);
},
verifyToken: async () => true,
listZones: async () => [],
} as unknown as CloudflareClient;
const domain = repos.createDomain(db, null, "example.com", "zone-1");
const local = repos.insertDnsRecord(
db,
domain.id,
"A",
"nsgt.example.com",
"130.49.213.176",
1,
false,
null,
SYNC_SYNCED,
"local",
"cf-stale-gone", // not present in Cloudflare
);
await dnsService.deleteRecord(db, cf, domain.id, local.id);
expect(deleted).toEqual(["cf-live"]);
expect(repos.listDnsByDomain(db, domain.id)).toEqual([]);
expect(remote).toEqual([]);
});
it("delete is no-op success when record already removed outside CFDM", async () => {
const db = setupDb();
const cf = {
listDnsRecords: async () => [],
deleteDnsRecord: async () => {
throw new Error("Record does not exist");
},
verifyToken: async () => true,
listZones: async () => [],
} as unknown as CloudflareClient;
const domain = repos.createDomain(db, null, "example.com", "zone-1");
const local = repos.insertDnsRecord(
db,
domain.id,
"A",
"nsgt.example.com",
"130.49.213.176",
1,
false,
null,
SYNC_SYNCED,
"local",
"cf-already-gone",
);
await expect(
dnsService.deleteRecord(db, cf, domain.id, local.id),
).resolves.toBeUndefined();
expect(repos.listDnsByDomain(db, domain.id)).toEqual([]);
});
it("updateConfig can remove extra FQDN when CF id is stale", async () => {
const db = setupDb();
const remote: CfRec[] = [
{
id: "cf-gt",
type: "A",
name: "gt.example.com",
content: "130.49.213.176",
ttl: 1,
proxied: false,
},
{
id: "cf-nsgt-live",
type: "A",
name: "nsgt.example.com",
content: "130.49.213.176",
ttl: 1,
proxied: false,
},
];
const deleted: string[] = [];
const cf = {
listDnsRecords: async () => [...remote],
createDnsRecord: async (
_zoneId: string,
payload: { type: string; name: string; content: string },
) => {
const rec: CfRec = {
id: `cf-new-${remote.length}`,
type: payload.type,
name: payload.name,
content: payload.content,
ttl: 1,
proxied: false,
};
remote.push(rec);
return rec;
},
updateDnsRecord: async (
_zoneId: string,
id: string,
payload: { type: string; name: string; content: string },
) => {
const idx = remote.findIndex((r) => r.id === id);
if (idx < 0) throw new Error("Record does not exist");
const rec: CfRec = {
id,
type: payload.type,
name: payload.name,
content: payload.content,
ttl: 1,
proxied: false,
};
remote[idx] = rec;
return rec;
},
deleteDnsRecord: async (_zoneId: string, id: string) => {
const idx = remote.findIndex((r) => r.id === id);
if (idx < 0) throw new Error("Record does not exist");
deleted.push(id);
remote.splice(idx, 1);
},
verifyToken: async () => true,
listZones: async () => [
{ id: "zone-1", name: "example.com", status: "active" },
],
} as unknown as CloudflareClient;
repos.createDomain(db, null, "example.com", "zone-1");
const service = repos.createService(db, "Main TG", "tg-pr");
repos.setServiceEnabled(db, service.id, true);
await updateConfig(db, cf, service.id, {
ips: ["130.49.213.176"],
domains: [
{ fqdn: "gt.example.com", target_ips: ["130.49.213.176"] },
{ fqdn: "nsgt.example.com", target_ips: ["130.49.213.176"] },
],
});
// Poison cached id on extra binding's DNS row.
const nsgtBinding = repos
.listBindingsByService(db, service.id)
.find((b) => b.hostname === "nsgt")!;
const nsgtRecords = repos.listRecordsForBinding(db, nsgtBinding.id);
for (const row of nsgtRecords) {
repos.updateDnsFields(
db,
row.id,
row.record_type,
row.name,
row.content,
row.ttl,
row.proxied,
row.priority,
SYNC_SYNCED,
"cf-stale-nsgt",
null,
);
}
const view = await updateConfig(db, cf, service.id, {
ips: ["130.49.213.176"],
domains: [{ fqdn: "gt.example.com", target_ips: ["130.49.213.176"] }],
});
expect(view.domains.map((d) => d.fqdn)).toEqual(["gt.example.com"]);
expect(deleted).toContain("cf-nsgt-live");
expect(remote.some((r) => r.name.includes("nsgt"))).toBe(false);
});
});
+224
View File
@@ -0,0 +1,224 @@
import { describe, expect, it } from "vitest";
import { createMemoryDb, repos, runMigrations } from "@cfdm/db";
import { SYNC_SYNCED } from "@cfdm/shared";
import type { CloudflareClient } from "../src/lib/cf-client.js";
import { updateConfig } from "../src/services/service-config-service.js";
type CfRec = {
id: string;
type: string;
name: string;
content: string;
ttl: number;
proxied: boolean;
};
function setupDb() {
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
return db;
}
describe("DNS reconcile for extra FQDN", () => {
it("publishes extra A even when local/CF already have CNAME on same name", async () => {
const db = setupDb();
const remote: CfRec[] = [
{
id: "cf-cname-nsgt",
type: "CNAME",
name: "nsgt.example.com",
content: "legacy.example.com",
ttl: 1,
proxied: false,
},
];
const created: Array<{ type: string; name: string; content: string }> = [];
const deleted: string[] = [];
const cf = {
listDnsRecords: async () => remote,
createDnsRecord: async (
_zoneId: string,
payload: { type: string; name: string; content: string },
) => {
created.push(payload);
const rec: CfRec = {
id: `cf-new-${created.length}`,
type: payload.type,
name: payload.name,
content: payload.content,
ttl: 1,
proxied: false,
};
remote.push(rec);
return rec;
},
updateDnsRecord: async (
_zoneId: string,
id: string,
payload: { type: string; name: string; content: string },
) => {
const idx = remote.findIndex((r) => r.id === id);
const rec: CfRec = {
id,
type: payload.type,
name: payload.name,
content: payload.content,
ttl: 1,
proxied: false,
};
if (idx >= 0) remote[idx] = rec;
else remote.push(rec);
return rec;
},
deleteDnsRecord: async (_zoneId: string, id: string) => {
deleted.push(id);
const idx = remote.findIndex((r) => r.id === id);
if (idx >= 0) remote.splice(idx, 1);
},
verifyToken: async () => true,
listZones: async () => [
{ id: "zone-1", name: "example.com", status: "active" },
],
} as unknown as CloudflareClient;
const domain = repos.createDomain(db, null, "example.com", "zone-1");
// Leftover local CNAME (previous service / manual import).
repos.insertDnsRecord(
db,
domain.id,
"CNAME",
"nsgt.example.com",
"legacy.example.com",
1,
false,
null,
SYNC_SYNCED,
"cloudflare",
"cf-cname-nsgt",
);
const service = repos.createService(db, "MSK Hip", "tg-msk-hip");
repos.setServiceEnabled(db, service.id, true);
const view = await updateConfig(db, cf, service.id, {
ips: ["130.49.213.176"],
domains: [
{
fqdn: "gt.example.com",
target_ips: ["130.49.213.176"],
},
{
fqdn: "nsgt.example.com",
target_ips: ["130.49.213.176"],
},
],
});
const nsgt = view.domains.find((d) => d.fqdn === "nsgt.example.com");
expect(nsgt?.record_type).toBe("A");
expect(nsgt?.target_ips).toEqual(["130.49.213.176"]);
expect(nsgt?.target_cname).toBeFalsy();
const binding = repos
.listBindingsByService(db, service.id)
.find((b) => b.hostname === "nsgt")!;
expect(repos.listBindingIps(db, binding.id)).toEqual(["130.49.213.176"]);
expect(deleted).toContain("cf-cname-nsgt");
expect(
created.some(
(r) =>
r.type === "A" &&
(r.name === "nsgt" || r.name === "nsgt.example.com") &&
r.content === "130.49.213.176",
),
).toBe(true);
expect(remote.some((r) => r.type === "CNAME" && r.name.includes("nsgt"))).toBe(
false,
);
expect(
remote.some(
(r) =>
r.type === "A" &&
r.name.includes("nsgt") &&
r.content === "130.49.213.176",
),
).toBe(true);
});
it("replaces stale A content for extra FQDN on the same hostname", async () => {
const db = setupDb();
const remote: CfRec[] = [
{
id: "cf-stale-a",
type: "A",
name: "nsgt.example.com",
content: "1.1.1.1",
ttl: 1,
proxied: false,
},
];
const deleted: string[] = [];
const created: Array<{ type: string; name: string; content: string }> = [];
const cf = {
listDnsRecords: async () => [...remote],
createDnsRecord: async (
_zoneId: string,
payload: { type: string; name: string; content: string },
) => {
created.push(payload);
const rec: CfRec = {
id: `cf-new-${created.length}`,
type: payload.type,
name: payload.name,
content: payload.content,
ttl: 1,
proxied: false,
};
remote.push(rec);
return rec;
},
updateDnsRecord: async (
_zoneId: string,
id: string,
payload: { type: string; name: string; content: string },
) => ({
id,
type: payload.type,
name: payload.name,
content: payload.content,
ttl: 1,
proxied: false,
}),
deleteDnsRecord: async (_zoneId: string, id: string) => {
deleted.push(id);
const idx = remote.findIndex((r) => r.id === id);
if (idx >= 0) remote.splice(idx, 1);
},
verifyToken: async () => true,
listZones: async () => [
{ id: "zone-1", name: "example.com", status: "active" },
],
} as unknown as CloudflareClient;
repos.createDomain(db, null, "example.com", "zone-1");
const service = repos.createService(db, "MSK", "msk");
repos.setServiceEnabled(db, service.id, true);
await updateConfig(db, cf, service.id, {
ips: ["130.49.213.176"],
domains: [
{ fqdn: "nsgt.example.com", target_ips: ["130.49.213.176"] },
],
});
expect(deleted).toContain("cf-stale-a");
expect(
created.some(
(r) => r.type === "A" && r.content === "130.49.213.176",
),
).toBe(true);
expect(remote.map((r) => r.content)).toEqual(["130.49.213.176"]);
});
});
+112
View File
@@ -0,0 +1,112 @@
import { describe, expect, it } from "vitest";
import { createMemoryDb, repos, runMigrations } from "@cfdm/db";
import { buildApp } from "../src/app.js";
import { loadConfig } from "../src/config.js";
import { failoverARecordDiff } from "../src/services/service-config-service.js";
async function authHeaders(app: Awaited<ReturnType<typeof buildApp>>) {
const config = loadConfig();
const res = await app.inject({
method: "POST",
url: "/api/v1/auth/login",
payload: { username: config.adminUsername, password: "admin" },
});
expect(res.statusCode).toBe(200);
const { token } = res.json() as { token: string };
return { authorization: `Bearer ${token}` };
}
describe("failoverARecordDiff", () => {
it("diffs added and removed A contents", () => {
expect(
failoverARecordDiff(
["130.49.213.153", "93.115.203.183"],
["93.115.203.183"],
),
).toEqual({
added: [],
removed: ["130.49.213.153"],
});
expect(failoverARecordDiff(["10.0.0.1"], ["10.0.0.1", "10.0.0.2"])).toEqual({
added: ["10.0.0.2"],
removed: [],
});
});
});
describe("GET /services/:id/failover-log", () => {
it("returns add/remove rows for the service", async () => {
const app = await buildApp({
config: { ...loadConfig(), staticDir: null },
memory: true,
});
const headers = await authHeaders(app);
const domain = repos.createDomain(app.db, null, "rkns.top", "zone-1");
const service = repos.createService(app.db, "MSK Hip", "msk-hip");
const binding = repos.insertBinding(app.db, domain.id, service.id, "gt", null);
repos.insertFailoverLog(app.db, {
serviceId: service.id,
bindingId: binding.id,
fqdn: "gt.rkns.top",
entries: [
{ ip: "130.49.213.153", action: "removed" },
{ ip: "93.115.203.183", action: "added" },
],
});
const res = await app.inject({
method: "GET",
url: `/api/v1/services/${service.id}/failover-log`,
headers,
});
expect(res.statusCode).toBe(200);
const body = res.json() as {
items: Array<{ ip: string; fqdn: string; action: string }>;
};
expect(body.items).toEqual(
expect.arrayContaining([
expect.objectContaining({
ip: "130.49.213.153",
fqdn: "gt.rkns.top",
action: "removed",
}),
expect.objectContaining({
ip: "93.115.203.183",
fqdn: "gt.rkns.top",
action: "added",
}),
]),
);
await app.close();
});
});
describe("failover_log table", () => {
it("lists newest first", () => {
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "example.com", "zone-1");
const service = repos.createService(db, "Panel", "panel");
const binding = repos.insertBinding(db, domain.id, service.id, "panel", null);
repos.insertFailoverLog(db, {
serviceId: service.id,
bindingId: binding.id,
fqdn: "panel.example.com",
entries: [{ ip: "1.1.1.1", action: "removed" }],
});
repos.insertFailoverLog(db, {
serviceId: service.id,
bindingId: binding.id,
fqdn: "panel.example.com",
entries: [{ ip: "1.1.1.1", action: "added" }],
});
const rows = repos.listFailoverLogForService(db, service.id);
expect(rows.map((row) => row.action)).toEqual(["added", "removed"]);
});
});
+154
View File
@@ -376,6 +376,7 @@ describe("CNAME health mapped onto service IPs", () => {
repos.replaceServiceIps(db, service.id, ["2.59.161.102"]);
const binding = repos.insertBinding(db, domain.id, service.id, "s", null);
repos.setBindingCnameTarget(db, binding.id, "ihome.rkns.top");
repos.updateBindingLbConfig(db, binding.id, { health_check_enabled: true });
repos.upsertIpHealthStatus(
db,
"binding",
@@ -397,4 +398,157 @@ describe("CNAME health mapped onto service IPs", () => {
}),
]);
});
it("getView is up when any binding IP is up", async () => {
const { createMemoryDb, repos, runMigrations } = await import("@cfdm/db");
const { getView } = await import("../src/services/service-config-service.js");
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "rkns.top", "zone-id");
const service = repos.createService(db, "MSK Hip", "msk-hip");
repos.replaceServiceIps(db, service.id, ["10.0.0.1", "10.0.0.2"]);
const binding = repos.insertBinding(db, domain.id, service.id, "gt", null);
repos.replaceBindingIpsWithMeta(db, binding.id, [
{ ip: "10.0.0.1", weight: 1, priority: 1 },
{ ip: "10.0.0.2", weight: 1, priority: 1 },
]);
repos.updateBindingLbConfig(db, binding.id, { health_check_enabled: true });
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"10.0.0.1",
"up",
12,
0,
null,
);
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"10.0.0.2",
"down",
null,
5,
"timeout",
);
const view = await getView(db, service.id);
expect(view.health_status).toBe("up");
});
it("getView shows live OK over hysteresis unknown", async () => {
const { createMemoryDb, repos, runMigrations } = await import("@cfdm/db");
const { getView } = await import("../src/services/service-config-service.js");
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "rkns.top", "zone-id");
const service = repos.createService(db, "RW Panel", "rw-panel");
repos.replaceServiceIps(db, service.id, ["2.59.161.102"]);
const binding = repos.insertBinding(db, domain.id, service.id, "c", null);
repos.replaceBindingIpsWithMeta(db, binding.id, [
{ ip: "2.59.161.102", weight: 1, priority: 1 },
]);
repos.updateBindingLbConfig(db, binding.id, { health_check_enabled: true });
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"2.59.161.102",
"unknown",
63,
0,
null,
1,
);
repos.insertHealthProbeLog(db, {
scope: "binding",
refId: binding.id,
ip: "2.59.161.102",
provider: "local",
status: "up",
ok: true,
latencyMs: 63,
colo: null,
error: null,
});
const view = await getView(db, service.id);
expect(view.health_status).toBe("up");
expect(view.ip_health[0]?.status).toBe("up");
expect(view.ip_health[0]?.latency_ms).toBe(63);
});
it("getView masks stale down when health-check is disabled", async () => {
const { createMemoryDb, repos, runMigrations } = await import("@cfdm/db");
const { getView } = await import("../src/services/service-config-service.js");
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "rkns.top", "zone-id");
const service = repos.createService(db, "Main TG", "main-tg");
repos.setServiceEnabled(db, service.id, true);
repos.replaceServiceIps(db, service.id, ["130.49.213.176"]);
const binding = repos.insertBinding(db, domain.id, service.id, "gt", null);
repos.replaceBindingIpsWithMeta(db, binding.id, [
{ ip: "130.49.213.176", weight: 1, priority: 1 },
]);
repos.updateBindingLbConfig(db, binding.id, { health_check_enabled: false });
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"130.49.213.176",
"down",
null,
5,
"timeout",
);
const view = await getView(db, service.id);
expect(view.health_status).toBe("unknown");
expect(view.ip_health).toEqual([
expect.objectContaining({
ip: "130.49.213.176",
status: "unknown",
latency_ms: null,
last_error: null,
}),
]);
});
it("getView masks stale down when IP is disabled in pool", async () => {
const { createMemoryDb, repos, runMigrations } = await import("@cfdm/db");
const { getView } = await import("../src/services/service-config-service.js");
const { db, sqlite } = createMemoryDb();
runMigrations(sqlite);
const domain = repos.createDomain(db, null, "rkns.top", "zone-id");
const service = repos.createService(db, "Main TG", "main-tg");
repos.setServiceEnabled(db, service.id, true);
repos.replaceServiceIps(db, service.id, ["130.49.213.176"]);
repos.setServiceIpEnabled(db, service.id, "130.49.213.176", false);
const binding = repos.insertBinding(db, domain.id, service.id, "gt", null);
repos.replaceBindingIpsWithMeta(db, binding.id, [
{ ip: "130.49.213.176", weight: 1, priority: 1 },
]);
repos.updateBindingLbConfig(db, binding.id, { health_check_enabled: true });
repos.upsertIpHealthStatus(
db,
"binding",
binding.id,
"130.49.213.176",
"down",
null,
5,
"timeout",
);
const view = await getView(db, service.id);
expect(view.health_status).toBe("unknown");
expect(view.ip_health[0]?.status).toBe("unknown");
});
});
+200 -12
View File
@@ -1,9 +1,12 @@
import { describe, expect, it } from "vitest";
import {
resolveDesiredAIps,
selectActiveIpsByMode,
shouldRecordFailoverDnsDiff,
type LbIpRow,
type LbTargetConfig,
} from "../src/services/service-config-service.js";
import { WEIGHTED_SLOT_MS } from "../src/services/routing/weighted.js";
function row(
ip: string,
@@ -17,6 +20,11 @@ function row(
};
}
const weightedConfig: LbTargetConfig = {
lb_mode: "weighted",
health_check_enabled: true,
};
describe("selectActiveIpsByMode", () => {
it("round_robin returns all healthy ips, falls back to all if none healthy", () => {
const config: LbTargetConfig = {
@@ -62,6 +70,18 @@ describe("selectActiveIpsByMode", () => {
expect(selectActiveIpsByMode(config, rows)).toEqual(["1.1.1.1"]);
});
it("failover returns recovering unknown primary immediately", () => {
const config: LbTargetConfig = {
lb_mode: "failover",
health_check_enabled: true,
};
const rows = [
row("1.1.1.1", { priority: 1, health: "unknown" }),
row("2.2.2.2", { priority: 2, health: "up" }),
];
expect(selectActiveIpsByMode(config, rows)).toEqual(["1.1.1.1"]);
});
it("failover falls back to min-priority ip among all when none healthy", () => {
const config: LbTargetConfig = {
lb_mode: "failover",
@@ -75,23 +95,61 @@ describe("selectActiveIpsByMode", () => {
expect(selectActiveIpsByMode(config, rows)).toEqual(["2.2.2.2"]);
});
it("weighted returns all healthy ips (one A per ip; weights stored for display)", () => {
const config: LbTargetConfig = {
lb_mode: "weighted",
health_check_enabled: true,
};
it("weighted 1:3 picks the lighter ip on slot 0 and the heavier on slot 1", () => {
const rows = [
row("1.1.1.1", { weight: 3, health: "up" }),
row("2.2.2.2", { weight: 1, health: "up" }),
row("3.3.3.3", { weight: 2, health: "down" }),
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "up" }),
];
expect(selectActiveIpsByMode(config, rows).sort()).toEqual([
"1.1.1.1",
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS)).toEqual([
"2.2.2.2",
]);
});
it("round_robin excludes unknown when another ip is up", () => {
it("weighted excludes down ips from the cycle", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "down" }),
];
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(
selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS),
).toEqual(["1.1.1.1"]);
});
it("weighted includes recovering unknown in the cycle", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "unknown" }),
];
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(
selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS),
).toEqual(["2.2.2.2"]);
});
it("weighted with one ip always returns that ip", () => {
expect(
selectActiveIpsByMode(weightedConfig, [row("1.1.1.1", { weight: 5 })], 0),
).toEqual(["1.1.1.1"]);
});
it("weighted with all unknown rotates across every ip", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "unknown" }),
row("2.2.2.2", { weight: 3, health: "unknown" }),
];
expect(selectActiveIpsByMode(weightedConfig, rows, 0)).toEqual(["1.1.1.1"]);
expect(selectActiveIpsByMode(weightedConfig, rows, WEIGHTED_SLOT_MS)).toEqual([
"2.2.2.2",
]);
});
it("weighted returns empty array for no rows", () => {
expect(selectActiveIpsByMode(weightedConfig, [], 0)).toEqual([]);
});
it("round_robin puts recovering unknown back with live ips", () => {
const config: LbTargetConfig = {
lb_mode: "round_robin",
health_check_enabled: true,
@@ -100,7 +158,25 @@ describe("selectActiveIpsByMode", () => {
row("1.1.1.1", { health: "up" }),
row("2.2.2.2", { health: "unknown" }),
];
expect(selectActiveIpsByMode(config, rows)).toEqual(["1.1.1.1"]);
expect(selectActiveIpsByMode(config, rows).sort()).toEqual([
"1.1.1.1",
"2.2.2.2",
]);
});
it("round_robin keeps degraded in the pool with live ips", () => {
const config: LbTargetConfig = {
lb_mode: "round_robin",
health_check_enabled: true,
};
const rows = [
row("1.1.1.1", { health: "up" }),
row("2.2.2.2", { health: "degraded" }),
];
expect(selectActiveIpsByMode(config, rows).sort()).toEqual([
"1.1.1.1",
"2.2.2.2",
]);
});
it("returns empty array for no rows", () => {
@@ -111,3 +187,115 @@ describe("selectActiveIpsByMode", () => {
expect(selectActiveIpsByMode(config, [])).toEqual([]);
});
});
describe("resolveDesiredAIps", () => {
it("keeps a dedicated single IP even when down", () => {
expect(
resolveDesiredAIps(
weightedConfig,
[row("1.1.1.1", { health: "down" })],
["1.1.1.1"],
),
).toEqual(["1.1.1.1"]);
});
it("does not drain when the service has only one unique IP", () => {
expect(
resolveDesiredAIps(
weightedConfig,
[row("1.1.1.1", { health: "down" })],
["1.1.1.1", "1.1.1.1"],
0,
["1.1.1.1"],
),
).toEqual(["1.1.1.1", "1.1.1.1"]);
});
it("does not apply overlay when service pool is a single IP", () => {
const rows = [
row("1.1.1.1", { health: "up" }),
row("2.2.2.2", { health: "down" }),
];
expect(
resolveDesiredAIps(
weightedConfig,
rows,
["1.1.1.1", "2.2.2.2"],
0,
["1.1.1.1"],
),
).toEqual(["1.1.1.1", "2.2.2.2"]);
});
it("applies weighted overlay on a shared pool", () => {
const rows = [
row("1.1.1.1", { weight: 1, health: "up" }),
row("2.2.2.2", { weight: 3, health: "up" }),
];
expect(
resolveDesiredAIps(weightedConfig, rows, ["1.1.1.1", "2.2.2.2"], 0),
).toEqual(selectActiveIpsByMode(weightedConfig, rows, 0));
});
});
describe("shouldRecordFailoverDnsDiff", () => {
it("skips duplicate listings of the same IP", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "1.1.1.1"],
lbMode: "failover",
added: [],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(false);
});
it("skips dedicated extra-FQDN", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1"],
lbMode: "failover",
added: [],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(false);
});
it("skips weighted live-to-live slot swap", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "2.2.2.2"],
lbMode: "weighted",
added: ["2.2.2.2"],
removed: ["1.1.1.1"],
downIps: new Set(),
}),
).toBe(false);
});
it("logs weighted swap when a down ip leaves the pool", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "2.2.2.2"],
lbMode: "weighted",
added: ["2.2.2.2"],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(true);
});
it("logs failover diffs on a shared pool", () => {
expect(
shouldRecordFailoverDnsDiff({
configuredIps: ["1.1.1.1", "2.2.2.2"],
lbMode: "failover",
added: ["2.2.2.2"],
removed: ["1.1.1.1"],
downIps: new Set(["1.1.1.1"]),
}),
).toBe(true);
});
});
+192 -28
View File
@@ -6,6 +6,7 @@ import { buildApp } from "../src/app.js";
import { loadConfig } from "../src/config.js";
import {
listGroupViews,
toggleServiceIp,
updateConfig,
} from "../src/services/service-config-service.js";
@@ -250,7 +251,7 @@ describe("create service then list groups", () => {
await app.close();
});
it("PATCH /services/:id/ips/toggle keeps IP in pool and removes it from A-binding", async () => {
it("PATCH /services/:id/ips/toggle keeps IP in pool and in A-binding", async () => {
const app = await buildApp({
config: { ...loadConfig(), staticDir: null },
memory: true,
@@ -280,6 +281,18 @@ describe("create service then list groups", () => {
expect(createRes.statusCode).toBe(200);
const created = createRes.json() as { id: number };
const domainPayload = {
lb_mode: "round_robin" as const,
health_check_enabled: false,
health_check_type: "tcp" as const,
health_check_port: 443,
health_check_path: null,
health_check_expected_status: null,
health_check_interval_sec: 30,
health_check_timeout_ms: 3000,
health_check_verify_tls: false,
};
await updateConfig(app.db, cf, created.id, {
ips: ["1.2.3.4", "5.6.7.8"],
service_group_id: group.id,
@@ -289,23 +302,81 @@ describe("create service then list groups", () => {
target_ips: ["1.2.3.4", "5.6.7.8"],
target_ip_weights: { "1.2.3.4": 1, "5.6.7.8": 1 },
target_ip_priorities: { "1.2.3.4": 1, "5.6.7.8": 1 },
lb_mode: "round_robin",
health_check_enabled: false,
health_check_type: "tcp",
health_check_port: 443,
health_check_path: null,
health_check_expected_status: null,
health_check_interval_sec: 30,
health_check_timeout_ms: 3000,
health_check_verify_tls: false,
...domainPayload,
},
{
fqdn: "extra.example.com",
target_ips: ["1.2.3.4"],
target_ip_weights: { "1.2.3.4": 1 },
target_ip_priorities: { "1.2.3.4": 1 },
...domainPayload,
},
],
});
// HTTP toggle uses request.server.cf; disable DNS push so the test
// does not call the real Cloudflare client.
repos.setServiceEnabled(app.db, created.id, false);
const commonBinding = repos
.listBindingsByService(app.db, created.id)
.find((b) => b.hostname === "panel")!;
const extraBinding = repos
.listBindingsByService(app.db, created.id)
.find((b) => b.hostname === "extra")!;
expect(repos.listBindingIps(app.db, commonBinding.id)).toEqual(
expect.arrayContaining(["1.2.3.4", "5.6.7.8"]),
);
expect(
repos
.listRecordsForBinding(app.db, commonBinding.id)
.map((r) => r.content)
.sort(),
).toEqual(["1.2.3.4", "5.6.7.8"]);
// Direct service call with mock CF — keep HTTP path free of real Cloudflare.
await toggleServiceIp(app.db, cf, created.id, "1.2.3.4", false);
expect(repos.listServiceIps(app.db, created.id)).toEqual(
expect.arrayContaining(["1.2.3.4", "5.6.7.8"]),
);
expect(
repos.listServiceIpRows(app.db, created.id).find((r) => r.ip === "1.2.3.4")
?.enabled,
).toBe(false);
// Common + per-IP bindings keep configured IPs (UI hydrate stays stable).
expect(repos.listBindingIps(app.db, commonBinding.id)).toEqual(
expect.arrayContaining(["1.2.3.4", "5.6.7.8"]),
);
expect(repos.listBindingIps(app.db, extraBinding.id)).toEqual(["1.2.3.4"]);
// DNS for common FQDN drops the disabled IP only.
expect(
repos
.listRecordsForBinding(app.db, commonBinding.id)
.map((r) => r.content)
.sort(),
).toEqual(["5.6.7.8"]);
// Per-IP extra FQDN has no enabled targets → A records removed.
expect(repos.listRecordsForBinding(app.db, extraBinding.id)).toEqual([]);
await toggleServiceIp(app.db, cf, created.id, "1.2.3.4", true);
expect(
repos.listServiceIpRows(app.db, created.id).find((r) => r.ip === "1.2.3.4")
?.enabled,
).toBe(true);
expect(repos.listBindingIps(app.db, commonBinding.id)).toEqual(
expect.arrayContaining(["1.2.3.4", "5.6.7.8"]),
);
expect(
repos
.listRecordsForBinding(app.db, commonBinding.id)
.map((r) => r.content)
.sort(),
).toEqual(["1.2.3.4", "5.6.7.8"]);
expect(
repos
.listRecordsForBinding(app.db, extraBinding.id)
.map((r) => r.content),
).toEqual(["1.2.3.4"]);
// HTTP toggle still updates ip_enabled without mutating bindings.
repos.setServiceEnabled(app.db, created.id, false);
const offRes = await app.inject({
method: "PATCH",
url: `/api/v1/services/${created.id}/ips/toggle`,
@@ -319,24 +390,117 @@ describe("create service then list groups", () => {
};
expect(offView.ips).toEqual(expect.arrayContaining(["1.2.3.4", "5.6.7.8"]));
expect(offView.ip_enabled["1.2.3.4"]).toBe(false);
expect(offView.ip_enabled["5.6.7.8"]).toBe(true);
const binding = repos.listBindingsByService(app.db, created.id)[0]!;
expect(repos.listBindingIps(app.db, binding.id)).toEqual(["5.6.7.8"]);
const onRes = await app.inject({
method: "PATCH",
url: `/api/v1/services/${created.id}/ips/toggle`,
headers,
payload: { ip: "1.2.3.4", enabled: true },
});
expect(onRes.statusCode).toBe(200);
const onView = onRes.json() as { ip_enabled: Record<string, boolean> };
expect(onView.ip_enabled["1.2.3.4"]).toBe(true);
expect(repos.listBindingIps(app.db, binding.id)).toEqual(
expect(repos.listBindingIps(app.db, commonBinding.id)).toEqual(
expect.arrayContaining(["1.2.3.4", "5.6.7.8"]),
);
await app.close();
});
it("GET /services/:id repairs multi-IP bindings shrunk below the pool", async () => {
const app = await buildApp({
config: { ...loadConfig(), staticDir: null },
memory: true,
});
const headers = await authHeaders(app);
const cf = mockCf();
repos.createDomain(app.db, null, "example.com", "zone-1");
const group = repos.createServiceGroup(
app.db,
"VPN",
"vpn-repair",
null,
"vpn.example.com",
);
const createRes = await app.inject({
method: "POST",
url: "/api/v1/services",
headers,
payload: {
name: "Repair",
slug: "panel-ip-repair",
service_group_id: group.id,
},
});
expect(createRes.statusCode).toBe(200);
const created = createRes.json() as { id: number };
await updateConfig(app.db, cf, created.id, {
ips: ["1.2.3.4", "5.6.7.8", "9.9.9.9"],
service_group_id: group.id,
domains: [
{
fqdn: "gw.example.com",
target_ips: ["1.2.3.4", "5.6.7.8", "9.9.9.9"],
target_ip_weights: { "1.2.3.4": 1, "5.6.7.8": 1, "9.9.9.9": 1 },
target_ip_priorities: { "1.2.3.4": 1, "5.6.7.8": 1, "9.9.9.9": 1 },
lb_mode: "round_robin",
health_check_enabled: false,
health_check_type: "tcp",
health_check_port: 443,
health_check_path: null,
health_check_expected_status: null,
health_check_interval_sec: 30,
health_check_timeout_ms: 3000,
health_check_verify_tls: false,
},
{
fqdn: "extra.example.com",
target_ips: ["1.2.3.4"],
target_ip_weights: { "1.2.3.4": 1 },
target_ip_priorities: { "1.2.3.4": 1 },
lb_mode: "round_robin",
health_check_enabled: false,
health_check_type: "tcp",
health_check_port: 443,
health_check_path: null,
health_check_expected_status: null,
health_check_interval_sec: 30,
health_check_timeout_ms: 3000,
health_check_verify_tls: false,
},
],
});
const commonBinding = repos
.listBindingsByService(app.db, created.id)
.find((b) => b.hostname === "gw")!;
const extraBinding = repos
.listBindingsByService(app.db, created.id)
.find((b) => b.hostname === "extra")!;
// Simulate legacy toggle damage: shrink common binding, leave extra alone.
repos.replaceBindingIpsWithMeta(app.db, commonBinding.id, [
{ ip: "1.2.3.4", weight: 1, priority: 1 },
{ ip: "5.6.7.8", weight: 1, priority: 1 },
]);
expect(repos.listBindingIps(app.db, commonBinding.id)).toEqual([
"1.2.3.4",
"5.6.7.8",
]);
const getRes = await app.inject({
method: "GET",
url: `/api/v1/services/${created.id}`,
headers,
});
expect(getRes.statusCode).toBe(200);
const view = getRes.json() as {
domains: Array<{ fqdn: string; target_ips: string[] }>;
};
const gw = view.domains.find((d) => d.fqdn === "gw.example.com");
const extra = view.domains.find((d) => d.fqdn === "extra.example.com");
expect(gw?.target_ips.sort()).toEqual(["1.2.3.4", "5.6.7.8", "9.9.9.9"]);
expect(extra?.target_ips).toEqual(["1.2.3.4"]);
expect(repos.listBindingIps(app.db, commonBinding.id).sort()).toEqual([
"1.2.3.4",
"5.6.7.8",
"9.9.9.9",
]);
expect(repos.listBindingIps(app.db, extraBinding.id)).toEqual(["1.2.3.4"]);
await app.close();
});
});
+77 -2
View File
@@ -1,6 +1,13 @@
import { describe, expect, it } from "vitest";
import type { ServiceBindingView } from "@cfdm/shared";
import { resolveBindingIpsForSync } from "../src/services/vps-tracker-sync.js";
import {
cfdmBindingSyncItemSchema,
type ServiceBindingView,
} from "@cfdm/shared";
import {
resolveBindingIpsForSync,
resolveLbModeForSync,
effectiveLbModeForSync,
} from "../src/services/vps-tracker-sync.js";
function binding(
partial: Partial<ServiceBindingView> &
@@ -160,3 +167,71 @@ describe("resolveBindingIpsForSync", () => {
expect(ips).toEqual(["203.0.113.55"]);
});
});
describe("resolveLbModeForSync", () => {
it("prefers binding lb_mode", () => {
expect(resolveLbModeForSync("failover", "round_robin")).toBe("failover");
});
it("falls back to service group lb_mode", () => {
expect(resolveLbModeForSync("off", "weighted")).toBe("weighted");
expect(resolveLbModeForSync(undefined, "round_robin")).toBe("round_robin");
});
it("returns undefined when neither is a known mode", () => {
expect(resolveLbModeForSync("off", "off")).toBeUndefined();
expect(resolveLbModeForSync(null, undefined)).toBeUndefined();
});
});
describe("effectiveLbModeForSync", () => {
it("omits mode when unique origin IPs are below two", () => {
expect(
effectiveLbModeForSync("round_robin", "failover", ["203.0.113.10"]),
).toBeUndefined();
expect(
effectiveLbModeForSync("round_robin", "failover", [
"203.0.113.10",
"203.0.113.10",
]),
).toBeUndefined();
});
it("emits configured mode when the service has a pool", () => {
expect(
effectiveLbModeForSync("failover", "round_robin", [
"203.0.113.10",
"203.0.113.20",
]),
).toBe("failover");
expect(
effectiveLbModeForSync("off", "weighted", [
"203.0.113.10",
"198.51.100.1",
]),
).toBe("weighted");
});
});
describe("cfdmBindingSyncItemSchema lbMode", () => {
const base = {
bindingId: 1,
serviceId: 10,
serviceName: "VPN",
serviceSlug: "vpn",
fqdn: "vpn.example.com",
zoneName: "example.com",
hostname: "vpn",
ips: ["203.0.113.10"],
};
it("accepts optional lbMode on sync payload", () => {
const parsed = cfdmBindingSyncItemSchema.parse({ ...base, lbMode: "failover" });
expect(parsed.lbMode).toBe("failover");
});
it("accepts payload without lbMode (legacy)", () => {
const parsed = cfdmBindingSyncItemSchema.parse(base);
expect(parsed.lbMode).toBeUndefined();
});
});
+105 -60
View File
@@ -13,7 +13,11 @@ import {
import { HealthCheckBadge } from '@/components/health-check-badge'
import { EmptyState } from '@/components/empty-state'
import { formatDate, formatRelative, sqliteUtcToIso } from '@/lib/format'
import type { FailoverEvent } from '@/lib/failover-events'
import {
failoverEventCopy,
type FailoverEvent,
type FailoverHistoryItem,
} from '@/lib/failover-events'
import { cn } from '@cfdm/ui/lib/utils'
import type { ComponentProps } from 'react'
@@ -29,32 +33,38 @@ function failStreakLabel(count: number): string {
return `${count} ошибок подряд`
}
function indicatorClass(status: string): string {
if (status === 'checking') {
return 'border-warning bg-warning/15 group-data-completed/timeline-item:border-warning'
function indicatorClass(tone: 'down' | 'added' | 'removed'): string {
if (tone === 'added') {
return 'border-success bg-success/15 group-data-completed/timeline-item:border-success'
}
return 'border-destructive bg-destructive/15 group-data-completed/timeline-item:border-destructive'
}
function separatorClass(status: string): string {
if (status === 'checking') return 'bg-warning/25'
function separatorClass(tone: 'down' | 'added' | 'removed'): string {
if (tone === 'added') return 'bg-success/25'
return 'bg-destructive/25'
}
/**
* Failover как sibling «Смены статуса»: ReUI Timeline + Badge, не степпер.
* Failover как sibling «Смены статуса»: ReUI Timeline + Badge.
* Preview: https://reui.io/preview/base/components/c-timeline-10
* Preview: https://reui.io/preview/base/empty-state-12
* Docs: https://reui.io/docs/components/base/timeline
* Docs: https://reui.io/docs/components/base/badge
*/
export function FailoverTimeline({ events }: { events: FailoverEvent[] }) {
if (events.length === 0) {
export function FailoverTimeline({
events,
history = [],
}: {
events: FailoverEvent[]
history?: readonly FailoverHistoryItem[]
}) {
if (events.length === 0 && history.length === 0) {
return (
<EmptyState
icon={ShieldCheckIcon}
title="Пул в DNS на месте"
description="Standby и last-resort не красные — только down, снятые с A-записей"
title="Нет инцидентов"
description="Нет Down и нет выходов из общего пула"
stackedIcon={false}
centered={false}
/>
@@ -62,55 +72,90 @@ export function FailoverTimeline({ events }: { events: FailoverEvent[] }) {
}
return (
<Timeline defaultValue={0} className="gap-0">
{events.map((event, index) => {
const checkedIso = event.lastCheckAt
? (sqliteUtcToIso(event.lastCheckAt) ?? event.lastCheckAt)
: null
const isChecking = event.status === 'checking'
<div className="flex flex-col gap-4">
{events.length > 0 ? (
<Timeline defaultValue={0} className="gap-0">
{events.map((event, index) => {
const checkedIso = event.lastCheckAt
? (sqliteUtcToIso(event.lastCheckAt) ?? event.lastCheckAt)
: null
return (
<TimelineItem key={event.id} step={index + 1}>
<TimelineSeparator className={separatorClass(event.status)} />
<TimelineIndicator className={indicatorClass(event.status)} />
<TimelineHeader>
<TimelineTitle className="flex flex-wrap items-center gap-2">
<span className="font-mono text-sm">{event.address}</span>
<HealthCheckBadge
status={event.status as HealthBadgeStatus}
lastError={event.lastFailureReason}
lastCheckedAt={checkedIso}
size="xs"
/>
</TimelineTitle>
<TimelineDate>
{event.consecutiveFailures > 0
? failStreakLabel(event.consecutiveFailures)
: null}
{checkedIso
? `${event.consecutiveFailures > 0 ? ' · ' : ''}${formatRelative(checkedIso)} · ${formatDate(checkedIso)}`
: null}
</TimelineDate>
</TimelineHeader>
<TimelineContent className="flex flex-col gap-2">
<p className="text-foreground text-sm">
{isChecking
? 'Снята с DNS, идёт восстановление'
: 'Снята с DNS — в A-записях этого адреса нет'}
</p>
{event.lastFailureReason ? (
<code
className={cn(
'bg-muted block overflow-x-auto rounded-md px-2 py-1.5 font-mono text-xs',
)}
>
{event.lastFailureReason}
</code>
) : null}
</TimelineContent>
</TimelineItem>
)
})}
</Timeline>
return (
<TimelineItem key={event.id} step={index + 1}>
<TimelineSeparator className={separatorClass('down')} />
<TimelineIndicator className={indicatorClass('down')} />
<TimelineHeader>
<TimelineTitle className="flex flex-wrap items-center gap-2">
<span className="font-mono text-sm">{event.address}</span>
<HealthCheckBadge
status={event.status as HealthBadgeStatus}
lastError={event.lastFailureReason}
lastCheckedAt={checkedIso}
size="xs"
/>
</TimelineTitle>
<TimelineDate>
{event.consecutiveFailures > 0
? failStreakLabel(event.consecutiveFailures)
: null}
{checkedIso
? `${event.consecutiveFailures > 0 ? ' · ' : ''}${formatRelative(checkedIso)} · ${formatDate(checkedIso)}`
: null}
</TimelineDate>
</TimelineHeader>
<TimelineContent className="flex flex-col gap-2">
<p className="text-foreground text-sm">
{failoverEventCopy(event)}
</p>
{event.lastFailureReason ? (
<code
className={cn(
'bg-muted block overflow-x-auto rounded-md px-2 py-1.5 font-mono text-xs',
)}
>
{event.lastFailureReason}
</code>
) : null}
</TimelineContent>
</TimelineItem>
)
})}
</Timeline>
) : null}
{history.length > 0 ? (
<Timeline defaultValue={0} className="gap-0">
{history.map((item, index) => {
const checkedIso = sqliteUtcToIso(item.created_at) ?? item.created_at
const tone = item.action === 'added' ? 'added' : 'removed'
return (
<TimelineItem key={item.id} step={index + 1}>
<TimelineSeparator className={separatorClass(tone)} />
<TimelineIndicator className={indicatorClass(tone)} />
<TimelineHeader>
<TimelineTitle className="flex flex-wrap items-center gap-2">
<span className="font-mono text-sm">{item.ip}</span>
<HealthCheckBadge
status={item.action === 'added' ? 'up' : 'down'}
size="xs"
/>
<span className="text-muted-foreground text-xs">
{item.fqdn}
</span>
</TimelineTitle>
<TimelineDate>
{formatRelative(checkedIso)} · {formatDate(checkedIso)}
</TimelineDate>
</TimelineHeader>
<TimelineContent>
<p className="text-foreground text-sm">{item.copy}</p>
</TimelineContent>
</TimelineItem>
)
})}
</Timeline>
) : null}
</div>
)
}
@@ -16,8 +16,8 @@ type HealthStatus =
function normalizeHealth(status: HealthStatus): IpHealthStatus['status'] {
if (status === 'healthy') return 'up'
if (status === 'unhealthy' || status === 'disabled') return 'down'
if (status === 'checking') return 'unknown'
if (status === 'unhealthy') return 'down'
if (status === 'disabled' || status === 'checking') return 'unknown'
return status
}
@@ -1,6 +1,7 @@
import { FormFieldSimple } from '@/components/form-field'
import { AppInput } from '@/components/app-input'
import { SettingRow } from '@/components/setting-row'
import { SelectField } from '@/components/select-field'
import { Badge } from '@/components/reui/badge'
import {
NumberField,
@@ -9,15 +10,8 @@ import {
NumberFieldIncrement,
NumberFieldInput,
} from '@/components/reui/number-field'
import {
Select,
SelectContent,
SelectItem,
SelectTrigger,
SelectValue,
} from '@cfdm/ui/components/select'
import { Switch } from '@cfdm/ui/components/switch'
import { FieldGroup } from '@cfdm/ui/components/field'
import { Field, FieldGroup, FieldLabel } from '@cfdm/ui/components/field'
import { Button } from '@cfdm/ui/components/button'
import { ButtonGroup } from '@cfdm/ui/components/button-group'
import { CableIcon, GlobeIcon } from 'lucide-react'
@@ -59,9 +53,14 @@ export interface LbAndHealthConfig extends HealthCheckConfig {
const defaultLbModeOptions = [
{ value: 'round_robin', label: 'Round Robin' },
{ value: 'failover', label: 'Failover (приоритет)' },
{ value: 'weighted', label: 'Weighted (веса)' },
{ value: 'weighted', label: 'Веса (подмена IP)' },
]
export interface LbPoolMetaChange {
weight?: number
priority?: number
}
function CompactNumberField({
id,
value,
@@ -95,6 +94,89 @@ function CompactNumberField({
)
}
function PoolLbMetaFields({
idPrefix,
mode,
ips,
weights,
priorities,
onMetaChange,
}: {
idPrefix: string
mode: Exclude<LbMode, 'round_robin'>
ips: readonly string[]
weights: Record<string, number>
priorities: Record<string, number>
onMetaChange?: (ip: string, meta: LbPoolMetaChange) => void
}) {
const isWeighted = mode === 'weighted'
const minPriority =
ips.length === 0
? 1
: Math.min(...ips.map((ip) => priorities[ip] ?? 1))
return (
<SettingRow
title={isWeighted ? 'Вес IP' : 'Приоритет IP'}
description={
isWeighted
? 'Доля времени на общем FQDN: 1 и 3 = ¼ и ¾ цикла (слот 60 с)'
: '1 — основной, больше — запасной'
}
compact
stacked
className="gap-3 px-0 py-3"
contentClassName="min-w-0"
>
{ips.length === 0 ? (
<p className="text-muted-foreground text-sm">Сначала добавьте IP выше</p>
) : (
<div className="flex w-full flex-col gap-2">
{ips.map((ip) => {
const isPrimary = (priorities[ip] ?? 1) === minPriority
const fieldId = isWeighted
? `${idPrefix}-weight-${ip}`
: `${idPrefix}-priority-${ip}`
return (
<div key={ip} className="flex items-center gap-3">
<span className="min-w-0 flex-1 truncate font-mono text-sm">{ip}</span>
{!isWeighted ? (
<Badge
variant={isPrimary ? 'success-light' : 'outline'}
size="xs"
radius="full"
>
{isPrimary ? 'Основной' : 'Запасной'}
</Badge>
) : null}
<Field className="w-28 gap-0">
<FieldLabel htmlFor={fieldId} className="sr-only">
{isWeighted ? `Вес ${ip}` : `Приоритет ${ip}`}
</FieldLabel>
<CompactNumberField
id={fieldId}
value={isWeighted ? (weights[ip] ?? 1) : (priorities[ip] ?? 1)}
min={1}
max={100}
onValueChange={(next) =>
onMetaChange?.(
ip,
isWeighted
? { weight: next ?? 1 }
: { priority: next ?? 1 },
)
}
/>
</Field>
</div>
)
})}
</div>
)}
</SettingRow>
)
}
export function HealthCheckConfigFields({
value,
onChange,
@@ -102,6 +184,10 @@ export function HealthCheckConfigFields({
lbModeOptions = defaultLbModeOptions,
idPrefix = 'health',
showLbMode = true,
ips = [],
weights = {},
priorities = {},
onMetaChange,
className,
}: {
value: LbAndHealthConfig
@@ -110,6 +196,10 @@ export function HealthCheckConfigFields({
lbModeOptions?: { value: string; label: string }[]
idPrefix?: string
showLbMode?: boolean
ips?: readonly string[]
weights?: Record<string, number>
priorities?: Record<string, number>
onMetaChange?: (ip: string, meta: LbPoolMetaChange) => void
className?: string
}) {
function patch(next: Partial<LbAndHealthConfig>) {
@@ -134,25 +224,28 @@ export function HealthCheckConfigFields({
compact
className={rowClass}
>
<Select
<SelectField
modal={false}
value={value.lb_mode}
onValueChange={(v) => patch({ lb_mode: (v ?? 'round_robin') as LbMode })}
>
<SelectTrigger id={`${idPrefix}-lb-mode`} className="w-full">
<SelectValue placeholder="Выберите режим" />
</SelectTrigger>
<SelectContent>
{lbModeOptions.map((item) => (
<SelectItem key={item.value} value={item.value}>
{item.label}
</SelectItem>
))}
</SelectContent>
</Select>
triggerId={`${idPrefix}-lb-mode`}
placeholder="Выберите режим"
options={lbModeOptions}
/>
</SettingRow>
) : null}
{showLbMode && value.lb_mode !== 'round_robin' ? (
<PoolLbMetaFields
idPrefix={idPrefix}
mode={value.lb_mode}
ips={ips}
weights={weights}
priorities={priorities}
onMetaChange={onMetaChange}
/>
) : null}
<SettingRow
title="Провайдер health-check"
description="Кто пробирует цель. Можно выбрать несколько источников."
@@ -16,10 +16,13 @@ import { parseFqdn } from '@/lib/parse-fqdn'
import {
addAddressNode,
addCommonFqdn,
addExtraFqdn,
addressHasFqdn,
removeAddressNode,
removeCommonFqdn,
removeExtraFqdn,
updateCommonFqdn,
updateExtraFqdn,
type AddressBlockState,
} from '@/lib/service-address'
import { Button } from '@cfdm/ui/components/button'
@@ -67,7 +70,7 @@ function ZoneAddon({
}
/**
* Единый блок адресов сервиса: список общих FQDN на весь пул + IP с доп. доменом.
* Единый блок адресов сервиса: список общих FQDN на весь пул + IP с доп. доменами.
* Preview: https://reui.io/preview/base/settings-3
* Preview: https://reui.io/preview/base/list-9
* Preview: https://reui.io/preview/base/form-7
@@ -88,6 +91,8 @@ export function ServiceAddressBlock({
const [ipInvalid, setIpInvalid] = useState(false)
const [pendingFqdn, setPendingFqdn] = useState('')
const [fqdnInvalid, setFqdnInvalid] = useState(false)
const [pendingExtraByIp, setPendingExtraByIp] = useState<Record<string, string>>({})
const [extraInvalidByIp, setExtraInvalidByIp] = useState<Record<string, boolean>>({})
const pool = value.nodes.map((node) => node.ip)
const pendingIpTrimmed = pendingIp.trim()
@@ -141,13 +146,26 @@ export function ServiceAddressBlock({
}
}
function handleNodeFqdn(ip: string, extraFqdn: string) {
onChange({
...value,
nodes: value.nodes.map((node) =>
node.ip === ip ? { ...node, extraFqdn } : node,
),
})
function tryAddExtra(ip: string, raw: string) {
const trimmed = raw.trim()
if (!trimmed) {
setExtraInvalidByIp((current) => ({ ...current, [ip]: false }))
return
}
if (addressHasFqdn(value, trimmed)) {
setExtraInvalidByIp((current) => ({ ...current, [ip]: true }))
return
}
onChange(addExtraFqdn(value, ip, trimmed))
setPendingExtraByIp((current) => ({ ...current, [ip]: '' }))
setExtraInvalidByIp((current) => ({ ...current, [ip]: false }))
}
function handleExtraKeyDown(ip: string, event: KeyboardEvent<HTMLInputElement>) {
if (event.key === 'Enter') {
event.preventDefault()
tryAddExtra(ip, pendingExtraByIp[ip] ?? '')
}
}
return (
@@ -156,7 +174,7 @@ export function ServiceAddressBlock({
<FrameHeader className="px-0 pt-0">
<FrameTitle>Адреса</FrameTitle>
<FrameDescription>
Общие FQDN на весь пул · у IP свой доп. домен
Общие FQDN на весь пул · у IP свои доп. домены
</FrameDescription>
</FrameHeader>
<Field>
@@ -224,7 +242,7 @@ export function ServiceAddressBlock({
<EmptyState
icon={ServerIcon}
title="Добавьте IP пула"
description="IPv4 сервиса. Для каждого адреса можно указать доп. FQDN."
description="IPv4 сервиса. Для каждого адреса можно указать несколько доп. FQDN, в том числе wildcard."
stackedIcon={false}
centered={false}
/>
@@ -265,27 +283,98 @@ export function ServiceAddressBlock({
</div>
<Field className="gap-1.5">
<FieldLabel
htmlFor={`service-ip-extra-${node.ip}`}
htmlFor={`service-ip-extra-add-${node.ip}`}
className="text-muted-foreground text-xs"
>
Доп. FQDN
</FieldLabel>
<InputGroup>
<InputGroupInput
id={`service-ip-extra-${node.ip}`}
className="font-mono"
value={node.extraFqdn}
placeholder={
zoneHints[0]
? `необязательно · spb.${zoneHints[0]}`
: 'необязательно · spb.example.com'
}
onChange={(event) =>
handleNodeFqdn(node.ip, event.target.value)
}
/>
<ZoneAddon fqdn={node.extraFqdn} zoneHints={zoneHints} />
</InputGroup>
<div className="flex w-full flex-col gap-2">
{node.extraFqdns.map((fqdn, index) => (
<InputGroup key={`extra-fqdn-${node.ip}-${index}`}>
<InputGroupInput
id={`service-ip-extra-${node.ip}-${index}`}
className="font-mono"
value={fqdn}
placeholder={
zoneHints[0]
? `*.mdns.${zoneHints[0]}`
: '*.mdns.example.com'
}
onChange={(event) =>
onChange(
updateExtraFqdn(
value,
node.ip,
index,
event.target.value,
),
)
}
/>
<ZoneAddon
fqdn={fqdn}
zoneHints={zoneHints}
trailing={
<InputGroupButton
size="icon-xs"
aria-label={`Удалить ${fqdn || 'FQDN'}`}
onClick={() =>
onChange(removeExtraFqdn(value, node.ip, index))
}
>
<Trash2Icon />
</InputGroupButton>
}
/>
</InputGroup>
))}
<InputGroup>
<InputGroupInput
id={`service-ip-extra-add-${node.ip}`}
className="font-mono"
value={pendingExtraByIp[node.ip] ?? ''}
placeholder={
zoneHints[0]
? `необязательно · *.mdns.${zoneHints[0]}`
: 'необязательно · *.mdns.example.com'
}
aria-invalid={
extraInvalidByIp[node.ip] &&
(pendingExtraByIp[node.ip] ?? '').trim().length > 0
? true
: undefined
}
onChange={(event) => {
setPendingExtraByIp((current) => ({
...current,
[node.ip]: event.target.value,
}))
setExtraInvalidByIp((current) => ({
...current,
[node.ip]: false,
}))
}}
onKeyDown={(event) => handleExtraKeyDown(node.ip, event)}
onBlur={() =>
tryAddExtra(node.ip, pendingExtraByIp[node.ip] ?? '')
}
/>
<ZoneAddon
fqdn={pendingExtraByIp[node.ip] ?? ''}
zoneHints={zoneHints}
trailing={
<InputGroupButton
size="sm"
onClick={() =>
tryAddExtra(node.ip, pendingExtraByIp[node.ip] ?? '')
}
>
Добавить
</InputGroupButton>
}
/>
</InputGroup>
</div>
</Field>
</ItemContent>
</Item>
@@ -2,9 +2,18 @@ import { UnplugIcon } from 'lucide-react'
import { FailoverTimeline } from '@/components/failover-timeline'
import {
mergeFailoverHistory,
toFailoverEvents,
type FailoverBindingPool,
type FailoverHealthInput,
} from '@/lib/failover-events'
import {
latestHealthByIp,
resolveIpDisplayHealth,
type HealthLogProbe,
type HealthLogStatus,
} from '@/lib/health-log'
import type { FailoverLogEntry, ServiceView } from '@/lib/schemas'
import { Badge } from '@/components/reui/badge'
import {
Frame,
@@ -19,18 +28,44 @@ import {
AlertTitle,
} from '@/components/reui/alert'
type LbMode = ServiceView['lb_mode']
const PANEL_COPY: Record<LbMode, { title: string; description: string }> = {
failover: {
title: 'Failover (приоритет)',
description: 'Down снимается с общего FQDN. История: кто вышел из пула и кто вернулся',
},
weighted: {
title: 'Веса (подмена IP)',
description: 'На общем FQDN один IP по весам. Down выводится из цикла',
},
round_robin: {
title: 'Round Robin',
description: 'На общем FQDN все живые A. Down снимается с пула',
},
}
function failoverCountLabel(count: number): string {
const mod10 = count % 10
const mod100 = count % 100
if (mod10 === 1 && mod100 !== 11) return `${count} нода вне пула`
if (mod10 === 1 && mod100 !== 11) return `${count} адрес Down`
if (mod10 >= 2 && mod10 <= 4 && (mod100 < 12 || mod100 > 14)) {
return `${count} ноды вне пула`
return `${count} адреса Down`
}
return `${count} нод вне пула`
return `${count} адресов Down`
}
function alertDescription(events: { kind: string; fqdns: string[] }[]): string {
const removedCount = events.filter((event) => event.kind === 'removed').length
if (removedCount > 0) return 'Сняты с общего FQDN'
if (events.some((event) => event.fqdns.length > 0)) {
return 'Остались в A-записях общего FQDN как last-resort'
}
return 'Down: персональные FQDN не меняются'
}
/**
* Failover — sibling ServiceHealthMonitor: Frame stacked + Alert + Timeline.
* Балансировка — текущие Down + кто вышел из общего пула.
* Preview: https://reui.io/preview/base/components/c-timeline-10
* Preview: https://reui.io/preview/base/empty-state-12
* Docs: https://reui.io/docs/components/base/frame
@@ -39,20 +74,52 @@ function failoverCountLabel(count: number): string {
* Docs: https://reui.io/docs/components/base/alert
*/
export function ServiceFailoverPanel({
lbMode = 'round_robin',
hasPool = true,
ipHealth,
activeAddresses,
bindings,
history,
probes = [],
}: {
lbMode?: LbMode
hasPool?: boolean
ipHealth: readonly FailoverHealthInput[]
activeAddresses: readonly string[]
bindings: readonly FailoverBindingPool[]
history: readonly FailoverLogEntry[]
probes?: readonly HealthLogProbe[]
}) {
const events = toFailoverEvents(ipHealth, activeAddresses)
const copy = hasPool
? (PANEL_COPY[lbMode] ?? PANEL_COPY.round_robin)
: {
title: 'без резервирования',
description: 'Один origin IP — балансировка не применяется',
}
const liveByIp = latestHealthByIp(probes)
const overlayHealth = ipHealth.map((row) => {
const live = liveByIp.get(row.ip)
return {
...row,
status: resolveIpDisplayHealth(
row.status as HealthLogStatus,
live?.status,
),
last_error:
live && live.status !== 'unknown' ? live.last_error : row.last_error,
last_checked_at:
live && live.status !== 'unknown'
? live.last_checked_at
: row.last_checked_at,
}
})
const events = toFailoverEvents(overlayHealth, bindings)
const mergedHistory = mergeFailoverHistory(history, probes, bindings)
return (
<Frame stacked spacing="sm" className="min-w-0 w-full">
<FramePanel className="flex flex-col gap-3">
<FrameHeader className="gap-1 px-0 py-0">
<FrameTitle className="flex flex-wrap items-center gap-2">
Failover
{copy.title}
{events.length > 0 ? (
<Badge variant="destructive-light" size="xs" radius="full">
{events.length}
@@ -63,22 +130,18 @@ export function ServiceFailoverPanel({
</Badge>
)}
</FrameTitle>
<FrameDescription>
Только down из IP Health вне DNS-пула · как таблица активов
</FrameDescription>
<FrameDescription>{copy.description}</FrameDescription>
</FrameHeader>
{events.length > 0 ? (
<Alert variant="destructive">
<UnplugIcon aria-hidden="true" />
<AlertTitle>{failoverCountLabel(events.length)}</AlertTitle>
<AlertDescription>
Эти IP сняты с A-записей
</AlertDescription>
<AlertDescription>{alertDescription(events)}</AlertDescription>
</Alert>
) : null}
<FailoverTimeline events={events} />
<FailoverTimeline events={events} history={mergedHistory} />
</FramePanel>
</Frame>
)
@@ -45,6 +45,28 @@ describe('toAlignedSeries', () => {
expect(keys).toEqual(['local', 'cloudflare', 'globalping'])
})
it('keeps live latency when another IP in the same bucket is down', () => {
const { points } = toAlignedSeries([
probe({
id: 1,
latency_ms: 18,
checked_at: '2026-01-01T00:00:10.000Z',
}),
probe({
id: 2,
status: 'down',
ok: false,
latency_ms: null,
checked_at: '2026-01-01T00:00:12.000Z',
}),
])
expect(points).toHaveLength(1)
expect(points[0]?.local).toBe(18)
expect(points[0]?.localOk).toBe(true)
expect(points[0]?.ok).toBe(true)
})
it('does not plot down probes as latency 0', () => {
const { points } = toAlignedSeries([
probe({
@@ -124,8 +124,13 @@ export function toAlignedSeries(items: UptimeProbe[]): {
}
const ok = probeOk(item)
row[`${key}Ok`] = ok
row[key] = ok ? item.latency_ms : null
const prevOk = row[`${key}Ok`]
row[`${key}Ok`] = prevOk === true || ok
if (ok && item.latency_ms != null) {
row[key] = item.latency_ms
} else if (row[key] === undefined) {
row[key] = null
}
}
const points = [...buckets.entries()]
@@ -17,6 +17,7 @@ import {
DEFAULT_BINDING_HEALTH,
emptyAddressBlock,
hydrateAddressBlock,
patchAddressIpMeta,
toBindingDrafts,
toDomainsPayload,
type AddressBlockState,
@@ -218,8 +219,8 @@ export function ServiceEditSheet({
<SheetHeader className="shrink-0 border-b pb-4">
<SheetTitle>{isCreate ? 'Новый сервис' : 'Редактирование сервиса'}</SheetTitle>
<SheetDescription>
Общие FQDN на весь пул IP. У каждого адреса можно указать свой доп.
FQDN.
Общие FQDN на весь пул IP. У каждого адреса можно указать несколько доп.
FQDN, в том числе wildcard.
</SheetDescription>
</SheetHeader>
@@ -282,6 +283,12 @@ export function ServiceEditSheet({
idPrefix="service-health"
value={primaryHealthValue}
onChange={handlePrimaryHealthChange}
ips={address.nodes.map((node) => node.ip)}
weights={address.target_ip_weights}
priorities={address.target_ip_priorities}
onMetaChange={(ip, meta) =>
setAddress((current) => patchAddressIpMeta(current, ip, meta))
}
/>
</section>
</div>
@@ -18,6 +18,11 @@ import { DataGridColumnHeader } from '@/components/reui/data-grid/data-grid-colu
import { IconTile } from '@/components/reui/icon-tile'
import { createFilter, type Filter, type FilterFieldConfig } from '@/components/reui/filters'
import { ResourcePage } from '@/components/reui-kit'
import {
latestHealthByIp,
resolveIpDisplayHealth,
type HealthLogProbe,
} from '@/lib/health-log'
import { certRelativeBadge } from '@/components/columns/certificates-columns'
import { certMonitoringOptions } from '@/lib/cert-monitoring'
import { formatDate } from '@/lib/format'
@@ -99,8 +104,12 @@ function mapNodeHealth(status: string): HealthStatus {
return 'unknown'
}
function buildIpRows(service: ServiceView): ServiceIpRow[] {
function buildIpRows(
service: ServiceView,
probes: readonly HealthLogProbe[] = [],
): ServiceIpRow[] {
const healthByIp = new Map(service.ip_health.map((row) => [row.ip, row]))
const liveByIp = latestHealthByIp(probes)
const weights = Object.assign(
{},
...service.domains.map((domain) => domain.target_ip_weights ?? {}),
@@ -113,19 +122,25 @@ function buildIpRows(service: ServiceView): ServiceIpRow[] {
return service.ips.map((ip) => {
const health = healthByIp.get(ip)
const live = liveByIp.get(ip)
const status = resolveIpDisplayHealth(health?.status, live?.status)
const extras = live && live.status !== 'unknown' ? live : health
return {
id: ip,
ip,
status: health?.status ?? 'unknown',
status,
enabled: service.ip_enabled[ip] !== false,
active: activeSet.has(ip),
weight: weights[ip] ?? 1,
priority: priorities[ip] ?? 1,
latency_ms: health?.latency_ms ?? null,
last_checked_at: health?.last_checked_at ?? null,
last_error: health?.last_error ?? null,
colo: health?.colo ?? null,
provider: health?.provider ?? null,
latency_ms: extras?.latency_ms ?? null,
last_checked_at: extras?.last_checked_at ?? null,
last_error:
live && live.status !== 'unknown'
? live.last_error
: (health?.last_error ?? null),
colo: extras?.colo ?? null,
provider: extras?.provider ?? null,
}
})
}
@@ -151,17 +166,23 @@ function buildNodeRows(
weight: number
priority: number
}>,
probes: readonly HealthLogProbe[] = [],
): ServiceNodeRow[] {
return nodes.map((node) => ({
id: String(node.id),
nodeId: node.id,
address: node.address,
protocol: node.protocol,
port: node.port,
health_status: mapNodeHealth(node.health_status),
weight: node.weight,
priority: node.priority,
}))
const liveByIp = latestHealthByIp(probes)
return nodes.map((node) => {
const stored = mapNodeHealth(node.health_status)
const live = liveByIp.get(node.address)
return {
id: String(node.id),
nodeId: node.id,
address: node.address,
protocol: node.protocol,
port: node.port,
health_status: resolveIpDisplayHealth(stored, live?.status),
weight: node.weight,
priority: node.priority,
}
})
}
function NameCell({
@@ -199,6 +220,7 @@ interface ServiceDetailGridProps {
weight: number
priority: number
}>
probes?: readonly HealthLogProbe[]
togglingIp: string | null
onToggleIp: (ip: string, enabled: boolean) => void
onChangeIp: (row: ServiceFqdnRow) => void
@@ -211,6 +233,7 @@ interface ServiceDetailGridProps {
export function ServiceDetailGrid({
service,
nodes,
probes = [],
togglingIp,
onToggleIp,
onChangeIp,
@@ -283,9 +306,9 @@ export function ServiceDetailGrid({
},
})
const ipRows = useMemo(() => buildIpRows(service), [service])
const ipRows = useMemo(() => buildIpRows(service, probes), [service, probes])
const fqdnRows = useMemo(() => buildFqdnRows(service), [service])
const nodeRows = useMemo(() => buildNodeRows(nodes), [nodes])
const nodeRows = useMemo(() => buildNodeRows(nodes, probes), [nodes, probes])
const markActive = service.lb_mode === 'failover' || service.lb_mode === 'weighted'
const tabs = TABS.map((entry) => ({
@@ -133,6 +133,7 @@ const VISIBLE_IP_LIMIT = 6
interface ServiceIpListProps {
ips: string[]
ipHealth?: ServiceView['ip_health']
healthCheckEnabled?: boolean
ipEnabled?: Record<string, boolean>
togglingIp?: string | null
ipToggleDisabled?: boolean
@@ -151,6 +152,7 @@ interface ServiceIpListProps {
export function ServiceIpList({
ips,
ipHealth = [],
healthCheckEnabled = true,
ipEnabled = {},
togglingIp = null,
ipToggleDisabled = false,
@@ -184,6 +186,10 @@ export function ServiceIpList({
{visible.map((ip) => {
const health = healthByIp.get(ip)
const enabled = ipEnabled[ip] !== false
const monitored = healthCheckEnabled && enabled
const badgeStatus = monitored
? (health?.status ?? 'unknown')
: 'disabled'
return (
<Item
key={ip}
@@ -192,7 +198,7 @@ export function ServiceIpList({
>
<ItemMedia>
<HealthCheckBadge
status={health?.status ?? 'unknown'}
status={badgeStatus}
latencyMs={health?.latency_ms}
lastCheckedAt={health?.last_checked_at}
lastError={health?.last_error}
@@ -5,6 +5,7 @@ import {
Repeat2Icon,
ScaleIcon,
ServerIcon,
UnplugIcon,
type LucideIcon,
} from 'lucide-react'
@@ -21,6 +22,7 @@ import {
ServiceIpList,
} from '@/components/services/service-fqdn-list'
import { serviceDisplayFqdn, serviceDisplayFqdns } from '@/lib/service-utils'
import { uniqueIpCount } from '@/lib/failover-events'
import type { ServiceView } from '@/lib/schemas'
import { Button } from '@cfdm/ui/components/button'
import {
@@ -71,11 +73,39 @@ const LB_MODE_META: Record<
weighted: {
icon: ScaleIcon,
className: 'text-info',
label: 'Weighted (веса)',
label: 'Веса (подмена IP)',
},
}
export function LbModeTile({ mode }: { mode: LbMode }) {
export function LbModeTile({
mode,
hasPool = true,
}: {
mode: LbMode
hasPool?: boolean
}) {
if (!hasPool) {
return (
<TooltipProvider>
<Tooltip>
<TooltipTrigger
render={
<IconTile
variant="elevated"
size="xs"
className="shrink-0 text-muted-foreground"
aria-label="без резервирования"
/>
}
>
<UnplugIcon aria-hidden="true" />
</TooltipTrigger>
<TooltipContent>без резервирования</TooltipContent>
</Tooltip>
</TooltipProvider>
)
}
const meta = LB_MODE_META[mode]
const Icon = meta.icon
@@ -148,7 +178,10 @@ export function ServiceUnitCard({
{service.name}
</Link>
</FrameTitle>
<LbModeTile mode={service.lb_mode} />
<LbModeTile
mode={service.lb_mode}
hasPool={uniqueIpCount(service.ips) >= 2}
/>
</div>
<div className="flex min-w-0 items-center gap-1">
<FrameDescription className="min-w-0 truncate font-mono text-xs">
@@ -240,6 +273,10 @@ export function ServiceUnitCard({
alignWithMenu
ips={service.ips ?? []}
ipHealth={service.ip_health ?? []}
healthCheckEnabled={
service.enabled &&
(service.domains ?? []).some((domain) => domain.health_check_enabled)
}
ipEnabled={service.ip_enabled ?? {}}
ipToggleDisabled={togglingId === service.id}
togglingIp={togglingIp}
+240 -9
View File
@@ -1,10 +1,16 @@
import { describe, expect, it } from 'vitest'
import {
failoverEventCopy,
isFailoverEventStatus,
mergeFailoverHistory,
toFailoverEvents,
toIpAliveTransitions,
type FailoverBindingPool,
type FailoverHealthInput,
} from '@/lib/failover-events'
import type { FailoverLogEntry } from '@/lib/schemas'
import type { HealthLogProbe } from '@/lib/health-log'
function row(
overrides: Partial<FailoverHealthInput> & Pick<FailoverHealthInput, 'ip'>,
@@ -18,6 +24,24 @@ function row(
}
}
const mskHip: FailoverBindingPool[] = [
{
fqdn: 'gt.rkns.top',
configured: ['130.49.213.153', '93.115.203.183'],
active: ['93.115.203.183'],
},
{
fqdn: 'nsgt.rkns.top',
configured: ['130.49.213.153'],
active: ['130.49.213.153'],
},
{
fqdn: 'rutg.rkns.top',
configured: ['93.115.203.183'],
active: ['93.115.203.183'],
},
]
describe('toFailoverEvents', () => {
it('инцидент только при binding down, не при unhealthy ноды', () => {
expect(isFailoverEventStatus('down')).toBe(true)
@@ -27,7 +51,28 @@ describe('toFailoverEvents', () => {
expect(isFailoverEventStatus('unhealthy')).toBe(false)
})
it('без DNS-пула ничего не показывает — нельзя врать про вывод', () => {
it('один IP у сервиса — не инцидент пула', () => {
const events = toFailoverEvents(
[
row({
ip: '10.0.0.1',
status: 'down',
consecutive_failures: 9,
last_error: 'fetch failed',
}),
],
[
{
fqdn: 'solo.example.com',
configured: ['10.0.0.1'],
active: [],
},
],
)
expect(events).toEqual([])
})
it('Down без shared pool не событие балансировки', () => {
const events = toFailoverEvents([
row({
ip: '130.49.213.153',
@@ -45,7 +90,13 @@ describe('toFailoverEvents', () => {
row({ ip: '10.0.0.1', status: 'up' }),
row({ ip: '130.49.213.153', status: 'up' }),
],
['10.0.0.1'],
[
{
fqdn: 'pool.example.com',
configured: ['10.0.0.1', '130.49.213.153'],
active: ['10.0.0.1'],
},
],
)
expect(events).toEqual([])
})
@@ -57,12 +108,18 @@ describe('toFailoverEvents', () => {
row({ ip: '10.0.0.2', status: 'unknown' }),
row({ ip: '10.0.0.3', status: 'degraded' }),
],
['10.0.0.1'],
[
{
fqdn: 'pool.example.com',
configured: ['10.0.0.1', '10.0.0.2', '10.0.0.3'],
active: ['10.0.0.1'],
},
],
)
expect(events).toEqual([])
})
it('down в DNS-пуле не инцидент (last-resort / ещё в A-записи)', () => {
it('Down на доп. FQDN last-resort, снятая с общего пула', () => {
const events = toFailoverEvents(
[
row({
@@ -70,15 +127,43 @@ describe('toFailoverEvents', () => {
status: 'down',
consecutive_failures: 9,
last_error: 'fetch failed',
last_checked_at: '2026-08-20 07:00:00',
}),
row({ ip: '10.0.0.3', status: 'down' }),
row({ ip: '93.115.203.183', status: 'up' }),
],
['130.49.213.153', '10.0.0.2'],
mskHip,
)
expect(events.map((event) => event.address)).toEqual(['10.0.0.3'])
expect(events).toEqual([
{
id: '130.49.213.153',
address: '130.49.213.153',
status: 'down',
kind: 'removed',
fqdns: ['gt.rkns.top'],
consecutiveFailures: 9,
lastFailureReason: 'fetch failed',
lastCheckAt: '2026-08-20 07:00:00',
},
])
expect(failoverEventCopy(events[0]!)).toBe('Снята с gt.rkns.top')
expect(events[0]?.fqdns).not.toContain('nsgt.rkns.top')
})
it('down без A-записи — реальный вывод из пула', () => {
it('Down только last-resort на своём FQDN', () => {
const events = toFailoverEvents(
[row({ ip: '130.49.213.153', status: 'down' })],
[
{
fqdn: 'nsgt.rkns.top',
configured: ['130.49.213.153'],
active: ['130.49.213.153'],
},
],
)
expect(events).toEqual([])
})
it('down без A-записи на configured FQDN — снятие', () => {
const events = toFailoverEvents(
[
row({
@@ -89,13 +174,21 @@ describe('toFailoverEvents', () => {
last_checked_at: '2026-08-20 07:00:00',
}),
],
['10.0.0.1'],
[
{
fqdn: 'pool.example.com',
configured: ['10.0.0.1', '130.49.213.153'],
active: ['10.0.0.1'],
},
],
)
expect(events).toEqual([
{
id: '130.49.213.153',
address: '130.49.213.153',
status: 'down',
kind: 'removed',
fqdns: ['pool.example.com'],
consecutiveFailures: 9,
lastFailureReason: 'fetch failed',
lastCheckAt: '2026-08-20 07:00:00',
@@ -103,3 +196,141 @@ describe('toFailoverEvents', () => {
])
})
})
function probe(
overrides: Partial<HealthLogProbe> & Pick<HealthLogProbe, 'id' | 'status' | 'checked_at'>,
): HealthLogProbe {
return {
ip: '130.49.213.153',
provider: 'local',
ok: overrides.status === 'up',
latency_ms: 12,
colo: null,
error: null,
...overrides,
}
}
function dns(
overrides: Partial<FailoverLogEntry> & Pick<FailoverLogEntry, 'id' | 'action' | 'created_at'>,
): FailoverLogEntry {
return {
service_id: 13,
binding_id: 1,
fqdn: 'gt.rkns.top',
ip: '130.49.213.153',
...overrides,
}
}
describe('toIpAliveTransitions', () => {
it('emits leave then return, not the initial up', () => {
const transitions = toIpAliveTransitions([
probe({ id: 1, status: 'up', checked_at: '2026-08-20T09:00:00Z' }),
probe({ id: 2, status: 'up', checked_at: '2026-08-20T09:01:00Z' }),
probe({ id: 3, status: 'down', checked_at: '2026-08-20T09:02:00Z' }),
probe({ id: 4, status: 'down', checked_at: '2026-08-20T09:03:00Z' }),
probe({ id: 5, status: 'up', checked_at: '2026-08-20T09:04:00Z' }),
])
expect(transitions).toEqual([
{ id: 'probe:3', ip: '130.49.213.153', alive: false, at: '2026-08-20T09:02:00Z' },
{ id: 'probe:5', ip: '130.49.213.153', alive: true, at: '2026-08-20T09:04:00Z' },
])
})
it('any-up across providers: leave only when every source is down', () => {
const transitions = toIpAliveTransitions([
probe({
id: 1,
provider: 'local',
status: 'up',
checked_at: '2026-08-20T09:00:00Z',
}),
probe({
id: 2,
provider: 'cloudflare',
status: 'down',
checked_at: '2026-08-20T09:01:00Z',
}),
probe({
id: 3,
provider: 'local',
status: 'down',
checked_at: '2026-08-20T09:02:00Z',
}),
probe({
id: 4,
provider: 'local',
status: 'up',
checked_at: '2026-08-20T09:03:00Z',
}),
])
expect(transitions.map((item) => item.id)).toEqual(['probe:3', 'probe:4'])
})
})
describe('mergeFailoverHistory', () => {
it('fills leave/return from probes when DNS has only the add', () => {
const history = mergeFailoverHistory(
[dns({ id: 10, action: 'added', created_at: '2026-08-20 09:26:00' })],
[
probe({ id: 1, status: 'up', checked_at: '2026-08-20T09:00:00Z' }),
probe({ id: 2, status: 'down', checked_at: '2026-08-20T09:10:00Z' }),
probe({ id: 3, status: 'up', checked_at: '2026-08-20T09:26:30Z' }),
],
mskHip,
)
expect(history.map((item) => `${item.action}:${item.source}`)).toEqual([
'added:dns',
'removed:probe',
])
expect(history[0]?.copy).toBe('130.49.213.153 добавлена на gt.rkns.top')
expect(history[1]?.copy).toBe(
'130.49.213.153 вышла из пула (gt.rkns.top)',
)
})
it('keeps DNS over a probe return in the same 2-minute window', () => {
const history = mergeFailoverHistory(
[dns({ id: 10, action: 'added', created_at: '2026-08-20 09:26:00' })],
[probe({ id: 3, status: 'up', checked_at: '2026-08-20T09:26:30Z' })],
mskHip,
)
expect(history).toHaveLength(1)
expect(history[0]?.source).toBe('dns')
})
it('drops probe leave/return when the service has no shared pool', () => {
const history = mergeFailoverHistory(
[],
[
probe({ id: 1, status: 'up', checked_at: '2026-08-20T09:00:00Z' }),
probe({ id: 2, status: 'down', checked_at: '2026-08-20T09:10:00Z' }),
],
[
{
fqdn: 'solo.example.com',
configured: ['130.49.213.153'],
active: ['130.49.213.153'],
},
],
)
expect(history).toEqual([])
})
it('drops dedicated extra-FQDN DNS rows', () => {
const history = mergeFailoverHistory(
[
dns({
id: 11,
fqdn: 'nsgt.rkns.top',
action: 'added',
created_at: '2026-08-20 09:26:00',
}),
],
[],
mskHip,
)
expect(history).toEqual([])
})
})
+227 -11
View File
@@ -1,7 +1,29 @@
import {
bestAliveHealthStatus,
probeTime,
type HealthLogProbe,
type HealthLogStatus,
} from '@/lib/health-log'
import type { FailoverLogEntry } from '@/lib/schemas'
export type FailoverEventKind = 'removed' | 'last-resort'
export interface FailoverHistoryItem {
id: string
ip: string
fqdn: string
action: 'added' | 'removed'
created_at: string
copy: string
source: 'dns' | 'probe'
}
export interface FailoverEvent {
id: string
address: string
status: string
kind: FailoverEventKind
fqdns: string[]
consecutiveFailures: number
lastFailureReason: string | null
lastCheckAt?: string | null
@@ -16,31 +38,225 @@ export interface FailoverHealthInput {
last_checked_at?: string | null
}
export interface FailoverBindingPool {
fqdn: string
configured: readonly string[]
active: readonly string[]
}
/** Unique A targets. Duplicates of the same IP are not a pool. */
export function uniqueIpCount(ips: readonly string[]): number {
return new Set(ips.filter(Boolean)).size
}
/** Shared pool FQDN — two or more unique A targets. */
export function isSharedPoolBinding(binding: FailoverBindingPool): boolean {
return uniqueIpCount(binding.configured) >= 2
}
export function hasSharedPool(
bindings: readonly FailoverBindingPool[],
): boolean {
return bindings.some(isSharedPoolBinding)
}
/** Инцидент только при down. up / degraded / unknown — не вывод из пула. */
export function isFailoverEventStatus(status: string): boolean {
return status === 'down'
}
export function failoverEventCopy(event: FailoverEvent): string {
if (event.kind === 'removed') {
return event.fqdns.length > 0
? `Снята с ${event.fqdns.join(', ')}`
: 'Снята с DNS'
}
return event.fqdns.length > 0
? `Down, в A-записях last-resort на ${event.fqdns.join(', ')}`
: 'Down, в A-записях last-resort'
}
/**
* Инцидент failover = binding health down и адреса нет в DNS-пуле.
* OK / unknown / degraded вне пула standby, не инцидент.
* Down в activeAddresses (last-resort) не «снята с DNS».
* Инциденты пула только если у сервиса есть shared FQDN (2+ уникальных IP).
* Один IP не балансировка и не вывод из пула; Down смотрит health-монитор.
*/
export function toFailoverEvents(
ipHealth: readonly FailoverHealthInput[],
activeAddresses: readonly string[] = [],
bindings: readonly FailoverBindingPool[] = [],
): FailoverEvent[] {
const pool = new Set(activeAddresses)
if (pool.size === 0) return []
return ipHealth
.filter((row) => isFailoverEventStatus(row.status) && !pool.has(row.ip))
.map((row) => ({
if (!hasSharedPool(bindings)) return []
return ipHealth.filter((row) => isFailoverEventStatus(row.status)).map((row) => {
const removedFqdns: string[] = []
const lastResortFqdns: string[] = []
for (const binding of bindings) {
if (!isSharedPoolBinding(binding)) continue
const configured = binding.configured.includes(row.ip)
const active = binding.active.includes(row.ip)
if (configured && !active) removedFqdns.push(binding.fqdn)
else if (configured && active) lastResortFqdns.push(binding.fqdn)
}
const kind: FailoverEventKind =
removedFqdns.length > 0 ? 'removed' : 'last-resort'
return {
id: row.ip,
address: row.ip,
status: row.status,
kind,
fqdns: kind === 'removed' ? removedFqdns : lastResortFqdns,
consecutiveFailures: row.consecutive_failures ?? 0,
lastFailureReason: row.last_error ?? null,
lastCheckAt: row.last_checked_at ?? null,
}))
}
})
}
const DEDUPE_WINDOW_MS = 2 * 60 * 1000
function fqdnsForIp(
ip: string,
bindings: readonly FailoverBindingPool[],
): string[] {
return bindings
.filter((binding) => isSharedPoolBinding(binding) && binding.configured.includes(ip))
.map((binding) => binding.fqdn)
}
function isPoolFqdn(
fqdn: string,
bindings: readonly FailoverBindingPool[],
): boolean {
const binding = bindings.find((item) => item.fqdn === fqdn)
if (!binding) return true
return isSharedPoolBinding(binding)
}
function fqdnLabel(fqdns: string[]): string {
return fqdns.join(', ') || 'пул'
}
function isAliveStatus(status: HealthLogStatus): boolean {
return status === 'up' || status === 'degraded'
}
/**
* Per-IP any-up flips: down вышла из пула, up после down вернулась.
* Initial state is not an event.
*/
export function toIpAliveTransitions(
probes: readonly HealthLogProbe[],
): Array<{ id: string; ip: string; alive: boolean; at: string }> {
const byIp = new Map<string, HealthLogProbe[]>()
for (const item of probes) {
const list = byIp.get(item.ip)
if (list) list.push(item)
else byIp.set(item.ip, [item])
}
const out: Array<{ id: string; ip: string; alive: boolean; at: string }> = []
for (const [ip, list] of byIp) {
list.sort(
(a, b) => probeTime(a.checked_at) - probeTime(b.checked_at) || a.id - b.id,
)
const latestByProvider = new Map<string, HealthLogProbe>()
let prevAlive: boolean | undefined
for (const probe of list) {
latestByProvider.set(probe.provider, probe)
const status = bestAliveHealthStatus(
[...latestByProvider.values()].map((item) => item.status),
)
if (status === 'unknown') continue
const alive = isAliveStatus(status)
if (prevAlive !== undefined && alive !== prevAlive) {
out.push({
id: `probe:${probe.id}`,
ip,
alive,
at: probe.checked_at,
})
}
prevAlive = alive
}
}
return out
}
function dnsHistoryItem(item: FailoverLogEntry): FailoverHistoryItem {
return {
id: `dns:${item.id}`,
ip: item.ip,
fqdn: item.fqdn,
action: item.action,
created_at: item.created_at,
copy:
item.action === 'removed'
? `${item.ip} убрана с ${item.fqdn}`
: `${item.ip} добавлена на ${item.fqdn}`,
source: 'dns',
}
}
function probeHistoryItem(
transition: { id: string; ip: string; alive: boolean; at: string },
bindings: readonly FailoverBindingPool[],
): FailoverHistoryItem {
const fqdns = fqdnsForIp(transition.ip, bindings)
const fqdn = fqdnLabel(fqdns)
const action = transition.alive ? 'added' : 'removed'
return {
id: transition.id,
ip: transition.ip,
fqdn,
action,
created_at: transition.at,
copy:
action === 'removed'
? `${transition.ip} вышла из пула (${fqdn})`
: `${transition.ip} вернулась в пул (${fqdn})`,
source: 'probe',
}
}
function eventTime(value: string): number {
return probeTime(value)
}
/**
* DNS add/remove + health leave/return, newest first.
* Same IP+action within 2 minutes: keep the DNS row (it has a concrete FQDN).
*/
export function mergeFailoverHistory(
dns: readonly FailoverLogEntry[],
probes: readonly HealthLogProbe[],
bindings: readonly FailoverBindingPool[] = [],
): FailoverHistoryItem[] {
const merged = [
...dns
.filter((item) => isPoolFqdn(item.fqdn, bindings))
.map(dnsHistoryItem),
...toIpAliveTransitions(probes)
.filter((transition) => fqdnsForIp(transition.ip, bindings).length > 0)
.map((transition) => probeHistoryItem(transition, bindings)),
]
merged.sort(
(a, b) => eventTime(b.created_at) - eventTime(a.created_at) || a.id.localeCompare(b.id),
)
const kept: FailoverHistoryItem[] = []
for (const item of merged) {
const duplicate = kept.find(
(other) =>
other.ip === item.ip &&
other.action === item.action &&
Math.abs(eventTime(other.created_at) - eventTime(item.created_at)) <=
DEDUPE_WINDOW_MS,
)
if (!duplicate) {
kept.push(item)
continue
}
if (duplicate.source === 'probe' && item.source === 'dns') {
kept[kept.indexOf(duplicate)] = item
}
}
return kept
}
+73 -2
View File
@@ -1,9 +1,13 @@
import { describe, expect, it } from 'vitest'
import {
bestAliveHealthStatus,
collapseStatusChanges,
enabledHealthProviders,
latestHealthByIp,
providerHealthStatuses,
resolveIpDisplayHealth,
resolveServiceDisplayHealth,
worstHealthStatus,
type HealthLogProbe,
} from '@/lib/health-log'
@@ -73,7 +77,7 @@ describe('enabledHealthProviders', () => {
})
describe('providerHealthStatuses', () => {
it('uses worst latest-per-ip status', () => {
it('uses any-up among latest-per-ip statuses', () => {
const items = [
probe({ id: 1, ip: '1.1.1.1', status: 'up', checked_at: '2026-01-01T00:02:00Z' }),
probe({ id: 2, ip: '2.2.2.2', status: 'down', checked_at: '2026-01-01T00:01:00Z' }),
@@ -84,7 +88,7 @@ describe('providerHealthStatuses', () => {
checked_at: '2026-01-01T00:00:00Z',
}),
]
expect(providerHealthStatuses(items, ['local']).local).toBe('down')
expect(providerHealthStatuses(items, ['local']).local).toBe('up')
})
})
@@ -95,3 +99,70 @@ describe('worstHealthStatus', () => {
expect(worstHealthStatus([])).toBe('unknown')
})
})
describe('bestAliveHealthStatus', () => {
it('is up if any IP is up', () => {
expect(bestAliveHealthStatus(['up', 'down'])).toBe('up')
expect(bestAliveHealthStatus(['degraded', 'down'])).toBe('degraded')
expect(bestAliveHealthStatus(['down'])).toBe('down')
expect(bestAliveHealthStatus([])).toBe('unknown')
})
})
describe('latestHealthByIp', () => {
it('any-up among latest-per-provider probes', () => {
const items = [
probe({
id: 1,
ip: '130.49.213.153',
provider: 'local',
status: 'up',
checked_at: '2026-08-20T09:00:00Z',
}),
probe({
id: 2,
ip: '130.49.213.153',
provider: 'cloudflare',
status: 'down',
checked_at: '2026-08-20T09:00:01Z',
}),
probe({
id: 3,
ip: '93.115.203.183',
status: 'unknown',
checked_at: '2026-08-20T08:59:00Z',
}),
]
const byIp = latestHealthByIp(items)
expect(byIp.get('130.49.213.153')?.status).toBe('up')
expect(byIp.get('93.115.203.183')?.status).toBe('unknown')
})
})
describe('resolveIpDisplayHealth', () => {
it('prefers a live probe over stored unknown', () => {
expect(resolveIpDisplayHealth('unknown', 'up')).toBe('up')
expect(resolveIpDisplayHealth('down', 'up')).toBe('up')
expect(resolveIpDisplayHealth('up', 'unknown')).toBe('up')
expect(resolveIpDisplayHealth('unknown', undefined)).toBe('unknown')
})
})
describe('resolveServiceDisplayHealth', () => {
it('shows OK when live probes recovered and stored is still unknown', () => {
expect(
resolveServiceDisplayHealth(
'unknown',
[{ ip: '2.59.161.102', status: 'unknown' }],
[
probe({
id: 1,
ip: '2.59.161.102',
status: 'up',
checked_at: '2026-08-20T18:00:00Z',
}),
],
),
).toBe('up')
})
})
+90 -2
View File
@@ -98,7 +98,18 @@ export function worstHealthStatus(statuses: readonly HealthLogStatus[]): HealthL
)
}
/** Latest probe per IP for a provider, then worst among those IPs. */
/** Any up → up; else degraded, then down, then unknown. */
export function bestAliveHealthStatus(
statuses: readonly HealthLogStatus[],
): HealthLogStatus {
if (statuses.length === 0) return 'unknown'
if (statuses.some((status) => status === 'up')) return 'up'
if (statuses.some((status) => status === 'degraded')) return 'degraded'
if (statuses.some((status) => status === 'down')) return 'down'
return 'unknown'
}
/** Latest probe per IP for a provider, then any-up among those IPs. */
export function providerHealthStatuses(
items: readonly HealthLogProbe[],
providers: readonly HealthCheckProvider[],
@@ -120,8 +131,85 @@ export function providerHealthStatuses(
const statuses = [...latestByIp.values()]
.filter((item) => item.provider === provider)
.map((item) => item.status)
result[provider] = worstHealthStatus(statuses)
result[provider] = bestAliveHealthStatus(statuses)
}
return result
}
export interface IpDisplayHealth {
status: HealthLogStatus
latency_ms: number | null
last_checked_at: string
last_error: string | null
colo: string | null
provider: HealthCheckProvider
}
/**
* Latest probe per provider+IP, then any-up among those providers.
* Used by the IP table so hysteresis `unknown` in ip_health does not hide a live OK.
*/
export function latestHealthByIp(
items: readonly HealthLogProbe[],
): Map<string, IpDisplayHealth> {
const latest = new Map<string, HealthLogProbe>()
const sorted = [...items].sort(
(a, b) => probeTime(b.checked_at) - probeTime(a.checked_at) || b.id - a.id,
)
for (const item of sorted) {
const key = `${item.provider}\0${item.ip}`
if (!latest.has(key)) latest.set(key, item)
}
const byIp = new Map<string, HealthLogProbe[]>()
for (const item of latest.values()) {
const list = byIp.get(item.ip)
if (list) list.push(item)
else byIp.set(item.ip, [item])
}
const result = new Map<string, IpDisplayHealth>()
for (const [ip, probes] of byIp) {
const status = bestAliveHealthStatus(probes.map((probe) => probe.status))
const preferred =
probes.find((probe) => probe.status === status) ?? probes[0]!
result.set(ip, {
status,
latency_ms: preferred.latency_ms,
last_checked_at: preferred.checked_at,
last_error: preferred.error,
colo: preferred.colo,
provider: preferred.provider,
})
}
return result
}
/** Prefer a concrete live probe over stored hysteresis `unknown`. */
export function resolveIpDisplayHealth(
stored: HealthLogStatus | undefined,
live: HealthLogStatus | undefined,
): HealthLogStatus {
if (live && live !== 'unknown') return live
return stored ?? live ?? 'unknown'
}
/** Service KPI / card: any-up of per-IP overlay (live probes beat hysteresis). */
export function resolveServiceDisplayHealth(
stored: HealthLogStatus | undefined,
ipHealth: readonly { ip: string; status: string }[],
probes: readonly HealthLogProbe[] = [],
): HealthLogStatus {
const liveByIp = latestHealthByIp(probes)
const statuses =
ipHealth.length > 0
? ipHealth.map((row) =>
resolveIpDisplayHealth(
row.status as HealthLogStatus,
liveByIp.get(row.ip)?.status,
),
)
: [...liveByIp.values()].map((row) => row.status)
return resolveIpDisplayHealth(stored, bestAliveHealthStatus(statuses))
}
+13
View File
@@ -84,6 +84,7 @@ export const serviceDomainBindingSchema = z
health_check_aggregate: z.enum(['any', 'all', 'majority']).catch('majority'),
cert_monitoring: z.enum(['auto', 'required', 'skipped']).default('auto'),
sync_status: z.string().nullable().default(null),
active_ips: z.array(z.string()).default([]),
})
.transform((binding) => ({
...binding,
@@ -125,6 +126,18 @@ export const healthProbeLogSchema = z.object({
checked_at: z.string(),
})
export const failoverLogSchema = z.object({
id: z.number(),
service_id: z.number(),
binding_id: z.number(),
fqdn: z.string(),
ip: z.string(),
action: z.enum(['added', 'removed']),
created_at: z.string(),
})
export type FailoverLogEntry = z.infer<typeof failoverLogSchema>
export const serviceViewSchema = serviceSchema.extend({
subdomain: z.string().default(''),
enabled: z.coerce.boolean().default(false),
+173 -7
View File
@@ -4,9 +4,11 @@ import {
DEFAULT_BINDING_HEALTH,
addAddressNode,
addCommonFqdn,
addExtraFqdn,
emptyAddressBlock,
emptyBindingDraft,
hydrateAddressBlock,
patchAddressIpMeta,
removeAddressNode,
toAddressBindings,
toDomainsPayload,
@@ -44,8 +46,8 @@ describe('hydrateAddressBlock', () => {
expect(state.commonFqdns).toEqual(['rutg.rkns.top'])
expect(state.nodes).toEqual([
{ ip: '93.115.203.183', extraFqdn: 'msk.rutg.rkns.top' },
{ ip: '185.244.181.61', extraFqdn: '' },
{ ip: '93.115.203.183', extraFqdns: ['msk.rutg.rkns.top'] },
{ ip: '185.244.181.61', extraFqdns: [] },
])
expect(state.preservedBindings).toEqual([])
})
@@ -65,7 +67,7 @@ describe('hydrateAddressBlock', () => {
const state = hydrateAddressBlock(drafts, ['1.1.1.1', '2.2.2.2'])
expect(state.commonFqdns).toEqual(['rutg.rkns.top', 'both.rkns.top'])
expect(state.nodes.every((node) => node.extraFqdn === '')).toBe(true)
expect(state.nodes.every((node) => node.extraFqdns.length === 0)).toBe(true)
expect(state.preservedBindings.map((item) => item.fqdn)).toEqual(['alias.rkns.top'])
})
@@ -77,11 +79,105 @@ describe('hydrateAddressBlock', () => {
const state = hydrateAddressBlock(drafts, ['10.0.0.1'])
expect(state.nodes).toEqual([{ ip: '10.0.0.1', extraFqdn: '' }])
expect(state.nodes).toEqual([{ ip: '10.0.0.1', extraFqdns: [] }])
expect(state.commonFqdns).toEqual(['gw.example.com'])
expect(state.preservedBindings).toHaveLength(1)
expect(state.preservedBindings[0]?.fqdn).toBe('edge.example.com')
})
it('при одном IP пула отделяет второй A в extraFqdn узла', () => {
const drafts = [
aRecord('dns.shnt.top', ['130.49.213.176']),
aRecord('ndns.shnt.top', ['130.49.213.176']),
]
const state = hydrateAddressBlock(drafts, ['130.49.213.176'])
expect(state.commonFqdns).toEqual(['dns.shnt.top'])
expect(state.nodes).toEqual([
{ ip: '130.49.213.176', extraFqdns: ['ndns.shnt.top'] },
])
expect(state.preservedBindings).toEqual([])
})
it('кладёт несколько extra A на один IP в extraFqdns, включая wildcard', () => {
const drafts = [
aRecord('dns.shnt.top', ['130.49.213.176']),
aRecord('ndns.shnt.top', ['130.49.213.176']),
aRecord('*.mdns.shnt.top', ['130.49.213.176']),
]
const state = hydrateAddressBlock(drafts, ['130.49.213.176'])
expect(state.commonFqdns).toEqual(['dns.shnt.top'])
expect(state.nodes).toEqual([
{
ip: '130.49.213.176',
extraFqdns: ['ndns.shnt.top', '*.mdns.shnt.top'],
},
])
expect(state.preservedBindings).toEqual([])
})
it('поднимает веса и приоритеты с общего FQDN', () => {
const drafts = [
aRecord('gt.rkns.top', ['130.49.213.153', '93.115.203.183'], {
target_ip_weights: { '130.49.213.153': 3, '93.115.203.183': 1 },
target_ip_priorities: { '130.49.213.153': 2, '93.115.203.183': 1 },
}),
aRecord('nsgt.rkns.top', ['130.49.213.153']),
]
const state = hydrateAddressBlock(drafts, ['130.49.213.153', '93.115.203.183'])
expect(state.target_ip_weights).toEqual({
'130.49.213.153': 3,
'93.115.203.183': 1,
})
expect(state.target_ip_priorities).toEqual({
'130.49.213.153': 2,
'93.115.203.183': 1,
})
expect(state.nodes[0]?.extraFqdns).toEqual(['nsgt.rkns.top'])
})
it('лечит урезанный общий FQDN (legacy toggle) как common, не preserved', () => {
const pool = ['130.49.213.153', '130.49.213.176', '93.115.203.183']
const drafts = [
// corrupted common — missing one pool IP
aRecord('gw.pngs.top', ['130.49.213.153', '130.49.213.176']),
aRecord('gt.rkns.top', pool),
aRecord('nsgt.rkns.top', ['130.49.213.176']),
aRecord('rutg.rkns.top', ['93.115.203.183']),
]
const state = hydrateAddressBlock(drafts, pool)
expect(state.commonFqdns).toEqual(['gw.pngs.top', 'gt.rkns.top'])
expect(state.nodes).toEqual([
{ ip: '130.49.213.153', extraFqdns: [] },
{ ip: '130.49.213.176', extraFqdns: ['nsgt.rkns.top'] },
{ ip: '93.115.203.183', extraFqdns: ['rutg.rkns.top'] },
])
expect(state.preservedBindings).toEqual([])
const payload = toDomainsPayload(state, primaryMeta)
expect(payload.map((item) => item.fqdn)).toEqual([
'gw.pngs.top',
'gt.rkns.top',
'nsgt.rkns.top',
'rutg.rkns.top',
])
expect(payload[0]?.target_ips).toEqual(pool)
expect(new Set(payload.map((item) => item.fqdn.toLowerCase())).size).toBe(4)
})
it('не дублирует FQDN при повторном binding в drafts', () => {
const pool = ['1.1.1.1', '2.2.2.2']
const state = hydrateAddressBlock(
[aRecord('gw.example.com', ['1.1.1.1']), aRecord('gw.example.com', pool)],
pool,
)
expect(state.commonFqdns).toEqual(['gw.example.com'])
expect(state.nodes.every((node) => node.extraFqdns.length === 0)).toBe(true)
})
})
describe('toDomainsPayload', () => {
@@ -122,6 +218,40 @@ describe('toDomainsPayload', () => {
expect(second.nodes).toEqual(first.nodes)
expect(second.preservedBindings).toEqual([])
})
it('круг hydrate → payload → hydrate сохраняет extra FQDN при одном IP', () => {
const drafts = [
aRecord('dns.shnt.top', ['130.49.213.176']),
aRecord('ndns.shnt.top', ['130.49.213.176']),
]
const first = hydrateAddressBlock(drafts, ['130.49.213.176'])
expect(first.commonFqdns).toEqual(['dns.shnt.top'])
expect(first.nodes).toEqual([
{ ip: '130.49.213.176', extraFqdns: ['ndns.shnt.top'] },
])
const rebound = toAddressBindings(first, primaryMeta)
const second = hydrateAddressBlock(rebound, ['130.49.213.176'])
expect(second.commonFqdns).toEqual(first.commonFqdns)
expect(second.nodes).toEqual(first.nodes)
expect(second.preservedBindings).toEqual([])
})
it('круг hydrate → payload → hydrate сохраняет несколько extra и wildcard при одном IP', () => {
const drafts = [
aRecord('dns.shnt.top', ['130.49.213.176']),
aRecord('ndns.shnt.top', ['130.49.213.176']),
aRecord('*.mdns.shnt.top', ['130.49.213.176']),
]
const first = hydrateAddressBlock(drafts, ['130.49.213.176'])
expect(first.nodes[0]?.extraFqdns).toEqual(['ndns.shnt.top', '*.mdns.shnt.top'])
const rebound = toAddressBindings(first, primaryMeta)
const second = hydrateAddressBlock(rebound, ['130.49.213.176'])
expect(second.commonFqdns).toEqual(first.commonFqdns)
expect(second.nodes).toEqual(first.nodes)
expect(second.preservedBindings).toEqual([])
})
})
describe('removeAddressNode', () => {
@@ -137,7 +267,7 @@ describe('removeAddressNode', () => {
const next = removeAddressNode(state, '10.0.0.1')
expect(next.nodes).toEqual([{ ip: '10.0.0.2', extraFqdn: '' }])
expect(next.nodes).toEqual([{ ip: '10.0.0.2', extraFqdns: [] }])
expect(next.preservedBindings).toHaveLength(1)
expect(next.preservedBindings[0]?.target_ips).toEqual(['9.9.9.9'])
})
@@ -146,7 +276,7 @@ describe('removeAddressNode', () => {
describe('addAddressNode / addCommonFqdn', () => {
it('не добавляет дубликат IP', () => {
const withIp = addAddressNode(
{ ...emptyAddressBlock(), nodes: [{ ip: '1.1.1.1', extraFqdn: '' }] },
{ ...emptyAddressBlock(), nodes: [{ ip: '1.1.1.1', extraFqdns: [] }] },
'1.1.1.1',
)
expect(withIp.nodes).toHaveLength(1)
@@ -159,6 +289,42 @@ describe('addAddressNode / addCommonFqdn', () => {
)
expect(state.commonFqdns).toEqual(['gt.rkns.top'])
})
it('добавляет extra FQDN к IP и отклоняет дубликат', () => {
const withIp = addAddressNode(emptyAddressBlock(), '1.1.1.1')
const withExtra = addExtraFqdn(withIp, '1.1.1.1', 'mdns.shnt.top')
expect(withExtra.nodes[0]?.extraFqdns).toEqual(['mdns.shnt.top'])
expect(addExtraFqdn(withExtra, '1.1.1.1', 'MDNS.shnt.top')).toBe(withExtra)
})
})
describe('patchAddressIpMeta', () => {
it('меняет вес одного IP и не трогает extraFqdns', () => {
const state = {
...addAddressNode(addAddressNode(emptyAddressBlock(), '1.1.1.1'), '2.2.2.2'),
nodes: [
{ ip: '1.1.1.1', extraFqdns: ['msk.example.com'] },
{ ip: '2.2.2.2', extraFqdns: [] },
],
}
const next = patchAddressIpMeta(state, '1.1.1.1', { weight: 7 })
expect(next.target_ip_weights).toEqual({ '1.1.1.1': 7, '2.2.2.2': 1 })
expect(next.target_ip_priorities).toEqual(state.target_ip_priorities)
expect(next.nodes).toEqual(state.nodes)
})
it('clamp веса и приоритета в 1–100', () => {
const state = addAddressNode(emptyAddressBlock(), '10.0.0.1')
expect(patchAddressIpMeta(state, '10.0.0.1', { weight: 0 }).target_ip_weights['10.0.0.1']).toBe(1)
expect(
patchAddressIpMeta(state, '10.0.0.1', { priority: 999 }).target_ip_priorities['10.0.0.1'],
).toBe(100)
})
it('игнорирует IP вне пула', () => {
const state = addAddressNode(emptyAddressBlock(), '10.0.0.1')
expect(patchAddressIpMeta(state, '8.8.8.8', { weight: 5 })).toBe(state)
})
})
describe('CNAME / preservedBindings', () => {
@@ -174,7 +340,7 @@ describe('CNAME / preservedBindings', () => {
cname,
]
const state = hydrateAddressBlock(drafts, ['1.1.1.1', '2.2.2.2'])
expect(state.nodes[0]?.extraFqdn).toBe('msk.rkns.top')
expect(state.nodes[0]?.extraFqdns).toEqual(['msk.rkns.top'])
expect(state.preservedBindings).toHaveLength(1)
const payload = toDomainsPayload(state, primaryMeta)
+214 -32
View File
@@ -33,7 +33,7 @@ export interface ServiceBindingDraft {
export interface AddressNode {
ip: string
extraFqdn: string
extraFqdns: string[]
}
export interface AddressBlockState {
@@ -113,6 +113,10 @@ function sameIpSet(left: string[], right: string[]): boolean {
return right.every((ip) => set.has(ip.trim()))
}
function fqdnKey(value: string): string {
return value.trim().toLowerCase()
}
export function toBindingDrafts(service: ServiceView): ServiceBindingDraft[] {
return (service.domains ?? []).map((binding) => ({
fqdn: bindingToFqdn(binding),
@@ -145,6 +149,36 @@ function isFullPoolA(draft: ServiceBindingDraft, pool: string[]): boolean {
return draft.record_type === 'A' && sameIpSet(draft.target_ips, pool)
}
/** UI contract: common = 2+ IPs all in pool (full or corrupted subset after old toggles). */
function isCommonPoolA(draft: ServiceBindingDraft, poolSet: Set<string>): boolean {
if (draft.record_type !== 'A') return false
const ips = draft.target_ips.map((ip) => ip.trim()).filter(Boolean)
if (ips.length < 2) return false
return ips.every((ip) => poolSet.has(ip))
}
function takeAsCommon(
draft: ServiceBindingDraft,
fqdn: string,
commonFqdns: string[],
seenCommon: Set<string>,
weights: Record<string, number>,
priorities: Record<string, number>,
): { weights: Record<string, number>; priorities: Record<string, number> } {
const key = fqdnKey(fqdn)
if (fqdn && key && !seenCommon.has(key)) {
seenCommon.add(key)
commonFqdns.push(draft.fqdn)
}
if (Object.keys(weights).length === 0) {
return {
weights: { ...draft.target_ip_weights },
priorities: { ...draft.target_ip_priorities },
}
}
return { weights, priorities }
}
export function hydrateAddressBlock(
drafts: ServiceBindingDraft[],
pool: string[] = [],
@@ -161,27 +195,73 @@ export function hydrateAddressBlock(
: uniqueIps(...(multiIpTargets.length > 0 ? multiIpTargets : allAIps))
const poolSet = new Set(ips)
const commonFqdns: string[] = []
const claimed = new Set<string>()
const extraByIp = new Map<string, string>()
const seenCommon = new Set<string>()
const seenExtra = new Set<string>()
const extraByIp = new Map<string, string[]>()
const preservedBindings: ServiceBindingDraft[] = []
let weights: Record<string, number> = {}
let priorities: Record<string, number> = {}
const splitSinglePool =
ips.length === 1 &&
drafts.filter((draft) => isFullPoolA(draft, ips) || isCommonPoolA(draft, poolSet))
.length > 1
let assignedFirstSinglePoolCommon = false
function pushExtra(ip: string, fqdn: string) {
const key = fqdnKey(fqdn)
if (!key || seenExtra.has(key) || seenCommon.has(key)) return
seenExtra.add(key)
const list = extraByIp.get(ip) ?? []
list.push(fqdn)
extraByIp.set(ip, list)
}
function promoteToCommon(draft: ServiceBindingDraft, fqdn: string) {
const key = fqdnKey(fqdn)
if (key && seenExtra.has(key)) {
seenExtra.delete(key)
for (const [ip, list] of extraByIp) {
extraByIp.set(
ip,
list.filter((item) => fqdnKey(item) !== key),
)
}
}
const next = takeAsCommon(
draft,
fqdn,
commonFqdns,
seenCommon,
weights,
priorities,
)
weights = next.weights
priorities = next.priorities
}
for (const draft of drafts) {
const fqdn = draft.fqdn.trim()
if (isFullPoolA(draft, ips)) {
if (fqdn) commonFqdns.push(draft.fqdn)
if (Object.keys(weights).length === 0) {
weights = { ...draft.target_ip_weights }
priorities = { ...draft.target_ip_priorities }
if (splitSinglePool && (isFullPoolA(draft, ips) || isCommonPoolA(draft, poolSet))) {
if (!assignedFirstSinglePoolCommon) {
assignedFirstSinglePoolCommon = true
promoteToCommon(draft, fqdn)
continue
}
const ip = draft.target_ips[0]?.trim() ?? ''
if (ip && poolSet.has(ip) && fqdn) {
pushExtra(ip, draft.fqdn)
continue
}
}
// Full pool OR multi-IP subset of pool → common (heals orphaned toggle damage).
if (isFullPoolA(draft, ips) || isCommonPoolA(draft, poolSet)) {
promoteToCommon(draft, fqdn)
continue
}
if (draft.record_type === 'A' && draft.target_ips.length === 1) {
const ip = draft.target_ips[0]?.trim() ?? ''
if (ip && poolSet.has(ip) && fqdn && !claimed.has(ip)) {
claimed.add(ip)
extraByIp.set(ip, draft.fqdn)
if (ip && poolSet.has(ip) && fqdn) {
pushExtra(ip, draft.fqdn)
continue
}
}
@@ -192,7 +272,7 @@ export function hydrateAddressBlock(
commonFqdns,
nodes: ips.map((ip) => ({
ip,
extraFqdn: extraByIp.get(ip) ?? '',
extraFqdns: extraByIp.get(ip) ?? [],
})),
preservedBindings,
target_ip_weights: weights,
@@ -237,28 +317,66 @@ export function addAddressNode(state: AddressBlockState, ip: string): AddressBlo
}
return {
...state,
nodes: [...state.nodes, { ip: trimmed, extraFqdn: '' }],
nodes: [...state.nodes, { ip: trimmed, extraFqdns: [] }],
target_ip_weights: { ...state.target_ip_weights, [trimmed]: 1 },
target_ip_priorities: { ...state.target_ip_priorities, [trimmed]: 1 },
}
}
function fqdnKey(value: string): string {
return value.trim().toLowerCase()
const LB_META_MIN = 1
const LB_META_MAX = 100
function clampLbMeta(value: number): number {
if (!Number.isFinite(value)) return LB_META_MIN
return Math.min(LB_META_MAX, Math.max(LB_META_MIN, Math.round(value)))
}
export function patchAddressIpMeta(
state: AddressBlockState,
ip: string,
meta: { weight?: number; priority?: number },
): AddressBlockState {
if (!state.nodes.some((node) => node.ip === ip)) return state
return {
...state,
target_ip_weights:
meta.weight === undefined
? state.target_ip_weights
: { ...state.target_ip_weights, [ip]: clampLbMeta(meta.weight) },
target_ip_priorities:
meta.priority === undefined
? state.target_ip_priorities
: { ...state.target_ip_priorities, [ip]: clampLbMeta(meta.priority) },
}
}
export function addressHasFqdn(state: AddressBlockState, fqdn: string): boolean {
const key = fqdnKey(fqdn)
if (!key) return false
if (state.commonFqdns.some((item) => fqdnKey(item) === key)) return true
if (state.nodes.some((node) => fqdnKey(node.extraFqdn) === key)) return true
return false
if (state.nodes.some((node) => node.extraFqdns.some((item) => fqdnKey(item) === key))) {
return true
}
return state.preservedBindings.some((item) => fqdnKey(item.fqdn) === key)
}
export function addCommonFqdn(state: AddressBlockState, fqdn: string): AddressBlockState {
const trimmed = fqdn.trim()
if (!trimmed || addressHasFqdn(state, trimmed)) return state
return { ...state, commonFqdns: [...state.commonFqdns, trimmed] }
if (!trimmed) return state
const key = fqdnKey(trimmed)
if (state.commonFqdns.some((item) => fqdnKey(item) === key)) return state
if (state.nodes.some((node) => node.extraFqdns.some((item) => fqdnKey(item) === key))) {
return state
}
// Promote out of invisible preserved (corrupted / CNAME-adjacent duplicates).
const preservedBindings = state.preservedBindings.filter(
(item) => fqdnKey(item.fqdn) !== key,
)
return {
...state,
commonFqdns: [...state.commonFqdns, trimmed],
preservedBindings,
}
}
export function removeCommonFqdn(state: AddressBlockState, index: number): AddressBlockState {
@@ -279,6 +397,62 @@ export function updateCommonFqdn(
}
}
export function addExtraFqdn(
state: AddressBlockState,
ip: string,
fqdn: string,
): AddressBlockState {
const trimmed = fqdn.trim()
if (!trimmed) return state
const key = fqdnKey(trimmed)
if (state.commonFqdns.some((item) => fqdnKey(item) === key)) return state
if (state.nodes.some((node) => node.extraFqdns.some((item) => fqdnKey(item) === key))) {
return state
}
if (!state.nodes.some((node) => node.ip === ip)) return state
return {
...state,
preservedBindings: state.preservedBindings.filter((item) => fqdnKey(item.fqdn) !== key),
nodes: state.nodes.map((node) =>
node.ip === ip ? { ...node, extraFqdns: [...node.extraFqdns, trimmed] } : node,
),
}
}
export function removeExtraFqdn(
state: AddressBlockState,
ip: string,
index: number,
): AddressBlockState {
return {
...state,
nodes: state.nodes.map((node) =>
node.ip === ip
? { ...node, extraFqdns: node.extraFqdns.filter((_, i) => i !== index) }
: node,
),
}
}
export function updateExtraFqdn(
state: AddressBlockState,
ip: string,
index: number,
fqdn: string,
): AddressBlockState {
return {
...state,
nodes: state.nodes.map((node) =>
node.ip === ip
? {
...node,
extraFqdns: node.extraFqdns.map((item, i) => (i === index ? fqdn : item)),
}
: node,
),
}
}
export function toAddressBindings(
state: AddressBlockState,
primary: AddressPrimaryMeta,
@@ -308,21 +482,29 @@ export function toAddressBindings(
}
for (const node of state.nodes) {
const extraFqdn = node.extraFqdn.trim()
if (!extraFqdn) continue
drafts.push({
fqdn: extraFqdn,
record_type: 'A',
target_ips: [node.ip],
target_cname: '',
lb_mode: primary.lb_mode,
health: { ...primary.health },
target_ip_weights: { [node.ip]: weights[node.ip] ?? 1 },
target_ip_priorities: { [node.ip]: priorities[node.ip] ?? 1 },
})
for (const raw of node.extraFqdns) {
const extraFqdn = raw.trim()
if (!extraFqdn) continue
drafts.push({
fqdn: extraFqdn,
record_type: 'A',
target_ips: [node.ip],
target_cname: '',
lb_mode: primary.lb_mode,
health: { ...primary.health },
target_ip_weights: { [node.ip]: weights[node.ip] ?? 1 },
target_ip_priorities: { [node.ip]: priorities[node.ip] ?? 1 },
})
}
}
drafts.push(...state.preservedBindings)
const seen = new Set(drafts.map((item) => fqdnKey(item.fqdn)).filter(Boolean))
for (const preserved of state.preservedBindings) {
const key = fqdnKey(preserved.fqdn)
if (!key || seen.has(key)) continue
seen.add(key)
drafts.push(preserved)
}
return drafts
}
+17
View File
@@ -1,6 +1,7 @@
import { queryOptions } from '@tanstack/react-query'
import { api } from '@/lib/api-client'
import {
failoverLogSchema,
healthProbeLogSchema,
serviceBindingSchema,
serviceGroupsResponseSchema,
@@ -35,6 +36,8 @@ export const serviceGroupsQueryOptions = () =>
}
return parsed.data
},
refetchInterval: 10_000,
staleTime: 5_000,
})
export const servicesQueryOptions = () =>
@@ -89,6 +92,7 @@ export const serviceDetailKeys = {
overview: (id: number) => [...serviceKeys.all, id, 'overview'] as const,
nodes: (id: number) => [...serviceKeys.all, id, 'nodes'] as const,
healthLog: (id: number) => [...serviceKeys.all, id, 'health-log'] as const,
failoverLog: (id: number) => [...serviceKeys.all, id, 'failover-log'] as const,
view: (id: number) => [...serviceKeys.all, id, 'view'] as const,
}
@@ -99,6 +103,8 @@ export const serviceViewQueryOptions = (id: number) =>
const data = await api.get<unknown>(`/api/v1/services/${id}`)
return serviceViewSchema.parse(data)
},
refetchInterval: 10_000,
staleTime: 5_000,
})
export const serviceHealthLogQueryOptions = (id: number) =>
@@ -108,6 +114,17 @@ export const serviceHealthLogQueryOptions = (id: number) =>
const data = await api.get<unknown>(`/api/v1/services/${id}/health-log`)
return z.object({ items: z.array(healthProbeLogSchema) }).parse(data)
},
refetchInterval: 10_000,
staleTime: 5_000,
})
export const serviceFailoverLogQueryOptions = (id: number) =>
queryOptions({
queryKey: serviceDetailKeys.failoverLog(id),
queryFn: async () => {
const data = await api.get<unknown>(`/api/v1/services/${id}/failover-log`)
return z.object({ items: z.array(failoverLogSchema) }).parse(data)
},
})
export const serviceOverviewQueryOptions = (id: number) =>
@@ -32,9 +32,11 @@ import {
ServiceHealthMonitor,
} from '@/components/reui-kit'
import { api } from '@/lib/api-client'
import { hasSharedPool, uniqueIpCount } from '@/lib/failover-events'
import {
enabledHealthProviders,
providerHealthStatuses,
resolveServiceDisplayHealth,
} from '@/lib/health-log'
import type { ServiceView, UpdateServiceConfigInput } from '@/lib/schemas'
import {
@@ -44,6 +46,7 @@ import {
domainsListQueryOptions,
serviceBindingKeys,
serviceDetailKeys,
serviceFailoverLogQueryOptions,
serviceGroupKeys,
serviceGroupsQueryOptions,
serviceHealthLogQueryOptions,
@@ -90,6 +93,7 @@ function ServiceDetailPage() {
const viewQuery = useQuery(serviceViewQueryOptions(id))
const overviewQuery = useQuery(serviceOverviewQueryOptions(id))
const logQuery = useQuery(serviceHealthLogQueryOptions(id))
const failoverLogQuery = useQuery(serviceFailoverLogQueryOptions(id))
const nodesQuery = useQuery(serviceNodesQueryOptions(id))
const groupsQuery = useQuery(serviceGroupsQueryOptions())
const domainsQuery = useQuery(domainsListQueryOptions())
@@ -100,6 +104,35 @@ function ServiceDetailPage() {
() => logQuery.data?.items ?? [],
[logQuery.data?.items],
)
const failoverHistory = useMemo(
() => failoverLogQuery.data?.items ?? [],
[failoverLogQuery.data?.items],
)
const failoverBindings = useMemo(
() =>
(service?.domains ?? []).map((domain) => ({
fqdn: domain.fqdn,
configured: domain.target_ips,
active: domain.active_ips ?? [],
})),
[service?.domains],
)
const uniqueEnabledIps = useMemo(
() =>
(service?.ips ?? []).filter((ip) => service?.ip_enabled[ip] !== false),
[service?.ips, service?.ip_enabled],
)
const displayHealth = useMemo(
() =>
resolveServiceDisplayHealth(
service?.health_status,
service?.ip_health ?? [],
logItems,
),
[service?.health_status, service?.ip_health, logItems],
)
const showPoolPanel =
uniqueEnabledIps.length >= 2 && hasSharedPool(failoverBindings)
const nodes = (nodesQuery.data as OverviewPayload['nodes']) ?? []
const [editOpen, setEditOpen] = useState(false)
@@ -129,6 +162,7 @@ function ServiceDetailPage() {
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.overview(id) }),
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.nodes(id) }),
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.healthLog(id) }),
queryClient.invalidateQueries({ queryKey: serviceDetailKeys.failoverLog(id) }),
])
}
@@ -234,8 +268,11 @@ function ServiceDetailPage() {
description="Domain → Service → Node → Health → Failover"
actions={
<>
<LbModeTile mode={service.lb_mode} />
<HealthCheckBadge status={service.health_status} />
<LbModeTile
mode={service.lb_mode}
hasPool={uniqueIpCount(service.ips) >= 2}
/>
<HealthCheckBadge status={displayHealth} />
<Tooltip>
<TooltipTrigger
render={
@@ -262,20 +299,20 @@ function ServiceDetailPage() {
id: 'status',
icon: <ActivityIcon />,
label: 'Статус',
value: service.health_status === 'up' ? 'OK' : service.health_status,
value: displayHealth === 'up' ? 'OK' : displayHealth,
variant:
service.health_status === 'down'
displayHealth === 'down'
? 'destructive'
: service.health_status === 'degraded'
: displayHealth === 'degraded'
? 'warning'
: 'default',
iconClassName:
service.health_status === 'down'
displayHealth === 'down'
? 'text-destructive'
: service.health_status === 'degraded'
: displayHealth === 'degraded'
? 'text-warning'
: 'text-success',
hint: <HealthCheckBadge status={service.health_status} size="xs" />,
hint: <HealthCheckBadge status={displayHealth} size="xs" />,
},
{
id: 'fqdn',
@@ -289,21 +326,33 @@ function ServiceDetailPage() {
icon: <NetworkIcon />,
label: 'IP',
value: String(service.ips.length),
hint: `${service.active_ips.length} в пуле`,
hint: showPoolPanel
? `${service.active_ips.length} в пуле`
: uniqueEnabledIps.length <= 1
? 'без балансировки'
: service.ips.join(', ') || 'нет',
},
{
id: 'pool',
icon: <ServerIcon />,
label: 'Активный пул',
value: String((overview?.active_addresses ?? service.active_ips).length),
hint: (overview?.active_addresses ?? service.active_ips).join(', ') || 'нет',
label: showPoolPanel ? 'Активный пул' : 'Адреса',
value: String(
(overview?.active_addresses ?? service.active_ips).length,
),
hint:
(overview?.active_addresses ?? service.active_ips).join(', ') ||
'нет',
},
]}
/>
<section
aria-label="Мониторинг"
className="grid min-w-0 items-start gap-2 @3xl:grid-cols-2"
className={
showPoolPanel
? 'grid min-w-0 items-start gap-2 @3xl:grid-cols-2'
: 'grid min-w-0 items-start gap-2'
}
>
<ServiceHealthMonitor
items={logItems}
@@ -311,10 +360,16 @@ function ServiceDetailPage() {
statuses={providerStatuses}
isLoading={logQuery.isLoading}
/>
<ServiceFailoverPanel
ipHealth={service.ip_health}
activeAddresses={overview?.active_addresses ?? service.active_ips}
/>
{showPoolPanel ? (
<ServiceFailoverPanel
lbMode={service.lb_mode}
hasPool={uniqueIpCount(service.ips) >= 2}
ipHealth={service.ip_health}
bindings={failoverBindings}
history={failoverHistory}
probes={logItems}
/>
) : null}
</section>
{service.ips.length === 0 && service.domains.length === 0 ? (
@@ -327,6 +382,7 @@ function ServiceDetailPage() {
<ServiceDetailGrid
service={service}
nodes={nodes}
probes={logItems}
togglingIp={togglingIp}
onToggleIp={(ip, enabled) => {
setTogglingIp(ip)
@@ -1,6 +1,7 @@
import { createFileRoute, Outlet } from '@tanstack/react-router'
import { PageShell } from '@/components/page-shell'
import {
serviceFailoverLogQueryOptions,
serviceHealthLogQueryOptions,
serviceNodesQueryOptions,
serviceOverviewQueryOptions,
@@ -14,6 +15,7 @@ export const Route = createFileRoute('/_auth/services/$serviceId')({
queryClient.ensureQueryData(serviceViewQueryOptions(id)),
queryClient.ensureQueryData(serviceOverviewQueryOptions(id)),
queryClient.ensureQueryData(serviceHealthLogQueryOptions(id)),
queryClient.ensureQueryData(serviceFailoverLogQueryOptions(id)),
queryClient.ensureQueryData(serviceNodesQueryOptions(id)),
])
return { breadcrumb: view.name }
+4 -3
View File
@@ -44,9 +44,10 @@ health-check работают на двух уровнях:
- **Change Domain** — перенос привязок между зонами `POST /api/v1/services/:id/change-domain`.
Режимы LB: `round_robin`, `failover`, `weighted`. В Cloudflare free `weighted`
работает как `round_robin` (одна A на IP). `unknown` **не** считается healthy и
не попадает в пул, пока нет успешных проб; восстановление — `UNHEALTHY → CHECKING → HEALTHY`
после `HEALTH_SUCCESS_RECOVERIES` (default 2). Пороги и cron движка задаются в
работает как `round_robin` (одна A на IP). В A-пул попадает всё, кроме **Down**
(`unknown` / checking / Slow возвращаются в DNS на первой успешной пробе).
Бейдж Healthy — `UNHEALTHY → CHECKING → HEALTHY` после `HEALTH_SUCCESS_RECOVERIES`
(default 2). Пороги и cron движка задаются в
**Настройки → Health-check** (env — fallback, пока значения не сохранены в UI).
### Источники проб: Local, Cloudflare Worker, Globalping
+295 -3
View File
File diff suppressed because one or more lines are too long
+40 -1
View File
@@ -355,6 +355,15 @@ var notificationLog = sqliteTable("notification_log", {
message: text("message").notNull(),
created_at: text("created_at").notNull().default(sql`datetime('now')`)
});
var failoverLog = sqliteTable("failover_log", {
id: integer("id").primaryKey({ autoIncrement: true }),
service_id: integer("service_id").notNull().references(() => services.id, { onDelete: "cascade" }),
binding_id: integer("binding_id").notNull().references(() => serviceBindings.id, { onDelete: "cascade" }),
fqdn: text("fqdn").notNull(),
ip: text("ip").notNull(),
action: text("action").notNull(),
created_at: text("created_at").notNull().default(sql`datetime('now')`)
});
var auditLog = sqliteTable("audit_log", {
id: text("id").primaryKey(),
event_id: text("event_id"),
@@ -395,6 +404,7 @@ var schema = {
domainMonitorResults,
healthProbeLog,
notificationLog,
failoverLog,
auditLog
};
@@ -715,6 +725,7 @@ __export(repos_exports, {
getSyncJob: () => getSyncJob,
insertBinding: () => insertBinding,
insertDnsRecord: () => insertDnsRecord,
insertFailoverLog: () => insertFailoverLog,
insertHealthProbeLog: () => insertHealthProbeLog,
insertNotificationLog: () => insertNotificationLog,
linkBindingRecord: () => linkBindingRecord,
@@ -738,6 +749,7 @@ __export(repos_exports, {
listDomains: () => listDomains,
listDomainsEnriched: () => listDomainsEnriched,
listEnabledDomainMonitors: () => listEnabledDomainMonitors,
listFailoverLogForService: () => listFailoverLogForService,
listGroupDnsRecords: () => listGroupDnsRecords,
listGroups: () => listGroups,
listHealthCheckTargets: () => listHealthCheckTargets,
@@ -1995,6 +2007,12 @@ var WORST_HEALTH_SQL = sql2.raw(`CASE
END) = 1 THEN 'up'
ELSE 'unknown'
END`);
var BEST_ALIVE_HEALTH_SQL = sql2.raw(`CASE
WHEN MAX(CASE WHEN status = 'up' THEN 1 ELSE 0 END) = 1 THEN 'up'
WHEN MAX(CASE WHEN status = 'degraded' THEN 1 ELSE 0 END) = 1 THEN 'degraded'
WHEN MAX(CASE WHEN status = 'down' THEN 1 ELSE 0 END) = 1 THEN 'down'
ELSE 'unknown'
END`);
function aggregateIpHealthByRefs(db, scope, refIds) {
const result = /* @__PURE__ */ new Map();
if (refIds.length === 0) return result;
@@ -2055,7 +2073,7 @@ function aggregateIpHealthByServiceIds(db, serviceIds) {
);
const rows = db.all(sql2`
SELECT sb.service_id AS service_id,
${WORST_HEALTH_SQL} AS health_status,
${BEST_ALIVE_HEALTH_SQL} AS health_status,
MAX(ihs.latency_ms) AS health_latency_ms
FROM ip_health_status ihs
INNER JOIN service_bindings sb
@@ -2545,6 +2563,26 @@ function listNotificationLog(db, limit = 50) {
LIMIT ${limit}
`);
}
function insertFailoverLog(db, input) {
for (const entry of input.entries) {
db.insert(failoverLog).values({
service_id: input.serviceId,
binding_id: input.bindingId,
fqdn: input.fqdn,
ip: entry.ip,
action: entry.action
}).run();
}
}
function listFailoverLogForService(db, serviceId, limit = 100) {
return db.all(sql2`
SELECT id, service_id, binding_id, fqdn, ip, action, created_at
FROM failover_log
WHERE service_id = ${serviceId}
ORDER BY created_at DESC, id DESC
LIMIT ${limit}
`);
}
export {
ConflictError,
NotFoundError,
@@ -2560,6 +2598,7 @@ export {
domainMonitors,
domainTags,
domains,
failoverLog,
getAppSettings,
getAppSettingsSecrets,
groups,
@@ -0,0 +1,11 @@
CREATE TABLE failover_log (
id INTEGER PRIMARY KEY AUTOINCREMENT,
service_id INTEGER NOT NULL REFERENCES services(id) ON DELETE CASCADE,
binding_id INTEGER NOT NULL REFERENCES service_bindings(id) ON DELETE CASCADE,
fqdn TEXT NOT NULL,
ip TEXT NOT NULL,
action TEXT NOT NULL CHECK (action IN ('added', 'removed')),
created_at TEXT NOT NULL DEFAULT (datetime('now'))
);
CREATE INDEX idx_failover_log_service ON failover_log(service_id, created_at DESC);
+166 -2
View File
@@ -49,6 +49,7 @@ import {
nodes,
bindingNodes,
notificationLog,
failoverLog,
serviceBindingIps,
serviceBindingRecords,
serviceBindings,
@@ -2163,6 +2164,14 @@ const WORST_HEALTH_SQL = sql.raw(`CASE
ELSE 'unknown'
END`);
/** Service is up if any binding IP is up. */
const BEST_ALIVE_HEALTH_SQL = sql.raw(`CASE
WHEN MAX(CASE WHEN status = 'up' THEN 1 ELSE 0 END) = 1 THEN 'up'
WHEN MAX(CASE WHEN status = 'degraded' THEN 1 ELSE 0 END) = 1 THEN 'degraded'
WHEN MAX(CASE WHEN status = 'down' THEN 1 ELSE 0 END) = 1 THEN 'down'
ELSE 'unknown'
END`);
/** Worst status across ip_health_status rows for each ref_id in a scope. */
export function aggregateIpHealthByRefs(
db: Db,
@@ -2236,7 +2245,7 @@ export function aggregateGroupScopeHealthByIds(
return result;
}
/** Worst binding-scope health rolled up per service_id. */
/** Binding-scope health rolled up per service_id: any up → service up. */
export function aggregateIpHealthByServiceIds(
db: Db,
serviceIds: number[],
@@ -2253,7 +2262,7 @@ export function aggregateIpHealthByServiceIds(
health_latency_ms: number | null;
}>(sql`
SELECT sb.service_id AS service_id,
${WORST_HEALTH_SQL} AS health_status,
${BEST_ALIVE_HEALTH_SQL} AS health_status,
MAX(ihs.latency_ms) AS health_latency_ms
FROM ip_health_status ihs
INNER JOIN service_bindings sb
@@ -2332,6 +2341,113 @@ export function listIpHealthByServiceIds(
return result;
}
function parseIpHealthState(status: string | null): IpHealthState | undefined {
if (
status === "up" ||
status === "down" ||
status === "degraded" ||
status === "unknown"
) {
return status;
}
return undefined;
}
function bestAliveIpState(statuses: readonly IpHealthState[]): IpHealthState {
if (statuses.some((status) => status === "up")) return "up";
if (statuses.some((status) => status === "degraded")) return "degraded";
if (statuses.some((status) => status === "down")) return "down";
return "unknown";
}
/**
* Latest probe per service+IP+provider from health_probe_log, then any-up per IP.
* Display overlay hysteresis in ip_health_status is unchanged (DNS).
*/
export function listLatestLiveHealthByServiceIds(
db: Db,
serviceIds: number[],
): Map<number, ServiceIpHealthRow[]> {
const result = new Map<number, ServiceIpHealthRow[]>();
if (serviceIds.length === 0) return result;
const idList = sql.join(
serviceIds.map((id) => sql`${id}`),
sql`, `,
);
const rows = db.all<{
service_id: number;
ip: string;
provider: string | null;
status: string | null;
latency_ms: number | null;
last_error: string | null;
colo: string | null;
checked_at: string | null;
}>(sql`
SELECT sb.service_id AS service_id,
l.ip AS ip,
l.provider AS provider,
l.status AS status,
l.latency_ms AS latency_ms,
l.error AS last_error,
l.colo AS colo,
l.checked_at AS checked_at
FROM health_probe_log l
INNER JOIN service_bindings sb
ON l.scope = 'binding' AND l.ref_id = sb.id
INNER JOIN (
SELECT sb2.service_id AS service_id,
l2.ip AS ip,
l2.provider AS provider,
MAX(l2.id) AS max_id
FROM health_probe_log l2
INNER JOIN service_bindings sb2
ON l2.scope = 'binding' AND l2.ref_id = sb2.id
WHERE sb2.service_id IN (${idList})
GROUP BY sb2.service_id, l2.ip, l2.provider
) latest
ON latest.max_id = l.id
`);
const byServiceIp = new Map<
string,
{ serviceId: number; ip: string; probes: ServiceIpHealthRow[] }
>();
for (const row of rows) {
const status = parseIpHealthState(row.status);
if (!status) continue;
const key = `${row.service_id}\0${row.ip}`;
const probe: ServiceIpHealthRow = {
ip: row.ip,
status,
latency_ms: row.latency_ms,
last_checked_at: row.checked_at,
last_error: row.last_error,
provider: normalizeStatusProvider(row.provider),
colo: row.colo,
};
const bucket = byServiceIp.get(key);
if (bucket) bucket.probes.push(probe);
else {
byServiceIp.set(key, {
serviceId: row.service_id,
ip: row.ip,
probes: [probe],
});
}
}
for (const { serviceId, ip, probes } of byServiceIp.values()) {
const status = bestAliveIpState(probes.map((probe) => probe.status));
const preferred =
probes.find((probe) => probe.status === status) ?? probes[0]!;
const list = result.get(serviceId) ?? [];
list.push({ ...preferred, ip, status });
result.set(serviceId, list);
}
return result;
}
export function mergeHealthAggregates(
parts: Array<HealthAggregate | undefined | null>,
): HealthAggregate {
@@ -3055,3 +3171,51 @@ export function listNotificationLog(
`);
}
export type FailoverLogAction = "added" | "removed";
export type FailoverLogRow = {
id: number;
service_id: number;
binding_id: number;
fqdn: string;
ip: string;
action: FailoverLogAction;
created_at: string;
};
export function insertFailoverLog(
db: Db,
input: {
serviceId: number;
bindingId: number;
fqdn: string;
entries: Array<{ ip: string; action: FailoverLogAction }>;
},
): void {
for (const entry of input.entries) {
db.insert(failoverLog)
.values({
service_id: input.serviceId,
binding_id: input.bindingId,
fqdn: input.fqdn,
ip: entry.ip,
action: entry.action,
})
.run();
}
}
export function listFailoverLogForService(
db: Db,
serviceId: number,
limit = 100,
): FailoverLogRow[] {
return db.all<FailoverLogRow>(sql`
SELECT id, service_id, binding_id, fqdn, ip, action, created_at
FROM failover_log
WHERE service_id = ${serviceId}
ORDER BY created_at DESC, id DESC
LIMIT ${limit}
`);
}
+17
View File
@@ -497,6 +497,22 @@ export const notificationLog = sqliteTable("notification_log", {
.default(sql`datetime('now')`),
});
export const failoverLog = sqliteTable("failover_log", {
id: integer("id").primaryKey({ autoIncrement: true }),
service_id: integer("service_id")
.notNull()
.references(() => services.id, { onDelete: "cascade" }),
binding_id: integer("binding_id")
.notNull()
.references(() => serviceBindings.id, { onDelete: "cascade" }),
fqdn: text("fqdn").notNull(),
ip: text("ip").notNull(),
action: text("action").notNull(),
created_at: text("created_at")
.notNull()
.default(sql`datetime('now')`),
});
export const auditLog = sqliteTable("audit_log", {
id: text("id").primaryKey(),
event_id: text("event_id"),
@@ -540,5 +556,6 @@ export const schema = {
domainMonitorResults,
healthProbeLog,
notificationLog,
failoverLog,
auditLog,
};
+40 -1
View File
@@ -143,6 +143,7 @@ interface ServiceDomainBindingView {
fqdn: string;
record_type: "A" | "CNAME";
target_ips: string[];
active_ips: string[];
target_ip_weights: Record<string, number>;
target_ip_priorities: Record<string, number>;
target_cname: string | null;
@@ -505,6 +506,19 @@ declare const healthProbeLogSchema: z.ZodObject<{
checked_at: z.ZodString;
}, z.core.$strip>;
type HealthProbeLog = z.infer<typeof healthProbeLogSchema>;
declare const failoverLogSchema: z.ZodObject<{
id: z.ZodNumber;
service_id: z.ZodNumber;
binding_id: z.ZodNumber;
fqdn: z.ZodString;
ip: z.ZodString;
action: z.ZodEnum<{
added: "added";
removed: "removed";
}>;
created_at: z.ZodString;
}, z.core.$strip>;
type FailoverLogEntry = z.infer<typeof failoverLogSchema>;
declare const groupSchema: z.ZodObject<{
id: z.ZodNumber;
name: z.ZodString;
@@ -643,6 +657,7 @@ declare const serviceDomainBindingSchema: z.ZodPipe<z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -668,6 +683,7 @@ declare const serviceDomainBindingSchema: z.ZodPipe<z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -690,6 +706,7 @@ declare const serviceDomainBindingSchema: z.ZodPipe<z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -763,6 +780,7 @@ declare const serviceViewSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -788,6 +806,7 @@ declare const serviceViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -810,6 +829,7 @@ declare const serviceViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -965,6 +985,7 @@ declare const serviceGroupViewSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -990,6 +1011,7 @@ declare const serviceGroupViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -1012,6 +1034,7 @@ declare const serviceGroupViewSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -1176,6 +1199,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -1201,6 +1225,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -1223,6 +1248,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -1338,6 +1364,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
skipped: "skipped";
}>>;
sync_status: z.ZodDefault<z.ZodNullable<z.ZodString>>;
active_ips: z.ZodDefault<z.ZodArray<z.ZodString>>;
}, z.core.$strip>, z.ZodTransform<{
target_ips: string[];
target_ip_weights: Record<string, number>;
@@ -1363,6 +1390,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ip?: string | null | undefined;
}, {
binding_id: number;
@@ -1385,6 +1413,7 @@ declare const serviceGroupsResponseSchema: z.ZodObject<{
health_check_aggregate: "any" | "all" | "majority";
cert_monitoring: "auto" | "required" | "skipped";
sync_status: string | null;
active_ips: string[];
target_ips?: string[] | undefined;
target_ip?: string | null | undefined;
target_ip_weights?: Record<string, number> | undefined;
@@ -2218,6 +2247,11 @@ declare const cfdmBindingSyncItemSchema: z.ZodObject<{
hostname: z.ZodString;
ips: z.ZodArray<z.ZodString>;
cnameTarget: z.ZodOptional<z.ZodString>;
lbMode: z.ZodOptional<z.ZodEnum<{
round_robin: "round_robin";
failover: "failover";
weighted: "weighted";
}>>;
deleted: z.ZodOptional<z.ZodBoolean>;
}, z.core.$strip>;
declare const cfdmSyncBindingsBodySchema: z.ZodObject<{
@@ -2231,6 +2265,11 @@ declare const cfdmSyncBindingsBodySchema: z.ZodObject<{
hostname: z.ZodString;
ips: z.ZodArray<z.ZodString>;
cnameTarget: z.ZodOptional<z.ZodString>;
lbMode: z.ZodOptional<z.ZodEnum<{
round_robin: "round_robin";
failover: "failover";
weighted: "weighted";
}>>;
deleted: z.ZodOptional<z.ZodBoolean>;
}, z.core.$strip>>;
fullSync: z.ZodOptional<z.ZodBoolean>;
@@ -2412,4 +2451,4 @@ declare const ingestAuditEventSchema: z.ZodObject<{
}, z.core.$strip>;
type IngestAuditEvent = z.infer<typeof ingestAuditEventSchema>;
export { AUDIT_SEVERITIES, AUDIT_SOURCE_APPS, AUDIT_TARGET_TYPES, type AppSettingsPatch, type AppSwitcherConfig, type AppSwitcherEntry, type AuditListQuery, type AuditLogEntry, type AuditSeverity, type AuditSourceApp, type AuditTargetType, type BulkUpdateDomainsInput, CERT_ERROR, CERT_EXPIRED, CERT_MONITORING_VALUES, CERT_MONITOR_AUTO, CERT_MONITOR_REQUIRED, CERT_MONITOR_SKIPPED, CERT_OK, CERT_UNKNOWN, CERT_WARNING, type CertMonitoring, type Certificate, type CfDnsRecord, type CfHealthCheck, type CfZone, type CfdmBindingSyncItem, type ChangeDomainInput, type ChangeIpInput, type CreateDnsRecordInput, type CreateDnsRecordPayload, type CreateDomainInput, type CreateDomainMonitorInput, type CreateGroupInput, type CreateOriginHealthCheckInput, type CreateServiceBindingInput, type CreateServiceGroupInput, type CreateServiceInput, type CreateServiceNodeInput, type CreateServiceWithConfigInput, type CreateSubdomainInput, type DnsRecord, type Domain, type DomainEnvironment, type DomainListItem, type DomainMonitor, type DomainMonitorResult, type DomainMonitorType, type Group, type GroupWithStats, HEALTH_CHECK_AGGREGATES, HEALTH_CHECK_PROVIDERS, HEALTH_KV_CURSOR_KEY, HEALTH_KV_RESULTS_KEY, HEALTH_KV_TARGETS_KEY, HEALTH_PROBE_BATCH, HEALTH_PROBE_CONCURRENCY, HEALTH_PROBE_KV_TITLE, HEALTH_PROBE_SCRIPT_NAME, HEALTH_STATUS_PROVIDERS, type HealthCheckAggregate, type HealthCheckConfig, type HealthCheckProvider, type HealthCheckScope, type HealthCheckTarget, type HealthCheckType, type HealthProbeLog, type HealthProbeResultItem, type HealthProbeResultsDoc, type HealthProbeTargetItem, type HealthProbeTargetsDoc, type HealthStatusProvider, type HealthStatusQuery, type HealthWorkerStatus, type IngestAuditEvent, type IpHealthState, type IpHealthStatus, type JwtClaims, type LbMode, type LoginInput, type LoginRequest, type LoginResponse, type NodeHealthState, type NotificationLog, type OriginHealthCheck, type OriginHealthCheckRecord, type ParsedFqdn, type PatchDnsRecordPayload, type ReorderServicesInput, SYNC_CONFLICT, SYNC_ERROR, SYNC_PENDING_DELETE, SYNC_PENDING_PUSH, SYNC_SYNCED, type Service, type ServiceBinding, type ServiceBindingView, type ServiceCertificateRow, type ServiceDomainBinding, type ServiceDomainBindingView, type ServiceGroup, type ServiceGroupView, type ServiceGroupsResponse, type ServiceIpHealth, type ServiceNode, type ServiceNodeRecord, type ServiceOverview, type ServiceView, type Subdomain, type SubdomainRecord, type SyncJob, type ToggleEnabledInput, type ToggleServiceIpInput, type UpdateDomainInput, type UpdateServiceConfigInput, type UpdateServiceGroupInput, type UpdateServiceNodeInput, type UpdateSubdomainInput, ValidationError, type VpsTrackerEvent, aggregateHealthOk, appSettingsPatchSchema, appSwitcherConfigSchema, appSwitcherEntrySchema, appSwitcherIconSchema, auditListQuerySchema, auditLogEntrySchema, auditSeveritySchema, auditSourceAppSchema, auditTargetTypeSchema, bindingToFqdn, bulkUpdateDomainsSchema, certMonitoringSchema, certStatusFromExpiry, certificateSchema, cfdmBindingSyncItemSchema, cfdmSyncBindingsBodySchema, changeDomainSchema, changeIpSchema, clampGlobalpingLimit, createDnsRecordSchema, createDomainMonitorSchema, createDomainSchema, createGroupSchema, createOriginHealthCheckSchema, createServiceBindingSchema, createServiceGroupSchema, createServiceNodeSchema, createServiceSchema, createServiceWithConfigSchema, createSubdomainSchema, derivePrimaryProvider, dnsNameToSubdomainLabel, dnsRecordNamesMatch, dnsRecordSchema, domainEnvironmentSchema, domainListItemSchema, domainMonitorResultSchema, domainMonitorSchema, domainMonitorTypeSchema, domainSchema, fqdnToDisplay, groupSchema, groupWithStatsSchema, healthCheckAggregateSchema, healthCheckConfigSchema, healthCheckProviderSchema, healthCheckProvidersSchema, healthCheckScopeSchema, healthCheckTypeSchema, healthProbeLogSchema, healthStatusProviderSchema, healthStatusQuerySchema, ingestAuditEventSchema, ipHealthStateSchema, ipHealthStatusSchema, isIpLiteral, isValidIpv4, lbModeSchema, loginSchema, nodeHealthStateSchema, normalizeDnsRecordName, normalizeProbeProvider, normalizeStatusProvider, notificationLogSchema, originHealthCheckSchema, parseFqdn, parseGlobalpingLocations, parseHealthAggregate, parseHealthProviders, reorderServicesSchema, serializeHealthProviders, serviceBindingSchema, serviceCertificateRowSchema, serviceDomainBindingSchema, serviceGroupSchema, serviceGroupTypeSchema, serviceGroupViewSchema, serviceGroupsResponseSchema, serviceIpHealthSchema, serviceNodeSchema, serviceSchema, serviceViewSchema, shouldMonitorService, subdomainLabelToFqdn, subdomainSchema, targetHasProvider, targetProviders, toggleEnabledSchema, toggleServiceIpSchema, uniqueHealthProviders, updateDomainGroupSchema, updateDomainSchema, updateServiceConfigSchema, updateServiceGroupSchema, updateServiceNodeSchema, updateSubdomainSchema, validateDnsRecord, vpsTrackerEventSchema };
export { AUDIT_SEVERITIES, AUDIT_SOURCE_APPS, AUDIT_TARGET_TYPES, type AppSettingsPatch, type AppSwitcherConfig, type AppSwitcherEntry, type AuditListQuery, type AuditLogEntry, type AuditSeverity, type AuditSourceApp, type AuditTargetType, type BulkUpdateDomainsInput, CERT_ERROR, CERT_EXPIRED, CERT_MONITORING_VALUES, CERT_MONITOR_AUTO, CERT_MONITOR_REQUIRED, CERT_MONITOR_SKIPPED, CERT_OK, CERT_UNKNOWN, CERT_WARNING, type CertMonitoring, type Certificate, type CfDnsRecord, type CfHealthCheck, type CfZone, type CfdmBindingSyncItem, type ChangeDomainInput, type ChangeIpInput, type CreateDnsRecordInput, type CreateDnsRecordPayload, type CreateDomainInput, type CreateDomainMonitorInput, type CreateGroupInput, type CreateOriginHealthCheckInput, type CreateServiceBindingInput, type CreateServiceGroupInput, type CreateServiceInput, type CreateServiceNodeInput, type CreateServiceWithConfigInput, type CreateSubdomainInput, type DnsRecord, type Domain, type DomainEnvironment, type DomainListItem, type DomainMonitor, type DomainMonitorResult, type DomainMonitorType, type FailoverLogEntry, type Group, type GroupWithStats, HEALTH_CHECK_AGGREGATES, HEALTH_CHECK_PROVIDERS, HEALTH_KV_CURSOR_KEY, HEALTH_KV_RESULTS_KEY, HEALTH_KV_TARGETS_KEY, HEALTH_PROBE_BATCH, HEALTH_PROBE_CONCURRENCY, HEALTH_PROBE_KV_TITLE, HEALTH_PROBE_SCRIPT_NAME, HEALTH_STATUS_PROVIDERS, type HealthCheckAggregate, type HealthCheckConfig, type HealthCheckProvider, type HealthCheckScope, type HealthCheckTarget, type HealthCheckType, type HealthProbeLog, type HealthProbeResultItem, type HealthProbeResultsDoc, type HealthProbeTargetItem, type HealthProbeTargetsDoc, type HealthStatusProvider, type HealthStatusQuery, type HealthWorkerStatus, type IngestAuditEvent, type IpHealthState, type IpHealthStatus, type JwtClaims, type LbMode, type LoginInput, type LoginRequest, type LoginResponse, type NodeHealthState, type NotificationLog, type OriginHealthCheck, type OriginHealthCheckRecord, type ParsedFqdn, type PatchDnsRecordPayload, type ReorderServicesInput, SYNC_CONFLICT, SYNC_ERROR, SYNC_PENDING_DELETE, SYNC_PENDING_PUSH, SYNC_SYNCED, type Service, type ServiceBinding, type ServiceBindingView, type ServiceCertificateRow, type ServiceDomainBinding, type ServiceDomainBindingView, type ServiceGroup, type ServiceGroupView, type ServiceGroupsResponse, type ServiceIpHealth, type ServiceNode, type ServiceNodeRecord, type ServiceOverview, type ServiceView, type Subdomain, type SubdomainRecord, type SyncJob, type ToggleEnabledInput, type ToggleServiceIpInput, type UpdateDomainInput, type UpdateServiceConfigInput, type UpdateServiceGroupInput, type UpdateServiceNodeInput, type UpdateSubdomainInput, ValidationError, type VpsTrackerEvent, aggregateHealthOk, appSettingsPatchSchema, appSwitcherConfigSchema, appSwitcherEntrySchema, appSwitcherIconSchema, auditListQuerySchema, auditLogEntrySchema, auditSeveritySchema, auditSourceAppSchema, auditTargetTypeSchema, bindingToFqdn, bulkUpdateDomainsSchema, certMonitoringSchema, certStatusFromExpiry, certificateSchema, cfdmBindingSyncItemSchema, cfdmSyncBindingsBodySchema, changeDomainSchema, changeIpSchema, clampGlobalpingLimit, createDnsRecordSchema, createDomainMonitorSchema, createDomainSchema, createGroupSchema, createOriginHealthCheckSchema, createServiceBindingSchema, createServiceGroupSchema, createServiceNodeSchema, createServiceSchema, createServiceWithConfigSchema, createSubdomainSchema, derivePrimaryProvider, dnsNameToSubdomainLabel, dnsRecordNamesMatch, dnsRecordSchema, domainEnvironmentSchema, domainListItemSchema, domainMonitorResultSchema, domainMonitorSchema, domainMonitorTypeSchema, domainSchema, failoverLogSchema, fqdnToDisplay, groupSchema, groupWithStatsSchema, healthCheckAggregateSchema, healthCheckConfigSchema, healthCheckProviderSchema, healthCheckProvidersSchema, healthCheckScopeSchema, healthCheckTypeSchema, healthProbeLogSchema, healthStatusProviderSchema, healthStatusQuerySchema, ingestAuditEventSchema, ipHealthStateSchema, ipHealthStatusSchema, isIpLiteral, isValidIpv4, lbModeSchema, loginSchema, nodeHealthStateSchema, normalizeDnsRecordName, normalizeProbeProvider, normalizeStatusProvider, notificationLogSchema, originHealthCheckSchema, parseFqdn, parseGlobalpingLocations, parseHealthAggregate, parseHealthProviders, reorderServicesSchema, serializeHealthProviders, serviceBindingSchema, serviceCertificateRowSchema, serviceDomainBindingSchema, serviceGroupSchema, serviceGroupTypeSchema, serviceGroupViewSchema, serviceGroupsResponseSchema, serviceIpHealthSchema, serviceNodeSchema, serviceSchema, serviceViewSchema, shouldMonitorService, subdomainLabelToFqdn, subdomainSchema, targetHasProvider, targetProviders, toggleEnabledSchema, toggleServiceIpSchema, uniqueHealthProviders, updateDomainGroupSchema, updateDomainSchema, updateServiceConfigSchema, updateServiceGroupSchema, updateServiceNodeSchema, updateSubdomainSchema, validateDnsRecord, vpsTrackerEventSchema };
+19 -2
View File
@@ -19,7 +19,10 @@ var CERT_MONITORING_VALUES = [
];
// src/validators.ts
var NAME_RE = /^(@|\*|[a-zA-Z0-9_]([a-zA-Z0-9_-]*[a-zA-Z0-9_])?(\.[a-zA-Z0-9_]([a-zA-Z0-9_-]*[a-zA-Z0-9_])?)*)$/;
var LABEL_RE = "[a-zA-Z0-9_](?:[a-zA-Z0-9_-]*[a-zA-Z0-9_])?";
var NAME_RE = new RegExp(
`^(@|\\*|(\\*\\.)?${LABEL_RE}(?:\\.${LABEL_RE})*)$`
);
var IPV4_RE = /^((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)$/;
var IPV6_RE = /^([0-9a-fA-F]{0,4}:){2,7}[0-9a-fA-F]{0,4}$/;
var ALLOWED_TYPES = ["A", "AAAA", "CNAME", "TXT", "MX", "NS", "SRV", "CAA"];
@@ -106,6 +109,7 @@ function dnsNameToSubdomainLabel(recordName, zoneName) {
const prefix = rn.slice(0, rn.length - zoneSuffix.length);
return prefix || "@";
}
if (rn.startsWith("*.")) return rn;
if (!rn.includes(".")) return rn;
return null;
}
@@ -322,6 +326,15 @@ var healthProbeLogSchema = z.object({
error: z.string().nullable(),
checked_at: z.string()
});
var failoverLogSchema = z.object({
id: z.number(),
service_id: z.number(),
binding_id: z.number(),
fqdn: z.string(),
ip: z.string(),
action: z.enum(["added", "removed"]),
created_at: z.string()
});
var groupSchema = z.object({
id: z.number(),
name: z.string(),
@@ -399,7 +412,8 @@ var serviceDomainBindingSchema = z.object({
health_check_providers: healthCheckProvidersSchema.catch(["local"]),
health_check_aggregate: healthCheckAggregateSchema.catch("majority"),
cert_monitoring: certMonitoringSchema.default("auto"),
sync_status: z.string().nullable().default(null)
sync_status: z.string().nullable().default(null),
active_ips: z.array(z.string()).default([])
}).transform((binding) => ({
...binding,
target_ips: binding.target_ips && binding.target_ips.length > 0 ? binding.target_ips : binding.target_ip ? [binding.target_ip] : [],
@@ -836,6 +850,8 @@ var cfdmBindingSyncItemSchema = z3.object({
ips: z3.array(z3.string()),
/** CNAME-цель (FQDN), если binding — CNAME; для матчинга в VPS Tracker. */
cnameTarget: z3.string().optional(),
/** HA-режим binding (fallback — service group). Optional для старых payload. */
lbMode: z3.enum(["round_robin", "failover", "weighted"]).optional(),
deleted: z3.boolean().optional()
});
var cfdmSyncBindingsBodySchema = z3.object({
@@ -1019,6 +1035,7 @@ export {
domainMonitorSchema,
domainMonitorTypeSchema,
domainSchema,
failoverLogSchema,
fqdnToDisplay,
groupSchema,
groupWithStatsSchema,
@@ -11,6 +11,8 @@ export const cfdmBindingSyncItemSchema = z.object({
ips: z.array(z.string()),
/** CNAME-цель (FQDN), если binding — CNAME; для матчинга в VPS Tracker. */
cnameTarget: z.string().optional(),
/** HA-режим binding (fallback — service group). Optional для старых payload. */
lbMode: z.enum(["round_robin", "failover", "weighted"]).optional(),
deleted: z.boolean().optional(),
});
+13
View File
@@ -100,6 +100,18 @@ export const healthProbeLogSchema = z.object({
export type HealthProbeLog = z.infer<typeof healthProbeLogSchema>
export const failoverLogSchema = z.object({
id: z.number(),
service_id: z.number(),
binding_id: z.number(),
fqdn: z.string(),
ip: z.string(),
action: z.enum(['added', 'removed']),
created_at: z.string(),
})
export type FailoverLogEntry = z.infer<typeof failoverLogSchema>
export const groupSchema = z.object({
id: z.number(),
name: z.string(),
@@ -184,6 +196,7 @@ export const serviceDomainBindingSchema = z
health_check_aggregate: healthCheckAggregateSchema.catch('majority'),
cert_monitoring: certMonitoringSchema.default('auto'),
sync_status: z.string().nullable().default(null),
active_ips: z.array(z.string()).default([]),
})
.transform((binding) => ({
...binding,
+1
View File
@@ -19,6 +19,7 @@ export function dnsNameToSubdomainLabel(
return prefix || "@";
}
if (rn.startsWith("*.")) return rn;
if (!rn.includes(".")) return rn;
return null;
+1
View File
@@ -190,6 +190,7 @@ export interface ServiceDomainBindingView {
fqdn: string;
record_type: "A" | "CNAME";
target_ips: string[];
active_ips: string[];
target_ip_weights: Record<string, number>;
target_ip_priorities: Record<string, number>;
target_cname: string | null;
+5 -2
View File
@@ -5,8 +5,11 @@ import {
} from "./constants.js";
import type { ServiceGroup } from "./types.js";
const NAME_RE =
/^(@|\*|[a-zA-Z0-9_]([a-zA-Z0-9_-]*[a-zA-Z0-9_])?(\.[a-zA-Z0-9_]([a-zA-Z0-9_-]*[a-zA-Z0-9_])?)*)$/;
const LABEL_RE = "[a-zA-Z0-9_](?:[a-zA-Z0-9_-]*[a-zA-Z0-9_])?";
/** Apex `@`, zone `*`, labels, or nested wildcard (`*.ndns`, `*.ndns.shnt.top`). */
const NAME_RE = new RegExp(
`^(@|\\*|(\\*\\.)?${LABEL_RE}(?:\\.${LABEL_RE})*)$`,
);
const IPV4_RE =
/^((25[0-5]|2[0-4]\d|[01]?\d\d?)\.){3}(25[0-5]|2[0-4]\d|[01]?\d\d?)$/;
const IPV6_RE = /^([0-9a-fA-F]{0,4}:){2,7}[0-9a-fA-F]{0,4}$/;
+16
View File
@@ -23,6 +23,13 @@ describe("normalizeDnsRecordName", () => {
expect(normalizeDnsRecordName("@", ZONE)).toBe("rkns.top");
expect(normalizeDnsRecordName("rkns.top", ZONE)).toBe("rkns.top");
});
it("normalizes nested wildcard relative name to FQDN", () => {
expect(normalizeDnsRecordName("*.mdns", ZONE)).toBe("*.mdns.rkns.top");
expect(normalizeDnsRecordName("*.mdns.rkns.top", ZONE)).toBe(
"*.mdns.rkns.top",
);
});
});
describe("dnsRecordNamesMatch", () => {
@@ -34,10 +41,19 @@ describe("dnsRecordNamesMatch", () => {
it("does not match different hosts", () => {
expect(dnsRecordNamesMatch("de", "mhome.rkns.top", ZONE)).toBe(false);
});
it("matches nested wildcard relative name and FQDN", () => {
expect(dnsRecordNamesMatch("*.mdns", "*.mdns.rkns.top", ZONE)).toBe(true);
});
});
describe("dnsNameToSubdomainLabel", () => {
it("extracts label from FQDN", () => {
expect(dnsNameToSubdomainLabel("de.rkns.top", ZONE)).toBe("de");
});
it("keeps nested wildcard relative names", () => {
expect(dnsNameToSubdomainLabel("*.mdns", ZONE)).toBe("*.mdns");
expect(dnsNameToSubdomainLabel("*.mdns.rkns.top", ZONE)).toBe("*.mdns");
});
});
+41
View File
@@ -0,0 +1,41 @@
import { describe, expect, it } from "vitest";
import { validateDnsRecord, ValidationError } from "../src/validators.js";
function assertValidName(name: string): void {
expect(() =>
validateDnsRecord("A", name, "1.2.3.4", 1, false),
).not.toThrow();
}
function assertInvalidName(name: string): void {
expect(() => validateDnsRecord("A", name, "1.2.3.4", 1, false)).toThrow(
ValidationError,
);
expect(() => validateDnsRecord("A", name, "1.2.3.4", 1, false)).toThrow(
`invalid record name: ${name}`,
);
}
describe("validateDnsRecord name", () => {
it("accepts apex and zone wildcard", () => {
assertValidName("@");
assertValidName("*");
});
it("accepts regular labels and FQDN", () => {
assertValidName("ndns");
assertValidName("ndns.shnt.top");
});
it("accepts nested wildcard relative name and FQDN", () => {
assertValidName("*.ndns");
assertValidName("*.ndns.shnt.top");
assertValidName("*.mdns");
assertValidName("*.mdns.rkns.top");
});
it("rejects wildcard not as leftmost label", () => {
assertInvalidName("foo.*.bar");
assertInvalidName("ndns.*");
});
});