Hub stage 6: blocking and cache actions on several servers

One endpoint, hub/servers/run (operate), runs a fixed set of actions on
the chosen servers in parallel with each server's admin token: add or
remove a domain in Blocked or Allowed, disable blocking for a time or
enable it, update block lists, replace the block list URLs, flush the
cache or delete a domain from it. Each server gets its own result and
the console offers Retry failed.

In "All servers", Blocking, Cache and Query Logs rows open this flow
instead of being disabled. The hub validates domains, minutes and the
server's list grammar (allow lists with !, switched-off lines with #)
and logs no URL list. Operators can now use the blocking switch.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

hub/README.md

@@ -58,7 +58,7 @@ atomically with mode 0600. The running hub notices a change on the next request
 | `apps-admin` | read, apps.write |
 | `admin` | everything, including audit.read |
 
-`apps.write` is what the app writes below need; `operate` and the rest are still unused. Every response also carries
+`apps.write` is what the app writes below need; `operate` and `admin` are what the per-server and several-server changes need. Every response also carries
 `hubRole` (and `info.hubRole`).
 
 ## Login protection
@@ -86,6 +86,31 @@ Public: `status`, `user/login`. After signing in: `user/session/get`, `user/logo
 
 Everything else answers `403 The hub is read-only.` and is audited.
 
+## Changing several servers at once
+
+`POST hub/servers/run` (`write-all`, role with `operate`) runs one fixed action on the chosen servers, in parallel, each with
+that server's admin token. Parameters: `action`, `targets` (comma separated server ids) and the action's own `domain`,
+`minutes` or `urls` (lines in the grammar of the block lists, up to 300: `<url>`, `!<url>` for an allow list, `#...` for a switched-off list or a heading; http, https or file URLs; a line with a comma is refused). The answer is `{results: [{server, ok, message}]}`,
+one per server; a server with no admin token or one that does not answer fails on its own and the others still run. Empty
+targets or an unknown server are refused with `400`, and so is a bad parameter, before any server is called.
+
+| action | Technitium call | parameters sent |
+|---|---|---|
+| `blocked.add`, `blocked.delete` | `blocked/add`, `blocked/delete` | `domain` |
+| `allowed.add`, `allowed.delete` | `allowed/add`, `allowed/delete` | `domain` |
+| `blocking.disable` | `settings/temporaryDisableBlocking` | `minutes` (1 to 1440) |
+| `blocking.enable` | `settings/set` | `enableBlocking=true` only |
+| `blocking.updateLists` | `settings/forceUpdateBlockLists` | none |
+| `blocking.setListUrls` | `settings/set` | `blockListUrls` only (comma joined) |
+| `cache.flush` | `cache/flush` | none |
+| `cache.delete` | `cache/delete` | `domain` |
+
+The block list URLs are part of blocking, so `blocking.setListUrls` needs `operate` like the rest (the per-server
+`settings/set` of any other field still needs `admin`). `settings/set` with `enableBlocking` as its only field is `operate`
+too, so an operator can use the blocking switch on one server. The audit log gets one line per server with the action, the
+server, the domain or `minutes=N` or `urls=N` and the outcome, never the URL list. On the servers, the hub-admin needs
+Blocked and Allowed Modify and Delete, Cache Delete, and Settings Modify (list URLs, blocking switch).
+
 ## Managing one server
 
 The sections Zones, Cache, Blocking (Blocked and Allowed), DNS Client, Settings and DHCP are managed on ONE server at a

hub/routes.ts

@@ -1,7 +1,8 @@
 /*
 What each `/api/<path>` does and who may call it. `kind` says what it touches:
 `read` (no change), `write-one` (changes one server) and `write-all` (changes every server).
-A path that is not listed is refused. The app writes need `apps.write`.
+A path that is not listed is refused. The app writes need `apps.write`. `hub/servers/run` (blocking, Blocked/Allowed and cache on several
+servers at once) needs `operate`.
 
 The sections a server is managed from (stage 5) are listed by what the console calls. A
 write there goes to ONE server (`node=<id>`) with that server's admin token. Capabilities:
@@ -116,7 +117,7 @@ export const ROUTES: Record<string, Route> = {
   'dnsClient/resolve': readOne(),
   // Settings. `settings/get` (above) is not `perServer`: the header's blocking mark reads it without a node.
   'settings/getTsigKeyNames': readOne(),
-  'settings/set': writeOne('admin'),
+  'settings/set': writeOne('admin'), // see `effectiveRoute` for the blocking switch
   'settings/temporaryDisableBlocking': writeOne('operate'),
   'settings/forceUpdateBlockLists': writeOne('operate'),
   // Logs (no delete).
@@ -156,6 +157,8 @@ export const ROUTES: Record<string, Route> = {
   'hub/apps/import': writeAll('apps.write'),
   'hub/apps/run': writeAll('apps.write'),
   'hub/apps/upload': writeAll('apps.write'),
+  // The same change on several servers, outside Apps (servers-run.ts). One endpoint with a fixed list of actions.
+  'hub/servers/run': writeAll('operate'),
 }
 
 export function routeFor(apiPath: string): Route | undefined {
@@ -173,6 +176,15 @@ export function effectiveRoute(apiPath: string, params: Record<string, string>):
     const flag = (params.import ?? '').trim().toLowerCase()
     if (flag !== '' && flag !== 'false') return writeOne('admin')
   }
+  /*
+  The blocking switch is blocking, not server settings: a `settings/set` that carries nothing but
+  `enableBlocking` needs `operate`, as the temporary switch does. Any other field in the call
+  makes it the usual `admin` write. (`node` and `token` are the hub's own.)
+  */
+  if (apiPath === 'settings/set') {
+    const fields = Object.keys(params).filter((k) => k !== 'node' && k !== 'token')
+    if (fields.length === 1 && fields[0] === 'enableBlocking') return writeOne('operate')
+  }
   return routeFor(apiPath)
 }
 

hub/server-write.test.ts

@@ -196,13 +196,17 @@ const OPERATE = [
   ['allowed/delete', { domain: 'ok.example' }],
   ['settings/temporaryDisableBlocking', { minutes: '5' }],
   ['settings/forceUpdateBlockLists', {}],
+  // The blocking switch alone is blocking, not server settings (routes.ts `effectiveRoute`).
+  ['settings/set', { enableBlocking: 'false' }],
 ] as const
 const ADMIN = [
   ['zones/records/add', { zone: 'example.com', domain: 'www.example.com', type: 'A', ipAddress: '192.0.2.1' }],
   ['zones/delete', { zone: 'example.com' }],
   ['zones/dnssec/sign', { zone: 'example.com', algorithm: 'ECDSA' }],
   ['dhcp/scopes/enable', { name: 'Default' }],
-  ['settings/set', { enableBlocking: 'true' }],
+  ['settings/set', { dnsServerDomain: 'x.example' }],
+  // The switch plus any other field is a settings change.
+  ['settings/set', { enableBlocking: 'true', dnsServerDomain: 'x.example' }],
 ] as const
 
 describe('who may write what', () => {

hub/server.ts

@@ -27,6 +27,7 @@ import { Audit } from './audit.ts'
 import { effectiveRoute, largeBodyLimit, routeFor, type Route } from './routes.ts'
 import { WEB_SERVICE_MESSAGE, isForbiddenSettingsField, redactSettings, safeTarget, webServiceChanges } from './guards.ts'
 import { buildMatrix } from './apps-matrix.ts'
+import { ServersRunner, planAction } from './servers-run.ts'
 import { AppsWriter, actionForPath, type FileAction, type TargetResult } from './apps-write.ts'
 import { TooLarge, parseMultipart, readLimited, type UploadedFile } from './multipart.ts'
 import { AppsConfig, UserError } from './apps-config.ts'
@@ -144,6 +145,7 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
   const limiter = new LoginLimiter()
   const audit = new Audit(config.dataDir)
   const writer = new AppsWriter(config, upstream)
+  const runner = new ServersRunner(config, upstream)
   const appsConfig = new AppsConfig(config, upstream)
   const byId = new Map(config.servers.map((s) => [s.id, s]))
 
@@ -384,6 +386,27 @@ export function createHub(config: HubConfig, users: Users = Users.load(config.da
         await configWrite(res, apiPath, params, who, ip)
         return
       }
+      if (apiPath === 'hub/servers/run') {
+        const planned = planAction(params.action, params)
+        const line = { user: who.user, role: who.role, ip, action: 'error' in planned ? 'servers.run' : `servers.${planned.plan.action}` }
+        if ('error' in planned) {
+          audit.log({ ...line, outcome: 'refused', detail: planned.error })
+          send(res, 400, fail(planned.error))
+          return
+        }
+        const outcome = await runner.run(planned.plan, csv(params.targets))
+        if ('error' in outcome) {
+          audit.log({ ...line, target: planned.plan.target, outcome: 'refused', detail: outcome.error })
+          send(res, 400, fail(outcome.error))
+          return
+        }
+        // One line per server. The target is a domain or a count, never the list of URLs.
+        for (const r of outcome.results) {
+          audit.log({ ...line, server: r.server, target: planned.plan.target, outcome: r.ok ? 'ok' : 'failed', detail: r.message })
+        }
+        send(res, 200, ok({ results: outcome.results }))
+        return
+      }
       if (apiPath === 'hub/apps/run') {
         const action = params.action
         if (action !== 'install-store' && action !== 'update-store' && action !== 'uninstall') {

hub/servers-run.test.ts

@@ -0,0 +1,354 @@
+// @vitest-environment node
+/*
+Stage 6: the same change on several servers at once, outside Apps (`hub/servers/run`). The servers
+are fakes that record how they were called.
+*/
+import http from 'node:http'
+import type { AddressInfo } from 'node:net'
+import { mkdtempSync, rmSync } from 'node:fs'
+import { tmpdir } from 'node:os'
+import path from 'node:path'
+import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest'
+import { parseConfig } from './config.ts'
+import { createHub } from './server.ts'
+import { addUser, writeUsersFile } from './users.ts'
+import { MAX_MINUTES, MAX_LINES, SERVER_ACTIONS, cleanDomain, cleanUrls, planAction } from './servers-run.ts'
+import { routeFor } from './routes.ts'
+
+interface Call {
+  path: string
+  method: string
+  token: string
+  query: URLSearchParams
+  form: URLSearchParams
+}
+
+interface Fake {
+  server: http.Server
+  url: string
+  calls: Call[]
+  failWith: string | null
+  down: boolean
+}
+
+function fake(): Promise<Fake> {
+  return new Promise((resolve) => {
+    const f: Fake = { server: http.createServer(), url: '', calls: [], failWith: null, down: false }
+    f.server.on('request', (req, res) => {
+      const chunks: Buffer[] = []
+      req.on('data', (c: Buffer) => chunks.push(c))
+      req.on('end', () => {
+        if (f.down) return req.socket.destroy()
+        const url = new URL(req.url ?? '/', 'http://fake')
+        const contentType = String(req.headers['content-type'] ?? '')
+        const form = new URLSearchParams(contentType.includes('x-www-form-urlencoded') ? Buffer.concat(chunks).toString('utf8') : '')
+        f.calls.push({ path: url.pathname.replace(/^\/api\//, ''), method: req.method ?? '', token: url.searchParams.get('token') ?? '', query: url.searchParams, form })
+        res.setHeader('content-type', 'application/json')
+        if (f.failWith !== null && req.method === 'POST') return res.end(JSON.stringify({ status: 'error', errorMessage: f.failWith }))
+        res.end(JSON.stringify({ status: 'ok', response: {} }))
+      })
+    })
+    f.server.listen(0, '127.0.0.1', () => {
+      f.url = `http://127.0.0.1:${(f.server.address() as AddressInfo).port}`
+      resolve(f)
+    })
+  })
+}
+
+let a: Fake
+let b: Fake
+let dataDir: string
+let staticDir: string
+let hub: http.Server | undefined
+let base: string
+const tokens: Record<string, string> = {}
+const ROLES = ['viewer', 'operator', 'apps-admin', 'admin']
+
+async function start(extra: Record<string, string> = {}): Promise<void> {
+  hub?.close()
+  hub = createHub(
+    parseConfig({
+      HUB_SERVERS: `a=${a.url},b=${b.url}`,
+      A_TOKEN: 'read-a',
+      B_TOKEN: 'read-b',
+      A_ADMIN_TOKEN: 'admin-a',
+      B_ADMIN_TOKEN: 'admin-b',
+      HUB_DATA: dataDir,
+      HUB_STATIC: staticDir,
+      HUB_CACHE_SECONDS: '0',
+      HUB_TIMEOUT_MS: '2000',
+      ...extra,
+    }),
+  )
+  await new Promise<void>((resolve) => hub!.listen(0, '127.0.0.1', resolve))
+  base = `http://127.0.0.1:${(hub.address() as AddressInfo).port}`
+  for (const role of ROLES) {
+    const res = await fetch(`${base}/api/user/login`, {
+      method: 'POST',
+      headers: { 'content-type': 'application/x-www-form-urlencoded' },
+      body: `user=${role}&pass=password-${role}`,
+    })
+    tokens[role] = ((await res.json()) as { token: string }).token
+  }
+}
+
+async function run(role: string, params: Record<string, string>) {
+  const res = await fetch(`${base}/api/hub/servers/run`, {
+    method: 'POST',
+    headers: { authorization: `Bearer ${tokens[role]}`, 'content-type': 'application/x-www-form-urlencoded' },
+    body: new URLSearchParams(params).toString(),
+  })
+  return { code: res.status, body: (await res.json()) as Record<string, any> }
+}
+
+const writes = (f: Fake) => f.calls.filter((c) => c.method === 'POST')
+
+async function auditEntries(): Promise<Record<string, string>[]> {
+  const res = await fetch(`${base}/api/hub/audit/list?limit=200`, { headers: { authorization: `Bearer ${tokens.admin}` } })
+  return ((await res.json()) as { response: { entries: Record<string, string>[] } }).response.entries
+}
+
+beforeAll(async () => {
+  a = await fake()
+  b = await fake()
+  dataDir = mkdtempSync(path.join(tmpdir(), 'hub-s6-data-'))
+  staticDir = mkdtempSync(path.join(tmpdir(), 'hub-s6-static-'))
+  let users = addUser([], 'viewer', 'viewer', 'password-viewer')
+  users = addUser(users, 'operator', 'operator', 'password-operator')
+  users = addUser(users, 'apps-admin', 'apps-admin', 'password-apps-admin')
+  users = addUser(users, 'admin', 'admin', 'password-admin')
+  writeUsersFile(dataDir, users)
+  await start()
+})
+
+beforeEach(() => {
+  for (const f of [a, b]) {
+    f.calls = []
+    f.failWith = null
+    f.down = false
+  }
+})
+
+afterAll(() => {
+  hub?.close()
+  a.server.close()
+  b.server.close()
+  rmSync(dataDir, { recursive: true, force: true })
+  rmSync(staticDir, { recursive: true, force: true })
+})
+
+describe('the plan of an action', () => {
+  const plan = (action: string, raw: Record<string, string> = {}) => {
+    const out = planAction(action, raw)
+    if ('error' in out) throw new Error(out.error)
+    return out.plan
+  }
+
+  it('maps each action to its Technitium call and sends only what it needs', () => {
+    expect(plan('blocked.add', { domain: 'Ads.Example.' })).toMatchObject({ path: 'blocked/add', params: { domain: 'Ads.Example' } })
+    expect(plan('blocked.delete', { domain: 'ads.example' })).toMatchObject({ path: 'blocked/delete', params: { domain: 'ads.example' } })
+    expect(plan('allowed.add', { domain: 'ok.example' })).toMatchObject({ path: 'allowed/add' })
+    expect(plan('allowed.delete', { domain: 'ok.example' })).toMatchObject({ path: 'allowed/delete' })
+    expect(plan('blocking.disable', { minutes: '15' })).toMatchObject({ path: 'settings/temporaryDisableBlocking', params: { minutes: '15' }, target: 'minutes=15' })
+    expect(plan('blocking.enable')).toMatchObject({ path: 'settings/set', params: { enableBlocking: 'true' } })
+    expect(plan('blocking.updateLists')).toMatchObject({ path: 'settings/forceUpdateBlockLists', params: {} })
+    expect(plan('blocking.setListUrls', { urls: 'https://a.example/x\nhttp://b.example/y' })).toMatchObject({
+      path: 'settings/set',
+      params: { blockListUrls: 'https://a.example/x,http://b.example/y' },
+      target: 'urls=2',
+    })
+    expect(plan('cache.flush')).toMatchObject({ path: 'cache/flush', params: {} })
+    expect(plan('cache.delete', { domain: 'x.example' })).toMatchObject({ path: 'cache/delete', params: { domain: 'x.example' } })
+    expect(SERVER_ACTIONS).toHaveLength(10)
+  })
+
+  it('refuses an action that is not on the list', () => {
+    for (const bad of [undefined, '', 'settings.set', 'zones/delete', 'cache.flush ', '__proto__']) expect(planAction(bad, {})).toHaveProperty('error')
+  })
+
+  it('checks the domain', () => {
+    expect(cleanDomain('example.com')).toBe('example.com')
+    expect(cleanDomain('*.example.com')).toBe('*.example.com')
+    for (const bad of ['', ' ', 'a b.example', 'a..example', '-a.example', 'a.example/x', 'a.example?x=1', 'a,b.example', 'a'.repeat(64) + '.example', ('a.'.repeat(130)) + 'com']) {
+      expect(cleanDomain(bad)).toBeNull()
+      expect(planAction('blocked.add', { domain: bad })).toHaveProperty('error')
+    }
+  })
+
+  it('checks the minutes', () => {
+    for (const bad of [undefined, '', '0', '-5', '1.5', 'abc', '5m', String(MAX_MINUTES + 1), '99999999']) {
+      expect(planAction('blocking.disable', { minutes: bad })).toHaveProperty('error')
+    }
+    expect(planAction('blocking.disable', { minutes: String(MAX_MINUTES) })).toHaveProperty('plan')
+  })
+
+  it('checks the lines', () => {
+    for (const bad of ['', '   \n  ', 'ftp://a.example/x', '!ftp://a.example/x', '! javascript:alert(1)', 'not a url', 'https://a.example/x,y', '# a, b', 'https://a.example/a b', '#' + 'x'.repeat(300)]) {
+      expect(cleanUrls(bad), bad).toHaveProperty('error')
+    }
+    expect(cleanUrls(Array.from({ length: MAX_LINES + 1 }, (_, i) => `https://a.example/${i}`).join('\n'))).toHaveProperty('error')
+  })
+
+  it('takes the grammar of the lists: allow lines, switched-off lines, headings, file addresses', () => {
+    const text = '# Privacy\nhttps://a.example/hosts\n!https://b.example/allow.txt\n#https://c.example/off\n\nfile:///etc/extra.txt\r\n  http://d.example/x  \n'
+    expect(cleanUrls(text)).toEqual({
+      urls: ['# Privacy', 'https://a.example/hosts', '!https://b.example/allow.txt', '#https://c.example/off', 'file:///etc/extra.txt', 'http://d.example/x'],
+    })
+  })
+
+  it('sends the same blockListUrls the console Settings save sends (the lines joined with commas, in order)', () => {
+    const lines = ['# Privacy', 'https://a.example/hosts', '!https://b.example/allow.txt', '#https://c.example/off', '# Ads', 'https://d.example/x']
+    const out = planAction('blocking.setListUrls', { urls: lines.join('\n') })
+    if ('error' in out) throw new Error(out.error)
+    // cleanList in src/screens/settings/model.ts: newlines to commas, repeated commas collapsed, none at the ends.
+    expect(out.plan.params).toEqual({ blockListUrls: lines.join(',') })
+    expect(out.plan.target).toBe('urls=6')
+  })
+})
+
+describe('who may run it', () => {
+  it('is a write-all route for operate', () => {
+    expect(routeFor('hub/servers/run')).toMatchObject({ kind: 'write-all', capability: 'operate' })
+  })
+
+  it('refuses a viewer and an apps admin, calling no server', async () => {
+    for (const role of ['viewer', 'apps-admin']) {
+      const r = await run(role, { action: 'cache.flush', targets: 'a,b' })
+      expect(r.code).toBe(403)
+    }
+    expect(writes(a)).toHaveLength(0)
+    expect(writes(b)).toHaveLength(0)
+  })
+
+  it('lets an operator and an admin run it', async () => {
+    for (const role of ['operator', 'admin']) {
+      const r = await run(role, { action: 'cache.flush', targets: 'a,b' })
+      expect(r.code).toBe(200)
+      expect(r.body.response.results).toHaveLength(2)
+    }
+  })
+
+  it('needs a POST', async () => {
+    const res = await fetch(`${base}/api/hub/servers/run?action=cache.flush&targets=a`, { headers: { authorization: `Bearer ${tokens.operator}` } })
+    expect(res.status).toBe(405)
+    expect(writes(a)).toHaveLength(0)
+  })
+})
+
+describe('running it', () => {
+  it('sends each action to each server with its admin token, parameters in the body', async () => {
+    const cases: [Record<string, string>, string, Record<string, string>][] = [
+      [{ action: 'blocked.add', domain: 'ads.example' }, 'blocked/add', { domain: 'ads.example' }],
+      [{ action: 'blocked.delete', domain: 'ads.example' }, 'blocked/delete', { domain: 'ads.example' }],
+      [{ action: 'allowed.add', domain: 'ok.example' }, 'allowed/add', { domain: 'ok.example' }],
+      [{ action: 'allowed.delete', domain: 'ok.example' }, 'allowed/delete', { domain: 'ok.example' }],
+      [{ action: 'blocking.disable', minutes: '15' }, 'settings/temporaryDisableBlocking', { minutes: '15' }],
+      [{ action: 'blocking.enable' }, 'settings/set', { enableBlocking: 'true' }],
+      [{ action: 'blocking.updateLists' }, 'settings/forceUpdateBlockLists', {}],
+      [{ action: 'blocking.setListUrls', urls: 'https://a.example/x\nhttps://b.example/y' }, 'settings/set', { blockListUrls: 'https://a.example/x,https://b.example/y' }],
+      [{ action: 'cache.flush' }, 'cache/flush', {}],
+      [{ action: 'cache.delete', domain: 'x.example' }, 'cache/delete', { domain: 'x.example' }],
+    ]
+    for (const [params, upstreamPath, sent] of cases) {
+      a.calls = []
+      b.calls = []
+      const r = await run('operator', { ...params, targets: 'a,b' })
+      expect(r.code, params.action).toBe(200)
+      expect(r.body.response.results.map((x: { server: string; ok: boolean }) => [x.server, x.ok])).toEqual([['a', true], ['b', true]])
+      for (const [f, token] of [[a, 'admin-a'], [b, 'admin-b']] as const) {
+        const call = writes(f).filter((c) => c.path !== 'settings/get')
+        expect(call, params.action).toHaveLength(1)
+        expect(call[0].path).toBe(upstreamPath)
+        expect(call[0].token).toBe(token)
+        expect(Object.fromEntries(call[0].form)).toEqual(sent)
+        // Nothing but the token in the URL.
+        expect([...call[0].query.keys()]).toEqual(['token'])
+      }
+    }
+  })
+
+  it('sends only the one setting, and never reads or writes anything else', async () => {
+    await run('operator', { action: 'blocking.setListUrls', urls: 'https://a.example/x', targets: 'a' })
+    expect(a.calls.map((c) => c.path)).toEqual(['settings/set'])
+    expect([...(a.calls[0].form.keys())]).toEqual(['blockListUrls'])
+  })
+
+  it('shows a partial failure per server, never a bare ok', async () => {
+    b.failWith = 'Access was denied to user x for Blocked.'
+    const r = await run('operator', { action: 'blocked.add', domain: 'ads.example', targets: 'a,b' })
+    expect(r.code).toBe(200)
+    const results = r.body.response.results as { server: string; ok: boolean; message: string }[]
+    expect(results.find((x) => x.server === 'a')).toMatchObject({ ok: true })
+    expect(results.find((x) => x.server === 'b')).toMatchObject({ ok: false })
+    expect(results.find((x) => x.server === 'b')?.message).toContain('Access was denied')
+    expect(results.every((x) => typeof x.message === 'string' && x.message !== '')).toBe(true)
+  })
+
+  it('reports a server that does not answer, and still runs the others', async () => {
+    b.down = true
+    const r = await run('operator', { action: 'cache.flush', targets: 'a,b' })
+    const results = r.body.response.results as { server: string; ok: boolean }[]
+    expect(results.map((x) => [x.server, x.ok])).toEqual([['a', true], ['b', false]])
+    expect(writes(a)).toHaveLength(1)
+  })
+
+  it('fails only the target that has no admin token, and one that is unknown refuses the request', async () => {
+    await start({ A_ADMIN_TOKEN: '' })
+    for (const f of [a, b]) f.calls = []
+    const r = await run('operator', { action: 'cache.flush', targets: 'a,b' })
+    expect(r.code).toBe(200)
+    const results = r.body.response.results as { server: string; ok: boolean; message: string }[]
+    expect(results.find((x) => x.server === 'a')).toMatchObject({ ok: false, message: 'no admin token configured' })
+    expect(results.find((x) => x.server === 'b')).toMatchObject({ ok: true })
+    expect(writes(a)).toHaveLength(0)
+    expect(writes(b)).toHaveLength(1)
+    const unknown = await run('operator', { action: 'cache.flush', targets: 'a,nope' })
+    expect(unknown.code).toBe(400)
+    expect(writes(a)).toHaveLength(0)
+    await start()
+  })
+
+  it('refuses empty targets and a bad parameter, calling no server', async () => {
+    const bad: Record<string, string>[] = [
+      { action: 'cache.flush', targets: '' },
+      { action: 'cache.flush' },
+      { action: 'cache.flush', targets: ' , ' },
+      { action: 'blocked.add', domain: 'a b', targets: 'a' },
+      { action: 'blocking.disable', minutes: '0', targets: 'a' },
+      { action: 'blocking.setListUrls', urls: 'ftp://x.example', targets: 'a' },
+      { action: 'settings.set', targets: 'a' },
+    ]
+    for (const params of bad) {
+      const r = await run('operator', params)
+      expect(r.code, JSON.stringify(params)).toBe(400)
+      expect(r.body.status).toBe('error')
+    }
+    expect(a.calls).toHaveLength(0)
+    expect(b.calls).toHaveLength(0)
+  })
+
+  it('runs on a server once even when it is named twice', async () => {
+    await run('operator', { action: 'cache.flush', targets: 'a,a' })
+    expect(writes(a)).toHaveLength(1)
+  })
+})
+
+describe('the audit log', () => {
+  it('has one line per server with the target, and never the URL list', async () => {
+    b.failWith = 'Disk full.'
+    await run('operator', { action: 'blocking.setListUrls', urls: 'https://secret.example/token123/list.txt\nhttps://x.example/y', targets: 'a,b' })
+    await run('operator', { action: 'blocked.add', domain: 'ads.example', targets: 'a,b' })
+    await run('operator', { action: 'blocking.disable', minutes: '15', targets: 'a' })
+    await run('operator', { action: 'cache.flush', targets: '' })
+    const entries = (await auditEntries()).filter((e) => e.action?.startsWith('servers.'))
+    const lists = entries.filter((e) => e.action === 'servers.blocking.setListUrls')
+    expect(lists.map((e) => [e.server, e.outcome, e.target])).toEqual(expect.arrayContaining([['a', 'ok', 'urls=2'], ['b', 'failed', 'urls=2']]))
+    expect(entries.filter((e) => e.action === 'servers.blocked.add').map((e) => [e.server, e.target])).toEqual(expect.arrayContaining([['a', 'ads.example'], ['b', 'ads.example']]))
+    expect(entries.find((e) => e.action === 'servers.blocking.disable')).toMatchObject({ server: 'a', target: 'minutes=15', outcome: 'ok', user: 'operator' })
+    expect(entries.find((e) => e.action === 'servers.cache.flush')).toMatchObject({ outcome: 'refused' })
+    const text = JSON.stringify(entries)
+    expect(text).not.toContain('secret.example')
+    expect(text).not.toContain('token123')
+    expect(text).not.toContain('admin-a')
+  })
+})

hub/servers-run.ts

@@ -0,0 +1,183 @@
+/*
+Actions on several servers at once, outside Apps (stage 6): Blocked and Allowed, the temporary
+blocking switch, the block lists, and the cache. One endpoint, `hub/servers/run`, takes a fixed
+`action` and the action's own parameters, checks them HERE, and runs the one matching Technitium
+call on every chosen server, in parallel, with that server's admin token.
+
+  * `planAction` is pure: the allowlist, the validation, and the path and parameters to send.
+  * `ServersRunner` runs a plan on targets. A server that is unknown or has no admin token fails
+    on its own; the others still run. The answer is one result per server, never a bare "ok".
+
+The block list URLs are part of blocking, so `blocking.setListUrls` needs `operate` like the rest,
+not `admin` like the per-server `settings/set`. Only `blockListUrls` is sent, so nothing else of
+the server's settings is touched, and the plan still goes through the checks of guards.ts.
+*/
+import type { HubConfig } from './config.ts'
+import { isForbiddenSettingsField } from './guards.ts'
+import { Upstream } from './upstream.ts'
+
+export const SERVER_ACTIONS = [
+  'blocked.add',
+  'blocked.delete',
+  'allowed.add',
+  'allowed.delete',
+  'blocking.disable',
+  'blocking.enable',
+  'blocking.updateLists',
+  'blocking.setListUrls',
+  'cache.flush',
+  'cache.delete',
+] as const
+
+export type ServerAction = (typeof SERVER_ACTIONS)[number]
+
+export interface TargetResult {
+  server: string
+  ok: boolean
+  message?: string
+}
+
+export interface Plan {
+  action: ServerAction
+  /* The Technitium endpoint. */
+  path: string
+  /* Exactly what is sent (a form body). */
+  params: Record<string, string>
+  /* What the audit line may say about it: a domain, `minutes=15`, `urls=3`. Never the URLs. */
+  target?: string
+  /* The sentence for a server where it worked. */
+  done: string
+}
+
+export const MAX_MINUTES = 1440
+const MAX_URL_LENGTH = 2000
+
+const LABEL = '[a-z0-9_](?:[a-z0-9_-]{0,61}[a-z0-9_])?'
+const DOMAIN = new RegExp(`^(?:\\*\\.)?(?:${LABEL}\\.)*${LABEL}$`, 'i')
+
+/* A domain as the hub accepts it (a trailing dot is dropped), or null. */
+export function cleanDomain(value: string | undefined): string | null {
+  const domain = (value ?? '').trim().replace(/\.$/, '')
+  return domain.length > 0 && domain.length <= 253 && DOMAIN.test(domain) ? domain : null
+}
+
+/*
+The lines of a block list request, one per line, in the grammar of the servers and of the console's
+Lists tab (src/screens/blocking/list-lines.ts): `<url>` a block list, `!<url>` an allow list,
+`#...` a switched-off list or a comment (a group heading), free text. Each URL is http, https or
+file. Blank lines go. They travel as the console's own Settings save sends them: joined with a
+comma. A comma cannot be told from the separator, so a line that has one is refused, not split.
+*/
+export const MAX_LINES = 300
+export const MAX_COMMENT = 200
+const SCHEMES = new Set(['http:', 'https:', 'file:'])
+
+export function cleanUrls(value: string | undefined): { urls: string[] } | { error: string } {
+  const lines = (value ?? '')
+    .split(/\r?\n/)
+    .map((l) => l.trim())
+    .filter((l) => l !== '')
+  if (lines.length === 0) return { error: 'Give at least one block list URL.' }
+  if (lines.length > MAX_LINES) return { error: `At most ${MAX_LINES} lines.` }
+  for (const line of lines) {
+    if (line.includes(',')) return { error: `A line cannot contain a comma: ${line.slice(0, 60)}` }
+    if (line.startsWith('#')) {
+      if (line.length > MAX_COMMENT) return { error: 'A comment line is too long.' }
+      continue
+    }
+    if (line.length > MAX_URL_LENGTH) return { error: 'A block list URL is too long.' }
+    const text = line.startsWith('!') ? line.slice(1).trim() : line
+    let url: URL
+    try {
+      url = new URL(text)
+    } catch {
+      return { error: `Not a valid http, https or file address: ${line.slice(0, 60)}` }
+    }
+    if (!SCHEMES.has(url.protocol) || /\s/.test(text)) return { error: `Not a valid http, https or file address: ${line.slice(0, 60)}` }
+  }
+  return { urls: lines }
+}
+
+export function planAction(action: string | undefined, raw: Record<string, string | undefined>): { error: string } | { plan: Plan } {
+  if (action === undefined || !(SERVER_ACTIONS as readonly string[]).includes(action)) {
+    return { error: `action must be one of: ${SERVER_ACTIONS.join(', ')}.` }
+  }
+  const kind = action as ServerAction
+  switch (kind) {
+    case 'blocked.add':
+    case 'blocked.delete':
+    case 'allowed.add':
+    case 'allowed.delete':
+    case 'cache.delete': {
+      const domain = cleanDomain(raw.domain)
+      if (domain === null) return { error: 'A valid domain name is required.' }
+      const [list, verb] = kind.split('.')
+      const path = list === 'cache' ? 'cache/delete' : `${list}/${verb}`
+      const done =
+        kind === 'cache.delete' ? 'Deleted from the cache.' : `${verb === 'add' ? 'Added to' : 'Removed from'} ${list === 'blocked' ? 'Blocked' : 'Allowed'}.`
+      return { plan: { action: kind, path, params: { domain }, target: domain, done } }
+    }
+    case 'blocking.disable': {
+      const text = (raw.minutes ?? '').trim()
+      const minutes = /^\d{1,5}$/.test(text) ? Number(text) : 0
+      if (minutes < 1 || minutes > MAX_MINUTES) return { error: `minutes must be a whole number from 1 to ${MAX_MINUTES}.` }
+      return {
+        plan: { action: kind, path: 'settings/temporaryDisableBlocking', params: { minutes: String(minutes) }, target: `minutes=${minutes}`, done: `Blocking disabled for ${minutes} minute(s).` },
+      }
+    }
+    case 'blocking.enable':
+      return settingsPlan(kind, { enableBlocking: 'true' }, undefined, 'Blocking enabled.')
+    case 'blocking.setListUrls': {
+      const cleaned = cleanUrls(raw.urls)
+      if ('error' in cleaned) return cleaned
+      return settingsPlan(kind, { blockListUrls: cleaned.urls.join(',') }, `urls=${cleaned.urls.length}`, `Block list URLs replaced (${cleaned.urls.length} line(s)).`)
+    }
+    case 'blocking.updateLists':
+      return { plan: { action: kind, path: 'settings/forceUpdateBlockLists', params: {}, done: 'Block list update started.' } }
+    case 'cache.flush':
+      return { plan: { action: kind, path: 'cache/flush', params: {}, done: 'Cache flushed.' } }
+  }
+}
+
+/* A `settings/set` of one field. The same checks as the per-server route, applied to what the hub itself builds. */
+function settingsPlan(action: ServerAction, params: Record<string, string>, target: string | undefined, done: string): { error: string } | { plan: Plan } {
+  if (Object.keys(params).some((k) => isForbiddenSettingsField(k) || k.startsWith('webService'))) return { error: 'That setting is not changed from the hub.' }
+  return { plan: { action, path: 'settings/set', params, target, done } }
+}
+
+const reason = (e: unknown): string => (e instanceof Error ? e.message : 'failed')
+
+export class ServersRunner {
+  #config: HubConfig
+  #upstream: Upstream
+
+  constructor(config: HubConfig, upstream: Upstream) {
+    this.#config = config
+    this.#upstream = upstream
+  }
+
+  /*
+  The plan on every target, in parallel. Returns `{ error }` when the request itself is refused
+  (no targets, an unknown server), else one result per target.
+  */
+  async run(plan: Plan, targets: string[]): Promise<{ error: string } | { results: TargetResult[] }> {
+    const known = new Map(this.#config.servers.map((s) => [s.id, s]))
+    const unique = [...new Set(targets)]
+    if (unique.length === 0) return { error: 'Choose at least one server.' }
+    const unknown = unique.filter((t) => !known.has(t))
+    if (unknown.length > 0) return { error: `Unknown server: ${unknown.join(', ')}.` }
+    const results = await Promise.all(
+      unique.map(async (id): Promise<TargetResult> => {
+        const server = known.get(id)!
+        if (server.adminToken === undefined) return { server: id, ok: false, message: 'no admin token configured' }
+        try {
+          await this.#upstream.write(server, plan.path, plan.params)
+          return { server: id, ok: true, message: plan.done }
+        } catch (e) {
+          return { server: id, ok: false, message: reason(e) }
+        }
+      }),
+    )
+    return { results }
+  }
+}

hub/session.ts

@@ -35,9 +35,9 @@ and is always there. Administration (users, SSO, LDAP of each server) is not off
 What a role may change comes from its capabilities, per key. Settings never get `canDelete`
 (that is the backup and restore, which the hub does not forward), nor do the Logs and the
 Dashboard (deleting logs and stats stays in each server's own console). DNS Client needs
-`admin` to modify because its Import writes a zone. The Settings key also gates the
-temporary blocking switch in the console, which the hub itself would let an operator use:
-the console is the stricter of the two there.
+`admin` to modify because its Import writes a zone. The blocking switch and the block list update are blocking,
+so behind a hub the console asks for Blocked/Allowed Modify there (which an operator has), not for Settings Modify (which
+only an admin has). Saving the list URLs of one server stays a settings change.
 */
 const PERMISSION_KEYS = [
   'Dashboard', 'Zones', 'Cache', 'Allowed', 'Blocked', 'Apps', 'DnsClient', 'Settings', 'DhcpServer', 'Administration', 'Logs',