Compare commits

...
Author SHA1 Message Date
DenozordecandClaude Sonnet 5.5 5ec95a7c95 chore(docs): пересмотреть D5, контроллер прогона A на VPS
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 10:20:55 +07:00
DenozordecandClaude Sonnet 5.5 c207e6b3a2 chore(docs): зафиксировать решения D5, D7, D9 и D10 прогона A
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 10:15:58 +07:00
DenozordecandClaude Sonnet 5.5 37196d5d54 chore(docs): зафиксировать решение D4 в плане прогона A
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 10:14:59 +07:00
DenozordecandClaude Sonnet 5.5 0fba78147c chore(docs): зафиксировать решение D3 в плане прогона A
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 10:14:38 +07:00
DenozordecandClaude Sonnet 5.5 cd1b67057d chore(docs): зафиксировать решение D2 в плане прогона A
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 10:13:57 +07:00
DenozordecandClaude Sonnet 5.5 e2d97b2ad7 chore(docs): зафиксировать решение D1 в плане прогона A
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 10:12:10 +07:00
DenozordecandClaude Sonnet 5.5 c62a3ae3e3 chore(docs): добавить план стендового прогона A
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 10:11:29 +07:00
DenozordecandClaude Sonnet 5.5 0cbdc282c4 chore(docs): отметить приёмку Inspector по HTTP и OAuth, закрыть этап 2
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 09:19:27 +07:00
DenozordecandClaude Sonnet 5.5 85a5fdb9aa chore(docs): сгруппировать стендовые приёмки этапа 3 в три прогона
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 09:13:50 +07:00
DenozordecandClaude Sonnet 5.5 44c1036942 chore(docs): отметить решение оставить settings.local.json под Git
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 09:10:28 +07:00
DenozordecandClaude Sonnet 5.5 4b92c20cb5 chore(docs): зафиксировать результат сверки tmp-патчей и черновиков
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 09:08:18 +07:00
DenozordecandClaude Sonnet 5.5 91d5535651 chore(docs): оформить причину unknown из CI2713 как неустановленную
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 09:06:41 +07:00
DenozordecandClaude Sonnet 5.5 6f2e88261f chore(docs): перенести RT-3 в этап 4
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 09:03:02 +07:00
DenozordecandClaude Sonnet 5.5 345288ed12 chore(docs): уточнить статус пунктов VRF, dhcp-client и RT-3 в этапе 2
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 09:02:24 +07:00
DenozordecandClaude Sonnet 5.5 420eb372ae chore(docs): оформить NetFlow wildcard как ограничение Node dgram
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 08:59:05 +07:00
DenozordecandClaude Sonnet 5.5 b0e2912c8e chore(docs): перенести нестабильные сценарии Edge и маршрутизации в этап 3
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 08:54:14 +07:00
DenozordecandClaude Sonnet 5.5 8d97388c40 fix(test): не зависать на закрытии мок-сервера после приёмки native CREATE ID
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 05:27:34 +07:00
DenozordecandClaude Sonnet 5.5 a7db013c7b chore(docs): отметить прогресс этапа 1 в плане MCP
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-16 00:08:05 +07:00
DenozordecandClaude Opus 5.5 77e52f157a fix(test): обновить заглушки standalone-тестов IPsec и восстановления БД
Устаревшие тесты, не дефекты продукта (профили test:ipsec и
test:system-database):
- ipsec-nat и ipsec-group: аудит конфигурации теперь пишется через
  requestAccountingWrite (BEGIN, pg_catalog.pg_try_advisory_xact_lock_shared,
  COMMIT), а SQL-заглушка знала только прежние запросы и падала, превращая
  ожидаемый отказ preflight 409 в 502. Заглушка принимает новые запросы
  блокировки и границы транзакции.
- database-restore-audit: bootstrap.ts теперь импортирует модули карантина и
  контрольной точки восстановления и читает process.env; в карту заглушек
  добавлены значения обычного запуска (без карантина и checkpoint), в
  контекст передан пустой process.env. Проверка порядка (guard → миграции →
  импорт → аудит) не менялась.
Проверено: --all --regression-only для test:ipsec и test:system-database → exit 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 23:38:32 +07:00
DenozordecandClaude Opus 5.5 1094acb5c5 fix(test): совместить перехват аудита с наблюдателем checkpoint и починить сбои purge
Устаревшие тесты, не дефекты продукта.
- Фикстур configuration-audit-commit оборачивал клиента через Proxy; вместе с
  committed-scoped-checkpoint-observer, который подменяет client.query на
  экземпляре, это давало бесконечную рекурсию (RangeError в приёмке scoped
  route community). Теперь фикстур, как и наблюдатель, подменяет query на
  экземпляре и восстанавливает его при release; отключение не затирает более
  поздние обёртки pool.connect.
- Приёмка NetFlow purge внедряла сбой журнала событий через pool.query, а
  событие пишется через withPartitionWrite на клиенте pool.connect(), поэтому
  режим journal-failure не срабатывал. Сбои журнала/обслуживания партиций
  проверяются в обоих перехватчиках; сообщения assert называют режим.
Проверено: node scripts/test-mcp-postgres.mjs --recovery-maintenance → exit 0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 23:15:17 +07:00
DenozordecandClaude Opus 5.5 dfb73555a7 fix(test): исправить устаревшие проверки монолита MCP и добавить режим продолжения
Устаревшие тесты, не дефекты продукта:
- Пауза во время списка шаблонов OSPF: план читает список шаблонов один раз
  (затем два print-запроса флагов, перед каждым повторная проверка доступа),
  поэтому второй итерации «пауза во время второго списка» не существует.
  Диагностический прогон подтвердил одну выборку списка; итерация 0 (пауза
  во время единственного списка → отказ UNAUTHORIZED) сохранена.
- Отказ хранилища аудита перед нативной записью: аудит намерения пишется на
  транзакционном клиенте, перехват pool.query заменён hookConfigurationAudits
  с beforeIntent.

Режим MCP_INTEGRATION_CONTINUE=1: помощник soft() оборачивает только
самостоятельные приёмочные вызовы после блока OSPF (каждый восстанавливает
свои фикстуры в finally), записывает ошибку и продолжает; в конце печатается
список и выставляется exitCode=1. Встроенные проверки монолита и служебные
вызовы (scheduler, ingest, refreshScheduler) не обёрнуты: их сбой искажает
общее состояние для последующих шагов. Без переменной поведение прежнее.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 22:37:53 +07:00
DenozordecandClaude Opus 5.5 268640b9f1 fix(test): перевести перехваты аудита WireGuard import и scoped routing на клиент
Устаревшие тесты, не дефект продукта. Как и в предыдущих исправлениях,
configuration.intent пишется через requestAccountingWrite на клиенте из
pool.connect(), поэтому перехваты pool.query в приёмках WireGuard import
(поздняя защита интерфейса/пира после намерения) и scoped route community
(afterIntent) не срабатывали. Они переведены на afterConfigurationIntentCommit
и hookConfigurationAudits; фикстур получил beforeIntent для имитации
недоступного аудита намерения. Перехват wireguard.import.checkpoint остаётся
на pool.query: этот аудит пишется через пул.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 22:37:53 +07:00
DenozordecandClaude Opus 5.5 84e34da4f9 fix(test): отдавать /interface с интерфейсом шаблона в сценарии passive OSPF
Устаревшая фикстура, не дефект продукта. Сценарий mixed passive добавляет
непассивный шаблон OSPF на fixture-loop без Full-соседа, а мок RouterOS в
этой точке отдаёт пустой /interface. resolveOspfInterfaceSelectors
закрыто отказывает (STATE_CONFLICT), если имя интерфейса шаблона нельзя
подтвердить ни соседом, ни инвентарём, — это корректная защита. На реальном
маршрутизаторе интерфейс шаблона есть в /interface, поэтому для этого цикла
мок отдаёт инвентарь с fixture-gre и fixture-loop. Ожидание теста
(шаблон без Full не блокирует и не входит в план) не менялось.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 22:19:14 +07:00
DenozordecandClaude Opus 5.5 73af143a53 fix(test): перевести оставшиеся перехваты аудита на транзакционный клиент
Устаревшие тесты, не дефект продукта. Аудиты configuration.intent и
configuration.acknowledged пишутся через requestAccountingWrite на клиенте
из pool.connect(), поэтому перехваты pool.query в Legacy OSPF, журнале
событий, IPsec peer update, IPsec native write и native conservation не
срабатывали. Фикстур configuration-audit-commit получил общий примитив
hookConfigurationAudits (после COMMIT намерения / перед подтверждением);
afterConfigurationIntentCommit и loseConfigurationAcknowledgement построены
на нём. Ожидаемые итоги (unknown/partial, без записи и повтора) не менялись;
перехват afterVerified в native conservation остаётся на pool.query, так как
runtime обновляет шаги через пул.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 22:10:01 +07:00
DenozordecandClaude Opus 5.5 07aaa40bff fix(test): удалять сервер-фикстуру после приёмок ARP и IPv6 ND
Устаревший тест, не дефект продукта. Приёмки ARP и IPv6 ND оставляли свою
строку servers (только enabled=false), тогда как все соседние приёмки
удаляют фикстуру. После 88e82702 они создаются стабильно, и лишние строки
доходили до Legacy OSPF, который по задокументированному правилу требует
допуск ко всему каталогу (не более 10 серверов, все в scope). Соединение
приёмки OSPF ограничено серверами [1,2], поэтому каталог отклонялся.
Продуктовое правило не ослабляется: фикстуры удаляются в finally (FK на
servers каскадные; на них ничего позже не ссылается).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 22:10:00 +07:00
DenozordecandClaude Sonnet 5.5 64cfb07c22 chore(test): добавить test:changed для регресса только затронутых тестов
Скрипт берёт изменённые файлы из Git и запускает только тесты, которые на них ссылаются. Тесты на PostgreSQL он не запускает, а выводит нужный npm-профиль.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 21:59:48 +07:00
DenozordecandClaude Opus 5.5 d3a347a8ca fix(test): внедрять позднее изменение GRE после COMMIT аудита намерения
Устаревший тест, не дефект продукта. Аудит configuration.intent для
RouterOS-записи пишется на транзакционном клиенте (requestAccountingWrite),
поэтому перехват pool.query не срабатывал: поздняя пауза, защита или
изменение ревизии не внедрялись, и восстановление законно завершалось
applied. Фикстур afterConfigurationIntentCommit принимает предикат по
payload (method), тест GRE использует его; ожидания (unknown, без записи
в RouterOS) не менялись.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 21:58:29 +07:00
DenozordecandClaude Opus 5.5 95b9d1cd52 fix(test): перехватывать аудит GeoIP на транзакционном клиенте
Устаревший тест, не дефект продукта. Аудиты configuration.intent и
configuration.acknowledged пишутся через requestAccountingWrite на клиенте
из pool.connect(), поэтому перехват pool.query в сценариях поздней отмены
доступа и потерянного подтверждения никогда не срабатывал (revoked=false).
Общий фикстур configuration-audit-commit получил помощник
loseConfigurationAcknowledgement; тест GeoIP использует его и
afterConfigurationIntentCommit. Ожидания продукта (нет повторного запроса,
итог unknown при потере подтверждения) не менялись.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 21:50:13 +07:00
DenozordecandClaude Opus 5.5 c70e55d5d4 fix(test): внедрять позднее изменение ACME после COMMIT аудита намерения
Устаревший тест, не дефект продукта. Аудит configuration.intent теперь
пишется через requestAccountingWrite (транзакционный клиент), а не через
pool.query, поэтому перехват pool.query в режиме late-intent не срабатывал:
конкурентное изменение не внедрялось и запись законно завершалась applied.
Сценарий переведён на общий фикстур afterConfigurationIntentCommit (как в
mcp-acme-test.ts): изменение updated_at после фиксации намерения ловит
fence перед SQL, запись не отправляется, итог unknown без повтора.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-11-15 21:42:45 +07:00
DenozordecandClaude Sonnet 5.5 ad0162d4a2 docs(mcp): отметить lint, сборки, test:mcp и сверку покрытия в плане
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 21:41:42 +07:00
DenozordecandClaude Sonnet 5.5 969d7cd50d fix(test): расширить лимит серверов на план в тесте NetFlow overlay
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 21:33:48 +07:00
DenozordecandClaude Sonnet 5.5 88e82702db fix(test): выбирать свободный id сервера в приёмках IPv6 ND и ARP
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 21:27:36 +07:00
DenozordecandClaude Sonnet 5.5 26cba792a8 fix(test): задать часовой бюджет изменений для OSPF-сценария интеграционного теста
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 21:24:31 +07:00
DenozordecandClaude Sonnet 5.5 44b5066f3a fix(lint): устранить ошибки lint и ожидание host в тесте артефактов логов
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 21:20:17 +07:00
DenozordecandClaude Sonnet 5.5 cf6dcd77fb fix(test): синхронизировать заглушки теста планировщика и правила lint
Тест scheduler-automatic-root отстал от исходников: не хватало заглушек карантина восстановления, общего шлюза БД и проверки активации. ESLint больше не обходит локальные каталоги tmp, сгенерированный .ci/docker и dist-*, ошибки Codex в новых скриптах и тестах устранены.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 21:13:52 +07:00
DenozordecandClaude Sonnet 5.5 eec070d2d6 docs(mcp): записать итог проверки LF/CRLF миграций 0008–0013
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:58:43 +07:00
DenozordecandClaude Sonnet 5.5 c4dc2f95c9 docs(mcp): зафиксировать решение оставить квитанции в docs как есть
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:57:13 +07:00
DenozordecandClaude Sonnet 5.5 98831dcdf4 docs(mcp): отметить копию tmp и итог изучения реорганизации docs
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:56:50 +07:00
DenozordecandClaude Sonnet 5.5 3c78c55c0d fix(docs): сохранить в Git CRLF-байты квитанций, закреплённых SHA-256
28 файлов закреплены хешами по CRLF-байтам рабочего дерева, а в индексе были LF-блобы.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:53:14 +07:00
DenozordecandClaude Sonnet 5.5 d644d0229b docs(mcp): отметить выполнение этапа 0 в плане завершения
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:37:51 +07:00
DenozordecandClaude Sonnet 5.5 42f59f14bf fix(db): сохранить в Git исходные CRLF-байты миграций 0008–0013
Хеши в migration-manifest.ts закреплены за CRLF-байтами; в индексе были LF-блобы из main, и на чистом чекауте проверка манифеста не прошла бы.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:37:27 +07:00
DenozordecandClaude Sonnet 5.5 1d519b1734 docs(mcp): добавить документацию, аудиты вызовов и квитанции приёмки MCP
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:36:15 +07:00
DenozordecandClaude Sonnet 5.5 531d822338 ci: обновить workflow и развёртывание под MCP
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:36:15 +07:00
DenozordecandClaude Sonnet 5.5 6cb550e179 feat(mcp): добавить веб-раздел MCP, клиентский API и e2e-тесты
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:36:14 +07:00
DenozordecandClaude Sonnet 5.5 18c0a52171 feat(mcp): реализовать серверную часть MCP, адаптеры, сервисы и скрипты приёмки
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:36:13 +07:00
DenozordecandClaude Sonnet 5.5 7380f18f15 feat(mcp): добавить контракты, схему БД и миграции MCP
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:36:08 +07:00
DenozordecandClaude Sonnet 5.5 bd4ce59058 chore(git): игнорировать временные артефакты и закрепить байты миграций
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
2026-11-15 20:36:02 +07:00
Denozordec 5efd0339fb feat(logs): добавить раздел логов с историей и живым потоком
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 4m35s
Docker images / frontend-image (push) Successful in 3m29s
Docker images / updater-image (push) Successful in 41s
Docker images / netflow-agent-image (push) Successful in 34s
Docker images / backend-image (push) Successful in 1m44s
Docker images / publish-release (push) Successful in 10s
Docker images / notify-webhook (push) Successful in 2s
2026-10-16 22:13:48 +07:00
Denozordec e461dc21bd fix(traffic): строить непрерывную историю коротких периодов
Docker images / prepare-release (push) Successful in 9s
Docker images / backend-test (push) Successful in 4m3s
Docker images / frontend-image (push) Successful in 2m13s
Docker images / updater-image (push) Successful in 39s
Docker images / netflow-agent-image (push) Successful in 31s
Docker images / backend-image (push) Successful in 1m37s
Docker images / publish-release (push) Successful in 13s
Docker images / notify-webhook (push) Successful in 2s
2026-10-16 19:27:30 +07:00
2762 changed files with 579537 additions and 5868 deletions

No files matched your search

+33
View File
@@ -1,2 +1,35 @@
deploy/netflow-agent/*.sh text eol=lf
deploy/netflow-agent/*.py text eol=lf
# Migration hashes authenticate exact reviewed bytes, including existing CRLF files.
# Preserve those bytes on every checkout instead of applying core.autocrlf.
backend/drizzle/*.sql -text
# Receipts pinned by SHA-256 over their exact CRLF bytes; keep them byte-for-byte.
/docs/bandwidth-operator-effect-ledger.md -text
/docs/bandwidth-operator-metadata-effect-contract.md -text
/docs/mcp-2gb-operating-progress-20261028.json -text
/docs/mcp-chr-ospf-source-cleanup.json -text
/docs/mcp-chr-ssh-forwarding-policy-observation-v1.json -text
/docs/mcp-db-connected-publisher-source-applied-20261102.json -text
/docs/mcp-db-publisher-journal-lease-source-applied-20261102.json -text
/docs/mcp-db-registry-fusion-acceptance-20261101.json -text
/docs/mcp-db-registry-read-contract-acceptance-20261101.json -text
/docs/mcp-guarded-peer-sources-acceptance-20261029.json -text
/docs/mcp-ipsec-user-create-acceptance-v2.json -text
/docs/mcp-linux-inventory-collector-raw-acceptance-20261101.json -text
/docs/mcp-netflow-settings-acceptance-v2.json -text
/docs/mcp-node-key-effect-progress-20261029.json -text
/docs/mcp-operator-peer-metadata-managed-acceptance-20261028.json -text
/docs/mcp-operator-public-metadata-progress-20261028.json -text
/docs/mcp-portal-permission-catalog-source-applied-20261103.json -text
/docs/mcp-public-registry-metadata-acceptance-20261028.json -text
/docs/mcp-publisher-current-operator-linux-acceptance-20261103.json -text
/docs/mcp-routing-manual-inverse-charge-reproduction-20261101.json -text
/docs/mcp-routing-manual-inverse-charge-reproduction-20261101.md -text
/docs/mcp-routing-manual-inverse-six-case-acceptance-20261101.json -text
/docs/mcp-routing-manual-inverse-six-case-acceptance-20261101.md -text
/docs/mcp-routing44-upgrade-bridge-acceptance-20261101.json -text
/docs/mcp-server-poll-expiry-alignment-source-applied-20261102.json -text
/docs/mcp-two-chr-v40-execution-20261106.json -text
/docs/mcp-vrf-production-ticket-acceptance-v1.json -text
/docs/netflow-agent.md -text
+2 -1
View File
@@ -153,8 +153,9 @@ jobs:
NODE
cp packages/contracts/package.json packages/contracts/tsconfig.json "$STAGING/packages/contracts/"
cp -R packages/contracts/src "$STAGING/packages/contracts/"
cp backend/package.json backend/tsconfig.json "$STAGING/backend/"
cp backend/package.json backend/tsconfig.json backend/mcp-coverage.json "$STAGING/backend/"
cp -R backend/src "$STAGING/backend/"
node backend/scripts/stage-database-operator.mjs "$STAGING/backend/scripts"
mkdir -p "$STAGING/deploy/netflow-agent"
cp deploy/netflow-agent/install.sh "$STAGING/deploy/netflow-agent/"
if [ -d backend/drizzle ]; then
+274
View File
@@ -0,0 +1,274 @@
name: MCP isolated acceptance preview images
'on':
workflow_dispatch:
inputs:
expected_sha:
description: Exact reviewed full commit SHA on the selected ref; includes all accepted source changes
required: true
type: string
auth_portal_origin:
description: Canonical isolated browser-facing Auth Portal HTTP(S) origin; no path, credentials or query
required: true
type: string
default: http://127.0.0.1:55175
permissions:
contents: read
packages: write
jobs:
backend-preview:
if: ${{ gitea.event_name == 'workflow_dispatch' }}
runs-on: ubuntu-latest
timeout-minutes: 40
env:
COMPONENT: backend
EXPECTED_SHA: ${{ inputs.expected_sha }}
DISPATCH_SHA: ${{ gitea.sha }}
DISPATCH_EVENT: ${{ gitea.event_name }}
REPOSITORY: ${{ gitea.repository }}
RUN_ID: ${{ gitea.run_id }}
RUN_ATTEMPT: ${{ gitea.run_attempt }}
steps:
- name: Checkout exact dispatch commit
uses: actions/checkout@v4
with:
ref: ${{ gitea.sha }}
persist-credentials: false
- name: Use existing Linux Node22 build toolchain
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Verify source and isolated preview destination
shell: bash
run: |
set -euo pipefail
test "$DISPATCH_EVENT" = workflow_dispatch
[[ "$EXPECTED_SHA" =~ ^[0-9a-f]{40}$ ]]
test "$EXPECTED_SHA" = "$DISPATCH_SHA"
test "$(git rev-parse HEAD)" = "$EXPECTED_SHA"
git diff --exit-code
git diff --cached --exit-code
[[ "$RUN_ID" =~ ^[0-9]+$ && "$RUN_ATTEMPT" =~ ^[0-9]+$ ]]
owner=$(printf '%s' "$REPOSITORY" | cut -d/ -f1 | tr '[:upper:]' '[:lower:]')
name=$(printf '%s' "$REPOSITORY" | cut -d/ -f2 | tr '[:upper:]' '[:lower:]')
[[ "$owner" =~ ^[a-z0-9][a-z0-9._-]*$ && "$name" =~ ^[a-z0-9][a-z0-9._-]*$ ]]
stem=$(printf '%s' "$name" | sed -E 's/-[0-9]+$//')
echo "IMAGE_NAME=git.shx.one/$owner/$stem-$COMPONENT-mcp-acceptance" >> "$GITHUB_ENV"
echo "PREVIEW_TAG=mcp-acceptance-$EXPECTED_SHA-$RUN_ID-$RUN_ATTEMPT" >> "$GITHUB_ENV"
echo "PREVIEW_CACHE=git.shx.one/$owner/$stem-$COMPONENT-mcp-acceptance-buildcache:node-buildcache" >> "$GITHUB_ENV"
mkdir -p .ci/acceptance-evidence
node <<'NODE'
const fs=require('node:fs'),cp=require('node:child_process'),crypto=require('node:crypto');
const files=cp.execFileSync('git',['ls-files','-z'],{encoding:'utf8'}).split('\0').filter(Boolean);
const inputs=files.map(path=>{const info=fs.lstatSync(path);if(!info.isFile()||info.isSymbolicLink())throw Error('Regular tracked source inputs required');const data=fs.readFileSync(path);return {path,sha256:crypto.createHash('sha256').update(data).digest('hex'),bytes:data.length}});
fs.writeFileSync('.ci/acceptance-evidence/source.json',JSON.stringify({format:'mcp-preview-source-v1',repository:process.env.REPOSITORY,sourceSha:process.env.EXPECTED_SHA,sourceTree:cp.execFileSync('git',['rev-parse','HEAD^{tree}'],{encoding:'utf8'}).trim(),platform:'linux/amd64',component:process.env.COMPONENT,runId:process.env.RUN_ID,runAttempt:process.env.RUN_ATTEMPT,authPortalOrigin:process.env.AUTH_PORTAL_ORIGIN??null,inputs},null,2)+'\n');
NODE
- name: Prepare existing scoped Docker context
shell: bash
run: |
set -euo pipefail
STAGING=".ci/docker/backend"
rm -rf "$STAGING"
mkdir -p "$STAGING/packages/contracts" "$STAGING/backend"
cp package-lock.json "$STAGING/"
node <<'NODE'
const fs = require("node:fs")
const pkg = JSON.parse(fs.readFileSync("package.json", "utf8"))
pkg.workspaces = ["packages/*", "backend"]
pkg.dependencies = {}
pkg.devDependencies = {}
delete pkg.scripts
fs.writeFileSync(".ci/docker/backend/package.json", `${JSON.stringify(pkg, null, 2)}\n`)
NODE
cp packages/contracts/package.json packages/contracts/tsconfig.json "$STAGING/packages/contracts/"
cp -R packages/contracts/src "$STAGING/packages/contracts/"
cp backend/package.json backend/tsconfig.json backend/mcp-coverage.json "$STAGING/backend/"
cp -R backend/src "$STAGING/backend/"
mkdir -p "$STAGING/deploy/netflow-agent"
cp deploy/netflow-agent/install.sh "$STAGING/deploy/netflow-agent/"
if [ -d backend/drizzle ]; then
cp -R backend/drizzle "$STAGING/backend/"
fi
node backend/scripts/stage-database-operator.mjs "$STAGING/backend/scripts"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to existing Gitea registry
uses: docker/login-action@v3
with:
registry: git.shx.one
username: ${{ gitea.actor }}
password: ${{ secrets.ACTIONS_PAT }}
- name: Build and push isolated acceptance image
id: build
uses: docker/build-push-action@v5
with:
context: .ci/docker/backend
file: backend/Dockerfile
platforms: linux/amd64
push: true
tags: ${{ env.IMAGE_NAME }}:${{ env.PREVIEW_TAG }}
build-args: APP_VERSION=${{ env.PREVIEW_TAG }}
cache-from: type=gha,scope=mcp-acceptance-backend
cache-to: type=gha,scope=mcp-acceptance-backend,mode=max
provenance: mode=min
sbom: true
labels: |-
org.opencontainers.image.revision=${{ gitea.sha }}
org.opencontainers.image.version=${{ env.PREVIEW_TAG }}
io.mmapp.acceptance-only=true
- name: Record exact digest and build metadata
shell: bash
env:
IMAGE_DIGEST: ${{ steps.build.outputs.digest }}
BUILD_METADATA: ${{ steps.build.outputs.metadata }}
run: |
set -euo pipefail
node <<'NODE'
const fs=require('node:fs'),crypto=require('node:crypto');
const digest=process.env.IMAGE_DIGEST;if(!/^sha256:[a-f0-9]{64}$/.test(digest))throw Error('Exact registry digest required');
const metadata=JSON.parse(process.env.BUILD_METADATA);const source=fs.readFileSync('.ci/acceptance-evidence/source.json');
fs.writeFileSync('.ci/acceptance-evidence/image.json',JSON.stringify({format:'mcp-preview-image-v1',sourceSha:process.env.EXPECTED_SHA,sourceProofSha256:crypto.createHash('sha256').update(source).digest('hex'),image:process.env.IMAGE_NAME,tag:process.env.PREVIEW_TAG,digest,pinnedReference:process.env.IMAGE_NAME+'@'+digest,platform:'linux/amd64',component:process.env.COMPONENT,authPortalOrigin:process.env.AUTH_PORTAL_ORIGIN??null,buildMetadata:metadata,productionDeployed:false,wholeInstallation2GbAccepted:false},null,2)+'\n');
console.log('Acceptance image digest: '+process.env.IMAGE_NAME+'@'+digest);
NODE
- name: Retain source and image binding artifacts
uses: actions/upload-artifact@v3
with:
name: mcp-acceptance-${{ env.COMPONENT }}-${{ gitea.sha }}-${{ gitea.run_id }}-${{ gitea.run_attempt }}
path: .ci/acceptance-evidence/*.json
if-no-files-found: error
retention-days: 14
frontend-preview:
if: ${{ gitea.event_name == 'workflow_dispatch' }}
runs-on: ubuntu-latest
timeout-minutes: 40
env:
COMPONENT: frontend
EXPECTED_SHA: ${{ inputs.expected_sha }}
DISPATCH_SHA: ${{ gitea.sha }}
DISPATCH_EVENT: ${{ gitea.event_name }}
REPOSITORY: ${{ gitea.repository }}
RUN_ID: ${{ gitea.run_id }}
RUN_ATTEMPT: ${{ gitea.run_attempt }}
AUTH_PORTAL_ORIGIN_REQUEST: ${{ inputs.auth_portal_origin }}
steps:
- name: Checkout exact dispatch commit
uses: actions/checkout@v4
with:
ref: ${{ gitea.sha }}
persist-credentials: false
- name: Use existing Linux Node22 build toolchain
uses: actions/setup-node@v4
with:
node-version: '22'
- name: Validate explicit isolated portal origin
shell: bash
run: |
set -euo pipefail
node <<'NODE'
const fs=require('node:fs'),assert=require('node:assert/strict');
const raw=process.env.AUTH_PORTAL_ORIGIN_REQUEST;
assert(typeof raw==='string'&&raw.length>0&&raw.length<=2048&&!/[\x00-\x20\x7f]/.test(raw),'Bounded canonical HTTP(S) origin required');
const origin=new URL(raw);
assert(['http:','https:'].includes(origin.protocol)&&!origin.username&&!origin.password&&origin.pathname==='/'&&!origin.search&&!origin.hash&&origin.origin!=='null'&&origin.origin===raw,'Canonical origin only; credentials/path/query/hash refused');
fs.appendFileSync(process.env.GITHUB_ENV,'AUTH_PORTAL_ORIGIN='+origin.origin+'\n');
NODE
- name: Verify source and isolated preview destination
shell: bash
run: |
set -euo pipefail
test "$DISPATCH_EVENT" = workflow_dispatch
[[ "$EXPECTED_SHA" =~ ^[0-9a-f]{40}$ ]]
test "$EXPECTED_SHA" = "$DISPATCH_SHA"
test "$(git rev-parse HEAD)" = "$EXPECTED_SHA"
git diff --exit-code
git diff --cached --exit-code
[[ "$RUN_ID" =~ ^[0-9]+$ && "$RUN_ATTEMPT" =~ ^[0-9]+$ ]]
owner=$(printf '%s' "$REPOSITORY" | cut -d/ -f1 | tr '[:upper:]' '[:lower:]')
name=$(printf '%s' "$REPOSITORY" | cut -d/ -f2 | tr '[:upper:]' '[:lower:]')
[[ "$owner" =~ ^[a-z0-9][a-z0-9._-]*$ && "$name" =~ ^[a-z0-9][a-z0-9._-]*$ ]]
stem=$(printf '%s' "$name" | sed -E 's/-[0-9]+$//')
echo "IMAGE_NAME=git.shx.one/$owner/$stem-$COMPONENT-mcp-acceptance" >> "$GITHUB_ENV"
echo "PREVIEW_TAG=mcp-acceptance-$EXPECTED_SHA-$RUN_ID-$RUN_ATTEMPT" >> "$GITHUB_ENV"
echo "PREVIEW_CACHE=git.shx.one/$owner/$stem-$COMPONENT-mcp-acceptance-buildcache:node-buildcache" >> "$GITHUB_ENV"
mkdir -p .ci/acceptance-evidence
node <<'NODE'
const fs=require('node:fs'),cp=require('node:child_process'),crypto=require('node:crypto');
const files=cp.execFileSync('git',['ls-files','-z'],{encoding:'utf8'}).split('\0').filter(Boolean);
const inputs=files.map(path=>{const info=fs.lstatSync(path);if(!info.isFile()||info.isSymbolicLink())throw Error('Regular tracked source inputs required');const data=fs.readFileSync(path);return {path,sha256:crypto.createHash('sha256').update(data).digest('hex'),bytes:data.length}});
fs.writeFileSync('.ci/acceptance-evidence/source.json',JSON.stringify({format:'mcp-preview-source-v1',repository:process.env.REPOSITORY,sourceSha:process.env.EXPECTED_SHA,sourceTree:cp.execFileSync('git',['rev-parse','HEAD^{tree}'],{encoding:'utf8'}).trim(),platform:'linux/amd64',component:process.env.COMPONENT,runId:process.env.RUN_ID,runAttempt:process.env.RUN_ATTEMPT,authPortalOrigin:process.env.AUTH_PORTAL_ORIGIN??null,inputs},null,2)+'\n');
NODE
- name: Prepare existing scoped Docker context
shell: bash
run: |
set -euo pipefail
STAGING=".ci/docker/frontend"
rm -rf "$STAGING"
mkdir -p "$STAGING/packages/contracts" "$STAGING/public"
cp package-lock.json next.config.ts tsconfig.json postcss.config.mjs eslint.config.mjs components.json "$STAGING/"
node <<'NODE'
const fs = require("node:fs")
const pkg = JSON.parse(fs.readFileSync("package.json", "utf8"))
pkg.workspaces = ["packages/*"]
fs.writeFileSync(".ci/docker/frontend/package.json", `${JSON.stringify(pkg, null, 2)}\n`)
NODE
cp packages/contracts/package.json packages/contracts/tsconfig.json "$STAGING/packages/contracts/"
cp -R packages/contracts/src "$STAGING/packages/contracts/"
for dir in app components lib shared entities public hooks; do
if [ -d "$dir" ]; then
cp -R "$dir" "$STAGING/"
fi
done
test -f "$STAGING/public/release-manifest.json"
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Log in to existing Gitea registry
uses: docker/login-action@v3
with:
registry: git.shx.one
username: ${{ gitea.actor }}
password: ${{ secrets.ACTIONS_PAT }}
- name: Build and push isolated acceptance image
id: build
uses: docker/build-push-action@v5
with:
context: .ci/docker/frontend
file: Dockerfile.frontend
platforms: linux/amd64
push: true
tags: ${{ env.IMAGE_NAME }}:${{ env.PREVIEW_TAG }}
build-args: |-
BACKEND_INTERNAL_URL=http://backend:8000
NEXT_PUBLIC_BACKEND_URL=same-origin
NEXT_PUBLIC_DEFAULT_DATA_SOURCE=live
NEXT_PUBLIC_ALLOW_MOCK_DATA=false
NEXT_PUBLIC_AUTH_PORTAL_URL=${{ env.AUTH_PORTAL_ORIGIN }}
NEXT_PUBLIC_APP_VERSION=${{ env.PREVIEW_TAG }}
NEXT_PUBLIC_RELEASE_URL=
cache-from: type=gha,scope=mcp-acceptance-frontend
cache-to: type=gha,scope=mcp-acceptance-frontend,mode=max
provenance: mode=min
sbom: true
labels: |-
org.opencontainers.image.revision=${{ gitea.sha }}
org.opencontainers.image.version=${{ env.PREVIEW_TAG }}
io.mmapp.acceptance-only=true
- name: Record exact digest and build metadata
shell: bash
env:
IMAGE_DIGEST: ${{ steps.build.outputs.digest }}
BUILD_METADATA: ${{ steps.build.outputs.metadata }}
run: |
set -euo pipefail
node <<'NODE'
const fs=require('node:fs'),crypto=require('node:crypto');
const digest=process.env.IMAGE_DIGEST;if(!/^sha256:[a-f0-9]{64}$/.test(digest))throw Error('Exact registry digest required');
const metadata=JSON.parse(process.env.BUILD_METADATA);const source=fs.readFileSync('.ci/acceptance-evidence/source.json');
fs.writeFileSync('.ci/acceptance-evidence/image.json',JSON.stringify({format:'mcp-preview-image-v1',sourceSha:process.env.EXPECTED_SHA,sourceProofSha256:crypto.createHash('sha256').update(source).digest('hex'),image:process.env.IMAGE_NAME,tag:process.env.PREVIEW_TAG,digest,pinnedReference:process.env.IMAGE_NAME+'@'+digest,platform:'linux/amd64',component:process.env.COMPONENT,authPortalOrigin:process.env.AUTH_PORTAL_ORIGIN??null,buildMetadata:metadata,productionDeployed:false,wholeInstallation2GbAccepted:false},null,2)+'\n');
console.log('Acceptance image digest: '+process.env.IMAGE_NAME+'@'+digest);
NODE
- name: Retain source and image binding artifacts
uses: actions/upload-artifact@v3
with:
name: mcp-acceptance-${{ env.COMPONENT }}-${{ gitea.sha }}-${{ gitea.run_id }}-${{ gitea.run_attempt }}
path: .ci/acceptance-evidence/*.json
if-no-files-found: error
retention-days: 14
+14
View File
@@ -52,6 +52,8 @@ Thumbs.db
.vercel/
tmp/
temp/
.mcp-state/
backend/.mcp-state/
.ci/docker/
.claude/settings.local.json
@@ -70,3 +72,15 @@ __pycache__/
# NetFlow agent host state (when Compose is installed in a checkout)
/data/netflow-wireguard/
/data/netflow-agent-socket/
# Compiled disposable memory-acceptance entry; not a production image artifact.
backend/dist-mcp-memory/
backend/dist-operator/
# Agent-local permissions can contain commands with deployment credentials.
/.claude/settings.local.json
# Local scratch and disposable build outputs (root-anchored).
/tmp-*
/.next-*/
/.serena/
/backend/dist-*/
+2
View File
@@ -4,6 +4,8 @@
## Содержание
Разрабатываемый MCP для AI-клиентов: [настройка, возможности и текущие ограничения](docs/mcp.md), [полный план реализации](docs/mcp-implementation-plan.md).
1. [Состав монорепозитория](#состав-монорепозитория)
2. [Архитектура](#архитектура)
3. [Локальная разработка](#локальная-разработка)
+4 -1
View File
@@ -58,6 +58,7 @@ import {
} from "@/components/data-grids/data-collection-scheduler-data-grid"
import { DataPageCard } from "@/components/data-page-card"
import { NetflowSettingsPanel } from "@/components/traffic/netflow-settings-panel"
import { LogsCollectorSettings } from "@/components/logs/logs-collector-settings"
import { cn } from "@/lib/utils"
import {
AlertCircleIcon,
@@ -84,6 +85,7 @@ function fmtMs(ms: number): string {
}
function SnapshotTables({ snap }: { snap: SchedulerRunSnapshot }) {
if (snap.job === "router_logs") return <div className="flex flex-col gap-2 text-sm">{snap.skipped && <p>Сбор логов выключен или приостановлен.</p>}{snap.servers.map(server => <p key={server.serverId}>{server.name}: +{server.added} событий{server.gap ? " · Возможен разрыв истории" : ""}{server.error ? " · " + server.error : ""}</p>)}</div>
if (snap.job === "traffic") {
const t = snap as TrafficRunSnapshot
return (
@@ -759,7 +761,7 @@ function DataCollectionContent() {
return SCHEDULER_JOB_KEYS.map((jobKey) => {
const j = schedulerJobsByKey[jobKey]
const settingsKey: SettingsKey = jobKey === "traffic" ? "traffic" : jobKey === "servers_rest_ping" ? "rest" : jobKey.startsWith("uptime_") ? "uptime" : jobKey === "certificates_renew" ? "cert" : "internet"
const fixedSchedule = jobKey === "gre_bgp" || jobKey === "alert_engine" || jobKey === "backups" || jobKey === "geoip_update"
const fixedSchedule = jobKey === "router_logs" || jobKey === "gre_bgp" || jobKey === "alert_engine" || jobKey === "backups" || jobKey === "geoip_update"
const enabled = fixedSchedule
? Boolean(j?.enabled ?? true)
: jobKey === "traffic"
@@ -959,6 +961,7 @@ function DataCollectionContent() {
</Alert>
)}
{isLive && can("mm:settings:admin") && <LogsCollectorSettings key={backendUrl} base={backendUrl} onSaved={settings => setSchedulerStatus(previous => previous ? { ...previous, jobs: previous.jobs.map(job => job.jobKey === "router_logs" ? { ...job, enabled: settings.enabled, intervalSec: settings.intervalSec } : job) } : previous)} />}
{isLive && (
<>
<KpiStatGrid
+58 -19
View File
@@ -53,6 +53,7 @@ import {
patchIpsecUser,
restoreIpsecRevision,
} from "@/shared/api/ipsec"
import type { IpsecExistingCertExportOptions } from "@/shared/api/ipsec"
import type {
IpsecCertBundle,
IpsecCertInfoDto,
@@ -203,7 +204,12 @@ function IpsecPage({ selectedServerId, setSelectedServerId, workspaceTab, setWor
const [certError, setCertError] = useState<string | null>(null)
const certEpoch = useRef(0)
const certPending = useRef(false)
useEffect(() => () => { certEpoch.current += 1 }, [])
const certFailedAttempts = useRef(new Set<string>())
const certActive = useRef<{ targetKey: string; controller: AbortController } | null>(null)
const [certAttemptFailed, setCertAttemptFailed] = useState(false)
const [certRole, setCertRole] = useState<IpsecCertInfoDto["role"]>("client")
const [certSession, setCertSession] = useState(0)
useEffect(() => () => { certEpoch.current += 1; certActive.current?.controller.abort() }, [])
const [search, setSearch] = useState("")
const [initOpen, setInitOpen] = useState(false)
@@ -460,6 +466,9 @@ function IpsecPage({ selectedServerId, setSelectedServerId, workspaceTab, setWor
setCertBundle(res.bundle)
setCertClient(res.client ?? null)
setCertByName(null)
setCertRole("client")
setCertAttemptFailed(false)
setCertSession(session => session + 1)
setCertOpen(true)
}
} finally {
@@ -487,52 +496,71 @@ function IpsecPage({ selectedServerId, setSelectedServerId, workspaceTab, setWor
}
}
const loadCert = async (serverId: string, request: () => Promise<IpsecCertBundle>) => {
const loadCert = async (serverId: string, targetKey: string, request: (signal: AbortSignal) => Promise<IpsecCertBundle>) => {
if (certPending.current) return
certPending.current = true
const controller = new AbortController()
certActive.current = { targetKey, controller }
const epoch = ++certEpoch.current
setCertBusy(true)
setCertError(null)
try {
assertDeviceReady(serverId)
const bundle = await request()
if (epoch === certEpoch.current) setCertBundle(bundle)
const bundle = await request(controller.signal)
if (epoch === certEpoch.current) {
setCertBundle(bundle)
certFailedAttempts.current.delete(targetKey)
setCertAttemptFailed(false)
}
} catch (error) {
if (epoch === certEpoch.current) setCertError(error instanceof Error ? error.message : "Ошибка экспорта сертификата")
certFailedAttempts.current.add(targetKey)
if (epoch === certEpoch.current) {
setCertAttemptFailed(true)
setCertError(error instanceof Error ? error.message : "Ошибка экспорта сертификата")
}
} finally {
if (epoch === certEpoch.current) { certPending.current = false; setCertBusy(false) }
if (epoch === certEpoch.current) { certPending.current = false; certActive.current = null; setCertBusy(false) }
}
}
const openCert = async (client: IpsecClientDto) => {
const byName = client.kind === "cert" && Boolean(client.certName)
const openCert = (client: IpsecClientDto) => {
if (certPending.current) return
const byName = !client.rosId.startsWith("identity:") && client.kind === "cert" && Boolean(client.certName)
const targetKey = JSON.stringify([client.serverId, byName ? "certificate" : "client", byName ? client.certName : client.rosId])
certEpoch.current += 1
certPending.current = false
setCertClient(byName ? null : client)
setCertByName(byName ? { serverId: client.serverId, name: client.certName! } : null)
setCertBundle(null)
setCertError(null)
setCertAttemptFailed(certFailedAttempts.current.has(targetKey))
setCertRole(summaries.find(summary => summary.serverId === client.serverId)?.certs?.find(cert => cert.name === client.certName)?.role ?? "client")
setCertSession(session => session + 1)
setCertOpen(true)
const passphrase = crypto.randomUUID()
await loadCert(client.serverId, () => byName
? exportIpsecCertByName(backendUrl, client.serverId, client.certName!, passphrase)
: exportIpsecUserCert(backendUrl, client.serverId, client.rosId, passphrase))
}
const openCertByName = async (serverId: string, cert: IpsecCertInfoDto) => {
const openCertByName = (serverId: string, cert: IpsecCertInfoDto) => {
if (certPending.current) return
certEpoch.current += 1
certPending.current = false
setCertByName({ serverId, name: cert.name })
setCertClient(null)
setCertBundle(null)
setCertError(null)
setCertAttemptFailed(certFailedAttempts.current.has(JSON.stringify([serverId, "certificate", cert.name])))
setCertRole(cert.role ?? "other")
setCertSession(session => session + 1)
setCertOpen(true)
await loadCert(serverId, () => exportIpsecCertByName(backendUrl, serverId, cert.name, crypto.randomUUID()))
}
const reexportCert = async (passphrase: string) => {
const reexportCert = async (passphrase: string, options: IpsecExistingCertExportOptions) => {
if (!certClient && !certByName) return
await loadCert(certByName?.serverId ?? certClient!.serverId, () => certByName
? exportIpsecCertByName(backendUrl, certByName.serverId, certByName.name, passphrase)
: exportIpsecUserCert(backendUrl, certClient!.serverId, certClient!.rosId, passphrase))
if (options.privateKeyExport !== true || (certRole === "ca" && !options.caPrivateKeyExport)) return
const serverId = certByName?.serverId ?? certClient!.serverId
const targetKey = JSON.stringify([serverId, certByName ? "certificate" : "client", certByName?.name ?? certClient!.rosId])
await loadCert(serverId, targetKey, signal => certByName
? exportIpsecCertByName(backendUrl, certByName.serverId, certByName.name, passphrase, options, { signal })
: exportIpsecUserCert(backendUrl, certClient!.serverId, certClient!.rosId, passphrase, options, { signal }))
}
const deletingCertificateIdentity = pendingDelete?.kind === "user" && pendingDelete.client.kind === "cert" && pendingDelete.client.rosId.startsWith("identity:")
@@ -919,14 +947,25 @@ function IpsecPage({ selectedServerId, setSelectedServerId, workspaceTab, setWor
/>
<IpsecCertSheet
key={certSession}
open={certOpen}
onOpenChange={(v) => {
if (!v) { certEpoch.current += 1; certPending.current = false; setCertBusy(false); setCertBundle(null); setCertClient(null); setCertByName(null) }
if (!v) {
if (certActive.current) {
certFailedAttempts.current.add(certActive.current.targetKey)
certActive.current.controller.abort()
certActive.current = null
}
certEpoch.current += 1; certPending.current = false; setCertBusy(false); setCertBundle(null); setCertClient(null); setCertByName(null)
}
setCertOpen(v)
}}
error={certError}
bundle={certBundle}
busy={certBusy}
certificateRole={certRole}
targetName={certByName?.name ?? certClient?.certName ?? certClient?.name}
previousAttemptFailed={certAttemptFailed}
onReexport={reexportCert}
/>
+2
View File
@@ -0,0 +1,2 @@
import { LogsWorkspace } from "@/components/logs/logs-workspace"
export default function LogsPage() { return <LogsWorkspace /> }
+6
View File
@@ -0,0 +1,6 @@
import { Suspense } from "react"
import { McpWorkspace } from "@/components/mcp/mcp-workspace"
export default function McpPage() {
return <Suspense fallback={<p role="status" className="p-6">Загрузка MCP…</p>}><McpWorkspace /></Suspense>
}
+9 -6
View File
@@ -924,18 +924,21 @@ export default function TrafficPage() {
})
return () => { cancelled = true }
}
void Promise.all([
getTrafficFlows(backendUrl, range),
getFlowExporters(backendUrl, range),
getFlowClients(backendUrl, range),
]).then(([stats, exporters, clients]) => {
const loadFlowCards = async () => {
// These reads share the collector's constrained admission budget.
const stats = await getTrafficFlows(backendUrl, range)
if (cancelled) return
const exporters = await getFlowExporters(backendUrl, range)
if (cancelled) return
const clients = await getFlowClients(backendUrl, range)
if (cancelled) return
setFlowResult({ key: flowKey, receivedAt: Date.now(), stats, exporters: exporters.exporters, clients: clients.clients })
setSelectedId(prev => {
const list = flowScope === "users" ? clients.clients : exporters.exporters
return list.some(x => x.id === prev) ? prev : (list[0]?.id ?? "")
})
}).catch(e => {
}
void loadFlowCards().catch(e => {
if (!cancelled) setFlowResult({ key: flowKey, error: e instanceof Error ? e.message : "Не удалось загрузить потоки" })
})
return () => { cancelled = true }
+4 -2
View File
@@ -74,6 +74,7 @@ function UsersContent() {
const isLive = mode === "live"
const canRead = can("mm:users:read")
const canWrite = can("mm:users:write")
const canManagePermissions = !isLive || can("mm:settings:admin")
const canReadServers = can("mm:servers:read")
const [users, setUsers] = useState<AppUser[]>(() => isLive ? [] : INIT_USERS)
@@ -184,8 +185,8 @@ function UsersContent() {
setSaving(true); setSaveError(null); setOperationNotice(null)
try {
const saved = await saveUserSteps(form, checkpoint, {
create: f => createAppUser(backendUrl, userProfileRequest(f), init()),
update: (id, f) => updateAppUser(backendUrl, id, userProfileRequest(f), init()),
create: f => createAppUser(backendUrl, userProfileRequest(f, canManagePermissions), init()),
update: (id, f) => updateAppUser(backendUrl, id, userProfileRequest(f, canManagePermissions), init()),
read: id => getAppUser(backendUrl, id, init()),
removeBinding: (userId, id) => deleteUserBinding(backendUrl, userId, id, init()),
addBinding: (userId, b) => createUserBinding(backendUrl, userId, {
@@ -328,6 +329,7 @@ function UsersContent() {
users={users}
servers={serverOptions}
isLive={isLive}
canManagePermissions={canManagePermissions}
backendUrl={backendUrl}
saving={saving}
saveError={saveError}
+10 -1
View File
@@ -1,7 +1,16 @@
@import "tailwindcss";
@import "tailwindcss" source(none);
@import "tw-animate-css";
@import "shadcn/tailwind.css";
/* Scan application sources only; local test archives and generated bundles
must not become utility classes in the production stylesheet. */
@source "./";
@source "../components";
@source "../entities";
@source "../hooks";
@source "../lib";
@source "../shared";
/* SVG focus follows the object shape, never the text/foreignObject bounding box. */
.network-map-focusable {
outline: none;
+60
View File
@@ -1,5 +1,33 @@
# PostgreSQL connection string
DATABASE_URL=postgres://mmapp:mmapp@127.0.0.1:5432/mmapp
# Optional operator-approved request and CA for runtime PostgreSQL TLS and SQL identity checks.
# DATABASE_RUNTIME_REQUEST_FILE=/run/config/mmapp/approved-request.json
# DATABASE_RUNTIME_REQUEST_SHA256=<operator-approved SHA-256 of request bytes>
# DATABASE_RUNTIME_CA_FILE=/run/config/mmapp/postgresql-ca.pem
# Optional live operator activation directory (mount the directory read-only).
# DATABASE_RUNTIME_ACTIVATION_DIRECTORY=/var/lib/mmapp/source-activation
# DATABASE_RUNTIME_ACTIVATION_SHA256=<confirmed activation SHA-256>
# Optional operator-approved logical schema SHA-256 from db:inspect-schema.
# DATABASE_RUNTIME_SCHEMA_SHA256=<approved schema SHA-256>
# Restore-aware signed writer profile. Requires source activation, schema pin and restore checkpoint.
# DATABASE_RUNTIME_WRITER_APPROVAL_FILE=/run/config/mmapp/writer-approval.json
# DATABASE_RUNTIME_WRITER_APPROVAL_SHA256=<operator-approved SHA-256 of approval bytes>
# Protected exports use the approved capture role and require this private password file.
# The PostgreSQL TLS peer must present the approved self-signed non-CA leaf.
# DATABASE_ARCHIVE_PASSWORD_FILE=/run/secrets/mmapp_database_archive_password
# Optional owner connection URI file for restore and its schema migrations.
# DATABASE_RESTORE_URL_FILE=/run/secrets/mmapp_database_restore_url
# Opt in on every backend/CLI after provisioning the restore quarantine in mmapp_control.
# DATABASE_RESTORE_QUARANTINE=true
# Optional operator-approved pair for verifying one archive's restored data.
# Requires DATABASE_RESTORE_QUARANTINE and DATABASE_RESTORE_URL_FILE.
# DATABASE_RESTORE_ARCHIVE_SHA256=<approved archive SHA-256>
# DATABASE_RESTORE_DATA_SHA256=<approved data SHA-256>
# Optional checkpoint in an operator-managed directory outside the PostgreSQL and MCP state backups.
# Mount the whole directory read-only. Restore requires a staged pending checkpoint.
# DATABASE_RESTORE_CHECKPOINT_FILE=/run/config/mmapp-restore/database-restore-checkpoint.json
# Local socket served by the operator-side db:serve-restore-claims process.
# DATABASE_RESTORE_CHECKPOINT_CLAIM_SOCKET=/run/mmapp-restore/claims.sock
# SQLite file used only for one-shot boot ETL (empty PG + existing mikrotik.db)
DATABASE_PATH=./mikrotik.db
@@ -17,5 +45,37 @@ AUTH_REQUIRED=false
AUTH_JWT_SECRET=dev-secret-change-me
AUTH_ISSUER=https://auth.shnt.top
AUTH_PORTAL_URL=http://localhost:5175
# Optional portal -> MCP revocation webhook. Separate random secret, at least 32 characters.
# Portal must send signed events to /_internal/mcp/portal/revocations; see docs/mcp.md.
# MCP_PORTAL_WEBHOOK_SECRET=
# MCP_PUBLIC_URL=https://manager.example/mcp
# MCP_STATE_DIR=/var/lib/mikrotik-manager/mcp-state
# Optional separate, externally provisioned bandwidth control source. No automatic
# schema/key/identity creation. See docs/mcp-bandwidth-control-source.md.
# Use the restricted control runtime role, never the owner/application role.
# MMAPP_BANDWIDTH_CONTROL_URL=postgresql://bandwidth_runtime:change-me@control-db:5432/bandwidth_control
# MMAPP_BANDWIDTH_CONTROL_ATTESTATION_FILE=/etc/mmapp/control/source.json
# MMAPP_BANDWIDTH_CONTROL_TRUST_KEY_FILE=/etc/mmapp/control/issuer-public.pem
# SHA256 of the Ed25519 public SPKI DER, pinned outside the attestation itself.
# MMAPP_BANDWIDTH_CONTROL_TRUST_KEY_SHA256=
# MMAPP_BANDWIDTH_CONTROL_EVENT_KEY_FILE=/etc/mmapp/control/event.key
# Separate persistent registry issuer key (raw 32 bytes, different from event.key)
# and operator bootstrap signed by the pinned control-source Ed25519 issuer.
# These inputs grant no actor/network permissions and create no registry rows.
# MMAPP_BANDWIDTH_NODE_ISSUER_KEY_FILE=/etc/mmapp/control/node-issuer.key
# MMAPP_BANDWIDTH_NODE_ISSUER_KEY_SHA256=
# MMAPP_BANDWIDTH_NODE_BOOTSTRAP_FILE=/etc/mmapp/control/node-bootstrap.json
# Optional: enabled by the NetFlow agent Compose installer. Linux Unix socket only.
# FLOW_AGENT_SOCKET=/run/mm-netflow-host/agent.sock
# Optional UDP NetFlow v9/IPFIX template lifetime in milliseconds (restart required).
# Collector policy defaults to 90 minutes, not a RouterOS default. Set at least
# 3 times the longest configured exporter template refresh interval; data does
# not refresh templates. Missing/expired templates refuse decoding, not zero traffic.
# FLOW_TEMPLATE_LIFETIME_MS=5400000
# Optional NetFlow read admission: standard (default) or constrained.
# Constrained admits two responses (UI + AI), limits SQL sources to 2 MiB,
# cumulative sources to 4 MiB and encoded responses to 2 MiB; large reads refuse.
# This is not a complete installation memory limit. See docs/mcp-2gb-resource-acceptance.md.
# MM_FLOW_READ_RESOURCE_PROFILE=standard
+2
View File
@@ -55,7 +55,9 @@ COPY --from=build /app/package.json /app/package-lock.json ./
COPY --from=build /app/node_modules ./node_modules
COPY --from=build /app/packages/contracts ./packages/contracts
COPY --from=build /app/backend/dist ./backend/dist
COPY --from=build /app/backend/dist-operator ./backend/dist-operator
COPY --from=build /app/backend/package.json ./backend/package.json
COPY --from=build /app/backend/mcp-coverage.json ./backend/mcp-coverage.json
COPY --from=build /app/backend/drizzle ./backend/drizzle
COPY deploy/netflow-agent/install.sh ./deploy/netflow-agent/install.sh
# Nested install from lockfile (dotenv etc.) — ESM resolves from /app/backend/dist → ../node_modules
+26
View File
@@ -0,0 +1,26 @@
CREATE TABLE IF NOT EXISTS router_log_settings (
id INTEGER PRIMARY KEY CHECK (id = 1), enabled BOOLEAN NOT NULL DEFAULT false,
interval_sec INTEGER NOT NULL DEFAULT 10 CHECK (interval_sec BETWEEN 5 AND 3600),
retention_days INTEGER NOT NULL DEFAULT 14 CHECK (retention_days BETWEEN 1 AND 365)
);
INSERT INTO router_log_settings(id) VALUES (1) ON CONFLICT DO NOTHING;
-- No foreign key: collected history survives device deletion.
CREATE TABLE IF NOT EXISTS router_log_collection_state (
server_id INTEGER PRIMARY KEY, server_name TEXT NOT NULL, host TEXT NOT NULL,
generation INTEGER NOT NULL DEFAULT 0, snapshot JSONB NOT NULL DEFAULT '[]',
last_collected_at TIMESTAMPTZ, last_error TEXT, gap_at TIMESTAMPTZ,
next_poll_at TIMESTAMPTZ, failures INTEGER NOT NULL DEFAULT 0
);
CREATE TABLE IF NOT EXISTS router_log_entries (
id BIGINT GENERATED ALWAYS AS IDENTITY PRIMARY KEY, server_id INTEGER NOT NULL,
server_name TEXT NOT NULL, host TEXT NOT NULL, router_id TEXT NOT NULL, raw_time TEXT NOT NULL,
event_at TIMESTAMPTZ, received_at TIMESTAMPTZ NOT NULL,
effective_at TIMESTAMPTZ NOT NULL, topics TEXT[] NOT NULL, level TEXT NOT NULL,
message TEXT NOT NULL, generation INTEGER NOT NULL
);
CREATE INDEX IF NOT EXISTS router_logs_server_time ON router_log_entries(server_id, effective_at DESC, id DESC);
CREATE INDEX IF NOT EXISTS router_logs_time ON router_log_entries(effective_at DESC, id DESC);
CREATE INDEX IF NOT EXISTS router_logs_level_time ON router_log_entries(level, effective_at DESC);
CREATE INDEX IF NOT EXISTS router_logs_topics ON router_log_entries USING gin(topics);
CREATE INDEX IF NOT EXISTS router_logs_received ON router_log_entries(received_at);
CREATE INDEX IF NOT EXISTS router_logs_server_cursor ON router_log_entries(server_id, id);
+101
View File
@@ -0,0 +1,101 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_settings (
id integer PRIMARY KEY CHECK(id=1), enabled boolean NOT NULL DEFAULT false,
requests_retention_days integer NOT NULL DEFAULT 30 CHECK(requests_retention_days BETWEEN 1 AND 365),
audit_retention_days integer NOT NULL DEFAULT 180 CHECK(audit_retention_days BETWEEN 30 AND 3650),
updated_at timestamptz NOT NULL DEFAULT now()
);
INSERT INTO mcp_settings(id) VALUES(1) ON CONFLICT DO NOTHING;
CREATE TABLE IF NOT EXISTS mcp_connections (
id uuid PRIMARY KEY, name text NOT NULL, owner_id text NOT NULL, owner_permissions jsonb NOT NULL,
policy jsonb NOT NULL, generation text NOT NULL, created_at timestamptz NOT NULL DEFAULT now(),
updated_at timestamptz NOT NULL DEFAULT now(), last_used_at timestamptz
);
CREATE TABLE IF NOT EXISTS mcp_credentials (
id uuid PRIMARY KEY, connection_id uuid NOT NULL REFERENCES mcp_connections(id) ON DELETE CASCADE,
digest text NOT NULL UNIQUE, kind text NOT NULL CHECK(kind IN ('bearer','access','refresh')),
expires_at timestamptz NOT NULL, revoked_at timestamptz, created_at timestamptz NOT NULL DEFAULT now(),
family_id uuid, client_id text, audience text
);
CREATE INDEX IF NOT EXISTS mcp_credentials_connection ON mcp_credentials(connection_id);
CREATE TABLE IF NOT EXISTS mcp_audit (
id bigint GENERATED ALWAYS AS IDENTITY PRIMARY KEY, actor_id text NOT NULL, action text NOT NULL,
connection_id uuid, payload jsonb NOT NULL DEFAULT '{}', created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS mcp_audit_time ON mcp_audit(created_at,id);
CREATE TABLE IF NOT EXISTS mcp_requests (
id uuid PRIMARY KEY, connection_id uuid REFERENCES mcp_connections(id) ON DELETE SET NULL,
actor_id text, tool text NOT NULL, server_ids bigint[] NOT NULL DEFAULT '{}',
status text NOT NULL, input jsonb NOT NULL DEFAULT '{}', result jsonb, error_code text,
duration_ms integer, created_at timestamptz NOT NULL DEFAULT now(), finished_at timestamptz
);
CREATE INDEX IF NOT EXISTS mcp_requests_connection_time ON mcp_requests(connection_id,created_at,id);
CREATE INDEX IF NOT EXISTS mcp_requests_server ON mcp_requests USING gin(server_ids);
CREATE TABLE IF NOT EXISTS mcp_protections (
id uuid PRIMARY KEY, object_ref jsonb NOT NULL, reason text NOT NULL DEFAULT '',
created_by text NOT NULL, created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS mcp_plans (
id uuid PRIMARY KEY, connection_id uuid NOT NULL REFERENCES mcp_connections(id), operation_id text NOT NULL,
operation_version integer NOT NULL, policy_version integer NOT NULL, generation text NOT NULL,
payload jsonb NOT NULL, evidence jsonb NOT NULL, state text NOT NULL, expires_at timestamptz NOT NULL,
approved_by text, approved_at timestamptz, created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS mcp_plans_connection ON mcp_plans(connection_id,created_at);
CREATE TABLE IF NOT EXISTS mcp_operations (
id uuid PRIMARY KEY, plan_id uuid NOT NULL UNIQUE REFERENCES mcp_plans(id),
connection_id uuid NOT NULL REFERENCES mcp_connections(id), idempotency_key text NOT NULL,
state text NOT NULL DEFAULT 'pending', verification text NOT NULL DEFAULT 'pending',
result jsonb, error_code text, created_at timestamptz NOT NULL DEFAULT now(), finished_at timestamptz,
UNIQUE(connection_id,idempotency_key)
);
CREATE TABLE IF NOT EXISTS mcp_operation_steps (
operation_id uuid NOT NULL REFERENCES mcp_operations(id), step_index integer NOT NULL,
server_id bigint, state text NOT NULL, before_state jsonb, intended_state jsonb, result jsonb,
created_at timestamptz NOT NULL DEFAULT now(), finished_at timestamptz,
PRIMARY KEY(operation_id,step_index)
);
CREATE TABLE IF NOT EXISTS mcp_budget_buckets (
connection_id uuid NOT NULL REFERENCES mcp_connections(id) ON DELETE CASCADE,
kind text NOT NULL, bucket timestamptz NOT NULL, used integer NOT NULL CHECK(used>=0),
PRIMARY KEY(connection_id,kind,bucket)
);
CREATE TABLE IF NOT EXISTS mcp_read_leases (
id uuid PRIMARY KEY, connection_id uuid NOT NULL REFERENCES mcp_connections(id) ON DELETE CASCADE,
expires_at timestamptz NOT NULL
);
CREATE TABLE IF NOT EXISTS mcp_grants (
id uuid PRIMARY KEY, connection_id uuid NOT NULL REFERENCES mcp_connections(id) ON DELETE CASCADE,
payload jsonb NOT NULL, revoked_at timestamptz
);
CREATE TABLE IF NOT EXISTS mcp_secrets (
id uuid PRIMARY KEY, purpose text NOT NULL, server_id bigint, ciphertext text NOT NULL,
created_by text NOT NULL, created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS mcp_artifacts (
id uuid PRIMARY KEY, connection_id uuid NOT NULL REFERENCES mcp_connections(id),
filename text NOT NULL, media_type text NOT NULL, ciphertext text NOT NULL,
expires_at timestamptz NOT NULL, created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS mcp_oauth_clients (
id text PRIMARY KEY, name text NOT NULL, redirect_uris jsonb NOT NULL, enabled boolean NOT NULL DEFAULT true,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS mcp_oauth_requests (
id uuid PRIMARY KEY, client_id text NOT NULL REFERENCES mcp_oauth_clients(id),
redirect_uri text NOT NULL, challenge text NOT NULL, state text, resource text NOT NULL,
scope text NOT NULL, expires_at timestamptz NOT NULL, consumed_at timestamptz,
connection_id uuid REFERENCES mcp_connections(id), code_digest text UNIQUE
);
CREATE TABLE IF NOT EXISTS mcp_routing_profiles (
id uuid PRIMARY KEY, connection_id uuid NOT NULL REFERENCES mcp_connections(id),
version integer NOT NULL DEFAULT 1, config jsonb NOT NULL, state jsonb NOT NULL DEFAULT '{}',
enabled boolean NOT NULL DEFAULT false, grant_expires_at timestamptz NOT NULL,
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS mcp_drift (
id uuid PRIMARY KEY, operation_id uuid NOT NULL REFERENCES mcp_operations(id),
server_id bigint, expected jsonb NOT NULL, observed jsonb NOT NULL,
acknowledged_at timestamptz, detected_at timestamptz NOT NULL DEFAULT now()
);
COMMIT;
@@ -0,0 +1,12 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_routing_observations (
id uuid PRIMARY KEY, src_server_id bigint NOT NULL, dst_server_id bigint NOT NULL,
src_interface text NOT NULL, dst_interface text NOT NULL, family text NOT NULL CHECK(family IN ('ipv4','ipv6')),
routing_table text NOT NULL DEFAULT 'main', ping_ms double precision, loss_pct double precision,
dl_mbps double precision, ul_mbps double precision, ping_at timestamptz, bandwidth_at timestamptz,
context_verified boolean NOT NULL DEFAULT false, source text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(), CHECK(ping_ms IS NULL OR ping_ms>=0),
CHECK(loss_pct IS NULL OR loss_pct BETWEEN 0 AND 100), CHECK(dl_mbps IS NULL OR dl_mbps>=0), CHECK(ul_mbps IS NULL OR ul_mbps>=0)
);
CREATE INDEX IF NOT EXISTS mcp_routing_observations_leg ON mcp_routing_observations(src_server_id,dst_server_id,src_interface,family,routing_table,created_at DESC);
COMMIT;
+7
View File
@@ -0,0 +1,7 @@
BEGIN;
ALTER TABLE mcp_routing_profiles ADD COLUMN IF NOT EXISTS credential_id uuid REFERENCES mcp_credentials(id);
ALTER TABLE mcp_routing_profiles ADD COLUMN IF NOT EXISTS generation text;
ALTER TABLE mcp_routing_profiles ADD COLUMN IF NOT EXISTS policy_version integer;
CREATE INDEX IF NOT EXISTS mcp_routing_profiles_enabled ON mcp_routing_profiles(updated_at) WHERE enabled;
CREATE UNIQUE INDEX IF NOT EXISTS mcp_drift_open_step ON mcp_drift(operation_id,(expected->>'stepIndex')) WHERE acknowledged_at IS NULL;
COMMIT;
@@ -0,0 +1,3 @@
BEGIN;
ALTER TABLE mcp_connections ADD COLUMN IF NOT EXISTS owner_valid_until timestamptz NOT NULL DEFAULT now();
COMMIT;
+7
View File
@@ -0,0 +1,7 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_operation_checkpoints (
operation_id uuid NOT NULL REFERENCES mcp_operations(id), step_index integer NOT NULL,
before_sealed text NOT NULL, after_sealed text, created_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY(operation_id,step_index)
);
COMMIT;
+8
View File
@@ -0,0 +1,8 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_management_channels (
id uuid PRIMARY KEY, server_id bigint NOT NULL UNIQUE REFERENCES servers(id) ON DELETE CASCADE,
config_sealed text NOT NULL, identity_hash text NOT NULL, attestation text NOT NULL,
created_by text NOT NULL, generation text NOT NULL, expires_at timestamptz NOT NULL,
verified_at timestamptz NOT NULL DEFAULT now(), revoked_at timestamptz
);
COMMIT;
+8
View File
@@ -0,0 +1,8 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_path_checks (
id uuid PRIMARY KEY, connection_id uuid NOT NULL REFERENCES mcp_connections(id),
profile_hash text NOT NULL, candidate_id text NOT NULL, status text NOT NULL CHECK(status IN ('passed','failed','unknown')),
evidence jsonb NOT NULL, created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS mcp_path_checks_latest ON mcp_path_checks(connection_id,profile_hash,candidate_id,created_at DESC);
COMMIT;
@@ -0,0 +1,7 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_recovery_intents (
operation_id uuid PRIMARY KEY REFERENCES mcp_operations(id), state text NOT NULL,
authority_profile_id uuid NOT NULL REFERENCES mcp_routing_profiles(id),
created_at timestamptz NOT NULL DEFAULT now(), finished_at timestamptz
);
COMMIT;
@@ -0,0 +1,6 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_unauthorized_buckets (
bucket timestamptz NOT NULL, client_hash text NOT NULL, reason text NOT NULL,
attempts integer NOT NULL DEFAULT 1, PRIMARY KEY(bucket,client_hash,reason)
);
COMMIT;
@@ -0,0 +1,4 @@
BEGIN;
ALTER TABLE user_interface_bindings DROP CONSTRAINT IF EXISTS user_interface_bindings_interface_type_check;
ALTER TABLE user_interface_bindings ADD CONSTRAINT user_interface_bindings_interface_type_check CHECK(interface_type IN ('ether','bridge','vlan','vxlan','gre','wg','ipsec','other'));
COMMIT;
@@ -0,0 +1,11 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_owner_revocations (
owner_id text PRIMARY KEY, invalid_before timestamptz NOT NULL,
updated_at timestamptz NOT NULL DEFAULT now()
);
CREATE TABLE IF NOT EXISTS mcp_portal_events (
id uuid PRIMARY KEY, owner_id text NOT NULL, payload_hash text NOT NULL,
created_at timestamptz NOT NULL DEFAULT now()
);
CREATE INDEX IF NOT EXISTS mcp_portal_events_time ON mcp_portal_events(created_at);
COMMIT;
@@ -0,0 +1,3 @@
BEGIN;
ALTER TABLE mcp_connections ADD COLUMN IF NOT EXISTS owner_claims_issued_at timestamptz NOT NULL DEFAULT to_timestamp(0);
COMMIT;
+8
View File
@@ -0,0 +1,8 @@
CREATE INDEX IF NOT EXISTS mcp_audit_handshake_cooldown
ON mcp_audit(connection_id, created_at DESC)
WHERE action = 'connection.handshake.start';
CREATE INDEX IF NOT EXISTS mcp_credentials_readiness_expired
ON mcp_credentials(connection_id, expires_at)
WHERE client_id = 'mcp-readiness-probe';
CREATE INDEX IF NOT EXISTS mcp_grants_credential
ON mcp_grants(connection_id, (payload->>'credentialId'));
@@ -0,0 +1,2 @@
ALTER TABLE mcp_artifacts ADD COLUMN IF NOT EXISTS access_scope jsonb;
CREATE INDEX IF NOT EXISTS mcp_artifacts_expiry ON mcp_artifacts(expires_at);
@@ -0,0 +1,23 @@
BEGIN;
ALTER TABLE servers ADD COLUMN IF NOT EXISTS routing_endpoint_epoch uuid NOT NULL DEFAULT gen_random_uuid();
CREATE OR REPLACE FUNCTION mm_routing_endpoint_epoch() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF ROW(NEW.host,NEW.port,NEW.username,NEW.password,NEW.use_ssl,NEW.verify_ssl,NEW.enabled)
IS DISTINCT FROM ROW(OLD.host,OLD.port,OLD.username,OLD.password,OLD.use_ssl,OLD.verify_ssl,OLD.enabled) THEN
NEW.routing_endpoint_epoch := gen_random_uuid();
ELSE
NEW.routing_endpoint_epoch := OLD.routing_endpoint_epoch;
END IF;
RETURN NEW;
END $$;
DROP TRIGGER IF EXISTS mm_routing_endpoint_epoch ON servers;
CREATE TRIGGER mm_routing_endpoint_epoch BEFORE UPDATE ON servers FOR EACH ROW EXECUTE FUNCTION mm_routing_endpoint_epoch();
ALTER TABLE mcp_routing_observations ADD COLUMN IF NOT EXISTS source_proof text;
DO $$ BEGIN
IF NOT EXISTS (SELECT 1 FROM pg_constraint WHERE conname='mcp_routing_observation_source_proof' AND conrelid='mcp_routing_observations'::regclass) THEN
ALTER TABLE mcp_routing_observations ADD CONSTRAINT mcp_routing_observation_source_proof CHECK(source_proof IS NULL OR source_proof ~ '^[a-f0-9]{64}$');
END IF;
END $$;
-- Existing measurements stay readable but unsigned. Never attest old samples
-- using today's endpoint: that would invent capture-time provenance.
COMMIT;
@@ -0,0 +1,6 @@
BEGIN;
-- Service-owned exact read-back must survive a subsequent network probe failure.
-- Generic after_sealed remains an array of resource snapshots.
ALTER TABLE mcp_operation_checkpoints ADD COLUMN IF NOT EXISTS configuration_sealed text;
ALTER TABLE mcp_operation_checkpoints ADD COLUMN IF NOT EXISTS configuration_proof text;
COMMIT;
@@ -0,0 +1,24 @@
BEGIN;
CREATE TABLE IF NOT EXISTS mcp_scoped_recovery_intents (
source_operation_id uuid NOT NULL REFERENCES mcp_operations(id),
source_step_index integer NOT NULL CHECK(source_step_index>=0),
source_receipt_id uuid NOT NULL,
source_configuration_proof text NOT NULL,
recovery_receipt_id uuid NOT NULL UNIQUE,
recovery_operation_id uuid REFERENCES mcp_operations(id),
mode text NOT NULL CHECK(mode IN ('manual','reserved')),
authority_profile_id uuid REFERENCES mcp_routing_profiles(id),
authority_profile_version integer,
intent_sealed text NOT NULL,
intent_proof text NOT NULL,
state text NOT NULL CHECK(state IN ('intent','verified','unknown')),
deadline timestamptz NOT NULL,
created_at timestamptz NOT NULL DEFAULT now(),
finished_at timestamptz,
verified_sealed text,
verified_proof text,
PRIMARY KEY(source_operation_id,source_step_index),
CHECK(mode='manual' OR authority_profile_id IS NOT NULL AND authority_profile_version IS NOT NULL),
CHECK((state='verified')=(verified_sealed IS NOT NULL AND verified_proof IS NOT NULL))
);
COMMIT;
@@ -0,0 +1,10 @@
-- Immutable commit evidence shared by ingestion and REST/MCP fact rebuild.
-- No raw flow payload, IP addresses, credentials or notification contents belong here.
CREATE TABLE IF NOT EXISTS flow_write_receipts (
batch_id uuid PRIMARY KEY,
kind text NOT NULL CHECK (kind IN ('ingest', 'facts', 'rebuild')),
source_proof text NOT NULL CHECK (source_proof ~ '^[a-f0-9]{64}$'),
result jsonb NOT NULL CHECK (jsonb_typeof(result) = 'object' AND octet_length(result::text) <= 65536),
committed_at timestamptz NOT NULL DEFAULT now()
);
-- Receipts are intentionally not pruned: an owned unknown batch may still need them.
@@ -0,0 +1,11 @@
-- A claim survives unknown results and is never recycled for another attempt.
CREATE TABLE IF NOT EXISTS flow_settings_intents (
receipt_id uuid NOT NULL,
phase text NOT NULL CHECK (phase IN ('configuration', 'listener')),
request_proof text NOT NULL,
state text NOT NULL DEFAULT 'claimed' CHECK (state IN ('claimed', 'completed')),
result jsonb,
created_at timestamptz NOT NULL DEFAULT now(),
PRIMARY KEY (receipt_id, phase)
);
INSERT INTO traffic_flow_settings(id) VALUES(1) ON CONFLICT DO NOTHING;
@@ -0,0 +1,19 @@
-- Precondition receipts only. No executor capability is granted by this table.
CREATE TABLE IF NOT EXISTS vrf_production_tickets (
ticket_id uuid PRIMARY KEY,
plan_id uuid NOT NULL UNIQUE REFERENCES mcp_plans(id),
connection_id uuid NOT NULL REFERENCES mcp_connections(id),
server_id bigint NOT NULL REFERENCES servers(id),
owner_id text NOT NULL,
action_id text NOT NULL CHECK (action_id IN ('ros.vrf.create','ros.vrf.update','ros.vrf.delete')),
action_version integer NOT NULL CHECK (action_version > 0),
source_proof text NOT NULL CHECK (source_proof ~ '^[a-f0-9]{64}$'),
digest text NOT NULL CHECK (digest ~ '^[a-f0-9]{64}$'),
ciphertext text NOT NULL,
state text NOT NULL DEFAULT 'issued' CHECK (state IN ('issued','claimed')),
issued_at timestamptz NOT NULL,
expires_at timestamptz NOT NULL,
claimed_at timestamptz,
CHECK (expires_at > issued_at AND expires_at <= issued_at + interval '5 minutes'),
CHECK ((state='issued' AND claimed_at IS NULL) OR (state='claimed' AND claimed_at IS NOT NULL))
);
@@ -0,0 +1,67 @@
-- Durable diagnostic attempts; this migration registers no MCP action.
BEGIN;
CREATE TABLE server_probe_executions (
receipt_id uuid PRIMARY KEY, binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'),
identity_kind text NOT NULL CHECK(identity_kind IN ('mcp','rest')),
execution_id uuid NOT NULL, actor_id text NOT NULL CHECK(length(actor_id) BETWEEN 1 AND 200),
authentication text NOT NULL CHECK(authentication IN ('mcp','jwt','local')),
mcp_connection_id uuid REFERENCES mcp_connections(id), mcp_credential_id uuid REFERENCES mcp_credentials(id), request_id text,
authority_issued_at bigint NOT NULL CHECK(authority_issued_at>=0), authority_expires_at timestamptz NOT NULL,
source_server_id integer NOT NULL CHECK(source_server_id>0), destination_server_id integer CHECK(destination_server_id>0),
tool text NOT NULL CHECK(tool IN ('ping','traceroute','bandwidth','dns','route','mtu')),
state text NOT NULL DEFAULT 'claimed' CHECK(state IN ('claimed','completed','failed','cancelled')),
attempted_stages integer CHECK(attempted_stages BETWEEN 0 AND 11), completed_stages integer CHECK(completed_stages BETWEEN 0 AND attempted_stages),
receipt_ciphertext text CHECK(octet_length(receipt_ciphertext)<=262144), receipt_proof text CHECK(receipt_proof ~ '^[a-f0-9]{64}$'),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(), finished_at timestamptz,
UNIQUE(identity_kind,execution_id),
CHECK((identity_kind='mcp' AND authentication='mcp' AND mcp_connection_id IS NOT NULL AND mcp_credential_id IS NOT NULL AND request_id IS NULL)
OR (identity_kind='rest' AND authentication IN ('jwt','local') AND mcp_connection_id IS NULL AND mcp_credential_id IS NULL AND request_id IS NOT NULL AND length(request_id) BETWEEN 1 AND 200)),
CHECK((tool='bandwidth')=(destination_server_id IS NOT NULL)),
CHECK((state='claimed' AND receipt_ciphertext IS NULL AND receipt_proof IS NULL AND finished_at IS NULL AND attempted_stages IS NULL AND completed_stages IS NULL)
OR (state<>'claimed' AND receipt_ciphertext IS NOT NULL AND receipt_proof IS NOT NULL AND finished_at IS NOT NULL AND attempted_stages IS NOT NULL AND completed_stages IS NOT NULL))
);
CREATE TABLE server_probe_execution_stages (
receipt_id uuid NOT NULL REFERENCES server_probe_executions(receipt_id), stage_index integer NOT NULL CHECK(stage_index BETWEEN 1 AND 11),
kind text NOT NULL CHECK(kind IN ('source-dns-A','native-ping','native-traceroute','native-bandwidth','native-route-read',
'manager-dns-A','manager-dns-AAAA','manager-dns-MX','manager-dns-NS','manager-dns-TXT','manager-dns-CNAME','manager-dns-PTR',
'native-mtu-1500','native-mtu-1492','native-mtu-1480','native-mtu-1476','native-mtu-1472','native-mtu-1468','native-mtu-1400','native-mtu-1280','native-mtu-1024','native-mtu-576')),
timeout_ms integer NOT NULL CHECK(timeout_ms BETWEEN 1 AND 120000),
state text NOT NULL DEFAULT 'intent' CHECK(state IN ('intent','received')),
outcome text CHECK(outcome IN ('received','source-unavailable')),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(), received_at timestamptz,
PRIMARY KEY(receipt_id,stage_index),
CHECK((state='intent' AND outcome IS NULL AND received_at IS NULL) OR (state='received' AND outcome IS NOT NULL AND received_at IS NOT NULL))
);
CREATE TABLE server_probe_execution_quarantines (
receipt_id uuid PRIMARY KEY, binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'),
identity_kind text NOT NULL CHECK(identity_kind IN ('mcp','rest')), execution_id uuid NOT NULL,
identity_proof text NOT NULL CHECK(identity_proof ~ '^[a-f0-9]{64}$'),
error_code text NOT NULL CHECK(error_code ~ '^[A-Z_]{1,64}$'),
attempted_stages integer NOT NULL CHECK(attempted_stages BETWEEN 0 AND 11), completed_stages integer NOT NULL CHECK(completed_stages BETWEEN 0 AND attempted_stages),
created_at timestamptz NOT NULL DEFAULT clock_timestamp(), UNIQUE(identity_kind,execution_id)
);
CREATE FUNCTION server_probe_execution_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
IF TG_OP='DELETE' THEN RAISE EXCEPTION 'probe execution journal is append only'; END IF;
IF OLD.state<>'claimed' OR (to_jsonb(OLD)-ARRAY['state','attempted_stages','completed_stages','receipt_ciphertext','receipt_proof','finished_at']) IS DISTINCT FROM
(to_jsonb(NEW)-ARRAY['state','attempted_stages','completed_stages','receipt_ciphertext','receipt_proof','finished_at']) OR NEW.state='claimed'
THEN RAISE EXCEPTION 'probe execution identity or terminal receipt is immutable'; END IF;
IF EXISTS(SELECT 1 FROM server_probe_execution_quarantines WHERE receipt_id=OLD.receipt_id) THEN RAISE EXCEPTION 'probe execution quarantined'; END IF;
RETURN NEW; END $$;
CREATE TRIGGER server_probe_execution_immutable BEFORE UPDATE OR DELETE ON server_probe_executions FOR EACH ROW EXECUTE FUNCTION server_probe_execution_immutable();
CREATE FUNCTION server_probe_stage_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
IF TG_OP='DELETE' THEN RAISE EXCEPTION 'probe stage journal is append only'; END IF;
PERFORM receipt_id FROM server_probe_executions WHERE receipt_id=NEW.receipt_id AND state='claimed' FOR UPDATE;
IF NOT FOUND OR EXISTS(SELECT 1 FROM server_probe_execution_quarantines WHERE receipt_id=NEW.receipt_id) THEN RAISE EXCEPTION 'probe attempt is not executable'; END IF;
IF TG_OP='INSERT' AND (NEW.stage_index<>(SELECT COALESCE(max(stage_index),0)+1 FROM server_probe_execution_stages WHERE receipt_id=NEW.receipt_id)
OR EXISTS(SELECT 1 FROM server_probe_execution_stages WHERE receipt_id=NEW.receipt_id AND state<>'received'))
THEN RAISE EXCEPTION 'probe stages must be serial and contiguous'; END IF;
IF TG_OP='UPDATE' AND (OLD.state<>'intent' OR NEW.state<>'received' OR (to_jsonb(OLD)-ARRAY['state','outcome','received_at']) IS DISTINCT FROM (to_jsonb(NEW)-ARRAY['state','outcome','received_at']))
THEN RAISE EXCEPTION 'probe stage terminal state is immutable'; END IF;
RETURN NEW; END $$;
CREATE TRIGGER server_probe_stage_immutable BEFORE INSERT OR UPDATE OR DELETE ON server_probe_execution_stages FOR EACH ROW EXECUTE FUNCTION server_probe_stage_immutable();
CREATE FUNCTION server_probe_quarantine_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RAISE EXCEPTION 'probe quarantine is immutable'; END $$;
CREATE TRIGGER server_probe_quarantine_immutable BEFORE UPDATE OR DELETE ON server_probe_execution_quarantines FOR EACH ROW EXECUTE FUNCTION server_probe_quarantine_immutable();
INSERT INTO schema_migrations(id) VALUES ('0035_server_probe_journal') ON CONFLICT (id) DO NOTHING;
COMMIT;
@@ -0,0 +1,14 @@
-- Durable stop requests for opt-in diagnostics, not configuration rollback.
BEGIN;
CREATE TABLE mcp_diagnostic_cancellations (
operation_id uuid PRIMARY KEY REFERENCES mcp_operations(id),
connection_id uuid NOT NULL REFERENCES mcp_connections(id),
actor_id text NOT NULL CHECK (length(actor_id) BETWEEN 1 AND 200),
created_at timestamptz NOT NULL DEFAULT clock_timestamp()
);
CREATE FUNCTION mcp_diagnostic_cancellation_immutable() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN RAISE EXCEPTION 'diagnostic cancellation request is immutable'; END $$;
CREATE TRIGGER mcp_diagnostic_cancellation_immutable BEFORE UPDATE OR DELETE ON mcp_diagnostic_cancellations
FOR EACH ROW EXECUTE FUNCTION mcp_diagnostic_cancellation_immutable();
INSERT INTO schema_migrations(id) VALUES ('0036_diagnostic_cancellations') ON CONFLICT (id) DO NOTHING;
COMMIT;
@@ -0,0 +1,226 @@
-- Fresh-only enrollment of empty diagnostic tables. No historical backfill,
-- source-generation claim, public registration or automatic cleanup is granted.
BEGIN;
LOCK TABLE server_probe_executions,server_probe_execution_stages,server_probe_execution_quarantines,mcp_diagnostic_cancellations IN ACCESS EXCLUSIVE MODE;
DO $$ BEGIN
IF EXISTS(SELECT 1 FROM server_probe_executions)
OR EXISTS(SELECT 1 FROM server_probe_execution_stages)
OR EXISTS(SELECT 1 FROM server_probe_execution_quarantines)
OR EXISTS(SELECT 1 FROM mcp_diagnostic_cancellations)
THEN RAISE EXCEPTION 'nonempty legacy diagnostic history prevents fresh storage enrollment'; END IF;
IF EXISTS(SELECT 1 FROM pg_catalog.pg_class WHERE relname IN
('server_probe_storage_usage','server_probe_storage_admissions','server_probe_execution_payloads'))
THEN RAISE EXCEPTION 'unmarked diagnostic storage relation prevents fresh storage enrollment'; END IF;
IF NOT EXISTS(SELECT 1 FROM mcp_settings WHERE id=1 AND requests_retention_days BETWEEN 1 AND 365)
THEN RAISE EXCEPTION 'authoritative probe retention setting unavailable'; END IF;
END $$;
CREATE TABLE server_probe_storage_usage (
scope text PRIMARY KEY CHECK(scope='global' OR (scope LIKE 'owner:%' AND length(scope) BETWEEN 7 AND 206)),
admissions bigint NOT NULL DEFAULT 0 CHECK(admissions>=0),
reserved_files bigint NOT NULL DEFAULT 0 CHECK(reserved_files>=0 AND reserved_files<=admissions),
reserved_bytes bigint NOT NULL DEFAULT 0 CHECK(reserved_bytes>=0),
CHECK(reserved_bytes=admissions*32768+reserved_files*262144)
);
CREATE TABLE server_probe_storage_admissions (
receipt_id uuid PRIMARY KEY,
owner_id text NOT NULL CHECK(length(owner_id) BETWEEN 1 AND 200),
identity_kind text NOT NULL CHECK(identity_kind IN ('mcp','rest')), execution_id uuid NOT NULL,
binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'),
authority_expires_at timestamptz NOT NULL,
claim_token uuid, claim_attempted_at timestamptz,
purged_at timestamptz, cancelled_at timestamptz, created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
retention_days integer NOT NULL CHECK(retention_days BETWEEN 1 AND 365),
safety_ms integer NOT NULL DEFAULT 60000 CHECK(safety_ms BETWEEN 60000 AND 86400000),
gc_not_before timestamptz NOT NULL DEFAULT 'infinity',
CHECK(gc_not_before>=authority_expires_at+make_interval(days=>retention_days)+safety_ms*interval '1 millisecond'),
UNIQUE(identity_kind,execution_id),
CHECK(claim_token IS NULL OR (claim_attempted_at IS NULL AND purged_at IS NULL AND cancelled_at IS NULL)),
CHECK(cancelled_at IS NULL OR (claim_token IS NULL AND claim_attempted_at IS NOT NULL))
);
CREATE INDEX server_probe_storage_admissions_owner_age ON server_probe_storage_admissions(owner_id,authority_expires_at,receipt_id);
CREATE TABLE server_probe_execution_payloads (
receipt_id uuid PRIMARY KEY REFERENCES server_probe_executions(receipt_id),
receipt_ciphertext text NOT NULL CHECK(octet_length(receipt_ciphertext) BETWEEN 1 AND 262144),
receipt_proof text NOT NULL CHECK(receipt_proof ~ '^[a-f0-9]{64}$')
);
INSERT INTO server_probe_storage_usage(scope,admissions,reserved_files,reserved_bytes) VALUES('global',0,0,0);
-- PostgreSQL drops table CHECKs dependent on this column together with it.
-- The locked empty-table guard precedes this column removal; no data is copied.
ALTER TABLE server_probe_executions DROP COLUMN receipt_ciphertext;
ALTER TABLE server_probe_executions ADD CONSTRAINT server_probe_storage_terminal_state CHECK(
(state='claimed' AND receipt_proof IS NULL AND finished_at IS NULL AND attempted_stages IS NULL AND completed_stages IS NULL)
OR(state<>'claimed' AND receipt_proof IS NOT NULL AND finished_at IS NOT NULL AND attempted_stages IS NOT NULL AND completed_stages IS NOT NULL));
-- Predicate only: it performs no DELETE. Every guarded exception below must
-- independently satisfy the persisted lifetime, actual current retention and
-- original terminal operation. The GUC alone never grants cleanup authority.
CREATE FUNCTION server_probe_storage_gc_allowed(id uuid) RETURNS boolean LANGUAGE plpgsql SET TimeZone='UTC' AS $$
DECLARE a server_probe_storage_admissions%ROWTYPE; activity timestamptz; current_retention integer; terminal_at timestamptz;
BEGIN
IF current_setting('mm.probe_storage_gc',true) IS DISTINCT FROM 'on' THEN RETURN false; END IF;
PERFORM pg_advisory_xact_lock(hashtextextended('server-probe-storage-v2',0));
SELECT * INTO a FROM server_probe_storage_admissions WHERE receipt_id=id FOR UPDATE;
IF NOT FOUND OR a.gc_not_before>=clock_timestamp() OR a.authority_expires_at>=clock_timestamp() THEN RETURN false; END IF;
SELECT settings.requests_retention_days INTO current_retention FROM mcp_settings settings WHERE settings.id=1;
IF current_retention IS NULL OR current_retention NOT BETWEEN 1 AND 365 THEN RETURN false; END IF;
terminal_at:=a.created_at;
IF a.identity_kind='mcp' THEN
SELECT GREATEST(o.finished_at,p.expires_at,COALESCE(cancellation.created_at,o.finished_at)) INTO terminal_at
FROM mcp_operations o JOIN mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN mcp_connections owner_connection ON owner_connection.id=o.connection_id
LEFT JOIN mcp_diagnostic_cancellations cancellation ON cancellation.operation_id=o.id
WHERE o.id=a.execution_id AND owner_connection.owner_id=a.owner_id AND p.operation_id='servers.probes.run'
AND o.state IN ('applied','failed','unknown','rejected','partial','cancelled') AND o.finished_at IS NOT NULL
AND (cancellation.operation_id IS NULL OR(cancellation.connection_id=o.connection_id AND cancellation.actor_id=a.owner_id))
FOR UPDATE OF o,p,owner_connection;
IF NOT FOUND OR terminal_at IS NULL THEN RETURN false; END IF;
END IF;
SELECT GREATEST(a.created_at,a.authority_expires_at,terminal_at,COALESCE(a.cancelled_at,a.created_at),
COALESCE((SELECT finished_at FROM server_probe_executions WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM server_probe_execution_quarantines WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(GREATEST(created_at,COALESCE(received_at,created_at))) FROM server_probe_execution_stages WHERE receipt_id=id),a.created_at)) INTO activity;
RETURN activity+make_interval(days=>GREATEST(a.retention_days,current_retention))+a.safety_ms*interval '1 millisecond'<clock_timestamp();
END $$;
CREATE FUNCTION server_probe_storage_admission_guard() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE current_retention integer; activity timestamptz;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-probe-storage-v2',0));
IF TG_OP='DELETE' THEN
IF server_probe_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'probe admission lifetime has not elapsed';
END IF;
SELECT requests_retention_days INTO current_retention FROM mcp_settings WHERE id=1;
IF current_retention IS NULL OR current_retention NOT BETWEEN 1 AND 365 THEN RAISE EXCEPTION 'probe retention unavailable'; END IF;
IF TG_OP='INSERT' THEN
IF NEW.purged_at IS NOT NULL OR (NEW.cancelled_at IS NULL AND
(NEW.authority_expires_at<=clock_timestamp() OR NEW.claim_token IS NULL OR NEW.claim_attempted_at IS NOT NULL))
OR (NEW.cancelled_at IS NOT NULL AND (NEW.claim_token IS NOT NULL OR NEW.claim_attempted_at IS NULL))
THEN RAISE EXCEPTION 'invalid fresh probe admission'; END IF;
IF NEW.cancelled_at IS NOT NULL THEN NEW.cancelled_at:=clock_timestamp(); NEW.claim_attempted_at:=NEW.cancelled_at; END IF;
NEW.retention_days:=current_retention; RETURN NEW;
END IF;
IF (to_jsonb(OLD)-ARRAY['claim_token','claim_attempted_at','purged_at','cancelled_at']) IS DISTINCT FROM
(to_jsonb(NEW)-ARRAY['claim_token','claim_attempted_at','purged_at','cancelled_at'])
THEN RAISE EXCEPTION 'immutable probe admission identity'; END IF;
IF OLD.cancelled_at IS DISTINCT FROM NEW.cancelled_at THEN
IF OLD.cancelled_at IS NULL AND NEW.cancelled_at IS NOT NULL AND NEW.claim_token IS NULL
AND NEW.purged_at IS NOT DISTINCT FROM OLD.purged_at AND NEW.claim_attempted_at IS NOT NULL
AND (OLD.claim_attempted_at IS NULL OR NEW.claim_attempted_at IS NOT DISTINCT FROM OLD.claim_attempted_at)
THEN NEW.cancelled_at:=clock_timestamp(); IF OLD.claim_attempted_at IS NULL THEN NEW.claim_attempted_at:=NEW.cancelled_at; END IF; RETURN NEW; END IF;
RAISE EXCEPTION 'probe cancellation is irreversible';
END IF;
IF OLD.purged_at IS NULL AND NEW.purged_at IS NULL AND OLD.claim_token IS NOT NULL AND NEW.claim_token IS NULL
AND OLD.cancelled_at IS NULL AND OLD.claim_attempted_at IS NULL AND NEW.claim_attempted_at IS NOT NULL AND OLD.authority_expires_at>clock_timestamp()
THEN RETURN NEW; END IF;
IF OLD.purged_at IS NULL AND NEW.purged_at IS NOT NULL AND NEW.claim_token IS NULL
AND NEW.claim_attempted_at IS NOT DISTINCT FROM OLD.claim_attempted_at THEN
SELECT GREATEST(OLD.authority_expires_at,COALESCE(e.finished_at,OLD.authority_expires_at),COALESCE(q.created_at,OLD.authority_expires_at)) INTO activity
FROM (SELECT 1) x LEFT JOIN server_probe_executions e ON e.receipt_id=OLD.receipt_id
LEFT JOIN server_probe_execution_quarantines q ON q.receipt_id=OLD.receipt_id;
IF activity<clock_timestamp()-make_interval(days=>GREATEST(OLD.retention_days,current_retention)) THEN RETURN NEW; END IF;
END IF;
RAISE EXCEPTION 'probe admission transition refused';
END $$;
CREATE TRIGGER server_probe_storage_admission_guard BEFORE INSERT OR UPDATE OR DELETE ON server_probe_storage_admissions
FOR EACH ROW EXECUTE FUNCTION server_probe_storage_admission_guard();
CREATE FUNCTION server_probe_storage_anchor_guard() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE a server_probe_storage_admissions%ROWTYPE;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-probe-storage-v2',0));
SELECT * INTO a FROM server_probe_storage_admissions WHERE receipt_id=NEW.receipt_id FOR UPDATE;
IF NOT FOUND OR a.owner_id<>NEW.actor_id OR a.identity_kind<>NEW.identity_kind OR a.execution_id<>NEW.execution_id
OR a.binding_proof<>NEW.binding_proof OR a.authority_expires_at<>NEW.authority_expires_at
OR a.purged_at IS NOT NULL OR a.cancelled_at IS NOT NULL OR a.claim_token IS NOT NULL OR a.claim_attempted_at IS NULL OR a.authority_expires_at<=clock_timestamp()
THEN RAISE EXCEPTION 'probe anchor has no live consumed admission'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER server_probe_storage_anchor_guard BEFORE INSERT OR UPDATE ON server_probe_executions
FOR EACH ROW EXECUTE FUNCTION server_probe_storage_anchor_guard();
CREATE OR REPLACE FUNCTION server_probe_execution_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
IF TG_OP='DELETE' THEN
IF server_probe_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'probe execution lifetime has not elapsed';
END IF;
IF OLD.state<>'claimed' OR (to_jsonb(OLD)-ARRAY['state','attempted_stages','completed_stages','receipt_proof','finished_at']) IS DISTINCT FROM
(to_jsonb(NEW)-ARRAY['state','attempted_stages','completed_stages','receipt_proof','finished_at']) OR NEW.state='claimed'
THEN RAISE EXCEPTION 'probe execution identity or terminal receipt is immutable'; END IF;
IF EXISTS(SELECT 1 FROM server_probe_execution_quarantines WHERE receipt_id=OLD.receipt_id)
THEN RAISE EXCEPTION 'probe execution quarantined'; END IF;
RETURN NEW; END $$;
CREATE FUNCTION server_probe_storage_stage_guard() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-probe-storage-v2',0));
PERFORM receipt_id FROM server_probe_storage_admissions WHERE receipt_id=NEW.receipt_id AND purged_at IS NULL AND cancelled_at IS NULL
AND claim_token IS NULL AND claim_attempted_at IS NOT NULL AND authority_expires_at>clock_timestamp() FOR UPDATE;
IF NOT FOUND THEN RAISE EXCEPTION 'probe stage has no live admission'; END IF;
RETURN NEW; END $$;
CREATE TRIGGER server_probe_storage_stage_guard BEFORE INSERT OR UPDATE ON server_probe_execution_stages
FOR EACH ROW EXECUTE FUNCTION server_probe_storage_stage_guard();
CREATE FUNCTION server_probe_storage_quarantine_guard() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-probe-storage-v2',0));
PERFORM receipt_id FROM server_probe_storage_admissions WHERE receipt_id=NEW.receipt_id AND identity_kind=NEW.identity_kind
AND execution_id=NEW.execution_id AND binding_proof=NEW.binding_proof FOR UPDATE;
IF NOT FOUND THEN RAISE EXCEPTION 'probe quarantine has no committed matching admission'; END IF;
RETURN NEW; END $$;
CREATE TRIGGER server_probe_storage_quarantine_guard BEFORE INSERT ON server_probe_execution_quarantines
FOR EACH ROW EXECUTE FUNCTION server_probe_storage_quarantine_guard();
CREATE FUNCTION server_probe_storage_payload_guard() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE a server_probe_storage_admissions%ROWTYPE; e server_probe_executions%ROWTYPE;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-probe-storage-v2',0));
IF TG_OP='UPDATE' THEN RAISE EXCEPTION 'probe payload is immutable'; END IF;
IF TG_OP='DELETE' THEN
PERFORM receipt_id FROM server_probe_storage_admissions WHERE receipt_id=OLD.receipt_id AND purged_at IS NOT NULL FOR UPDATE;
IF NOT FOUND AND NOT server_probe_storage_gc_allowed(OLD.receipt_id) THEN RAISE EXCEPTION 'probe payload is not purged'; END IF;
RETURN OLD;
END IF;
SELECT * INTO a FROM server_probe_storage_admissions WHERE receipt_id=NEW.receipt_id FOR UPDATE;
IF NOT FOUND OR a.purged_at IS NOT NULL OR a.cancelled_at IS NOT NULL OR a.claim_token IS NOT NULL OR a.claim_attempted_at IS NULL OR a.authority_expires_at<=clock_timestamp()
THEN RAISE EXCEPTION 'probe payload has no live admission'; END IF;
SELECT * INTO e FROM server_probe_executions WHERE receipt_id=NEW.receipt_id FOR UPDATE;
IF NOT FOUND OR e.state='claimed' OR e.receipt_proof<>NEW.receipt_proof OR e.actor_id<>a.owner_id OR e.binding_proof<>a.binding_proof
OR e.identity_kind<>a.identity_kind OR e.execution_id<>a.execution_id OR EXISTS(SELECT 1 FROM server_probe_execution_quarantines q
WHERE q.receipt_id=NEW.receipt_id OR(q.identity_kind=a.identity_kind AND q.execution_id=a.execution_id))
THEN RAISE EXCEPTION 'probe payload terminal identity mismatch or quarantine'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER server_probe_storage_payload_guard BEFORE INSERT OR UPDATE OR DELETE ON server_probe_execution_payloads
FOR EACH ROW EXECUTE FUNCTION server_probe_storage_payload_guard();
-- Preserve every original INSERT/UPDATE rule; only the already-validated
-- lifetime predicate can admit DELETE; no cleanup is scheduled by migration.
CREATE OR REPLACE FUNCTION server_probe_stage_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
IF TG_OP='DELETE' THEN
IF server_probe_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'probe stage lifetime has not elapsed';
END IF;
PERFORM receipt_id FROM server_probe_executions WHERE receipt_id=NEW.receipt_id AND state='claimed' FOR UPDATE;
IF NOT FOUND OR EXISTS(SELECT 1 FROM server_probe_execution_quarantines WHERE receipt_id=NEW.receipt_id) THEN RAISE EXCEPTION 'probe attempt is not executable'; END IF;
IF TG_OP='INSERT' AND (NEW.stage_index<>(SELECT COALESCE(max(stage_index),0)+1 FROM server_probe_execution_stages WHERE receipt_id=NEW.receipt_id)
OR EXISTS(SELECT 1 FROM server_probe_execution_stages WHERE receipt_id=NEW.receipt_id AND state<>'received'))
THEN RAISE EXCEPTION 'probe stages must be serial and contiguous'; END IF;
IF TG_OP='UPDATE' AND (OLD.state<>'intent' OR NEW.state<>'received' OR (to_jsonb(OLD)-ARRAY['state','outcome','received_at']) IS DISTINCT FROM (to_jsonb(NEW)-ARRAY['state','outcome','received_at']))
THEN RAISE EXCEPTION 'probe stage terminal state is immutable'; END IF;
RETURN NEW; END $$;
CREATE OR REPLACE FUNCTION server_probe_quarantine_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
IF TG_OP='DELETE' AND server_probe_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'probe quarantine remains immutable'; END $$;
CREATE OR REPLACE FUNCTION mcp_diagnostic_cancellation_immutable() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE original_receipt uuid;
BEGIN
IF TG_OP='DELETE' THEN
SELECT a.receipt_id INTO original_receipt FROM server_probe_storage_admissions a
JOIN mcp_operations o ON o.id=a.execution_id AND o.connection_id=OLD.connection_id
WHERE a.identity_kind='mcp' AND a.execution_id=OLD.operation_id AND a.owner_id=OLD.actor_id;
IF FOUND AND server_probe_storage_gc_allowed(original_receipt) THEN RETURN OLD; END IF;
END IF;
RAISE EXCEPTION 'diagnostic cancellation lifetime has not elapsed';
END $$;
INSERT INTO schema_migrations(id) VALUES ('0037_server_probe_storage_v2') ON CONFLICT (id) DO NOTHING;
COMMIT;
@@ -0,0 +1,59 @@
-- Initial poll journal and adopted partition-writer backstop.
-- Mixed old replicas MUST be drained before activation: triggers alone run
-- after relation access and cannot replace pre-DML shared writer admission.
BEGIN;
DO $$ BEGIN
PERFORM pg_catalog.pg_advisory_xact_lock(8723110::bigint);
IF EXISTS(SELECT 1 FROM pg_catalog.pg_class WHERE relname=ANY(ARRAY['server_poll_executions','server_poll_intents','server_poll_acks','server_poll_receipts','server_poll_quarantines']))
OR EXISTS(SELECT 1 FROM pg_catalog.pg_proc WHERE proname=ANY(ARRAY['server_poll_journal_immutable','server_partition_writer_guard']))
OR EXISTS(SELECT 1 FROM pg_catalog.pg_trigger WHERE tgname='server_partition_writer_guard')
THEN RAISE EXCEPTION 'unmarked poll journal or partition guard requires explicit reconciliation'; END IF;
END $$;
CREATE TABLE server_poll_executions (
receipt_id uuid PRIMARY KEY,identity_kind text NOT NULL CHECK(identity_kind IN ('mcp','rest')),execution_id uuid NOT NULL,
binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'),context_ciphertext text NOT NULL CHECK(octet_length(context_ciphertext)<=2097152),
context_proof text NOT NULL CHECK(context_proof ~ '^[a-f0-9]{64}$'),created_at timestamptz NOT NULL DEFAULT clock_timestamp(),UNIQUE(identity_kind,execution_id));
CREATE TABLE server_poll_intents (
receipt_id uuid REFERENCES server_poll_executions(receipt_id),stage text NOT NULL CHECK(stage ~ '^(native\.(identity|resource|interfaces|addresses)|publish\.[0-9]{1,4})$'),
before_proof text NOT NULL CHECK(before_proof ~ '^[a-f0-9]{64}$'),effect_proof text NOT NULL CHECK(effect_proof ~ '^[a-f0-9]{64}$'),
intent_proof text NOT NULL CHECK(intent_proof ~ '^[a-f0-9]{64}$'),created_at timestamptz NOT NULL DEFAULT clock_timestamp(),PRIMARY KEY(receipt_id,stage));
CREATE TABLE server_poll_acks (
receipt_id uuid NOT NULL,stage text NOT NULL,result_proof text NOT NULL CHECK(result_proof ~ '^[a-f0-9]{64}$'),outcome text NOT NULL CHECK(outcome IN ('confirmed','read-failed')),
ack_proof text NOT NULL CHECK(ack_proof ~ '^[a-f0-9]{64}$'),created_at timestamptz NOT NULL DEFAULT clock_timestamp(),PRIMARY KEY(receipt_id,stage),FOREIGN KEY(receipt_id,stage) REFERENCES server_poll_intents(receipt_id,stage),CHECK(stage LIKE 'native.%' OR outcome='confirmed'));
CREATE TABLE server_poll_receipts (
receipt_id uuid PRIMARY KEY REFERENCES server_poll_executions(receipt_id),receipt_ciphertext text NOT NULL CHECK(octet_length(receipt_ciphertext)<=262144),receipt_proof text NOT NULL CHECK(receipt_proof ~ '^[a-f0-9]{64}$'),created_at timestamptz NOT NULL DEFAULT clock_timestamp());
CREATE TABLE server_poll_quarantines (
receipt_id uuid PRIMARY KEY,identity_kind text NOT NULL CHECK(identity_kind IN ('mcp','rest')),execution_id uuid NOT NULL,context_proof text NOT NULL CHECK(context_proof ~ '^[a-f0-9]{64}$'),created_at timestamptz NOT NULL DEFAULT clock_timestamp(),UNIQUE(identity_kind,execution_id));
CREATE FUNCTION server_poll_journal_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN RAISE EXCEPTION 'server poll journal is append only'; END $$;
CREATE TRIGGER server_poll_execution_immutable BEFORE UPDATE OR DELETE ON server_poll_executions FOR EACH ROW EXECUTE FUNCTION server_poll_journal_immutable();
CREATE TRIGGER server_poll_intent_immutable BEFORE UPDATE OR DELETE ON server_poll_intents FOR EACH ROW EXECUTE FUNCTION server_poll_journal_immutable();
CREATE TRIGGER server_poll_ack_immutable BEFORE UPDATE OR DELETE ON server_poll_acks FOR EACH ROW EXECUTE FUNCTION server_poll_journal_immutable();
CREATE TRIGGER server_poll_receipt_immutable BEFORE UPDATE OR DELETE ON server_poll_receipts FOR EACH ROW EXECUTE FUNCTION server_poll_journal_immutable();
CREATE TRIGGER server_poll_quarantine_immutable BEFORE UPDATE OR DELETE ON server_poll_quarantines FOR EACH ROW EXECUTE FUNCTION server_poll_journal_immutable();
CREATE FUNCTION public.server_partition_writer_guard() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN PERFORM pg_advisory_xact_lock_shared(8723110::bigint); IF TG_OP='DELETE' THEN RETURN OLD; END IF; RETURN NEW; END; $$;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."flow_buckets" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."flow_buckets" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."flow_minute_stats" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."flow_minute_stats" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."flow_minute_dims" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."flow_minute_dims" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."flow_daily_dims" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."flow_daily_dims" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."flow_hour_facts" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."flow_hour_facts" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."flow_daily_facts" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."flow_daily_facts" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."traffic_samples" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."traffic_samples" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."servers_rest_ping_samples" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."servers_rest_ping_samples" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."uptime_probe_samples" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."uptime_probe_samples" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."uptime_resource_samples" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."uptime_resource_samples" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."server_snapshots" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."server_snapshots" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
CREATE TRIGGER server_partition_writer_guard BEFORE INSERT OR UPDATE OR DELETE ON public."internet_path_snapshots" FOR EACH ROW EXECUTE FUNCTION public.server_partition_writer_guard();
ALTER TABLE public."internet_path_snapshots" ENABLE ALWAYS TRIGGER server_partition_writer_guard;
INSERT INTO schema_migrations(id) VALUES ('0038_server_poll_journal_and_writer_guards') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,175 @@
-- Fresh-only enrollment of the reviewed bounded poll journal. No historical backfill.
-- Fixed logical quota; physical heap/index/WAL and vacuum remain independent.
-- Requires the empty 0038 poll journal. No scheduling or public activation.
BEGIN;
LOCK TABLE server_poll_executions,server_poll_intents,server_poll_acks,server_poll_receipts,server_poll_quarantines IN ACCESS EXCLUSIVE MODE;
DO $$ BEGIN
IF EXISTS(SELECT 1 FROM server_poll_executions) OR EXISTS(SELECT 1 FROM server_poll_intents)
OR EXISTS(SELECT 1 FROM server_poll_acks) OR EXISTS(SELECT 1 FROM server_poll_receipts)
OR EXISTS(SELECT 1 FROM server_poll_quarantines)
THEN RAISE EXCEPTION 'nonempty poll journal needs explicit reconciliation'; END IF;
IF NOT EXISTS(SELECT 1 FROM mcp_settings WHERE id=1 AND requests_retention_days BETWEEN 1 AND 365)
THEN RAISE EXCEPTION 'poll retention unavailable'; END IF;
IF EXISTS(SELECT 1 FROM pg_catalog.pg_class WHERE relname IN('server_poll_storage_usage','server_poll_storage_admissions'))
OR EXISTS(SELECT 1 FROM pg_catalog.pg_proc WHERE proname IN('server_poll_storage_gc_allowed','server_poll_storage_admission_guard','server_poll_storage_row_guard','server_poll_storage_row_meter'))
OR EXISTS(SELECT 1 FROM pg_catalog.pg_trigger WHERE tgname LIKE 'server_poll_storage_%')
THEN RAISE EXCEPTION 'unmarked poll storage requires explicit reconciliation'; END IF;
END $$;
CREATE TABLE server_poll_storage_usage (
scope text PRIMARY KEY CHECK(scope='global' OR(scope LIKE 'owner:%' AND length(scope) BETWEEN 7 AND 206)),
admissions bigint NOT NULL DEFAULT 0 CHECK(admissions>=0 AND admissions<=128),
reserved_rows bigint NOT NULL DEFAULT 0 CHECK(reserved_rows=admissions*4019),
reserved_bytes bigint NOT NULL DEFAULT 0 CHECK(reserved_bytes=admissions*8421376),
CHECK(scope='global' OR admissions<=8)
);
CREATE TABLE server_poll_storage_admissions (
receipt_id uuid PRIMARY KEY, identity_kind text NOT NULL CHECK(identity_kind IN('mcp','rest')),
execution_id uuid NOT NULL, connection_id uuid, owner_id text NOT NULL CHECK(length(owner_id) BETWEEN 1 AND 200 AND btrim(owner_id)=owner_id AND owner_id !~ '[[:cntrl:]]'),
binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'), context_proof text NOT NULL CHECK(context_proof ~ '^[a-f0-9]{64}$'),
authority_issued_at timestamptz NOT NULL CHECK(isfinite(authority_issued_at)),
authority_expires_at timestamptz NOT NULL CHECK(isfinite(authority_expires_at) AND authority_expires_at>authority_issued_at),
max_stages integer NOT NULL CHECK(max_stages BETWEEN 8 AND 2008),
claim_token uuid, claim_attempted_at timestamptz, created_at timestamptz NOT NULL DEFAULT clock_timestamp(),
retention_days integer NOT NULL CHECK(retention_days BETWEEN 1 AND 365),
safety_ms integer NOT NULL DEFAULT 60000 CHECK(safety_ms=60000),
gc_not_before timestamptz NOT NULL CHECK(isfinite(gc_not_before)),
used_rows integer NOT NULL DEFAULT 0 CHECK(used_rows BETWEEN 0 AND 4019 AND used_rows<=max_stages*2+3),
used_bytes bigint NOT NULL DEFAULT 0 CHECK(used_bytes BETWEEN 0 AND 8388608),
UNIQUE(identity_kind,execution_id),
CHECK((identity_kind='mcp' AND connection_id IS NOT NULL) OR(identity_kind='rest' AND connection_id IS NULL)),
CHECK((claim_token IS NOT NULL AND claim_attempted_at IS NULL) OR(claim_token IS NULL AND claim_attempted_at IS NOT NULL)),
CHECK(gc_not_before>=authority_expires_at+make_interval(days=>retention_days)+interval '60 seconds')
);
CREATE INDEX server_poll_storage_expiry ON server_poll_storage_admissions(gc_not_before,receipt_id);
INSERT INTO server_poll_storage_usage(scope) VALUES('global');
-- Every guard is installed in this same empty-enrollment transaction;
-- no historical admission or journal identity is inferred or imported.
CREATE FUNCTION server_poll_storage_gc_allowed(id uuid) RETURNS boolean LANGUAGE plpgsql SET TimeZone='UTC' AS $$
DECLARE a server_poll_storage_admissions%ROWTYPE; retention integer; activity timestamptz; terminal_at timestamptz;
BEGIN
IF current_setting('mm.poll_storage_gc',true) IS DISTINCT FROM 'on' THEN RETURN false; END IF;
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
SELECT * INTO a FROM server_poll_storage_admissions WHERE receipt_id=id FOR UPDATE;
IF NOT FOUND OR a.authority_expires_at>=clock_timestamp() OR a.gc_not_before>=clock_timestamp() THEN RETURN false; END IF;
SELECT requests_retention_days INTO retention FROM mcp_settings WHERE mcp_settings.id=1 FOR SHARE;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365 THEN RETURN false; END IF;
terminal_at:=a.created_at;
IF a.identity_kind='mcp' THEN
SELECT GREATEST(o.finished_at,p.expires_at) INTO terminal_at FROM mcp_operations o
JOIN mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN mcp_connections c ON c.id=o.connection_id
WHERE o.id=a.execution_id AND o.connection_id=a.connection_id AND c.owner_id=a.owner_id AND p.operation_id='servers.poll'
AND o.state IN('applied','failed','unknown','rejected','partial','cancelled') AND o.finished_at IS NOT NULL
FOR UPDATE OF o,p,c;
IF NOT FOUND OR terminal_at IS NULL THEN RETURN false; END IF;
END IF;
SELECT GREATEST(a.authority_expires_at,a.created_at,COALESCE(a.claim_attempted_at,a.created_at),terminal_at,
COALESCE((SELECT created_at FROM server_poll_executions WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(created_at) FROM server_poll_intents WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(created_at) FROM server_poll_acks WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM server_poll_receipts WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM server_poll_quarantines WHERE receipt_id=id),a.created_at)) INTO activity;
RETURN activity+make_interval(days=>GREATEST(a.retention_days,retention))+a.safety_ms*interval '1 millisecond'<clock_timestamp();
END $$;
CREATE FUNCTION server_poll_storage_admission_guard() RETURNS trigger LANGUAGE plpgsql SET TimeZone='UTC' AS $$
DECLARE retention integer; charged integer;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
IF TG_OP='DELETE' THEN
IF NOT server_poll_storage_gc_allowed(OLD.receipt_id) OR EXISTS(SELECT 1 FROM server_poll_executions WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM server_poll_intents WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM server_poll_acks WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM server_poll_receipts WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM server_poll_quarantines WHERE receipt_id=OLD.receipt_id)
THEN RAISE EXCEPTION 'poll lifetime or children prevent admission deletion'; END IF;
UPDATE server_poll_storage_usage SET admissions=admissions-1,reserved_rows=reserved_rows-4019,reserved_bytes=reserved_bytes-8421376
WHERE scope IN('global','owner:'||OLD.owner_id) AND admissions>=1 AND reserved_rows>=4019 AND reserved_bytes>=8421376;
GET DIAGNOSTICS charged=ROW_COUNT;
IF charged<>2 THEN RAISE EXCEPTION 'poll quota refund mismatch'; END IF;
RETURN OLD;
END IF;
IF TG_OP='UPDATE' THEN
IF(to_jsonb(OLD)-ARRAY['claim_token','claim_attempted_at','used_rows','used_bytes']) IS DISTINCT FROM
(to_jsonb(NEW)-ARRAY['claim_token','claim_attempted_at','used_rows','used_bytes']) THEN RAISE EXCEPTION 'poll admission identity is immutable'; END IF;
IF OLD.claim_token IS NOT NULL AND NEW.claim_token IS NULL AND OLD.claim_attempted_at IS NULL AND NEW.claim_attempted_at IS NOT NULL
AND NEW.used_rows=OLD.used_rows AND NEW.used_bytes=OLD.used_bytes AND OLD.authority_expires_at>clock_timestamp()
THEN NEW.claim_attempted_at:=clock_timestamp(); RETURN NEW; END IF;
IF OLD.claim_token IS NOT DISTINCT FROM NEW.claim_token AND OLD.claim_attempted_at IS NOT DISTINCT FROM NEW.claim_attempted_at
AND NEW.used_rows=OLD.used_rows+1 AND NEW.used_bytes>OLD.used_bytes THEN RETURN NEW; END IF;
RAISE EXCEPTION 'poll admission cannot refresh, reopen or refund itself';
END IF;
SELECT requests_retention_days INTO retention FROM mcp_settings WHERE id=1;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365 OR NEW.authority_issued_at>clock_timestamp()
OR NEW.authority_expires_at<=clock_timestamp() OR NEW.claim_token IS NULL OR NEW.claim_attempted_at IS NOT NULL
OR NEW.used_rows<>0 OR NEW.used_bytes<>0
THEN RAISE EXCEPTION 'new poll admission needs original live authority'; END IF;
IF EXISTS(SELECT 1 FROM server_poll_storage_admissions WHERE receipt_id=NEW.receipt_id OR(identity_kind=NEW.identity_kind AND execution_id=NEW.execution_id))
THEN RAISE EXCEPTION 'poll identity already spent'; END IF;
NEW.created_at:=clock_timestamp(); NEW.retention_days:=retention; NEW.safety_ms:=60000;
NEW.gc_not_before:=NEW.authority_expires_at+make_interval(days=>retention)+interval '60 seconds';
INSERT INTO server_poll_storage_usage(scope) VALUES('owner:'||NEW.owner_id) ON CONFLICT DO NOTHING;
IF(SELECT count(*) FROM server_poll_storage_usage WHERE scope IN('global','owner:'||NEW.owner_id)
AND admissions<CASE WHEN scope='global' THEN 128 ELSE 8 END)<>2 THEN RAISE EXCEPTION 'poll admission quota exhausted'; END IF;
UPDATE server_poll_storage_usage SET admissions=admissions+1,reserved_rows=reserved_rows+4019,reserved_bytes=reserved_bytes+8421376
WHERE scope IN('global','owner:'||NEW.owner_id);
GET DIAGNOSTICS charged=ROW_COUNT; IF charged<>2 THEN RAISE EXCEPTION 'poll admission quota unavailable'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER server_poll_storage_admission_guard BEFORE INSERT OR UPDATE OR DELETE ON server_poll_storage_admissions
FOR EACH ROW EXECUTE FUNCTION server_poll_storage_admission_guard();
CREATE FUNCTION server_poll_storage_row_guard() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE a server_poll_storage_admissions%ROWTYPE; n integer;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
SELECT * INTO a FROM server_poll_storage_admissions WHERE receipt_id=NEW.receipt_id FOR UPDATE;
IF NOT FOUND THEN RAISE EXCEPTION 'poll journal needs a committed admission'; END IF;
IF TG_TABLE_NAME='server_poll_quarantines' THEN
IF NEW.identity_kind<>a.identity_kind OR NEW.execution_id<>a.execution_id OR NEW.context_proof<>a.context_proof
THEN RAISE EXCEPTION 'poll quarantine admission mismatch'; END IF;
ELSE
IF a.claim_token IS NOT NULL OR a.claim_attempted_at IS NULL OR a.authority_expires_at<=clock_timestamp()
THEN RAISE EXCEPTION 'poll admission is not consumed and live'; END IF;
END IF;
IF TG_TABLE_NAME='server_poll_executions' THEN
IF NEW.identity_kind<>a.identity_kind OR NEW.execution_id<>a.execution_id OR NEW.context_proof<>a.context_proof OR NEW.binding_proof<>a.binding_proof
THEN RAISE EXCEPTION 'poll anchor admission mismatch'; END IF;
ELSIF TG_TABLE_NAME IN('server_poll_intents','server_poll_acks') THEN
IF NEW.stage LIKE 'publish.%' AND substring(NEW.stage FROM 9)::integer>=a.max_stages-4 THEN RAISE EXCEPTION 'poll stage exceeds private reservation'; END IF;
IF TG_TABLE_NAME='server_poll_intents' THEN SELECT count(*) INTO n FROM server_poll_intents WHERE receipt_id=NEW.receipt_id;
ELSE SELECT count(*) INTO n FROM server_poll_acks WHERE receipt_id=NEW.receipt_id; END IF;
IF n>=a.max_stages THEN RAISE EXCEPTION 'poll stage row budget exhausted'; END IF;
END IF;
IF TG_TABLE_NAME<>'server_poll_quarantines' AND EXISTS(SELECT 1 FROM server_poll_quarantines WHERE receipt_id=NEW.receipt_id)
THEN RAISE EXCEPTION 'poll identity quarantined'; END IF;
NEW.created_at:=clock_timestamp();
RETURN NEW;
END $$;
-- Meter actual inserts, not BEFORE attempts: ON CONFLICT DO NOTHING must not
-- spend another row/byte allowance for a marker which was never inserted.
CREATE FUNCTION server_poll_storage_row_meter() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE cost bigint; n integer;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
cost:=pg_column_size(NEW);
UPDATE server_poll_storage_admissions SET used_rows=used_rows+1,used_bytes=used_bytes+cost
WHERE receipt_id=NEW.receipt_id AND used_rows<max_stages*2+3 AND used_bytes<=8388608-cost;
GET DIAGNOSTICS n=ROW_COUNT; IF n<>1 THEN RAISE EXCEPTION 'poll journal byte or row budget exhausted'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER server_poll_storage_anchor_guard BEFORE INSERT ON server_poll_executions FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_guard();
CREATE TRIGGER server_poll_storage_intent_guard BEFORE INSERT ON server_poll_intents FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_guard();
CREATE TRIGGER server_poll_storage_ack_guard BEFORE INSERT ON server_poll_acks FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_guard();
CREATE TRIGGER server_poll_storage_receipt_guard BEFORE INSERT ON server_poll_receipts FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_guard();
CREATE TRIGGER server_poll_storage_quarantine_guard BEFORE INSERT ON server_poll_quarantines FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_guard();
CREATE TRIGGER server_poll_storage_anchor_meter AFTER INSERT ON server_poll_executions FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_meter();
CREATE TRIGGER server_poll_storage_intent_meter AFTER INSERT ON server_poll_intents FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_meter();
CREATE TRIGGER server_poll_storage_ack_meter AFTER INSERT ON server_poll_acks FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_meter();
CREATE TRIGGER server_poll_storage_receipt_meter AFTER INSERT ON server_poll_receipts FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_meter();
CREATE TRIGGER server_poll_storage_quarantine_meter AFTER INSERT ON server_poll_quarantines FOR EACH ROW EXECUTE FUNCTION server_poll_storage_row_meter();
CREATE OR REPLACE FUNCTION server_poll_journal_immutable() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN
IF TG_OP='DELETE' AND server_poll_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'server poll journal is immutable before its full lifetime elapses';
END $$;
INSERT INTO schema_migrations(id) VALUES ('0039_server_poll_storage') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,56 @@
-- Additive, inactive reserved recovery ledger. Existing manual evidence is retained.
-- Legacy reserved rows cannot prove an original two-write debit: never backfill them.
BEGIN;
LOCK TABLE public.mcp_scoped_recovery_intents IN ACCESS EXCLUSIVE MODE;
DO $$ BEGIN
IF EXISTS(SELECT 1 FROM public.mcp_scoped_recovery_intents WHERE mode='reserved')
THEN RAISE EXCEPTION 'legacy reserved recovery needs explicit reconciliation'; END IF;
IF EXISTS(SELECT 1 FROM pg_catalog.pg_class c JOIN pg_catalog.pg_namespace n ON n.oid=c.relnamespace
WHERE n.nspname='public' AND c.relname IN('mcp_routing_reservations','mcp_routing_reservation_slots','mcp_routing_reservation_slots_window'))
OR EXISTS(SELECT 1 FROM pg_catalog.pg_attribute a WHERE a.attrelid='public.mcp_scoped_recovery_intents'::regclass
AND NOT a.attisdropped AND a.attname IN('reservation_id','slot_phase','inverse_intent_sealed','inverse_intent_proof','inverse_configuration_sealed','inverse_configuration_proof'))
OR EXISTS(SELECT 1 FROM pg_catalog.pg_constraint c WHERE c.conrelid='public.mcp_scoped_recovery_intents'::regclass
AND c.conname IN('scoped_reservation_slot_shape','reserved_inverse_evidence_pairs'))
THEN RAISE EXCEPTION 'unmarked routing reservation objects need explicit reconciliation'; END IF;
END $$;
CREATE TABLE public.mcp_routing_reservations (
id uuid PRIMARY KEY,
source_plan_id uuid NOT NULL REFERENCES public.mcp_plans(id),
source_operation_id uuid NOT NULL REFERENCES public.mcp_operations(id),
source_step_index integer NOT NULL CHECK(source_step_index=0),
profile_id uuid NOT NULL REFERENCES public.mcp_routing_profiles(id),
connection_id uuid NOT NULL REFERENCES public.mcp_connections(id),
binding_sealed text NOT NULL,binding_proof text NOT NULL,
state_sealed text NOT NULL,state_proof text NOT NULL,revision integer NOT NULL CHECK(revision>=1),
deadline timestamptz NOT NULL,created_at timestamptz NOT NULL DEFAULT now(),
UNIQUE(source_operation_id,source_step_index)
);
CREATE TABLE public.mcp_routing_reservation_slots (
reservation_id uuid NOT NULL REFERENCES public.mcp_routing_reservations(id),
profile_id uuid NOT NULL REFERENCES public.mcp_routing_profiles(id),
lane text NOT NULL CHECK(lane IN ('forward','inverse')),
state text NOT NULL CHECK(state IN ('held','spent','proven-unused','unknown')),
created_at timestamptz NOT NULL DEFAULT now(),PRIMARY KEY(reservation_id,lane)
);
CREATE INDEX mcp_routing_reservation_slots_window ON public.mcp_routing_reservation_slots(profile_id,created_at);
ALTER TABLE public.mcp_scoped_recovery_intents ADD COLUMN reservation_id uuid REFERENCES public.mcp_routing_reservations(id);
ALTER TABLE public.mcp_scoped_recovery_intents ADD COLUMN slot_phase text NOT NULL DEFAULT 'recovery-intent';
ALTER TABLE public.mcp_scoped_recovery_intents ALTER COLUMN source_configuration_proof DROP NOT NULL;
ALTER TABLE public.mcp_scoped_recovery_intents ADD COLUMN inverse_intent_sealed text;
ALTER TABLE public.mcp_scoped_recovery_intents ADD COLUMN inverse_intent_proof text;
ALTER TABLE public.mcp_scoped_recovery_intents ADD COLUMN inverse_configuration_sealed text;
ALTER TABLE public.mcp_scoped_recovery_intents ADD COLUMN inverse_configuration_proof text;
ALTER TABLE public.mcp_scoped_recovery_intents ADD CONSTRAINT scoped_reservation_slot_shape CHECK (
(mode='manual' AND reservation_id IS NULL AND slot_phase='recovery-intent' AND source_configuration_proof IS NOT NULL)
OR (mode='reserved' AND reservation_id IS NOT NULL AND
((slot_phase='reservation-held' AND source_configuration_proof IS NULL)
OR (slot_phase IN ('reservation-bound','reserved-attempt') AND source_configuration_proof IS NOT NULL)))
);
ALTER TABLE public.mcp_scoped_recovery_intents ADD CONSTRAINT reserved_inverse_evidence_pairs CHECK (
(inverse_intent_sealed IS NULL)=(inverse_intent_proof IS NULL)
AND (inverse_configuration_sealed IS NULL)=(inverse_configuration_proof IS NULL)
AND (inverse_configuration_sealed IS NULL OR inverse_intent_sealed IS NOT NULL)
AND (inverse_intent_sealed IS NULL OR mode='reserved')
);
INSERT INTO public.schema_migrations(id) VALUES ('0040_mcp_routing_reservations') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,333 @@
-- Additive terminal poll archives. Existing 0040 full journals are retained.
-- No compaction, retention run, source backfill or worker activation occurs here.
BEGIN;
DO $$ BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
IF EXISTS(SELECT 1 FROM pg_class WHERE relname='server_poll_archives')
OR EXISTS(SELECT 1 FROM pg_attribute WHERE attrelid='public.server_poll_storage_admissions'::regclass AND NOT attisdropped AND attname IN('storage_state','compact_at','archive_bytes','original_activity_at','compaction_claim','compaction_claimed_at','compaction_claim_xid'))
OR EXISTS(SELECT 1 FROM pg_attribute WHERE attrelid='public.server_poll_storage_usage'::regclass AND NOT attisdropped AND attname IN('compacted_admissions','archive_bytes'))
OR EXISTS(SELECT 1 FROM pg_proc WHERE proname IN('server_poll_storage_source_payload','server_poll_storage_compact_allowed','server_poll_storage_accounting_valid','server_poll_archive_insert_guard','server_poll_archive_insert_finish','server_poll_archive_immutable','server_poll_archive_final_guard','server_poll_compacted_row_guard'))
OR EXISTS(SELECT 1 FROM pg_trigger WHERE tgname IN('server_poll_archive_insert_guard','server_poll_archive_insert_finish','server_poll_archive_immutable','server_poll_archive_final_guard','server_poll_admission_final_guard','server_poll_compact_final_guard','server_poll_compacted_anchor_guard','server_poll_compacted_intent_guard','server_poll_compacted_ack_guard','server_poll_compacted_receipt_guard','server_poll_compacted_quarantine_guard'))
THEN RAISE EXCEPTION 'unmarked poll archive objects require explicit reconciliation'; END IF;
END $$;
LOCK TABLE server_poll_storage_usage,server_poll_storage_admissions,server_poll_executions,server_poll_intents,server_poll_acks,server_poll_receipts,server_poll_quarantines IN ACCESS EXCLUSIVE MODE;
DO $$ BEGIN
IF (SELECT count(*) FROM server_poll_storage_usage WHERE scope='global')<>1
OR EXISTS(SELECT 1 FROM server_poll_storage_usage u WHERE
u.admissions<>(SELECT count(*) FROM server_poll_storage_admissions a WHERE u.scope='global' OR u.scope='owner:'||a.owner_id)
OR u.reserved_rows<>u.admissions*4019 OR u.reserved_bytes<>u.admissions*8421376)
OR EXISTS(SELECT 1 FROM server_poll_storage_admissions a WHERE NOT EXISTS(SELECT 1 FROM server_poll_storage_usage u WHERE u.scope='owner:'||a.owner_id))
THEN RAISE EXCEPTION 'existing poll accounting requires explicit reconciliation'; END IF;
IF NOT EXISTS(SELECT 1 FROM pg_constraint WHERE conrelid='public.server_poll_storage_usage'::regclass AND conname='server_poll_storage_usage_check'
AND pg_get_constraintdef(oid)='CHECK ((reserved_rows = (admissions * 4019)))')
OR NOT EXISTS(SELECT 1 FROM pg_constraint WHERE conrelid='public.server_poll_storage_usage'::regclass AND conname='server_poll_storage_usage_check1'
AND pg_get_constraintdef(oid)='CHECK ((reserved_bytes = (admissions * 8421376)))')
THEN RAISE EXCEPTION 'original poll accounting constraints require explicit reconciliation'; END IF;
END $$;
ALTER TABLE server_poll_storage_admissions
ADD COLUMN storage_state text NOT NULL DEFAULT 'full' CHECK(storage_state IN('full','compacted')),
ADD COLUMN compact_at timestamptz,
ADD COLUMN archive_bytes integer,
ADD COLUMN original_activity_at timestamptz,
ADD COLUMN compaction_claim uuid,
ADD COLUMN compaction_claimed_at timestamptz,
ADD COLUMN compaction_claim_xid xid8,
ADD CONSTRAINT server_poll_compaction_claim_shape CHECK(
(compaction_claim IS NULL AND compaction_claimed_at IS NULL AND compaction_claim_xid IS NULL)
OR(compaction_claim IS NOT NULL AND compaction_claimed_at IS NOT NULL AND compaction_claim_xid IS NOT NULL AND isfinite(compaction_claimed_at))),
ADD CONSTRAINT server_poll_storage_compact_shape CHECK(
(storage_state='full' AND compact_at IS NULL AND archive_bytes IS NULL AND original_activity_at IS NULL)
OR(storage_state='compacted' AND identity_kind='mcp' AND claim_token IS NULL AND claim_attempted_at IS NOT NULL
AND compaction_claim IS NOT NULL AND compaction_claimed_at IS NOT NULL AND compaction_claim_xid IS NOT NULL
AND compact_at IS NOT NULL AND archive_bytes IS NOT NULL AND original_activity_at IS NOT NULL
AND isfinite(compact_at) AND archive_bytes BETWEEN 1 AND 131072 AND isfinite(original_activity_at)
AND original_activity_at>=created_at AND compact_at>authority_expires_at+safety_ms*interval '1 millisecond'));
ALTER TABLE server_poll_storage_usage
DROP CONSTRAINT server_poll_storage_usage_check,
DROP CONSTRAINT server_poll_storage_usage_check1,
ADD COLUMN compacted_admissions bigint NOT NULL DEFAULT 0 CHECK(compacted_admissions>=0),
ADD COLUMN archive_bytes bigint NOT NULL DEFAULT 0 CHECK(archive_bytes>=0),
ADD CONSTRAINT server_poll_storage_rows_charge CHECK(reserved_rows=admissions*4019+compacted_admissions*2
AND reserved_rows<=CASE WHEN scope='global' THEN 514432 ELSE 32152 END),
ADD CONSTRAINT server_poll_storage_bytes_charge CHECK(reserved_bytes=admissions*8421376+compacted_admissions*32768+archive_bytes
AND reserved_bytes<=CASE WHEN scope='global' THEN 1077936128 ELSE 67371008 END),
ADD CONSTRAINT server_poll_storage_archive_charge CHECK(archive_bytes>=compacted_admissions AND archive_bytes<=compacted_admissions*131072);
CREATE TABLE server_poll_archives (
receipt_id uuid PRIMARY KEY REFERENCES server_poll_storage_admissions(receipt_id),version smallint NOT NULL CHECK(version=1),
execution_id uuid NOT NULL UNIQUE,connection_id uuid NOT NULL,owner_id text NOT NULL,
binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'),context_proof text NOT NULL CHECK(context_proof ~ '^[a-f0-9]{64}$'),
source_digest text NOT NULL CHECK(source_digest ~ '^[a-f0-9]{64}$'),source_sha256 text NOT NULL CHECK(source_sha256 ~ '^[a-f0-9]{64}$'),
terminal_digest text NOT NULL CHECK(terminal_digest ~ '^[a-f0-9]{64}$'),
original_activity_at timestamptz NOT NULL CHECK(isfinite(original_activity_at)),original_expires_at timestamptz NOT NULL CHECK(isfinite(original_expires_at)),
decoded_bytes integer NOT NULL CHECK(decoded_bytes BETWEEN 1 AND 16777216),compressed_bytes integer NOT NULL CHECK(compressed_bytes BETWEEN 1 AND 98048),
ciphertext text NOT NULL,ciphertext_bytes integer NOT NULL CHECK(ciphertext_bytes BETWEEN 1 AND 131072 AND ciphertext_bytes=octet_length(ciphertext)),
archive_proof text NOT NULL CHECK(archive_proof ~ '^[a-f0-9]{64}$'),
created_at timestamptz NOT NULL,compaction_xid xid8 NOT NULL
);
-- SQL checks structure/ownership/current rows. The fixed private producer and
-- historical reader additionally authenticate every existing keyed proof and
-- the archive MAC; SQL never receives an encryption/HMAC key or boolean grant.
CREATE FUNCTION server_poll_storage_compact_allowed(id uuid) RETURNS boolean LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE a public.server_poll_storage_admissions%ROWTYPE; o public.mcp_operations%ROWTYPE; p public.mcp_plans%ROWTYPE; s public.mcp_operation_steps%ROWTYPE; n integer; pubs integer; activity timestamptz;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
SELECT * INTO a FROM public.server_poll_storage_admissions WHERE receipt_id=id FOR UPDATE;
IF NOT FOUND OR a.storage_state<>'full' OR a.identity_kind<>'mcp' OR a.claim_token IS NOT NULL OR a.claim_attempted_at IS NULL
OR a.authority_expires_at+a.safety_ms*interval '1 millisecond'>=clock_timestamp()
OR EXISTS(SELECT 1 FROM public.server_poll_quarantines WHERE receipt_id=id OR(identity_kind=a.identity_kind AND execution_id=a.execution_id))
THEN RETURN false; END IF;
IF NOT pg_try_advisory_xact_lock(hashtextextended('mcp-operation:'||a.execution_id::text,0)) THEN RETURN false; END IF;
IF NOT pg_try_advisory_xact_lock(hashtextextended('server-poll-execution-v1:mcp:'||a.execution_id::text,0)) THEN RETURN false; END IF;
IF NOT pg_try_advisory_xact_lock(hashtextextended('server-poll-receipt-v1:'||a.receipt_id::text,0)) THEN RETURN false; END IF;
BEGIN
SELECT * INTO o FROM public.mcp_operations WHERE mcp_operations.id=a.execution_id AND connection_id=a.connection_id FOR UPDATE NOWAIT;
IF NOT FOUND OR o.state<>'applied' OR o.verification<>'passed' OR o.error_code IS NOT NULL OR o.finished_at IS NULL THEN RETURN false; END IF;
SELECT * INTO p FROM public.mcp_plans WHERE mcp_plans.id=o.plan_id AND connection_id=a.connection_id FOR UPDATE NOWAIT;
IF NOT FOUND OR p.operation_id<>'servers.poll' OR p.operation_version<>1 OR p.state<>'completed' OR p.approved_by IS NULL OR p.approved_at IS NULL
OR p.created_at>p.approved_at OR p.approved_at>o.created_at OR o.created_at>=LEAST(p.expires_at,a.authority_expires_at) THEN RETURN false; END IF;
SELECT * INTO s FROM public.mcp_operation_steps WHERE operation_id=o.id AND step_index=0 FOR UPDATE NOWAIT;
IF NOT FOUND OR s.state<>'verified' OR s.server_id IS NOT NULL OR s.result IS NULL OR s.finished_at IS NULL
OR(SELECT count(*) FROM public.mcp_operation_steps WHERE operation_id=o.id)<>1
OR NOT EXISTS(SELECT 1 FROM public.mcp_connections c WHERE c.id=a.connection_id AND c.owner_id=a.owner_id)
OR s.intended_state->>'kind' IS DISTINCT FROM 'service' OR s.intended_state->>'service' IS DISTINCT FROM 'servers.poll'
OR s.intended_state#>>'{input,receiptId}' IS DISTINCT FROM id::text
OR s.result->>'operationId' IS DISTINCT FROM o.id::text OR s.result->>'receiptId' IS DISTINCT FROM id::text
OR s.result->'nativeReads' IS DISTINCT FROM '4'::jsonb
OR s.result->'complete' IS DISTINCT FROM 'true'::jsonb OR s.result->'maintenanceComplete' IS DISTINCT FROM 'true'::jsonb
OR s.result->'configurationVerified' IS DISTINCT FROM 'true'::jsonb OR s.result->'automaticReplay' IS DISTINCT FROM 'false'::jsonb
THEN RETURN false; END IF;
EXCEPTION WHEN lock_not_available THEN RETURN false; END;
IF NOT EXISTS(SELECT 1 FROM public.server_poll_executions e WHERE e.receipt_id=id AND e.identity_kind=a.identity_kind AND e.execution_id=a.execution_id AND e.binding_proof=a.binding_proof AND e.context_proof=a.context_proof)
OR NOT EXISTS(SELECT 1 FROM public.server_poll_receipts r WHERE r.receipt_id=id AND r.created_at<a.authority_expires_at) THEN RETURN false; END IF;
SELECT count(*) INTO n FROM public.server_poll_intents WHERE receipt_id=id;
SELECT count(*) INTO pubs FROM public.server_poll_intents WHERE receipt_id=id AND stage LIKE 'publish.%';
IF n<>pubs+4 OR pubs NOT IN(a.max_stages-6,a.max_stages-4) OR a.used_rows<>2*n+2
OR(SELECT count(*) FROM public.server_poll_acks WHERE receipt_id=id)<>n
OR s.result->'confirmedPublications' IS DISTINCT FROM to_jsonb(pubs)
OR s.result->'completedPublications' IS DISTINCT FROM to_jsonb(pubs)
OR EXISTS(SELECT 1 FROM public.server_poll_acks WHERE receipt_id=id AND outcome<>'confirmed')
OR EXISTS(SELECT 1 FROM public.server_poll_acks WHERE receipt_id=id AND created_at>=a.authority_expires_at)
OR EXISTS(SELECT 1 FROM unnest(ARRAY['native.identity','native.resource','native.interfaces','native.addresses']) wanted(stage)
WHERE NOT EXISTS(SELECT 1 FROM public.server_poll_intents i JOIN public.server_poll_acks k USING(receipt_id,stage) WHERE i.receipt_id=id AND i.stage=wanted.stage AND k.outcome='confirmed'))
OR EXISTS(SELECT 1 FROM generate_series(0,pubs-1) wanted(stage) WHERE NOT EXISTS(SELECT 1 FROM public.server_poll_intents i JOIN public.server_poll_acks k USING(receipt_id,stage) WHERE i.receipt_id=id AND i.stage='publish.'||wanted.stage::text AND k.outcome='confirmed'))
THEN RETURN false; END IF;
SELECT GREATEST(a.authority_expires_at,a.created_at,a.claim_attempted_at,o.finished_at,p.expires_at,s.finished_at,
(SELECT created_at FROM public.server_poll_executions WHERE receipt_id=id),(SELECT max(created_at) FROM public.server_poll_intents WHERE receipt_id=id),
(SELECT max(created_at) FROM public.server_poll_acks WHERE receipt_id=id),(SELECT created_at FROM public.server_poll_receipts WHERE receipt_id=id)) INTO activity;
RETURN isfinite(activity) AND activity+a.safety_ms*interval '1 millisecond'<clock_timestamp();
END $$;
CREATE FUNCTION server_poll_storage_source_payload(receipt_uuid uuid) RETURNS jsonb LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE payload jsonb;
BEGIN
IF NOT public.server_poll_storage_compact_allowed(receipt_uuid) THEN RETURN NULL; END IF;
SELECT jsonb_build_object('admission',to_jsonb(a),'journal',jsonb_build_object(
'execution',(SELECT to_jsonb(e) FROM public.server_poll_executions e WHERE e.receipt_id=receipt_uuid),
'intents',(SELECT COALESCE(jsonb_agg(to_jsonb(i) ORDER BY i.stage),'[]'::jsonb) FROM public.server_poll_intents i WHERE i.receipt_id=receipt_uuid),
'acks',(SELECT COALESCE(jsonb_agg(to_jsonb(k) ORDER BY k.stage),'[]'::jsonb) FROM public.server_poll_acks k WHERE k.receipt_id=receipt_uuid),
'receipt',(SELECT to_jsonb(r) FROM public.server_poll_receipts r WHERE r.receipt_id=receipt_uuid),
'quarantines',(SELECT COALESCE(jsonb_agg(to_jsonb(q) ORDER BY q.created_at,q.receipt_id),'[]'::jsonb) FROM public.server_poll_quarantines q WHERE q.receipt_id=receipt_uuid)),
'terminal',jsonb_build_object('operation',to_jsonb(o),'plan',to_jsonb(p),'step',to_jsonb(s))) INTO payload
FROM public.server_poll_storage_admissions a JOIN public.mcp_operations o ON o.id=a.execution_id AND o.connection_id=a.connection_id
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id JOIN public.mcp_operation_steps s ON s.operation_id=o.id AND s.step_index=0
WHERE a.receipt_id=receipt_uuid;
IF octet_length(payload::text)>16777216 THEN RETURN NULL; END IF;
RETURN payload;
END $$;
CREATE FUNCTION server_poll_storage_accounting_valid() RETURNS boolean LANGUAGE sql SET search_path=pg_catalog,public AS $$
WITH charges AS (
SELECT owner_id,CASE WHEN storage_state='full' THEN 1 ELSE 0 END::bigint AS heavy,
CASE WHEN storage_state='compacted' THEN 1 ELSE 0 END::bigint AS compact,
COALESCE(archive_bytes,0)::bigint AS archives,
CASE WHEN storage_state='full' THEN 4019 ELSE 2 END::bigint AS rows,
CASE WHEN storage_state='full' THEN 8421376 ELSE 32768+archive_bytes END::bigint AS bytes
FROM public.server_poll_storage_admissions
), expected AS (
SELECT 'global' AS scope,COALESCE(sum(heavy),0) AS heavy,COALESCE(sum(compact),0) AS compact,COALESCE(sum(archives),0) AS archives,COALESCE(sum(rows),0) AS rows,COALESCE(sum(bytes),0) AS bytes FROM charges
UNION ALL SELECT 'owner:'||owner_id,sum(heavy),sum(compact),sum(archives),sum(rows),sum(bytes) FROM charges GROUP BY owner_id
) SELECT NOT EXISTS(SELECT 1 FROM public.server_poll_storage_usage u FULL JOIN expected e USING(scope)
WHERE u.scope IS NULL OR u.admissions<>COALESCE(e.heavy,0) OR u.compacted_admissions<>COALESCE(e.compact,0)
OR u.archive_bytes<>COALESCE(e.archives,0) OR u.reserved_rows<>COALESCE(e.rows,0) OR u.reserved_bytes<>COALESCE(e.bytes,0))
AND NOT EXISTS(SELECT 1 FROM public.server_poll_storage_admissions a LEFT JOIN public.server_poll_archives r USING(receipt_id)
WHERE (a.storage_state='full' AND r.receipt_id IS NOT NULL) OR(a.storage_state='compacted' AND(r.receipt_id IS NULL OR a.archive_bytes<>r.ciphertext_bytes)))
$$;
CREATE FUNCTION server_poll_archive_insert_guard() RETURNS trigger LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE a public.server_poll_storage_admissions%ROWTYPE; payload jsonb; activity timestamptz;
BEGIN
payload:=public.server_poll_storage_source_payload(NEW.receipt_id);
IF payload IS NULL OR NOT public.server_poll_storage_accounting_valid() THEN RAISE EXCEPTION 'poll source is not eligible for terminal archive'; END IF;
SELECT * INTO a FROM public.server_poll_storage_admissions WHERE receipt_id=NEW.receipt_id FOR UPDATE;
SELECT GREATEST(a.authority_expires_at,a.created_at,a.claim_attempted_at,(payload#>>'{terminal,operation,finished_at}')::timestamptz,
(payload#>>'{terminal,plan,expires_at}')::timestamptz,(payload#>>'{terminal,step,finished_at}')::timestamptz,
(payload#>>'{journal,execution,created_at}')::timestamptz,(payload#>>'{journal,receipt,created_at}')::timestamptz,
(SELECT max((v->>'created_at')::timestamptz) FROM jsonb_array_elements(payload#>'{journal,intents}') v),
(SELECT max((v->>'created_at')::timestamptz) FROM jsonb_array_elements(payload#>'{journal,acks}') v)) INTO activity;
IF a.compaction_claim IS NULL OR a.compaction_claimed_at IS NULL OR NOT isfinite(a.compaction_claimed_at)
OR a.compaction_claim_xid IS NULL OR a.compaction_claim_xid=pg_current_xact_id()
OR NEW.version<>1 OR NEW.execution_id<>a.execution_id OR NEW.connection_id<>a.connection_id OR NEW.owner_id<>a.owner_id
OR NEW.binding_proof<>a.binding_proof OR NEW.context_proof<>a.context_proof OR NEW.original_expires_at<>a.authority_expires_at
OR NEW.original_activity_at<>activity OR NEW.decoded_bytes<>octet_length(payload::text)
OR NEW.source_sha256<>encode(sha256(convert_to(payload::text,'UTF8')),'hex')
OR NEW.ciphertext_bytes<>octet_length(NEW.ciphertext) OR NEW.ciphertext_bytes>131072
OR NEW.ciphertext !~ '^[A-Za-z0-9+/]+={0,2}$'
OR replace(encode(decode(NEW.ciphertext,'base64'),'base64'),E'\n','')<>NEW.ciphertext
OR octet_length(decode(NEW.ciphertext,'base64'))<>NEW.compressed_bytes+28
THEN RAISE EXCEPTION 'poll archive does not match the entire immutable source'; END IF;
NEW.created_at:=clock_timestamp(); NEW.compaction_xid:=pg_current_xact_id();
RETURN NEW;
END $$;
CREATE TRIGGER server_poll_archive_insert_guard BEFORE INSERT ON server_poll_archives FOR EACH ROW EXECUTE FUNCTION server_poll_archive_insert_guard();
CREATE OR REPLACE FUNCTION server_poll_journal_immutable() RETURNS trigger LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$ BEGIN
IF TG_OP='DELETE' AND EXISTS(SELECT 1 FROM public.server_poll_archives r JOIN public.server_poll_storage_admissions a USING(receipt_id)
WHERE r.receipt_id=OLD.receipt_id AND r.compaction_xid=pg_current_xact_id() AND a.storage_state='full') THEN RETURN OLD; END IF;
IF TG_OP='DELETE' AND public.server_poll_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'server poll journal is immutable outside complete archive or original lifetime GC';
END $$;
CREATE OR REPLACE FUNCTION server_poll_storage_admission_guard() RETURNS trigger LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE retention integer; charged integer; r public.server_poll_archives%ROWTYPE;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
IF TG_OP='DELETE' THEN
IF NOT public.server_poll_storage_gc_allowed(OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_executions WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_intents WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_acks WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_receipts WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_quarantines WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_archives WHERE receipt_id=OLD.receipt_id) THEN RAISE EXCEPTION 'poll lifetime or children prevent admission deletion'; END IF;
UPDATE public.server_poll_storage_usage SET admissions=admissions-CASE WHEN OLD.storage_state='full' THEN 1 ELSE 0 END,
compacted_admissions=compacted_admissions-CASE WHEN OLD.storage_state='compacted' THEN 1 ELSE 0 END,
archive_bytes=archive_bytes-COALESCE(OLD.archive_bytes,0),
reserved_rows=reserved_rows-CASE WHEN OLD.storage_state='full' THEN 4019 ELSE 2 END,
reserved_bytes=reserved_bytes-CASE WHEN OLD.storage_state='full' THEN 8421376 ELSE 32768+OLD.archive_bytes END
WHERE scope IN('global','owner:'||OLD.owner_id);
GET DIAGNOSTICS charged=ROW_COUNT; IF charged<>2 THEN RAISE EXCEPTION 'poll quota refund mismatch'; END IF; RETURN OLD;
END IF;
IF TG_OP='UPDATE' THEN
-- This durable one-shot marker is written and ACKed in its OWN transaction
-- before archive construction. A committed/unknown claim never resets and
-- the worker never retries destructive work against an already claimed full
-- source. Claim time is maintenance metadata, not renewed execution activity.
IF OLD.storage_state='full' AND NEW.storage_state='full' AND OLD.compaction_claim IS NULL AND OLD.compaction_claimed_at IS NULL AND OLD.compaction_claim_xid IS NULL
AND NEW.compaction_claim IS NOT NULL AND NEW.compaction_claim_xid IS NULL
AND(to_jsonb(OLD)-ARRAY['compaction_claim','compaction_claimed_at','compaction_claim_xid'])=(to_jsonb(NEW)-ARRAY['compaction_claim','compaction_claimed_at','compaction_claim_xid'])
AND public.server_poll_storage_compact_allowed(OLD.receipt_id)
THEN NEW.compaction_claimed_at:=clock_timestamp(); NEW.compaction_claim_xid:=pg_current_xact_id(); RETURN NEW; END IF;
IF(to_jsonb(OLD)-ARRAY['claim_token','claim_attempted_at','used_rows','used_bytes','storage_state','compact_at','archive_bytes','original_activity_at']) IS DISTINCT FROM
(to_jsonb(NEW)-ARRAY['claim_token','claim_attempted_at','used_rows','used_bytes','storage_state','compact_at','archive_bytes','original_activity_at']) THEN RAISE EXCEPTION 'poll admission identity is immutable'; END IF;
IF OLD.storage_state<>'full' THEN RAISE EXCEPTION 'compacted poll identity cannot reopen or change'; END IF;
IF NEW.storage_state='compacted' THEN
SELECT * INTO r FROM public.server_poll_archives WHERE receipt_id=OLD.receipt_id AND compaction_xid=pg_current_xact_id();
IF NOT FOUND OR NEW.used_rows<>OLD.used_rows OR NEW.used_bytes<>OLD.used_bytes OR NEW.claim_token IS DISTINCT FROM OLD.claim_token
OR NEW.claim_attempted_at IS DISTINCT FROM OLD.claim_attempted_at OR NEW.archive_bytes<>r.ciphertext_bytes OR NEW.original_activity_at<>r.original_activity_at
OR EXISTS(SELECT 1 FROM public.server_poll_executions WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_intents WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_acks WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_receipts WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_quarantines WHERE receipt_id=OLD.receipt_id)
THEN RAISE EXCEPTION 'poll compaction requires the complete current-transaction archive'; END IF;
NEW.compact_at:=r.created_at;
UPDATE public.server_poll_storage_usage SET admissions=admissions-1,compacted_admissions=compacted_admissions+1,
archive_bytes=archive_bytes+r.ciphertext_bytes,reserved_rows=reserved_rows-4019+2,reserved_bytes=reserved_bytes-8421376+32768+r.ciphertext_bytes
WHERE scope IN('global','owner:'||OLD.owner_id) AND admissions>=1;
GET DIAGNOSTICS charged=ROW_COUNT; IF charged<>2 THEN RAISE EXCEPTION 'poll compaction charge mismatch'; END IF;
RETURN NEW;
END IF;
IF NEW.compact_at IS NOT NULL OR NEW.archive_bytes IS NOT NULL OR NEW.original_activity_at IS NOT NULL THEN RAISE EXCEPTION 'full poll cannot carry archive metadata'; END IF;
IF OLD.claim_token IS NOT NULL AND NEW.claim_token IS NULL AND OLD.claim_attempted_at IS NULL AND NEW.claim_attempted_at IS NOT NULL
AND NEW.used_rows=OLD.used_rows AND NEW.used_bytes=OLD.used_bytes AND OLD.authority_expires_at>clock_timestamp()
THEN NEW.claim_attempted_at:=clock_timestamp(); RETURN NEW; END IF;
IF OLD.claim_token IS NOT DISTINCT FROM NEW.claim_token AND OLD.claim_attempted_at IS NOT DISTINCT FROM NEW.claim_attempted_at
AND NEW.used_rows=OLD.used_rows+1 AND NEW.used_bytes>OLD.used_bytes THEN RETURN NEW; END IF;
RAISE EXCEPTION 'poll admission cannot refresh, reopen or refund itself';
END IF;
IF NEW.storage_state<>'full' OR NEW.compact_at IS NOT NULL OR NEW.archive_bytes IS NOT NULL OR NEW.original_activity_at IS NOT NULL
OR NEW.compaction_claim IS NOT NULL OR NEW.compaction_claimed_at IS NOT NULL OR NEW.compaction_claim_xid IS NOT NULL
OR NOT public.server_poll_storage_accounting_valid() THEN RAISE EXCEPTION 'new poll requires full reservation and exact ledger'; END IF;
SELECT requests_retention_days INTO retention FROM public.mcp_settings WHERE id=1;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365 OR NEW.authority_issued_at>clock_timestamp() OR NEW.authority_expires_at<=clock_timestamp()
OR NEW.claim_token IS NULL OR NEW.claim_attempted_at IS NOT NULL OR NEW.used_rows<>0 OR NEW.used_bytes<>0 THEN RAISE EXCEPTION 'new poll admission needs original live authority'; END IF;
IF EXISTS(SELECT 1 FROM public.server_poll_storage_admissions WHERE receipt_id=NEW.receipt_id OR(identity_kind=NEW.identity_kind AND execution_id=NEW.execution_id)) THEN RAISE EXCEPTION 'poll identity already spent'; END IF;
NEW.created_at:=clock_timestamp(); NEW.retention_days:=retention; NEW.safety_ms:=60000;
NEW.gc_not_before:=NEW.authority_expires_at+make_interval(days=>retention)+interval '60 seconds';
INSERT INTO public.server_poll_storage_usage(scope) VALUES('owner:'||NEW.owner_id) ON CONFLICT DO NOTHING;
IF(SELECT count(*) FROM public.server_poll_storage_usage WHERE scope IN('global','owner:'||NEW.owner_id)
AND admissions<CASE WHEN scope='global' THEN 128 ELSE 8 END
AND reserved_rows<=CASE WHEN scope='global' THEN 514432 ELSE 32152 END-4019
AND reserved_bytes<=CASE WHEN scope='global' THEN 1077936128 ELSE 67371008 END-8421376)<>2 THEN RAISE EXCEPTION 'poll admission quota exhausted'; END IF;
UPDATE public.server_poll_storage_usage SET admissions=admissions+1,reserved_rows=reserved_rows+4019,reserved_bytes=reserved_bytes+8421376 WHERE scope IN('global','owner:'||NEW.owner_id);
GET DIAGNOSTICS charged=ROW_COUNT; IF charged<>2 THEN RAISE EXCEPTION 'poll admission quota unavailable'; END IF;
RETURN NEW;
END $$;
CREATE FUNCTION server_poll_archive_insert_finish() RETURNS trigger LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$ BEGIN
DELETE FROM public.server_poll_acks WHERE receipt_id=NEW.receipt_id;
DELETE FROM public.server_poll_intents WHERE receipt_id=NEW.receipt_id;
DELETE FROM public.server_poll_receipts WHERE receipt_id=NEW.receipt_id;
DELETE FROM public.server_poll_executions WHERE receipt_id=NEW.receipt_id;
UPDATE public.server_poll_storage_admissions SET storage_state='compacted',compact_at=NEW.created_at,archive_bytes=NEW.ciphertext_bytes,original_activity_at=NEW.original_activity_at WHERE receipt_id=NEW.receipt_id AND storage_state='full';
IF NOT FOUND THEN RAISE EXCEPTION 'poll admission did not transition with archive'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER server_poll_archive_insert_finish AFTER INSERT ON server_poll_archives FOR EACH ROW EXECUTE FUNCTION server_poll_archive_insert_finish();
CREATE OR REPLACE FUNCTION server_poll_storage_gc_allowed(id uuid) RETURNS boolean LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE a public.server_poll_storage_admissions%ROWTYPE; retention integer; activity timestamptz; terminal_at timestamptz;
BEGIN
IF current_setting('mm.poll_storage_gc',true) IS DISTINCT FROM 'on' THEN RETURN false; END IF;
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
SELECT * INTO a FROM public.server_poll_storage_admissions WHERE receipt_id=id FOR UPDATE;
IF NOT FOUND OR a.authority_expires_at>=clock_timestamp() OR a.gc_not_before>=clock_timestamp() THEN RETURN false; END IF;
SELECT requests_retention_days INTO retention FROM public.mcp_settings WHERE mcp_settings.id=1 FOR SHARE;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365 THEN RETURN false; END IF;
terminal_at:=a.created_at;
IF a.identity_kind='mcp' THEN
IF NOT pg_try_advisory_xact_lock(hashtextextended('mcp-operation:'||a.execution_id::text,0)) THEN RETURN false; END IF;
BEGIN
SELECT GREATEST(o.finished_at,p.expires_at) INTO terminal_at FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id JOIN public.mcp_connections c ON c.id=o.connection_id
WHERE o.id=a.execution_id AND o.connection_id=a.connection_id AND c.owner_id=a.owner_id AND p.operation_id='servers.poll'
AND o.state IN('applied','failed','unknown','rejected','partial','cancelled') AND o.finished_at IS NOT NULL FOR UPDATE OF o,p NOWAIT;
IF NOT FOUND OR terminal_at IS NULL THEN RETURN false; END IF;
EXCEPTION WHEN lock_not_available THEN RETURN false; END;
END IF;
IF a.storage_state='compacted' THEN activity:=GREATEST(a.original_activity_at,a.authority_expires_at,terminal_at);
ELSE
SELECT GREATEST(a.authority_expires_at,a.created_at,COALESCE(a.claim_attempted_at,a.created_at),terminal_at,
COALESCE((SELECT created_at FROM public.server_poll_executions WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(created_at) FROM public.server_poll_intents WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(created_at) FROM public.server_poll_acks WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM public.server_poll_receipts WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM public.server_poll_quarantines WHERE receipt_id=id),a.created_at)) INTO activity;
END IF;
RETURN isfinite(activity) AND activity+make_interval(days=>GREATEST(a.retention_days,retention))+a.safety_ms*interval '1 millisecond'<clock_timestamp();
END $$;
CREATE FUNCTION server_poll_archive_immutable() RETURNS trigger LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$ BEGIN
IF TG_OP='DELETE' AND public.server_poll_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'poll archive is immutable until original full retention GC';
END $$;
CREATE TRIGGER server_poll_archive_immutable BEFORE UPDATE OR DELETE ON server_poll_archives FOR EACH ROW EXECUTE FUNCTION server_poll_archive_immutable();
CREATE FUNCTION server_poll_archive_final_guard() RETURNS trigger LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
DECLARE id uuid; a public.server_poll_storage_admissions%ROWTYPE;
BEGIN
id:=CASE WHEN TG_OP='DELETE' THEN OLD.receipt_id ELSE NEW.receipt_id END;
SELECT * INTO a FROM public.server_poll_storage_admissions WHERE receipt_id=id;
IF FOUND AND a.storage_state='compacted' AND(EXISTS(SELECT 1 FROM public.server_poll_executions WHERE receipt_id=id)
OR EXISTS(SELECT 1 FROM public.server_poll_intents WHERE receipt_id=id) OR EXISTS(SELECT 1 FROM public.server_poll_acks WHERE receipt_id=id)
OR EXISTS(SELECT 1 FROM public.server_poll_receipts WHERE receipt_id=id) OR EXISTS(SELECT 1 FROM public.server_poll_quarantines WHERE receipt_id=id))
THEN RAISE EXCEPTION 'compacted poll retained incomplete original children'; END IF;
IF NOT public.server_poll_storage_accounting_valid() THEN RAISE EXCEPTION 'poll final archive or exact ledger mismatch'; END IF;
RETURN NULL;
END $$;
CREATE CONSTRAINT TRIGGER server_poll_archive_final_guard AFTER INSERT OR DELETE ON server_poll_archives DEFERRABLE INITIALLY DEFERRED FOR EACH ROW EXECUTE FUNCTION server_poll_archive_final_guard();
CREATE CONSTRAINT TRIGGER server_poll_admission_final_guard AFTER INSERT OR DELETE ON server_poll_storage_admissions DEFERRABLE INITIALLY DEFERRED FOR EACH ROW EXECUTE FUNCTION server_poll_archive_final_guard();
CREATE CONSTRAINT TRIGGER server_poll_compact_final_guard AFTER UPDATE ON server_poll_storage_admissions DEFERRABLE INITIALLY DEFERRED FOR EACH ROW WHEN(OLD.storage_state IS DISTINCT FROM NEW.storage_state) EXECUTE FUNCTION server_poll_archive_final_guard();
-- Existing row guard additionally refuses compacted tombstones, including late
-- quarantine attempts; the original meter and immutable identity stay intact.
CREATE FUNCTION server_poll_compacted_row_guard() RETURNS trigger LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$ BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
IF EXISTS(SELECT 1 FROM public.server_poll_storage_admissions WHERE receipt_id=NEW.receipt_id AND storage_state='compacted') THEN RAISE EXCEPTION 'compacted poll journal cannot reopen'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER server_poll_compacted_anchor_guard BEFORE INSERT ON server_poll_executions FOR EACH ROW EXECUTE FUNCTION server_poll_compacted_row_guard();
CREATE TRIGGER server_poll_compacted_intent_guard BEFORE INSERT ON server_poll_intents FOR EACH ROW EXECUTE FUNCTION server_poll_compacted_row_guard();
CREATE TRIGGER server_poll_compacted_ack_guard BEFORE INSERT ON server_poll_acks FOR EACH ROW EXECUTE FUNCTION server_poll_compacted_row_guard();
CREATE TRIGGER server_poll_compacted_receipt_guard BEFORE INSERT ON server_poll_receipts FOR EACH ROW EXECUTE FUNCTION server_poll_compacted_row_guard();
CREATE TRIGGER server_poll_compacted_quarantine_guard BEFORE INSERT ON server_poll_quarantines FOR EACH ROW EXECUTE FUNCTION server_poll_compacted_row_guard();
INSERT INTO schema_migrations(id) VALUES('0041_server_poll_terminal_archives') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,49 @@
-- Approval-time configuration identity. DDL and its marker commit atomically.
BEGIN;
-- Existing rows retain all fields; the new epoch invalidates old complete proofs.
-- This schema addition does not activate the delayed-approval execution protocol.
LOCK TABLE public.traffic_settings, public.uptime_settings,
public.internet_path_settings IN ACCESS EXCLUSIVE MODE;
ALTER TABLE public.traffic_settings ADD COLUMN poll_configuration_epoch uuid NOT NULL DEFAULT pg_catalog.gen_random_uuid();
ALTER TABLE public.uptime_settings ADD COLUMN poll_configuration_epoch uuid NOT NULL DEFAULT pg_catalog.gen_random_uuid();
ALTER TABLE public.internet_path_settings ADD COLUMN poll_configuration_epoch uuid NOT NULL DEFAULT pg_catalog.gen_random_uuid();
CREATE FUNCTION public.server_poll_settings_epoch_guard() RETURNS trigger
LANGUAGE plpgsql SET search_path = pg_catalog AS $epoch$
BEGIN
IF TG_TABLE_SCHEMA <> 'public' OR TG_TABLE_NAME NOT IN
('traffic_settings', 'uptime_settings', 'internet_path_settings')
OR TG_LEVEL <> 'ROW' OR TG_WHEN <> 'BEFORE' THEN
RAISE EXCEPTION 'unexpected poll settings epoch trigger target';
END IF;
IF TG_OP = 'INSERT' THEN
-- Never accept an epoch supplied by an importer or application caller.
NEW.poll_configuration_epoch := pg_catalog.gen_random_uuid();
ELSIF TG_OP = 'UPDATE' THEN
-- Every unknown/future field is configuration by default. Only these four
-- current collector fields are excluded; created_at stays significant.
IF (pg_catalog.to_jsonb(NEW) - ARRAY['poll_configuration_epoch',
'last_collected_at', 'last_duration_ms', 'last_error', 'updated_at'])
IS DISTINCT FROM
(pg_catalog.to_jsonb(OLD) - ARRAY['poll_configuration_epoch',
'last_collected_at', 'last_duration_ms', 'last_error', 'updated_at']) THEN
NEW.poll_configuration_epoch := pg_catalog.gen_random_uuid();
ELSE
NEW.poll_configuration_epoch := OLD.poll_configuration_epoch;
END IF;
ELSE
RAISE EXCEPTION 'unexpected poll settings epoch trigger operation';
END IF;
RETURN NEW;
END;
$epoch$;
CREATE TRIGGER server_poll_settings_epoch BEFORE INSERT OR UPDATE ON public.traffic_settings
FOR EACH ROW EXECUTE FUNCTION public.server_poll_settings_epoch_guard();
CREATE TRIGGER server_poll_settings_epoch BEFORE INSERT OR UPDATE ON public.uptime_settings
FOR EACH ROW EXECUTE FUNCTION public.server_poll_settings_epoch_guard();
CREATE TRIGGER server_poll_settings_epoch BEFORE INSERT OR UPDATE ON public.internet_path_settings
FOR EACH ROW EXECUTE FUNCTION public.server_poll_settings_epoch_guard();
ALTER TABLE public.traffic_settings ENABLE ALWAYS TRIGGER server_poll_settings_epoch;
ALTER TABLE public.uptime_settings ENABLE ALWAYS TRIGGER server_poll_settings_epoch;
ALTER TABLE public.internet_path_settings ENABLE ALWAYS TRIGGER server_poll_settings_epoch;
INSERT INTO schema_migrations(id) VALUES('0042_server_poll_settings_epoch') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,62 @@
-- Durable persistent-forward charge; no execution authority is issued.
-- Reuses the accepted B2.a immutable-parent charging rule, narrowed to ONE
-- forward charge. Existing 0040 two-slot reservations/0031 remain untouched.
-- This SQL cannot authenticate a binding or grant execution authority.
BEGIN;
DO $$ BEGIN
IF to_regclass('public.mcp_routing_forward_charges') IS NOT NULL
THEN RAISE EXCEPTION 'unmarked forward charge storage requires explicit reconciliation'; END IF;
END $$;
CREATE TABLE public.mcp_routing_forward_charges (
-- Logical historical identities, intentionally NOT lifetime FKs. INSERT is
-- checked against the actual current source below; later source/profile
-- retention cannot erase this charge or be blocked forever by old charges.
source_operation_id uuid PRIMARY KEY,
source_plan_id uuid NOT NULL,
source_step_index integer NOT NULL CHECK(source_step_index=0),
profile_id uuid NOT NULL,
connection_id uuid NOT NULL,
charge_units integer NOT NULL DEFAULT 1 CHECK(charge_units=1),
binding_sealed text NOT NULL CHECK(octet_length(binding_sealed) BETWEEN 1 AND 262144),
binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'),
created_at timestamptz NOT NULL
);
CREATE INDEX mcp_routing_forward_charges_window
ON public.mcp_routing_forward_charges(profile_id,created_at);
CREATE INDEX mcp_routing_forward_charges_retention
ON public.mcp_routing_forward_charges(created_at,source_operation_id);
-- The immutable parent IS the charged unit. No mutable status/child can drop it
-- from the hourly projection after reject/unknown, human save or restart.
CREATE FUNCTION public.mcp_routing_forward_charge_source() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF NOT EXISTS(
SELECT 1 FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_routing_profiles r ON r.id=NEW.profile_id AND r.connection_id=o.connection_id
WHERE o.id=NEW.source_operation_id AND p.id=NEW.source_plan_id
AND o.connection_id=NEW.connection_id AND o.state='pending' AND p.state='executing'
AND p.operation_id='routing.community.switch' AND p.operation_version=7
) THEN RAISE EXCEPTION 'forward charge differs from actual pending admission source'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER mcp_routing_forward_charge_source BEFORE INSERT ON public.mcp_routing_forward_charges
FOR EACH ROW EXECUTE FUNCTION public.mcp_routing_forward_charge_source();
CREATE FUNCTION public.mcp_routing_forward_charge_immutable() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE retention integer;
BEGIN
IF TG_OP<>'DELETE' THEN RAISE EXCEPTION 'immutable forward charge; rewrite/refund forbidden'; END IF;
SELECT requests_retention_days INTO retention FROM public.mcp_settings WHERE id=1 FOR SHARE;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365
OR OLD.created_at>clock_timestamp()-interval '1 hour'
OR OLD.created_at>=clock_timestamp()-make_interval(days=>retention)
OR EXISTS(SELECT 1 FROM public.mcp_operations WHERE id=OLD.source_operation_id AND state IN('pending','running'))
THEN RAISE EXCEPTION 'forward charge is still retained or unresolved'; END IF;
RETURN OLD;
END $$;
CREATE TRIGGER mcp_routing_forward_charge_immutable BEFORE UPDATE OR DELETE ON public.mcp_routing_forward_charges
FOR EACH ROW EXECUTE FUNCTION public.mcp_routing_forward_charge_immutable();
CREATE TRIGGER mcp_routing_forward_charge_no_truncate BEFORE TRUNCATE ON public.mcp_routing_forward_charges
FOR EACH STATEMENT EXECUTE FUNCTION public.mcp_routing_forward_charge_immutable();
-- DDL and exact file marker commit atomically. No SECURITY DEFINER or grants.
INSERT INTO public.schema_migrations (id) VALUES ('0043_mcp_routing_forward_charges') ON CONFLICT (id) DO NOTHING;
COMMIT;
@@ -0,0 +1,93 @@
-- Additive durable scoped manual inverse-one charge. No execution grant.
-- Existing 0043 is immutable. Logical charge identities outlive family FKs.
BEGIN;
DO $$ BEGIN
IF to_regclass('public.mcp_routing_manual_inverse_charges') IS NOT NULL
THEN RAISE EXCEPTION 'unmarked manual inverse storage requires reconciliation'; END IF;
END $$;
CREATE TABLE public.mcp_routing_manual_inverse_charges (
recovery_operation_id uuid PRIMARY KEY,
recovery_plan_id uuid NOT NULL,
source_operation_id uuid NOT NULL,
source_plan_id uuid NOT NULL,
source_step_index integer NOT NULL CHECK(source_step_index=0),
profile_id uuid NOT NULL,
profile_version integer NOT NULL CHECK(profile_version>0),
connection_id uuid NOT NULL,
owner_id text NOT NULL CHECK(octet_length(owner_id) BETWEEN 1 AND 1000),
charge_units integer NOT NULL CHECK(charge_units=1),
binding_sealed text NOT NULL CHECK(octet_length(binding_sealed) BETWEEN 1 AND 262144),
binding_proof text NOT NULL CHECK(binding_proof ~ '^[a-f0-9]{64}$'),
created_at timestamptz NOT NULL,
CONSTRAINT mcp_manual_inverse_source_step_unique UNIQUE(source_operation_id,source_step_index)
);
CREATE INDEX mcp_routing_manual_inverse_charges_window
ON public.mcp_routing_manual_inverse_charges(profile_id,created_at);
CREATE INDEX mcp_routing_manual_inverse_charges_retention
ON public.mcp_routing_manual_inverse_charges(created_at,recovery_operation_id);
CREATE INDEX mcp_scoped_recovery_manual_history
ON public.mcp_scoped_recovery_intents(mode,source_operation_id,source_step_index)
WHERE mode='manual';
CREATE FUNCTION public.mcp_routing_manual_inverse_charge_source() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF NOT EXISTS(
SELECT 1 FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_routing_profiles r ON r.id=NEW.profile_id AND r.connection_id=o.connection_id
JOIN public.mcp_operations source ON source.id=NEW.source_operation_id AND source.connection_id=o.connection_id
JOIN public.mcp_plans sp ON sp.id=source.plan_id AND sp.connection_id=source.connection_id
JOIN public.mcp_connections c ON c.id=o.connection_id
WHERE o.id=NEW.recovery_operation_id AND p.id=NEW.recovery_plan_id
AND o.connection_id=NEW.connection_id AND c.owner_id=NEW.owner_id
AND r.version=NEW.profile_version AND o.state='pending' AND p.state='executing'
AND p.operation_id='operations.recover' AND p.operation_version=7
AND sp.id=NEW.source_plan_id AND sp.operation_id='routing.community.switch' AND sp.operation_version=7
AND source.state IN('applied','partial')
) THEN RAISE EXCEPTION 'manual inverse charge differs from pending admission'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER mcp_routing_manual_inverse_charge_source BEFORE INSERT ON public.mcp_routing_manual_inverse_charges
FOR EACH ROW EXECUTE FUNCTION public.mcp_routing_manual_inverse_charge_source();
CREATE FUNCTION public.mcp_routing_manual_inverse_charge_immutable() RETURNS trigger LANGUAGE plpgsql AS $$
DECLARE retention integer;
BEGIN
IF TG_OP<>'DELETE' THEN RAISE EXCEPTION 'immutable manual inverse charge; rewrite/refund forbidden'; END IF;
SELECT requests_retention_days INTO retention FROM public.mcp_settings WHERE id=1 FOR SHARE;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365
OR OLD.created_at>clock_timestamp()-interval '1 hour'
OR OLD.created_at>=clock_timestamp()-make_interval(days=>retention)
OR EXISTS(SELECT 1 FROM public.mcp_operations WHERE id=OLD.recovery_operation_id AND state IN('pending','running'))
THEN RAISE EXCEPTION 'manual inverse charge still retained or unresolved'; END IF;
RETURN OLD;
END $$;
CREATE TRIGGER mcp_routing_manual_inverse_charge_immutable BEFORE UPDATE OR DELETE ON public.mcp_routing_manual_inverse_charges
FOR EACH ROW EXECUTE FUNCTION public.mcp_routing_manual_inverse_charge_immutable();
CREATE TRIGGER mcp_routing_manual_inverse_charge_no_truncate BEFORE TRUNCATE ON public.mcp_routing_manual_inverse_charges
FOR EACH STATEMENT EXECUTE FUNCTION public.mcp_routing_manual_inverse_charge_immutable();
-- This is a version/identity fence, never authentication of encrypted source.
-- Old binaries cannot create a fresh v6 manual intent after this DDL commits.
-- Already-claimed old work requires the normal drained-writer cutover.
CREATE FUNCTION public.mcp_scoped_recovery_manual_version() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF NEW.mode='manual' AND NOT pg_try_advisory_xact_lock(hashtextextended('mcp-runner-admission',0))
THEN RAISE EXCEPTION 'manual intent admission snapshot busy'; END IF;
IF NEW.mode='manual' AND NOT EXISTS(
SELECT 1 FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_operation_steps s ON s.operation_id=o.id AND s.step_index=0
JOIN public.mcp_operations source ON source.id=NEW.source_operation_id AND source.connection_id=o.connection_id
WHERE o.id=NEW.recovery_operation_id AND o.state='running'
AND p.state='executing' AND p.operation_id='operations.recover' AND p.operation_version=7
AND NEW.source_step_index=0 AND s.state='intent'
AND s.intended_state->>'kind'='service'
AND s.intended_state->>'service'='routing.community.scoped.recover'
AND s.intended_state->'input'->>'mode'='manual'
AND s.intended_state->'input'->>'sourceOperationId'=NEW.source_operation_id::text
AND s.intended_state->'input'->>'receiptId'=NEW.recovery_receipt_id::text
) THEN RAISE EXCEPTION 'manual inverse requires current exact running recovery'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER mcp_scoped_recovery_manual_version BEFORE INSERT ON public.mcp_scoped_recovery_intents
FOR EACH ROW EXECUTE FUNCTION public.mcp_scoped_recovery_manual_version();
INSERT INTO public.schema_migrations(id) VALUES('0044_mcp_routing_manual_inverse_charges') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,13 @@
BEGIN;
CREATE TABLE public.mcp_operation_replay_tombstones (
operation_id uuid PRIMARY KEY,
plan_id uuid NOT NULL UNIQUE,
connection_id uuid NOT NULL REFERENCES public.mcp_connections(id),
idempotency_digest text NOT NULL CHECK (idempotency_digest ~ '^[0-9a-f]{64}$'),
action_id text NOT NULL CHECK (length(action_id) BETWEEN 1 AND 200),
target_server_ids bigint[] NOT NULL CHECK (array_position(target_server_ids, NULL) IS NULL),
target_global boolean NOT NULL,
UNIQUE (connection_id,idempotency_digest)
);
INSERT INTO public.schema_migrations(id) VALUES ('0045_mcp_operation_replay_tombstones') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,186 @@
-- Inactive poll-history V1-to-V2 retirement. Application owner authenticates
-- both ciphertext versions; SQL independently fences identity and lifetime.
BEGIN;
LOCK TABLE public.server_poll_archives,public.server_poll_storage_admissions IN ACCESS EXCLUSIVE MODE;
ALTER TABLE public.server_poll_archives DROP CONSTRAINT server_poll_archives_version_check;
ALTER TABLE public.server_poll_archives ADD CONSTRAINT server_poll_archives_version_check
CHECK(version=1 OR (version=2 AND decoded_bytes<=24576 AND compressed_bytes<=24576 AND ciphertext_bytes<=33000));
CREATE OR REPLACE FUNCTION public.server_poll_archive_immutable() RETURNS trigger
LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='DELETE' AND public.server_poll_storage_gc_allowed(OLD.receipt_id) THEN RETURN OLD; END IF;
IF TG_OP='UPDATE' AND OLD.version=1 AND NEW.version=2
AND current_setting('mm.poll_retire_receipt',true)=OLD.receipt_id::text
AND NEW.ciphertext_bytes<=OLD.ciphertext_bytes
AND NEW.ciphertext ~ '^[A-Za-z0-9+/]+={0,2}$'
AND replace(encode(decode(NEW.ciphertext,'base64'),'base64'),E'\n','')=NEW.ciphertext
AND octet_length(decode(NEW.ciphertext,'base64'))=NEW.compressed_bytes+28
AND (to_jsonb(OLD)-ARRAY['version','source_digest','source_sha256','decoded_bytes','compressed_bytes','ciphertext','ciphertext_bytes','archive_proof'])=
(to_jsonb(NEW)-ARRAY['version','source_digest','source_sha256','decoded_bytes','compressed_bytes','ciphertext','ciphertext_bytes','archive_proof'])
AND EXISTS(
SELECT 1 FROM public.server_poll_storage_admissions a JOIN public.mcp_operations o ON o.id=a.execution_id AND o.connection_id=a.connection_id
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_operation_steps s ON s.operation_id=o.id AND s.step_index=0
JOIN public.mcp_operation_replay_tombstones t ON t.operation_id=o.id AND t.plan_id=p.id AND t.connection_id=o.connection_id
JOIN public.mcp_connections c ON c.id=o.connection_id AND c.owner_id=a.owner_id
WHERE a.receipt_id=OLD.receipt_id AND a.identity_kind='mcp' AND a.storage_state='compacted'
AND a.execution_id=OLD.execution_id AND a.connection_id=OLD.connection_id AND a.owner_id=OLD.owner_id
AND a.binding_proof=OLD.binding_proof AND a.context_proof=OLD.context_proof AND a.archive_bytes=OLD.ciphertext_bytes
AND a.original_activity_at=OLD.original_activity_at AND a.authority_expires_at=OLD.original_expires_at
AND o.state='applied' AND o.verification='passed' AND o.error_code IS NULL AND isfinite(o.finished_at)
AND o.finished_at<transaction_timestamp()-interval '180 days'
AND p.state='completed' AND p.operation_id='servers.poll' AND p.operation_version=1 AND p.approved_by IS NOT NULL
AND s.state='verified' AND s.finished_at IS NOT NULL AND t.action_id=p.operation_id
AND t.target_global AND cardinality(t.target_server_ids)>0
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'poll archive immutable outside exact retirement or original lifetime GC';
END $$;
CREATE FUNCTION public.server_poll_retirement_final_guard() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
BEGIN
IF NOT EXISTS(SELECT 1 FROM public.mcp_operation_replay_tombstones t
WHERE t.operation_id=NEW.execution_id AND t.connection_id=NEW.connection_id AND t.action_id='servers.poll'
AND t.target_global AND NOT EXISTS(SELECT 1 FROM public.mcp_plans p WHERE p.id=t.plan_id))
OR EXISTS(SELECT 1 FROM public.mcp_operations o WHERE o.id=NEW.execution_id)
OR NOT public.server_poll_storage_accounting_valid()
THEN RAISE EXCEPTION 'poll retirement is incomplete'; END IF;
RETURN NULL;
END $$;
CREATE CONSTRAINT TRIGGER server_poll_retirement_final_guard AFTER UPDATE ON public.server_poll_archives
DEFERRABLE INITIALLY DEFERRED FOR EACH ROW WHEN(OLD.version=1 AND NEW.version=2)
EXECUTE FUNCTION public.server_poll_retirement_final_guard();
CREATE OR REPLACE FUNCTION server_poll_storage_admission_guard() RETURNS trigger LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE retention integer; charged integer; r public.server_poll_archives%ROWTYPE;
BEGIN
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
IF TG_OP='DELETE' THEN
IF NOT public.server_poll_storage_gc_allowed(OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_executions WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_intents WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_acks WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_receipts WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_quarantines WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_archives WHERE receipt_id=OLD.receipt_id) THEN RAISE EXCEPTION 'poll lifetime or children prevent admission deletion'; END IF;
UPDATE public.server_poll_storage_usage SET admissions=admissions-CASE WHEN OLD.storage_state='full' THEN 1 ELSE 0 END,
compacted_admissions=compacted_admissions-CASE WHEN OLD.storage_state='compacted' THEN 1 ELSE 0 END,
archive_bytes=archive_bytes-COALESCE(OLD.archive_bytes,0),
reserved_rows=reserved_rows-CASE WHEN OLD.storage_state='full' THEN 4019 ELSE 2 END,
reserved_bytes=reserved_bytes-CASE WHEN OLD.storage_state='full' THEN 8421376 ELSE 32768+OLD.archive_bytes END
WHERE scope IN('global','owner:'||OLD.owner_id);
GET DIAGNOSTICS charged=ROW_COUNT; IF charged<>2 THEN RAISE EXCEPTION 'poll quota refund mismatch'; END IF; RETURN OLD;
END IF;
IF TG_OP='UPDATE' THEN
IF OLD.storage_state='compacted' AND NEW.storage_state='compacted'
AND current_setting('mm.poll_retire_receipt',true)=OLD.receipt_id::text
AND (to_jsonb(OLD)-'archive_bytes')=(to_jsonb(NEW)-'archive_bytes') THEN
SELECT * INTO r FROM public.server_poll_archives WHERE receipt_id=OLD.receipt_id AND version=2;
IF NOT FOUND OR r.ciphertext_bytes<>NEW.archive_bytes OR NEW.archive_bytes>OLD.archive_bytes
OR r.execution_id<>OLD.execution_id OR r.connection_id<>OLD.connection_id OR r.owner_id<>OLD.owner_id
THEN RAISE EXCEPTION 'retired poll charge source mismatch'; END IF;
UPDATE public.server_poll_storage_usage
SET archive_bytes=archive_bytes-OLD.archive_bytes+NEW.archive_bytes,
reserved_bytes=reserved_bytes-OLD.archive_bytes+NEW.archive_bytes
WHERE scope IN ('global','owner:'||OLD.owner_id) AND compacted_admissions>=1 AND archive_bytes>=OLD.archive_bytes;
GET DIAGNOSTICS charged=ROW_COUNT;
IF charged<>2 THEN RAISE EXCEPTION 'retired poll charge adjustment mismatch'; END IF;
RETURN NEW;
END IF;
-- This durable one-shot marker is written and ACKed in its OWN transaction
-- before archive construction. A committed/unknown claim never resets and
-- the worker never retries destructive work against an already claimed full
-- source. Claim time is maintenance metadata, not renewed execution activity.
IF OLD.storage_state='full' AND NEW.storage_state='full' AND OLD.compaction_claim IS NULL AND OLD.compaction_claimed_at IS NULL AND OLD.compaction_claim_xid IS NULL
AND NEW.compaction_claim IS NOT NULL AND NEW.compaction_claim_xid IS NULL
AND(to_jsonb(OLD)-ARRAY['compaction_claim','compaction_claimed_at','compaction_claim_xid'])=(to_jsonb(NEW)-ARRAY['compaction_claim','compaction_claimed_at','compaction_claim_xid'])
AND public.server_poll_storage_compact_allowed(OLD.receipt_id)
THEN NEW.compaction_claimed_at:=clock_timestamp(); NEW.compaction_claim_xid:=pg_current_xact_id(); RETURN NEW; END IF;
IF(to_jsonb(OLD)-ARRAY['claim_token','claim_attempted_at','used_rows','used_bytes','storage_state','compact_at','archive_bytes','original_activity_at']) IS DISTINCT FROM
(to_jsonb(NEW)-ARRAY['claim_token','claim_attempted_at','used_rows','used_bytes','storage_state','compact_at','archive_bytes','original_activity_at']) THEN RAISE EXCEPTION 'poll admission identity is immutable'; END IF;
IF OLD.storage_state<>'full' THEN RAISE EXCEPTION 'compacted poll identity cannot reopen or change'; END IF;
IF NEW.storage_state='compacted' THEN
SELECT * INTO r FROM public.server_poll_archives WHERE receipt_id=OLD.receipt_id AND compaction_xid=pg_current_xact_id();
IF NOT FOUND OR NEW.used_rows<>OLD.used_rows OR NEW.used_bytes<>OLD.used_bytes OR NEW.claim_token IS DISTINCT FROM OLD.claim_token
OR NEW.claim_attempted_at IS DISTINCT FROM OLD.claim_attempted_at OR NEW.archive_bytes<>r.ciphertext_bytes OR NEW.original_activity_at<>r.original_activity_at
OR EXISTS(SELECT 1 FROM public.server_poll_executions WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_intents WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_acks WHERE receipt_id=OLD.receipt_id)
OR EXISTS(SELECT 1 FROM public.server_poll_receipts WHERE receipt_id=OLD.receipt_id) OR EXISTS(SELECT 1 FROM public.server_poll_quarantines WHERE receipt_id=OLD.receipt_id)
THEN RAISE EXCEPTION 'poll compaction requires the complete current-transaction archive'; END IF;
NEW.compact_at:=r.created_at;
UPDATE public.server_poll_storage_usage SET admissions=admissions-1,compacted_admissions=compacted_admissions+1,
archive_bytes=archive_bytes+r.ciphertext_bytes,reserved_rows=reserved_rows-4019+2,reserved_bytes=reserved_bytes-8421376+32768+r.ciphertext_bytes
WHERE scope IN('global','owner:'||OLD.owner_id) AND admissions>=1;
GET DIAGNOSTICS charged=ROW_COUNT; IF charged<>2 THEN RAISE EXCEPTION 'poll compaction charge mismatch'; END IF;
RETURN NEW;
END IF;
IF NEW.compact_at IS NOT NULL OR NEW.archive_bytes IS NOT NULL OR NEW.original_activity_at IS NOT NULL THEN RAISE EXCEPTION 'full poll cannot carry archive metadata'; END IF;
IF OLD.claim_token IS NOT NULL AND NEW.claim_token IS NULL AND OLD.claim_attempted_at IS NULL AND NEW.claim_attempted_at IS NOT NULL
AND NEW.used_rows=OLD.used_rows AND NEW.used_bytes=OLD.used_bytes AND OLD.authority_expires_at>clock_timestamp()
THEN NEW.claim_attempted_at:=clock_timestamp(); RETURN NEW; END IF;
IF OLD.claim_token IS NOT DISTINCT FROM NEW.claim_token AND OLD.claim_attempted_at IS NOT DISTINCT FROM NEW.claim_attempted_at
AND NEW.used_rows=OLD.used_rows+1 AND NEW.used_bytes>OLD.used_bytes THEN RETURN NEW; END IF;
RAISE EXCEPTION 'poll admission cannot refresh, reopen or refund itself';
END IF;
IF NEW.storage_state<>'full' OR NEW.compact_at IS NOT NULL OR NEW.archive_bytes IS NOT NULL OR NEW.original_activity_at IS NOT NULL
OR NEW.compaction_claim IS NOT NULL OR NEW.compaction_claimed_at IS NOT NULL OR NEW.compaction_claim_xid IS NOT NULL
OR NOT public.server_poll_storage_accounting_valid() THEN RAISE EXCEPTION 'new poll requires full reservation and exact ledger'; END IF;
SELECT requests_retention_days INTO retention FROM public.mcp_settings WHERE id=1;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365 OR NEW.authority_issued_at>clock_timestamp() OR NEW.authority_expires_at<=clock_timestamp()
OR NEW.claim_token IS NULL OR NEW.claim_attempted_at IS NOT NULL OR NEW.used_rows<>0 OR NEW.used_bytes<>0 THEN RAISE EXCEPTION 'new poll admission needs original live authority'; END IF;
IF EXISTS(SELECT 1 FROM public.server_poll_storage_admissions WHERE receipt_id=NEW.receipt_id OR(identity_kind=NEW.identity_kind AND execution_id=NEW.execution_id)) THEN RAISE EXCEPTION 'poll identity already spent'; END IF;
NEW.created_at:=clock_timestamp(); NEW.retention_days:=retention; NEW.safety_ms:=60000;
NEW.gc_not_before:=NEW.authority_expires_at+make_interval(days=>retention)+interval '60 seconds';
INSERT INTO public.server_poll_storage_usage(scope) VALUES('owner:'||NEW.owner_id) ON CONFLICT DO NOTHING;
IF(SELECT count(*) FROM public.server_poll_storage_usage WHERE scope IN('global','owner:'||NEW.owner_id)
AND admissions<CASE WHEN scope='global' THEN 128 ELSE 8 END
AND reserved_rows<=CASE WHEN scope='global' THEN 514432 ELSE 32152 END-4019
AND reserved_bytes<=CASE WHEN scope='global' THEN 1077936128 ELSE 67371008 END-8421376)<>2 THEN RAISE EXCEPTION 'poll admission quota exhausted'; END IF;
UPDATE public.server_poll_storage_usage SET admissions=admissions+1,reserved_rows=reserved_rows+4019,reserved_bytes=reserved_bytes+8421376 WHERE scope IN('global','owner:'||NEW.owner_id);
GET DIAGNOSTICS charged=ROW_COUNT; IF charged<>2 THEN RAISE EXCEPTION 'poll admission quota unavailable'; END IF;
RETURN NEW;
END $$;
CREATE OR REPLACE FUNCTION server_poll_storage_gc_allowed(id uuid) RETURNS boolean LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE a public.server_poll_storage_admissions%ROWTYPE; retention integer; activity timestamptz; terminal_at timestamptz;
BEGIN
IF current_setting('mm.poll_storage_gc',true) IS DISTINCT FROM 'on' THEN RETURN false; END IF;
PERFORM pg_advisory_xact_lock(hashtextextended('server-poll-storage-v1',0));
SELECT * INTO a FROM public.server_poll_storage_admissions WHERE receipt_id=id FOR UPDATE;
IF NOT FOUND OR a.authority_expires_at>=clock_timestamp() OR a.gc_not_before>=clock_timestamp() THEN RETURN false; END IF;
SELECT requests_retention_days INTO retention FROM public.mcp_settings WHERE mcp_settings.id=1 FOR SHARE;
IF retention IS NULL OR retention NOT BETWEEN 1 AND 365 THEN RETURN false; END IF;
terminal_at:=a.created_at;
IF a.identity_kind='mcp' THEN
IF NOT pg_try_advisory_xact_lock(hashtextextended('mcp-operation:'||a.execution_id::text,0)) THEN RETURN false; END IF;
BEGIN
SELECT GREATEST(o.finished_at,p.expires_at) INTO terminal_at FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id JOIN public.mcp_connections c ON c.id=o.connection_id
WHERE o.id=a.execution_id AND o.connection_id=a.connection_id AND c.owner_id=a.owner_id AND p.operation_id='servers.poll'
AND o.state IN('applied','failed','unknown','rejected','partial','cancelled') AND o.finished_at IS NOT NULL FOR UPDATE OF o,p NOWAIT;
IF NOT FOUND THEN
IF a.storage_state<>'compacted' OR EXISTS(SELECT 1 FROM public.mcp_operations o WHERE o.id=a.execution_id) OR NOT EXISTS (
SELECT 1 FROM public.mcp_operation_replay_tombstones t JOIN public.mcp_connections c ON c.id=t.connection_id
WHERE t.operation_id=a.execution_id AND t.connection_id=a.connection_id AND c.owner_id=a.owner_id
AND t.action_id='servers.poll' AND t.target_global AND cardinality(t.target_server_ids)>0
AND NOT EXISTS(SELECT 1 FROM public.mcp_plans WHERE mcp_plans.id=t.plan_id)
) OR EXISTS(SELECT 1 FROM public.server_poll_archives r WHERE r.receipt_id=id AND
(r.version<>2 OR r.execution_id<>a.execution_id OR r.connection_id<>a.connection_id OR r.owner_id<>a.owner_id
OR r.original_activity_at IS DISTINCT FROM a.original_activity_at OR r.original_expires_at<>a.authority_expires_at
OR r.binding_proof<>a.binding_proof OR r.context_proof<>a.context_proof OR r.ciphertext_bytes<>a.archive_bytes))
THEN RETURN false; END IF;
terminal_at:=a.original_activity_at;
END IF;
IF terminal_at IS NULL THEN RETURN false; END IF;
EXCEPTION WHEN lock_not_available THEN RETURN false; END;
END IF;
IF a.storage_state='compacted' THEN activity:=GREATEST(a.original_activity_at,a.authority_expires_at,terminal_at);
ELSE
SELECT GREATEST(a.authority_expires_at,a.created_at,COALESCE(a.claim_attempted_at,a.created_at),terminal_at,
COALESCE((SELECT created_at FROM public.server_poll_executions WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(created_at) FROM public.server_poll_intents WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(created_at) FROM public.server_poll_acks WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM public.server_poll_receipts WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM public.server_poll_quarantines WHERE receipt_id=id),a.created_at)) INTO activity;
END IF;
RETURN isfinite(activity) AND activity+make_interval(days=>GREATEST(a.retention_days,retention))+a.safety_ms*interval '1 millisecond'<clock_timestamp();
END $$;
INSERT INTO public.schema_migrations(id) VALUES ('0046_server_poll_operation_retirement') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,40 @@
BEGIN;
CREATE TABLE public.server_poll_retirement_claims (
receipt_id uuid PRIMARY KEY REFERENCES public.server_poll_storage_admissions(receipt_id) ON DELETE CASCADE,
old_archive_proof text NOT NULL CHECK(old_archive_proof ~ '^[0-9a-f]{64}$'),
claim_token uuid NOT NULL,
claim_xid xid8 NOT NULL DEFAULT pg_current_xact_id()
);
CREATE FUNCTION public.server_poll_retirement_claim_immutable() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='INSERT' AND NEW.claim_xid=pg_current_xact_id() AND EXISTS(
SELECT 1 FROM public.server_poll_archives r WHERE r.receipt_id=NEW.receipt_id
AND r.version=1 AND r.archive_proof=NEW.old_archive_proof
) THEN RETURN NEW; END IF;
IF TG_OP='DELETE' AND pg_trigger_depth()>1 AND NOT EXISTS(
SELECT 1 FROM public.server_poll_storage_admissions a WHERE a.receipt_id=OLD.receipt_id
) THEN RETURN OLD; END IF;
RAISE EXCEPTION 'poll retirement claim cannot be replaced or reopened';
END $$;
CREATE TRIGGER server_poll_retirement_claim_immutable
BEFORE INSERT OR UPDATE OR DELETE ON public.server_poll_retirement_claims
FOR EACH ROW EXECUTE FUNCTION public.server_poll_retirement_claim_immutable();
CREATE TRIGGER server_poll_retirement_claim_no_truncate
BEFORE TRUNCATE ON public.server_poll_retirement_claims
FOR EACH STATEMENT EXECUTE FUNCTION public.server_poll_retirement_claim_immutable();
CREATE FUNCTION public.server_poll_retirement_requires_claim() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
BEGIN
IF NOT EXISTS(SELECT 1 FROM public.server_poll_retirement_claims c
WHERE c.receipt_id=OLD.receipt_id AND c.old_archive_proof=OLD.archive_proof
AND c.claim_token::text=current_setting('mm.poll_retire_claim',true)
AND c.claim_xid<>pg_current_xact_id())
THEN RAISE EXCEPTION 'poll retirement requires prior exact claim'; END IF;
RETURN NEW;
END $$;
CREATE TRIGGER server_poll_retirement_requires_claim BEFORE UPDATE ON public.server_poll_archives
FOR EACH ROW WHEN(OLD.version=1 AND NEW.version=2)
EXECUTE FUNCTION public.server_poll_retirement_requires_claim();
INSERT INTO public.schema_migrations(id) VALUES ('0047_server_poll_retirement_claims') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,27 @@
BEGIN;
CREATE TABLE public.mcp_ordinary_retirement_claims (
operation_id uuid PRIMARY KEY,
plan_id uuid NOT NULL UNIQUE,
connection_id uuid NOT NULL REFERENCES public.mcp_connections(id),
source_proof text NOT NULL CHECK(source_proof ~ '^[0-9a-f]{64}$'),
claim_token uuid NOT NULL,
claim_xid xid8 NOT NULL DEFAULT pg_current_xact_id()
);
CREATE FUNCTION public.mcp_ordinary_retirement_claim_immutable() RETURNS trigger
LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='INSERT' AND NEW.claim_xid=pg_current_xact_id() AND EXISTS(
SELECT 1 FROM public.mcp_operations o JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
WHERE o.id=NEW.operation_id AND o.plan_id=NEW.plan_id AND o.connection_id=NEW.connection_id
AND o.state='applied' AND o.verification='passed' AND o.error_code IS NULL
AND isfinite(o.finished_at) AND o.finished_at<transaction_timestamp()-interval '180 days'
AND p.operation_id='logs.settings.update' AND p.operation_version=1 AND p.state='completed'
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'ordinary operation retirement claim cannot be replaced or reopened';
END $$;
CREATE TRIGGER mcp_ordinary_retirement_claim_immutable BEFORE INSERT OR UPDATE OR DELETE
ON public.mcp_ordinary_retirement_claims FOR EACH ROW EXECUTE FUNCTION public.mcp_ordinary_retirement_claim_immutable();
CREATE TRIGGER mcp_ordinary_retirement_claim_no_truncate BEFORE TRUNCATE
ON public.mcp_ordinary_retirement_claims FOR EACH STATEMENT EXECUTE FUNCTION public.mcp_ordinary_retirement_claim_immutable();
INSERT INTO public.schema_migrations(id) VALUES ('0048_mcp_ordinary_retirement_claims') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,18 @@
BEGIN;
CREATE OR REPLACE FUNCTION public.mcp_ordinary_retirement_claim_immutable() RETURNS trigger
LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='INSERT' AND NEW.claim_xid=pg_current_xact_id() AND EXISTS(
SELECT 1 FROM public.mcp_operations o JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
WHERE o.id=NEW.operation_id AND o.plan_id=NEW.plan_id AND o.connection_id=NEW.connection_id
AND o.state='applied' AND o.verification='passed' AND o.error_code IS NULL
AND isfinite(o.finished_at) AND o.finished_at<transaction_timestamp()-interval '180 days'
AND (p.operation_id,p.operation_version) IN (
('logs.settings.update',1),('traffic.settings.update',2),('uptime.settings.update',2),
('servers.api-ping.settings.update',2),('internet-path.settings.update',1)
) AND p.state='completed'
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'ordinary operation retirement claim cannot be replaced or reopened';
END $$;
INSERT INTO public.schema_migrations(id) VALUES ('0049_mcp_ordinary_telemetry_retirement') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,22 @@
BEGIN;
CREATE TABLE public.server_probe_retired_sources (
receipt_id uuid PRIMARY KEY REFERENCES public.server_probe_storage_admissions(receipt_id),
operation_id uuid NOT NULL UNIQUE REFERENCES public.mcp_operation_replay_tombstones(operation_id),
source jsonb NOT NULL CHECK(jsonb_typeof(source)='object' AND octet_length(source::text) BETWEEN 1 AND 8192),
proof text NOT NULL CHECK(proof ~ '^[0-9a-f]{64}$'),
CHECK((source->>'version') IS NOT DISTINCT FROM '1'
AND (source->>'receiptId') IS NOT DISTINCT FROM receipt_id::text
AND (source->>'executionId') IS NOT DISTINCT FROM operation_id::text)
);
CREATE FUNCTION public.server_probe_retired_source_immutable() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
BEGIN
RAISE EXCEPTION 'retired diagnostic source is immutable';
END $$;
CREATE TRIGGER server_probe_retired_source_immutable
BEFORE UPDATE OR DELETE ON public.server_probe_retired_sources
FOR EACH ROW EXECUTE FUNCTION public.server_probe_retired_source_immutable();
CREATE TRIGGER server_probe_retired_source_no_truncate BEFORE TRUNCATE ON public.server_probe_retired_sources
FOR EACH STATEMENT EXECUTE FUNCTION public.server_probe_retired_source_immutable();
INSERT INTO public.schema_migrations(id) VALUES ('0050_server_probe_retired_sources') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,52 @@
BEGIN;
CREATE TABLE public.server_probe_retirement_claims (
receipt_id uuid PRIMARY KEY,
operation_id uuid NOT NULL UNIQUE,
plan_id uuid NOT NULL UNIQUE,
connection_id uuid NOT NULL,
source_proof text NOT NULL CHECK(source_proof ~ '^[a-f0-9]{64}$'),
claim_token uuid NOT NULL,
claim_xid xid8 NOT NULL DEFAULT pg_current_xact_id()
);
CREATE FUNCTION public.server_probe_retirement_claim_guard() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public SET TimeZone='UTC' AS $$
BEGIN
IF TG_OP='INSERT' AND NEW.claim_xid=pg_current_xact_id() AND EXISTS(
SELECT 1 FROM public.server_probe_storage_admissions a
JOIN public.mcp_operations o ON o.id=a.execution_id
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_connections c ON c.id=o.connection_id AND c.owner_id=a.owner_id
WHERE a.receipt_id=NEW.receipt_id AND a.identity_kind='mcp' AND a.claim_token IS NULL
AND a.claim_attempted_at IS NOT NULL AND a.cancelled_at IS NULL AND a.purged_at IS NULL
AND o.id=NEW.operation_id AND p.id=NEW.plan_id AND c.id=NEW.connection_id
AND o.state='applied' AND o.verification='inconclusive' AND o.error_code IS NULL
AND isfinite(o.finished_at) AND o.finished_at<transaction_timestamp()-interval '180 days'
AND p.operation_id='servers.probes.run' AND p.operation_version=1 AND p.state='completed'
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'diagnostic retirement claim cannot be replaced or reopened';
END $$;
CREATE TRIGGER server_probe_retirement_claim_guard BEFORE INSERT OR UPDATE OR DELETE
ON public.server_probe_retirement_claims FOR EACH ROW EXECUTE FUNCTION public.server_probe_retirement_claim_guard();
CREATE TRIGGER server_probe_retirement_claim_no_truncate BEFORE TRUNCATE
ON public.server_probe_retirement_claims FOR EACH STATEMENT EXECUTE FUNCTION public.server_probe_retirement_claim_guard();
CREATE FUNCTION public.server_probe_retired_source_insert_guard() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
BEGIN
IF EXISTS(
SELECT 1 FROM public.server_probe_retirement_claims r
JOIN public.mcp_operation_replay_tombstones t ON t.operation_id=r.operation_id AND t.plan_id=r.plan_id
AND t.connection_id=r.connection_id AND t.action_id='servers.probes.run' AND NOT t.target_global
JOIN public.mcp_operations o ON o.id=r.operation_id AND o.plan_id=r.plan_id AND o.connection_id=r.connection_id
JOIN public.server_probe_storage_admissions a ON a.receipt_id=r.receipt_id AND a.execution_id=r.operation_id
WHERE r.receipt_id=NEW.receipt_id AND r.operation_id=NEW.operation_id AND r.source_proof=NEW.proof
AND r.claim_xid<>pg_current_xact_id() AND r.claim_token::text=current_setting('mm.probe_retirement_claim',true)
AND NEW.source->>'planId'=r.plan_id::text AND NEW.source->>'connectionId'=r.connection_id::text
AND NEW.source->>'ownerId'=a.owner_id AND NEW.source->>'bindingProof'=a.binding_proof
AND NEW.source->'serverIds'=to_jsonb(t.target_server_ids)
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'diagnostic archive requires an exact committed retirement claim';
END $$;
CREATE TRIGGER server_probe_retired_source_insert_guard BEFORE INSERT ON public.server_probe_retired_sources
FOR EACH ROW EXECUTE FUNCTION public.server_probe_retired_source_insert_guard();
INSERT INTO public.schema_migrations(id) VALUES ('0051_server_probe_retirement_claims') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,96 @@
BEGIN;
ALTER TABLE public.server_probe_retired_sources
DROP CONSTRAINT server_probe_retired_sources_receipt_id_fkey,
ADD CONSTRAINT server_probe_retired_sources_receipt_id_fkey
FOREIGN KEY(receipt_id) REFERENCES public.server_probe_storage_admissions(receipt_id) ON DELETE CASCADE;
CREATE FUNCTION public.server_probe_retired_gc_allowed(target_receipt uuid,current_retention integer) RETURNS boolean
LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
DECLARE activity timestamptz; retained_days integer; safety integer;
BEGIN
IF current_setting('mm.probe_storage_gc',true) IS DISTINCT FROM 'on'
OR current_setting('mm.probe_retired_gc_receipt',true) IS DISTINCT FROM target_receipt::text THEN RETURN false; END IF;
SELECT (r.source->>'activityAt')::timestamptz,a.retention_days,a.safety_ms
INTO activity,retained_days,safety
FROM public.server_probe_retired_sources r
JOIN public.server_probe_storage_admissions a ON a.receipt_id=r.receipt_id AND a.execution_id=r.operation_id
JOIN public.mcp_operation_replay_tombstones t ON t.operation_id=r.operation_id
JOIN public.mcp_connections c ON c.id=t.connection_id AND c.owner_id=a.owner_id
WHERE r.receipt_id=target_receipt AND r.proof=current_setting('mm.probe_retired_gc_proof',true)
AND a.identity_kind='mcp' AND a.cancelled_at IS NULL AND a.claim_token IS NULL AND isfinite(a.claim_attempted_at)
AND t.action_id='servers.probes.run' AND NOT t.target_global
AND r.source->>'planId'=t.plan_id::text AND r.source->>'connectionId'=c.id::text
AND r.source->'serverIds'=to_jsonb(t.target_server_ids) AND r.source->>'ownerId'=a.owner_id
AND r.source->>'bindingProof'=a.binding_proof
AND (r.source->>'authorityExpiresAt')::timestamptz=a.authority_expires_at
AND (r.source->>'gcNotBefore')::timestamptz=a.gc_not_before
AND (r.source->>'retentionDays')::int=a.retention_days AND (r.source->>'safetyMs')::int=a.safety_ms
AND isfinite(a.authority_expires_at) AND isfinite(a.gc_not_before)
AND a.authority_expires_at<clock_timestamp() AND a.gc_not_before<clock_timestamp()
AND NOT EXISTS(SELECT 1 FROM public.mcp_operations WHERE id=t.operation_id)
AND NOT EXISTS(SELECT 1 FROM public.mcp_plans WHERE id=t.plan_id)
AND NOT EXISTS(SELECT 1 FROM public.mcp_operation_steps WHERE operation_id=t.operation_id)
AND NOT EXISTS(SELECT 1 FROM public.mcp_operation_checkpoints WHERE operation_id=t.operation_id)
AND NOT EXISTS(SELECT 1 FROM public.mcp_diagnostic_cancellations WHERE operation_id=t.operation_id)
AND NOT EXISTS(SELECT 1 FROM public.server_probe_execution_quarantines q
WHERE q.receipt_id=r.receipt_id OR(q.identity_kind='mcp' AND q.execution_id=r.operation_id));
IF NOT FOUND OR activity IS NULL OR NOT isfinite(activity)
OR current_retention IS NULL OR current_retention NOT BETWEEN 1 AND 365 THEN RETURN false; END IF;
IF EXISTS(SELECT 1 FROM public.server_probe_executions j WHERE j.receipt_id=target_receipt
AND (j.finished_at IS NULL OR j.finished_at>activity))
OR EXISTS(SELECT 1 FROM public.server_probe_execution_stages s WHERE s.receipt_id=target_receipt
AND GREATEST(s.created_at,COALESCE(s.received_at,s.created_at))>activity)
THEN RETURN false; END IF;
RETURN activity+make_interval(days=>GREATEST(retained_days,current_retention))
+GREATEST(safety,60000)*interval '1 millisecond'<clock_timestamp();
END $$;
CREATE OR REPLACE FUNCTION public.server_probe_retired_source_immutable() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='DELETE' THEN
IF current_setting('mm.probe_storage_gc',true)='on'
AND current_setting('mm.probe_retired_gc_receipt',true)=OLD.receipt_id::text
AND current_setting('mm.probe_retired_gc_proof',true)=OLD.proof
AND NOT EXISTS(SELECT 1 FROM public.server_probe_storage_admissions WHERE receipt_id=OLD.receipt_id)
AND NOT EXISTS(SELECT 1 FROM public.server_probe_executions WHERE receipt_id=OLD.receipt_id)
AND NOT EXISTS(SELECT 1 FROM public.server_probe_execution_payloads WHERE receipt_id=OLD.receipt_id)
AND NOT EXISTS(SELECT 1 FROM public.server_probe_execution_stages WHERE receipt_id=OLD.receipt_id)
THEN RETURN OLD; END IF;
END IF;
RAISE EXCEPTION 'retired diagnostic source is immutable';
END $$;
CREATE OR REPLACE FUNCTION public.server_probe_storage_gc_allowed(id uuid) RETURNS boolean
LANGUAGE plpgsql SET TimeZone='UTC' AS $$
DECLARE a server_probe_storage_admissions%ROWTYPE; activity timestamptz; current_retention integer; terminal_at timestamptz;
BEGIN
IF current_setting('mm.probe_storage_gc',true) IS DISTINCT FROM 'on' THEN RETURN false; END IF;
PERFORM pg_advisory_xact_lock(hashtextextended('server-probe-storage-v2',0));
SELECT * INTO a FROM server_probe_storage_admissions WHERE receipt_id=id FOR UPDATE;
IF NOT FOUND OR a.gc_not_before>=clock_timestamp() OR a.authority_expires_at>=clock_timestamp() THEN RETURN false; END IF;
SELECT settings.requests_retention_days INTO current_retention FROM mcp_settings settings WHERE settings.id=1;
IF current_retention IS NULL OR current_retention NOT BETWEEN 1 AND 365 THEN RETURN false; END IF;
IF EXISTS(SELECT 1 FROM public.server_probe_retired_sources WHERE receipt_id=id) THEN
RETURN public.server_probe_retired_gc_allowed(id,current_retention);
END IF;
terminal_at:=a.created_at;
IF a.identity_kind='mcp' THEN
SELECT GREATEST(o.finished_at,p.expires_at,COALESCE(cancellation.created_at,o.finished_at)) INTO terminal_at
FROM mcp_operations o JOIN mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN mcp_connections owner_connection ON owner_connection.id=o.connection_id
LEFT JOIN mcp_diagnostic_cancellations cancellation ON cancellation.operation_id=o.id
WHERE o.id=a.execution_id AND owner_connection.owner_id=a.owner_id AND p.operation_id='servers.probes.run'
AND o.state IN ('applied','failed','unknown','rejected','partial','cancelled') AND o.finished_at IS NOT NULL
AND (cancellation.operation_id IS NULL OR(cancellation.connection_id=o.connection_id AND cancellation.actor_id=a.owner_id))
FOR UPDATE OF o,p,owner_connection;
IF NOT FOUND OR terminal_at IS NULL THEN RETURN false; END IF;
END IF;
SELECT GREATEST(a.created_at,a.authority_expires_at,terminal_at,COALESCE(a.cancelled_at,a.created_at),
COALESCE((SELECT finished_at FROM server_probe_executions WHERE receipt_id=id),a.created_at),
COALESCE((SELECT created_at FROM server_probe_execution_quarantines WHERE receipt_id=id),a.created_at),
COALESCE((SELECT max(GREATEST(created_at,COALESCE(received_at,created_at))) FROM server_probe_execution_stages WHERE receipt_id=id),a.created_at)) INTO activity;
RETURN activity+make_interval(days=>GREATEST(a.retention_days,current_retention))+a.safety_ms*interval '1 millisecond'<clock_timestamp();
END $$;
INSERT INTO public.schema_migrations(id) VALUES ('0052_server_probe_retired_gc') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,74 @@
BEGIN;
CREATE TABLE public.server_probe_terminal_proofs (
operation_id uuid PRIMARY KEY REFERENCES public.mcp_operations(id),
receipt_id uuid NOT NULL UNIQUE,
source jsonb NOT NULL CHECK(jsonb_typeof(source)='object' AND octet_length(source::text) BETWEEN 1 AND 8192),
source_proof text NOT NULL CHECK(source_proof ~ '^[a-f0-9]{64}$'),
idempotency_digest text NOT NULL CHECK(idempotency_digest ~ '^[a-f0-9]{64}$'),
capture_proof text NOT NULL CHECK(capture_proof ~ '^[a-f0-9]{64}$'),
CHECK((source->>'version') IS NOT DISTINCT FROM '1'
AND (source->>'executionId') IS NOT DISTINCT FROM operation_id::text
AND (source->>'receiptId') IS NOT DISTINCT FROM receipt_id::text)
);
CREATE FUNCTION public.server_probe_terminal_proof_guard() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='INSERT' THEN
IF EXISTS(
SELECT 1 FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_connections c ON c.id=o.connection_id
JOIN public.server_probe_storage_admissions a ON a.execution_id=o.id AND a.identity_kind='mcp' AND a.owner_id=c.owner_id
JOIN public.server_probe_executions j ON j.receipt_id=a.receipt_id
JOIN public.server_probe_execution_payloads b ON b.receipt_id=a.receipt_id AND b.receipt_proof=j.receipt_proof
WHERE o.id=NEW.operation_id AND a.receipt_id=NEW.receipt_id AND a.purged_at IS NULL AND a.cancelled_at IS NULL
AND a.claim_token IS NULL AND a.claim_attempted_at IS NOT NULL
AND o.state='applied' AND o.verification='inconclusive' AND o.error_code IS NULL
AND p.state='completed' AND p.operation_id='servers.probes.run' AND p.operation_version=1
AND j.state='completed' AND j.execution_id=o.id AND j.mcp_connection_id=c.id AND j.actor_id=c.owner_id
AND NEW.source->>'connectionId'=c.id::text AND NEW.source->>'planId'=p.id::text
AND NEW.source->>'ownerId'=a.owner_id AND NEW.source->>'bindingProof'=a.binding_proof
AND NEW.source->>'receiptProof'=b.receipt_proof
AND NEW.source->>'ciphertextSha256'=encode(sha256(convert_to(b.receipt_ciphertext,'UTF8')),'hex')
) THEN RETURN NEW; END IF;
ELSIF TG_OP='DELETE' THEN
IF EXISTS(
SELECT 1 FROM public.server_probe_retirement_claims r
JOIN public.mcp_operation_replay_tombstones t ON t.operation_id=r.operation_id AND t.plan_id=r.plan_id AND t.connection_id=r.connection_id
WHERE r.operation_id=OLD.operation_id AND r.receipt_id=OLD.receipt_id AND r.source_proof=OLD.source_proof
AND r.claim_xid<>pg_current_xact_id() AND r.claim_token::text=current_setting('mm.probe_retirement_claim',true)
AND t.action_id='servers.probes.run' AND NOT t.target_global AND t.idempotency_digest=OLD.idempotency_digest
AND OLD.source->'serverIds'=to_jsonb(t.target_server_ids)
) THEN RETURN OLD; END IF;
END IF;
RAISE EXCEPTION 'diagnostic terminal proof must match its original source or exact retirement claim';
END $$;
CREATE TRIGGER server_probe_terminal_proof_guard BEFORE INSERT OR UPDATE OR DELETE
ON public.server_probe_terminal_proofs FOR EACH ROW EXECUTE FUNCTION public.server_probe_terminal_proof_guard();
CREATE TRIGGER server_probe_terminal_proof_no_truncate BEFORE TRUNCATE
ON public.server_probe_terminal_proofs FOR EACH STATEMENT EXECUTE FUNCTION public.server_probe_terminal_proof_guard();
CREATE OR REPLACE FUNCTION public.server_probe_retirement_claim_guard() RETURNS trigger
LANGUAGE plpgsql SET search_path=pg_catalog,public SET TimeZone='UTC' AS $$
BEGIN
IF TG_OP='INSERT' AND NEW.claim_xid=pg_current_xact_id() AND EXISTS(
SELECT 1 FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_connections c ON c.id=o.connection_id
WHERE o.id=NEW.operation_id AND p.id=NEW.plan_id AND c.id=NEW.connection_id
AND o.state='applied' AND o.verification='inconclusive' AND o.error_code IS NULL
AND isfinite(o.finished_at) AND o.finished_at<transaction_timestamp()-interval '180 days'
AND p.operation_id='servers.probes.run' AND p.operation_version=1 AND p.state='completed'
AND (EXISTS(SELECT 1 FROM public.server_probe_storage_admissions a
WHERE a.receipt_id=NEW.receipt_id AND a.execution_id=o.id AND a.identity_kind='mcp' AND a.owner_id=c.owner_id
AND a.claim_token IS NULL AND a.claim_attempted_at IS NOT NULL AND a.cancelled_at IS NULL AND a.purged_at IS NULL)
OR(EXISTS(SELECT 1 FROM public.server_probe_terminal_proofs f
WHERE f.operation_id=o.id AND f.receipt_id=NEW.receipt_id AND f.source_proof=NEW.source_proof
AND f.source->>'ownerId'=c.owner_id AND f.source->>'connectionId'=c.id::text AND f.source->>'planId'=p.id::text)
AND NOT EXISTS(SELECT 1 FROM public.server_probe_storage_admissions a
WHERE a.receipt_id=NEW.receipt_id OR(a.identity_kind='mcp' AND a.execution_id=o.id))))
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'diagnostic retirement claim cannot be replaced or reopened';
END $$;
INSERT INTO public.schema_migrations(id) VALUES ('0053_server_probe_terminal_proofs') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,18 @@
BEGIN;
CREATE OR REPLACE FUNCTION public.mcp_ordinary_retirement_claim_immutable() RETURNS trigger
LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='INSERT' AND NEW.claim_xid=pg_current_xact_id() AND EXISTS(
SELECT 1 FROM public.mcp_operations o JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
WHERE o.id=NEW.operation_id AND o.plan_id=NEW.plan_id AND o.connection_id=NEW.connection_id
AND o.state='applied' AND o.verification='passed' AND o.error_code IS NULL
AND isfinite(o.finished_at) AND o.finished_at<transaction_timestamp()-interval '180 days'
AND (p.operation_id,p.operation_version) IN (
('logs.settings.update',1),('traffic.settings.update',2),('uptime.settings.update',2),
('servers.api-ping.settings.update',2),('internet-path.settings.update',1),('geoip.settings.update',1)
) AND p.state='completed'
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'ordinary operation retirement claim cannot be replaced or reopened';
END $$;
INSERT INTO public.schema_migrations(id) VALUES ('0054_mcp_geoip_retirement') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,18 @@
BEGIN;
CREATE OR REPLACE FUNCTION public.mcp_ordinary_retirement_claim_immutable() RETURNS trigger
LANGUAGE plpgsql SET TimeZone='UTC' SET search_path=pg_catalog,public AS $$
BEGIN
IF TG_OP='INSERT' AND NEW.claim_xid=pg_current_xact_id() AND EXISTS(
SELECT 1 FROM public.mcp_operations o JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
WHERE o.id=NEW.operation_id AND o.plan_id=NEW.plan_id AND o.connection_id=NEW.connection_id
AND o.state='applied' AND o.verification='passed' AND o.error_code IS NULL
AND isfinite(o.finished_at) AND o.finished_at<transaction_timestamp()-interval '180 days'
AND (p.operation_id,p.operation_version) IN (
('logs.settings.update',1),('traffic.settings.update',2),('uptime.settings.update',2),
('servers.api-ping.settings.update',2),('internet-path.settings.update',1),('geoip.settings.update',1),('backups.schedule.update',1)
) AND p.state='completed'
) THEN RETURN NEW; END IF;
RAISE EXCEPTION 'ordinary operation retirement claim cannot be replaced or reopened';
END $$;
INSERT INTO public.schema_migrations(id) VALUES ('0055_mcp_backup_schedule_retirement') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,43 @@
-- Keep the live charge and intent fences in sync with recovery action v8.
-- Existing authenticated v6/v7 history remains unchanged.
BEGIN;
CREATE OR REPLACE FUNCTION public.mcp_routing_manual_inverse_charge_source() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF NOT EXISTS(
SELECT 1 FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_routing_profiles r ON r.id=NEW.profile_id AND r.connection_id=o.connection_id
JOIN public.mcp_operations source ON source.id=NEW.source_operation_id AND source.connection_id=o.connection_id
JOIN public.mcp_plans sp ON sp.id=source.plan_id AND sp.connection_id=source.connection_id
JOIN public.mcp_connections c ON c.id=o.connection_id
WHERE o.id=NEW.recovery_operation_id AND p.id=NEW.recovery_plan_id
AND o.connection_id=NEW.connection_id AND c.owner_id=NEW.owner_id
AND r.version=NEW.profile_version AND o.state='pending' AND p.state='executing'
AND p.operation_id='operations.recover' AND p.operation_version=8
AND sp.id=NEW.source_plan_id AND sp.operation_id='routing.community.switch' AND sp.operation_version=7
AND source.state IN('applied','partial')
) THEN RAISE EXCEPTION 'manual inverse charge differs from pending admission'; END IF;
RETURN NEW;
END $$;
CREATE OR REPLACE FUNCTION public.mcp_scoped_recovery_manual_version() RETURNS trigger LANGUAGE plpgsql AS $$
BEGIN
IF NEW.mode='manual' AND NOT pg_try_advisory_xact_lock(hashtextextended('mcp-runner-admission',0))
THEN RAISE EXCEPTION 'manual intent admission snapshot busy'; END IF;
IF NEW.mode='manual' AND NOT EXISTS(
SELECT 1 FROM public.mcp_operations o
JOIN public.mcp_plans p ON p.id=o.plan_id AND p.connection_id=o.connection_id
JOIN public.mcp_operation_steps s ON s.operation_id=o.id AND s.step_index=0
JOIN public.mcp_operations source ON source.id=NEW.source_operation_id AND source.connection_id=o.connection_id
WHERE o.id=NEW.recovery_operation_id AND o.state='running'
AND p.state='executing' AND p.operation_id='operations.recover' AND p.operation_version=8
AND NEW.source_step_index=0 AND s.state='intent'
AND s.intended_state->>'kind'='service'
AND s.intended_state->>'service'='routing.community.scoped.recover'
AND s.intended_state->'input'->>'mode'='manual'
AND s.intended_state->'input'->>'sourceOperationId'=NEW.source_operation_id::text
AND s.intended_state->'input'->>'receiptId'=NEW.recovery_receipt_id::text
) THEN RAISE EXCEPTION 'manual inverse requires current exact running recovery'; END IF;
RETURN NEW;
END $$;
INSERT INTO public.schema_migrations(id) VALUES('0056_mcp_manual_inverse_recovery_v8') ON CONFLICT(id) DO NOTHING;
COMMIT;
@@ -0,0 +1,8 @@
BEGIN;
CREATE INDEX mcp_audit_routing_charge_profile_window
ON public.mcp_audit (connection_id, (payload -> 'anchor' ->> 'profileId'), created_at, id)
WHERE action = 'routing.charge.admitted';
INSERT INTO public.schema_migrations (id) VALUES ('0057_mcp_routing_charge_audit_index') ON CONFLICT (id) DO NOTHING;
COMMIT;
File diff suppressed because it is too large. Load diff
+174 -24
View File
@@ -1,46 +1,191 @@
{
"name": "mikrotik-manager-backend",
"version": "1.0.0",
"description": "MikroTik Manager backend — Fastify + Drizzle + PostgreSQL",
"description": "MikroTik Manager backend — Fastify + Drizzle + PostgreSQL",
"type": "module",
"scripts": {
"test:sidebar-counts": "tsx src/services/complete-count.test.ts && tsx src/routes/sidebar-counts.test.ts",
"test:mcp:module-completion": "tsx src/services/ipsec-server-init-model.test.ts && tsx src/modules/mcp/ipsec-server-init-actions.test.ts && tsx src/services/wireguard-revision-restore.test.ts && tsx src/modules/mcp/vrf-ticket-runtime.test.ts && tsx src/services/ipsec-restore.test.ts && tsx src/services/ipsec-pool-dependencies.test.ts && tsx src/services/vrf-production-transition.test.ts && tsx src/services/vrf-production-action-binding.test.ts && tsx src/services/ipsec-revision-restore-model.test.ts && tsx src/modules/mcp/ipsec-revision-actions.test.ts && tsx src/modules/mcp/service-native-progress.test.ts && tsx src/services/ipsec-revision-restore.test.ts && tsx src/services/ipsec-revision-secret.test.ts",
"test:mcp:module-completion:pg": "node scripts/test-mcp-postgres.mjs --module-completion --restore",
"test:mcp:pg": "node scripts/test-mcp-postgres.mjs",
"test:mcp:users-bindings:pg": "node scripts/test-mcp-postgres.mjs --users-bindings",
"test:mcp:bridge-vlan:pg": "node scripts/test-mcp-postgres.mjs --bridge-vlan-configuration",
"test:mcp:external-process:pg": "node scripts/mcp-external-process/run-owned.mjs",
"test:mcp:bridge-vlan:sdk": "node scripts/mcp-external-process/run-owned.mjs .. bridge-vlan",
"test:traffic-flow:ownership": "tsx src/services/traffic-flow-batch-proof.test.ts && tsx src/services/traffic-flow-owned-batch.test.ts && tsx src/services/traffic-flow-owned-writer.test.ts && tsx src/services/traffic-flow-write-transaction.test.ts && tsx src/services/traffic-flow-writer-exclusion.test.ts && tsx src/services/traffic-flow-drain-protocol.test.ts && tsx src/services/traffic-flow-facts-rebuild-execution.test.ts && tsx src/services/traffic-flow-facts-rebuild-source.test.ts && tsx src/services/traffic-flow-facts-rebuild-classifier.test.ts && tsx src/services/traffic-flow-facts-rebuild-service.test.ts && tsx src/services/traffic-flow-ripe-prune-owned.test.ts && tsx src/services/traffic-flow-engine-owned.test.ts && tsx src/routes/traffic-flow-rebuild.test.ts && tsx src/services/traffic-flow-settings-execution.test.ts",
"test:traffic-flow:rebuild:pg": "node scripts/test-mcp-postgres.mjs --flow-rebuild",
"test:traffic-flow:pg": "node scripts/test-mcp-postgres.mjs --flow-writes",
"test:traffic-flow:quality:pg": "node scripts/test-mcp-postgres.mjs --flow-quality",
"test:traffic-flow:chr-evidence": "tsx --test src/test/traffic-flow-chr-capture-check.test.ts src/test/traffic-flow-chr-wire.test.ts src/test/traffic-flow-chr-target-harness.test.ts src/test/traffic-flow-chr-owned-sequence.test.ts src/test/traffic-flow-chr-reconcile.test.ts",
"test:changed": "node scripts/test-changed.mjs",
"test:mcp:pg:all": "node scripts/test-mcp-postgres.mjs --all",
"mcp:stdio": "node dist/scripts/mcp-stdio.js",
"mcp:stdio:dev": "tsx src/scripts/mcp-stdio.ts",
"test:logs": "tsx src/services/router-logs-export.test.ts && tsx src/services/router-logs-model.test.ts && tsx src/plugins/auth.logs.test.ts && tsx src/services/mikrotik-logs.test.ts",
"test:logs:pg": "tsx src/services/router-logs-sql.test.ts",
"test:sidebar-counts": "tsx src/services/sidebar-counts.test.ts && tsx src/services/complete-count.test.ts && tsx src/routes/sidebar-counts.test.ts",
"dev": "tsx watch src/index.ts",
"build": "npm run build -w @mmapp/contracts && tsc",
"build": "npm run build -w @mmapp/contracts && tsc && npm run build:database-operator",
"build:database-operator": "node scripts/build-database-operator.mjs",
"test:alerts-telegram": "tsx src/services/alerts-outbox-retry.test.ts && tsx src/services/alerts-outbox-overview.test.ts && tsx src/services/alert-engine/outbox-execution.test.ts && tsx src/services/alerts-telegram-delivery.test.ts && tsx src/services/alerts-revision.test.ts && tsx src/services/alerts-overview.test.ts && tsx ../packages/contracts/src/alerts-normalization.test.ts && tsx src/services/alerts-config-execution.test.ts && tsx src/services/alerts-telegram-settings.test.ts && tsx src/services/alerts-telegram-send.test.ts && tsx src/routes/alerts.test.ts && tsx src/plugins/auth.alerts.test.ts",
"test:alert-engine": "tsx src/services/alert-engine/server-signal-identity.test.ts && tsx src/services/alert-engine/collector-source-signals.test.ts && tsx src/services/alert-engine/monitoring-history.test.ts && tsx src/services/alert-engine/groups.test.ts && tsx src/services/alert-engine/evaluate-rule.test.ts && tsx src/services/gre-tunnels-live.test.ts && tsx src/services/bgp-peers-live.test.ts && tsx src/services/gre-bgp-snapshot-collector.test.ts && tsx src/services/gre-bgp-snapshot-history.test.ts && tsx src/services/gre-bgp-snapshot-sql.test.ts && tsx src/services/alert-engine/collector-source-sql.test.ts && tsx src/services/alert-engine/confirm-stability.test.ts",
"test:alert-engine": "tsx src/services/alert-engine/execution-access.test.ts && tsx src/services/alert-engine/server-signal-identity.test.ts && tsx src/services/alert-engine/collector-source-signals.test.ts && tsx src/services/alert-engine/monitoring-history.test.ts && tsx src/services/alert-engine/groups.test.ts && tsx src/services/alert-engine/evaluate-rule.test.ts && tsx src/services/gre-tunnels-live.test.ts && tsx src/services/bgp-peers-live.test.ts && tsx src/services/gre-bgp-snapshot-collector.test.ts && tsx src/services/gre-bgp-snapshot-history.test.ts && tsx src/services/gre-bgp-snapshot-sql.test.ts && tsx src/services/alert-engine/collector-source-sql.test.ts && tsx src/services/alert-engine/confirm-stability.test.ts",
"start": "node dist/index.js",
"db:generate": "drizzle-kit generate",
"db:migrate": "drizzle-kit migrate",
"db:prepare": "tsx src/scripts/prepare-database.ts",
"db:complete-restore": "tsx src/scripts/complete-database-restore.ts",
"db:serve-restore-claims": "tsx src/scripts/serve-restore-checkpoint-claims.ts",
"db:seal-restore-checkpoint": "tsx src/scripts/seal-database-restore-checkpoint.ts",
"db:stage-restore-checkpoint": "tsx src/scripts/stage-database-restore-checkpoint.ts",
"db:inspect-restore-checkpoint": "tsx src/scripts/inspect-database-restore-checkpoint.ts",
"db:inspect-schema": "tsx src/scripts/inspect-database-schema.ts",
"db:studio": "drizzle-kit studio",
"db:migrate-from-sqlite": "tsx src/scripts/migrate-sqlite-to-pg.ts",
"facts:rebuild": "tsx src/scripts/rebuild-flow-facts.ts",
"test:probes": "tsx src/routes/probes.test.ts && tsx src/routes/probes.cancel.test.ts && tsx src/services/ping-measurement.test.ts && tsx src/services/uptime-probe-observations.test.ts && tsx src/services/ping-measurement-sql.test.ts",
"test:exec": "tsx src/services/mikrotik-logs.test.ts && tsx src/routes/exec.test.ts && tsx src/plugins/auth.exec.test.ts",
"test:auth": "tsx src/plugins/auth-path.test.ts && tsx src/lib/permissions.test.ts && tsx src/plugins/auth.smoke.test.ts && tsx src/plugins/auth.ospf-optimizer.test.ts",
"test:wireguard": "npx tsx src/services/wireguard-config.test.ts && npx tsx src/services/wireguard-ipv6.test.ts && npx tsx --test src/services/wireguard-import.test.ts src/services/wireguard-import-apply.test.ts src/services/wireguard-import-revision.test.ts src/services/wireguard-delete.test.ts && npx tsx src/routes/wireguard-import.test.ts",
"test:mcp:vrf-cleanup": "tsx src/test/chr-vrf-companion-cleanup-proof.test.ts && tsx src/test/chr-vrf-companion-disabled-cleanup-proof.test.ts && tsx src/test/chr-vrf-companion-transfer-cleanup-proof.test.ts",
"test:probes": "npm run test:speed-workflow && tsx src/services/manual-data-collection-execution.test.ts && tsx src/routes/manual-data-collection.test.ts && tsx src/services/network-host-resolution.test.ts && tsx src/routes/probes.test.ts && tsx src/routes/probes.cancel.test.ts && tsx src/services/ping-measurement.test.ts && tsx src/services/uptime-probe-observations.test.ts && tsx src/services/ping-measurement-sql.test.ts",
"test:exec": "tsx src/services/mikrotik-logs.test.ts && tsx src/services/terminal-diagnostics.test.ts && tsx src/routes/exec.test.ts && tsx src/plugins/auth.exec.test.ts",
"test:auth": "tsx --test src/modules/mcp/routing-manual-inverse-history.test.ts && tsx src/db/application-writer-profile-registry-structure.test.ts && tsx src/db/application-writer-profile-registry-read.test.ts && tsx src/db/application-writer-profile-registry-read-parity.test.ts && tsx src/db/application-writer-profile-format.test.ts && tsx src/plugins/auth-path.test.ts && tsx src/lib/permissions.test.ts && tsx src/plugins/auth-provenance.test.ts && tsx src/services/bandwidth-rest-speed-original.test.ts && tsx src/plugins/auth.smoke.test.ts && tsx src/plugins/auth.ospf-optimizer.test.ts && npm run test:mcp",
"test:mcp": "node scripts/vrf-production-profile.test.mjs && node src/services/vrf-production-peer-telemetry.test.mjs && node src/services/vrf-execution-diagnostic.test.mjs && node src/services/vrf-transition-refusal.test.mjs && node src/services/vrf-production-gre-cycle.test.mjs && tsx src/services/vrf-production-gre-link-local.test.ts && tsx src/services/vrf-production-bridge-delete.test.ts && tsx src/services/vrf-production-transition.test.ts && tsx --test src/services/routeros-snapshot-read.test.ts src/services/bridge-vlan-configuration.test.ts src/modules/mcp/bridge-vlan-projection.test.ts src/modules/mcp/vrf-preparation-diagnostic.test.ts && tsx --test src/services/server-poll-error-provenance.test.ts && tsx --test src/modules/mcp/plan-expiry.test.ts src/modules/mcp/result-budget.test.ts src/modules/mcp/result-delivery.test.ts src/modules/mcp/wire-admission.test.ts src/modules/mcp/flow-envelope-allowance.test.ts src/modules/mcp/sdk-terminal-budget.test.ts src/modules/mcp/inflight-owner.test.ts && npm run test:database-source-binding && tsx --test src/modules/mcp/integration-fixture-expiry.test.ts && npm run test:mcp:operation-origin && npm run test:mcp:service-context && npm run test:mcp:operation-locks && tsx --test src/modules/mcp/routing-community-public.test.ts src/modules/mcp/management-transport.test.ts src/modules/mcp/routing-community-context-model.test.ts src/modules/mcp/routing-community-context-native.test.ts ../packages/contracts/src/mcp-routing.test.ts && tsx src/modules/mcp/human-artifact-receipts.test.ts && tsx src/modules/mcp/chr-overlay-readiness.test.ts && tsx src/modules/mcp/chr-recovery-source.test.ts && tsx src/modules/mcp/chr-native-failure.test.ts && tsx src/modules/mcp/ospf-owner-graph.test.ts && tsx src/modules/mcp/routing-measurement-address.test.ts && tsx src/modules/mcp/routing-observation-source.test.ts && tsx src/services/traffic-flow-talkers-model.test.ts && tsx src/services/bgp-prefix-model.test.ts && tsx src/services/vrf-members.test.ts && tsx src/services/routing-gateway-reference.test.ts && tsx src/services/bgp-export-model.test.ts && tsx src/services/traffic-history-model.test.ts && tsx src/services/internet-path-snapshot.test.ts && tsx src/services/internet-path-observation.test.ts && tsx src/modules/mcp/coverage.test.ts && tsx src/modules/mcp/policy.test.ts && tsx src/modules/mcp/events-batch-model.test.ts && tsx src/modules/mcp/operation-revisions.test.ts && tsx src/modules/mcp/native-create-identity.test.ts && tsx src/modules/mcp/ros-step-verification.test.ts && tsx src/modules/mcp/ros-prefix.test.ts && tsx src/modules/mcp/native-conservation-http.test.ts && tsx src/modules/mcp/security.test.ts && tsx src/modules/mcp/state-file-publish.test.ts && tsx src/modules/mcp/routing-model.test.ts && tsx src/modules/mcp/routing-observations-model.test.ts && tsx src/modules/mcp/ospf-health.test.ts && tsx src/modules/mcp/ecmp-preflight.test.ts && tsx src/modules/mcp/recursive-recovery.test.ts && tsx src/modules/mcp/control-path-model.test.ts && tsx src/modules/mcp/routing-community-scoped-model.test.ts && tsx src/modules/mcp/routing-community-scoped-native.test.ts && tsx src/modules/mcp/path-verification.test.ts && tsx src/modules/mcp/database-maintenance.test.ts && tsx src/routes/mcp.test.ts && tsx ../lib/auth-mcp.test.ts && tsx --test src/modules/mcp/routing-controller-fence.test.ts && tsx src/test/chr-vrf-companion-proof.test.ts && tsx src/test/chr-vrf-companion-telemetry.test.ts && tsx src/test/chr-vrf-companion-address-space.test.ts && tsx src/modules/mcp/flow-facts-rebuild-actions.test.ts && tsx src/modules/mcp/ipsec-peer-create-dependencies.test.ts && tsx src/services/ipsec-certificate-update-model.test.ts && tsx src/modules/mcp/ipsec-native-dispatch.test.ts && tsx src/modules/mcp/ipsec-user-update-actions.test.ts && tsx src/modules/mcp/ipsec-server-delete-actions.test.ts && tsx src/services/ipsec-proof-telemetry.test.ts && tsx src/services/vrf-production-admission.test.ts && tsx src/modules/mcp/flow-host-keys-actions.test.ts && tsx src/services/vrf-production-read.test.ts && tsx --test src/services/vrf-production-wireguard-alias.test.ts && tsx src/modules/mcp/ipsec-user-create-actions.test.ts && tsx src/modules/mcp/flow-settings-actions.test.ts && tsx src/services/vrf-production-ticket-store.test.ts && npm run test:mcp:routing-baseline",
"test:wireguard": "tsx src/services/mikrotik-response-bounds.test.ts && tsx --test src/services/wireguard-dependency-read.test.ts && tsx src/services/wireguard-interface-update.test.ts && tsx src/services/wireguard-peer-update.test.ts && npx tsx src/services/wireguard-config.test.ts && npx tsx src/services/wireguard-ipv6.test.ts && npx tsx --test src/services/wireguard-import.test.ts src/services/wireguard-import-apply.test.ts src/services/wireguard-import-revision.test.ts src/services/wireguard-delete.test.ts && npx tsx src/routes/wireguard-import.test.ts && tsx src/services/wireguard-interface-create.test.ts && tsx src/services/wireguard-peer-create.test.ts && tsx src/services/wireguard-import-plan.test.ts && tsx src/services/wireguard-import-execution.test.ts && tsx src/modules/mcp/wireguard-import-actions.test.ts && tsx --test src/services/wireguard-export.test.ts",
"test:traffic-rate": "tsx src/services/traffic-rate.test.ts && tsx src/services/traffic-observation.test.ts && tsx src/services/traffic-freshness.test.ts && tsx src/services/traffic-live.test.ts && tsx src/services/traffic-observation-sql.test.ts && tsx src/services/traffic-history-sql.test.ts && tsx src/services/traffic-live-sql.test.ts && tsx src/services/traffic-volume-sql.test.ts && tsx src/services/traffic-binding-status.test.ts && tsx src/services/traffic-status-sql.test.ts && tsx src/services/traffic-gap.test.ts && tsx src/services/traffic-identity.test.ts && tsx src/services/traffic-membership.test.ts && tsx src/services/traffic-aggregate-peak.test.ts && tsx src/services/traffic-users-cache.test.ts && tsx src/services/traffic-collection-lock.test.ts && tsx src/services/traffic-concurrency-sql.test.ts",
"test:traffic-flow": "tsx src/routes/traffic-flow-query.test.ts && tsx src/services/traffic-flow-month.test.ts && tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-ipv6-pipeline.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-dest.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/geoip-download.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/traffic-flow-facts-rebuild.test.ts && tsx src/services/statistics-aggregate.test.ts",
"test:users": "tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/bindings.test.ts && tsx src/modules/users/interface-inventory.test.ts && tsx src/routes/users-catalog.test.ts",
"test:traffic-flow": "tsx --test src/services/traffic-flow-heartbeat.test.ts src/services/traffic-flow-heartbeat-wiring.test.ts src/services/traffic-flow-heartbeat-receiver.test.ts src/services/traffic-flow-ingest-quality.test.ts src/services/traffic-flow-quality-contract.test.ts && tsx src/services/traffic-flow-catalog-read.test.ts && tsx src/routes/traffic-flow-query.test.ts && tsx src/services/traffic-flow-month.test.ts && tsx src/services/traffic-flow-parse.test.ts && tsx src/services/traffic-flow-ipv6-pipeline.test.ts && tsx src/services/traffic-flow-map-exporter.test.ts && tsx src/services/traffic-flow-ifaces.test.ts && tsx src/services/traffic-flow-ifindex.test.ts && tsx src/services/traffic-flow-dedup.test.ts && tsx src/services/traffic-flow-planes.test.ts && tsx src/services/traffic-flow-ip.test.ts && tsx src/services/traffic-flow-dest.test.ts && tsx src/services/traffic-flow-classify.test.ts && tsx src/services/traffic-flow-ripe.test.ts && tsx src/services/traffic-flow-brands.test.ts && tsx src/services/traffic-flow-ingest.test.ts && tsx src/services/traffic-flow-analytics.test.ts && tsx src/services/traffic-flow-map-hops.test.ts && tsx src/services/traffic-flow-purge.test.ts && tsx src/services/geoip-download.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/services/traffic-flow-fact-accumulator.test.ts && tsx src/services/traffic-flow-facts.test.ts && tsx src/services/traffic-flow-facts-filter.test.ts && tsx src/services/traffic-flow-facts-rebuild.test.ts && tsx src/services/statistics-aggregate.test.ts && tsx src/services/geoip-manual-update.test.ts && tsx src/services/traffic-flow-overlay-model.test.ts && npm run test:traffic-flow:ownership && tsx src/services/traffic-flow-worker-platform.test.ts && tsx src/services/traffic-flow-template-cache.test.ts && tsx --test src/services/traffic-flow-template-lifetime.test.ts && tsx src/services/traffic-flow-protocol-scope.test.ts && npm run test:traffic-flow:read-budget && npm run test:traffic-flow:resource-profile",
"test:users": "tsx ../packages/contracts/src/users.test.ts && tsx src/modules/users/iface-type.test.ts && tsx src/modules/users/binding-target.test.ts && tsx src/modules/users/bindings.test.ts && tsx src/modules/users/interface-inventory.test.ts && tsx src/routes/users-catalog.test.ts",
"test:pg": "tsx src/db/pg-pool.test.ts && tsx src/db/sql-bind.test.ts && tsx src/db/sqlite-json.test.ts && tsx src/db/traffic-flags.test.ts && tsx src/db/pg-schema.test.ts && tsx src/db/scheduler-retention-index.test.ts && tsx src/services/config-revisions.test.ts",
"test:recursive-routes": "tsx src/routes/recursive-routes.test.ts && tsx src/services/recursive-route-catalog.test.ts && tsx src/services/recursive-effective-properties.test.ts && tsx src/services/recursive-apply.test.ts",
"test:config-sync": "tsx src/utils/server-id.test.ts && tsx src/services/config-apply-plan.test.ts && tsx src/services/entity-snapshots.test.ts && tsx src/services/gre-native-identity.test.ts && tsx src/services/gre-ipv6.test.ts && tsx src/routes/gre-ipv6-validation.test.ts && tsx src/services/network-disabled-server.test.ts",
"test:ros-file": "tsx src/services/ros-file-contents.test.ts && tsx src/services/mikrotik-response.test.ts",
"test:recursive-routes": "tsx src/services/recursive-route-identity.test.ts && tsx src/routes/recursive-routes.test.ts && tsx src/services/recursive-route-catalog.test.ts && tsx src/services/recursive-effective-properties.test.ts && tsx src/services/recursive-apply.test.ts",
"test:config-sync": "tsx src/services/owned-routing-policy.test.ts && tsx src/services/routing-filter-body.test.ts && tsx src/services/bgp-community-policy-model.test.ts && tsx src/services/bgp-connection-read.test.ts && tsx src/services/bgp-filter-binding-model.test.ts && tsx src/services/filter-rules-write-model.test.ts && tsx src/services/filter-rules-mutation.test.ts && tsx src/utils/server-id.test.ts && tsx src/services/config-apply-plan.test.ts && tsx src/services/entity-snapshots.test.ts && tsx src/services/gre-native-identity.test.ts && tsx src/services/gre-ipv6.test.ts && tsx src/routes/gre-ipv6-validation.test.ts && tsx src/services/network-disabled-server.test.ts",
"test:ros-file": "tsx src/services/mikrotik-get-cancel.test.ts && tsx src/services/ros-file-contents.test.ts && tsx src/services/mikrotik-response.test.ts && tsx src/services/mikrotik-owned-file.test.ts && tsx --test src/services/mikrotik-owned-file-zeroization.test.ts",
"test:scheduler": "tsx src/services/scheduler-running.test.ts && tsx src/services/scheduler-recorded-run.test.ts && tsx src/routes/scheduler.test.ts",
"test:backups": "tsx src/services/backup-local-file-state.test.ts && tsx src/services/router-config-export.test.ts && tsx src/services/backup-scheduler-run.test.ts && tsx src/services/backup-restore.test.ts && tsx src/services/backup-deletion.test.ts && tsx src/services/backup-dto.test.ts && tsx src/services/backup-storage-update.test.ts && tsx src/services/backup-job-execution.test.ts && tsx src/routes/backups-operations.test.ts && tsx src/services/s3-backup-client.test.ts",
"test:ipsec": "tsx src/services/ipsec-config.test.ts && tsx src/services/ipsec-state.test.ts && tsx src/routes/ipsec-history.test.ts && tsx src/services/ipsec-restore.test.ts && tsx src/services/ipsec-operation.test.ts && tsx src/routes/ipsec-mutation.test.ts && tsx src/routes/ipsec-nat.test.ts && tsx src/routes/ipsec-group.test.ts",
"test:live-maps": "tsx src/services/gre-keepalive.test.ts && tsx src/services/gre-read.test.ts && tsx src/services/ospf-route-parse.test.ts && tsx src/services/vxlan-live.test.ts && tsx src/routes/vxlan.test.ts && tsx src/services/containers-live.test.ts && tsx src/services/firewall-poll.test.ts && tsx src/services/gre-catalog-poll.test.ts && tsx src/routes/filters-gre-catalog.test.ts && tsx src/routes/bgp-catalog.test.ts && tsx src/services/bgp-catalog-parse.test.ts && tsx src/routes/ospf-catalog.test.ts && tsx src/services/ospf-observed-values.test.ts && tsx src/routes/ospf-legacy.test.ts",
"test:backups": "tsx src/modules/mcp/backup-storage-actions.test.ts && tsx src/services/backup-job-read.test.ts && tsx src/services/backup-local-file-state.test.ts && tsx src/services/router-config-export.test.ts && tsx src/services/backup-scheduler-run.test.ts && tsx src/services/backup-restore.test.ts && tsx src/services/backup-deletion.test.ts && tsx src/services/backup-dto.test.ts && tsx src/services/backup-storage-update.test.ts && tsx src/services/backup-job-execution.test.ts && tsx src/routes/backups-operations.test.ts && tsx src/services/s3-backup-client.test.ts && tsx src/services/backup-content-bounds.test.ts",
"test:ipsec": "tsx src/services/ipsec-config.test.ts && tsx src/services/ipsec-state.test.ts && tsx src/routes/ipsec-history.test.ts && tsx src/services/ipsec-restore.test.ts && tsx src/services/ipsec-operation.test.ts && tsx src/routes/ipsec-mutation.test.ts && tsx src/routes/ipsec-nat.test.ts && tsx src/routes/ipsec-group.test.ts && tsx src/services/ipsec-peer-delete.test.ts && tsx src/modules/mcp/ipsec-peer-delete-actions.test.ts && tsx src/services/ipsec-user-delete.test.ts && tsx src/modules/mcp/ipsec-user-delete-actions.test.ts && tsx src/services/ipsec-certificate-delete.test.ts && tsx src/modules/mcp/ipsec-certificate-delete-actions.test.ts && tsx src/services/ipsec-peer-update.test.ts && tsx src/modules/mcp/ipsec-peer-update-actions.test.ts && tsx src/services/ipsec-user-update-model.test.ts && tsx src/modules/mcp/ipsec-user-update-actions.test.ts && tsx src/services/ipsec-server-delete-model.test.ts && tsx src/modules/mcp/ipsec-server-delete-actions.test.ts && tsx src/services/ipsec-proof-telemetry.test.ts && tsx src/services/ipsec-user-create-model.test.ts",
"test:live-maps": "tsx src/services/gre-revision-restore.test.ts && tsx src/services/gre-native-mutation.test.ts && tsx src/services/gre-mutation.test.ts && tsx src/services/filter-rules-observed.test.ts && tsx src/services/firewall-revision-restore.test.ts && tsx src/services/firewall-address-write.test.ts && tsx src/services/firewall-rule-write.test.ts && tsx src/services/gre-observed-read.test.ts && tsx src/services/network-observed-read.test.ts && tsx src/services/firewall-rule-move.test.ts && tsx src/services/ospf-state-read.test.ts && tsx src/modules/mcp/readiness-http.test.ts && tsx src/services/probe-interface-model.test.ts && tsx src/services/wan-runtime-model.test.ts && tsx src/services/gre-keepalive.test.ts && tsx src/services/gre-read.test.ts && tsx src/services/ospf-route-parse.test.ts && tsx src/services/vxlan-live.test.ts && tsx src/routes/vxlan.test.ts && tsx src/services/containers-live.test.ts && tsx src/services/firewall-poll.test.ts && tsx src/services/gre-catalog-poll.test.ts && tsx src/routes/filters-gre-catalog.test.ts && tsx src/routes/bgp-catalog.test.ts && tsx src/services/bgp-catalog-parse.test.ts && tsx src/routes/ospf-catalog.test.ts && tsx src/services/ospf-observed-values.test.ts && tsx src/routes/ospf-legacy.test.ts",
"test": "node scripts/test.mjs",
"test:evolution": "tsx src/services/evobgp-catalog.test.ts && tsx src/routes/evobgp-catalog.test.ts && tsx src/routes/evobgp-settings.test.ts && tsx src/plugins/auth.evolution-settings.test.ts",
"test:netflow-purge": "tsx src/db/operation-lock.test.ts && tsx src/services/traffic-flow-purge-execution.test.ts && tsx src/routes/traffic-flow-purge.test.ts",
"test:netflow-settings": "tsx src/services/traffic-flow-host-file-builders.test.ts && tsx src/services/traffic-flow-key-execution.test.ts && tsx src/routes/traffic-flow-settings.test.ts && tsx src/plugins/auth.netflow-settings.test.ts && tsx src/services/traffic-flow-agent-install.test.ts && tsx src/services/traffic-flow-agent-client.test.ts && tsx src/services/traffic-flow-operation-queue.test.ts && tsx src/routes/traffic-flow-host.test.ts",
"test:evolution": "tsx src/services/evobgp-catalog.test.ts && tsx src/services/evobgp-client.test.ts && tsx src/routes/evobgp-catalog.test.ts && tsx src/routes/evobgp-settings.test.ts && tsx src/plugins/auth.evolution-settings.test.ts",
"test:netflow-purge": "tsx src/db/operation-lock.test.ts && tsx src/services/traffic-flow-purge-execution.test.ts && tsx src/services/flow-purge-partitions.test.ts && tsx --test src/services/traffic-flow-worker-lifecycle.test.ts && tsx src/routes/traffic-flow-purge.test.ts",
"test:netflow-settings": "tsx src/services/traffic-flow-host-file-builders.test.ts && tsx src/services/traffic-flow-key-execution.test.ts && tsx src/routes/traffic-flow-settings.test.ts && tsx src/plugins/auth.netflow-settings.test.ts && tsx src/services/traffic-flow-agent-install.test.ts && tsx src/services/traffic-flow-agent-client.test.ts && tsx src/services/traffic-flow-operation-queue.test.ts && tsx src/routes/traffic-flow-host.test.ts && tsx src/services/traffic-flow-settings-execution.test.ts && tsx src/modules/mcp/flow-host-keys-actions.test.ts && tsx src/services/traffic-flow-lifecycle-fence.test.ts && tsx src/services/traffic-flow-settings-store.test.ts && tsx src/services/traffic-flow-settings-service.test.ts && tsx src/modules/mcp/flow-settings-actions.test.ts",
"test:geoip": "tsx src/services/geoip-download.test.ts && tsx src/services/traffic-flow-geoip.test.ts && tsx src/plugins/auth.geoip.test.ts",
"test:ospf-optimizer": "tsx src/services/ospf-optimizer-identity.test.ts && tsx src/services/ospf-optimizer-metrics.test.ts && tsx src/services/ospf-optimizer-plan.test.ts && tsx src/services/ospf-effective-disabled.test.ts && tsx src/routes/ospf-optimizer.test.ts",
"test:ospf-optimizer": "tsx src/services/ospf-cost-optimization.test.ts && tsx src/modules/mcp/ospf-legacy-source.test.ts && tsx src/services/ospf-legacy-optimization.test.ts && tsx src/services/ospf-template-flags.test.ts && tsx src/services/ospf-route-proof.test.ts && tsx src/services/ospf-link-catalog.test.ts && tsx src/services/ospf-identity-catalog.test.ts && tsx src/services/ospf-template-cost-write.test.ts && tsx src/services/ospf-template-cost-http.test.ts && tsx src/services/ospf-optimizer-identity.test.ts && tsx src/services/ospf-optimizer-metrics.test.ts && tsx src/services/ospf-optimizer-plan.test.ts && tsx src/services/ospf-effective-disabled.test.ts && tsx src/routes/ospf-optimizer.test.ts",
"test:containers": "tsx src/services/containers-live.test.ts && tsx src/services/containers-state.test.ts && tsx src/services/container-removal.test.ts && tsx src/routes/containers-state.test.ts",
"test:certificates": "tsx src/services/certificate-installation.test.ts && tsx src/services/certificate-renew-collector.test.ts && tsx src/services/certificate-issue-execution.test.ts && tsx src/services/certificate-issue-request.test.ts && tsx src/routes/certificates-issue.test.ts && tsx src/services/certificate-issue-dto.test.ts && tsx src/services/certificate-parse.test.ts && tsx src/services/certificates-state.test.ts && tsx src/routes/certificates-state.test.ts",
"test:system-database": "tsx src/services/system-database-partition-restore.test.ts && tsx src/services/system-database-cache.test.ts && tsx src/services/database-background-work.test.ts && tsx src/services/database-request-gate.test.ts && tsx src/services/traffic-flow-worker-shutdown.test.ts && tsx src/services/system-database-restore-execution.test.ts && tsx src/services/system-database-operation.test.ts && tsx src/routes/system-database.test.ts && tsx src/plugins/auth.system-database.test.ts",
"test:all": "npm run test:probes && npm run test:exec && npm run test:alert-engine && npm run test:alerts-telegram && npm run test:auth && npm run test:wireguard && npm run test:ipsec && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:backups && npm run test:scheduler && npm run test:live-maps && npm run test:ospf-optimizer && npm run test:recursive-routes && npm run test:config-sync && npm run test:ros-file && npm run test:containers && npm run test:certificates && npm run test:netflow-settings && npm run test:netflow-purge && npm run test:evolution && npm run test:system-database && npm run test:server-privacy",
"test:server-privacy": "tsx src/routes/server-secrets-sql.test.ts"
"test:certificates": "tsx src/services/certificate-installation.test.ts && tsx src/services/certificate-renew-collector.test.ts && tsx src/services/certificate-issue-execution.test.ts && tsx src/services/certificate-issue-request.test.ts && tsx src/routes/certificates-issue.test.ts && tsx src/services/certificate-issue-dto.test.ts && tsx src/services/certificate-parse.test.ts && tsx src/services/certificates-state.test.ts && tsx src/routes/certificates-state.test.ts && tsx src/services/acme-settings-test.test.ts && tsx src/services/acme-settings-update-model.test.ts && tsx src/modules/mcp/acme-settings-actions.test.ts",
"test:system-database": "tsx src/services/bounded-database-dump.test.ts && tsx src/services/database-dump-connection.test.ts && tsx src/services/system-database-partition-restore.test.ts && tsx src/services/system-database-cache.test.ts && tsx src/services/database-background-work.test.ts && tsx src/services/database-request-gate.test.ts && tsx --test src/services/detached-database-admission.test.ts && tsx src/services/traffic-flow-worker-shutdown.test.ts && tsx src/services/system-database-restore-execution.test.ts && tsx src/services/system-database-operation.test.ts && tsx src/services/system-database-schema-readiness.test.ts && tsx --test src/routes/mcp-oauth-database-admission.test.ts && tsx src/routes/system-database.test.ts && tsx src/plugins/auth.system-database.test.ts && tsx src/services/database-control-archive-boundary.test.ts && tsx src/db/database-schema-control-model.test.ts && tsx src/db/database-schema-control.test.ts && tsx src/db/controlled-migrations.test.ts && tsx src/db/controlled-schema-verification.test.ts && tsx --test src/db/migration-data-inventory.test.ts && tsx --test src/services/database-maintenance-startup.test.ts && tsx --test src/services/database-restore-audit.test.ts && tsx --test src/services/database-maintenance-resolution.test.ts && tsx --test scripts/database-maintenance-resolution.test.mts",
"test:all": "npm run test:logs && npm run test:probes && npm run test:exec && npm run test:alert-engine && npm run test:alerts-telegram && npm run test:auth && npm run test:wireguard && npm run test:ipsec && npm run test:traffic-rate && npm run test:traffic-flow && npm run test:users && npm run test:pg && npm run test:sidebar-counts && npm run test:backups && npm run test:scheduler && npm run test:live-maps && npm run test:ospf-optimizer && npm run test:recursive-routes && npm run test:config-sync && npm run test:ros-file && npm run test:containers && npm run test:certificates && npm run test:netflow-settings && npm run test:netflow-purge && npm run test:evolution && npm run test:system-database && npm run test:server-privacy && npm run test:mcp:module-completion && npm run test:mcp:loopback-agent && npm run test:server-probe-management && npm run test:server-poll-management && npm run test:mcp:routing-authority && npm run test:mcp:native-readers && npm run test:mcp:chr-acquisition && npm run test:mcp:shared-context && npm run test:mcp:telemetry-configuration",
"test:server-privacy": "tsx ../packages/contracts/src/servers.test.ts && tsx src/routes/server-secrets-sql.test.ts && tsx --test src/services/server-connection-test.test.ts",
"test:mcp:flow-rebuild:pg": "node scripts/test-mcp-postgres.mjs --flow-rebuild-mcp",
"test:mcp:flow-host-keys:pg": "node scripts/test-mcp-postgres.mjs --flow-host-keys",
"test:mcp:loopback-agent": "tsx src/test/loopback-http-agent.test.ts",
"test:mcp:pending-foundations": "tsx src/services/server-poll-execution.test.ts && tsx src/services/server-probe-execution.test.ts && tsx src/services/ipsec-existing-certificate-export.test.ts && tsx src/modules/mcp/ipsec-existing-certificate-export-actions.test.ts && tsx src/modules/mcp/ipsec-existing-certificate-export-production.test.ts && tsx src/services/ipsec-existing-certificate-export-rest.test.ts && tsx src/services/ipsec-existing-certificate-export-rest-production.test.ts && tsx src/services/ipsec-existing-certificate-export-rest-journal.test.ts && tsx src/services/ipsec-existing-certificate-export-rest-http.test.ts && tsx src/services/ipsec-existing-certificate-export-artifact.test.ts && tsx src/modules/mcp/ipsec-existing-certificate-export-registration.test.ts",
"test:mcp:ipsec-create:pg": "node scripts/test-mcp-postgres.mjs --module-completion --only=ipsec-create --restore",
"test:mcp:ipsec-export-journal:pg": "node scripts/test-mcp-postgres.mjs --ipsec-export-journal --restore",
"test:mcp:ipsec-export:http:pg": "node scripts/test-mcp-postgres.mjs --ipsec-export-http --restore",
"test:mcp:diagnostic-cancel-host:pg": "node scripts/test-mcp-postgres.mjs --diagnostic-cancel-host --restore",
"test:server-probe-management": "tsx src/services/server-probe-authority.test.ts && tsx src/services/server-probe-public-result.test.ts && tsx src/services/server-probe-execution.test.ts && tsx src/services/server-probe-rest-contract.test.ts && tsx src/services/server-probe-rest-execution.test.ts && tsx src/services/server-probe-journal.test.ts && tsx src/services/server-probe-journal-history.test.ts && tsx src/modules/mcp/server-probe-actions.test.ts && tsx src/modules/mcp/diagnostic-cancellation.test.ts && tsx src/modules/mcp/service-diagnostic-progress.test.ts && tsx src/services/mikrotik-get-admission.test.ts && tsx src/services/server-probe-storage-journal-adapter.test.ts && npm run test:server-probe-history && npm run test:server-probe-storage-models",
"test:server-poll-http:pg": "node scripts/test-server-poll-http-owned.mjs",
"test:server-poll-approval:pg": "node scripts/test-server-poll-approval-owned.mjs",
"test:server-poll-atomic:pg": "node scripts/test-server-poll-atomic-owned.mjs",
"test:source-capture-adopted:pg": "node scripts/test-source-capture-adopted-postgres.mjs",
"test:server-poll-settings-epoch:pg": "node scripts/test-server-poll-settings-epoch-owned.mjs",
"test:server-poll-management": "tsx --test src/services/server-poll-maintenance-planner.test.ts src/services/server-poll-maintenance-groups.test.ts && tsx src/services/server-poll-execution.test.ts && tsx src/services/server-poll-production.test.ts && tsx src/services/server-poll-journal.test.ts && tsx src/services/server-poll-authority.test.ts && tsx src/services/server-poll-rest.test.ts && tsx src/services/server-partition-maintenance-lease.test.ts && npm run test:server-poll-storage && npm run test:partition-writers && tsx --test src/services/server-poll-history-verification.test.ts src/services/server-poll-terminal-source.test.ts src/services/server-poll-terminal-archive.test.ts src/services/server-poll-archive-accounting.test.ts src/services/server-poll-terminal-compaction.test.ts src/services/server-poll-history-reader.test.ts src/modules/mcp/server-poll-actions.test.ts src/modules/mcp/server-poll-production.test.ts src/modules/mcp/server-poll-registration.test.ts && tsx src/modules/mcp/effective-connection-source.test.ts && tsx src/services/server-poll-cost-diagnostics.test.ts && tsx src/services/server-poll-journal-snapshot.test.ts && tsx src/modules/mcp/principal-security-source.test.ts && tsx --test src/modules/mcp/poll-runtime-policy-source.test.ts src/modules/mcp/poll-policy-coordinator.test.ts && tsx src/services/server-poll-http.test.ts && tsx --test src/services/server-poll-approval-context.test.ts",
"test:server-poll-storage": "tsx src/services/server-poll-capacity.test.ts && tsx src/services/server-poll-storage-model.test.ts && tsx src/services/server-poll-storage.test.ts && tsx src/services/server-poll-storage-journal.test.ts && tsx src/services/server-poll-storage-retention.test.ts",
"test:bandwidth-admission:pg": "node scripts/test-mcp-postgres.mjs --bandwidth-admission --restore",
"test:bandwidth-control-source:pg": "node scripts/test-mcp-postgres.mjs --bandwidth-control-source --restore",
"test:bandwidth-control-source": "tsx src/services/bandwidth-control-role.test.ts && tsx src/services/bandwidth-control-catalog.test.ts && tsx --test src/services/bandwidth-control-identity.test.ts",
"bandwidth:control:seal": "tsx src/scripts/seal-bandwidth-control-source.ts",
"test:bandwidth-node-observation": "tsx src/services/bandwidth-node-observation.test.ts",
"test:bandwidth-peers": "tsx src/services/bandwidth-peer-observation.test.ts && tsx src/services/bandwidth-peer-network-observation.test.ts && tsx src/services/bandwidth-peer-direct-observation.test.ts && tsx src/services/bandwidth-operator-enrollment.test.ts && tsx --test --test-concurrency=1 src/services/bandwidth-control-registry-frame.test.ts src/services/bandwidth-control-registry-identity.test.ts src/services/bandwidth-node-identity.test.ts src/services/bandwidth-node-registry-batch.test.ts src/services/bandwidth-operator-docker-stop.test.ts src/services/bandwidth-operator-writer-window.test.ts src/services/bandwidth-operator-writer-composition.test.ts src/services/bandwidth-operator-registry.test.ts src/services/bandwidth-external-node-registry.test.ts src/services/bandwidth-external-node-registry-catalog.test.ts src/services/bandwidth-external-node-registry-role.test.ts src/modules/mcp/routing-bandwidth-original.test.ts src/services/bandwidth-peer-identity.test.ts src/services/bandwidth-peer-resolver.test.ts && node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/bandwidth-node-proof-batch.test.ts src/services/bandwidth-node-key-effect.test.ts src/services/bandwidth-control-file.test.ts src/services/bandwidth-node-bootstrap-v2.test.ts src/services/bandwidth-peer-fence.test.ts src/services/bandwidth-operator-public-files.test.ts src/services/bandwidth-registry-combined.test.ts && npm run test:bandwidth-public-metadata && npm run test:bandwidth-public-registry && npm run test:bandwidth-peer-metadata && npm run test:bandwidth-guarded-peer-sources",
"test:bandwidth-peers:pg": "node scripts/test-mcp-postgres.mjs --bandwidth-external-registry",
"test:bandwidth-rest-origin:pg": "node scripts/test-mcp-postgres.mjs --rest-speed-original",
"test:peer-speed-workflow:pg": "node scripts/test-mcp-postgres.mjs --bandwidth-speed-workflow",
"test:scheduled-speed-original:pg": "node scripts/test-mcp-postgres.mjs --scheduled-speed-original",
"test:operator-docker:pg": "node scripts/test-mcp-postgres.mjs --operator-docker-sql",
"test:bandwidth-public-metadata": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/bandwidth-control-public-metadata.test.ts src/services/bandwidth-node-public-metadata.test.ts",
"test:bandwidth-public-registry:pg": "node scripts/test-mcp-postgres.mjs --public-registry-metadata",
"test:bandwidth-public-registry": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/bandwidth-control-public-registry.test.ts src/services/bandwidth-node-public-registry.test.ts",
"test:bandwidth-peer-metadata": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/bandwidth-peer-metadata.test.ts src/services/bandwidth-peer-metadata-source.test.ts src/services/bandwidth-operator-metadata.test.ts",
"test:operator-peer-metadata:pg": "node scripts/test-mcp-postgres.mjs --operator-peer-metadata",
"test:bandwidth-guarded-peer-sources": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/server-probe-storage-connection.test.ts src/services/bandwidth-operator-peer-source-guard.test.ts src/services/bandwidth-external-peer-source.test.ts src/services/bandwidth-managed-peer-source.test.ts src/services/bandwidth-operator-peer-source-composition.test.ts",
"test:guarded-peer-sources:pg": "node scripts/test-mcp-postgres.mjs --external-peer-metadata",
"test:mcp:operation-origin": "node --experimental-test-module-mocks --import tsx --test src/modules/mcp/operation-execution-origin.test.ts",
"test:mcp:operation-origin:pg": "node scripts/test-mcp-postgres.mjs --mcp-operation-origin",
"test:mcp:speed-original": "node --import tsx --test src/modules/mcp/speed-bandwidth-original.test.ts",
"test:mcp:speed-original:pg": "node scripts/test-mcp-postgres.mjs --mcp-speed-original",
"test:router-configuration-guard:pg": "node scripts/test-mcp-postgres.mjs --router-configuration-guard",
"test:router-configuration-writer": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/router-configuration-writer-context.test.ts src/services/router-configuration-native-transport.test.ts",
"test:scheduled-speed-original": "node --experimental-test-module-mocks --import tsx src/services/bandwidth-scheduled-speed-original.test.ts",
"test:peer-speed-workflow": "node --experimental-test-module-mocks --import tsx --test src/services/bandwidth-speed-run-identity.test.ts src/services/bandwidth-speed-workflow.test.ts",
"test:bandwidth-node-registry:pg": "node scripts/test-mcp-postgres.mjs --bandwidth-node-registry --restore",
"test:mcp:poll-terminal-claim-loss:pg": "node scripts/test-mcp-postgres.mjs --poll-terminal-archive-claim-loss --restore",
"test:mcp:poll-terminal-history-cold:pg": "node scripts/test-mcp-postgres.mjs --poll-terminal-archive-history-cold --restore",
"test:mcp:poll-terminal-history:pg": "node scripts/test-mcp-postgres.mjs --poll-terminal-archive-history --restore",
"test:mcp:poll-terminal-archive-lost-ack:pg": "node scripts/test-mcp-postgres.mjs --poll-terminal-archive-lost-ack --restore",
"test:mcp:poll-terminal-archive:pg": "node scripts/test-mcp-postgres.mjs --poll-terminal-archive --restore",
"test:mcp:poll-archive-schema:pg": "node scripts/test-mcp-postgres.mjs --poll-archive-schema",
"test:mcp:telemetry-configuration:pg": "node scripts/test-mcp-postgres.mjs --telemetry-settings-configuration",
"test:mcp:telemetry-configuration": "tsx --test src/modules/mcp/telemetry-settings-configuration.test.ts",
"test:mcp:routing-schema:pg": "node scripts/test-mcp-postgres.mjs --routing-reservation-schema",
"test:mcp:reserved-step": "tsx --test src/modules/mcp/routing-reserved-step.test.ts",
"test:mcp:routing-forward-rounds:pg": "node scripts/test-mcp-postgres.mjs --routing-network-forward-rounds",
"test:mcp:routing-original-effects:pg": "node scripts/test-mcp-postgres.mjs --routing-original-effects",
"test:flow-active-services:pg": "node scripts/test-mcp-postgres.mjs --flow-active-services",
"test:flow-active-rebuild:pg": "node scripts/test-mcp-postgres.mjs --flow-active-rebuild",
"test:flow-worker-ipc:pg": "node scripts/test-mcp-postgres.mjs --flow-worker-ipc",
"test:manual-uptime:pg": "node scripts/test-mcp-postgres.mjs --manual-uptime",
"test:manual-ping-group:pg": "node scripts/test-mcp-postgres.mjs --manual-ping-group",
"test:manual-traffic:pg": "node scripts/test-mcp-postgres.mjs --manual-traffic",
"test:manual-remaining": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/modules/mcp/manual-uptime-actions.test.ts src/modules/mcp/manual-ping-group-actions.test.ts src/modules/mcp/manual-traffic-actions.test.ts src/services/manual-data-collection-execution.test.ts src/routes/manual-data-collection.test.ts",
"test:manual-api-ping:pg": "node scripts/test-mcp-postgres.mjs --manual-api-ping",
"test:manual-api-ping": "node --experimental-test-module-mocks --import tsx --test src/modules/mcp/manual-api-ping-actions.test.ts src/services/manual-data-collection-execution.test.ts src/routes/manual-data-collection.test.ts",
"test:simple-speed:pg": "node scripts/test-mcp-postgres.mjs --simple-speed",
"test:simple-speed:chr": "node scripts/test-mcp-postgres.mjs --simple-speed-chr",
"test:simple-speed": "node --experimental-test-module-mocks --import tsx --test src/modules/mcp/speed-test-actions.test.ts src/services/uptime-speed-service.test.ts && node --import tsx --test src/routes/speed-test.test.ts src/services/speed-run-persistence.test.ts src/services/btest-node-locks.test.ts src/services/bandwidth-test-status.test.ts src/services/bandwidth-test-rates.test.ts",
"test:bandwidth-node-key-effect": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/bandwidth-node-key-effect.test.ts src/services/bandwidth-node-proof-batch.test.ts",
"test:bandwidth-node-source": "tsx src/services/bandwidth-node-registry-role.test.ts && tsx src/services/bandwidth-node-registry-catalog.test.ts",
"test:bandwidth-node-source:pg": "node scripts/test-mcp-postgres.mjs --bandwidth-node-source --restore",
"test:mcp:service-context": "tsx --test src/services/config-mcp-service-execution.test.ts src/modules/mcp/temporary-grant-origin.test.ts src/modules/mcp/routing-measurement-origin.test.ts",
"test:mcp:service-context:pg": "node scripts/test-mcp-postgres.mjs --service-execution-origin",
"test:mcp:operation-locks": "tsx --test src/services/operation-config-lock-lease.test.ts && npm run test:scheduler-automatic",
"test:mcp:operation-locks:pg": "node scripts/test-mcp-postgres.mjs --operation-locks",
"test:scheduler-automatic": "node --test src/services/scheduler-automatic-root.test.mjs src/services/scheduler-automatic-authority.test.mjs",
"test:speed-workflow": "npm run test:router-configuration-writer && npm run test:mcp:speed-original && npm run test:scheduled-speed-original && npm run test:peer-speed-workflow && npm run test:bandwidth-node-observation && npm run test:bandwidth-native-permit && tsx src/services/bandwidth-admission.test.ts && tsx src/services/btest-node-locks.test.ts && tsx src/services/mikrotik-speed-admission.test.ts && tsx src/services/bandwidth-test-status.test.ts && tsx src/services/bandwidth-test-rates.test.ts && tsx src/services/speed-run-persistence.test.ts && tsx src/routes/speed-test.test.ts && tsx src/routes/speed-probes-write.test.ts",
"test:bandwidth-native-dispatch": "node --experimental-test-module-mocks --import tsx --test --test-concurrency=1 src/services/mikrotik-managed-source.test.ts src/services/bandwidth-native-dispatch-composite.test.ts src/services/bandwidth-peer-execution-host-fence.test.ts src/services/bandwidth-native-dispatch-source.test.ts src/services/bandwidth-native-dispatch.test.ts src/services/bandwidth-native-permit.test.ts src/services/bandwidth-admission.test.ts src/services/mikrotik-speed-admission.test.ts",
"test:bandwidth-native-permit": "tsx src/services/bandwidth-native-permit.test.ts",
"test:bandwidth-native-permit:pg": "node scripts/test-mcp-postgres.mjs --bandwidth-control-source --restore",
"test:server-poll-mcp": "tsx src/services/server-poll-boundary.test.ts && tsx src/modules/mcp/server-poll-actions.test.ts && tsx src/modules/mcp/server-poll-production.test.ts && tsx src/modules/mcp/server-poll-registration.test.ts",
"test:server-poll-storage:pg": "node scripts/test-mcp-postgres.mjs --server-poll-storage --restore",
"test:server-poll-storage-fresh:pg": "node scripts/test-mcp-postgres.mjs --server-poll-storage-fresh --restore",
"test:partition-writers": "tsx src/services/partition-write-transaction.test.ts && tsx src/services/collector-partition-writes.test.ts && tsx src/db/partitions-writer.test.ts && tsx src/db/migrate-partition-admission.test.ts && tsx src/db/migration-manifest.test.ts && tsx src/db/sqlite-import-partition-admission.test.ts && tsx src/services/system-database-partition-restore.test.ts && tsx src/services/traffic-flow-write-transaction.test.ts && tsx src/services/traffic-flow-facts-rebuild-cli.test.ts",
"test:mcp:routing-authority": "tsx --test src/modules/mcp/routing-switch-ledger-contract.test.ts && tsx src/modules/mcp/routing-profile-authority.test.ts && tsx src/modules/mcp/routing-recovery-authority.test.ts && tsx src/modules/mcp/routing-reservation.test.ts && tsx src/modules/mcp/routing-community-reserved-source.test.ts && tsx src/modules/mcp/routing-network-verifier-model.test.ts && tsx --test src/test/committed-scoped-checkpoint-observer.test.ts && node --test src/modules/mcp/routing-forward-transport-charge.test.cjs",
"test:mcp:native-readers": "tsx src/modules/mcp/routing-network-verifier-model.test.ts && tsx src/modules/mcp/routing-network-native-parser.test.ts && tsx src/modules/mcp/routing-network-native-session.test.ts && tsx src/db/migration-schema-inventory.test.ts && tsx src/modules/mcp/routing-network-observation-assembly.test.ts && tsx src/modules/mcp/routing-network-configuration-bridge.test.ts && tsx --test src/services/routing-table-fib.test.ts src/services/routing-topology-observation.test.ts && node --test src/modules/mcp/routing-original-effects.test.mjs src/modules/mcp/routing-original-effects-caller.test.mjs src/modules/mcp/routing-original-effects-controller.test.mjs src/modules/mcp/routing-network-execution-origin.test.mjs",
"test:mcp:native-readers:pg": "node scripts/test-mcp-postgres.mjs --routing-network-native --restore",
"test:mcp:schema-inventory:pg": "node scripts/test-mcp-postgres.mjs --migration-schema-inventory --restore",
"test:mcp:native-tls": "tsx src/modules/mcp/routing-network-native-tls.test.ts",
"test:mcp:schema-control:pg": "node scripts/test-mcp-postgres.mjs --database-schema-control --restore",
"test:mcp:chr-acquisition": "node --test scripts/acceptance/chr-recursive-observation.test.mjs",
"test:mcp:controlled-migrations:pg": "node scripts/test-mcp-postgres.mjs --controlled-migrations --restore",
"test:mcp:shared-context": "tsx --test src/modules/mcp/routing-network-scoped-context.test.ts",
"test:mcp:controlled-reference:pg": "node scripts/test-mcp-postgres.mjs --controlled-schema-reference --restore",
"test:server-probe-storage-fresh:pg": "node scripts/test-mcp-postgres.mjs --probe-storage-fresh --restore",
"test:server-poll-durable:pg": "node scripts/test-mcp-postgres.mjs --server-poll-durable --restore",
"test:server-probe-legacy:pg": "node scripts/test-mcp-postgres.mjs --probe-legacy-orphans",
"test:server-probe-storage:pg": "node scripts/test-mcp-postgres.mjs --probe-storage-journal",
"test:server-probe-history": "tsx --test src/services/server-probe-storage-history.test.ts src/services/server-probe-retired-source.test.ts",
"test:server-probe-history:pg": "node scripts/test-mcp-postgres.mjs --probe-storage-history",
"test:server-probe-retirement:pg": "node scripts/test-mcp-postgres.mjs --probe-operation-retirement",
"test:server-probe-storage-models": "tsx src/services/server-probe-journal-maintenance.test.ts && tsx src/modules/mcp/diagnostic-cancellation-source.test.ts && tsx src/services/server-probe-storage-source.test.ts && tsx src/services/server-probe-storage-connection.test.ts && tsx src/modules/mcp/server-probe-storage-host.test.ts && tsx src/services/server-probe-storage-rest.test.ts && tsx src/services/server-probe-storage-metadata.test.ts && tsx src/services/server-probe-legacy-verifier.test.ts && tsx src/services/server-probe-legacy-snapshot.test.ts && tsx src/services/server-probe-legacy-inspection.test.ts && tsx src/db/database-source-observation.test.ts && tsx src/services/server-probe-retention.test.ts && tsx src/modules/mcp/server-probe-registration.test.ts",
"test:mcp:migration-data:pg": "node scripts/test-mcp-postgres.mjs --migration-data-inventory --restore",
"test:mcp:poll-memory:pg": "npm run build -w @mmapp/contracts && node scripts/acceptance/build-poll-memory.mjs && node scripts/test-mcp-postgres.mjs --poll-memory",
"test:mcp:poll-memory-bootstrap:pg": "npm run build -w @mmapp/contracts && node scripts/acceptance/build-poll-memory.mjs && node scripts/test-mcp-postgres.mjs --poll-memory-bootstrap",
"test:mcp:poll-memory-cost:pg": "npm run build -w @mmapp/contracts && node scripts/acceptance/build-poll-memory.mjs && node scripts/test-mcp-postgres.mjs --poll-memory-cost",
"test:mcp:poll-journal-snapshot:pg": "node scripts/test-mcp-postgres.mjs --poll-journal-snapshot",
"test:mcp:principal-security-source:pg": "node scripts/test-mcp-postgres.mjs --principal-security-source",
"test:traffic-flow:read-budget": "tsx --test src/services/traffic-flow-read-budget.test.ts src/modules/mcp/flow-read-quality.test.ts && node scripts/test-flow-read-adapters.mjs",
"test:traffic-flow:read-http:pg": "node scripts/test-flow-read-http-owned.mjs",
"test:traffic-flow:read-source:pg": "tsx scripts/test-flow-read-sql-owned.mts",
"test:mcp:routing-baseline": "tsx --test src/modules/mcp/routing-controller-active-writer.test.ts src/modules/mcp/routing-controller-baseline.test.ts src/modules/mcp/routing-controller-fence.test.ts src/modules/mcp/routing-community-public.test.ts src/modules/mcp/routing-controller-overlap.test.ts",
"test:mcp:routing-baseline:pg": "node scripts/test-routing-controller-baseline.mjs",
"test:mcp:routing-baseline-inverse:pg": "node scripts/test-routing-controller-baseline.mjs --default-scoped",
"test:sqlite-import-control:pg": "node scripts/test-sqlite-import-control-postgres.mjs",
"test:traffic-flow:resource-profile": "tsx --test src/services/traffic-flow-read-resource-profile.test.ts && node scripts/test-flow-read-profile-startup.mjs",
"test:traffic-flow:settings-read-profiles:pg": "node scripts/test-flow-settings-read-profiles-owned.mjs",
"test:database-source-binding": "node scripts/test-database-source-binding.mjs"
},
"dependencies": {
"@aws-sdk/client-s3": "^3.888.0",
@@ -48,6 +193,10 @@
"@fastify/jwt": "^10.2.2",
"@fastify/type-provider-zod": "^1.0.0",
"@mmapp/contracts": "1.0.0",
"@modelcontextprotocol/client": "2.3.1",
"@modelcontextprotocol/fastify": "2.0.1",
"@modelcontextprotocol/node": "2.1.1",
"@modelcontextprotocol/server": "2.3.1",
"acme-client": "^5.4.0",
"better-sqlite3": "^12.9.0",
"dotenv": "^16.4.7",
@@ -64,6 +213,7 @@
"@types/node": "^22.15.3",
"@types/pg": "^8.23.1",
"drizzle-kit": "^0.31.10",
"embedded-postgres": "18.4.0-beta.17",
"jose": "^6.2.11",
"pino-pretty": "^13.1.3",
"tsx": "^4.19.3",
@@ -0,0 +1,72 @@
import { createHash } from 'node:crypto';
import { execFileSync } from 'node:child_process';
import { cpSync, mkdirSync, readFileSync, readdirSync, statSync, writeFileSync } from 'node:fs';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const root = fileURLToPath(new URL('../../../', import.meta.url));
const baseRevision = 'bf3d50e1ff2ee3b5c9268991a255c3b49ce6378e';
const sha = bytes => createHash('sha256').update(bytes).digest('hex');
const require = (value, reason) => { if (!value) throw Error(reason); };
const git = (...args) => execFileSync('git', args, { cwd: root, stdio: ['ignore', 'pipe', 'pipe'], maxBuffer: 4 * 1024 * 1024 });
const fromRoot = filename => path.join(root, filename);
const readJson = bytes => JSON.parse(bytes.toString('utf8'));
const manifestPath = ['backend/package.json', 'packages/contracts/package.json'];
const runtimeKeys = ['type', 'exports', 'engines', 'dependencies', 'optionalDependencies', 'peerDependencies'];
const normalizeLock = bytes => bytes.toString('utf8').replace(/\r\n/g, '\n');
const sort = values => [...new Set(values)].sort();
function runtimePins() {
const lockFile = readFileSync(fromRoot('package-lock.json'));
require(normalizeLock(lockFile) === normalizeLock(git('show', baseRevision + ':package-lock.json')), 'Runtime lock differs from cached source');
const lock = readJson(lockFile).packages;
return manifestPath.flatMap(filename => {
const current = readJson(readFileSync(fromRoot(filename)));
const cached = readJson(git('show', baseRevision + ':' + filename));
for (const key of runtimeKeys) {
require(JSON.stringify(current[key]) === JSON.stringify(cached[key]), 'Runtime package manifest differs from cached source');
}
const workspace = path.posix.dirname(filename);
return sort(Object.keys(current.dependencies ?? {})).filter(name => name !== '@mmapp/contracts').map(name => {
const nested = workspace + '/node_modules/' + name, shared = 'node_modules/' + name;
const installed = lock[nested] ? nested : shared;
const version = lock[installed]?.version;
require(typeof version === 'string', 'Runtime dependency pin unavailable');
return { path: installed + '/package.json', name, version };
});
}).filter((item, index, all) => all.findIndex(other => other.path === item.path) === index);
}
function compile(project, outDir) {
execFileSync(process.execPath, [fromRoot('node_modules/typescript/bin/tsc'), '--project', fromRoot(project), '--outDir', outDir,
'--declaration', 'false', '--declarationMap', 'false', '--sourceMap', 'false', '--incremental', 'false'],
{ cwd: root, stdio: 'inherit' });
}
function files(directory, prefix = '') {
return readdirSync(directory, { withFileTypes: true }).sort((a, b) => a.name.localeCompare(b.name)).flatMap(entry => {
const key = prefix + entry.name, item = path.join(directory, entry.name);
if (entry.isDirectory()) return files(item, key + '/');
require(entry.isFile(), 'Packet contains a nonregular entry');
const bytes = readFileSync(item);
return [{ path: key, size: bytes.length, sha256: sha(bytes) }];
});
}
const [runId, ...extra] = process.argv.slice(2);
require(/^[a-f0-9]{8}$/.test(runId ?? '') && !extra.length, 'Expected one eight-digit hexadecimal run ID');
const directory = fromRoot('tmp/mcp-netflow-smoke-packet-' + runId);
const dependencies = runtimePins();
mkdirSync(directory);
compile('packages/contracts/tsconfig.json', path.join(directory, 'contracts-dist'));
compile('backend/tsconfig.json', path.join(directory, 'dist'));
require(statSync(path.join(directory, 'dist/test/traffic-flow-chr-collector-bootstrap.js')).isFile(), 'Compiled collector entry unavailable');
cpSync(fromRoot('backend/drizzle'), path.join(directory, 'drizzle'), { recursive: true, errorOnExist: true });
const entries = files(directory);
require(entries.length > 0 && entries.length <= 4096 && entries.reduce((sum, entry) => sum + entry.size, 0) <= 128 * 1024 * 1024,
'Packet budget exceeded');
const body = Buffer.from(JSON.stringify({ format: 'mmapp-netflow-collector-smoke-v1', cachedSourceRevision: baseRevision, dependencies,
files: entries }, null, 2) + '\n');
writeFileSync(path.join(directory, 'manifest.json'), body, { flag: 'wx' });
console.log(JSON.stringify({ directory, manifestSha256: sha(body), files: entries.length, dependencies: dependencies.length,
collectorStarted: false, networkUsed: false }));
@@ -0,0 +1,22 @@
/** Separate output: never replace an in-flight or accepted poll-memory tree. */
import assert from 'node:assert/strict'
import { mkdir, lstat } from 'node:fs/promises'
import { dirname, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import ts from 'typescript'
assert(process.argv.length===2||process.argv.length===3,'Supply at most one new output name')
const outputName=process.argv[2]??'dist-installation-memory'
assert(/^dist-installation-memory(?:-[a-z0-9-]{3,64})?$/.test(outputName),'Invalid compiled output name')
const backend=resolve(dirname(fileURLToPath(import.meta.url)),'../..'), output=resolve(backend,outputName)
const existing=await lstat(output).catch(error=>{if(error.code!=='ENOENT')throw error})
assert(!existing,'Use a new output directory; no implicit cleanup or concurrent rebuild')
await mkdir(output)
const configPath=resolve(backend,'tsconfig.json'),config=ts.readConfigFile(configPath,ts.sys.readFile)
assert(!config.error)
const parsed=ts.parseJsonConfigFileContent(config.config,ts.sys,backend,undefined,configPath)
const options={...parsed.options,rootDir:resolve(backend,'src'),outDir:output,noEmit:false,noEmitOnError:true,declaration:false,declarationMap:false,sourceMap:false,incremental:false,composite:false,tsBuildInfoFile:undefined}
const roots=['src/services/installation-co-load.owned.test.ts','src/services/traffic-flow-collector-worker.ts'].map(file=>resolve(backend,file))
const program=ts.createProgram({rootNames:roots,options}),diagnostics=ts.getPreEmitDiagnostics(program)
if(diagnostics.length)throw Error(ts.formatDiagnosticsWithColorAndContext(diagnostics,{getCurrentDirectory:()=>backend,getCanonicalFileName:file=>file,getNewLine:()=> '\n'}))
assert(!program.emit().emitSkipped)
console.log(JSON.stringify({output,rootEntries:roots.length,actualWorkerIncluded:true,productionModulesSubstituted:false}))
@@ -0,0 +1,36 @@
/** Emit one owned memory entry plus its actual production import tree. The
* measured child runs plain node, not the TypeScript compiler or tsx. */
import assert from 'node:assert/strict'
import { lstat, rm } from 'node:fs/promises'
import { dirname, basename, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import ts from 'typescript'
const backend = resolve(dirname(fileURLToPath(import.meta.url)), '../..')
const output = resolve(backend, 'dist-mcp-memory')
assert.equal(dirname(output), backend)
assert.equal(basename(output), 'dist-mcp-memory')
const existing = await lstat(output).catch(error => { if (error.code !== 'ENOENT') throw error })
assert(!existing?.isSymbolicLink(), 'Refuse to clean a redirected output directory')
if (existing) await rm(output, { recursive: true })
const configPath = resolve(backend, 'tsconfig.json')
const config = ts.readConfigFile(configPath, ts.sys.readFile)
const format = diagnostics => ts.formatDiagnosticsWithColorAndContext(diagnostics, {
getCurrentDirectory: () => backend, getCanonicalFileName: name => name, getNewLine: () => '\n',
})
if (config.error) throw Error(format([config.error]))
const parsed = ts.parseJsonConfigFileContent(config.config, ts.sys, backend, undefined, configPath)
if (parsed.errors.length) throw Error(format(parsed.errors))
const entry = resolve(backend, 'src/services/server-poll-memory.owned.test.ts')
const options = { ...parsed.options, rootDir: resolve(backend, 'src'), outDir: output,
noEmit: false, noEmitOnError: true, declaration: false, declarationMap: false, sourceMap: false,
incremental: false, composite: false, tsBuildInfoFile: undefined }
// Deliberately override tsconfig include/exclude with this single test root.
// TypeScript still follows all normal imports; no module/path/IO substitutes.
const program = ts.createProgram({ rootNames: [entry], options })
const diagnostics = ts.getPreEmitDiagnostics(program)
if (diagnostics.length) throw Error(format(diagnostics))
const emitted = program.emit()
if (emitted.emitSkipped || emitted.diagnostics.length) throw Error(format(emitted.diagnostics) || 'Memory entry emit skipped')
console.log(JSON.stringify({ compiledPollMemoryEntry: resolve(output, 'services/server-poll-memory.owned.test.js'),
outputDirectory: output, rootEntries: 1, noEmitOnError: true, measuredChildUsesTsx: false }))
@@ -0,0 +1,325 @@
"""Fixed, opt-in local Docker owner for the disposable NetFlow collector."""
import argparse
import json
import os
import re
import secrets
import stat
import subprocess
import sys
from pathlib import Path
PG_IMAGE = 'postgres@sha256:77f585114c32fbca283dc835b0596f4e52b51b4c6662d7810b2f4084f60a1873'
NODE_IMAGE = 'git.shx.one/denozord/mikrotikmanager-backend-mcp-acceptance@sha256:151ab635caca5f2f6b50a9d0c27980a1559d7bd67f0e3a8d2b74cfa3f9ec6945'
CID = re.compile(r'[a-f0-9]{64}\Z')
RUN = re.compile(r'[a-f0-9]{8}\Z')
LABEL = 'io.mmapp.netflow.run'
ENV = {'PATH': '/usr/sbin:/usr/bin:/sbin:/bin', 'LC_ALL': 'C',
'HOME': '/nonexistent', 'DOCKER_CONFIG': '/nonexistent'}
def require(value, code):
if not value:
raise ValueError(code)
def root(run):
require(isinstance(run, str) and RUN.fullmatch(run), 'RUN_ID_REFUSED')
return '/tmp/mcp-netflow-' + run
def mounts(source, target, readonly=False):
return ['--mount', 'type=bind,src=' + source + ',dst=' + target + (',readonly' if readonly else '')]
def build_pg(run, uid, gid):
base = root(run)
require(type(uid) is int and 0 < uid <= 65534 and type(gid) is int and 0 < gid <= 65534, 'FIXTURE_ACCOUNT_REFUSED')
return [
'create', '--pull=never', '--name', 'mm-nf-pg-' + run,
'--label', LABEL + '=' + run, '--label', 'io.mmapp.netflow.role=postgres',
'--network=none', '--user', f'{uid}:{gid}', '--read-only',
'--cap-drop=ALL', '--security-opt=no-new-privileges',
'--memory=192m', '--memory-swap=192m',
'--tmpfs', '/tmp:rw,nosuid,nodev,noexec,mode=1777,size=32m',
'--tmpfs', '/var/run/postgresql:rw,nosuid,nodev,noexec,mode=1777,size=1m',
*mounts(base + '/pg', '/owned-pg'),
*mounts(base + '/pg-socket', base + '/pg-socket'),
*mounts(base + '/pg-password', '/run/mm-netflow-pg-secret', True),
'--env', 'PGDATA=/owned-pg/data', '--env', 'POSTGRES_USER=mcp_test',
'--env', 'POSTGRES_DB=mcp_netflow_' + run,
'--env', 'POSTGRES_PASSWORD_FILE=/run/mm-netflow-pg-secret',
'--env', 'POSTGRES_INITDB_ARGS=--auth-local=scram-sha-256 --auth-host=scram-sha-256',
PG_IMAGE, 'postgres', '-c', 'listen_addresses=', '-c', 'unix_socket_directories=' + base + '/pg-socket',
'-c', 'unix_socket_permissions=0700',
]
def build_node(run, uid, gid, anchor_id, port, namespace, host_namespace, packet_digest):
base = root(run)
require(type(uid) is int and 0 < uid <= 65534 and type(gid) is int and 0 < gid <= 65534, 'FIXTURE_ACCOUNT_REFUSED')
require(isinstance(anchor_id, str) and CID.fullmatch(anchor_id), 'ANCHOR_ID_REFUSED')
require(isinstance(packet_digest, str) and CID.fullmatch(packet_digest), 'PACKET_PIN_REFUSED')
require(type(port) is int and 49152 <= port <= 65535, 'COLLECTOR_PORT_REFUSED')
require(all(type(pair) in (tuple, list) and len(pair) == 2 and
all(type(value) is int and 0 < value < 2**64 for value in pair)
for pair in (namespace, host_namespace)) and list(namespace) != list(host_namespace), 'NAMESPACE_PINS_REFUSED')
return [
'create', '--pull=never', '--name', 'mm-nf-node-' + run,
'--label', LABEL + '=' + run, '--label', 'io.mmapp.netflow.role=collector',
'--network=container:' + anchor_id, '--user', f'{uid}:{gid}', '--read-only',
'--cap-drop=ALL', '--security-opt=no-new-privileges', '--interactive', '--tty',
'--memory=320m', '--memory-swap=320m',
'--tmpfs', '/tmp:rw,nosuid,nodev,noexec,mode=1777,size=32m',
*mounts(base + '/collector', base),
*mounts(base + '/pg-socket', base + '/pg-socket'),
*mounts(base + '/pg-password', '/run/mm-netflow-pg-secret', True),
*mounts(base + '/packet/dist', '/app/backend/dist', True),
*mounts(base + '/packet/drizzle', '/app/backend/drizzle', True),
*mounts(base + '/packet/contracts-dist', '/app/packages/contracts/dist', True),
*mounts(base + '/packet/manifest.json', '/run/mm-netflow-packet.json', True),
'--workdir', base, '--env', 'MMAPP_OWN_CHR_NETFLOW_COLLECTOR=true',
'--env', 'MMAPP_NETFLOW_PACKET_SHA256=' + packet_digest,
'--entrypoint', 'node', NODE_IMAGE,
'--max-old-space-size=160', '--max-semi-space-size=8',
'/app/backend/dist/test/traffic-flow-chr-collector-bootstrap.js',
'--container-collector-cycle', run, str(port), *map(str, namespace), *map(str, host_namespace),
]
def capture_args(run, first, second, port):
root(run)
require((first, second) == ('198.18.241.2', '198.18.241.9') and
type(port) is int and 49152 <= port <= 65535, 'CAPTURE_SCOPE_REFUSED')
return ['nsenter', '--net=/run/netns/nf' + run + 'nc', '--',
'tcpdump', '-n', '-p', '-U', '-i', 'nf' + run + 'cv', '-y', 'EN10MB', '-s', '4138',
'-c', '64', '-w', '-', 'udp and dst host 198.18.242.3 and dst port ' + str(port) +
' and (src host ' + first + ' or src host ' + second + ')']
def local_docker():
require(sys.platform == 'linux' and os.geteuid() == 0 and
os.environ.get('MMAPP_OWN_CHR_NETFLOW_CONTAINER') == 'true', 'DOCKER_OWNER_REFUSED')
socket = Path('/var/run/docker.sock')
require(stat.S_ISSOCK(socket.stat().st_mode), 'LOCAL_DOCKER_SOCKET_UNAVAILABLE')
choices = [Path(folder, 'docker') for folder in ('/usr/bin', '/usr/sbin', '/bin')]
binary = next((path.resolve(strict=True) for path in choices if path.is_file()), None)
require(binary is not None and binary.stat().st_uid == 0 and not binary.stat().st_mode & 0o022,
'DOCKER_BINARY_UNPINNED')
def call(*args):
result = subprocess.run([str(binary), '--host', 'unix:///var/run/docker.sock', *args],
env=ENV, stdin=subprocess.DEVNULL, stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL, timeout=30)
require(result.returncode == 0 and len(result.stdout) <= 256 * 1024, 'DOCKER_OUTCOME_UNCONFIRMED')
return result.stdout.decode('utf-8', 'strict').strip()
def attach(cid):
require(CID.fullmatch(cid) and sys.stdin.isatty() and sys.stdout.isatty(), 'EXACT_TTY_REFUSED')
return subprocess.run([str(binary), '--host', 'unix:///var/run/docker.sock', 'start', '--attach', '--interactive', cid],
env=ENV, timeout=330).returncode
return call, attach
def prepare_private_storage(run, uid, gid):
"""Create only new private bind sources. A retained fixture is never reused."""
require(sys.platform == 'linux' and os.geteuid() == 0 and
os.environ.get('MMAPP_OWN_CHR_NETFLOW_CONTAINER') == 'true', 'DOCKER_OWNER_REFUSED')
require(type(uid) is int and 0 < uid <= 65534 and type(gid) is int and 0 < gid <= 65534, 'FIXTURE_ACCOUNT_REFUSED')
base = Path(root(run))
info = base.lstat()
require(base.resolve(strict=True) == base and stat.S_ISDIR(info.st_mode) and info.st_uid == uid and
stat.S_IMODE(info.st_mode) == 0o700, 'FIXTURE_ROOT_REFUSED')
for name in ('pg', 'pg-socket', 'collector'):
directory = base / name
directory.mkdir(mode=0o700)
os.chown(directory, uid, gid, follow_symlinks=False)
fd = os.open(base / 'pg-password', os.O_WRONLY | os.O_CREAT | os.O_EXCL | os.O_NOFOLLOW, 0o400)
with os.fdopen(fd, 'wb') as output:
output.write((secrets.token_hex(32) + '\n').encode('ascii'))
output.flush()
os.fsync(output.fileno())
os.fchown(output.fileno(), uid, gid)
class CollectorContainerOwner:
"""Outer Linux parent retains each acknowledged ID and journals it.
An unknown create/start/retire acknowledgement requires reconciliation.
It never searches by label, restarts, prunes, or removes a matched name.
"""
def __init__(self, run, docker, checkpoint):
root(run)
self.run, self.docker, self.checkpoint = run, docker, checkpoint
self.ids = {}
self.attempted = set()
self.verified_exits = {}
self.anchor_start = None
self.adopted = None
self.topology_retired = False
def image(self, reference):
rows = json.loads(self.docker('image', 'inspect', reference))
require(len(rows) == 1 and reference in rows[0]['RepoDigests'] and rows[0]['Os'] == 'linux' and
re.fullmatch(r'sha256:[a-f0-9]{64}', rows[0]['Id']), 'CACHED_IMAGE_UNPINNED')
return rows[0]['Id']
def inspect(self, cid, role, image_id, network):
require(CID.fullmatch(cid), 'CONTAINER_ID_UNPINNED')
rows = json.loads(self.docker('container', 'inspect', cid))
require(len(rows) == 1, 'CONTAINER_UNCONFIRMED')
row = rows[0]
labels = row['Config']['Labels']
memory = (192 if role == 'postgres' else 320) * 1024 * 1024
require(row['Id'] == cid and row['Image'] == image_id and labels[LABEL] == self.run and
labels['io.mmapp.netflow.role'] == role and row['HostConfig']['NetworkMode'] == network and
not row['HostConfig']['Privileged'] and not row['HostConfig']['PortBindings'] and
row['HostConfig']['ReadonlyRootfs'] is True and row['HostConfig']['CapDrop'] == ['ALL'] and
not row['HostConfig']['CapAdd'] and row['HostConfig']['Memory'] == memory and
row['HostConfig']['MemorySwap'] == memory, 'CONTAINER_SCOPE_DRIFT')
return row['State']
def _create(self, role, argv, reference, network):
require(role not in self.attempted, 'CONTAINER_REPLAY_REFUSED')
self.attempted.add(role)
image_id = self.image(reference)
self.checkpoint(role, 'create-intent', None)
cid = self.docker(*argv)
require(CID.fullmatch(cid), 'CREATE_ACK_UNCONFIRMED')
self.checkpoint(role, 'create-acknowledged', cid)
state = self.inspect(cid, role, image_id, network)
require(state['Status'] == 'created' and state['Running'] is False, 'CREATED_STATE_UNCONFIRMED')
self.ids[role] = (cid, image_id, network)
self.checkpoint(role, 'create-verified', cid)
return cid
def create_anchor(self, uid, gid):
return self._create('postgres', build_pg(self.run, uid, gid), PG_IMAGE, 'none')
def create_collector(self, uid, gid, port, namespace, host_namespace, packet_digest, assert_packet):
anchor_id, image_id, network = self.ids['postgres']
anchor = self.inspect(anchor_id, 'postgres', image_id, network)
require(anchor['Running'] is True and anchor['Pid'] > 0, 'ANCHOR_PID_UNCONFIRMED')
require(self.adopted is not None, 'C_ADOPTION_UNCONFIRMED')
adopted, read_mount = self.adopted
require(adopted == self.anchor_namespace() and adopted['namespace'] == list(namespace) and
adopted['hostNamespace'] == list(host_namespace) and read_mount() == adopted['namespace'],
'C_ADOPTION_DRIFT')
assert_packet()
return self._create('collector', build_node(self.run, uid, gid, anchor_id, port, namespace, host_namespace, packet_digest),
NODE_IMAGE, 'container:' + anchor_id)
def start_anchor(self):
cid, image_id, network = self.ids['postgres']
require('postgres-start' not in self.attempted, 'CONTAINER_REPLAY_REFUSED')
self.attempted.add('postgres-start')
self.checkpoint('postgres', 'start-intent', cid)
require(self.docker('start', cid) == cid, 'START_ACK_UNCONFIRMED')
state = self.inspect(cid, 'postgres', image_id, network)
require(state['Running'] is True and type(state['Pid']) is int and state['Pid'] > 0 and
re.fullmatch(r'\d{4}-\d{2}-\d{2}T[0-9:.]+Z', state['StartedAt']), 'ANCHOR_PID_UNCONFIRMED')
self.checkpoint('postgres', 'start-verified', cid)
self.anchor_start = (state['Pid'], state['StartedAt'])
return state['Pid']
def anchor_namespace(self):
"""Return live anchor and host netns pins for the topology owner."""
require(sys.platform == 'linux' and os.geteuid() == 0, 'LINUX_OWNER_REQUIRED')
cid, image_id, network = self.ids['postgres']
state = self.inspect(cid, 'postgres', image_id, network)
pid = state['Pid']
started_at = state['StartedAt']
require(state['Running'] is True and self.anchor_start == (pid, started_at), 'ANCHOR_PID_UNCONFIRMED')
pin = Path('/proc', str(pid), 'ns/net').stat()
host = Path('/proc/self/ns/net').stat()
namespace, host_namespace = [pin.st_dev, pin.st_ino], [host.st_dev, host.st_ino]
latest = self.inspect(cid, 'postgres', image_id, network)
require(latest['Running'] is True and latest['Pid'] == pid and latest['StartedAt'] == started_at and
namespace != host_namespace,
'ANCHOR_NAMESPACE_DRIFT')
return {'containerId': cid, 'pid': pid, 'startedAt': started_at,
'namespace': namespace, 'hostNamespace': host_namespace}
def create_linux_topology(self, executor, backend, journal, planned, pins):
"""Bridge the exact live anchor into the existing guarded Linux owner."""
name = 'nf' + self.run + 'nc'
require(pins['nonce'][:8] == self.run and planned['names']['C'] == name, 'TOPOLOGY_RUN_DIFFERS')
require('topology-adopt' not in self.attempted, 'C_ADOPTION_REPLAY_REFUSED')
self.attempted.add('topology-adopt')
result = executor(backend, journal, planned, pins, read_collector_anchor=self.anchor_namespace)
adopted = journal.value['collectorAnchor']
require(adopted == self.anchor_namespace() and backend.namespace(name) == adopted['namespace'],
'C_ADOPTION_DRIFT')
self.adopted = (adopted, lambda: backend.namespace(name))
return result
def assert_linux_anchor_exited(self, recorded):
"""Validate the journalled container and its exact exit before C cleanup."""
cid, image_id, network = self.ids['postgres']
require(recorded['containerId'] == cid and 'postgres' in self.verified_exits and
('collector' not in self.ids or 'collector' in self.verified_exits), 'ANCHOR_EXIT_NOT_OWNED')
state = self.inspect(cid, 'postgres', image_id, network)
require(state['Running'] is False and state['Status'] == 'exited' and state['Pid'] == 0 and
state['StartedAt'] == recorded['startedAt'] == self.verified_exits['postgres'], 'ANCHOR_EXIT_UNCONFIRMED')
def cleanup_linux_topology(self, executor, backend, journal, planned):
require(self.adopted is not None and self.adopted[0] == journal.value.get('collectorAnchor') and
backend.root_baseline() == journal.value.get('rootBaseline'), 'TOPOLOGY_CLEANUP_SCOPE_DRIFT')
executor(backend, journal, planned, assert_collector_anchor_exited=self.assert_linux_anchor_exited)
require(journal.value['state'] == 'retired' and backend.root_baseline() == journal.value['rootBaseline'],
'TOPOLOGY_CLEANUP_UNCONFIRMED')
self.topology_retired = True
def start_collector(self, attach):
cid, image_id, network = self.ids['collector']
require('collector-start' not in self.attempted, 'CONTAINER_REPLAY_REFUSED')
require(self.adopted is not None and self.adopted[0] == self.anchor_namespace() and
self.adopted[1]() == self.adopted[0]['namespace'], 'C_ADOPTION_DRIFT')
self.attempted.add('collector-start')
self.checkpoint('collector', 'start-intent', cid)
status = attach(cid)
state = self.inspect(cid, 'collector', image_id, network)
require(state['Running'] is False and state['Status'] == 'exited' and state['Pid'] == 0 and
state['ExitCode'] == status and re.fullmatch(r'\d{4}-\d{2}-\d{2}T[0-9:.]+Z', state['StartedAt']),
'COLLECTOR_EXIT_UNCONFIRMED')
self.checkpoint('collector', 'exit-verified', cid)
self.verified_exits['collector'] = state['StartedAt']
require(status == 0, 'COLLECTOR_RUN_REFUSED')
def stop_anchor(self, assert_restored):
cid, image_id, network = self.ids['postgres']
require('postgres-stop' not in self.attempted, 'CONTAINER_REPLAY_REFUSED')
require('collector' not in self.ids or 'collector' in self.verified_exits, 'COLLECTOR_EXIT_UNCONFIRMED')
self.attempted.add('postgres-stop')
assert_restored()
self.checkpoint('postgres', 'stop-intent', cid)
current = self.inspect(cid, 'postgres', image_id, network)
require(current['Running'] is True and self.anchor_start == (current['Pid'], current['StartedAt']), 'ANCHOR_START_DRIFT')
require(self.docker('stop', '--time', '15', cid) == cid, 'STOP_ACK_UNCONFIRMED')
state = self.inspect(cid, 'postgres', image_id, network)
require(state['Running'] is False and state['Status'] == 'exited' and state['Pid'] == 0 and
state['StartedAt'] == self.anchor_start[1] and state['ExitCode'] == 0, 'ANCHOR_EXIT_UNCONFIRMED')
self.checkpoint('postgres', 'exit-verified', cid)
self.verified_exits['postgres'] = state['StartedAt']
def retire_exited(self, role):
cid, image_id, network = self.ids[role]
operation = role + '-retire'
require(operation not in self.attempted, 'CONTAINER_REPLAY_REFUSED')
require(role != 'postgres' or self.adopted is None or self.topology_retired, 'C_RETIREMENT_UNCONFIRMED')
self.attempted.add(operation)
state = self.inspect(cid, role, image_id, network)
require(role in self.verified_exits and state['Running'] is False and state['Status'] == 'exited' and
state['Pid'] == 0 and state['StartedAt'] == self.verified_exits[role], 'EXIT_NOT_CONFIRMED')
self.checkpoint(role, 'retire-intent', cid)
require(self.docker('rm', cid) == cid, 'RETIRE_ACK_UNCONFIRMED')
self.checkpoint(role, 'retire-acknowledged', cid)
if __name__ == '__main__':
cli = argparse.ArgumentParser()
cli.add_argument('--check-only', required=True, metavar='RUN_ID')
args = cli.parse_args()
print(json.dumps({'fixtureRoot': root(args.check_only), 'postgresImage': PG_IMAGE, 'nodeImage': NODE_IMAGE,
'requiresOwnedCollectorNamespace': True, 'dockerUsed': False, 'networkUsed': False}))
@@ -0,0 +1,143 @@
import json
import runpy
import unittest
from pathlib import Path
owner = runpy.run_path(str(Path(__file__).with_name('chr-netflow-collector-container.py')))
RUN = 'a012bcde'
PG_ID = 'a' * 64
NODE_ID = 'b' * 64
PACKET = 'd' * 64
IMAGE_ID = 'sha256:' + 'c' * 64
STARTED = '2026-11-04T12:00:00.000Z'
class DockerFixture:
def __init__(self):
self.calls = []
self.records = {}
self.lose_create = False
def __call__(self, *args):
self.calls.append(args)
if args[:2] == ('image', 'inspect'):
return json.dumps([{'Id': IMAGE_ID, 'Os': 'linux', 'RepoDigests': [args[2]]}])
if args[0] == 'create':
if self.lose_create:
raise RuntimeError('lost ACK')
pg = '--network=none' in args
role, cid = ('postgres', PG_ID) if pg else ('collector', NODE_ID)
self.records[cid] = {'Id': cid, 'Image': IMAGE_ID, 'Config': {'Labels': {
owner['LABEL']: RUN, 'io.mmapp.netflow.role': role}},
'HostConfig': {'NetworkMode': 'none' if pg else 'container:' + PG_ID, 'Privileged': False,
'PortBindings': None, 'ReadonlyRootfs': True, 'CapDrop': ['ALL'], 'CapAdd': None,
'Memory': (192 if pg else 320) * 1024 * 1024,
'MemorySwap': (192 if pg else 320) * 1024 * 1024},
'State': {'Running': False, 'Status': 'created', 'Pid': 0}}
return cid
cid = args[-1]
if args[:2] == ('container', 'inspect'):
return json.dumps([self.records[cid]])
if args[0] == 'start':
self.records[cid]['State'] = {'Running': True, 'Status': 'running', 'Pid': 42, 'StartedAt': STARTED}
elif args[0] == 'stop':
self.records[cid]['State'] = {'Running': False, 'Status': 'exited', 'Pid': 0, 'StartedAt': STARTED, 'ExitCode': 0}
elif args[0] == 'rm':
del self.records[cid]
else:
raise AssertionError(args)
return cid
class TestCollectorLauncher(unittest.TestCase):
def test_exact_images_no_pull_no_privilege_and_fixed_namespace(self):
pg = owner['build_pg'](RUN, 1000, 1000)
node = owner['build_node'](RUN, 1000, 1000, PG_ID, 50100, (4, 5), (4, 6), PACKET)
self.assertIn('--network=none', pg)
self.assertIn('--network=container:' + PG_ID, node)
self.assertIn('listen_addresses=', pg)
for argv in (pg, node):
self.assertIn('--pull=never', argv)
self.assertIn('--cap-drop=ALL', argv)
self.assertNotIn('--privileged', argv)
self.assertNotIn('/var/run/docker.sock', ' '.join(argv))
self.assertNotIn('POSTGRES_PASSWORD=', ' '.join(pg))
with self.assertRaisesRegex(ValueError, 'NAMESPACE_PINS'):
owner['build_node'](RUN, 1000, 1000, PG_ID, 50100, (4, 5), (4, 5), PACKET)
self.assertIn('EN10MB', owner['capture_args'](RUN, '198.18.241.2', '198.18.241.9', 50100))
def test_one_shot_ack_exact_ids_and_verified_cleanup(self):
docker, journal = DockerFixture(), []
launcher = owner['CollectorContainerOwner'](RUN, docker, lambda *entry: journal.append(entry))
self.assertEqual(launcher.create_anchor(1000, 1000), PG_ID)
self.assertEqual(launcher.start_anchor(), 42)
with self.assertRaisesRegex(ValueError, 'ADOPTION_UNCONFIRMED'):
launcher.create_collector(1000, 1000, 50100, (4, 5), (4, 6), PACKET, lambda: None)
anchor = {'containerId': PG_ID, 'pid': 42, 'startedAt': STARTED, 'namespace': [4, 5], 'hostNamespace': [4, 6]}
launcher.anchor_namespace = lambda: anchor
name = 'nf' + RUN + 'nc'
class Topology:
def namespace(self, target):
return [4, 5] if target == name else None
def root_baseline(self):
return 'root-baseline'
class Journal:
value = {}
def provision(backend, journal, planned, pins, read_collector_anchor):
journal.value.update(collectorAnchor=read_collector_anchor(), rootBaseline='root-baseline', state='created')
return {'ns:' + name: backend.namespace(name)}
def cleanup(backend, journal, planned, assert_collector_anchor_exited):
assert_collector_anchor_exited(journal.value['collectorAnchor'])
journal.value['state'] = 'retired'
topology, topology_journal, planned = Topology(), Journal(), {'names': {'C': name}}
launcher.create_linux_topology(provision, topology, topology_journal, planned, {'nonce': RUN + '0' * 24})
self.assertEqual(launcher.create_collector(1000, 1000, 50100, (4, 5), (4, 6), PACKET, lambda: None), NODE_ID)
with self.assertRaisesRegex(ValueError, 'EXIT_NOT_OWNED'):
launcher.cleanup_linux_topology(cleanup, topology, topology_journal, planned)
def attach(cid):
self.assertEqual(cid, NODE_ID)
docker.records[cid]['State'] = {'Running': False, 'Status': 'exited', 'Pid': 0, 'ExitCode': 0, 'StartedAt': STARTED}
return 0
launcher.start_collector(attach)
launcher.retire_exited('collector')
launcher.stop_anchor(lambda: None)
with self.assertRaisesRegex(ValueError, 'C_RETIREMENT_UNCONFIRMED'):
launcher.retire_exited('postgres')
launcher.cleanup_linux_topology(cleanup, topology, topology_journal, planned)
launcher.retire_exited('postgres')
self.assertEqual(docker.records, {})
self.assertIn(('collector', 'exit-verified', NODE_ID), journal)
self.assertEqual([call for call in docker.calls if call[0] == 'rm'], [('rm', NODE_ID), ('rm', PG_ID)])
def test_lost_create_ack_is_not_replayed_or_cleaned_by_name(self):
docker, journal = DockerFixture(), []
docker.lose_create = True
launcher = owner['CollectorContainerOwner'](RUN, docker, lambda *entry: journal.append(entry))
with self.assertRaisesRegex(RuntimeError, 'lost ACK'):
launcher.create_anchor(1000, 1000)
with self.assertRaisesRegex(ValueError, 'REPLAY'):
launcher.create_anchor(1000, 1000)
self.assertEqual(journal, [('postgres', 'create-intent', None)])
self.assertEqual([call[0] for call in docker.calls], ['image', 'create'])
def test_anchor_process_replaced_under_same_container_id_is_never_stopped(self):
docker = DockerFixture()
launcher = owner['CollectorContainerOwner'](RUN, docker, lambda *_: None)
launcher.create_anchor(1000, 1000)
launcher.start_anchor()
docker.records[PG_ID]['State']['StartedAt'] = '2026-11-04T13:00:00.000Z'
with self.assertRaisesRegex(ValueError, 'ANCHOR_START_DRIFT'):
launcher.stop_anchor(lambda: None)
self.assertFalse(any(call[0] == 'stop' for call in docker.calls))
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,242 @@
"""Disposable collector-only cycle. No RouterOS clients or capture are opened."""
import argparse
import gzip
import hashlib
import io
import json
import os
import re
import runpy
import stat
import subprocess
import sys
import tarfile
import time
from pathlib import Path
owner = runpy.run_path(str(Path(__file__).with_name('chr-netflow-collector-container.py')))
require = owner['require']
MAX_PACKET = 128 * 1024 * 1024
PACKET_PATH = re.compile(r'(?:dist|drizzle|contracts-dist)/(?:[A-Za-z0-9_.-]+/)*[A-Za-z0-9_.-]+\Z')
def verify_packet(directory, digest, require_root=False):
require(re.fullmatch(r'[a-f0-9]{64}', digest), 'PACKET_DIGEST_REFUSED')
require(directory.resolve(strict=True) == directory and directory.is_dir(), 'PACKET_DIRECTORY_REFUSED')
manifest = directory / 'manifest.json'
info = manifest.lstat()
require(stat.S_ISREG(info.st_mode) and 0 < info.st_size <= 1048576, 'PACKET_MANIFEST_REFUSED')
raw = manifest.read_bytes()
require(hashlib.sha256(raw).hexdigest() == digest, 'PACKET_MANIFEST_DRIFT')
value = json.loads(raw)
require(value.get('format') == 'mmapp-netflow-collector-smoke-v1' and
value.get('cachedSourceRevision') == 'bf3d50e1ff2ee3b5c9268991a255c3b49ce6378e' and
type(value.get('files')) is list and 1 <= len(value['files']) <= 4096, 'PACKET_MANIFEST_REFUSED')
total, listed = 0, {'manifest.json'}
for row in value['files']:
relative = row.get('path', '')
require(PACKET_PATH.fullmatch(relative) and
not any(part in ('.', '..') for part in relative.split('/')) and relative not in listed and
type(row.get('size')) is int and 0 < row['size'] <= 2 * 1024 * 1024 and
re.fullmatch(r'[a-f0-9]{64}', row.get('sha256', '')), 'PACKET_FILE_REFUSED')
item = directory / relative
info = item.lstat()
total += row['size']
require(total <= MAX_PACKET and stat.S_ISREG(info.st_mode) and item.resolve(strict=True) == item and
info.st_size == row['size'] and hashlib.sha256(item.read_bytes()).hexdigest() == row['sha256'], 'PACKET_FILE_DRIFT')
listed.add(relative)
observed = set()
for folder, directories, filenames in os.walk(directory, followlinks=False):
parent = Path(folder)
for entry in [parent, *(parent / name for name in directories + filenames)]:
info = entry.lstat()
require(not stat.S_ISLNK(info.st_mode) and
(not require_root or info.st_uid == 0 and not info.st_mode & 0o022), 'PACKET_OWNERSHIP_REFUSED')
observed.update((parent / filename).relative_to(directory).as_posix() for filename in filenames)
require(observed == listed and 'dist/test/traffic-flow-chr-collector-bootstrap.js' in listed, 'PACKET_FILE_SET_DRIFT')
return {'files': len(listed) - 1, 'bytes': total, 'manifestSha256': digest}
def pack(directory, digest):
match = re.fullmatch(r'mcp-netflow-smoke-packet-([a-f0-9]{8})', directory.name)
require(match is not None, 'PACKET_DIRECTORY_REFUSED')
verified = verify_packet(directory, digest)
archive = directory.parent / ('mcp-netflow-collector-smoke-' + match[1] + '.tar.gz')
manifest = json.loads((directory / 'manifest.json').read_bytes())
script_root = Path(__file__).resolve().parent
scripts = ('chr-netflow-collector-smoke.py', 'chr-netflow-collector-container.py')
with archive.open('xb') as destination, gzip.GzipFile(fileobj=destination, mode='wb', filename='', mtime=0) as compressed:
with tarfile.open(fileobj=compressed, mode='w|', format=tarfile.USTAR_FORMAT) as tar:
def add(name, body):
item = tarfile.TarInfo(name)
item.mode = 0o644
item.size = len(body)
tar.addfile(item, io.BytesIO(body))
for name in scripts:
add(name, (script_root / name).read_bytes())
add('packet/manifest.json', (directory / 'manifest.json').read_bytes())
for row in manifest['files']:
add('packet/' + row['path'], (directory / row['path']).read_bytes())
with archive.open('rb') as stream:
archive_sha = hashlib.file_digest(stream, 'sha256').hexdigest()
return {**verified, 'archive': str(archive), 'archiveSha256': archive_sha, 'dockerUsed': False, 'networkUsed': False}
class SmokeNamespace:
"""Only the C alias and one local dummy address, never the transit topology."""
def __init__(self, run, ip, namespace, checkpoint):
self.name, self.device = 'nf' + run + 'nc', 'nf' + run + 'cv'
self.ip, self.namespace, self.checkpoint = ip, namespace, checkpoint
self.anchor = None
def create(self, read_anchor):
require(self.anchor is None and self.namespace(self.name) is None, 'SMOKE_NAMESPACE_PRESENT')
anchor = read_anchor()
require(anchor['namespace'] != anchor['hostNamespace'] and read_anchor() == anchor, 'SMOKE_ANCHOR_DRIFT')
self.checkpoint('smoke', 'attach-intent', anchor)
self.ip('netns', 'attach', self.name, str(anchor['pid']))
require(self.namespace(self.name) == anchor['namespace'] and read_anchor() == anchor, 'SMOKE_ATTACH_UNCONFIRMED')
self.checkpoint('smoke', 'attach-verified', anchor)
self.anchor = anchor
def current():
require(self.namespace(self.name) == anchor['namespace'] and read_anchor() == anchor, 'SMOKE_ANCHOR_DRIFT')
links = json.loads(self.ip('-n', self.name, '-j', 'link', 'show'))
require([item['ifname'] for item in links] == ['lo'], 'SMOKE_NAMESPACE_NOT_EMPTY')
self.checkpoint('smoke', 'dummy-intent', self.device)
current()
self.ip('-n', self.name, 'link', 'add', self.device, 'type', 'dummy')
current()
created = json.loads(self.ip('-n', self.name, '-j', '-d', 'link', 'show', 'dev', self.device))
require(len(created) == 1 and created[0]['ifname'] == self.device and
created[0]['linkinfo']['info_kind'] == 'dummy' and created[0]['ifindex'] > 0, 'SMOKE_DUMMY_UNCONFIRMED')
self.ip('-n', self.name, 'address', 'add', '198.18.242.3/32', 'dev', self.device)
current()
self.ip('-n', self.name, 'link', 'set', self.device, 'up')
rows = json.loads(self.ip('-n', self.name, '-j', '-d', 'address', 'show', 'dev', self.device))
routes = json.loads(self.ip('-n', self.name, '-j', '-4', 'route', 'show', 'table', 'all'))
require(len(rows) == 1 and rows[0]['ifname'] == self.device and rows[0]['linkinfo']['info_kind'] == 'dummy' and
rows[0]['ifindex'] == created[0]['ifindex'] and
'UP' in rows[0]['flags'] and any(row.get('local') == '198.18.242.3' and row.get('prefixlen') == 32
for row in rows[0]['addr_info']) and all(row.get('dst') != 'default' for row in routes) and
self.namespace(self.name) == anchor['namespace'] and read_anchor() == anchor, 'SMOKE_DUMMY_UNCONFIRMED')
self.checkpoint('smoke', 'dummy-verified', rows[0]['ifindex'])
return anchor
def restore(self, assert_anchor_exited):
require(self.anchor is not None and self.namespace(self.name) == self.anchor['namespace'],
'SMOKE_NAMESPACE_DRIFT')
assert_anchor_exited(self.anchor)
require(not self.ip('netns', 'pids', self.name).strip(), 'SMOKE_NAMESPACE_BUSY')
self.checkpoint('smoke', 'delete-intent', self.anchor)
self.ip('netns', 'delete', self.name)
require(self.namespace(self.name) is None, 'SMOKE_NAMESPACE_RETIREMENT_UNCONFIRMED')
self.checkpoint('smoke', 'delete-verified', self.anchor)
def linux_ip():
binary = next((item.resolve(strict=True) for item in (Path('/usr/sbin/ip'), Path('/usr/bin/ip'), Path('/sbin/ip'))
if item.is_file()), None)
require(binary is not None and binary.stat().st_uid == 0 and not binary.stat().st_mode & 0o022, 'IP_BINARY_UNPINNED')
def call(*args):
result = subprocess.run([str(binary), *args], env=owner['ENV'], stdin=subprocess.DEVNULL,
stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, timeout=8)
require(result.returncode == 0 and len(result.stdout) <= 128 * 1024, 'IP_OUTCOME_UNCONFIRMED')
return result.stdout.decode('utf-8', 'strict')
return call
def ns_pin(name):
try:
row = Path('/run/netns', name).lstat()
except FileNotFoundError:
return None
require(stat.S_ISREG(row.st_mode) and row.st_uid == 0, 'SMOKE_NAMESPACE_HANDLE_UNSAFE')
return [row.st_dev, row.st_ino]
def run_smoke(run, uid, gid, port, digest):
docker, attach = owner['local_docker']()
base = Path(owner['root'](run))
packet = base / 'packet'
verify_packet(packet, digest, require_root=True)
ip = linux_ip()
require(ns_pin('nf' + run + 'nc') is None, 'SMOKE_NAMESPACE_PRESENT')
journal = os.open(base / 'collector-smoke.jsonl', os.O_CREAT | os.O_EXCL | os.O_WRONLY | os.O_NOFOLLOW, 0o600)
with os.fdopen(journal, 'w', encoding='utf-8') as report:
def checkpoint(role, phase, value):
report.write(json.dumps({'role': role, 'phase': phase, 'value': value}, separators=(',', ':')) + '\n')
report.flush()
os.fsync(report.fileno())
checkpoint('smoke', 'packet-verified', digest)
owner['prepare_private_storage'](run, uid, gid)
runner = owner['CollectorContainerOwner'](run, docker, checkpoint)
runner.create_anchor(uid, gid)
runner.start_anchor()
scope = SmokeNamespace(run, ip, ns_pin, checkpoint)
# Reuse the container owner's live-C admission and retirement checks.
class Journal:
value = {}
class Backend:
def namespace(self, name):
return ns_pin(name)
def root_baseline(self):
raw = [ip('-j', '-4', 'route', 'show', 'table', 'all'), ip('-j', '-d', 'link', 'show')]
# Link counters/traffic are outside root configuration; use exact names and ifindex.
links = sorted((row['ifname'], row['ifindex']) for row in json.loads(raw[1]))
return hashlib.sha256(json.dumps([json.loads(raw[0]), links], sort_keys=True).encode()).hexdigest()
def provision(backend, ledger, _plan, _pins, read_collector_anchor):
baseline = backend.root_baseline()
anchor = scope.create(read_collector_anchor)
require(backend.root_baseline() == baseline, 'ROOT_NETWORK_DRIFT')
ledger.value.update(collectorAnchor=anchor, rootBaseline=baseline, state='created')
return anchor
def cleanup(_backend, ledger, _plan, assert_collector_anchor_exited):
scope.restore(assert_collector_anchor_exited)
ledger.value['state'] = 'retired'
backend, ledger, plan = Backend(), Journal(), {'names': {'C': scope.name}}
pin = runner.create_linux_topology(provision, backend, ledger, plan, {'nonce': run + '0' * 24})
wait_until = time.monotonic() + 20
socket = base / 'pg-socket' / '.s.PGSQL.5432'
while not socket.exists() and time.monotonic() < wait_until:
time.sleep(.1)
require(stat.S_ISSOCK(socket.lstat().st_mode), 'PRIVATE_PG_NOT_READY')
runner.create_collector(uid, gid, port, pin['namespace'], pin['hostNamespace'], digest,
lambda: verify_packet(packet, digest, require_root=True))
runner.start_collector(attach)
runner.retire_exited('collector')
runner.stop_anchor(lambda: require(ledger.value['state'] == 'created', 'SMOKE_TOPOLOGY_DRIFT'))
runner.cleanup_linux_topology(cleanup, backend, ledger, plan)
runner.retire_exited('postgres')
checkpoint('smoke', 'complete', None)
return {'collectorSmokeCompleted': True, 'chrMutations': 0, 'capturePerformed': False, 'nativeAcceptance': False}
if __name__ == '__main__':
parser = argparse.ArgumentParser()
sub = parser.add_subparsers(dest='mode', required=True)
check = sub.add_parser('check')
check.add_argument('packet', type=Path)
check.add_argument('sha256')
package = sub.add_parser('pack')
package.add_argument('packet', type=Path)
package.add_argument('sha256')
run = sub.add_parser('run')
run.add_argument('run_id')
run.add_argument('uid', type=int)
run.add_argument('gid', type=int)
run.add_argument('port', type=int)
run.add_argument('sha256')
args = parser.parse_args()
try:
if args.mode == 'check':
result = {**verify_packet(args.packet.resolve(), args.sha256), 'dockerUsed': False, 'networkUsed': False}
elif args.mode == 'pack':
result = pack(args.packet.resolve(), args.sha256)
else:
result = run_smoke(args.run_id, args.uid, args.gid, args.port, args.sha256)
print(json.dumps(result))
except (OSError, ValueError, RuntimeError, subprocess.SubprocessError, KeyError) as error:
print(json.dumps({'completed': False, 'failureClass': type(error).__name__, 'automaticReplay': False}))
raise SystemExit(1) from None
@@ -0,0 +1,69 @@
import hashlib
import json
from pathlib import Path
import runpy
import tempfile
import unittest
smoke = runpy.run_path(str(Path(__file__).with_name('chr-netflow-collector-smoke.py')))
class SmokeTests(unittest.TestCase):
def test_pin_and_file_set_drift_fail_closed(self):
with tempfile.TemporaryDirectory(dir=Path(__file__).resolve().parents[3] / 'tmp') as temp:
root = Path(temp).resolve()
entry = root / 'dist/test/traffic-flow-chr-collector-bootstrap.js'
entry.parent.mkdir(parents=True)
entry.write_bytes(b'owned fixture\n')
body = json.dumps({'format': 'mmapp-netflow-collector-smoke-v1',
'cachedSourceRevision': 'bf3d50e1ff2ee3b5c9268991a255c3b49ce6378e',
'files': [{'path': entry.relative_to(root).as_posix(), 'size': entry.stat().st_size,
'sha256': hashlib.sha256(entry.read_bytes()).hexdigest()}]}).encode()
(root / 'manifest.json').write_bytes(body)
digest = hashlib.sha256(body).hexdigest()
self.assertEqual(smoke['verify_packet'](root, digest)['files'], 1)
entry.write_bytes(b'foreign data!\n')
with self.assertRaisesRegex(ValueError, 'FILE_DRIFT'):
smoke['verify_packet'](root, digest)
with self.assertRaisesRegex(ValueError, 'MANIFEST_DRIFT'):
smoke['verify_packet'](root, '0' * 64)
def test_exact_dummy_namespace_and_busy_cleanup(self):
anchor = {'containerId': 'a' * 64, 'pid': 42, 'namespace': [10, 20], 'hostNamespace': [10, 21]}
calls, journal = [], []
namespace = None
busy = True
name, device = 'nfa012bcdenc', 'nfa012bcdecv'
link = {'ifname': device, 'ifindex': 5, 'linkinfo': {'info_kind': 'dummy'}, 'flags': ['UP'],
'addr_info': [{'local': '198.18.242.3', 'prefixlen': 32}]}
def ip(*args):
nonlocal namespace
calls.append(args)
if args[:2] == ('netns', 'attach'):
self.assertEqual(args, ('netns', 'attach', name, '42'))
namespace = anchor['namespace']
elif args[:2] == ('netns', 'pids'):
return '42' if busy else ''
elif args[:2] == ('netns', 'delete'):
self.assertEqual(args, ('netns', 'delete', name))
namespace = None
elif args == ('-n', name, '-j', 'link', 'show'):
return json.dumps([{'ifname': 'lo'}])
elif '-d' in args:
return json.dumps([link])
elif 'route' in args:
return '[]'
return ''
fixture = smoke['SmokeNamespace']('a012bcde', ip, lambda _name: namespace, lambda *row: journal.append(row))
self.assertEqual(fixture.create(lambda: anchor), anchor)
self.assertIn(('-n', name, 'address', 'add', '198.18.242.3/32', 'dev', device), calls)
with self.assertRaisesRegex(ValueError, 'NAMESPACE_BUSY'):
fixture.restore(lambda pin: self.assertEqual(pin, anchor))
self.assertNotIn(('netns', 'delete', name), calls)
busy = False
fixture.restore(lambda pin: self.assertEqual(pin, anchor))
self.assertEqual(journal[-1][1], 'delete-verified')
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,123 @@
import assert from 'node:assert/strict'
import { execFileSync } from 'node:child_process'
import { randomBytes } from 'node:crypto'
import { lstat, mkdtemp, rm } from 'node:fs/promises'
import { createServer } from 'node:net'
import { tmpdir } from 'node:os'
import { isAbsolute, join, resolve } from 'node:path'
import { fileURLToPath } from 'node:url'
import EmbeddedPostgres from 'embedded-postgres'
import pg from 'pg'
import { applySqlMigrations } from '../../src/db/migrate.ts'
import { MIGRATION_MANIFEST } from '../../src/db/migration-manifest.ts'
assert.deepEqual(process.argv.slice(2), ['--owned-embedded'])
const bin = process.env.MMAPP_PG_CLIENT_BIN
assert(bin && isAbsolute(bin) && resolve(bin) === bin, 'Set MMAPP_PG_CLIENT_BIN to the exact local PostgreSQL client directory')
const psql = join(bin, process.platform === 'win32' ? 'psql.exe' : 'psql')
const executable = await lstat(psql)
assert(executable.isFile() && !executable.isSymbolicLink(), 'PostgreSQL client must be a regular file')
const clientEnv = Object.fromEntries(Object.entries(process.env).filter(([name]) => !/^PG/i.test(name)))
const run = (command, args, options = {}) => execFileSync(command, args, {
timeout: 10000, maxBuffer: 16384, encoding: 'utf8', windowsHide: true, ...options,
})
assert.match(run(psql, ['--version'], { env: clientEnv }), /^psql \(PostgreSQL\) (?:1[6-9]|[2-9][0-9])\./)
const witness = fileURLToPath(new URL('../../../tmp/mcp-netflow-linux-topology-20261104/collector-drain-witness.py', import.meta.url))
const { IDENTITY_SQL, SQL } = JSON.parse(run('python', ['-I', '-B', '-c',
'import json,runpy,sys; m=runpy.run_path(sys.argv[1]); print(json.dumps({k:m[k] for k in ("IDENTITY_SQL","SQL")}))', witness]))
assert(typeof SQL === 'string' && typeof IDENTITY_SQL === 'string')
const directory = await mkdtemp(join(tmpdir(), 'mm-netflow-drain-pg-'))
const password = randomBytes(24).toString('hex')
const host = '127.0.0.1', database = 'postgres', user = 'mcp_test'
const listener = createServer()
let embedded, pool, started = false
try {
await new Promise((done, reject) => { listener.once('error', reject); listener.listen(0, host, done) })
const { port } = listener.address()
await new Promise((done, reject) => listener.close(error => error ? reject(error) : done()))
embedded = new EmbeddedPostgres({
databaseDir: join(directory, 'pg'), port, user, password, persistent: true, createPostgresUser: false,
initdbFlags: ['--encoding=UTF8'], postgresFlags: ['-h', host, '-c', 'unix_socket_directories='],
onLog: () => {}, onError: () => {},
})
await embedded.initialise()
await embedded.start()
started = true
pool = new pg.Pool({ host, port, database, user, password, max: 1, connectionTimeoutMillis: 5000 })
await applySqlMigrations(pool)
const { rows: [version] } = await pool.query("SELECT current_setting('server_version_num')::int AS number, (SELECT count(*)::int FROM schema_migrations) AS migrations")
assert(version.number >= 180000 && version.number <= 189999)
assert.equal(version.migrations, MIGRATION_MANIFEST.length)
const runId = 'a012bcde', owner = 'netflow-operator-' + runId, collectorPort = 51000
const read = (sql, probePort = collectorPort) => JSON.parse(run(psql, ['-X', '-w', '-qAt',
'-h', host, '-p', String(port), '-U', user, '-d', database, '-v', 'ON_ERROR_STOP=1',
'-v', 'database=' + database, '-v', 'socket_dir=/tmp/mcp-netflow-' + runId + '/pg-socket',
'-v', 'run_id=' + runId, '-v', 'port=' + probePort],
{ env: { ...clientEnv, PGCONNECT_TIMEOUT: '2', PGPASSWORD: password }, input: sql }).trim())
const identity = read(IDENTITY_SQL)
assert.deepEqual(Object.keys(identity).sort(), ['oid', 'source'])
assert.match(identity.oid, /^[1-9][0-9]{0,9}$/)
// The production source predicate deliberately rejects this TCP fixture.
assert.equal(identity.source, false)
const expect = (planCount, enabled, verified, ordered = true, sqlPort = collectorPort) =>
assert.deepEqual(read(SQL, sqlPort), { ...identity, connectionCount: 1, planCount, enabled, verified, ordered })
const uuid = index => 'a012bcde-0000-4000-8000-' + String(index).padStart(12, '0')
const connection = uuid(1)
await pool.query(`INSERT INTO mcp_connections(id,name,owner_id,owner_permissions,policy,generation)
VALUES($1,$2,$3,'[]','{}','inert-witness')`, [connection, 'Owned NetFlow ' + runId, owner])
await pool.query("UPDATE traffic_flow_settings SET collector_ip='198.18.242.3', flow_listen_port=$1, enabled=false WHERE id=1", [collectorPort])
expect(0, false, [])
async function seed(index, enabled) {
const plan = uuid(index), operation = uuid(index + 1), receipt = uuid(index + 2)
const base = Date.UTC(2026, 0, 1) + index * 1000
const at = offset => new Date(base + offset)
const result = { receiptId: receipt, configurationConfirmed: true, receiptConfirmed: true,
listener: 'request-acknowledged', automaticReplay: false }
await pool.query(`INSERT INTO mcp_plans(id,connection_id,operation_id,operation_version,policy_version,generation,payload,evidence,state,expires_at,approved_by,approved_at,created_at)
VALUES($1,$2,'netflow.settings.update',3,1,'inert-witness','{}','{}','completed',$3,$4,$5,$6)`,
[plan, connection, at(9000), owner, at(100), at(0)])
await pool.query(`INSERT INTO mcp_operations(id,plan_id,connection_id,idempotency_key,state,verification,created_at,finished_at)
VALUES($1,$2,$3,$4,'applied','passed',$5,$6)`, [operation, plan, connection, 'inert-' + index, at(200), at(400)])
await pool.query(`INSERT INTO mcp_operation_steps(operation_id,step_index,state,intended_state,result,created_at,finished_at)
VALUES($1,0,'verified',$2,$3,$4,$5)`,
[operation, { service: 'netflow.settings.update', input: { receiptId: receipt, enabled } }, result, at(300), at(400)])
await pool.query(`INSERT INTO mcp_audit(actor_id,action,connection_id,payload,created_at)
VALUES($1,'plan.approve',$2,$3,$4)`, [owner, connection, { planId: plan }, at(100)])
await pool.query(`INSERT INTO flow_settings_intents(receipt_id,phase,request_proof,state,result,created_at)
VALUES($1,'configuration','inert-proof','completed',$2,$3),($1,'listener','inert-proof','completed',$2,$3)`, [receipt, result, at(300)])
await pool.query('UPDATE traffic_flow_settings SET enabled=$1 WHERE id=1', [enabled])
return [plan, operation, receipt]
}
const enabled = await seed(10, true)
expect(1, true, [enabled])
const disabled = await seed(20, false)
expect(2, false, [enabled, disabled])
expect(2, null, [enabled, disabled], true, collectorPort + 1)
await pool.query("UPDATE flow_settings_intents SET request_proof='foreign' WHERE receipt_id=$1 AND phase='listener'", [disabled[2]])
expect(2, false, [enabled])
await pool.query("UPDATE flow_settings_intents SET request_proof='inert-proof' WHERE receipt_id=$1 AND phase='listener'", [disabled[2]])
const { rows: [duplicate] } = await pool.query(`INSERT INTO mcp_audit(actor_id,action,connection_id,payload)
VALUES($1,'plan.approve',$2,$3) RETURNING id`, [owner, connection, { planId: disabled[0] }])
expect(2, false, [enabled])
await pool.query('DELETE FROM mcp_audit WHERE id=$1', [duplicate.id])
const { rows: [original] } = await pool.query(`UPDATE mcp_operations SET finished_at=(SELECT created_at + interval '1 second' FROM mcp_plans WHERE id=$2)
WHERE id=$1 RETURNING created_at`, [enabled[1], disabled[0]])
expect(2, false, [enabled, disabled], false)
await pool.query("UPDATE mcp_operations SET finished_at=$2::timestamptz + interval '200 milliseconds' WHERE id=$1", [enabled[1], original.created_at])
expect(2, false, [enabled, disabled])
console.log(JSON.stringify({ migrations: version.migrations, exactPsqlQueries: true, inertEnableDisableAndRejections: true,
linuxSocketSourceAccepted: false, realMcpExecutionClaimed: false }))
} finally {
try {
await pool?.end()
} finally {
if (started) await embedded.stop()
if (listener.listening) await new Promise(done => listener.close(done))
await rm(directory, { recursive: true, force: true, maxRetries: 5, retryDelay: 200 })
}
}
@@ -0,0 +1,221 @@
import { createHash } from 'node:crypto';
import { readFile } from 'node:fs/promises';
import { isIPv4 } from 'node:net';
import { pathToFileURL } from 'node:url';
const routes = Object.freeze(['/system/resource', '/ip/traffic-flow', '/ip/traffic-flow/target']);
const policyRoutes = Object.freeze(['/system/resource', '/ip/settings', '/ip/firewall/connection/tracking',
'/ip/firewall/filter', '/ip/firewall/raw', '/ip/firewall/mangle', '/ip/firewall/nat', '/routing/rule', '/ip/vrf']);
const counterFields = ['ifindex', 'rx-packet', 'tx-packet', 'fp-rx-byte', 'fp-tx-byte', 'fp-rx-packet', 'fp-tx-packet'];
const nativeId = value => typeof value === 'string' && /^\*[a-f0-9]{1,16}$/i.test(value);
const ports = Object.freeze({ '194.226.121.150': 19081, '104.165.244.7': 19082 });
const attestation = 'verified-known-host-161.104.46.222-owned-forward';
const reviewedTransportHash = '67d86dbae2710a25145df8ab9469029182f21a31972c49f063cdb6bc3daa8ea6';
const sha = bytes => createHash('sha256').update(bytes).digest('hex');
const deny = () => { throw new Error('NetFlow discovery refused'); };
export async function loadPinnedNetFlowTransport(policy = false) {
const allowedPaths = policy ? policyRoutes : routes;
return loadTransport(allowedPaths, allowedPaths.length * 2, policy ? 131072 : 65536);
}
async function loadTransport(allowedPaths, requests, responseLimit, totalLimit = responseLimit * requests) {
const bytes = await readFile(new URL('./chr-recursive-observation.mjs', import.meta.url));
if (sha(bytes) !== reviewedTransportHash) deny();
const reviewed = bytes.toString('utf8');
const source = reviewed.slice(0, reviewed.indexOf('async function privateStdin()'))
.replace(/const ROOT = .*?;/, 'const ROOT = null;')
.replace(/export const PATHS = Object.freeze\(\[[\s\S]*?\]\);/, `export const PATHS = Object.freeze(${JSON.stringify(allowedPaths)});`)
.replace('response: 1048576, bytes: 8388608, requests: 256',
`response: ${responseLimit}, bytes: ${totalLimit}, requests: ${requests}`);
const transport = await import('data:text/javascript;base64,' + Buffer.from(source).toString('base64'));
if (JSON.stringify(transport.PATHS) !== JSON.stringify(allowedPaths)) deny();
for (const [host, port] of Object.entries(ports)) {
if (endpoint(transport, host) !== `http://127.0.0.1:${port}`) deny();
}
return transport;
}
export async function discoverPinnedInterfaceCounterShape(read) {
const devices = [];
for (const host of Object.keys(ports)) {
const inventory = await read(host, '/interface');
if (!Array.isArray(inventory) || inventory.length > 256 ||
inventory.some(row => !nativeId(row?.['.id'])) ||
new Set(inventory.map(row => row['.id'])).size !== inventory.length) deny();
const preferred = type => inventory.find(row => row.type === type &&
observedFlag(row.running) === true && observedFlag(row.disabled) === false);
const chosen = preferred('wg') ?? preferred('ether');
if (!chosen) {
devices.push({ status: 'no-eligible-interface' });
continue;
}
const exact = single(await read(host, '/interface/' + encodeURIComponent(chosen['.id'])));
if (exact['.id'] !== chosen['.id'] || exact.type !== chosen.type ||
observedFlag(exact.running) !== true || observedFlag(exact.disabled) !== false) deny();
const fields = Object.fromEntries(counterFields.map(key => {
const value = exact[key];
return [key, { present: Object.hasOwn(exact, key),
type: value === null ? 'null' : Array.isArray(value) ? 'array' : typeof value }];
}));
devices.push({ status: 'shape-observed', fields });
}
return { devices, ingressVerified: false, capturePerformed: false, mutationsPerformed: false };
}
function endpoint(transport, host) {
return transport.configuration(['--source', host, '--transport', 'ssh-loopback', '--output',
'tmp/netflow-endpoint-discovery', '--owned-tunnel-attestation', attestation]).endpoint;
}
function bool(value) {
if (value === true || value === 'true') return true;
if (value === false || value === 'false') return false;
return deny();
}
function single(rows) {
if (rows.length !== 1) deny();
return rows[0];
}
function observedFlag(value) {
if ([true, 'true', 'yes'].includes(value)) return true;
if ([false, 'false', 'no'].includes(value)) return false;
return null;
}
export async function discoverPinnedNetFlowPolicy(read) {
const exporters = [];
for (const host of Object.keys(ports)) {
const resource = single(await read(host, policyRoutes[0]));
if (typeof resource.version !== 'string' || !/^\d+\.\d+(?:\.\d+)?(?: \([a-z-]+\))?$/.test(resource.version)) deny();
const ip = single(await read(host, policyRoutes[1]));
const tracking = single(await read(host, policyRoutes[2]));
const firewall = {};
const vetoes = [];
const ipForward = observedFlag(ip['ip-forward']);
const allowFastPath = observedFlag(ip['allow-fast-path']);
const conntrack = tracking.enabled === 'auto' ? 'auto' : observedFlag(tracking.enabled);
if (ipForward !== true) vetoes.push('ip-forward-not-confirmed');
if (allowFastPath !== false) vetoes.push('fast-path-not-confirmed-disabled');
if (conntrack !== true) vetoes.push('conntrack-not-explicitly-enabled');
for (const path of policyRoutes.slice(3, 7)) {
const rows = await read(host, path);
const enabled = rows.filter(row => observedFlag(row.disabled) !== true);
firewall[path.split('/').at(-1)] = {
total: rows.length,
notExplicitlyDisabled: enabled.length,
fastTrack: enabled.filter(row => row.action === 'fasttrack-connection').length,
};
}
const routingRules = await read(host, policyRoutes[7]);
const activeRoutingRules = routingRules.filter(row => observedFlag(row.disabled) !== true).length;
if (activeRoutingRules) vetoes.push('routing-policy-present');
const vrfs = await read(host, policyRoutes[8]);
const mainVrfOnly = vrfs.length === 1 && vrfs[0].name === 'main' && vrfs[0].interfaces === 'all' &&
observedFlag(vrfs[0].disabled) !== true;
if (!mainVrfOnly) vetoes.push('main-vrf-only-not-confirmed');
exporters.push({ host, version: resource.version, ipForward, allowFastPath, conntrack, firewall,
activeRoutingRules, vrfCount: vrfs.length, mainVrfOnly, vetoes });
}
return {
status: exporters.some(item => item.vetoes.length) ? 'pre-topology-veto' : 'no-global-veto-observed',
exporters, routeSelectionVerified: false, tuplePolicyVerified: false,
ingressVerified: false, capturePerformed: false, mutationsPerformed: false,
};
}
function target(row) {
if (bool(row.disabled) || row.version !== 'ipfix') return null;
const destination = row['dst-address'];
const portText = String(row.port);
if (!isIPv4(destination) || !/^[1-9][0-9]{0,4}$/.test(portText) || Number(portText) > 65535) deny();
const source = row['src-address'];
if (source !== undefined && source !== '' && !isIPv4(source)) deny();
return { destination, port: Number(portText), configuredSource: source && source !== '0.0.0.0' ? source : null };
}
export async function discoverPinnedNetFlowEndpoints(read) {
const exporters = [];
for (const host of Object.keys(ports)) {
const resource = single(await read(host, routes[0]));
if (typeof resource.version !== 'string' || !/^\d+\.\d+(?:\.\d+)?(?: \([a-z-]+\))?$/.test(resource.version)) deny();
const enabled = bool(single(await read(host, routes[1])).enabled);
const rows = await read(host, routes[2]);
if (rows.length > 64) deny();
const configuredTargets = rows.map(target).filter(Boolean);
exporters.push({ host, version: resource.version, enabled, configuredTargets });
}
const [first, second] = exporters;
const sameEndpoint = (a, b) => a.destination === b.destination && a.port === b.port;
const common = first.enabled && second.enabled
? first.configuredTargets.filter(a => second.configuredTargets.some(b => sameEndpoint(a, b)))
: [];
const unique = common.length === 1 && second.configuredTargets.filter(b => sameEndpoint(common[0], b)).length === 1;
return {
status: common.length === 0 ? 'no-common-active-ipfix-target' : unique ? 'configured-endpoint' : 'ambiguous-endpoint',
configuredEndpoint: unique ? { destination: common[0].destination, port: common[0].port } : null,
exporters,
ingressVerified: false,
capturePerformed: false,
mutationsPerformed: false,
};
}
async function main() {
const mode = process.argv.slice(2);
if (mode.length !== 1) deny();
const policy = mode[0] === '--check-policy' || mode[0] === '--read-pinned-policy';
const counters = mode[0] === '--check-counters' || mode[0] === '--read-pinned-counters';
const transport = counters ? await loadTransport(['/interface'], 4, 131072) : await loadPinnedNetFlowTransport(policy);
if (mode[0] === '--check-only' || mode[0] === '--check-policy') {
console.log(JSON.stringify({ transportSha256: reviewedTransportHash, loopbackPorts: ports, readPaths: transport.PATHS, networkUsed: false }));
return;
}
if (mode[0] === '--check-counters') {
const detail = await loadTransport(['/interface/*A'], 4, 16384);
if (detail.PATHS[0] !== '/interface/*A') deny();
console.log(JSON.stringify({ transportSha256: reviewedTransportHash, loopbackPorts: ports,
readPaths: ['/interface', '/interface/<observed-native-id>'], networkUsed: false }));
return;
}
if (!['--read-pinned', '--read-pinned-policy', '--read-pinned-counters'].includes(mode[0]) || process.stdin.isTTY) deny();
const chunks = [];
let length = 0;
for await (const chunk of process.stdin) {
length += chunk.length;
if (length > 4096) deny();
chunks.push(chunk);
}
const input = JSON.parse(new TextDecoder('utf-8', { fatal: true }).decode(Buffer.concat(chunks)));
if (!input || input.attestation !== attestation ||
Object.keys(input).length !== 2 || !input.passwords || Object.keys(input.passwords).length !== 2) deny();
const passwords = Object.keys(ports).map(host => input.passwords[host]);
if (passwords.some(password => typeof password !== 'string' || !/^[^\u0000-\u001f\u007f]{1,512}$/u.test(password))) deny();
const authorizations = passwords.map(password => 'Basic ' + Buffer.from('admin:' + password).toString('base64'));
const secrets = [...passwords, ...authorizations, ...authorizations.map(value => value.slice(6))].map(value => Buffer.from(value));
const budget = { requests: 0, bytes: 0, rows: 0, deadline: Date.now() + 30000 };
const discovery = counters ? discoverPinnedInterfaceCounterShape : policy ? discoverPinnedNetFlowPolicy : discoverPinnedNetFlowEndpoints;
const report = await discovery(async (host, route) => {
const authorization = authorizations[Object.keys(ports).indexOf(host)];
let selected = transport;
if (counters && route !== '/interface') {
if (!route.startsWith('/interface/') || !nativeId(route.slice('/interface/'.length))) deny();
selected = await loadTransport([route], 4, 16384, 524288);
}
const response = await selected.readResponse(endpoint(selected, host), route, authorization, budget);
if (secrets.some(secret => response.body.includes(secret))) deny();
return response.rows;
});
const output = JSON.stringify(report);
if (secrets.some(secret => Buffer.from(output).includes(secret))) deny();
console.log(output);
}
if (process.argv[1] && pathToFileURL(process.argv[1]).href === import.meta.url) {
main().catch(() => {
console.error('NetFlow discovery refused');
process.exitCode = 1;
});
}
@@ -0,0 +1,102 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { discoverPinnedInterfaceCounterShape, discoverPinnedNetFlowEndpoints, discoverPinnedNetFlowPolicy, loadPinnedNetFlowTransport } from './chr-netflow-endpoint-discovery.mjs';
const hosts = ['194.226.121.150', '104.165.244.7'];
const routes = ['/system/resource', '/ip/traffic-flow', '/ip/traffic-flow/target'];
const target = { disabled: 'false', version: 'ipfix', 'dst-address': '192.0.2.10', port: '2055', 'src-address': '0.0.0.0' };
function fixture(targets, enabled = ['true', 'true']) {
const calls = [];
const read = async (host, path) => {
calls.push([host, path]);
if (path === routes[0]) return [{ version: '7.24.5 (stable)' }];
if (path === routes[1]) return [{ enabled: enabled[hosts.indexOf(host)] }];
assert.equal(path, routes[2]);
return targets[hosts.indexOf(host)];
};
return { read, calls };
}
test('loads the reviewed source with the exact read-only path and relay fence', async () => {
const transport = await loadPinnedNetFlowTransport();
assert.deepEqual(transport.PATHS, routes);
assert.throws(() => transport.readResponse('http://127.0.0.1:19082', '/ip/address', '', {}), /request-budget/);
assert.throws(() => transport.readResponse('http://127.0.0.1:19083', routes[0], '', {}), /non-whitelisted-endpoint/);
});
test('reports only a common configured endpoint, with no ingress or capture claim', async () => {
const { read, calls } = fixture([[target], [{ ...target, 'src-address': '192.0.2.20' }]]);
const report = await discoverPinnedNetFlowEndpoints(read);
assert.deepEqual(calls, hosts.flatMap(host => routes.map(path => [host, path])));
assert.equal(report.status, 'configured-endpoint');
assert.deepEqual(report.configuredEndpoint, { destination: '192.0.2.10', port: 2055 });
assert.deepEqual(report.exporters.map(item => item.configuredTargets[0].configuredSource), [null, '192.0.2.20']);
assert.equal(report.ingressVerified, false);
assert.equal(report.capturePerformed, false);
assert.equal(report.mutationsPerformed, false);
});
test('returns successful absence for missing or disabled baseline', async () => {
for (const input of [
fixture([[], [target]]),
fixture([[target], [target]], [true, false]),
fixture([[target], [{ ...target, disabled: true }]]),
fixture([[target], [{ ...target, version: '9' }]]),
]) {
const result = await discoverPinnedNetFlowEndpoints(input.read);
assert.equal(result.status, 'no-common-active-ipfix-target');
assert.equal(result.configuredEndpoint, null);
}
});
test('refuses malformed relevant rows and withholds ambiguous common endpoints', async () => {
const result = await discoverPinnedNetFlowEndpoints(fixture([[target], [target, target]]).read);
assert.equal(result.status, 'ambiguous-endpoint');
assert.equal(result.configuredEndpoint, null);
await assert.rejects(discoverPinnedNetFlowEndpoints(fixture([[target], [{ ...target, port: '65536' }]]).read), /^Error: NetFlow discovery refused$/);
});
test('reports preliminary global vetoes and counts with no claim on tuple policy or routes', async () => {
const transport = await loadPinnedNetFlowTransport(true);
const calls = [];
const report = await discoverPinnedNetFlowPolicy(async (host, path) => {
calls.push([host, path]);
return ({
'/system/resource': [{ version: '7.24.5 (stable)' }],
'/ip/settings': [{ 'ip-forward': 'yes', 'allow-fast-path': host === hosts[0] ? 'yes' : 'no' }],
'/ip/firewall/connection/tracking': [{ enabled: host === hosts[0] ? 'auto' : 'yes' }],
'/ip/firewall/filter': [{ action: 'fasttrack-connection' }, { action: 'drop', disabled: 'true' }],
'/ip/firewall/raw': [], '/ip/firewall/mangle': [], '/ip/firewall/nat': [],
'/routing/rule': host === hosts[0] ? [{}] : [],
'/ip/vrf': [{ name: 'main', interfaces: 'all' }],
})[path];
});
assert.deepEqual(calls, hosts.flatMap(host => transport.PATHS.map(path => [host, path])));
assert.equal(report.status, 'pre-topology-veto');
assert.deepEqual(report.exporters[0].vetoes, ['fast-path-not-confirmed-disabled', 'conntrack-not-explicitly-enabled', 'routing-policy-present']);
assert.deepEqual(report.exporters[1].vetoes, []);
assert.deepEqual(report.exporters[0].firewall.filter, { total: 2, notExplicitlyDisabled: 1, fastTrack: 1 });
assert.equal(report.routeSelectionVerified, false);
assert.equal(report.tuplePolicyVerified, false);
assert.equal(report.ingressVerified, false);
});
test('counter shape reads exact selected native IDs and emits no identities or counter values', async () => {
const ether = { '.id': '*A', name: 'private-ether', type: 'ether', running: 'true', disabled: 'false' };
const wg = { ...ether, '.id': '*B', name: 'private-wg', type: 'wg' };
const calls = [];
const report = await discoverPinnedInterfaceCounterShape(async (host, route) => {
calls.push([host, route]);
if (route === '/interface') return host === hosts[0] ? [ether, wg] : [ether];
return [{ ...(host === hosts[0] ? wg : ether), ifindex: '61455', 'rx-packet': 71922, 'fp-rx-byte': null }];
});
assert.deepEqual(calls, [[hosts[0], '/interface'], [hosts[0], '/interface/*B'], [hosts[1], '/interface'], [hosts[1], '/interface/*A']]);
assert.deepEqual(report.devices[0].fields.ifindex, { present: true, type: 'string' });
assert.deepEqual(report.devices[0].fields['rx-packet'], { present: true, type: 'number' });
assert.deepEqual(report.devices[0].fields['fp-rx-byte'], { present: true, type: 'null' });
assert.deepEqual(report.devices[0].fields['tx-packet'], { present: false, type: 'undefined' });
assert.doesNotMatch(JSON.stringify(report), /private|61455|71922|104\.165|194\.226|\*[AB]/);
await assert.rejects(discoverPinnedInterfaceCounterShape(async (_host, route) =>
route === '/interface' ? [ether] : [{ ...ether, '.id': '*C' }]), /discovery refused/);
});
@@ -0,0 +1,86 @@
import assert from 'node:assert/strict';
import { createHash } from 'node:crypto';
import { lstat, readFile } from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
import { decodeBody, SOURCES } from './chr-recursive-observation.mjs';
const reviewedReportSha256 = 'fccf5171c8e50730a3e7c1fcceeeb77f1e946589ac4a56c6e04668e0ff4663f9';
const sha256 = bytes => createHash('sha256').update(bytes).digest('hex');
const opaqueId = value => typeof value === 'string' && /^\*[a-f0-9]+$/i.test(value);
const decimalIndex = value => typeof value === 'string' && /^[1-9][0-9]{0,15}$/.test(value);
export function inspectDirectLinks(routes, nexthops, interfaces, tables) {
function keyed(rows, field, valid) {
const index = new Map();
for (const row of rows) {
const key = row[field];
assert(valid(key) && !index.has(key), 'Missing or ambiguous ' + field);
index.set(key, row);
}
return index;
}
const byNexthopId = keyed(nexthops, '.id', opaqueId);
const byInterfaceIndex = keyed(interfaces, 'ifindex', decimalIndex);
const byTableName = keyed(tables, 'name', value => typeof value === 'string' && value !== '' && !/^[0-9]+$/.test(value));
let opaqueReferences = 0, namedInterfaceCorroborations = 0, interfaceIdSubstitutionRefusals = 0;
for (const route of routes) {
assert(byTableName.has(route['routing-table']), 'Missing named routing table');
if (!Object.hasOwn(route, 'nexthop-id')) continue;
const nexthop = byNexthopId.get(route['nexthop-id']);
assert(nexthop, 'No exact opaque nexthop match');
opaqueReferences++;
const gateway = route['immediate-gw'];
const name = typeof gateway === 'string' ? /%([^%@]+)$/.exec(gateway)?.[1] : undefined;
if (!name) continue;
const index = nexthop['immediate-gw.interface-idx'];
const observed = byInterfaceIndex.get(index);
assert(observed?.name === name && interfaces.filter(row => row.name === name).length === 1,
'No exact named interface corroboration for ifindex');
namedInterfaceCorroborations++;
if (!interfaces.some(row => row.id === index && row.name === name)) interfaceIdSubstitutionRefusals++;
}
assert(opaqueReferences && namedInterfaceCorroborations && interfaceIdSubstitutionRefusals, 'Capture lacks discriminating direct links');
return { opaqueReferences, namedInterfaceCorroborations, interfaceIdSubstitutionRefusals,
tableNames: byTableName.size, recursiveLinkRows: nexthops.filter(row => Object.keys(row).some(key => key === 'links' || key.startsWith('links.'))).length };
}
async function readPinned(directory, filename, digest) {
const file = path.join(directory, filename), stat = await lstat(file);
assert(stat.isFile() && !stat.isSymbolicLink() && stat.size <= 1048576, 'Unavailable bounded raw file');
const bytes = await readFile(file);
assert(bytes.length === stat.size && sha256(bytes) === digest, 'Raw capture hash changed');
return bytes;
}
export async function checkReviewedDirectLinks(directory) {
const report = JSON.parse(await readPinned(directory, 'report.json', reviewedReportSha256));
const sources = [];
for (const source of report.sources) {
assert(source.complete && source.versionBracketEqual && SOURCES[source.source]?.serverId === source.serverId);
const read = async (index, resource) => {
const record = source.records[index];
assert(record.resource === resource && record.status === 200 && record.version === '7.24.5 (stable)');
assert(record.file === `server${source.serverId}-${String(index + 1).padStart(2, '0')}-${resource.slice(1).replaceAll('/', '-')}.json`);
return decodeBody(await readPinned(directory, record.file, record.rawSha256)).rows;
};
const routes = await read(1, '/routing/route'), nexthops = await read(2, '/routing/nexthop');
assert.deepEqual(await read(7, '/routing/route'), routes);
assert.deepEqual(await read(6, '/routing/nexthop'), nexthops);
assert.equal(source.records[1].rawSha256, source.records[7].rawSha256);
assert.equal(source.records[2].rawSha256, source.records[6].rawSha256);
const interfaces = await read(3, '/interface'), tables = await read(4, '/routing/table');
sources.push({ serverId: source.serverId, ...inspectDirectLinks(routes, nexthops, interfaces, tables) });
}
return { reviewedReportSha256, sources, recursiveIdentity: 'unknown', semanticProof: false };
}
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
try {
assert(process.argv.length === 3, 'Supply the reviewed offline capture directory');
process.stdout.write(JSON.stringify(await checkReviewedDirectLinks(process.argv[2])) + '\n');
} catch {
process.stderr.write('Reviewed direct-link corroboration refused.\n');
process.exitCode = 1;
}
}
@@ -0,0 +1,17 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { inspectDirectLinks } from './chr-recursive-direct-links.mjs';
test('direct evidence requires exact opaque reference and corroborated ifindex namespace', () => {
const route = { 'routing-table': 'main', 'nexthop-id': '*A', 'immediate-gw': '192.0.2.1%ether1' };
const nexthop = { '.id': '*A', 'immediate-gw.interface-idx': '4' };
const iface = { id: '9', ifindex: '4', name: 'ether1' };
const table = { '.id': '*0', name: 'main' };
assert.deepEqual(inspectDirectLinks([route], [nexthop], [iface], [table]), {
opaqueReferences: 1, namedInterfaceCorroborations: 1, interfaceIdSubstitutionRefusals: 1, tableNames: 1, recursiveLinkRows: 0,
});
assert.throws(() => inspectDirectLinks([{ ...route, 'nexthop-id': '*a' }], [nexthop], [iface], [table]), /exact opaque/);
assert.throws(() => inspectDirectLinks([{ ...route, 'routing-table': '0' }], [nexthop], [iface], [table]), /named routing table/);
assert.throws(() => inspectDirectLinks([route], [nexthop], [{ ...iface, id: '4', ifindex: '9' }], [table]), /corroboration/);
assert.throws(() => inspectDirectLinks([route], [nexthop], [iface, { ...iface, id: '3' }], [table]), /ambiguous ifindex/);
});
@@ -0,0 +1,177 @@
import { request as httpRequest } from 'node:http';
import { request as httpsRequest } from 'node:https';
import { createHash } from 'node:crypto';
import { mkdir, realpath, lstat, writeFile } from 'node:fs/promises';
import path from 'node:path';
import { fileURLToPath } from 'node:url';
const ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../..');
export const SOURCES = Object.freeze({
'194.226.121.150': Object.freeze({ serverId: 1, port: 19081 }),
'104.165.244.7': Object.freeze({ serverId: 2, port: 19082 }),
});
export const PATHS = Object.freeze([
'/system/resource', '/routing/route', '/routing/nexthop', '/interface',
'/routing/table', '/ip/vrf', '/routing/nexthop', '/routing/route', '/system/resource',
]);
const LIMITS = Object.freeze({ response: 1048576, bytes: 8388608, requests: 256, rows: 1000, totalRows: 10000, duration: 120000 });
const sha = value => createHash('sha256').update(value).digest('hex');
const deny = code => { throw new Error(code); };
export function configuration(args) {
const options = {};
for (let i = 0; i < args.length; i += 2) {
if (!['--source', '--transport', '--output', '--owned-tunnel-attestation'].includes(args[i]) || !args[i + 1] || options[args[i]] !== undefined) deny('invalid-options');
options[args[i]] = args[i + 1];
}
const source = options['--source'], transport = options['--transport'];
if (!Object.hasOwn(SOURCES, source) || !['https', 'ssh-loopback'].includes(transport) || !options['--output']) deny('invalid-source-or-transport');
const attestation = options['--owned-tunnel-attestation'];
if (transport === 'ssh-loopback' && attestation !== 'verified-known-host-161.104.46.222-owned-forward') deny('owned-tunnel-attestation-required');
if (transport === 'https' && attestation !== undefined) deny('irrelevant-tunnel-attestation');
return { source, serverId: SOURCES[source].serverId, transport, output: options['--output'],
endpoint: transport === 'https' ? `https://${source}:443` : `http://127.0.0.1:${SOURCES[source].port}`,
tunnelAttestation: attestation ?? null };
}
export async function privateDirectory(output, root = ROOT) {
// Reject symlinks/junctions at every existing component, including tmp itself.
const base = path.resolve(root, 'tmp'), target = path.resolve(root, output);
const relative = path.relative(base, target);
if (!relative || relative.startsWith('..') || path.isAbsolute(relative)) deny('output-must-be-inside-tmp');
await mkdir(base, { recursive: true, mode: 0o700 });
if ((await lstat(base)).isSymbolicLink() || await realpath(base) !== base) deny('output-symlink-refused');
let current = base;
for (const part of relative.split(path.sep)) {
current = path.join(current, part);
try { await mkdir(current, { mode: 0o700 }); } catch (error) { if (error.code !== 'EEXIST') throw error; }
const stat = await lstat(current);
if (!stat.isDirectory() || stat.isSymbolicLink() || await realpath(current) !== current) deny('output-symlink-refused');
}
// Exclusive claim prevents mixing runs and refuses existing captures.
await writeFile(path.join(target, 'capture.claim'), 'chr-recursive-observation-v1\n', { flag: 'wx', mode: 0o600 });
return target;
}
export function decodeBody(body) {
let text, payload;
try { text = new TextDecoder('utf-8', { fatal: true }).decode(body); payload = JSON.parse(text); } catch { deny('invalid-utf8-or-json'); }
const rows = Array.isArray(payload) ? payload : payload && typeof payload === 'object' ? [payload] : deny('invalid-row-shape');
if (rows.length > LIMITS.rows || rows.some(row => !row || typeof row !== 'object' || Array.isArray(row))) deny('row-limit-or-shape');
return { rows };
}
export function readResponse(endpoint, resource, authorization, budget) {
const allowed = Object.entries(SOURCES).flatMap(([host, item]) => [`https://${host}:443`, `http://127.0.0.1:${item.port}`]);
if (!allowed.includes(endpoint)) deny('non-whitelisted-endpoint');
if (!PATHS.includes(resource) || budget.bytes >= LIMITS.bytes || budget.rows >= LIMITS.totalRows || ++budget.requests > LIMITS.requests || Date.now() >= budget.deadline) deny('request-budget');
const url = new URL('/rest' + resource, endpoint);
return new Promise((resolve, reject) => {
const fail = (code, status) => reject(Object.assign(new Error(code), Number.isInteger(status) ? { httpStatus: status } : {}));
const req = (url.protocol === 'https:' ? httpsRequest : httpRequest)(url, {
method: 'GET', agent: false, rejectUnauthorized: true, maxHeaderSize: 16384,
headers: { Authorization: authorization, Accept: 'application/json', 'Accept-Encoding': 'identity' },
}, res => {
const chunks = []; let size = 0;
res.on('data', chunk => {
size += chunk.length; budget.bytes += chunk.length;
if (size > LIMITS.response || budget.bytes > LIMITS.bytes) { req.destroy(); fail('byte-limit'); return; }
chunks.push(chunk);
});
res.on('aborted', () => fail('response-aborted'));
res.on('error', () => fail('response-failed'));
res.on('end', () => {
const body = Buffer.concat(chunks);
// Raw failed HTTP responses are excluded: error bodies can reflect secrets.
if (res.statusCode !== 200) return fail('http-status-refused', res.statusCode);
const type = res.headers['content-type'];
if (typeof type !== 'string' || !/^application\/json(?:\s*;\s*charset=utf-8)?$/i.test(type) || res.headers['content-encoding'] && res.headers['content-encoding'] !== 'identity') return fail('response-encoding-refused');
try {
const { rows } = decodeBody(body);
budget.rows += rows.length;
if (budget.rows > LIMITS.totalRows || Date.now() >= budget.deadline) return fail('response-budget');
resolve({ body, rows, status: res.statusCode, contentType: type });
} catch { fail('invalid-response'); }
});
});
const timer = setTimeout(() => { req.destroy(); fail('absolute-deadline'); }, Math.max(1, budget.deadline - Date.now()));
req.on('error', () => fail('transport-failed'));
req.on('close', () => clearTimeout(timer));
req.end();
});
}
export async function acquire(config, credentials) {
if (typeof credentials?.username !== 'string' || !credentials.username || credentials.username.length > 256 || /[:\r\n]/.test(credentials.username) || typeof credentials.password !== 'string' || !credentials.password || credentials.password.length > 4096 || /[\r\n]/.test(credentials.password)) deny('invalid-private-credentials');
// Validate caller configuration again; exported API cannot supply a custom host.
const fixed = configuration(['--source', config.source, '--transport', config.transport, '--output', config.output,
...(config.tunnelAttestation ? ['--owned-tunnel-attestation', config.tunnelAttestation] : [])]);
const directory = await privateDirectory(fixed.output);
const budget = { requests: 0, bytes: 0, rows: 0, deadline: Date.now() + LIMITS.duration };
const authorization = 'Basic ' + Buffer.from(credentials.username + ':' + credentials.password).toString('base64');
const report = { format: 'chr-recursive-raw-acquisition-v1', source: fixed.source, serverId: fixed.serverId,
transport: fixed.transport, tunnelAttestation: fixed.tunnelAttestation, startedAt: new Date().toISOString(),
semanticProof: false, complete: false, snapshotBracket: null, records: [], limits: LIMITS };
let version, attemptedResource;
try {
for (const [index, resource] of PATHS.entries()) {
attemptedResource = resource;
const startedAt = new Date().toISOString();
const response = await readResponse(fixed.endpoint, resource, authorization, budget);
// Never persist a body reflecting the private credential or Basic value.
if ([credentials.password, authorization].some(secret => response.body.includes(Buffer.from(secret)))) deny('credential-reflection-refused');
if (resource === '/system/resource') {
const current = response.rows.length === 1 && response.rows[0].version;
if (typeof current !== 'string' || !/^7\.2[0-6](?:\.\d{1,3})?(?: \((?:stable|long-term)\))?$/.test(current) || version && version !== current) deny('unsupported-or-drifting-version');
version = current;
}
const file = `${String(index + 1).padStart(2, '0')}-${resource.slice(1).replaceAll('/', '-')}.json`;
await writeFile(path.join(directory, file), response.body, { flag: 'wx', mode: 0o600 });
report.records.push({ file, resource, startedAt, finishedAt: new Date().toISOString(), version,
status: response.status, contentType: response.contentType, bytes: response.body.length, rows: response.rows.length, rawSha256: sha(response.body) });
}
const hashFor = resource => report.records.filter(record => record.resource === resource).map(record => record.rawSha256);
const [firstRoute, lastRoute] = hashFor('/routing/route');
const [firstNexthop, lastNexthop] = hashFor('/routing/nexthop');
report.snapshotBracket = {
routeByteEqual: firstRoute === lastRoute,
nexthopByteEqual: firstNexthop === lastNexthop,
};
report.complete = true;
} catch (error) {
report.failure = 'acquisition-incomplete';
report.failedResource = attemptedResource;
if (Number.isInteger(error.httpStatus) && error.httpStatus >= 100 && error.httpStatus <= 599) report.failedHttpStatus = error.httpStatus;
}
report.finishedAt = new Date().toISOString();
report.usage = { requests: budget.requests, bytes: budget.bytes, rows: budget.rows };
await writeFile(path.join(directory, 'report.json'), JSON.stringify(report, null, 2) + '\n', { flag: 'wx', mode: 0o600 });
return report;
}
async function privateStdin() {
// Piped stdin is private (no terminal echo). TTY raw mode hides all input.
const input = process.stdin, chunks = []; let size = 0;
if (input.isTTY) { process.stderr.write('Private credentials JSON (hidden), then Enter: '); input.setRawMode(true); }
try {
for await (const chunk of input) {
size += chunk.length;
if (size > 8192) deny('private-input-limit');
if (input.isTTY && chunk.includes(3)) deny('private-input-cancelled');
chunks.push(chunk);
if (input.isTTY && (chunk.includes(13) || chunk.includes(10))) break;
}
return JSON.parse(Buffer.concat(chunks).toString('utf8').trim());
} finally { if (input.isTTY) { input.setRawMode(false); process.stderr.write('\n'); } }
}
if (process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url)) {
try {
const config = configuration(process.argv.slice(2));
const report = await acquire(config, await privateStdin());
// Stdout contains metadata only; no native bodies, credentials or errors.
process.stdout.write(JSON.stringify({ complete: report.complete, snapshotBracket: report.snapshotBracket,
semanticProof: false, source: report.source, records: report.records.length }) + '\n');
if (!report.complete) process.exitCode = 1;
} catch { process.stderr.write('Read-only acquisition refused or incomplete.\n'); process.exitCode = 1; }
}
@@ -0,0 +1,96 @@
import assert from 'node:assert/strict';
import { test } from 'node:test';
import { createServer } from 'node:http';
import { randomUUID, createHash } from 'node:crypto';
import { readFile, readdir } from 'node:fs/promises';
import { fileURLToPath } from 'node:url';
import { configuration, decodeBody, readResponse, acquire, privateDirectory, PATHS } from './chr-recursive-observation.mjs';
test('source/transport allowlist refuses arbitrary hosts, plaintext external, missing tunnel attestation and option injection', () => {
const args = ['--source', '194.226.121.150', '--transport', 'https', '--output', 'tmp/acquisition-test'];
assert.equal(configuration(args).endpoint, 'https://194.226.121.150:443');
for (const changes of [args.map(x => x === '194.226.121.150' ? 'example.org' : x), args.map(x => x === 'https' ? 'http' : x),
args.map(x => x === 'https' ? 'ssh-loopback' : x), [...args, '--password', 'forbidden'], [...args, '--source', '104.165.244.7']]) assert.throws(() => configuration(changes));
const tunnel = configuration([...args.map(x => x === 'https' ? 'ssh-loopback' : x), '--owned-tunnel-attestation', 'verified-known-host-161.104.46.222-owned-forward']);
assert.equal(tunnel.endpoint, 'http://127.0.0.1:19081');
});
test('full own fixture saves nine exact immutable captures and metadata; reflected errors never enter reports', async () => {
const output = fileURLToPath(new URL('../../../tmp/chr-recursive-own-fixture-' + randomUUID(), import.meta.url)), seen = [];
let reflect = false, driftingResource;
const server = createServer((req, res) => {
seen.push(req.url); res.setHeader('Content-Type', 'application/json');
if (reflect) { res.statusCode = 401; res.end('{"message":"own-fixture-secret-only"}'); return; }
if (req.url === '/rest/system/resource') res.end('[{"version":"7.26"}]\n');
else if (req.url === driftingResource && seen.indexOf(req.url) !== seen.length - 1) res.end('[{".id":"*2"}]\n');
else res.end('[{".id":"*1"}]\n');
});
await new Promise((resolve, reject) => { server.once('error', reject); server.listen(19081, '127.0.0.1', resolve); });
const makeConfig = destination => configuration(['--source', '194.226.121.150', '--transport', 'ssh-loopback', '--output', destination,
'--owned-tunnel-attestation', 'verified-known-host-161.104.46.222-owned-forward']);
try {
const report = await acquire(makeConfig(output), { username: 'own-fixture', password: 'own-fixture-secret-only' });
assert.equal(report.complete, true); assert.equal(report.semanticProof, false);
assert.deepEqual(report.snapshotBracket, { routeByteEqual: true, nexthopByteEqual: true });
assert.deepEqual(seen, PATHS.map(resource => '/rest' + resource));
for (const record of report.records) {
const body = await readFile(output + '/' + record.file);
assert.equal(createHash('sha256').update(body).digest('hex'), record.rawSha256);
assert.equal(body.length, record.bytes); assert.equal(record.version, '7.26');
}
await assert.rejects(acquire(makeConfig(output), { username: 'own-fixture', password: 'own-fixture-secret-only' }));
assert.equal(seen.length, 9);
await assert.rejects(privateDirectory('../outside-tmp'));
for (const resource of ['route', 'nexthop']) {
seen.length = 0;
driftingResource = '/rest/routing/' + resource;
const driftOutput = output + '-' + resource;
const drift = await acquire(makeConfig(driftOutput), { username: 'own-fixture', password: 'own-fixture-secret-only' });
assert.equal(drift.complete, true);
assert.equal(drift.semanticProof, false);
assert.deepEqual(drift.snapshotBracket, { routeByteEqual: resource !== 'route', nexthopByteEqual: resource !== 'nexthop' });
assert.equal(drift.records.length, 9);
assert.deepEqual(JSON.parse(await readFile(driftOutput + '/report.json', 'utf8')).snapshotBracket, drift.snapshotBracket);
}
reflect = true;
const failedOutput = output + '-failed';
const failed = await acquire(makeConfig(failedOutput), { username: 'own-fixture', password: 'own-fixture-secret-only' });
assert.equal(failed.complete, false); assert.equal(failed.records.length, 0);
assert.equal(failed.snapshotBracket, null);
assert.equal(failed.failedResource, '/system/resource'); assert.equal(failed.failedHttpStatus, 401);
assert.deepEqual((await readdir(failedOutput)).sort(), ['capture.claim', 'report.json']);
assert(!String(await readFile(failedOutput + '/report.json')).includes('own-fixture-secret-only'));
} finally { await new Promise(resolve => server.close(resolve)); }
});
test('exact UTF8 and row bounds fail closed', () => {
assert.equal(decodeBody(Buffer.from('[{"version":"7.26"}]')).rows.length, 1);
for (const body of [Buffer.from([0xff]), Buffer.from('null'), Buffer.from('[[]]'), Buffer.from(JSON.stringify(Array.from({ length: 1001 }, () => ({}))))]) assert.throws(() => decodeBody(body));
});
test('own loopback fixture observes exact GET paths, refuses redirect/error/encoding/bytes, no retry or follow', async () => {
let mode = 'normal', requests = 0;
const server = createServer((req, res) => {
requests++;
assert.equal(req.method, 'GET'); assert.equal(req.url, '/rest/system/resource');
res.setHeader('Content-Type', 'application/json');
if (mode === 'redirect') { res.statusCode = 302; res.setHeader('Location', 'http://example.invalid'); res.end('[]'); }
else if (mode === 'error') { res.statusCode = 401; res.end('{"error":"reflected-private-input"}'); }
else if (mode === 'encoding') { res.setHeader('Content-Encoding', 'gzip'); res.end('[]'); }
else if (mode === 'oversized') res.end(' '.repeat(1048577));
else res.end('[{"version":"7.26"}]\n');
});
await new Promise((resolve, reject) => { server.once('error', reject); server.listen(19081, '127.0.0.1', resolve); });
const budget = () => ({ requests: 0, bytes: 0, rows: 0, deadline: Date.now() + 5000 });
try {
const first = await readResponse('http://127.0.0.1:19081', PATHS[0], 'Basic own-fixture', budget());
assert.equal(first.body.toString(), '[{"version":"7.26"}]\n');
for (mode of ['redirect', 'error', 'encoding', 'oversized']) await assert.rejects(readResponse('http://127.0.0.1:19081', PATHS[0], 'Basic own-fixture', budget()));
assert.equal(requests, 5);
assert.throws(() => readResponse('http://194.226.121.150', PATHS[0], '', budget()));
assert.throws(() => readResponse('http://127.0.0.1:19081', '/system/reboot', '', budget()));
assert.throws(() => readResponse('http://127.0.0.1:19081', PATHS[0], '', { ...budget(), requests: 256 }));
assert.throws(() => readResponse('http://127.0.0.1:19081', PATHS[0], '', { ...budget(), deadline: 0 }));
assert.equal(requests, 5);
} finally { await new Promise(resolve => server.close(resolve)); }
});
@@ -0,0 +1,155 @@
/** Arithmetic preflight only. No Docker/host access, changes or workload admission.
* Roles are operator-supplied labels, not authenticated service identities.
* Even a fitting budget cannot prove runtime <=2GB or permission to change limits. */
const MiB = 1024 * 1024
const roles = new Set(['backend', 'postgres', 'frontend', 'portal', 'proxy', 'agent', 'other'])
const singleton = new Set(['backend', 'postgres', 'frontend', 'portal'])
const integer = value => Number.isSafeInteger(value) && value >= 0
const exact = (value, fields) => value !== null && typeof value === 'object' && !Array.isArray(value)
&& Object.keys(value).sort().join('|') === [...fields].sort().join('|')
const id = value => typeof value === 'string' && /^[a-f0-9]{64}$/.test(value)
export function evaluateInstallationMemoryBudget(input) {
if (!exact(input, ['hostMemTotalBytes', 'osReserveBytes', 'observedContainerIds', 'containers'])
|| !integer(input.hostMemTotalBytes) || !integer(input.osReserveBytes)
|| !Array.isArray(input.observedContainerIds) || !Array.isArray(input.containers)
|| input.observedContainerIds.length > 128 || input.containers.length > 128) throw Error('Invalid bounded installation inventory')
if (input.observedContainerIds.some(value => !id(value)) || new Set(input.observedContainerIds).size !== input.observedContainerIds.length) throw Error('Invalid observed container set')
const seen = new Set(), counts = new Map(), reasons = new Set()
let configuredBytes = 0
for (const row of input.containers) {
if (!exact(row, ['id', 'role', 'configuredLimitBytes']) || !id(row.id) || !roles.has(row.role)
|| !integer(row.configuredLimitBytes) || seen.has(row.id)) throw Error('Invalid container budget')
seen.add(row.id); counts.set(row.role, (counts.get(row.role) ?? 0) + 1)
if (row.configuredLimitBytes === 0) reasons.add('UNBOUNDED_CONTAINER')
if (row.configuredLimitBytes > input.hostMemTotalBytes) reasons.add('CONTAINER_LIMIT_EXCEEDS_HOST')
if (!Number.isSafeInteger(configuredBytes + row.configuredLimitBytes)) throw Error('Container budget overflow')
configuredBytes += row.configuredLimitBytes
}
if (seen.size !== input.observedContainerIds.length || input.observedContainerIds.some(value => !seen.has(value))) reasons.add('INCOMPLETE_HOST_CONTAINER_INVENTORY')
for (const role of ['backend', 'postgres', 'frontend']) if (!counts.has(role)) reasons.add('REQUIRED_SERVICE_MISSING')
for (const role of singleton) if ((counts.get(role) ?? 0) > 1) reasons.add('DUPLICATE_APPLICATION_STACK')
if (input.hostMemTotalBytes === 0 || input.hostMemTotalBytes > 2 * 1024 * MiB) reasons.add('HOST_OUTSIDE_2GIB_TARGET')
if (input.osReserveBytes < 512 * MiB) reasons.add('OS_RESERVE_BELOW_CANDIDATE_FLOOR')
if (!Number.isSafeInteger(configuredBytes + input.osReserveBytes)) throw Error('Installation budget overflow')
const plannedBytes = configuredBytes + input.osReserveBytes
if (plannedBytes > input.hostMemTotalBytes) reasons.add('INSTALLATION_BUDGET_EXCEEDS_HOST')
return Object.freeze({ scope: 'arithmetic-preflight-only', budgetFits: reasons.size === 0,
wholeInstallation2GbAccepted: false, runtimeAcceptanceClaimed: false, changesAuthorized: false,
configuredContainerBytes: configuredBytes, osReserveBytes: input.osReserveBytes,
plannedBytes, remainingBytes: input.hostMemTotalBytes - plannedBytes,
observedContainers: input.observedContainerIds.length, reasons: Object.freeze([...reasons].sort()) })
}
/** Project the whitelist-only read-only inventory. Never substitute idle RSS or
* Docker's reported host ceiling for HostConfig.Memory. Labels/freshness are
* not authenticated here; this result must not be used as mutation authority. */
export function evaluateObservedInstallationBudget(inventory, roleById, osReserveBytes = 512 * MiB) {
if (!inventory || inventory.readOnly !== true || inventory.containerSetStable !== true
|| inventory.environmentRead !== false || inventory.serviceChanges !== false || inventory.newFixtureStarted !== false
|| !Array.isArray(inventory.containers) || inventory.containers.length > 128
|| !roleById || typeof roleById !== 'object' || Array.isArray(roleById)) throw Error('Invalid read-only installation inventory')
const containers = inventory.containers.map(row => {
if (!row || !id(row.id) || row.running !== true || !integer(row.memoryLimitBytes)
|| !/^sha256:[a-f0-9]{64}$/.test(row.imageId) || !Number.isFinite(Date.parse(row.startedAt))) throw Error('Incomplete configured-limit observation')
const label = Object.getOwnPropertyDescriptor(roleById, row.id)
if (!label || typeof label.value !== 'string' || !roles.has(label.value)) throw Error('Explicit container role missing')
return { id: row.id, role: label.value, configuredLimitBytes: row.memoryLimitBytes }
})
const observedContainerIds = containers.map(row => row.id)
if (Object.keys(roleById).length !== observedContainerIds.length || Object.keys(roleById).some(key => !observedContainerIds.includes(key))) throw Error('Foreign or incomplete role inventory')
const budget = evaluateInstallationMemoryBudget({ hostMemTotalBytes: inventory.hostMemTotalBytes, osReserveBytes, observedContainerIds, containers })
return Object.freeze({ ...budget, configuredLimitsSource: 'observed-HostConfig.Memory', roleLabelsAuthenticated: false, inventoryFreshnessProven: false })
}
/** NEW pure raw-text bridge for measure-memory schema2/3. This is NOT the
* separate GET-only inventory contract consumed by evaluateObserved above.
* No file/container/socket access or mutation authority. Parse raw text only:
* precision lost by a caller's previous JSON.parse cannot be recovered. */
export function evaluateMeasuredInstallationBudget(rawText, roleById, osReserveBytes = 512 * MiB) {
if (typeof rawText !== 'string') throw Error('Raw measurement JSON text required; preparsed precision cannot be recovered')
if (Buffer.byteLength(rawText, 'utf8') > 32 * 1024 * 1024) throw Error('Measurement text bound exceeded')
// A narrow numeric lexical gate BEFORE JSON.parse, not a JSON parser. The
// Python producer emits only integer JSON numbers (PSI averages are strings).
// Reject decimal/exponent/unsafe integers even where JSON.parse would round
// a fractional lexeme into a seemingly safe integer. JSON.parse checks syntax.
const number = /-?(?:0|[1-9][0-9]*)(?:\.[0-9]+)?(?:[eE][+-]?[0-9]+)?/y
let quoted = false, escaped = false
for (let index = 0; index < rawText.length; index++) {
const char = rawText[index]
if (quoted) { if (escaped) escaped = false; else if (char === '\\') escaped = true; else if (char === '"') quoted = false; continue }
if (char === '"') { quoted = true; continue }
if (char === '-' || char >= '0' && char <= '9') {
number.lastIndex = index; const token = number.exec(rawText)?.[0]
if (!token || token === '-0' || !/^-?(?:0|[1-9][0-9]*)$/.test(token) || token.replace('-', '').length > 16) throw Error('Unsafe or noninteger measurement JSON number')
const value = BigInt(token)
if (value < -9007199254740991n || value > 9007199254740991n) throw Error('Unsafe measurement JSON integer')
index = number.lastIndex - 1
}
}
const data = JSON.parse(rawText)
if (!data || ![2, 3].includes(data.schemaVersion) || data.readOnlyDockerAndProc !== true || data.serviceChanges !== 0
|| data.summary?.acceptancePassed !== false || !Array.isArray(data.observations) || !data.observations.length || data.observations.length > 300
|| !roleById || typeof roleById !== 'object' || Array.isArray(roleById)) throw Error('Unsupported or incomplete memory measurement')
const nonnegativeSafe = value => Number.isSafeInteger(value) && value >= 0
const limit = value => value === 'max' || nonnegativeSafe(value) ? value : null
const swap = row => row.configuredMemorySwapStatus === 'available' && Number.isSafeInteger(row.configuredMemorySwapBytes) && row.configuredMemorySwapBytes >= -1 ? row.configuredMemorySwapBytes : null
const cpu = value => value && ['nr_periods', 'nr_throttled', 'throttled_usec'].every(key => nonnegativeSafe(value[key]))
? Object.fromEntries(['nr_periods', 'nr_throttled', 'throttled_usec'].map(key => [key, value[key]])) : null
const pressure = host => {
if (host.memoryPressure?.status !== 'available') return null
const result = {}
for (const key of ['some', 'full']) {
const value = host.memoryPressure.metrics?.[key]
if (!value || !nonnegativeSafe(value.totalMicros)) return null
for (const avg of ['avg10Percent', 'avg60Percent', 'avg300Percent']) {
if (typeof value[avg] !== 'string' || !/^[0-9]{1,3}\.[0-9]{2}$/.test(value[avg]) || Number(value[avg]) > 100) return null
}
result[key] = Object.fromEntries(['avg10Percent', 'avg60Percent', 'avg300Percent', 'totalMicros'].map(name => [name, value[name]]))
}
return result
}
let baseline, hostMemory, priorElapsed = -1, complete = data.schemaVersion === 3
const extendedObservations = []
for (const observation of data.observations) {
if (!observation?.host || !nonnegativeSafe(observation.elapsedMs) || observation.elapsedMs < priorElapsed || observation.elapsedMs > 360000
|| !nonnegativeSafe(observation.host.MemTotalBytes) || !observation.host.MemTotalBytes
|| !Array.isArray(observation.containers) || !observation.containers.length || observation.containers.length > 16) throw Error('Incomplete bounded measurement sample')
priorElapsed = observation.elapsedMs
hostMemory ??= observation.host.MemTotalBytes
if (hostMemory !== observation.host.MemTotalBytes) throw Error('Measurement host memory changed')
const ids = new Set(), identities = [], extended = []
for (const row of observation.containers) {
if (!row || !id(row.id) || ids.has(row.id) || !/^sha256:[a-f0-9]{64}$/.test(row.imageId) || (typeof row.startedAt !== 'string' || !Number.isFinite(Date.parse(row.startedAt)))
|| row.running !== true || row.runningAfter !== true || row.identityStableDuringSample !== true || row.oomKilled !== false || row.oomKilledAfter !== false
|| !nonnegativeSafe(row.restartCount) || !Number.isSafeInteger(row.pid) || row.pid < 1 || row.pid > 2147483647 || !nonnegativeSafe(row.configuredLimitBytes)) throw Error('Unavailable measurement container identity or configured cap')
ids.add(row.id); identities.push([row.id, row.imageId, row.startedAt, row.pid, row.restartCount, row.configuredLimitBytes, data.schemaVersion === 3 ? swap(row) : null])
if (data.schemaVersion === 3) {
const metrics = row.cgroupV2?.metrics, memoryMax = limit(metrics?.['memory.max']), swapMax = limit(metrics?.['memory.swap.max'])
const cpuThrottling = cpu(metrics?.['cpu.stat']), configuredMemorySwapBytes = swap(row)
const available = row.cgroupV2?.status === 'available' && row.resourceLimitsStableDuringSample === true
&& memoryMax !== null && swapMax !== null && cpuThrottling !== null && configuredMemorySwapBytes !== null
complete &&= available
extended.push({id: row.id, memoryMax, swapMax, configuredMemorySwapBytes, cpuThrottling, available})
} else extended.push({id: row.id, memoryMax: null, swapMax: null, configuredMemorySwapBytes: null, cpuThrottling: null, available: false})
}
identities.sort((a, b) => a[0].localeCompare(b[0]))
const encoded = JSON.stringify(identities)
baseline ??= encoded
if (encoded !== baseline) throw Error('Measurement container set, identity or memory cap changed')
const memoryPressure = data.schemaVersion === 3 ? pressure(observation.host) : null
complete &&= memoryPressure !== null
extendedObservations.push({elapsedMs: observation.elapsedMs, memoryPressure, containers: extended})
}
const captured = JSON.parse(baseline)
if (Object.keys(roleById).length !== captured.length || Object.keys(roleById).some(key => !captured.some(row => row[0] === key))) throw Error('Incomplete or foreign measured role inventory')
const containers = captured.map(([identifier, _image, _start, _pid, _restart, configuredLimitBytes]) => {
const descriptor = Object.getOwnPropertyDescriptor(roleById, identifier)
if (!descriptor || typeof descriptor.value !== 'string' || !roles.has(descriptor.value)) throw Error('Explicit measured container role missing')
return {id: identifier, role: descriptor.value, configuredLimitBytes}
})
const budget = evaluateInstallationMemoryBudget({hostMemTotalBytes: hostMemory, osReserveBytes, observedContainerIds: containers.map(row => row.id), containers})
return Object.freeze({...budget, measurementSchemaVersion: data.schemaVersion, measurementSamples: data.observations.length,
extendedMetricsComplete: complete, extendedObservations: Object.freeze(extendedObservations),
roleLabelsAuthenticated: false, inventoryFreshnessProven: false, hostContainerInventoryComplete: false, measurementSelectedContainersOnly: true, noSwapAccepted: false, sourceGenerationAuthenticated: false})
}
@@ -0,0 +1,46 @@
import assert from 'node:assert/strict'
import { test } from 'node:test'
import { evaluateInstallationMemoryBudget as evaluate } from './installation-memory-budget.mjs'
const MiB = 1024 * 1024
function fixture() {
const containers = [['backend',512],['postgres',384],['frontend',160],['portal',192]].map(([role, limit], i) => ({ id: String(i+1).repeat(64), role, configuredLimitBytes: limit * MiB }))
return { hostMemTotalBytes: 2021212 * 1024, osReserveBytes: 512 * MiB, observedContainerIds: containers.map(row => row.id), containers }
}
test('complete four-service candidate fits arithmetic, never runtime acceptance or change permission', () => {
const input = fixture(), before = structuredClone(input), result = evaluate(input)
assert.equal(result.budgetFits,true); assert.equal(result.plannedBytes,1760 * MiB)
assert.equal(result.wholeInstallation2GbAccepted,false); assert.equal(result.runtimeAcceptanceClaimed,false); assert.equal(result.changesAuthorized,false)
assert(Object.isFrozen(result)); assert(Object.isFrozen(result.reasons)); assert.deepEqual(input,before)
})
test('coexisting baseline and owned app/PG cannot represent one installation', () => {
const input = fixture()
for (const [i,role,limit] of [[5,'backend',320],[6,'postgres',192]]) { input.containers.push({id:String(i).repeat(64),role,configuredLimitBytes:limit*MiB}); input.observedContainerIds.push(String(i).repeat(64)) }
const result = evaluate(input)
assert.equal(result.budgetFits,false); assert(result.reasons.includes('DUPLICATE_APPLICATION_STACK')); assert(result.reasons.includes('INSTALLATION_BUDGET_EXCEEDS_HOST'))
})
test('unknown host containers, missing and foreign inventory entries refuse', () => {
for (const mode of ['omitted','foreign','missing-service']) {
const input = fixture()
if (mode === 'omitted') input.observedContainerIds.push('a'.repeat(64))
if (mode === 'foreign') input.containers[0].id='b'.repeat(64)
if (mode === 'missing-service') { input.containers.splice(0,1); input.observedContainerIds.splice(0,1) }
const result = evaluate(input); assert.equal(result.budgetFits,false)
assert(result.reasons.includes(mode === 'missing-service'?'REQUIRED_SERVICE_MISSING':'INCOMPLETE_HOST_CONTAINER_INVENTORY'))
}
})
test('unlimited cap is distinct from observed idle RSS or Docker host limit display', () => {
const input = fixture(); input.containers[0].configuredLimitBytes=0
assert(evaluate(input).reasons.includes('UNBOUNDED_CONTAINER'))
input.containers[0].configuredLimitBytes=3*1024*MiB
assert(evaluate(input).reasons.includes('CONTAINER_LIMIT_EXCEEDS_HOST'))
})
test('extra service, reserve and physical host all count toward the same budget', () => {
const input = fixture(); input.containers.push({id:'a'.repeat(64),role:'other',configuredLimitBytes:256*MiB}); input.observedContainerIds.push('a'.repeat(64))
assert(evaluate(input).reasons.includes('INSTALLATION_BUDGET_EXCEEDS_HOST'))
const low = fixture(); low.osReserveBytes=128*MiB; assert(evaluate(low).reasons.includes('OS_RESERVE_BELOW_CANDIDATE_FLOOR'))
const large = fixture(); large.hostMemTotalBytes=4*1024*MiB; assert(evaluate(large).reasons.includes('HOST_OUTSIDE_2GIB_TARGET'))
})
test('malformed, duplicate, fractional, oversized or unsafe inventory rejects without I/O', () => {
const mutations = [input=>{input.inject='unknown'},input=>{input.containers[0].role='invented'},input=>{input.containers[0].configuredLimitBytes=-1},input=>{input.containers[0].configuredLimitBytes=0.5},input=>{input.containers[0].id='../foreign'},input=>{input.containers[1].id=input.containers[0].id},input=>{input.observedContainerIds.push(input.observedContainerIds[0])},input=>{input.containers[0].configuredLimitBytes=Number.MAX_SAFE_INTEGER},input=>{input.observedContainerIds=Array(129).fill('a'.repeat(64))}]
for (const mutate of mutations) { const input=fixture();mutate(input);assert.throws(()=>evaluate(input)) }
})
@@ -0,0 +1,25 @@
import assert from 'node:assert/strict'
import {test} from 'node:test'
import {evaluateMeasuredInstallationBudget as evaluate,evaluateInstallationMemoryBudget as legacy} from './installation-memory-budget.mjs'
const MiB=1024*1024
function fixture(version=3){
const roles=['backend','postgres','frontend','portal'],caps=[512,384,160,192]
const containers=roles.map((role,i)=>({id:String(i+1).repeat(64),imageId:'sha256:'+'a'.repeat(64),startedAt:'2026-11-02T00:00:00Z',pid:i+1,restartCount:0,running:true,runningAfter:true,identityStableDuringSample:true,oomKilled:false,oomKilledAfter:false,configuredLimitBytes:caps[i]*MiB,reportedLimitBytes:2*1024*MiB,
...(version===3?{configuredMemorySwapBytes:caps[i]*MiB,configuredMemorySwapStatus:'available',resourceLimitsStableDuringSample:true,cgroupV2:{status:'available',metrics:{'memory.max':caps[i]*MiB,'memory.swap.max':0,'cpu.stat':{nr_periods:10,nr_throttled:2,throttled_usec:1234,notWhitelisted:999}}}}:{cgroupV2:{status:'available',metrics:{'memory.current':1,'memory.peak':2}}})}))
const p={avg10Percent:'0.00',avg60Percent:'0.01',avg300Percent:'1.25',totalMicros:123456}
const host={MemTotalBytes:2069721088,MemAvailableBytes:1000000000,pswpin:0,pswpout:0,oom_kill:0,...(version===3?{memoryPressure:{status:'available',metrics:{some:p,full:{...p,totalMicros:0}}}}:{})}
const data={schemaVersion:version,readOnlyDockerAndProc:true,serviceChanges:0,summary:{acceptancePassed:false},observations:[{elapsedMs:0,host,containers},{elapsedMs:1000,host:structuredClone(host),containers:structuredClone(containers)}]}
return {data,roleById:Object.fromEntries(containers.map((c,i)=>[c.id,roles[i]])),raw:()=>JSON.stringify(data)}
}
test('real schema2 producer contract remains arithmetic-only; missing extended measurements stay unknown',()=>{const f=fixture(2),r=evaluate(f.raw(),f.roleById);assert.equal(r.budgetFits,true);assert.equal(r.plannedBytes,1760*MiB);assert.equal(r.measurementSchemaVersion,2);assert.equal(r.extendedMetricsComplete,false);assert.equal(r.extendedObservations[0].memoryPressure,null);assert.equal(r.extendedObservations[0].containers[0].memoryMax,null);assert.equal(r.wholeInstallation2GbAccepted,false)})
test('schema3 projects only fixed new fields and never converts data to runtime/no-swap authority',()=>{const f=fixture(),r=evaluate(f.raw(),f.roleById);assert.equal(r.extendedMetricsComplete,true);assert.equal(r.measurementSamples,2);assert.equal(r.extendedObservations[0].containers[0].cpuThrottling.throttled_usec,1234);assert.equal(r.extendedObservations[0].memoryPressure.some.avg300Percent,'1.25');assert(!JSON.stringify(r).includes('notWhitelisted'));assert.equal(r.runtimeAcceptanceClaimed,false);assert.equal(r.changesAuthorized,false);assert.equal(r.noSwapAccepted,false);assert.equal(r.inventoryFreshnessProven,false);assert.equal(r.hostContainerInventoryComplete,false);assert.equal(r.measurementSelectedContainersOnly,true)})
test('numeric unlimited max/zero/-1/daemon-default preserve their different observed meanings',()=>{const f=fixture();for(const o of f.data.observations){const c=o.containers[0];c.cgroupV2.metrics['memory.max']='max';c.cgroupV2.metrics['memory.swap.max']='max';c.configuredMemorySwapBytes=-1}let r=evaluate(f.raw(),f.roleById);assert.equal(r.extendedObservations[0].containers[0].memoryMax,'max');assert.equal(r.extendedObservations[0].containers[0].configuredMemorySwapBytes,-1);assert.equal(r.noSwapAccepted,false);for(const o of f.data.observations)o.containers[0].configuredMemorySwapBytes=0;r=evaluate(f.raw(),f.roleById);assert.equal(r.extendedObservations[0].containers[0].configuredMemorySwapBytes,0)})
test('missing schema3 fields, partial cgroup and invalid PSI are unknown rather than zero',()=>{for(const mutate of [o=>{delete o.containers[0].configuredMemorySwapBytes},o=>{o.containers[0].cgroupV2.status='partial'},o=>{o.containers[0].cgroupV2.metrics['memory.max']=null},o=>{delete o.containers[0].cgroupV2.metrics['cpu.stat'].nr_periods},o=>{o.host.memoryPressure.metrics.some.avg10Percent='100.01'},o=>{o.host.memoryPressure.status='unavailable'}]){const f=fixture();for(const o of f.data.observations)mutate(o);const r=evaluate(f.raw(),f.roleById);assert.equal(r.extendedMetricsComplete,false);assert.equal(r.wholeInstallation2GbAccepted,false)}})
test('unsafe uint64 raw lexemes fail before JSON.parse; no rounded Number is recovered',()=>{for(const token of ['9007199254740992','9007199254740993','18446744073709551615','1.0000000000000001','9007199254740991.1','1e0','-0']){const f=fixture(),raw=f.raw().replace('123456',token);assert.throws(()=>evaluate(raw,f.roleById),/Unsafe|noninteger/);assert.throws(()=>evaluate(JSON.parse(raw),f.roleById),/Raw measurement JSON text required/)} })
test('safe integer boundary is exact and large number-like text inside strings is not parsed as a counter',()=>{const f=fixture();for(const o of f.data.observations)o.host.memoryPressure.metrics.some.totalMicros=Number.MAX_SAFE_INTEGER;f.data.ignored='9007199254740993 1.0000000000000001 \\" escaped';const r=evaluate(f.raw(),f.roleById);assert.equal(r.extendedObservations[0].memoryPressure.some.totalMicros,Number.MAX_SAFE_INTEGER);assert(!JSON.stringify(r).includes('ignored'))})
test('malformed JSON, unsupported schemas or claimed producer acceptance refuse',()=>{const f=fixture();assert.throws(()=>evaluate('{bad',f.roleById));for(const mutate of [d=>{d.schemaVersion=1},d=>{d.schemaVersion=4},d=>{d.serviceChanges=1},d=>{d.readOnlyDockerAndProc=false},d=>{d.summary.acceptancePassed=true},d=>{d.observations=[]}]){const x=fixture();mutate(x.data);assert.throws(()=>evaluate(x.raw(),x.roleById))}})
test('full IDs, complete stable container set, OOM/running/process/image/caps stay mandatory',()=>{for(const mutate of [d=>{d.observations[1].containers.pop()},d=>{d.observations[1].containers.push(d.observations[1].containers[0])},d=>{d.observations[1].containers[0].pid++},d=>{d.observations[1].containers[0].runningAfter=false},d=>{d.observations[1].containers[0].oomKilledAfter=true},d=>{d.observations[1].containers[0].configuredLimitBytes++},d=>{d.observations[1].containers[0].configuredMemorySwapBytes++},d=>{d.observations[1].host.MemTotalBytes++},d=>{d.observations[1].containers[0].startedAt=123}]){const f=fixture();mutate(f.data);assert.throws(()=>evaluate(f.raw(),f.roleById))}})
test('role labels remain exact own DATA descriptors, never an inherited/accessor authority',()=>{for(const kind of ['getter','inherited','missing','foreign']){const f=fixture(),key=f.data.observations[0].containers[0].id;if(kind==='getter')Object.defineProperty(f.roleById,key,{enumerable:true,get(){assert.fail('Getter executed')}});if(kind==='inherited'){const x=Object.create({[key]:'backend'});for(const [id,role]of Object.entries(f.roleById))if(id!==key)x[id]=role;f.roleById=x}if(kind==='missing')delete f.roleById[key];if(kind==='foreign')f.roleById['f'.repeat(64)]='other';assert.throws(()=>evaluate(f.raw(),f.roleById))}})
test('legacy arithmetic export keeps soft metadata and no-runtime semantics',()=>{const f=fixture(),input={hostMemTotalBytes:2069721088,osReserveBytes:512*MiB,observedContainerIds:Object.keys(f.roleById),containers:f.data.observations[0].containers.map(c=>({id:c.id,role:f.roleById[c.id],configuredLimitBytes:c.configuredLimitBytes}))};assert.equal(legacy(input).budgetFits,true);assert.equal(legacy(input).changesAuthorized,false)})
import {readFileSync} from 'node:fs'
test('actual base/candidate Python producers interoperate through synthetic raw wire files',()=>{for(const version of [2,3]){const raw=readFileSync(new URL('./schema'+version+'.main.synthetic.json',import.meta.url),'utf8');const r=evaluate(raw,{['a'.repeat(64)]:'backend'});assert.equal(r.measurementSchemaVersion,version);assert.equal(r.extendedMetricsComplete,version===3);assert.equal(r.budgetFits,false);assert.equal(r.wholeInstallation2GbAccepted,false);if(version===3){assert.equal(r.extendedObservations[0].containers[0].configuredMemorySwapBytes,1024);assert.equal(r.extendedObservations[0].memoryPressure.some.totalMicros,123456)}}})
@@ -0,0 +1,31 @@
import assert from 'node:assert/strict'
import { test } from 'node:test'
import { evaluateObservedInstallationBudget as evaluate } from './installation-memory-budget.mjs'
const MiB=1024*1024
function fixture() {
const containers=['backend','postgres','frontend','portal'].map((role,index)=>({id:String(index+1).repeat(64),role,running:true,imageId:'sha256:'+'a'.repeat(64),startedAt:'2026-10-28T12:00:00Z',memoryLimitBytes:0,reportedLimitBytes:2*1024*MiB}))
const roleById=Object.fromEntries(containers.map(row=>[row.id,row.role]))
return {inventory:{readOnly:true,containerSetStable:true,environmentRead:false,serviceChanges:false,newFixtureStarted:false,hostMemTotalBytes:2021212*1024,containers},roleById}
}
test('observed zero configured caps stay unbounded despite positive Docker reported host limits',()=>{
const f=fixture(), result=evaluate(f.inventory,f.roleById)
assert.equal(result.budgetFits,false);assert(result.reasons.includes('UNBOUNDED_CONTAINER'));assert.equal(result.configuredContainerBytes,0)
assert.equal(result.wholeInstallation2GbAccepted,false);assert.equal(result.roleLabelsAuthenticated,false);assert.equal(result.inventoryFreshnessProven,false)
})
test('complete explicit configured caps can fit arithmetic without becoming runtime acceptance',()=>{
const f=fixture();[512,384,160,192].forEach((limit,index)=>{f.inventory.containers[index].memoryLimitBytes=limit*MiB})
const result=evaluate(f.inventory,f.roleById);assert(result.budgetFits);assert.equal(result.plannedBytes,1760*MiB);assert.equal(result.changesAuthorized,false)
})
test('uncertain snapshots, started fixtures and missing limit/identity fields refuse',()=>{
for(const mutate of [f=>{f.inventory.containerSetStable=false},f=>{f.inventory.serviceChanges=true},f=>{f.inventory.newFixtureStarted=true},f=>{delete f.inventory.containers[0].memoryLimitBytes},f=>{f.inventory.containers[0].running=false},f=>{f.inventory.containers[0].imageId='unknown'},f=>{f.inventory.containers[0].startedAt='invalid'}]){const f=fixture();mutate(f);assert.throws(()=>evaluate(f.inventory,f.roleById))}
})
test('missing, foreign, inherited and accessor role labels cannot silently classify a service',()=>{
for(const mode of ['missing','foreign','inherited','accessor']){
const f=fixture(),key=f.inventory.containers[0].id
if(mode==='missing')delete f.roleById[key]
if(mode==='foreign')f.roleById['f'.repeat(64)]='other'
if(mode==='inherited'){const labels=Object.create({[key]:'backend'});for(const [id,role]of Object.entries(f.roleById))if(id!==key)labels[id]=role;f.roleById=labels}
if(mode==='accessor')Object.defineProperty(f.roleById,key,{enumerable:true,get(){assert.fail('Role getter must not execute')}})
assert.throws(()=>evaluate(f.inventory,f.roleById))
}
})
@@ -0,0 +1,58 @@
import assert from 'node:assert/strict'
import { setTimeout as delay } from 'node:timers/promises'
import { Client, StreamableHTTPClientTransport } from '@modelcontextprotocol/client'
import { resumeMcpSteps } from './mcp-step-journal.mjs'
const [phase, port, planId, connectionId] = process.argv.slice(2)
assert(process.send && process.env.MCP_FIXTURE_TOKEN && ['dispatch', 'resume'].includes(phase))
const uuid = /^[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}$/i
assert.match(planId, uuid)
assert.match(connectionId, uuid)
const client = new Client({ name: 'owned-routing-observer', version: '1.0.0' })
async function call(name, args) {
const reply = await client.callTool({ name, arguments: args })
assert(!reply.isError, 'MCP fixture call failed: ' + name)
return reply.structuredContent?.result ?? JSON.parse(reply.content[0].text)
}
try {
await client.connect(new StreamableHTTPClientTransport(new URL(`http://127.0.0.1:${port}/mcp`), {
requestInit: { headers: { Authorization: 'Bearer ' + process.env.MCP_FIXTURE_TOKEN } },
}))
let operationId
const result = await resumeMcpSteps({ journal: 'routing-forward.jsonl', fixture: `routing-scoped-forward:${connectionId}:${planId}`, steps: [{
id: 'forward-apply', kind: 'apply',
async run({ idempotencyKey, accept }) {
assert.equal(phase, 'dispatch', 'A resumed observer cannot send apply')
const receipt = await call('mm_change_apply', { planId, idempotencyKey })
assert.deepEqual([receipt.plan_id, receipt.connection_id], [planId, connectionId])
accept(receipt.id)
// The supervisor terminates this process after the journal fsync.
process.send(receipt.id)
await new Promise(resolve => process.once('disconnect', resolve))
throw new Error('Observer was disconnected by its supervisor')
},
async reconcile({ accepted }) {
assert.equal(phase, 'resume')
assert.ok(accepted)
const deadline = Date.now() + 60_000
while (Date.now() < deadline) {
const receipt = await call('mm_operation_get', { operationId: accepted })
assert.deepEqual([receipt.id, receipt.plan_id, receipt.connection_id], [accepted, planId, connectionId])
if (!['pending', 'running'].includes(receipt.state)) {
assert.deepEqual([receipt.state, receipt.verification], ['applied', 'passed'])
operationId = accepted
return { state: 'verified', evidence: accepted }
}
await delay(400)
}
return { state: 'unknown' }
},
}] })
assert.deepEqual(result, { status: 'passed' })
assert.ok(operationId)
await new Promise((resolve, reject) => process.send(operationId, error => error ? reject(error) : resolve()))
} finally {
await client.close()
if (process.connected) process.disconnect()
}
@@ -0,0 +1,84 @@
import assert from 'node:assert/strict'
import { resumeMcpSteps } from './mcp-step-journal.mjs'
const uuid = /^[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}$/i
function isProtocolStorageBusy(error) {
const text = error?.data?.text
if (error?.name !== 'SdkHttpError' || error.status !== 503 ||
typeof text !== 'string' || Buffer.byteLength(text) > 1024) return false
try {
const body = JSON.parse(text)
return body?.code === 'PROTOCOL_STORAGE_BUSY'
} catch {
return false
}
}
/** The caller supplies an already registered synthetic test server and owner review. */
export function resumeServerPoll({ journal, fixture, serverId, connectionId, call, findPlan, findAcceptedOperation, assertApproved }) {
assert(Number.isSafeInteger(serverId) && serverId > 0)
assert.match(connectionId, uuid)
const find = async args => {
const planId = await findPlan?.({ ...args, connectionId, serverId })
return planId ? { state: 'verified', evidence: planId } : { state: 'unknown' }
}
return resumeMcpSteps({ journal, fixture: `${fixture}:server-${serverId}:connection-${connectionId}`, steps: [
{ id: 'mm_server_poll_plan', kind: 'plan', reconcile: find, async run({ accept }) {
const result = await call('mm_server_poll_plan', { serverId })
accept(result.planId)
return result.planId
} },
{ id: 'manager.plan-approved', kind: 'read', async run({ evidence }) {
const planId = evidence.get('mm_server_poll_plan')
await assertApproved(planId)
return planId
} },
{ id: 'mm_change_apply', kind: 'apply',
async reconcile(args) {
const id = args.accepted ?? await findAcceptedOperation?.({
...args, connectionId, planId: args.evidence.get('mm_server_poll_plan'),
})
return id ? { state: 'verified', evidence: id } : { state: 'unknown' }
},
async run({ evidence, idempotencyKey, accept }) {
const planId = evidence.get('mm_server_poll_plan')
const result = await call('mm_change_apply', { planId, idempotencyKey })
accept(result.id)
return result.id
} },
{ id: 'mm_operation_get', kind: 'read', async run({ evidence }) {
const operationId = evidence.get('mm_change_apply')
let result
try {
result = await call('mm_operation_get', { operationId })
} catch (error) {
if (!isProtocolStorageBusy(error)) throw error
throw Object.assign(new Error('MCP протокольное хранилище занято', { cause: error }), {
code: 'PROTOCOL_STORAGE_BUSY',
})
}
assert.equal(result.id, operationId)
assert.equal(result.plan_id, evidence.get('mm_server_poll_plan'))
assert.equal(result.connection_id, connectionId)
if (result.state !== 'applied' || result.verification !== 'passed') {
const code = ['pending', 'running'].includes(result.state) ? 'POLL_PENDING' : 'POLL_UNVERIFIED'
throw Object.assign(new Error('Poll terminal result is unverified'), { code })
}
assert.equal(result.steps.length, 1)
assert.equal(result.steps[0].state, 'verified')
const receipt = result.steps[0].result
assert.equal(receipt.operationId, operationId)
assert.match(receipt.receiptId, uuid)
assert.equal(receipt.nativeReads, 4)
assert.equal(receipt.complete, true)
assert.equal(receipt.maintenanceComplete, true)
assert.equal(receipt.configurationVerified, true)
assert.equal(receipt.automaticReplay, false)
assert(Number.isSafeInteger(receipt.completedPublications) && receipt.completedPublications >= 2)
assert.equal(receipt.confirmedPublications, receipt.completedPublications)
assert.equal(receipt.snapshot.serverId, serverId)
return operationId
} },
] })
}
@@ -0,0 +1,119 @@
import assert from 'node:assert/strict'
import { createHash } from 'node:crypto'
import { closeSync, constants, fstatSync, fsyncSync, openSync, readFileSync, unlinkSync, writeFileSync } from 'node:fs'
import { isDeepStrictEqual } from 'node:util'
const key = /^[a-z0-9][a-z0-9_.:-]{0,127}$/i
const evidenceId = /^[0-9a-f]{8}(?:-[0-9a-f]{4}){3}-[0-9a-f]{12}$/i
const errorCode = /^[A-Z][A-Z0-9_]{0,63}$/
const validEvidence = (receipt, kind) => receipt === null ? kind === 'read' : typeof receipt === 'string' && evidenceId.test(receipt)
/** A test journal only. The caller owns the fixture and all reconciliation reads. */
export async function resumeMcpSteps({ journal, fixture, steps }) {
assert(key.test(fixture))
const scope = steps.map(({ id, kind }) => ({ id, kind }))
assert(scope.length && new Set(scope.map(step => step.id)).size === scope.length)
assert(scope.every(({ id, kind }) => key.test(id) && ['read', 'plan', 'apply'].includes(kind)))
// One process owns a journal at a time. A leftover lock requires operator reconciliation.
const lock = openSync(journal + '.lock', 'wx', 0o600)
let records
try {
let fd
try {
fd = openSync(journal, constants.O_RDWR | constants.O_APPEND | constants.O_CREAT | (constants.O_NOFOLLOW ?? 0), 0o600)
assert(fstatSync(fd).isFile(), 'Test journal must be a regular file')
const raw = readFileSync(fd, 'utf8')
assert(!raw || raw.endsWith('\n'), 'Truncated test receipt')
records = raw.trimEnd().split('\n').filter(Boolean).map(line => JSON.parse(line))
const header = { phase: 'fixture', fixture, steps: scope }
const append = entry => {
writeFileSync(fd, JSON.stringify(entry) + '\n', 'utf8')
fsyncSync(fd)
records.push(entry)
}
if (!records.length) append(header)
assert(isDeepStrictEqual(records[0], header), 'Test fixture or ordered step list changed')
const state = new Map()
let cursor = 0
for (const entry of records.slice(1)) {
const previous = state.get(entry.id)
assert(scope[cursor]?.id === entry.id, 'Unexpected test receipt order')
if (entry.phase === 'started') {
assert(!previous || previous.phase === 'retryable', 'Unexpected resumed dispatch')
assert(entry.attempt === (previous?.attempt ?? 0) + 1, 'Unexpected test attempt')
} else {
assert(['started', 'accepted', 'failed'].includes(previous?.phase) && entry.attempt === previous.attempt, 'Unbound test receipt')
assert(['passed', 'retryable', 'accepted', 'failed'].includes(entry.phase), 'Unknown test receipt')
if (entry.phase === 'accepted') {
assert(previous.phase === 'started' && scope[cursor].kind !== 'read' &&
validEvidence(entry.evidence, scope[cursor].kind), 'Invalid MCP acceptance receipt')
}
if (entry.phase === 'failed') {
assert(previous.phase !== 'failed' && typeof entry.code === 'string' && errorCode.test(entry.code), 'Invalid failure receipt')
assert(entry.accepted === (previous.phase === 'accepted' ? previous.evidence : null), 'MCP acceptance receipt changed')
}
if (entry.phase === 'passed') {
assert(validEvidence(entry.evidence, scope[cursor].kind), 'Invalid stored MCP receipt ID')
const accepted = previous.phase === 'accepted' ? previous.evidence : previous.accepted
assert(!accepted || accepted === entry.evidence, 'MCP acceptance receipt changed')
cursor++
}
if (entry.phase === 'retryable') {
assert(entry.accepted === null && (previous.phase === 'started' ||
previous.phase === 'failed' && previous.accepted === null), 'Cannot retry an accepted MCP step')
}
}
state.set(entry.id, entry)
}
const evidence = new Map([...state].filter(([, entry]) => entry.phase === 'passed').map(([id, entry]) => [id, entry.evidence]))
const attemptKey = (step, attempt) => 'acceptance-' + createHash('sha256')
.update(JSON.stringify([fixture, step.id, step.kind, attempt])).digest('hex')
const record = (step, attempt, phase, details = {}) => append({ id: step.id, attempt, phase, ...details })
const complete = (step, attempt, receipt) => {
assert(validEvidence(receipt, step.kind), 'Invalid MCP receipt ID')
record(step, attempt, 'passed', { evidence: receipt })
evidence.set(step.id, receipt)
}
for (const step of steps.slice(cursor)) {
const previous = state.get(step.id)
let attempt = previous?.attempt ?? 0
if (previous && ['started', 'accepted', 'failed'].includes(previous.phase)) {
const accepted = previous.phase === 'accepted' ? previous.evidence : previous.accepted ?? null
const result = step.kind === 'read' ? { state: 'no-effect' } :
await step.reconcile?.({ attempt, accepted, idempotencyKey: attemptKey(step, attempt), evidence: new Map(evidence) })
if (result?.state === 'verified') {
assert(!accepted || accepted === result.evidence, 'Reconciled a different MCP receipt')
complete(step, attempt, result.evidence)
continue
}
if (accepted || result?.state !== 'no-effect') return { status: 'reconcile', step: step.id }
record(step, attempt, 'retryable', { accepted })
}
attempt++
record(step, attempt, 'started')
let receipt
let accepted = null
try {
receipt = await step.run({ attempt, idempotencyKey: attemptKey(step, attempt), evidence: new Map(evidence), accept(id) {
assert(accepted === null && step.kind !== 'read' && validEvidence(id, step.kind), 'Invalid MCP acceptance receipt')
record(step, attempt, 'accepted', { evidence: id })
accepted = id
} })
} catch (error) {
const code = typeof error?.code === 'string' && errorCode.test(error.code) ? error.code : 'TEST_STEP_FAILED'
record(step, attempt, 'failed', { code, accepted })
return { status: 'failed', step: step.id }
}
assert(!accepted || accepted === receipt, 'Verified a different MCP receipt')
complete(step, attempt, receipt)
}
return { status: 'passed' }
} finally {
if (fd !== undefined) closeSync(fd)
}
} finally {
closeSync(lock)
unlinkSync(journal + '.lock')
}
}
@@ -0,0 +1,176 @@
import assert from 'node:assert/strict'
import { appendFileSync, mkdtempSync, readFileSync, rmSync } from 'node:fs'
import { tmpdir } from 'node:os'
import { join } from 'node:path'
import test from 'node:test'
import { resumeMcpSteps } from './mcp-step-journal.mjs'
import { resumeServerPoll } from './mcp-server-poll-steps.mjs'
const uuid = 'ec4c6a09-3c3b-4a37-a370-c8821321807c'
const connectionId = 'd0d96834-948b-4cea-81d8-d8f9dc5e1429'
const operationId = 'c2cc9f5d-6d62-43b5-a6d2-ba7bedbd4fc1'
const pollReceipt = { operationId, receiptId: uuid, nativeReads: 4, complete: true,
maintenanceComplete: true, configurationVerified: true, automaticReplay: false,
confirmedPublications: 22, completedPublications: 22, snapshot: { serverId: 3, status: 'online' } }
function options(t, steps) {
const dir = mkdtempSync(join(tmpdir(), 'mm-mcp-step-test-'))
t.after(() => rmSync(dir, { recursive: true }))
return { journal: join(dir, 'history.jsonl'), fixture: 'owned-test', steps }
}
test('resumes the failed read and preserves successful reads and failed attempts', async t => {
const calls = []
const config = options(t, [
{ id: 'first', kind: 'read', run: async () => { calls.push('first'); return null } },
{ id: 'second', kind: 'read', run: async ({ attempt }) => {
calls.push('second:' + attempt)
if (attempt === 1) throw Error('test failure')
return null
} },
])
assert.deepEqual(await resumeMcpSteps(config), { status: 'failed', step: 'second' })
assert.deepEqual(await resumeMcpSteps(config), { status: 'passed' })
assert.deepEqual(await resumeMcpSteps(config), { status: 'passed' })
assert.deepEqual(calls, ['first', 'second:1', 'second:2'])
assert.deepEqual(readFileSync(config.journal, 'utf8').trim().split('\n').map(line => JSON.parse(line).phase),
['fixture', 'started', 'passed', 'started', 'failed', 'retryable', 'started', 'passed'])
})
test('unknown apply awaits reconciliation and a verified receipt advances without resending', async t => {
let sends = 0
let verified = false
let sentKey
const config = options(t, [
{ id: 'plan', kind: 'plan', run: async () => uuid },
{ id: 'apply', kind: 'apply', run: async ({ evidence, accept, idempotencyKey }) => {
assert.equal(evidence.get('plan'), uuid)
sentKey = idempotencyKey
sends++
accept(uuid)
throw Error('lost reply')
}, reconcile: async ({ accepted, idempotencyKey }) => {
assert.equal(accepted, uuid)
assert.equal(idempotencyKey, sentKey)
return verified ? { state: 'verified', evidence: uuid } : { state: 'no-effect' }
} },
{ id: 'next', kind: 'read', run: async ({ evidence }) => { assert.equal(evidence.get('apply'), uuid); return null } },
])
assert.deepEqual(await resumeMcpSteps(config), { status: 'failed', step: 'apply' })
assert.deepEqual(await resumeMcpSteps(config), { status: 'reconcile', step: 'apply' })
verified = true
assert.deepEqual(await resumeMcpSteps(config), { status: 'passed' })
assert.equal(sends, 1)
})
test('cannot resume a different fixture or an altered ordered step list', async t => {
const config = options(t, [{ id: 'read', kind: 'read', run: async () => null }])
await resumeMcpSteps(config)
await assert.rejects(resumeMcpSteps({ ...config, fixture: 'other' }), /fixture or ordered step/)
await assert.rejects(resumeMcpSteps({ ...config, steps: [{ ...config.steps[0], id: 'other' }] }), /fixture or ordered step/)
})
test('refuses a stored retry marker after an accepted apply', async t => {
const config = options(t, [{ id: 'apply', kind: 'apply', async run({ accept }) {
accept(uuid)
throw new Error('lost reply')
} }])
await resumeMcpSteps(config)
appendFileSync(config.journal, JSON.stringify({ id: 'apply', attempt: 1, phase: 'retryable', accepted: null }) + '\n')
await assert.rejects(resumeMcpSteps(config), /Cannot retry an accepted MCP step/)
})
test('the poll scenario resumes owner review and then reads the same accepted operation', async t => {
const { journal, fixture } = options(t, [])
const calls = []
let approved = false
let running = true
let storageBusy = true
const config = { journal, fixture, serverId: 3, connectionId, assertApproved: async id => {
assert.equal(id, uuid)
if (!approved) throw Error('approval not visible')
}, async call(tool, args) {
calls.push(tool)
if (tool === 'mm_server_poll_plan') { assert.deepEqual(args, { serverId: 3 }); return { planId: uuid } }
if (tool === 'mm_change_apply') {
assert.equal(args.planId, uuid)
assert.match(args.idempotencyKey, /^acceptance-[0-9a-f]{64}$/)
return { id: operationId }
}
assert.deepEqual(args, { operationId })
if (storageBusy) throw Object.assign(new Error('MCP storage busy'), {
name: 'SdkHttpError', status: 503, data: { text: '{"code":"PROTOCOL_STORAGE_BUSY"}' },
})
return { id: operationId, plan_id: uuid, connection_id: connectionId,
...(running ? { state: 'running' } : { state: 'applied', verification: 'passed',
steps: [{ state: 'verified', result: pollReceipt }] }) }
} }
assert.equal((await resumeServerPoll(config)).step, 'manager.plan-approved')
approved = true
assert.equal((await resumeServerPoll(config)).step, 'mm_operation_get')
storageBusy = false
assert.equal((await resumeServerPoll(config)).step, 'mm_operation_get')
running = false
assert.deepEqual(await resumeServerPoll(config), { status: 'passed' })
await assert.rejects(resumeServerPoll({ ...config, connectionId: uuid }), /fixture or ordered step/)
assert.deepEqual(calls, ['mm_server_poll_plan', 'mm_change_apply', 'mm_operation_get', 'mm_operation_get', 'mm_operation_get'])
const failures = readFileSync(journal, 'utf8').trim().split('\n').map(line => JSON.parse(line))
.filter(entry => entry.id === 'mm_operation_get' && entry.phase === 'failed')
assert.deepEqual(failures.map(entry => entry.code), ['PROTOCOL_STORAGE_BUSY', 'POLL_PENDING'])
})
test('recovers a lost poll apply response by the exact connection, plan and idempotency key', async t => {
const { journal, fixture } = options(t, [])
let key, sends = 0, wrongBinding = true
const config = { journal, fixture, serverId: 3, connectionId, assertApproved: async () => {},
async findAcceptedOperation(binding) {
assert.equal(binding.connectionId, connectionId)
assert.equal(binding.planId, uuid)
assert.equal(binding.idempotencyKey, key)
return operationId
},
async call(tool, args) {
if (tool === 'mm_server_poll_plan') return { planId: uuid }
if (tool === 'mm_change_apply') {
sends++
key = args.idempotencyKey
throw Error('apply response lost')
}
return { id: operationId, plan_id: uuid, connection_id: wrongBinding ? uuid : connectionId,
state: 'applied', verification: 'passed', steps: [{ state: 'verified', result: pollReceipt }] }
},
}
assert.equal((await resumeServerPoll(config)).step, 'mm_change_apply')
assert.equal((await resumeServerPoll(config)).step, 'mm_operation_get')
wrongBinding = false
assert.deepEqual(await resumeServerPoll(config), { status: 'passed' })
assert.equal(sends, 1)
})
test('rejects incomplete or misbound terminal poll receipts before recording success', async t => {
const { journal, fixture } = options(t, [])
const calls = []
const receipts = [
{ ...pollReceipt, operationId: uuid },
{ ...pollReceipt, receiptId: undefined },
{ ...pollReceipt, confirmedPublications: 21 },
{ ...pollReceipt, confirmedPublications: 0, completedPublications: 0 },
{ ...pollReceipt, maintenanceComplete: false },
{ ...pollReceipt, snapshot: { serverId: 4, status: 'online' } },
pollReceipt,
]
const config = { journal, fixture, serverId: 3, connectionId, assertApproved: async () => {},
async call(tool, args) {
calls.push(tool)
if (tool === 'mm_server_poll_plan') return { planId: uuid }
if (tool === 'mm_change_apply') return { id: operationId }
assert.deepEqual(args, { operationId })
return { id: operationId, plan_id: uuid, connection_id: connectionId, state: 'applied', verification: 'passed',
steps: [{ state: 'verified', result: receipts.shift() }] }
},
}
for (let i = 0; i < 6; i++) {
assert.deepEqual(await resumeServerPoll(config), { status: 'failed', step: 'mm_operation_get' })
}
assert.deepEqual(await resumeServerPoll(config), { status: 'passed' })
assert.deepEqual(calls, ['mm_server_poll_plan', 'mm_change_apply', ...Array(7).fill('mm_operation_get')])
})
@@ -0,0 +1,193 @@
"""Synthetic-only: no Docker/socket, real /proc/cgroup reads, process enumeration or main()."""
import copy
import importlib.util
from pathlib import Path
from types import SimpleNamespace
import unittest
from unittest.mock import patch
AREA = Path(__file__).parent
spec = importlib.util.spec_from_file_location('memory_measurement_extended', AREA / 'measure-memory.py')
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)
MEMINFO = 'MemTotal: 2048 kB\nMemAvailable: 1024 kB\nSwapTotal: 4096 kB\nSwapFree: 3072 kB\n'
VMSTAT = 'pswpin 20\npswpout 30\noom_kill 0\n'
PSI = 'some avg10=0.00 avg60=1.25 avg300=100.00 total=123456\nfull avg10=0.00 avg60=0.01 avg300=1.00 total=0\n'
ID = 'a' * 64
CG = '/sys/fs/cgroup/system.slice/docker-' + ID + '.scope'
class Synthetic(unittest.TestCase):
def setUp(self):
self.reads = []
self.gets = []
self.before = {'Id': ID, 'Image': 'sha256:' + 'b' * 64, 'RestartCount': 0,
'State': {'StartedAt': '2026-11-02T00:00:00Z', 'Pid': 123, 'Running': True, 'OOMKilled': False},
'HostConfig': {'Memory': 512, 'MemorySwap': 1024}, 'Config': {'Env': ['SECRET=never-retain']}}
self.stats = {'id': ID, 'memory_stats': {'usage': 100, 'limit': 2048, 'stats': {'inactive_file': 30}}}
self.files = {'/proc/meminfo': MEMINFO, '/proc/vmstat': VMSTAT, '/proc/pressure/memory': PSI,
'/proc/self/mountinfo': '1 0 0:1 / /sys/fs/cgroup rw - cgroup2 cgroup rw\n',
'/proc/123/cgroup': '0::/system.slice/docker-' + ID + '.scope\n',
'/proc/123/stat': '123 (owned fixture) S ' + ' '.join(['0'] * 18 + ['987'] + ['0'] * 8),
CG + '/memory.current': '100\n', CG + '/memory.peak': '200\n',
CG + '/memory.events': 'low 0\nhigh 0\nmax 0\noom 0\noom_kill 0\n',
CG + '/memory.swap.current': '0\n', CG + '/memory.swap.peak': '0\n',
CG + '/memory.swap.events': 'high 0\nmax 0\nfail 0\n',
CG + '/memory.max': '512\n', CG + '/memory.swap.max': '512\n',
CG + '/cpu.stat': 'usage_usec 50\nuser_usec 30\nsystem_usec 20\nnr_periods 10\nnr_throttled 2\nthrottled_usec 1234\nnr_bursts 0\n'}
def deny(*_args, **_kwargs):
raise AssertionError('Real external/OS I/O is forbidden by synthetic test')
# Module-local OS proxy avoids changing unittest/Python's filesystem operations.
fake_os = SimpleNamespace(**vars(m.os))
fake_os.path = SimpleNamespace(**vars(m.os.path))
fake_os.sep = '/' # synthetic Linux path contract, not this Windows host
fake_os.open = deny
fake_os.stat = deny
fake_os.path.realpath = deny
self.enterContext(patch.object(m, 'os', fake_os))
self.enterContext(patch.object(m, 'read_bounded', side_effect=deny))
self.enterContext(patch.object(m, 'docker_get', side_effect=deny))
self.enterContext(patch.object(m.socket, 'socket', side_effect=deny))
def read(self, name, expected_device=None, deadline=None):
self.reads.append(name)
if name not in self.files:
raise FileNotFoundError('synthetic missing')
value = self.files[name]
if isinstance(value, BaseException):
raise value
return value
def observe(self, after=None, realpath=None, statter=None):
after = copy.deepcopy(self.before) if after is None else after
def get(name, deadline=None):
self.gets.append(name)
if self.gets == ['/containers/' + ID + '/json']:
return copy.deepcopy(self.before)
if self.gets == ['/containers/' + ID + '/json', '/containers/' + ID + '/stats?stream=false&one-shot=true']:
return copy.deepcopy(self.stats)
if self.gets == ['/containers/' + ID + '/json', '/containers/' + ID + '/stats?stream=false&one-shot=true', '/containers/' + ID + '/json']:
return copy.deepcopy(after)
raise AssertionError('Unexpected Docker path or extra request')
def statter_default(name):
allowed = {'/sys/fs/cgroup': (8, 1), CG: (8, 2), '/proc/123/root': (8, 3)}
if name not in allowed:
raise AssertionError('Unexpected stat path')
dev, ino = allowed[name]
return SimpleNamespace(st_dev=dev, st_ino=ino)
with patch.object(m, 'read_bounded', side_effect=self.read), patch.object(m, 'docker_get', side_effect=get), \
patch.object(m.os.path, 'realpath', side_effect=realpath or (lambda p: p)), \
patch.object(m.os, 'stat', side_effect=statter or statter_default):
return m.collect_container(ID)
def test_limits_max_zero_finite_are_not_missing_or_host_ram(self):
for value, expected in [('max\n', 'max'), ('max', 'max'), ('0\n', 0), ('512\n', 512), (str(2**64-1), 2**64-1)]:
self.assertEqual(m.parse_limit(value), expected)
for bad in ('MAX', '-1', ' 0', '00', '1.0', str(2**64), 'max\n\n'):
with self.assertRaises(ValueError): m.parse_limit(bad)
def test_cpu_throttling_units_required_fields_and_output_whitelist(self):
parsed = m.parse_cpu_throttling(self.files[CG + '/cpu.stat'])
self.assertEqual(parsed, {'nr_periods':10,'nr_throttled':2,'throttled_usec':1234})
for bad in ('nr_periods 1\nnr_throttled 0\n', 'nr_periods 1\nnr_periods 2\nnr_throttled 0\nthrottled_usec 0\n', 'nr_periods -1\nnr_throttled 0\nthrottled_usec 0\n'):
with self.assertRaises(ValueError): m.parse_cpu_throttling(bad)
def test_pressure_fixed_groups_percent_and_microseconds(self):
parsed = m.parse_memory_pressure(PSI)
self.assertEqual(parsed['some'], {'avg10Percent':'0.00','avg60Percent':'1.25','avg300Percent':'100.00','totalMicros':123456})
self.assertEqual(parsed['full']['totalMicros'], 0)
for bad in (PSI.replace('100.00','100.01'), PSI.replace('1.25','nan'), PSI.replace('total=0','total=-1'), PSI.splitlines()[0], PSI.replace('full','some'), PSI+'extra\n', PSI.replace('total=0','total='+str(2**64))):
with self.assertRaises(ValueError): m.parse_memory_pressure(bad)
def test_docker_memoryswap_raw_minus1_zero_finite_and_missing_differ(self):
for raw in (-1,0,512,2**63-1):
item = copy.deepcopy(self.before); item['HostConfig']['MemorySwap'] = raw
self.assertEqual(m.configured_memory_swap(item), raw)
for bad in (None,False,True,'0',-2,2**63):
item = copy.deepcopy(self.before); item['HostConfig']['MemorySwap'] = bad
self.assertIsNone(m.configured_memory_swap(item))
for host in ({},None,[]):
self.assertIsNone(m.configured_memory_swap({'HostConfig':host}))
def test_real_collection_paths_are_only_fixed_host_and_retained_cgroup(self):
row = self.observe()
self.assertTrue(row['identityStableDuringSample'])
self.assertTrue(row['resourceLimitsStableDuringSample'])
self.assertEqual(row['configuredMemorySwapBytes'], 1024)
self.assertEqual(row['cgroupV2']['status'], 'available')
self.assertEqual(row['cgroupV2']['metrics']['memory.max'], 512)
self.assertEqual(row['cgroupV2']['metrics']['cpu.stat']['throttled_usec'], 1234)
self.assertTrue(set(self.reads).issubset(set(self.files)))
self.assertNotIn('SECRET', str(row)); self.assertNotIn('Env', str(row))
self.assertEqual(len(self.gets), 3)
def test_unlimited_limits_do_not_become_numeric_zero(self):
self.files[CG+'/memory.max']='max\n'; self.files[CG+'/memory.swap.max']='max\n'
row=self.observe();self.assertEqual(row['cgroupV2']['metrics']['memory.max'],'max');self.assertEqual(row['cgroupV2']['metrics']['memory.swap.max'],'max')
def test_each_new_cgroup_source_missing_or_malformed_is_unknown(self):
for name in ('memory.max','memory.swap.max','cpu.stat'):
for value in (None,'not-a-counter\n'):
with self.subTest(name=name,value=value):
self.setUp_fixture_only()
if value is None: self.files.pop(CG+'/'+name)
else: self.files[CG+'/'+name]=value
row=self.observe();self.assertEqual(row['cgroupV2']['status'],'partial');self.assertIsNone(row['cgroupV2']['metrics'][name]);self.assertIn(name,row['cgroupV2']['unavailable'])
def setUp_fixture_only(self):
# Recreate dictionary, not guards/patches; no real I/O.
self.files.update({CG+'/memory.max':'512\n',CG+'/memory.swap.max':'512\n',CG+'/cpu.stat':'nr_periods 10\nnr_throttled 2\nthrottled_usec 1234\n'});self.gets=[]
def test_missing_or_changed_swap_configuration_cannot_authenticate_limits(self):
after=copy.deepcopy(self.before);after['HostConfig']['MemorySwap']=2048
row=self.observe(after);self.assertFalse(row['resourceLimitsStableDuringSample']);self.assertEqual(row['cgroupV2']['status'],'partial');self.assertIsNone(row['cgroupV2']['metrics']['memory.swap.max'])
self.gets=[];self.before['HostConfig'].pop('MemorySwap');after=copy.deepcopy(self.before)
row=self.observe(after);self.assertIsNone(row['configuredMemorySwapBytes']);self.assertEqual(row['configuredMemorySwapStatus'],'unavailable');self.assertIsNone(row['resourceLimitsStableDuringSample']);self.assertEqual(row['cgroupV2']['status'],'partial')
def test_memory_limit_change_cannot_reuse_stable_container_identity_as_limit_proof(self):
after=copy.deepcopy(self.before);after['HostConfig']['Memory']=1024
row=self.observe(after)
self.assertTrue(row['identityStableDuringSample'])
self.assertFalse(row['resourceLimitsStableDuringSample'])
for key in ('memory.max','memory.swap.max'):
self.assertIsNone(row['cgroupV2']['metrics'][key])
self.assertEqual(row['cgroupV2']['unavailable'][key],'container-resource-limits-changed-or-unavailable')
def test_legacy_projection_fields_keep_cache_limit_and_rss_distinctions(self):
item=copy.deepcopy(self.before);item['HostConfig'].pop('MemorySwap');item['HostConfig']['Memory']=0
row=m.project_container(item,self.stats,ID)
self.assertEqual(row['usageIncludingCacheBytes'],100)
self.assertEqual(row['usageMinusInactiveFileBytes'],70)
self.assertEqual(row['configuredLimitBytes'],0)
self.assertEqual(row['reportedLimitBytes'],2048)
self.assertEqual(row['configuredMemorySwapStatus'],'unavailable')
self.assertIsNone(row['configuredMemorySwapBytes'])
def test_identity_drift_symlink_and_pid_generation_still_refuse(self):
after=copy.deepcopy(self.before);after['State']['Pid']=456
row=self.observe(after);self.assertFalse(row['identityStableDuringSample']);self.assertIsNone(row['resourceLimitsStableDuringSample']);self.assertEqual(row['cgroupV2']['status'],'unavailable')
self.gets=[];row=self.observe(realpath=lambda p:'/other' if p.endswith('memory.swap.max') else p);self.assertEqual(row['cgroupV2']['status'],'partial');self.assertIsNone(row['cgroupV2']['metrics']['memory.swap.max'])
self.gets=[]
with patch.object(m,'process_generation',side_effect=[(1,8,3),(2,8,3)]):
self.assertEqual(self.observe()['cgroupV2']['status'],'unavailable')
def test_host_collection_whitelists_one_psi_file_and_missing_is_explicit(self):
with patch.object(m,'read_bounded',side_effect=self.read):
host=m.collect_host();self.assertEqual(host['memoryPressure']['status'],'available');self.assertEqual(host['MemTotalBytes'],2048*1024)
self.assertEqual(self.reads,['/proc/meminfo','/proc/vmstat','/proc/pressure/memory'])
self.files.pop('/proc/pressure/memory');host=m.collect_host();self.assertEqual(host['memoryPressure']['status'],'unavailable');self.assertIsNone(host['memoryPressure']['metrics'])
def test_deadline_never_becomes_partial_or_success(self):
self.files[CG+'/cpu.stat']=m.ObservationDeadlineExceeded('fixture-deadline')
with self.assertRaises(m.ObservationDeadlineExceeded): self.observe()
with patch.object(m,'read_bounded',side_effect=[MEMINFO,VMSTAT,m.ObservationDeadlineExceeded('fixture-deadline')]):
with self.assertRaises(m.ObservationDeadlineExceeded): m.collect_host()
def test_summary_exposes_unknown_and_never_authorizes_runtime(self):
row=self.observe()
with patch.object(m,'read_bounded',side_effect=self.read):host=m.collect_host()
sample={'elapsedMs':0,'host':host,'containers':[row]}
out=m.summarize([sample]);self.assertTrue(out['allResourceLimitsStableDuringSample']);self.assertTrue(out['allHostMemoryPressureAvailable']);self.assertFalse(out['acceptancePassed'])
row['resourceLimitsStableDuringSample']=None;host['memoryPressure']['status']='unavailable'
out=m.summarize([sample]);self.assertFalse(out['allResourceLimitsStableDuringSample']);self.assertFalse(out['allHostMemoryPressureAvailable']);self.assertFalse(out['acceptancePassed'])
if __name__=='__main__': unittest.main()
@@ -0,0 +1,459 @@
"""Bounded Linux/Docker memory observation; GET only, no service changes.
Run alongside the named acceptance workload. Output is evidence, never an
automatic declaration that a 2 GB deployment has passed its load acceptance.
Requires Python 3 and read access to /proc and the local Docker socket.
"""
import argparse
import http.client
import json
import os
import re
import socket
import stat
import threading
import time
MAX_RESPONSE = 4 * 1024 * 1024
MAX_FILE_BYTES = 65536
MAX_FILE_LINES = 256
CGROUP_ROOT = '/sys/fs/cgroup'
CGROUP_FILES = ('memory.current', 'memory.peak', 'memory.events',
'memory.swap.current', 'memory.swap.peak', 'memory.swap.events',
'memory.max', 'memory.swap.max', 'cpu.stat')
class CgroupUnavailable(ValueError):
"""Fixed reason codes only; never include private paths or raw content."""
class ObservationDeadlineExceeded(RuntimeError):
pass
def check_deadline(deadline):
if deadline is not None and time.monotonic() >= deadline:
raise ObservationDeadlineExceeded('Observation time bound exceeded')
def read_bounded(path, expected_device=None, deadline=None):
# Fixed production paths only; no CLI path or executable is accepted.
check_deadline(deadline)
descriptor = os.open(path, os.O_RDONLY | getattr(os, 'O_NOFOLLOW', 0) | getattr(os, 'O_NONBLOCK', 0))
with os.fdopen(descriptor, 'rb') as handle:
identity = os.fstat(handle.fileno())
if not stat.S_ISREG(identity.st_mode) or (expected_device is not None and identity.st_dev != expected_device):
raise ValueError('Non-regular observation')
raw = handle.read(MAX_FILE_BYTES + 1)
check_deadline(deadline)
if len(raw) > MAX_FILE_BYTES:
raise ValueError('Observation file too large')
text = raw.decode('ascii')
if len(text.splitlines()) > MAX_FILE_LINES:
raise ValueError('Too many observation lines')
return text
def parse_counter(text):
if not re.fullmatch(r'(?:0|[1-9][0-9]{0,19})\n?', text):
raise ValueError('Invalid counter')
value = int(text)
if value > 2 ** 64 - 1:
raise ValueError('Counter overflow')
return value
def parse_events(text, required):
lines = text.splitlines()
if not 1 <= len(lines) <= 32:
raise ValueError('Invalid event count')
result = {}
for line in lines:
fields = line.split(' ')
if len(fields) != 2 or not re.fullmatch(r'[a-z_]{1,64}', fields[0]) or fields[0] in result:
raise ValueError('Invalid event')
result[fields[0]] = parse_counter(fields[1])
if not set(required).issubset(result):
raise ValueError('Incomplete events')
return result
def parse_limit(text):
# Preserve unlimited vs zero vs unavailable; never substitute host RAM.
return 'max' if text in ('max', 'max\n') else parse_counter(text)
def parse_cpu_throttling(text):
required = ('nr_periods', 'nr_throttled', 'throttled_usec')
parsed = parse_events(text, required)
# Kernel extensions are validated but do not expand the output whitelist.
return {key: parsed[key] for key in required}
def parse_memory_pressure(text):
rows = text.splitlines()
if len(rows) != 2:
raise ValueError('Incomplete memory pressure')
result = {}
for line in rows:
match = re.fullmatch(r'(some|full) avg10=([0-9]{1,3}\.[0-9]{2}) avg60=([0-9]{1,3}\.[0-9]{2}) avg300=([0-9]{1,3}\.[0-9]{2}) total=((?:0|[1-9][0-9]{0,19}))', line)
if not match or match[1] in result:
raise ValueError('Invalid memory pressure')
for value in match.group(2, 3, 4):
whole, fraction = value.split('.')
if int(whole) * 100 + int(fraction) > 10000:
raise ValueError('Invalid memory pressure percentage')
result[match[1]] = {'avg10Percent': match[2], 'avg60Percent': match[3],
'avg300Percent': match[4], 'totalMicros': parse_counter(match[5])}
if set(result) != {'some', 'full'}:
raise ValueError('Incomplete memory pressure')
return result
def configured_memory_swap(inspect):
# Docker MemorySwap is TOTAL memory+swap; 0 means daemon default, -1 unlimited.
host = inspect.get('HostConfig')
value = host.get('MemorySwap') if isinstance(host, dict) else None
if type(value) is not int or not -1 <= value <= 2 ** 63 - 1:
return None
return value
def resource_limits_stable(before, after):
values = []
for item in (before, after):
host = item.get('HostConfig')
memory = host.get('Memory') if isinstance(host, dict) else None
swap = configured_memory_swap(item)
if type(memory) is not int or not 0 <= memory <= 2 ** 63 - 1 or swap is None:
return None
values.append((memory, swap))
return values[0] == values[1]
def collect_host(deadline=None):
memory = parse_host(read_bounded('/proc/meminfo', deadline=deadline),
read_bounded('/proc/vmstat', deadline=deadline))
try:
pressure = parse_memory_pressure(read_bounded('/proc/pressure/memory', deadline=deadline))
memory['memoryPressure'] = {'status': 'available', 'metrics': pressure, 'reason': None}
except (OSError, ValueError, UnicodeError):
memory['memoryPressure'] = {'status': 'unavailable', 'metrics': None,
'reason': 'missing-unreadable-or-malformed'}
return memory
def container_identity(inspect, identifier):
state = inspect['State']
if (inspect.get('Id') != identifier or not re.fullmatch(r'[a-f0-9]{64}', identifier)
or type(state.get('Pid')) is not int or not 0 < state['Pid'] <= 2 ** 31 - 1
or state.get('Running') is not True or type(state.get('OOMKilled')) is not bool
or not isinstance(state.get('StartedAt'), str) or not state['StartedAt']
or type(inspect.get('RestartCount')) is not int or inspect['RestartCount'] < 0
or not isinstance(inspect.get('Image'), str) or not inspect['Image']):
raise ValueError('Unavailable container identity')
return (identifier, inspect['Image'], state['StartedAt'], state['Pid'], inspect['RestartCount'], state['OOMKilled'])
def cgroup_location(pid, identifier, deadline=None):
check_deadline(deadline)
mounts = read_bounded('/proc/self/mountinfo', deadline=deadline).splitlines()
# Namespace-relative/subtree mounts are deliberately unsupported: their
# root cannot authenticate a host container's complete cgroup membership.
matches = [line.split(' - ', 1) for line in mounts if ' - ' in line]
if not any(left.split()[3:5] == ['/', CGROUP_ROOT] and right.split()[0] == 'cgroup2'
for left, right in matches):
raise CgroupUnavailable('unsupported-or-unavailable-host-cgroup2-mount')
membership = read_bounded('/proc/' + str(pid) + '/cgroup', deadline=deadline)
lines = membership.splitlines()
if len(lines) != 1 or not lines[0].startswith('0::/'):
raise CgroupUnavailable('unsupported-v1-hybrid-or-malformed-membership')
relative = lines[0][3:]
parts = relative.split('/')[1:]
if (not parts or any(not p or p in ('.', '..') or not re.fullmatch(r'[A-Za-z0-9_.:-]{1,128}', p) for p in parts)
or parts[-1] not in (identifier, 'docker-' + identifier + '.scope')):
raise ValueError('Unbound cgroup membership')
root = os.path.realpath(CGROUP_ROOT)
path = os.path.realpath(CGROUP_ROOT + relative)
if root != CGROUP_ROOT or path != root + relative or not path.startswith(root + os.sep):
raise ValueError('Cgroup path escaped')
return path, membership
def process_generation(pid, deadline=None):
check_deadline(deadline)
raw = read_bounded('/proc/' + str(pid) + '/stat', deadline=deadline)
prefix, suffix = raw.rsplit(')', 1)
if prefix.split(' ', 1)[0] != str(pid) or ' (' not in prefix:
raise ValueError('Invalid process identity')
fields = suffix.split()
if len(fields) < 20:
raise ValueError('Incomplete process identity')
start = parse_counter(fields[19])
root = os.stat('/proc/' + str(pid) + '/root')
return start, root.st_dev, root.st_ino
def unavailable_cgroup(reason):
return {'version': 2, 'status': 'unavailable', 'reason': reason,
'metrics': {name: None for name in CGROUP_FILES}, 'unavailable': {name: reason for name in CGROUP_FILES}}
def collect_container(identifier, deadline=None):
check_deadline(deadline)
before = docker_get('/containers/' + identifier + '/json', deadline)
stats = docker_get('/containers/' + identifier + '/stats?stream=false&one-shot=true', deadline)
check_deadline(deadline)
result = project_container(before, stats, identifier)
cgroup = unavailable_cgroup('identity-unavailable')
try:
identity = container_identity(before, identifier)
pid = identity[3]
generation = process_generation(pid, deadline)
path, membership = cgroup_location(pid, identifier, deadline)
directory = os.stat(path)
if directory.st_dev != os.stat(CGROUP_ROOT).st_dev:
raise ValueError('Cgroup filesystem changed')
metrics, missing = {}, {}
for name in CGROUP_FILES:
check_deadline(deadline)
try:
candidate = path + '/' + name
if os.path.realpath(candidate) != candidate:
raise ValueError('Symlinked metric')
raw = read_bounded(candidate, directory.st_dev, deadline=deadline)
check_deadline(deadline)
if name == 'memory.events':
metrics[name] = parse_events(raw, ('low', 'high', 'max', 'oom', 'oom_kill'))
elif name == 'memory.swap.events':
metrics[name] = parse_events(raw, ('high', 'max', 'fail'))
elif name in ('memory.max', 'memory.swap.max'):
metrics[name] = parse_limit(raw)
elif name == 'cpu.stat':
metrics[name] = parse_cpu_throttling(raw)
else:
metrics[name] = parse_counter(raw)
except (OSError, ValueError, UnicodeError):
metrics[name] = None
missing[name] = 'missing-unreadable-or-malformed'
check_deadline(deadline)
after_path, after_membership = cgroup_location(pid, identifier, deadline)
after_directory = os.stat(path)
if (process_generation(pid, deadline) != generation or after_path != path or after_membership != membership
or (directory.st_dev, directory.st_ino) != (after_directory.st_dev, after_directory.st_ino)):
cgroup = unavailable_cgroup('process-or-cgroup-changed')
else:
cgroup = {'version': 2, 'status': 'partial' if missing else 'available',
'reason': 'some-metrics-unavailable' if missing else None,
'metrics': metrics, 'unavailable': missing,
'peakScope': 'cgroup lifetime or since external reset; not an isolated workload peak',
'provenance': 'fixed host cgroup2 mount + exact Docker ID membership + process/root generation'}
except CgroupUnavailable as error:
cgroup = unavailable_cgroup(str(error))
except (OSError, ValueError, KeyError, IndexError, UnicodeError):
cgroup = unavailable_cgroup('identity-process-or-cgroup-unavailable-or-unsupported')
# Even failed cgroup observations are bracketed by a fresh Docker inspect.
check_deadline(deadline)
after = docker_get('/containers/' + identifier + '/json', deadline)
check_deadline(deadline)
try:
stable = container_identity(before, identifier) == container_identity(after, identifier)
except (ValueError, KeyError, TypeError):
stable = False
if not stable:
cgroup = unavailable_cgroup('container-identity-changed-or-unavailable')
limits_stable = resource_limits_stable(before, after) if stable else None
if stable and limits_stable is not True and cgroup['status'] != 'unavailable':
for key in ('memory.max', 'memory.swap.max'):
cgroup['metrics'][key] = None
cgroup['unavailable'][key] = 'container-resource-limits-changed-or-unavailable'
cgroup['status'] = 'partial'
cgroup['reason'] = 'some-metrics-unavailable'
result.update({'resourceLimitsStableDuringSample': limits_stable,
'pid': before['State'].get('Pid'), 'identityStableDuringSample': stable,
'runningAfter': after.get('State', {}).get('Running'),
'oomKilledAfter': after.get('State', {}).get('OOMKilled'), 'cgroupV2': cgroup})
return result
def docker_get(path, deadline=None):
deadline = time.monotonic() + 5 if deadline is None else deadline
check_deadline(deadline)
connection = http.client.HTTPConnection('localhost', timeout=min(5, deadline - time.monotonic()))
owned_socket = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
connection.sock = owned_socket
# A socket timeout alone permits an endless body that drips bytes. The
# watchdog owns precisely this request's socket, including after HTTP's
# Connection:close detaches it from HTTPConnection into HTTPResponse.
def expire():
try:
owned_socket.shutdown(socket.SHUT_RDWR)
except OSError:
pass
finally:
owned_socket.close()
watchdog = threading.Timer(max(0, deadline - time.monotonic()), expire)
watchdog.daemon = True
watchdog_started = False
response = None
try:
watchdog.start()
watchdog_started = True
owned_socket.settimeout(min(5, max(0.001, deadline - time.monotonic())))
check_deadline(deadline)
owned_socket.connect('/var/run/docker.sock')
check_deadline(deadline)
connection.request('GET', '/v1.45' + path, headers={'Connection': 'close'})
response = connection.getresponse()
raw = response.read(MAX_RESPONSE + 1)
check_deadline(deadline)
if response.status != 200 or len(raw) > MAX_RESPONSE:
raise ValueError('Docker observation unavailable')
result = json.loads(raw)
check_deadline(deadline)
return result
except Exception:
check_deadline(deadline)
raise
finally:
watchdog.cancel()
if watchdog_started:
watchdog.join()
try:
if response is not None:
response.close()
finally:
try:
connection.close()
finally:
owned_socket.close()
def nonnegative(value):
if type(value) is not int or value < 0:
raise ValueError('Invalid numeric observation')
return value
def parse_host(meminfo, vmstat):
memory = {}
for line in meminfo.splitlines():
fields = line.split()
if fields and fields[0] in ('MemTotal:', 'MemAvailable:', 'SwapTotal:', 'SwapFree:'):
if len(fields) != 3 or fields[2] != 'kB':
raise ValueError('Invalid meminfo units')
memory[fields[0][:-1] + 'Bytes'] = nonnegative(int(fields[1])) * 1024
if len(memory) != 4 or memory['MemAvailableBytes'] > memory['MemTotalBytes'] or memory['SwapFreeBytes'] > memory['SwapTotalBytes']:
raise ValueError('Incomplete host memory observation')
for line in vmstat.splitlines():
fields = line.split()
if fields and fields[0] in ('pswpin', 'pswpout', 'oom_kill'):
memory[fields[0]] = nonnegative(int(fields[1]))
return memory
def project_container(inspect, stats, identifier):
if inspect.get('Id') != identifier or stats.get('id') != identifier:
raise ValueError('Pinned container changed')
state = inspect['State']
memory = stats['memory_stats']
usage = nonnegative(memory['usage'])
inactive = memory.get('stats', {}).get('inactive_file')
if inactive is not None:
inactive = nonnegative(inactive)
if inactive > usage:
raise ValueError('Invalid cache observation')
return {
'id': identifier, 'imageId': inspect['Image'],
'startedAt': state['StartedAt'], 'running': state['Running'] is True,
'oomKilled': state.get('OOMKilled'), 'restartCount': inspect.get('RestartCount'),
'usageIncludingCacheBytes': usage, 'inactiveFileBytes': inactive,
'usageMinusInactiveFileBytes': usage - inactive if inactive is not None else None,
'configuredLimitBytes': nonnegative(inspect['HostConfig']['Memory']),
'configuredMemorySwapBytes': configured_memory_swap(inspect),
'configuredMemorySwapStatus': 'available' if configured_memory_swap(inspect) is not None else 'unavailable',
'reportedLimitBytes': nonnegative(memory['limit']),
'swapUsageBytes': memory.get('swap_usage'),
}
def summarize(samples):
if not samples:
raise ValueError('No observations')
baseline = {c['id']: (c['imageId'], c['startedAt'], c['restartCount'], c.get('pid'), c.get('oomKilled')) for c in samples[0]['containers']}
stable = all({c['id']: (c['imageId'], c['startedAt'], c['restartCount'], c.get('pid'), c.get('oomKilled')) for c in sample['containers']} == baseline
and all(c.get('identityStableDuringSample', True) for c in sample['containers']) for sample in samples)
counters = {}
for field in ('pswpin', 'pswpout', 'oom_kill'):
first = samples[0]['host'].get(field)
last = samples[-1]['host'].get(field)
counters[field + 'Delta'] = last - first if first is not None and last is not None and last >= first else None
return {
'samples': len(samples), 'elapsedMs': samples[-1]['elapsedMs'],
'hostMemTotalBytes': samples[0]['host']['MemTotalBytes'],
'minHostAvailableBytes': min(s['host']['MemAvailableBytes'] for s in samples),
'maxObservedContainerUsageIncludingCacheBytes': max(sum(c['usageIncludingCacheBytes'] for c in s['containers']) for s in samples),
'containerIdentityAndStartStable': stable,
'allObservedRunning': all(c['running'] is True and c.get('runningAfter') is True
and c.get('identityStableDuringSample') is True for s in samples for c in s['containers']),
'anyObservedOomKilled': any(c['oomKilled'] is True or c.get('oomKilledAfter') is True for s in samples for c in s['containers']),
'allCgroupV2MetricsAvailable': all(s['containers'] and all(c.get('cgroupV2', {}).get('status') == 'available' for c in s['containers']) for s in samples),
**counters,
'allResourceLimitsStableDuringSample': all(c.get('resourceLimitsStableDuringSample') is True for s in samples for c in s['containers']) and all(s['containers'] for s in samples),
'allHostMemoryPressureAvailable': all(s['host'].get('memoryPressure', {}).get('status') == 'available' for s in samples),
'acceptancePassed': False,
'scope': 'sampled evidence only; correlate with workload, limits, errors and latency; unsampled peaks are unknown',
}
def main():
parser = argparse.ArgumentParser(description=__doc__)
parser.add_argument('--daemon-id', required=True)
parser.add_argument('--container', action='append', required=True)
parser.add_argument('--samples', type=int, default=30)
parser.add_argument('--interval', type=float, default=1)
parser.add_argument('--output', required=True)
args = parser.parse_args()
if os.name != 'posix' or not hasattr(socket, 'AF_UNIX'):
parser.error('Run on the Linux Docker host')
if not re.fullmatch(r'[A-Za-z0-9-]{8,128}', args.daemon_id) or not 1 <= len(args.container) <= 16 or len(set(args.container)) != len(args.container) or any(not re.fullmatch(r'[a-f0-9]{64}', c) for c in args.container):
parser.error('Pin exact daemon ID and 1..16 distinct full container IDs')
if not 1 <= args.samples <= 300 or not 0.2 <= args.interval <= 10 or args.samples * args.interval > 300:
parser.error('Observation bound: 300 samples / 300 seconds')
started = time.monotonic()
deadline = started + 360
if docker_get('/info', deadline).get('ID') != args.daemon_id:
raise ValueError('Pinned Docker daemon changed')
samples = []
for index in range(args.samples):
check_deadline(deadline)
host = collect_host(deadline)
containers = []
for identifier in args.container:
check_deadline(deadline)
containers.append(collect_container(identifier, deadline))
check_deadline(deadline)
samples.append({'elapsedMs': round((time.monotonic() - started) * 1000), 'host': host, 'containers': containers})
if index + 1 < args.samples:
time.sleep(min(args.interval, max(0, deadline - time.monotonic())))
check_deadline(deadline)
if docker_get('/info', deadline).get('ID') != args.daemon_id:
raise ValueError('Pinned Docker daemon changed')
result = {'schemaVersion': 3, 'readOnlyDockerAndProc': True, 'serviceChanges': 0, 'summary': summarize(samples), 'observations': samples}
encoded = json.dumps(result, indent=2) + '\n'
check_deadline(deadline)
# Refuse to replace an existing evidence file. Never dump Docker env/logs.
with open(args.output, 'x', encoding='utf-8') as handle:
check_deadline(deadline)
handle.write(encoded)
check_deadline(deadline)
print(json.dumps(result['summary']))
if __name__ == '__main__':
try:
main()
except Exception as error:
print(json.dumps({'status': 'observation-refused', 'errorClass': type(error).__name__, 'serviceChanges': 0}))
raise SystemExit(1)
@@ -0,0 +1,384 @@
import copy
import importlib.util
from pathlib import Path
import os
import tempfile
import io
import sys
import threading
import time
import unittest
from unittest.mock import patch
spec = importlib.util.spec_from_file_location('memory_measure', Path(__file__).with_name('measure-memory.py'))
measure = importlib.util.module_from_spec(spec)
spec.loader.exec_module(measure)
class MemoryEvidenceTests(unittest.TestCase):
def test_host_units_and_missing_fields(self):
raw = 'MemTotal: 2048 kB\nMemAvailable: 1024 kB\nSwapTotal: 4096 kB\nSwapFree: 3072 kB\n'
parsed = measure.parse_host(raw, 'pswpin 20\npswpout 30\noom_kill 1\n')
self.assertEqual(parsed['MemTotalBytes'], 2048 * 1024)
self.assertEqual(parsed['pswpin'], 20)
for invalid in (raw.replace('kB', 'MB'), raw.replace('MemAvailable: 1024 kB\n', ''), raw.replace('MemAvailable: 1024', 'MemAvailable: 4096')):
with self.assertRaises(ValueError):
measure.parse_host(invalid, '')
def test_cache_and_no_configured_limit_are_distinct(self):
item = {'Id': 'a' * 64, 'Image': 'sha256:fixed', 'State': {'StartedAt': 't0', 'Running': True}, 'HostConfig': {'Memory': 0}}
stats = {'id': 'a' * 64, 'memory_stats': {'usage': 100, 'limit': 2048, 'stats': {'inactive_file': 30}}}
result = measure.project_container(item, stats, 'a' * 64)
self.assertEqual(result['usageIncludingCacheBytes'], 100)
self.assertEqual(result['usageMinusInactiveFileBytes'], 70)
self.assertEqual(result['configuredLimitBytes'], 0)
with self.assertRaises(ValueError):
measure.project_container(item, stats, 'b' * 64)
def test_simultaneous_peak_is_not_sum_of_separate_peaks(self):
host = {'MemTotalBytes': 2048, 'MemAvailableBytes': 1024, 'pswpin': 3, 'pswpout': 5, 'oom_kill': 1}
def container(identifier, usage):
return {'id': identifier, 'imageId': 'i', 'startedAt': 't0', 'restartCount': 0, 'running': True, 'oomKilled': False, 'usageIncludingCacheBytes': usage}
samples = [{'elapsedMs': 0, 'host': host, 'containers': [container('a', 100), container('b', 20)]},
{'elapsedMs': 1000, 'host': {**host, 'pswpin': 4}, 'containers': [container('a', 20), container('b', 100)]}]
result = measure.summarize(samples)
self.assertEqual(result['maxObservedContainerUsageIncludingCacheBytes'], 120)
self.assertEqual(result['pswpinDelta'], 1)
self.assertTrue(result['containerIdentityAndStartStable'])
self.assertFalse(result['acceptancePassed'])
changed = copy.deepcopy(samples)
changed[-1]['containers'][0]['startedAt'] = 't1'
changed[-1]['containers'][1]['oomKilled'] = True
self.assertFalse(measure.summarize(changed)['containerIdentityAndStartStable'])
self.assertTrue(measure.summarize(changed)['anyObservedOomKilled'])
def test_counter_rollback_is_unknown(self):
samples = [{'elapsedMs': i, 'host': {'MemTotalBytes': 2, 'MemAvailableBytes': 1, 'pswpin': value}, 'containers': []} for i, value in enumerate([100, 2])]
self.assertIsNone(measure.summarize(samples)['pswpinDelta'])
class CgroupEvidenceTests(unittest.TestCase):
identifier = 'a' * 64
path = '/sys/fs/cgroup/system.slice/docker-' + identifier + '.scope'
def setUp(self):
self.before = {'Id': self.identifier, 'Image': 'sha256:fixed', 'RestartCount': 0,
'State': {'StartedAt': 'original-start', 'Pid': 123, 'Running': True, 'OOMKilled': False},
'HostConfig': {'Memory': 512, 'MemorySwap': 1024}}
self.stats = {'id': self.identifier, 'memory_stats': {'usage': 100, 'limit': 512, 'stats': {'inactive_file': 30}}}
self.files = {
'/proc/self/mountinfo': '1 0 0:1 / /sys/fs/cgroup rw - cgroup2 cgroup rw\n',
'/proc/123/cgroup': '0::/system.slice/docker-' + self.identifier + '.scope\n',
'/proc/123/stat': '123 (a name with ) parentheses) S ' + ' '.join(['0'] * 18 + ['987'] + ['0'] * 8),
self.path + '/memory.max': '512\n', self.path + '/memory.swap.max': '512\n',
self.path + '/cpu.stat': 'nr_periods 10\nnr_throttled 2\nthrottled_usec 1234\n',
self.path + '/memory.current': '100\n', self.path + '/memory.peak': '200\n',
self.path + '/memory.events': 'low 0\nhigh 1\nmax 2\noom 3\noom_kill 4\n',
self.path + '/memory.swap.current': '5\n', self.path + '/memory.swap.peak': '6\n',
self.path + '/memory.swap.events': 'high 0\nmax 2\nfail 1\n',
}
def observe(self, after=None, reader=None, resolver=None, statter=None):
def read(path, _expected_device=None, deadline=None):
if path not in self.files:
raise FileNotFoundError()
return self.files[path]
queries = []
def get(path, deadline=None):
queries.append(path)
if '/stats?' in path:
return self.stats
return self.before if len(queries) == 1 else (after or self.before)
with patch.object(measure, 'docker_get', side_effect=get), \
patch.object(measure, 'read_bounded', side_effect=reader or read), \
patch.object(measure.os.path, 'realpath', side_effect=resolver or (lambda path: path)), \
patch.object(measure.os, 'stat', side_effect=statter or (lambda path: os.stat_result((0, 7, 8, 0, 0, 0, 0, 0, 0, 0)))), \
patch.object(measure.os, 'sep', '/'):
result = measure.collect_container(self.identifier)
self.assertEqual(queries, ['/containers/' + self.identifier + '/json',
'/containers/' + self.identifier + '/stats?stream=false&one-shot=true',
'/containers/' + self.identifier + '/json'])
return result
def test_exact_identity_metrics_and_docker_cache_are_distinct(self):
result = self.observe()
self.assertTrue(result['identityStableDuringSample'])
self.assertEqual(result['usageMinusInactiveFileBytes'], 70)
self.assertEqual(result['cgroupV2']['status'], 'available')
self.assertEqual(result['cgroupV2']['metrics']['memory.current'], 100)
self.assertEqual(result['cgroupV2']['metrics']['memory.peak'], 200)
self.assertEqual(result['cgroupV2']['metrics']['memory.events']['oom_kill'], 4)
self.assertEqual(result['cgroupV2']['metrics']['memory.swap.current'], 5)
self.assertEqual(result['cgroupV2']['metrics']['memory.swap.peak'], 6)
self.assertEqual(result['cgroupV2']['metrics']['memory.swap.events']['fail'], 1)
def test_optional_swap_peak_missing_is_unknown_not_zero(self):
del self.files[self.path + '/memory.swap.peak']
result = self.observe()['cgroupV2']
self.assertEqual(result['status'], 'partial')
self.assertIsNone(result['metrics']['memory.swap.peak'])
self.assertEqual(result['metrics']['memory.peak'], 200)
self.assertIn('memory.swap.peak', result['unavailable'])
def test_each_missing_metric_is_explicit_unknown(self):
for name in measure.CGROUP_FILES:
with self.subTest(name=name):
value = self.files.pop(self.path + '/' + name)
result = self.observe()['cgroupV2']
self.assertIsNone(result['metrics'][name])
self.assertEqual(result['status'], 'partial')
self.files[self.path + '/' + name] = value
def test_malformed_metrics_never_coerce_to_zero(self):
for value in ('-1\n', '+1\n', ' 1\n', '01\n', '1\n2\n', '18446744073709551616\n', 'max\n'):
with self.subTest(value=value):
self.files[self.path + '/memory.current'] = value
self.assertIsNone(self.observe()['cgroupV2']['metrics']['memory.current'])
for value in ('oom 1\noom 2\n', 'low 0\nhigh -1\nmax 0\noom 0\noom_kill 0\n', 'low 0\n', 'low 0\n'):
with self.subTest(value=value):
self.files[self.path + '/memory.events'] = value
self.assertIsNone(self.observe()['cgroupV2']['metrics']['memory.events'])
def test_docker_identity_drift_invalidates_entire_cgroup_sample(self):
for field, value in [('Pid', 124), ('StartedAt', 'new'), ('OOMKilled', True), ('Running', False)]:
after = copy.deepcopy(self.before); after['State'][field] = value
result = self.observe(after)
self.assertFalse(result['identityStableDuringSample'])
self.assertTrue(all(value is None for value in result['cgroupV2']['metrics'].values()))
for field, value in [('Id', 'b' * 64), ('RestartCount', 1), ('Image', 'sha256:new')]:
after = copy.deepcopy(self.before); after[field] = value
self.assertEqual(self.observe(after)['cgroupV2']['status'], 'unavailable')
def test_process_reuse_rootfs_swap_and_membership_move_are_unknown(self):
for changed in ('/proc/123/stat', '/proc/123/cgroup'):
count = 0
def read(path, _expected_device=None, deadline=None):
nonlocal count
if path == changed:
count += 1
if count == 2:
return self.files[path].replace('987', '988') if path.endswith('/stat') else '0::/docker/' + 'b' * 64 + '\n'
return self.files[path]
self.assertEqual(self.observe(reader=read)['cgroupV2']['status'], 'unavailable')
count = 0
def statter(path):
nonlocal count
if path == '/proc/123/root':
count += 1
return os.stat_result((0, count, 8, 0, 0, 0, 0, 0, 0, 0))
self.assertEqual(self.observe(statter=statter)['cgroupV2']['status'], 'unavailable')
def test_unsupported_v1_namespace_subtree_and_unbound_membership(self):
original = self.files['/proc/123/cgroup']
for value in ('2:memory:/docker/' + self.identifier + '\n', '0::/\n', '0::/docker/' + 'b' * 64 + '\n',
'0::/docker/' + self.identifier + '/../other\n', original.rstrip() + '/child\n', original + original):
self.files['/proc/123/cgroup'] = value
self.assertEqual(self.observe()['cgroupV2']['status'], 'unavailable')
self.files['/proc/123/cgroup'] = original
self.files['/proc/self/mountinfo'] = '1 0 0:1 /subtree /sys/fs/cgroup rw - cgroup2 cgroup rw\n'
self.assertEqual(self.observe()['cgroupV2']['status'], 'unavailable')
def test_symlink_outside_or_to_other_container_is_refused(self):
for destination in ('/etc', '/sys/fs/cgroup/docker/' + 'b' * 64):
self.assertEqual(self.observe(resolver=lambda path: destination if path == self.path else path)['cgroupV2']['status'], 'unavailable')
result = self.observe(resolver=lambda path: '/etc/passwd' if path.endswith('/memory.current') else path)
self.assertIsNone(result['cgroupV2']['metrics']['memory.current'])
def test_cgroup_directory_replacement_or_nested_filesystem_is_unknown(self):
for nested in (False, True):
calls = 0
def statter(path):
nonlocal calls
if path == self.path:
calls += 1
inode = 8 if path == self.path and calls > 1 else 7
device = 9 if nested and path == self.path else 8
return os.stat_result((0, inode, device, 0, 0, 0, 0, 0, 0, 0))
result = self.observe(statter=statter)['cgroupV2']
self.assertEqual(result['status'], 'unavailable')
self.assertTrue(all(value is None for value in result['metrics'].values()))
def test_pid_reuse_unreadable_process_and_after_oom_stay_unknown(self):
after = copy.deepcopy(self.before); after['State']['OOMKilled'] = True
result = self.observe(after)
host = {'MemTotalBytes': 2048, 'MemAvailableBytes': 1024}
summary = measure.summarize([{'elapsedMs': 0, 'host': host, 'containers': [result]}])
self.assertTrue(summary['anyObservedOomKilled'])
self.assertFalse(summary['containerIdentityAndStartStable'])
self.assertFalse(summary['allCgroupV2MetricsAvailable'])
self.assertFalse(summary['acceptancePassed'])
def read(path, _expected_device=None, deadline=None):
if path == '/proc/123/stat':
raise PermissionError()
return self.files[path]
self.assertEqual(self.observe(reader=read)['cgroupV2']['status'], 'unavailable')
def test_real_temp_files_are_bounded_and_strictly_ascii(self):
with tempfile.TemporaryDirectory(dir=Path(__file__).resolve().parent) as directory:
path = Path(directory) / 'memory.current'
path.write_bytes(b'123\n')
self.assertEqual(measure.parse_counter(measure.read_bounded(path)), 123)
with self.assertRaises(ValueError):
measure.read_bounded(path, path.stat().st_dev + 1)
for raw in (b'0' * (measure.MAX_FILE_BYTES + 1), b'0\n' * (measure.MAX_FILE_LINES + 1), b'\xff'):
path.write_bytes(raw)
with self.assertRaises((ValueError, UnicodeError)):
measure.read_bounded(path)
def test_after_stopped_or_unknown_identity_cannot_summarize_running(self):
for value in (False, None):
after = copy.deepcopy(self.before); after['State']['Running'] = value
result = self.observe(after)
self.assertIs(result['runningAfter'], value)
summary = measure.summarize([{'elapsedMs': 0, 'host': {'MemTotalBytes': 2048, 'MemAvailableBytes': 1024}, 'containers': [result]}])
self.assertFalse(summary['allObservedRunning'])
result = self.observe()
summary = measure.summarize([{'elapsedMs': 0, 'host': {'MemTotalBytes': 2048, 'MemAvailableBytes': 1024}, 'containers': [result]}])
self.assertTrue(summary['allObservedRunning'])
def test_event_future_counters_retained_but_duplicates_and_overflow_refuse(self):
self.assertEqual(measure.parse_events('high 0\nmax 1\nfail 2\nfuture 3\n', ('high', 'max', 'fail'))['future'], 3)
for value in ('x 1\nx 2\n', 'x 18446744073709551616\n', ''.join('x_' + str(i) + ' 0\n' for i in range(33))):
with self.assertRaises(ValueError):
measure.parse_events(value, ())
class DeadlineEvidenceTests(unittest.TestCase):
def test_expired_http_never_opens_socket_or_schedules_watchdog(self):
with patch.object(measure.time, 'monotonic', return_value=10), \
patch.object(measure.socket, 'socket') as own_socket, patch.object(measure.threading, 'Timer') as timer:
with self.assertRaises(measure.ObservationDeadlineExceeded):
measure.docker_get('/info', 10)
own_socket.assert_not_called()
timer.assert_not_called()
def test_expired_container_read_never_starts_io(self):
with patch.object(measure.time, 'monotonic', return_value=10), patch.object(measure, 'docker_get') as get:
with self.assertRaises(measure.ObservationDeadlineExceeded):
measure.collect_container('a' * 64, 10)
get.assert_not_called()
def test_near_cutoff_after_stats_stops_before_proc_or_more_gets(self):
clock, queries = [9.9], []
def get(path, deadline):
self.assertEqual(deadline, 10)
queries.append(path)
if len(queries) == 2:
clock[0] = 10
return {}
with patch.object(measure.time, 'monotonic', side_effect=lambda: clock[0]), \
patch.object(measure, 'docker_get', side_effect=get), patch.object(measure, 'read_bounded') as read:
with self.assertRaises(measure.ObservationDeadlineExceeded):
measure.collect_container('a' * 64, 10)
self.assertEqual(len(queries), 2)
read.assert_not_called()
def test_metric_deadline_is_not_downgraded_to_partial_or_followed_by_io(self):
fixture = CgroupEvidenceTests(); fixture.setUp()
clock, reads, queries = [9.9], [], []
def read(path, _device=None, deadline=None):
self.assertEqual(deadline, 10)
reads.append(path)
if path.endswith('/memory.current'):
clock[0] = 10
return fixture.files[path]
def get(path, deadline):
queries.append(path)
return fixture.stats if '/stats?' in path else fixture.before
with patch.object(measure.time, 'monotonic', side_effect=lambda: clock[0]), \
patch.object(measure, 'docker_get', side_effect=get), patch.object(measure, 'read_bounded', side_effect=read), \
patch.object(measure.os.path, 'realpath', side_effect=lambda path: path), \
patch.object(measure.os, 'stat', return_value=os.stat_result((0, 7, 8, 0, 0, 0, 0, 0, 0, 0))), \
patch.object(measure.os, 'sep', '/'):
with self.assertRaises(measure.ObservationDeadlineExceeded):
measure.collect_container(fixture.identifier, 10)
self.assertEqual(reads[-1], fixture.path + '/memory.current')
self.assertEqual(len(queries), 2)
def test_final_daemon_check_crosses_deadline_without_writing_report(self):
clock, calls = [0], []
def get(path, deadline):
self.assertEqual(deadline, 360)
calls.append(path)
if len(calls) == 2:
clock[0] = 361
return {'ID': 'owned-daemon'}
host = 'MemTotal: 2048 kB\nMemAvailable: 1024 kB\nSwapTotal: 0 kB\nSwapFree: 0 kB\n'
args = ['measure-memory.py', '--daemon-id', 'owned-daemon', '--container', 'a' * 64, '--samples', '1', '--output', 'NEVER_CREATED']
with patch.object(sys, 'argv', args), patch.object(measure.os, 'name', 'posix'), \
patch.object(measure.socket, 'AF_UNIX', 1, create=True), \
patch.object(measure.time, 'monotonic', side_effect=lambda: clock[0]), patch.object(measure, 'docker_get', side_effect=get), \
patch.object(measure, 'read_bounded', side_effect=[host, '', 'some avg10=0.00 avg60=0.00 avg300=0.00 total=0\nfull avg10=0.00 avg60=0.00 avg300=0.00 total=0\n']), patch.object(measure, 'collect_container', return_value={}), \
patch.object(measure, 'summarize', return_value={}), patch('builtins.open') as output:
with self.assertRaises(measure.ObservationDeadlineExceeded):
measure.main()
output.assert_not_called()
def test_slow_body_absolute_deadline_closes_only_own_socket_and_joins(self):
# A real HTTPResponse/BufferedReader and real watchdog timer, with an
# in-memory socket transport. No AF_UNIX availability or local network
# permissions are needed on Windows. Inactivity timeouts never fire:
# the body keeps producing bytes until shutdown interrupts the read.
class DripBody(io.RawIOBase):
def __init__(self, stopped):
self.header = b'HTTP/1.1 200 OK\r\nContent-Length: 1000\r\nConnection: close\r\n\r\n'
self.stopped = stopped
self.body_reads = 0
def readable(self):
return True
def readinto(self, buffer):
if self.header:
value, self.header = self.header[:len(buffer)], self.header[len(buffer):]
buffer[:len(value)] = value
return len(value)
if self.stopped.wait(0.01):
return 0
self.body_reads += 1
buffer[0] = 32
return 1
class SocketTransport:
def __init__(self):
self.stopped = threading.Event()
self.body = DripBody(self.stopped)
self.closed = False
self.request = b''
def connect(self, path):
self.path = path
def settimeout(self, value):
self.timeout = value
def sendall(self, value):
self.request += value
def makefile(self, _mode):
return io.BufferedReader(self.body)
def shutdown(self, _how):
self.stopped.set()
def close(self):
self.closed = True
client, unrelated = SocketTransport(), SocketTransport()
started = time.monotonic()
original_timer = measure.threading.Timer
timers = []
def timer(*args, **kwargs):
value = original_timer(*args, **kwargs)
timers.append(value)
return value
with patch.object(measure.socket, 'AF_UNIX', 1, create=True), \
patch.object(measure.socket, 'socket', return_value=client), \
patch.object(measure.threading, 'Timer', side_effect=timer):
with self.assertRaises(measure.ObservationDeadlineExceeded):
measure.docker_get('/info', started + 0.12)
self.assertLess(time.monotonic() - started, 1.5)
self.assertTrue(client.closed)
self.assertTrue(client.stopped.is_set())
self.assertGreater(client.body.body_reads, 0)
self.assertFalse(unrelated.closed)
self.assertFalse(unrelated.stopped.is_set())
self.assertEqual(client.path, '/var/run/docker.sock')
self.assertIn(b'GET /v1.45/info ', client.request)
self.assertEqual(len(timers), 1)
self.assertFalse(timers[0].is_alive(), 'No deadline callback can perform late I/O after return')
if __name__ == '__main__':
unittest.main()
@@ -0,0 +1,63 @@
/** Local smoke preparation only: owns a new embedded PG and one compiled child.
* Does not represent frontend/portal/Docker/whole-installation acceptance. */
import assert from 'node:assert/strict'
import fs from 'node:fs/promises'
import { createRequire } from 'node:module'
import { tmpdir } from 'node:os'
import { join,resolve,dirname,basename } from 'node:path'
import { fileURLToPath } from 'node:url'
import { createServer } from 'node:net'
import { randomBytes,createHash } from 'node:crypto'
import { spawn } from 'node:child_process'
const root=resolve(dirname(fileURLToPath(import.meta.url)),'../../..')
assert(process.argv.length>=3&&process.argv.length<=5,'Supply a unique evidence name, optional compiled output and delayed-approval scenario')
const delayedApproval=process.argv[4]==='delayed-approval'
assert(process.argv[4]===undefined||delayedApproval,'Unknown isolated scenario')
const name=process.argv[2];assert(/^[a-z0-9][a-z0-9-]{3,99}$/.test(name),'Invalid evidence name')
const outputName=process.argv[3]??'dist-installation-memory'
assert(/^dist-installation-memory(?:-[a-z0-9-]{3,64})?$/.test(outputName),'Invalid compiled output name')
const prefix=resolve(root,'tmp',name), statusPath=prefix+'-status.json'
const exclusive=await fs.open(statusPath,'wx');await exclusive.close()
const state={scope:'local-backend-coload-only',wholeInstallation2GbAccepted:false,compiledOutputName:outputName,delayedHumanApprovalScenario:delayedApproval,status:'starting',nodeExitCode:null,sourcePinsUnchanged:false,cleanupConfirmed:false,startedAt:new Date().toISOString(),wrapperPid:process.pid}
const save=()=>fs.writeFile(statusPath,JSON.stringify(state,null,2)+'\n');await save()
const require=createRequire(new URL('../../package.json',import.meta.url)),EmbeddedPostgres=require('embedded-postgres').default
const directory=await fs.mkdtemp(join(tmpdir(),'mm-mcp-test-')),server=createServer()
await new Promise(done=>server.listen(0,'127.0.0.1',done));const port=server.address().port;await new Promise(done=>server.close(done))
const password=randomBytes(24).toString('hex')
const postgres=new EmbeddedPostgres({databaseDir:join(directory,'pg'),port,user:'mcp_test',password,persistent:true,createPostgresUser:false,
initdbFlags:['--encoding=UTF8'],postgresFlags:['-h','127.0.0.1'],onLog:()=>{},onError:()=>{}})
const output=resolve(root,'backend',outputName),entry=resolve(output,'services/installation-co-load.owned.test.js')
async function pins() {
const rows=[]
async function walk(dir){for(const item of await fs.readdir(dir,{withFileTypes:true})){const file=join(dir,item.name);if(item.isDirectory())await walk(file);else rows.push({path:file.slice(root.length+1).replaceAll('\\','/'),sha256:createHash('sha256').update(await fs.readFile(file)).digest('hex')})}}
await walk(output);return rows.sort((a,b)=>a.path.localeCompare(b.path))
}
const before=await pins();await fs.writeFile(prefix+'-pins.json',JSON.stringify({before},null,2)+'\n')
let log,timer
try {
await postgres.initialise();await postgres.start()
log=await fs.open(prefix+'.log','wx')
const retained=Object.fromEntries(['SystemRoot','WINDIR','TEMP','TMP','LANG','LC_ALL'].filter(key=>process.env[key]).map(key=>[key,process.env[key]]))
const child=spawn(process.execPath,['--max-old-space-size=192','--max-semi-space-size=16',entry],{cwd:directory,windowsHide:true,stdio:['ignore',log.fd,log.fd],env:{...retained,
NODE_OPTIONS:'',NODE_ENV:'test',PORT:'8000',AUTH_ISSUER:'https://auth.shnt.top',AUTH_PORTAL_URL:'http://localhost:5175',SQLITE_IMPORT_FULL_HISTORY:'false',
MM_FLOW_READ_RESOURCE_PROFILE:'constrained',DATABASE_URL:`postgresql://mcp_test:${password}@127.0.0.1:${port}/postgres`,DATABASE_PATH:join(directory,'no-legacy.sqlite'),
MCP_STATE_DIR:join(directory,'state'),AUTH_REQUIRED:'true',AUTH_JWT_SECRET:randomBytes(32).toString('hex'),MCP_PUBLIC_URL:'http://localhost:8000/mcp',MMAPP_OWN_INSTALLATION_COLOAD:'true',MMAPP_OWN_COLOAD_DELAYED_APPROVAL:String(delayedApproval)}})
state.status='running';state.childPid=child.pid;await save()
timer=setTimeout(()=>{state.outerTimedOut=true;child.kill()},300000)
state.nodeExitCode=await new Promise((done,reject)=>{child.once('error',reject);child.once('exit',code=>done(code??1))})
clearTimeout(timer);await log.close();log=undefined
const after=await pins();state.sourcePinsUnchanged=JSON.stringify(before)===JSON.stringify(after)
await fs.writeFile(prefix+'-pins.json',JSON.stringify({before,after,unchanged:state.sourcePinsUnchanged},null,2)+'\n')
process.exitCode=state.nodeExitCode===0&&state.sourcePinsUnchanged?0:1
}catch(error){state.failure=error instanceof Error?error.name:'UnknownError';process.exitCode=1}
finally {
if(timer)clearTimeout(timer);if(log)await log.close()
try{
if(postgres.process?.exitCode===null)await postgres.stop()
const actual=await fs.realpath(directory),parent=await fs.realpath(tmpdir())
assert.equal(dirname(actual),parent);assert.match(basename(actual),/^mm-mcp-test-[A-Za-z0-9-]+$/);assert.equal((await fs.lstat(directory)).isSymbolicLink(),false)
await fs.rm(actual,{recursive:true,force:true,maxRetries:20,retryDelay:100});state.cleanupConfirmed=true
}catch{state.cleanupConfirmed=false;process.exitCode=1}
state.status='terminal';state.finishedAt=new Date().toISOString();await save();console.log(JSON.stringify(state))
}
if(state.cleanupConfirmed)process.exit(process.exitCode??1)
@@ -1,3 +1,4 @@
/* eslint-disable @typescript-eslint/no-require-imports -- standalone CommonJS Playwright script */
const fs = require('fs'), assert = require('node:assert/strict'), { createHash } = require('crypto');
const { chromium, expect } = require('@playwright/test');
(async () => {
@@ -0,0 +1,81 @@
{
"schemaVersion": 2,
"readOnlyDockerAndProc": true,
"serviceChanges": 0,
"summary": {
"samples": 1,
"elapsedMs": 0,
"hostMemTotalBytes": 2097152,
"minHostAvailableBytes": 1048576,
"maxObservedContainerUsageIncludingCacheBytes": 100,
"containerIdentityAndStartStable": true,
"allObservedRunning": true,
"anyObservedOomKilled": false,
"allCgroupV2MetricsAvailable": true,
"pswpinDelta": 0,
"pswpoutDelta": 0,
"oom_killDelta": 0,
"acceptancePassed": false,
"scope": "sampled evidence only; correlate with workload, limits, errors and latency; unsampled peaks are unknown"
},
"observations": [
{
"elapsedMs": 0,
"host": {
"MemTotalBytes": 2097152,
"MemAvailableBytes": 1048576,
"SwapTotalBytes": 4194304,
"SwapFreeBytes": 3145728,
"pswpin": 20,
"pswpout": 30,
"oom_kill": 0
},
"containers": [
{
"id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"imageId": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"startedAt": "2026-11-02T00:00:00Z",
"running": true,
"oomKilled": false,
"restartCount": 0,
"usageIncludingCacheBytes": 100,
"inactiveFileBytes": 30,
"usageMinusInactiveFileBytes": 70,
"configuredLimitBytes": 512,
"reportedLimitBytes": 2048,
"swapUsageBytes": null,
"pid": 123,
"identityStableDuringSample": true,
"runningAfter": true,
"oomKilledAfter": false,
"cgroupV2": {
"version": 2,
"status": "available",
"reason": null,
"metrics": {
"memory.current": 100,
"memory.peak": 200,
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0
},
"memory.swap.current": 0,
"memory.swap.peak": 0,
"memory.swap.events": {
"high": 0,
"max": 0,
"fail": 0
}
},
"unavailable": {},
"peakScope": "cgroup lifetime or since external reset; not an isolated workload peak",
"provenance": "fixed host cgroup2 mount + exact Docker ID membership + process/root generation"
}
}
]
}
]
}
@@ -0,0 +1,111 @@
{
"schemaVersion": 3,
"readOnlyDockerAndProc": true,
"serviceChanges": 0,
"summary": {
"samples": 1,
"elapsedMs": 0,
"hostMemTotalBytes": 2097152,
"minHostAvailableBytes": 1048576,
"maxObservedContainerUsageIncludingCacheBytes": 100,
"containerIdentityAndStartStable": true,
"allObservedRunning": true,
"anyObservedOomKilled": false,
"allCgroupV2MetricsAvailable": true,
"pswpinDelta": 0,
"pswpoutDelta": 0,
"oom_killDelta": 0,
"allResourceLimitsStableDuringSample": true,
"allHostMemoryPressureAvailable": true,
"acceptancePassed": false,
"scope": "sampled evidence only; correlate with workload, limits, errors and latency; unsampled peaks are unknown"
},
"observations": [
{
"elapsedMs": 0,
"host": {
"MemTotalBytes": 2097152,
"MemAvailableBytes": 1048576,
"SwapTotalBytes": 4194304,
"SwapFreeBytes": 3145728,
"pswpin": 20,
"pswpout": 30,
"oom_kill": 0,
"memoryPressure": {
"status": "available",
"metrics": {
"some": {
"avg10Percent": "0.00",
"avg60Percent": "1.25",
"avg300Percent": "100.00",
"totalMicros": 123456
},
"full": {
"avg10Percent": "0.00",
"avg60Percent": "0.01",
"avg300Percent": "1.00",
"totalMicros": 0
}
},
"reason": null
}
},
"containers": [
{
"id": "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa",
"imageId": "sha256:bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb",
"startedAt": "2026-11-02T00:00:00Z",
"running": true,
"oomKilled": false,
"restartCount": 0,
"usageIncludingCacheBytes": 100,
"inactiveFileBytes": 30,
"usageMinusInactiveFileBytes": 70,
"configuredLimitBytes": 512,
"configuredMemorySwapBytes": 1024,
"configuredMemorySwapStatus": "available",
"reportedLimitBytes": 2048,
"swapUsageBytes": null,
"resourceLimitsStableDuringSample": true,
"pid": 123,
"identityStableDuringSample": true,
"runningAfter": true,
"oomKilledAfter": false,
"cgroupV2": {
"version": 2,
"status": "available",
"reason": null,
"metrics": {
"memory.current": 100,
"memory.peak": 200,
"memory.events": {
"low": 0,
"high": 0,
"max": 0,
"oom": 0,
"oom_kill": 0
},
"memory.swap.current": 0,
"memory.swap.peak": 0,
"memory.swap.events": {
"high": 0,
"max": 0,
"fail": 0
},
"memory.max": 512,
"memory.swap.max": 512,
"cpu.stat": {
"nr_periods": 10,
"nr_throttled": 2,
"throttled_usec": 1234
}
},
"unavailable": {},
"peakScope": "cgroup lifetime or since external reset; not an isolated workload peak",
"provenance": "fixed host cgroup2 mount + exact Docker ID membership + process/root generation"
}
}
]
}
]
}
+103
View File
@@ -0,0 +1,103 @@
/** Build-time only. Fixed operator artifact; never invoked by server startup. */
import assert from 'node:assert/strict'
import {createHash} from 'node:crypto'
import {isBuiltin} from 'node:module'
import {readFile,writeFile,mkdir,stat,lstat,readdir} from 'node:fs/promises'
import path from 'node:path'
import {fileURLToPath,pathToFileURL} from 'node:url'
import ts from 'typescript'
import {OPERATOR_SOURCES} from './database-operator-sources.mjs'
export {OPERATOR_SOURCES}
const emitted=source=>source.replace(/\.mts$/,'.mjs').replace(/\.ts$/,'.js')
const digest=value=>createHash('sha256').update(value).digest('hex')
const packageName=specifier=>specifier.startsWith('@')?specifier.split('/').slice(0,2).join('/'):specifier.split('/')[0]
/** The transformer changes only module specifier AST nodes, never SQL/string data. */
export function compileOperatorModule(source,fileName){
const rewrite=specifier=>specifier.replace(/^(\.\.\/|\.\.\/\.\.\/)src\//,'$1dist/')
const result=ts.transpileModule(source,{fileName,reportDiagnostics:true,compilerOptions:{target:ts.ScriptTarget.ES2022,module:ts.ModuleKind.ESNext,verbatimModuleSyntax:true,newLine:ts.NewLineKind.LineFeed},transformers:{before:[context=>{
const visit=node=>{
if(ts.isImportDeclaration(node)&&ts.isStringLiteral(node.moduleSpecifier))return context.factory.updateImportDeclaration(node,node.modifiers,node.importClause,context.factory.createStringLiteral(rewrite(node.moduleSpecifier.text)),node.attributes)
if(ts.isExportDeclaration(node)&&node.moduleSpecifier&&ts.isStringLiteral(node.moduleSpecifier))return context.factory.updateExportDeclaration(node,node.modifiers,node.isTypeOnly,node.exportClause,context.factory.createStringLiteral(rewrite(node.moduleSpecifier.text)),node.attributes)
if(ts.isCallExpression(node)&&node.expression.kind===ts.SyntaxKind.ImportKeyword){
assert(node.arguments.length===1&&ts.isStringLiteral(node.arguments[0]),'Only fixed literal operator dynamic imports are supported')
return context.factory.updateCallExpression(node,node.expression,node.typeArguments,[context.factory.createStringLiteral(rewrite(node.arguments[0].text))])
}
return ts.visitEachChild(node,visit,context)
}
return sourceFile=>ts.visitNode(sourceFile,visit)
}]}})
const errors=result.diagnostics?.filter(item=>item.category===ts.DiagnosticCategory.Error)??[]
assert.equal(errors.length,0,'Operator transpilation diagnostics: '+errors.map(item=>ts.flattenDiagnosticMessageText(item.messageText,' ')).join('; '))
return result.outputText
}
export function moduleSpecifiers(source,fileName){
const parsed=ts.createSourceFile(fileName,source,ts.ScriptTarget.ES2022,true,ts.ScriptKind.JS),specifiers=[]
const visit=node=>{
if((ts.isImportDeclaration(node)||ts.isExportDeclaration(node))&&node.moduleSpecifier){assert(ts.isStringLiteral(node.moduleSpecifier));specifiers.push(node.moduleSpecifier.text)}
if(ts.isCallExpression(node)&&node.expression.kind===ts.SyntaxKind.ImportKeyword){assert(node.arguments.length===1&&ts.isStringLiteral(node.arguments[0]),'Computed runtime import refused');specifiers.push(node.arguments[0].text)}
ts.forEachChild(node,visit)
}
visit(parsed);return specifiers
}
export async function buildDatabaseOperator({backendDirectory=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'..'),outputDirectory=path.join(backendDirectory,'dist-operator')}={}){
const backend=path.resolve(backendDirectory),output=path.resolve(outputDirectory),packageJson=JSON.parse(await readFile(path.join(backend,'package.json'),'utf8'))
const modules=new Map(),allowed=new Set(OPERATOR_SOURCES.map(emitted)),runtimeDependencies=new Set(),distImports=new Set()
const productionDependency=specifier=>{
if(isBuiltin(specifier))return
const dependency=packageName(specifier);assert(Object.hasOwn(packageJson.dependencies,dependency),'Operator runtime dependency must be production: '+dependency);runtimeDependencies.add(dependency)
}
for(const file of OPERATOR_SOURCES){
const raw=await readFile(path.join(backend,'scripts',file)),code=compileOperatorModule(new TextDecoder('utf-8',{fatal:true}).decode(raw),file),target=emitted(file)
const imports=moduleSpecifiers(code,target)
for(const specifier of imports){
assert(!/\.(?:ts|mts)$/.test(specifier)&&!/(?:^|\/)src\//.test(specifier),'Source runtime import refused')
if(specifier.startsWith('.')){
const resolved=path.posix.normalize(path.posix.join(path.posix.dirname(target),specifier))
if(resolved.startsWith('../dist/')){
const relative=resolved.slice('../dist/'.length)
assert(relative.endsWith('.js')&&!relative.split('/').includes('..'),'Fixed compiled backend helper required')
assert((await stat(path.join(backend,'dist',relative))).isFile(),'Run backend tsc before operator build')
distImports.add(relative)
}else assert(allowed.has(resolved),'Operator import outside fixed build graph: '+specifier)
}else productionDependency(specifier)
}
modules.set(target,{code,source:file,sourceSha256:digest(raw),sha256:digest(code),imports})
}
// Validate the referenced compiled helper closure as part of the production
// build, not only in tests. Keep this bounded to the small operator graph.
const backendDistModules=new Map()
async function visitDist(file){
assert(file.endsWith('.js')&&!/[\\:]/.test(file)&&!file.startsWith('/')&&!file.split('/').includes('..')&&!file.split('/').includes('src'),'Compiled helper escaped backend/dist')
if(backendDistModules.has(file))return
assert(backendDistModules.size<64,'Operator compiled helper graph exceeded 64 modules')
const raw=await readFile(path.join(backend,'dist',file)),imports=moduleSpecifiers(new TextDecoder('utf-8',{fatal:true}).decode(raw),file)
backendDistModules.set(file,{file,sha256:digest(raw),imports})
for(const specifier of imports){
assert(!/\.(?:ts|mts)$/.test(specifier)&&!/(?:^|\/)src\//.test(specifier),'Source runtime import refused')
if(specifier.startsWith('.'))await visitDist(path.posix.normalize(path.posix.join(path.posix.dirname(file),specifier)))
else productionDependency(specifier)
}
}
for(const file of distImports)await visitDist(file)
// Refuse stale/unlisted output rather than shipping an extra file (e.g. a key)
// through Docker's directory COPY. Never delete/adopt unknown build contents.
try{
assert((await lstat(output)).isDirectory(),'Ordinary operator output directory required')
for(const item of await readdir(output,{recursive:true,withFileTypes:true})){
const relative=path.relative(output,path.join(item.parentPath,item.name)).split(path.sep).join('/')
assert(item.isDirectory()?relative==='writer-profile':item.isFile()&&(allowed.has(relative)||relative==='build-manifest.json'),'Unexpected operator output entry: '+relative)
}
}catch(error){if(error.code!=='ENOENT')throw error}
// Validate the whole fixed graph before publishing any generated module.
await mkdir(output,{recursive:true})
for(const [file,module] of modules){await mkdir(path.dirname(path.join(output,file)),{recursive:true});await writeFile(path.join(output,file),module.code)}
const manifest={format:'application-database-operator-build-v1',compilerVersion:ts.version,entrypoints:['database-writer-bootstrap-operator.mjs','database-writer-profile-operator.mjs'],modules:[...modules].map(([file,module])=>({file,source:module.source,sourceSha256:module.sourceSha256,sha256:module.sha256,imports:module.imports})),backendDistImports:[...distImports].sort(),backendDistModules:[...backendDistModules.values()].sort((a,b)=>a.file.localeCompare(b.file)),runtimeDependencies:[...runtimeDependencies].sort(),automaticStartup:false,readyAuthorized:false}
await writeFile(path.join(output,'build-manifest.json'),JSON.stringify(manifest,null,2)+'\n')
return manifest
}
if(process.argv[1]&&import.meta.url===pathToFileURL(path.resolve(process.argv[1])).href){
assert.equal(process.argv.length,2,'Build takes no runtime configuration or key arguments')
const result=await buildDatabaseOperator();console.log(JSON.stringify({event:'database-operator-built',modules:result.modules.length,automaticStartup:false,readyAuthorized:false}))
}
@@ -0,0 +1,101 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {readFile,writeFile,mkdir,copyFile,mkdtemp,rename} from 'node:fs/promises'
import {spawnSync} from 'node:child_process'
import path from 'node:path'
import {fileURLToPath,pathToFileURL} from 'node:url'
import {buildDatabaseOperator,compileOperatorModule,moduleSpecifiers,OPERATOR_SOURCES} from './build-database-operator.mjs'
import {stageDatabaseOperatorScripts} from './stage-database-operator.mjs'
const backend=process.env.OPERATOR_BUILD_TEST_BACKEND??path.resolve(path.dirname(fileURLToPath(import.meta.url)),'..')
test('AST rewrite changes static/export/dynamic module specifiers and preserves SQL/string values',()=>{
const code=compileOperatorModule(`import {x} from '../src/db/x.js'; export {y} from '../../src/db/y.js'; const module=()=>import('../src/db/z.js'); export const sql="SELECT '../src/db/x.js'";`,'fixture.mts')
assert.deepEqual(moduleSpecifiers(code,'fixture.mjs'),['../dist/db/x.js','../../dist/db/y.js','../dist/db/z.js'])
assert(code.includes(`SELECT '../src/db/x.js'`));assert.throws(()=>compileOperatorModule('const load=name=>import(name)','fixture.mts'),/fixed literal/)
})
test('fixed artifact is reproducible, resolves the compiled backend graph, and runs with plain Node22',async()=>{
assert.equal(Number(process.versions.node.split('.')[0]),22,'Run this acceptance with the production Node22 major')
const temporaryParent=process.env.OPERATOR_BUILD_TEST_TMP??path.resolve(backend,'../tmp');await mkdir(temporaryParent,{recursive:true})
const temporary=await mkdtemp(path.join(temporaryParent,'db-operator-build-')),projectBackend=path.join(temporary,'backend'),output=path.join(projectBackend,'dist-operator')
const packageJson=JSON.parse(await readFile(path.join(backend,'package.json'),'utf8'))
await mkdir(projectBackend,{recursive:true});await writeFile(path.join(projectBackend,'package.json'),JSON.stringify({type:'module',dependencies:packageJson.dependencies}))
const first=await buildDatabaseOperator({backendDirectory:backend,outputDirectory:output}),second=await buildDatabaseOperator({backendDirectory:backend,outputDirectory:output})
assert.deepEqual(first,second);assert.equal(first.modules.length,OPERATOR_SOURCES.length);assert.equal(first.modules.length,31);assert.deepEqual(first.runtimeDependencies,['dotenv','pg','zod']);assert.equal(first.backendDistModules.length,30)
assert(first.modules.some(item=>item.file==='writer-profile/registry-enrollment.mjs'))
assert(first.modules.some(item=>item.file==='writer-profile/marker-fence-enrollment.mjs'))
assert(first.modules.some(item=>item.file==='writer-profile/fixed58-build-definition.js'))
assert(first.backendDistImports.includes('db/database-restore-quarantine-profile.js'))
assert(first.backendDistImports.includes('db/database-source-activation-record.js'))
assert(first.backendDistImports.includes('services/server-probe-legacy-source-inventory.js'))
assert(first.backendDistImports.includes('services/server-probe-legacy-archive-approval.js'))
assert(first.backendDistImports.includes('services/server-probe-legacy-handoff-inspection.js'))
assert(first.backendDistImports.includes('db/legacy-probe-0037-body.js'))
assert(first.backendDistImports.includes('db/controlled-schema-reference-core.js'))
assert(first.backendDistImports.includes('services/database-dump-connection.js'))
const copied=new Set()
async function copyDist(relative){
if(copied.has(relative))return;copied.add(relative)
const original=path.join(backend,'dist',relative),target=path.join(projectBackend,'dist',relative),source=await readFile(original,'utf8')
await mkdir(path.dirname(target),{recursive:true});await copyFile(original,target)
for(const specifier of moduleSpecifiers(source,relative)){
assert(!specifier.includes('/src/'));assert(!/\.(?:ts|mts)$/.test(specifier))
if(specifier.startsWith('.')){const dependency=path.posix.normalize(path.posix.join(path.posix.dirname(relative),specifier));assert(!dependency.startsWith('../'));await copyDist(dependency)}
else if(!specifier.startsWith('node:')){const name=specifier.startsWith('@')?specifier.split('/').slice(0,2).join('/'):specifier.split('/')[0];assert(Object.hasOwn(packageJson.dependencies,name),'Compiled helper has undeclared production dependency '+name)}
}
}
for(const relative of first.backendDistImports)await copyDist(relative)
assert.deepEqual([...copied].sort(),first.backendDistModules.map(item=>item.file).sort())
// Exercise the real build entry point with its default backend/output paths.
const scripts=path.join(projectBackend,'scripts')
assert.equal(await stageDatabaseOperatorScripts(scripts,path.join(backend,'scripts')),OPERATOR_SOURCES.length+2)
const buildScript=path.join(scripts,'build-database-operator.mjs')
const built=spawnSync(process.execPath,[buildScript],{cwd:temporary,env:{...process.env,NODE_OPTIONS:''},encoding:'utf8',timeout:30000})
assert.equal(built.status,0,built.stderr);assert.equal(JSON.parse(built.stdout.trim()).modules,first.modules.length)
assert.deepEqual(JSON.parse(await readFile(path.join(output,'build-manifest.json'),'utf8')),first)
const configFile=path.join(projectBackend,'dist','config.js'),originalConfig=await readFile(configFile,'utf8')
for(const injected of ["import 'tsx';","import '../src/config.js';","import '../outside.js';"]){
await writeFile(configFile,originalConfig+'\n'+injected)
await assert.rejects(buildDatabaseOperator({backendDirectory:projectBackend,outputDirectory:output}),/production|Source runtime import|escaped backend\/dist/)
}
await writeFile(configFile,originalConfig)
await writeFile(path.join(output,'unexpected-private.pem'),'not a key; stale-output test')
await assert.rejects(buildDatabaseOperator({backendDirectory:backend,outputDirectory:output}),/Unexpected operator output entry/)
const {unlink}=await import('node:fs/promises');await unlink(path.join(output,'unexpected-private.pem'))
await rename(path.join(projectBackend,'scripts'),path.join(temporary,'build-inputs'))
const guard=path.join(temporary,'runtime-guard.mjs'),telemetry=path.join(temporary,'telemetry.json')
const denied=[...new Set([...Object.keys(packageJson.devDependencies),'typescript','tsx'])]
// Block development imports and connections in the actual plain-Node process.
await writeFile(guard,`import {registerHooks,syncBuiltinESMExports} from 'node:module';import net from 'node:net';import tls from 'node:tls';import {writeFileSync} from 'node:fs';const denied=new Set(${JSON.stringify(denied)});let connections=0;const refuse=()=>{connections++;throw Error('Unexpected operator network connection')};net.Socket.prototype.connect=refuse;tls.connect=refuse;syncBuiltinESMExports();registerHooks({resolve(specifier,context,next){const name=specifier.startsWith('@')?specifier.split('/').slice(0,2).join('/'):specifier.split('/')[0];if(denied.has(name)||/\\.(?:ts|mts)$/.test(specifier)||specifier.includes('/backend/src/'))throw Error('Nonproduction import '+specifier);return next(specifier,context)}});process.on('exit',()=>writeFileSync(${JSON.stringify(telemetry)},JSON.stringify({connections})));`)
const env={...process.env,NODE_ENV:'production',NODE_OPTIONS:'',AUTH_REQUIRED:'false',DOTENV_CONFIG_QUIET:'true'}
// The workspace tmp projection resolves installed packages from its ancestor
// node_modules; the runtime hook rejects every development dependency.
const modulesToImport=first.modules.map(item=>path.join(output,item.file))
const importer=path.join(temporary,'import-all.mjs');await writeFile(importer,`import {pathToFileURL} from 'node:url';for(const file of ${JSON.stringify(modulesToImport)})await import(pathToFileURL(file).href);console.log(JSON.stringify({imported:${modulesToImport.length},readyAuthorized:false}));`)
const imported=spawnSync(process.execPath,['--import',pathToFileURL(guard).href,importer],{cwd:temporary,env,encoding:'utf8',timeout:30000})
assert.equal(imported.status,0,imported.stderr);assert.equal(JSON.parse(imported.stdout.trim()).imported,first.modules.length);assert.equal(JSON.parse(await readFile(telemetry,'utf8')).connections,0)
const refusals=[]
for(const [entry,event,args] of [['database-writer-bootstrap-operator.mjs','application-writer-bootstrap-refused',['invalid']],['database-writer-profile-operator.mjs','application-writer-profile-publication-refused',['invalid']],['database-writer-profile-operator.mjs','database-maintenance-command-refused',['maintenance-inspect']],['database-writer-profile-operator.mjs','database-maintenance-command-refused',['maintenance-resolve']],['database-writer-profile-operator.mjs','database-source-activation-refused',['source-activate']],['database-writer-profile-operator.mjs','database-legacy-inspection-refused',['legacy-inspect']],['database-writer-profile-operator.mjs','application-writer-registry-enrollment-refused',['registry-enroll']],['database-writer-profile-operator.mjs','application-writer-marker-fence-enrollment-refused',['marker-fence-enroll']],['database-writer-profile-operator.mjs','application-writer-restore-profile-enrollment-refused',['restore-profile-enroll']]]){
const child=spawnSync(process.execPath,['--import',pathToFileURL(guard).href,path.join(output,entry),...args],{cwd:temporary,env,encoding:'utf8',timeout:30000})
assert.equal(child.status,1,child.stderr);assert.equal(child.stdout,'');assert.equal(JSON.parse(child.stderr.trim()).event,event);assert.equal(JSON.parse(await readFile(telemetry,'utf8')).connections,0);refusals.push(event)
}
const catalogue=spawnSync(process.execPath,['--import',pathToFileURL(guard).href,path.join(output,'database-writer-profile-operator.mjs'),'source-catalogue-inspect'],{cwd:temporary,env,encoding:'utf8',timeout:30000})
assert.equal(catalogue.status,1,catalogue.stderr);assert.equal(catalogue.stdout,'');assert.equal(JSON.parse(catalogue.stderr.trim()).event,'application-writer-source-catalogue-inspection-refused')
assert.equal(JSON.parse(await readFile(telemetry,'utf8')).connections,0);refusals.push('application-writer-source-catalogue-inspection-refused')
for(const command of ['legacy-export','legacy-custody','legacy-handoff-inspect','legacy-cold-verify','legacy-handoff-execute','legacy-handoff-adopt','legacy-handoff-release','legacy-control-reference','legacy-source58-preflight','source58-foundation-enroll','source57-admission-enroll']){
const invalid=spawnSync(process.execPath,['--import',pathToFileURL(guard).href,path.join(output,'database-writer-profile-operator.mjs'),command],{cwd:temporary,env,encoding:'utf8',timeout:30000})
const event={
'legacy-handoff-inspect':'database-legacy-handoff-inspection-refused','legacy-cold-verify':'database-legacy-cold-proof-refused',
'legacy-handoff-execute':'database-legacy-handoff-execution-refused',
'legacy-handoff-adopt':'database-legacy-handoff-adoption-refused',
'legacy-handoff-release':'database-legacy-handoff-release-refused',
'legacy-control-reference':'database-legacy-control-reference-refused',
'legacy-source58-preflight':'database-legacy-source58-preflight-refused',
'source58-foundation-enroll':'database-source58-foundation-refused',
'source57-admission-enroll':'database-source57-admission-refused',
}[command]??`database-${command}-refused`
assert.equal(invalid.status,1,invalid.stderr);assert.equal(invalid.stdout,'');assert.equal(JSON.parse(invalid.stderr.trim()).event,event)
assert.equal(JSON.parse(await readFile(telemetry,'utf8')).connections,0);refusals.push(event)
}
const receipt={node:process.version,projection:temporary,modules:first.modules.length,compiledBackendModules:copied.size,buildCliExitCode:built.status,importedModules:modulesToImport.length,invalidArgumentsRefused:refusals,developmentDependenciesBlocked:denied,networkConnections:0,productionActivated:false,readyAuthorized:false}
await writeFile(path.join(temporary,'acceptance-receipt.json'),JSON.stringify(receipt,null,2)+'\n');console.log(JSON.stringify(receipt))
})
@@ -0,0 +1,100 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {mkdtemp,mkdir,writeFile,readFile,readdir,rm,unlink,lstat} from 'node:fs/promises'
import {createHash,randomUUID} from 'node:crypto'
import {rootCertificates} from 'node:tls'
import {EventEmitter} from 'node:events'
import path from 'node:path'
import {fileURLToPath,pathToFileURL} from 'node:url'
import {runInNewContext} from 'node:vm'
import {createRequire} from 'node:module'
import * as strictReader from '../src/services/database-maintenance-startup.js'
import * as grammar from '../src/services/database-maintenance-resolution.js'
import * as manifest from '../src/db/migration-manifest.js'
import * as contract from './writer-profile/operator-contract.mjs'
const backend=path.resolve(path.dirname(fileURLToPath(import.meta.url)),'..'),require=createRequire(import.meta.url),ts=require('typescript'),hash=(value:Buffer|string)=>createHash('sha256').update(value).digest('hex')
async function own(work:(root:string)=>Promise<void>){const temp=path.resolve(backend,'../tmp');await mkdir(temp,{recursive:true});const root=await mkdtemp(path.join(temp,'owned-resolution-command-'));try{await work(root)}finally{assert.equal(path.dirname(root),temp);assert(path.basename(root).startsWith('owned-resolution-command-'));await rm(root,{recursive:true,force:true})}}
async function model(root:string,localHost?:string){
let owners=0,sql=0,releaseFailure=false,poolEndFailure=false,published=0,wrongOwner=false,controlSchema=false,observationFailure:Error|undefined
const capturedScopes:unknown[]=[]
const state=path.join(root,'state');await mkdir(state);await mkdir(path.join(state,'restore-vault'));await writeFile(path.join(state,'access-generation'),Buffer.alloc(32,7));const uncertain=randomUUID(),at=new Date().toISOString(),record=(value:string)=>JSON.stringify({operationId:uncertain,state:value,at})+'\n';await writeFile(path.join(state,'database-maintenance.jsonl'),record('intent')+record('unconfirmed'))
const source={systemIdentifier:'1234567890',databaseOid:'12345',databaseName:'managed',serverVersionNum:180004,inRecovery:false}
const schema={sha256:'a'.repeat(64),observedRaw:{sha256:'b'.repeat(64)},observedPhysical:{sha256:'c'.repeat(64)},sections:{relations:manifest.CURRENT_REQUIRED_TABLES.map(name=>({namespace:'public',name,kind:'r',persistence:'p',row_security:false,force_row_security:false})),columns:[{namespace:'public',relation:'schema_migrations',name:'id',dropped:false,type:'text'}]}}
class Pool {on(){};constructor(options:unknown){const ssl=(options as {ssl:false|{rejectUnauthorized:boolean}}).ssl;if(ssl===false)assert(['127.0.0.1','::1'].includes((options as {host:string}).host));else assert(ssl.rejectUnauthorized)}async connect(){owners++;const client=new EventEmitter();return Object.assign(client,{release:()=>{owners--;if(releaseFailure)throw Error('Synthetic release uncertainty')},query:async(text:string)=>{sql++;if(text.includes('pg_try_advisory_lock'))return {rows:[{locked:true}],command:'SELECT'};if(text.includes('pg_advisory_unlock'))return {rows:[{unlocked:true}],command:'SELECT'};if(text.includes('pg_namespace'))return {rows:controlSchema?[{nspname:'mmapp_control'}]:[],command:'SELECT'};if(text.includes('pg_control_system')){if(observationFailure)throw observationFailure;return {rows:[source],command:'SELECT'}};if(text.startsWith('SELECT id FROM public.schema_migrations'))return {rows:manifest.MIGRATION_MANIFEST.map(pin=>({id:pin.file.slice(0,-4)})),command:'SELECT'};return {rows:[],command:text.split(' ')[0]}}})}async end(){assert.equal(owners,0);if(poolEndFailure)throw Error('Synthetic pool retirement uncertainty')}}
const protectedSource=await readFile(path.join(backend,'scripts/writer-profile/operator-files.mjs'),'utf8'),protectedExports:Record<string,unknown>={},actualUid=(await lstat(state)).uid
const protectedCode=ts.transpileModule(protectedSource,{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022,esModuleInterop:true}}).outputText
runInNewContext(protectedCode,{exports:protectedExports,require,Buffer,process:{platform:'linux',getuid:()=>actualUid+1},Object,Error})
const protectedDirectory=async(directory:string)=>{assert(path.resolve(directory).startsWith(root+path.sep)||path.resolve(directory)===root);if(wrongOwner)await (protectedExports.protectedDirectory as (path:string)=>Promise<void>)(directory);await readdir(directory)}
// Explicit MODELED protection adapter: native POSIX mode/ownership and hardlink
// fsync publication remain separate Linux acceptance, never a Windows claim.
const files={sha:hash,protectedDirectory,readProtected:async(file:string,maximum:number,pin?:string)=>{assert(file.startsWith(root+path.sep));const raw=await readFile(file);assert(raw.length>0&&raw.length<=maximum);if(pin)assert.equal(hash(raw),pin);return raw},writeFile:async(file:string,raw:Buffer)=>{assert(file.startsWith(root+path.sep));await writeFile(file,raw,{flag:'wx'});published++},withJournalLease:async(directory:string,work:()=>Promise<unknown>)=>{const lock=path.join(directory,'publisher-lease.lock');await writeFile(lock,'owned',{flag:'wx'});try{return await work()}finally{await unlink(lock)}}}
const readRecords=async(directory:string)=>{const records=path.join(directory,'maintenance-acceptances');let names:string[];try{names=await readdir(records)}catch(error){if((error as NodeJS.ErrnoException).code==='ENOENT')return [];throw error};assert(names.every(name=>/^[a-f0-9-]{36}\.json$/.test(name)));return Promise.all(names.sort().map(name=>readFile(path.join(records,name))))}
const moduleFile=path.join(backend,'scripts/writer-profile/maintenance-resolution.mts'),sourceText=(await readFile(moduleFile,'utf8')).replaceAll('import.meta.url',JSON.stringify(pathToFileURL(moduleFile).href)),exports:Record<string,unknown>={}
const dependencies:Record<string,unknown>={'./operator-files.mjs':files,'./operator-contract.mjs':contract,'../../src/services/database-maintenance-startup.js':{...strictReader,readMaintenanceAcceptanceRecords:readRecords},'../../src/services/database-maintenance-resolution.js':grammar,'../../src/db/migration-schema-inventory.js':{captureMigrationSchema:async(_client:unknown,options:{scope:unknown})=>{capturedScopes.push(options.scope);return schema}},'../../src/db/migration-manifest.js':manifest,pg:{Pool}}
const code=ts.transpileModule(sourceText,{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022,esModuleInterop:true}}).outputText
runInNewContext(code,{exports,require:(name:string)=>name in dependencies?dependencies[name]:require(name),Buffer,URL,process:{env:{}},Date,Object,Map,Set,Error,console})
const command=exports.maintenanceMain as (args:string[])=>Promise<Record<string,unknown>>
const tlsTransport={format:'database-maintenance-transport-v1',connection:{host:'db.example.invalid',port:5432,user:'existing-maintenance-user',password:'synthetic-only',database:'managed'},tls:{serverName:'db.example.invalid',spkiSha256:'f'.repeat(64)}},localTransport=(host:string)=>({format:'database-maintenance-transport-v1',mode:'local-loopback',connection:{...tlsTransport.connection,host}})
const configs={transport:localHost===undefined?tlsTransport:localTransport(localHost),'runtime-target':{format:'database-maintenance-runtime-target-v1',databaseUrl:'postgresql://db.example.invalid:5432/managed?sslmode=require'}}
const args=['--state-dir',state]
for(const [name,value]of Object.entries(configs)){const raw=Buffer.from(JSON.stringify(value)),file=path.join(root,name+'.json');await writeFile(file,raw);args.push('--'+name,file,'--'+name+'-sha256',hash(raw))}
const ca=Buffer.from(rootCertificates[0]),caFile=path.join(root,'ca.pem');await writeFile(caFile,ca);if(localHost===undefined)args.push('--ca',caFile,'--ca-sha256',hash(ca))
const output=path.join(root,'inspection.json')
async function inspect(destination=output){return command(['maintenance-inspect',...args,'--output',destination])}
async function approve(change?:(value:Record<string,unknown>)=>void){const raw=await readFile(output),inspection=JSON.parse(raw.toString()),approval={format:'database-maintenance-current-state-approval-v1',resolutionId:randomUUID(),decision:'accept-observed-current-state',actorId:'host-human-reviewer',inspectionSha256:hash(raw),issuedAt:new Date().toISOString(),expiresAt:new Date(Date.now()+60000).toISOString(),managedBackendStopped:true,historicOutcomesRemainUnknown:true,automaticRestore:false};assert(inspection.createdAt<=approval.issuedAt);change?.(approval);const bytes=Buffer.from(JSON.stringify(approval)),file=path.join(root,'approval.json');await writeFile(file,bytes);return ['maintenance-resolve',...args,'--inspection',output,'--inspection-sha256',hash(raw),'--approval',file,'--approval-sha256',hash(bytes)]}
const toLocal=async(host='127.0.0.1')=>{const raw=Buffer.from(JSON.stringify(localTransport(host)));await writeFile(path.join(root,'transport.json'),raw);args[args.indexOf('--transport-sha256')+1]=hash(raw);for(const flag of ['--ca','--ca-sha256']){const at=args.indexOf(flag);if(at>=0)args.splice(at,2)}}
const configuration=exports.maintenancePoolConfiguration as (value:unknown,ca?:Buffer,pin?:string)=>{options:Record<string,unknown>;transportBinding:{mode:string;endpointSha256:string}}
return {command,inspect,approve,toLocal,configuration,readRecords,state,output,sql:()=>sql,published:()=>published,failRelease:()=>{releaseFailure=true},failEnd:()=>{poolEndFailure=true},driftSchema:()=>{schema.sha256='0'.repeat(64)},driftTarget:()=>{source.databaseOid='54321'},wrongOwner:()=>{wrongOwner=true},withControlSchema:()=>{controlSchema=true},scopes:()=>capturedScopes,failObservation:(error:Error)=>{observationFailure=error}}
}
test('actual operator commands inspect then separately approved resolve; old uncertain bytes never rewritten',()=>own(async root=>{
const fixture=await model(root),journal=path.join(fixture.state,'database-maintenance.jsonl'),before=await readFile(journal),inspected=await fixture.inspect();assert.equal(inspected.humanApprovalRequired,true);assert.equal((await fixture.readRecords(fixture.state)).length,0)
const result=await fixture.command(await fixture.approve());assert.equal(result.outcome,'committed');assert.equal(result.historicOutcomesChanged,false);assert.equal(result.archiveDataEquivalenceProved,false);assert.equal(result.automaticRestore,false)
assert.deepEqual(await readFile(journal),before);const records=await fixture.readRecords(fixture.state),facts=await strictReader.readMaintenanceRecoveryFacts(fixture.state);assert.equal(records.length,1);const accepted=grammar.evaluateMaintenanceAcceptances(records,facts.journal,facts.vault.evidence,grammar.runtimeDatabaseTargetSha256('postgresql://db.example.invalid:5432/managed?sslmode=require'));assert.equal(accepted.count,1)
fixture.driftSchema();const afterStatus=await fixture.inspect(path.join(root,'after-inspection.json'));assert.equal(afterStatus.state,'historically-accepted');assert.equal(afterStatus.humanApprovalRequired,false);assert.equal(afterStatus.historicOutcomesChanged,false)
await assert.rejects(fixture.command(await fixture.approve()),/No new unresolved|already|assert|Assertion/i)
}))
for(const type of ['expired','not-stopped','wrong-inspection'])test('actual resolve refuses '+type+' approval before connecting SQL',()=>own(async root=>{
const fixture=await model(root);await fixture.inspect();const args=await fixture.approve(value=>{if(type==='expired')value.expiresAt=new Date(Date.now()-1).toISOString();if(type==='not-stopped')value.managedBackendStopped=false;if(type==='wrong-inspection')value.inspectionSha256='0'.repeat(64)}),before=fixture.sql();await assert.rejects(fixture.command(args));assert.equal(fixture.sql(),before);assert.equal((await fixture.readRecords(fixture.state)).length,0)
}))
for(const type of ['release','pool-end','schema-drift','target-drift','external-drift'])test('actual resolve refuses '+type+' before acceptance publication',()=>own(async root=>{
const fixture=await model(root);await fixture.inspect();const args=await fixture.approve();if(type==='release')fixture.failRelease();if(type==='pool-end')fixture.failEnd()
if(type==='schema-drift')fixture.driftSchema();if(type==='target-drift')fixture.driftTarget()
if(type==='external-drift')await appendFileSafe(path.join(fixture.state,'database-maintenance.jsonl'))
await assert.rejects(fixture.command(args));assert.equal((await fixture.readRecords(fixture.state)).length,0)
}))
async function appendFileSafe(file:string){await writeFile(file,await readFile(file,'utf8')+JSON.stringify({operationId:randomUUID(),state:'intent',at:new Date().toISOString()})+'\n')}
test('existing shipped main dispatches both commands before writer-profile role/source flags',async()=>{
const source=await readFile(path.join(backend,'scripts/database-writer-profile-operator.mts'),'utf8');assert(source.indexOf("['maintenance-inspect','maintenance-resolve']")<source.indexOf("const flags="));assert(source.includes("import('./writer-profile/maintenance-resolution.mjs')"))
})
test('existing protectedDirectory owner contract refuses a foreign CLI UID before SQL or writes',()=>own(async root=>{
const fixture=await model(root);fixture.wrongOwner();await assert.rejects(fixture.inspect());assert.equal(fixture.sql(),0);assert.equal(fixture.published(),0);assert.equal((await fixture.readRecords(fixture.state)).length,0)
}))
test('explicit local-loopback supports both IP families and binds selected mode in actual inspect/resolve',async()=>{
for(const host of ['127.0.0.1','::1'])await own(async root=>{const fixture=await model(root,host);await fixture.inspect();const result=await fixture.command(await fixture.approve());assert.equal(result.outcome,'committed');const records=await fixture.readRecords(fixture.state);assert.equal(JSON.parse(records[0].toString()).knownCurrent.transport.mode,'local-loopback')})
})
test('plaintext remote/DNS/unspecified/mapped hosts refuse before SQL and never fall back',async()=>{
for(const host of ['db.example.invalid','localhost','0.0.0.0','192.0.2.1','[::1]','::ffff:127.0.0.1'])await own(async root=>{const fixture=await model(root,host);await assert.rejects(fixture.inspect());assert.equal(fixture.sql(),0);assert.equal(fixture.published(),0)})
})
test('an approval for TLS observation cannot resolve through a changed local-loopback transport',()=>own(async root=>{
const fixture=await model(root);await fixture.inspect();await fixture.toLocal();await assert.rejects(fixture.command(await fixture.approve()));assert.equal((await fixture.readRecords(fixture.state)).length,0)
}))
test('local mode explicitly disables SSL/environment substitution; TLS remains verified default without fallback',()=>own(async root=>{
const fixture=await model(root),ConnectionParameters=require('pg/lib/connection-parameters'),raw={format:'database-maintenance-transport-v1',mode:'local-loopback',connection:{host:'127.0.0.1',port:5432,user:'existing-user',password:'synthetic-only',database:'managed'}}
const previous={PGHOST:process.env.PGHOST,PGSSLMODE:process.env.PGSSLMODE,PGOPTIONS:process.env.PGOPTIONS};process.env.PGHOST='remote.invalid';process.env.PGSSLMODE='require';process.env.PGOPTIONS='-c role=foreign'
try{const config=fixture.configuration(raw),actual=new ConnectionParameters(config.options);assert.equal(actual.host,'127.0.0.1');assert.equal(actual.ssl,false);assert.equal(actual.options,contract.STARTUP_OPTIONS);assert.equal(config.transportBinding.mode,'local-loopback')}
finally{for(const [key,value]of Object.entries(previous)){if(value===undefined)delete process.env[key];else process.env[key]=value}}
const ca=Buffer.from(rootCertificates[0]),tls={format:'database-maintenance-transport-v1',connection:{...raw.connection,host:'db.example.invalid'},tls:{serverName:'db.example.invalid',spkiSha256:'f'.repeat(64)}}
assert.throws(()=>fixture.configuration(tls));const secure=fixture.configuration(tls,ca,hash(ca));assert.equal(secure.transportBinding.mode,'tls');assert.equal((secure.options.ssl as {rejectUnauthorized:boolean}).rejectUnauthorized,true)
assert.throws(()=>fixture.configuration({...raw,mode:'plaintext'}));assert.throws(()=>fixture.configuration({...tls,mode:null},ca,hash(ca)));assert.throws(()=>fixture.configuration(raw,ca,hash(ca)))
}))
// A normal installation has no optional writer-profile schema. Observation must
// select its exact namespace set without requiring installation of that system.
test('maintenance observes with and without the optional control namespace',async()=>{
for(const present of [false,true])await own(async root=>{const fixture=await model(root);if(present)fixture.withControlSchema();await fixture.inspect();assert.equal(JSON.stringify(fixture.scopes()),JSON.stringify([{applicationNamespaces:['public'],excludedControlNamespaces:present?['mmapp_control']:[]}]))})
})
test('original observation failure survives confirmed rollback/unlock/release',()=>own(async root=>{
const fixture=await model(root),failure=Error('Synthetic original observation failure');fixture.failObservation(failure);await assert.rejects(fixture.inspect(),error=>error===failure);assert.equal(fixture.published(),0);assert.equal((await fixture.readRecords(fixture.state)).length,0)
}))
@@ -0,0 +1,17 @@
/** The fixed build inputs shared by the operator and Docker staging. */
export const OPERATOR_SOURCES=Object.freeze([
'database-writer-bootstrap-operator.mjs','database-writer-profile-operator.mts',
'writer-profile/operator-files.mjs','writer-profile/operator-contract.mjs','writer-profile/publisher.mjs','writer-profile/context.mjs',
'writer-profile/protected-source-registry.mts','writer-profile/maintenance-resolution.mts','writer-profile/source-activation.mts',
'writer-profile/legacy-cold-verification.mts',
'writer-profile/legacy-handoff-execution.mts',
'writer-profile/legacy-control-reference.mts',
'writer-profile/source58-foundation.mts',
'writer-profile/source57-admission-plan.mts',
'writer-profile/registry-enrollment.mts',
'writer-profile/marker-fence-enrollment.mts',
'writer-profile/catalogue.mts','writer-profile/catalogue-value-bridge.ts','writer-profile/catalogue-projection.ts','writer-profile/catalogue-leaves.ts',
'writer-profile/application-writer-profile-catalogue.ts','writer-profile/application-writer-profile-catalogue-values.ts',
'writer-profile/application-writer-profile-catalogue-parent-policy.ts','writer-profile/application-writer-profile-canonical.ts',
'writer-profile/fixed44-build-sql.ts','writer-profile/fixed44-build-definition.ts','writer-profile/fixed56-build-definition.ts','writer-profile/fixed56-build-delta.ts','writer-profile/fixed57-build-definition.ts','writer-profile/fixed58-build-definition.ts','writer-profile/expanded44-data-policy.ts',
])
@@ -0,0 +1,74 @@
/** Offline operator-only issuer proposal. Never imported by runtime/startup.
* Effective key/registry ACL isolation is required before deployment acceptance;
* mode bits and possession of CLI arguments do not establish that isolation. */
import { createPrivateKey, createPublicKey, sign } from "node:crypto"
import { mkdir, open, realpath } from "node:fs/promises"
import path from "node:path"
import { z } from "zod"
import { DATABASE_SOURCE_SIGNATURE_DOMAINS, sourceBindingBodySchema, sourceBindingBytes, sourceBindingCanonical, sourceBindingSha256,
readDatabaseSourceBindingRegistry, type DatabaseSourceDeploymentPin } from "../src/db/database-source-binding.js"
const requestSchema = sourceBindingBodySchema.omit({ format: true, ordinal: true })
export type DatabaseSourceOperatorRequest = z.infer<typeof requestSchema>
function outside(parent: string, child: string) {
const relative = path.relative(parent, child)
return !!relative && (relative.startsWith(".." + path.sep) || relative === ".." || path.isAbsolute(relative))
}
export async function provisionDatabaseSourceBindingOffline(options: { request: DatabaseSourceOperatorRequest; privateKeyFile: string;
outputDirectory: string; previousPin?: DatabaseSourceDeploymentPin }) {
const request = requestSchema.parse(options.request)
const privatePath = await realpath(options.privateKeyFile), output = path.resolve(options.outputDirectory)
const parent = await realpath(path.dirname(output))
if (parent !== path.dirname(output) || !outside(output, privatePath)) throw Error("Private operator key must be outside new registry, without ambiguous parent links")
const stat = await open(privatePath, "r")
let privateBytes: Buffer
try { const meta = await stat.stat(); if (!meta.isFile() || meta.size < 1 || meta.size > 8192) throw Error("Bounded operator private key required"); privateBytes = await stat.readFile() }
finally { await stat.close() }
const privateKey = createPrivateKey(privateBytes); privateBytes.fill(0)
if (privateKey.type !== "private" || privateKey.asymmetricKeyType !== "ed25519") throw Error("Independent Ed25519 operator key required")
const publicKey = createPublicKey(privateKey), spki = publicKey.export({ type: "spki", format: "der" })
const publicKeySpkiSha256 = sourceBindingSha256(spki)
const previous = options.previousPin ? await readDatabaseSourceBindingRegistry(options.previousPin) : undefined
if (previous && (previous.body.installationId !== request.installationId || previous.body.sourceIdentityId !== request.sourceIdentityId ||
previous.publicKeySpkiSha256 !== publicKeySpkiSha256 || previous.body.generation === request.generation || previous.body.ordinal >= Number.MAX_SAFE_INTEGER)) throw Error("Previous current head/identity/key or distinct new generation required")
const body = sourceBindingBodySchema.parse({ ...request, format: "database-source-binding-v1", ordinal: previous ? previous.body.ordinal + 1 : 1 })
const signed = (domain: string, payload: unknown) => sourceBindingBytes({ body: payload,
signature: sign(null, Buffer.from(domain + sourceBindingCanonical(payload)), privateKey).toString("base64url") })
const bindingBytes = signed(DATABASE_SOURCE_SIGNATURE_DOMAINS.binding, body), bindingSha256 = sourceBindingSha256(bindingBytes)
const head = { format: "database-source-registry-head-v1", installationId: body.installationId, sourceIdentityId: body.sourceIdentityId,
generation: body.generation, ordinal: body.ordinal, bindingSha256, previousHeadSha256: previous?.headSha256 ?? null }
const headBytes = signed(DATABASE_SOURCE_SIGNATURE_DOMAINS.head, head), currentHeadSha256 = sourceBindingSha256(headBytes)
// No recursive mkdir and no existing-state adoption. A partial output remains
// for explicit operator review; uncertainty is never retried/overwritten.
await mkdir(output, { mode: 0o700 })
const publish = async (filename: string, bytes: Buffer | string) => {
const handle = await open(path.join(output, filename), "wx", 0o600)
try { await handle.writeFile(bytes); await handle.sync() } finally { await handle.close() }
}
await publish("operator-public.pem", publicKey.export({ type: "spki", format: "pem" }))
await publish("binding-" + bindingSha256 + ".json", bindingBytes)
await publish("head-" + currentHeadSha256 + ".json", headBytes)
const deploymentPin = { registryDirectory: output, installationId: body.installationId, publicKeySpkiSha256, currentHeadSha256 }
// Public review artifact only; not installed into any application config.
await publish("proposed-deployment-pin.json", sourceBindingBytes(deploymentPin))
return Object.freeze({ deploymentPin: Object.freeze(deploymentPin), keyIsolationVerified: false as const,
platformProtectionVerified: false as const, generationAuthenticated: false as const, readyAuthorized: false as const })
}
export async function databaseSourceBindingOperatorCli(args: string[]) {
const allowed = new Set(["request", "private-key", "output", "previous-pin"]), values = new Map<string, string>()
for (const arg of args) {
const match = /^--([a-z-]+)=(.+)$/.exec(arg)
if (!match || !allowed.has(match[1]) || values.has(match[1])) throw Error("Exact explicit offline operator arguments required")
values.set(match[1], match[2])
}
if (!["request", "private-key", "output"].every(key => values.has(key))) throw Error("Missing explicit offline operator arguments")
const boundedJson = async (filename: string) => {
const handle = await open(filename, "r")
try { const stat = await handle.stat(); if (!stat.isFile() || stat.size < 1 || stat.size > 8192) throw Error("Bounded operator input required"); return JSON.parse(await handle.readFile("utf8")) as unknown }
finally { await handle.close() }
}
const request = requestSchema.parse(await boundedJson(values.get("request")!))
const previousPin = values.has("previous-pin") ? await boundedJson(values.get("previous-pin")!) as DatabaseSourceDeploymentPin : undefined
return provisionDatabaseSourceBindingOffline({ request, privateKeyFile: values.get("private-key")!, outputDirectory: values.get("output")!, previousPin })
}
@@ -0,0 +1,105 @@
/** Explicit offline operator init. No database, application imports or activation. */
import assert from 'node:assert/strict'
import {createHash, generateKeyPairSync} from 'node:crypto'
import {constants} from 'node:fs'
import {lstat, realpath, open, mkdir, link, unlink} from 'node:fs/promises'
import path from 'node:path'
import {pathToFileURL} from 'node:url'
const digest = bytes => createHash('sha256').update(bytes).digest('hex')
const hex = value => typeof value === 'string' && /^[a-f0-9]{64}$/.test(value)
const uuid = value => typeof value === 'string' && /^[a-f0-9]{8}-[a-f0-9]{4}-[1-5][a-f0-9]{3}-[89ab][a-f0-9]{3}-[a-f0-9]{12}$/.test(value)
const exact = (value, keys) => value && typeof value === 'object' && !Array.isArray(value) && Object.keys(value).length === keys.length && keys.every(key => Object.hasOwn(value, key))
const roles = ['runtime', 'operator', 'capture', 'publisher', 'control']
export function deploymentRequest(value) {
assert(exact(value, ['format', 'installationId', 'sourceHeadSha256', 'source', 'roles', 'tls']))
assert(value.format === 'application-writer-bootstrap-request-v1' && uuid(value.installationId) && hex(value.sourceHeadSha256))
assert(exact(value.source, ['systemIdentifier', 'databaseOid', 'databaseName', 'serverVersionNum', 'inRecovery']))
assert(typeof value.source.systemIdentifier === 'string' && /^[1-9][0-9]{0,19}$/.test(value.source.systemIdentifier) && BigInt(value.source.systemIdentifier) <= 18446744073709551615n)
assert(typeof value.source.databaseOid === 'string' && /^[1-9][0-9]{0,9}$/.test(value.source.databaseOid) && BigInt(value.source.databaseOid) <= 4294967295n)
assert(typeof value.source.databaseName === 'string' && /^[^\x00-\x1f\x7f]{1,63}$/.test(value.source.databaseName))
assert(Number.isInteger(value.source.serverVersionNum) && value.source.serverVersionNum >= 180000 && value.source.serverVersionNum <= 189999 && value.source.inRecovery === false)
assert(exact(value.roles, roles))
for (const role of roles) {
const pair = value.roles[role]
assert(exact(pair, ['name', 'signedOid']) && typeof pair.name === 'string' && /^[a-z_][a-z0-9_]{0,62}$/.test(pair.name))
assert(typeof pair.signedOid === 'string' && /^[1-9][0-9]{0,9}$/.test(pair.signedOid) && BigInt(pair.signedOid) <= 4294967295n)
}
assert(new Set(roles.map(role => value.roles[role].name)).size === 5 && new Set(roles.map(role => value.roles[role].signedOid)).size === 5)
assert(exact(value.tls, ['serverName', 'caSha256', 'spkiSha256']) && typeof value.tls.serverName === 'string' && /^[a-zA-Z0-9][a-zA-Z0-9.-]{0,252}$/.test(value.tls.serverName) && hex(value.tls.caSha256) && hex(value.tls.spkiSha256))
return structuredClone(value)
}
export function createBootstrapMaterial(raw, requestSha256) {
const request = deploymentRequest(raw)
assert(hex(requestSha256))
const {privateKey, publicKey} = generateKeyPairSync('ed25519')
const publicKeyPem = publicKey.export({format: 'pem', type: 'spki'}).toString()
const publicKeySpkiSha256 = digest(publicKey.export({format: 'der', type: 'spki'}))
return {
privateKeyPem: privateKey.export({format: 'pem', type: 'pkcs8'}).toString(), publicKeyPem,
bootstrap: {format: 'application-writer-bootstrap-v1', verificationSubject: {installationId: request.installationId, publicKeyPem, publicKeySpkiSha256},
deploymentRequest: request, deploymentRequestSha256: requestSha256, keyOrigin: 'explicit-offline-operator-init',
databaseObserved: false, rolesAuthenticated: false, sourceAuthenticated: false, approvedDefinitionSha256: null,
publicationAuthorized: false, generationAuthenticated: false, readyAuthorized: false},
}
}
async function protectedDirectory(directory) {
assert(process.platform !== 'win32' && typeof process.getuid === 'function', 'Protected POSIX ownership required; Windows ACL backend unavailable')
assert(path.isAbsolute(directory) && await realpath(directory) === directory)
const uid = process.getuid(), stat = await lstat(directory)
assert(stat.isDirectory() && !stat.isSymbolicLink() && stat.uid === uid && (stat.mode & 0o077) === 0)
for (let ancestor = path.dirname(directory);; ancestor = path.dirname(ancestor)) {
const metadata = await lstat(ancestor)
assert(metadata.isDirectory() && !metadata.isSymbolicLink() && (metadata.uid === uid || metadata.uid === 0))
assert((metadata.mode & 0o022) === 0 || metadata.uid === 0 && (metadata.mode & 0o1000) !== 0)
if (ancestor === path.dirname(ancestor)) break
}
return stat
}
async function publish(directory, name, bytes) {
await protectedDirectory(directory)
const temporary = path.join(directory, '.' + name + '.pending'), destination = path.join(directory, name)
const handle = await open(temporary, constants.O_WRONLY | constants.O_CREAT | constants.O_EXCL | constants.O_NOFOLLOW, 0o600)
try { await handle.writeFile(bytes); await handle.sync() } finally { await handle.close() }
await link(temporary, destination) // Atomic no-replace publication, never rename-overwrite.
await unlink(temporary)
const parent = await open(directory, constants.O_RDONLY | constants.O_DIRECTORY | constants.O_NOFOLLOW)
try { await parent.sync() } finally { await parent.close() }
}
export async function initializeBootstrap({outputDirectory, deploymentFile, deploymentSha256}) {
assert(path.isAbsolute(outputDirectory) && path.resolve(outputDirectory) === outputDirectory && hex(deploymentSha256))
await protectedDirectory(path.dirname(outputDirectory))
assert(path.isAbsolute(deploymentFile) && await realpath(deploymentFile) === deploymentFile)
const requestHandle = await open(deploymentFile, constants.O_RDONLY | constants.O_NOFOLLOW)
let bytes
try {
const before = await requestHandle.stat()
assert(before.isFile() && before.uid === process.getuid() && (before.mode & 0o077) === 0 && before.size > 0 && before.size <= 16384)
bytes = Buffer.alloc(before.size + 1)
let size = 0
while (size < bytes.length) { const read = await requestHandle.read(bytes, size, bytes.length - size, size); if (!read.bytesRead) break; size += read.bytesRead }
const after = await requestHandle.stat()
assert(size === before.size && after.size === before.size && after.mtimeMs === before.mtimeMs && after.ctimeMs === before.ctimeMs)
bytes = bytes.subarray(0, size)
} finally { await requestHandle.close() }
assert(digest(bytes) === deploymentSha256, 'Explicit independently approved deployment digest required')
const request = deploymentRequest(JSON.parse(new TextDecoder('utf-8', {fatal: true}).decode(bytes)))
await mkdir(outputDirectory, {mode: 0o700}) // EEXIST refuses; never adopt/overwrite.
const material = createBootstrapMaterial(request, deploymentSha256)
await publish(outputDirectory, 'signing-private.pem', material.privateKeyPem)
await publish(outputDirectory, 'operator-public.pem', material.publicKeyPem)
await publish(outputDirectory, 'bootstrap.json', JSON.stringify(material.bootstrap, null, 2) + '\n') // Completion marker last.
return {event: 'application-writer-bootstrap-initialized', installationId: request.installationId,
publicKeySpkiSha256: material.bootstrap.verificationSubject.publicKeySpkiSha256,
deploymentRequestSha256: deploymentSha256, databaseObserved: false, publicationAuthorized: false, readyAuthorized: false}
}
export async function main(args) {
assert(args.length === 7 && args[0] === 'init')
const values = new Map()
for (let i = 1; i < args.length; i += 2) { assert(['--output', '--deployment', '--deployment-sha256'].includes(args[i]) && !values.has(args[i])); values.set(args[i], args[i + 1]) }
return initializeBootstrap({outputDirectory: values.get('--output'), deploymentFile: values.get('--deployment'), deploymentSha256: values.get('--deployment-sha256')})
}
if (process.argv[1] && import.meta.url === pathToFileURL(path.resolve(process.argv[1])).href) {
try { console.log(JSON.stringify(await main(process.argv.slice(2)))) }
catch { console.error(JSON.stringify({event: 'application-writer-bootstrap-refused', publicationAuthorized: false, readyAuthorized: false})); process.exitCode = 1 }
}
@@ -0,0 +1,43 @@
import test from 'node:test'
import assert from 'node:assert/strict'
import {createHash, createPrivateKey, createPublicKey, sign, verify} from 'node:crypto'
import {mkdtemp, rm, writeFile, readdir, readFile, stat, chmod} from 'node:fs/promises'
import path from 'node:path'
import {tmpdir} from 'node:os'
import {deploymentRequest, createBootstrapMaterial, initializeBootstrap, main} from './database-writer-bootstrap-operator.mjs'
const hash = b => createHash('sha256').update(b).digest('hex')
const request = () => ({format: 'application-writer-bootstrap-request-v1', installationId: 'f8b410ef-11ba-4419-b6a6-e735e505450b', sourceHeadSha256: 'a'.repeat(64),
source: {systemIdentifier: '7387529813445422155', databaseOid: '16384', databaseName: 'operator-fixture', serverVersionNum: 180004, inRecovery: false},
roles: Object.fromEntries(['runtime', 'operator', 'capture', 'publisher', 'control'].map((name, i) => [name, {name: 'fixture_' + name, signedOid: String(20000 + i)}])),
tls: {serverName: 'db.example.invalid', caSha256: 'b'.repeat(64), spkiSha256: 'c'.repeat(64)}})
test('real distinct Ed25519 keypairs sign/verify and preserve externally pinned subject without authorization', () => {
const value = request(), a = createBootstrapMaterial(value, 'd'.repeat(64)), b = createBootstrapMaterial(value, 'd'.repeat(64))
const privateKey = createPrivateKey(a.privateKeyPem), publicKey = createPublicKey(a.publicKeyPem), message = Buffer.from('owned-key-roundtrip')
assert(verify(null, message, publicKey, sign(null, message, privateKey))); assert(!verify(null, Buffer.from('changed'), publicKey, sign(null, message, privateKey)))
assert.equal(hash(publicKey.export({format: 'der', type: 'spki'})), a.bootstrap.verificationSubject.publicKeySpkiSha256)
assert.notEqual(a.publicKeyPem, b.publicKeyPem); assert.equal(a.bootstrap.verificationSubject.installationId, value.installationId)
assert.equal(a.bootstrap.publicationAuthorized, false); assert.equal(a.bootstrap.readyAuthorized, false); assert.equal(a.bootstrap.approvedDefinitionSha256, null)
assert.equal(a.bootstrap.databaseObserved, false); assert(!JSON.stringify(a.bootstrap).includes('PRIVATE KEY')); value.roles.runtime.name = 'changed'; assert.notEqual(a.bootstrap.deploymentRequest.roles.runtime.name, 'changed')
})
test('missing roles/head/transport pin and malformed identities refuse rather than invent trust', () => {
for (const change of [v => delete v.roles.control, v => delete v.sourceHeadSha256, v => delete v.tls.spkiSha256, v => v.roles.operator = v.roles.runtime, v => v.source.databaseOid = 16384, v => v.source.inRecovery = true, v => v.extra = true]) {
const value = request(); change(value); assert.throws(() => deploymentRequest(value))
}
})
test('unknown/duplicate CLI options refuse without an implicit startup mode', async () => {
for (const args of [[], ['publish'], ['init', '--output', '/x', '--output', '/y', '--deployment-sha256', 'd'.repeat(64)]]) await assert.rejects(main(args))
})
test('actual protected filesystem init, no-replace and mismatched digest; unsupported Windows writes nothing', async () => {
const parent = await mkdtemp(path.join(tmpdir(), 'mm-bootstrap-init-')), output = path.join(parent, 'keys'), file = path.join(parent, 'deployment.json'), bytes = Buffer.from(JSON.stringify(request()))
try {
await chmod(parent, 0o700); await writeFile(file, bytes, {mode: 0o600})
const options = {outputDirectory: output, deploymentFile: file, deploymentSha256: hash(bytes)}
if (process.platform === 'win32') { await assert.rejects(initializeBootstrap(options), /Windows ACL backend unavailable/); assert.deepEqual(await readdir(parent), ['deployment.json']); return }
await assert.rejects(initializeBootstrap({...options, deploymentSha256: '0'.repeat(64)})); assert.deepEqual(await readdir(parent), ['deployment.json'])
const receipt = await initializeBootstrap(options); assert.equal(receipt.publicationAuthorized, false); assert(!JSON.stringify(receipt).includes('PRIVATE KEY'))
assert.equal((await stat(output)).mode & 0o777, 0o700); assert.equal((await stat(path.join(output, 'signing-private.pem'))).mode & 0o777, 0o600)
const first = await readFile(path.join(output, 'signing-private.pem')); await assert.rejects(initializeBootstrap(options), {code: 'EEXIST'}); assert.deepEqual(await readFile(path.join(output, 'signing-private.pem')), first)
assert.deepEqual((await readdir(output)).sort(), ['bootstrap.json', 'operator-public.pem', 'signing-private.pem'])
assert.equal(JSON.parse(await readFile(path.join(output, 'bootstrap.json'), 'utf8')).verificationSubject.publicKeySpkiSha256, receipt.publicKeySpkiSha256)
} finally { await rm(parent, {recursive: true, force: true}) }
})
@@ -0,0 +1,186 @@
/** Explicit operator process only. No HTTP, runtime import, role provisioning or READY. */
import assert from 'node:assert/strict'
import path from 'node:path'
import {pathToFileURL} from 'node:url'
import {createPrivateKey,createPublicKey,sign} from 'node:crypto'
import {readdir} from 'node:fs/promises'
import {readProtected,protectedDirectory,sha,withJournalLease,writeFile} from './writer-profile/operator-files.mjs'
import {approval,upgradeApproval,reconciliationApproval,poolOptions,roleKeys,assertJsDriverEnvironment} from './writer-profile/operator-contract.mjs'
import {deploymentRequest} from './database-writer-bootstrap-operator.mjs'
const hex=(v:unknown)=>typeof v==='string'&&/^[a-f0-9]{64}$/.test(v)
const parse=(b:Buffer)=>JSON.parse(new TextDecoder('utf-8',{fatal:true}).decode(b))
const sourceCommands=['source-activate','source-revoke','source-status']
export async function main(args:string[]){
if(args[0]==='legacy-cold-verify')return (await import('./writer-profile/legacy-cold-verification.mjs')).verifyLegacyColdMain(args)
if(args[0]==='source58-foundation-enroll'||args[0]==='source57-admission-enroll')return (await import('./writer-profile/source58-foundation.mjs')).source58FoundationMain(args)
if(args[0]==='legacy-control-reference'||args[0]==='legacy-source58-preflight')return (await import('./writer-profile/legacy-control-reference.mjs')).legacyControlReferenceMain(args)
if(['legacy-handoff-execute','legacy-handoff-adopt','legacy-handoff-release'].includes(args[0]))return (await import('./writer-profile/legacy-handoff-execution.mjs')).legacyHandoffExecuteMain(args)
if(['registry-enroll','registry-inspect','restore-profile-enroll'].includes(args[0]))return (await import('./writer-profile/registry-enrollment.mjs')).registryEnrollmentMain(args)
if(args[0]==='marker-fence-enroll')return (await import('./writer-profile/marker-fence-enrollment.mjs')).markerFenceEnrollmentMain(args)
if(sourceCommands.includes(args[0]))return (await import('./writer-profile/source-activation.mjs')).sourceActivationMain(args)
if(['maintenance-inspect','maintenance-resolve'].includes(args[0]))return (await import('./writer-profile/maintenance-resolution.mjs')).maintenanceMain(args)
const legacyCustody=args[0]==='legacy-custody'
const legacyHandoff=args[0]==='legacy-handoff-inspect'
const legacyExport=legacyCustody||args[0]==='legacy-export'
const legacyInspection=legacyExport||legacyHandoff||args[0]==='legacy-inspect'
const catalogueInspection=args[0]==='source-catalogue-inspect'
const activePreflight=args[0]==='source-preflight',preflight=activePreflight||args[0]==='preflight'
const upgrade=args[0]==='upgrade'
const writes=upgrade||args[0]==='publish'
const flags=['bootstrap','bootstrap-sha256','transport','transport-sha256','ca','source-registry','source-spki-sha256']
if(!legacyInspection)flags.push('definition','definition-sha256')
if(!legacyInspection&&!catalogueInspection)flags.push('approval','approval-sha256','journal')
if(legacyInspection||catalogueInspection||activePreflight||writes)flags.push('activation-directory','activation-sha256')
if(legacyExport)flags.push('archive-directory','source-signing-key')
if(legacyHandoff)flags.push('archive-directory','handoff-intent-sha256')
assert((legacyInspection||catalogueInspection||preflight||writes||args[0]==='reconcile')&&args.length===1+2*flags.length)
assertJsDriverEnvironment(process.env)
const values=new Map<string,string>()
for(let i=1;i<args.length;i+=2){assert(args[i].startsWith('--')&&flags.includes(args[i].slice(2))&&!values.has(args[i].slice(2))&&args[i+1]);values.set(args[i].slice(2),args[i+1])}
const get=(name:string)=>values.get(name)!
for(const flag of flags.filter(f=>f.endsWith('sha256')))assert(hex(get(flag)))
const bootstrap=parse(await readProtected(get('bootstrap'),32768,get('bootstrap-sha256')))
assert(bootstrap.format==='application-writer-bootstrap-v1'&&bootstrap.publicationAuthorized===false&&bootstrap.readyAuthorized===false)
const request=deploymentRequest(bootstrap.deploymentRequest),subject=bootstrap.verificationSubject
assert(subject.installationId===request.installationId&&hex(subject.publicKeySpkiSha256)&&sha(createPublicKey(subject.publicKeyPem).export({format:'der',type:'spki'}))===subject.publicKeySpkiSha256)
const pinned=legacyInspection||catalogueInspection?undefined:(args[0]==='reconcile'?reconciliationApproval:upgrade?upgradeApproval:approval)(parse(await readProtected(get('approval'),16384,get('approval-sha256'))))
const definition=legacyInspection?undefined:parse(await readProtected(get('definition'),4194304,get('definition-sha256')))
const transport=parse(await readProtected(get('transport'),16384,get('transport-sha256')))
const ca=await readProtected(get('ca'),65536,request.tls.caSha256)
const options=poolOptions(transport,request,ca)
if(!legacyInspection&&!catalogueInspection)await protectedDirectory(get('journal'))
await protectedDirectory(get('source-registry'))
const privateKeyFile=path.join(path.dirname(get('bootstrap')),'signing-private.pem')
if(writes){const key=await readProtected(privateKeyFile,8192);key.fill(0)}
// All explicit trust files are checked before app helper/PG imports. No .env URL is used.
// Existing source helper loads config/dotenv. Complete it BEFORE pg's first
// import so a newly loaded NODE_PG_FORCE_NATIVE cannot select another driver.
const {createProtectedDatabaseSourceBindingVerifier}=await import('./writer-profile/protected-source-registry.mjs')
assertJsDriverEnvironment(process.env)
const [{default:pg},{createConnectedPublisher,requireCurrentCatalogueDefinition},{collectSource44Catalogue,collectSource56Catalogue,collectSource57Catalogue,collectSource58Catalogue,approvedDefinition}]=await Promise.all([
import('pg'),import('./writer-profile/publisher.mjs'),import('./writer-profile/catalogue.mjs')])
assertJsDriverEnvironment(process.env)
const reviewedDefinition=legacyInspection?undefined:approvedDefinition(definition)
const collectCatalogues={44:collectSource44Catalogue,56:collectSource56Catalogue,57:collectSource57Catalogue,58:collectSource58Catalogue}
const collectCatalogue=collectCatalogues[reviewedDefinition&&'edition' in reviewedDefinition?reviewedDefinition.edition:44]
if(!legacyInspection&&!catalogueInspection){
assert.equal(reviewedDefinition!.definitionSha256,pinned!.definitionSha256)
if(args[0]!=='reconcile')requireCurrentCatalogueDefinition(pinned,reviewedDefinition)
}
const installation=Object.freeze({...subject,roles:Object.fromEntries(roleKeys.map(k=>[k,request.roles[k].signedOid])),roleBindings:request.roles,expectedSource:request.source,tls:Object.freeze({...request.tls})})
const sourcePin={registryDirectory:get('source-registry'),installationId:subject.installationId,publicKeySpkiSha256:get('source-spki-sha256'),currentHeadSha256:request.sourceHeadSha256}
const archiveDirectory=legacyExport?get('archive-directory'):undefined
let handoff
if(legacyHandoff){
const {verifyLegacyProbeHandoffIntent}=await import('../src/services/server-probe-legacy-handoff-inspection.js')
const directory=get('archive-directory'),maximum=64*1024*1024
const [archive,signedApproval,coldArchive,intent,operatorPublicKeyPem]=await Promise.all([
readProtected(path.join(directory,'legacy-probe.dump'),maximum),readProtected(path.join(directory,'legacy-probe-approval.json'),4096),
readProtected(path.join(directory,'legacy-probe-cold.dump'),maximum),
readProtected(path.join(directory,'legacy-probe-handoff-intent.json'),4096,get('handoff-intent-sha256')),
readProtected(path.join(sourcePin.registryDirectory,'operator-public.pem'),4096),
])
handoff=await verifyLegacyProbeHandoffIntent(sourcePin,{archive,signedApproval,coldArchive,intent,operatorPublicKeyPem})
}
// Use the same protected registry evidence for inspection and publication.
const verifier=createProtectedDatabaseSourceBindingVerifier(sourcePin)
async function connected(sourceVerifier:typeof verifier,deadlineAt:number){
const publisher=createConnectedPublisher(installation,(s:Parameters<typeof collectCatalogue>[0])=>collectCatalogue(s,installation),sourceVerifier,pinned,reviewedDefinition)
const pool=new pg.Pool(options)
// A pool-level async error never becomes raw stdout or a retry signal.
let asyncFailure=false;pool.on('error',()=>{asyncFailure=true})
let result,primary:unknown,failed=false
try{
const journalDirectory=get('journal')
if(legacyInspection)result=await publisher.inspectLegacyStorage(pool,{deadlineAt,archiveDirectory,custody:legacyCustody,handoff})
else if(catalogueInspection)result=await publisher.inspectCurrentCatalogue(pool,{deadlineAt})
else if(preflight)result=await publisher.preflightCurrentProfile(pool,{deadlineAt})
else if(args[0]==='reconcile')result=await publisher.reconcilePendingPublication(pool,{journalDirectory,deadlineAt})
else{
const publication={privateKeyFile,expectedHeadSha256:pinned.expectedHeadSha256,journalDirectory,validityMs:pinned.validityMs,deadlineAt}
result=upgrade?await publisher.upgradeCurrentProfile(pool,publication):await publisher.publishCurrentProfile(pool,publication)
}
}
catch(error){primary=error;failed=true}
try{await pool.end()}catch(error){primary??=error;failed=true}
if(legacyInspection&&result?.state==='observed'&&result.source.platformProtectionVerified===true&&!failed&&!asyncFailure){
const {installationId,sourceIdentityId,generation,ordinal,headSha256,bindingSha256}=result.source
if(legacyHandoff){assert(result.resolution);return {event:'database-legacy-handoff-inspection',...result.resolution,sourceActivationMatched:true,conversionAllowed:false}}
return {...(legacyExport?{archive:result.archive}:{}),...(result.lineageSnapshot?{lineageSnapshot:result.lineageSnapshot}:{}),event:'database-legacy-inspection',outcome:'observed',snapshot:result.snapshot,
source:{installationId,sourceIdentityId,generation,ordinal,headSha256,bindingSha256},sourceActivationMatched:true,
platformProtectionVerified:true,generationAuthenticated:false,allWriterDrainAuthenticated:false,
conversionAllowed:false,readyAuthorized:false,automaticReplay:false}
}
if(catalogueInspection&&result?.state==='observed'&&result.platformProtectionVerified===true&&!failed&&!asyncFailure){
const {state:_state,...catalogue}=result
return {event:'application-writer-source-catalogue-inspection',outcome:'observed',...catalogue,automaticReplay:false,writerAuthorized:false}
}
if(preflight&&result?.state==='matched'&&result.platformProtectionVerified===true&&!failed&&!asyncFailure)return {event:'application-writer-profile-preflight',outcome:'matched',epoch:result.epoch,headSha256:result.headSha256,notAfter:result.notAfter,sourceMatched:true,rolesMatched:true,catalogueMatched:true,platformProtectionVerified:true,writerAuthorized:false,cleanupUnconfirmed:false,automaticReplay:false,generationAuthenticated:false,readyAuthorized:false}
if(result?.outcome==='committed')return {event:'application-writer-profile-publication',outcome:'committed',epoch:result.epoch,headSha256:result.headSha256,notAfter:result.notAfter,
cleanupUnconfirmed:result.cleanupUnconfirmed||asyncFailure||failed,journalUnconfirmed:!!result.journalUnconfirmed,automaticReplay:false,generationAuthenticated:false,readyAuthorized:false}
if(args[0]==='reconcile'&&result&&!failed&&!asyncFailure&&['idle','confirmed','not-committed'].includes(result.state))return {event:'application-writer-profile-reconciliation',outcome:result.state,headSha256:result.headSha256??null,cleanupUnconfirmed:false,journalUnconfirmed:false,automaticReplay:false,generationAuthenticated:false,readyAuthorized:false}
throw primary??Error('Publisher terminal unconfirmed')
}
const deadlineAt=Date.now()+120000
if(!legacyInspection&&!catalogueInspection&&!activePreflight&&!writes)return connected(verifier,deadlineAt)
const {withActiveSource}=await import('./writer-profile/source-activation.mjs')
const publisherRole=request.roles.publisher.name
const expectedSource={...request.source,databaseOid:Number(request.source.databaseOid),sessionUser:publisherRole,currentUser:publisherRole}
const active=()=>withActiveSource({directory:get('activation-directory'),expectedSha256:get('activation-sha256'),sourcePin,expectedSource,tlsSpkiSha256:request.tls.spkiSha256,deadlineAt},verifier,
guardedVerifier=>connected(guardedVerifier,deadlineAt))
if(archiveDirectory)return withJournalLease(archiveDirectory,async()=>{
assert.deepEqual(await readdir(archiveDirectory),['publisher-lease.lock'],'Empty protected archive directory required')
const keyBytes=await readProtected(get('source-signing-key'),8192)
let key
try{key=createPrivateKey(keyBytes)}finally{keyBytes.fill(0)}
assert(key.asymmetricKeyType==='ed25519'&&sha(createPublicKey(key).export({type:'spki',format:'der'}))===sourcePin.publicKeySpkiSha256,'Source signer differs from deployment pin')
const {sourceBindingBytes,sourceBindingCanonical}=await import('../src/db/database-source-binding.js')
const {LEGACY_PROBE_ARCHIVE_APPROVAL_DOMAIN,LEGACY_PROBE_CUSTODY_APPROVAL_DOMAIN}=await import('../src/services/server-probe-legacy-archive-approval.js')
const result=await active()
assert(result.event==='database-legacy-inspection'&&Buffer.isBuffer(result.archive)&&Date.now()<deadlineAt)
const {archive,...receipt}=result
const base={...receipt.source,archiveSha256:sha(archive),storageSha256:receipt.snapshot.sha256}
assert(!legacyCustody||receipt.lineageSnapshot)
const body=legacyCustody?{format:'legacy-probe-custody-approval-v1',...base,lineageSha256:receipt.lineageSnapshot.sha256}
:{format:'legacy-probe-archive-approval-v1',...base}
const domain=legacyCustody?LEGACY_PROBE_CUSTODY_APPROVAL_DOMAIN:LEGACY_PROBE_ARCHIVE_APPROVAL_DOMAIN
const signedApproval=sourceBindingBytes({body,signature:sign(null,Buffer.from(domain+sourceBindingCanonical(body)),key).toString('base64url')})
const dumpFile='legacy-probe.dump',approvalFile='legacy-probe-approval.json'
await writeFile(path.join(archiveDirectory,dumpFile),archive,{flag:'wx',mode:0o600})
await writeFile(path.join(archiveDirectory,approvalFile),signedApproval,{flag:'wx',mode:0o600})
return {...receipt,event:legacyCustody?'database-legacy-custody':'database-legacy-export',outcome:'exported',dumpFile,approvalFile,archiveSha256:body.archiveSha256,approvalSha256:sha(signedApproval)}
})
const result=await active()
if(writes||legacyInspection||catalogueInspection)return result
assert.equal(result.event,'application-writer-profile-preflight')
return {...result,event:'application-writer-profile-source-preflight',sourceActivationMatched:true}
}
if(process.argv[1]&&import.meta.url===pathToFileURL(path.resolve(process.argv[1])).href){
try{const result=await main(process.argv.slice(2));console.log(JSON.stringify(result));if(result.cleanupUnconfirmed||result.journalUnconfirmed)process.exitCode=2}
catch(error){
const sourceCommand=sourceCommands.includes(process.argv[2])
const command=process.argv[2], code=(error as NodeJS.ErrnoException|null)?.code
const unknown=sourceCommand&&code==='SOURCE_ACTIVATION_TERMINAL_UNKNOWN'||['registry-enroll','restore-profile-enroll'].includes(command)&&code==='WRITER_REGISTRY_TERMINAL_UNKNOWN'||
command==='marker-fence-enroll'&&code==='MARKER_FENCE_TERMINAL_UNKNOWN'||command==='legacy-handoff-execute'&&code==='LEGACY_HANDOFF_EXECUTION_UNKNOWN'||
command==='legacy-handoff-adopt'&&code==='LEGACY_HANDOFF_ADOPTION_UNKNOWN'||command==='legacy-handoff-release'&&code==='LEGACY_HANDOFF_RELEASE_UNKNOWN'||
command==='source58-foundation-enroll'&&code==='SOURCE58_FOUNDATION_UNKNOWN'||command==='source57-admission-enroll'&&code==='SOURCE57_ADMISSION_UNKNOWN'
console.error(JSON.stringify({event:sourceCommand?(unknown?'database-source-activation-unknown':'database-source-activation-refused'):
command==='registry-inspect'?'application-writer-registry-inspection-refused':
command==='registry-enroll'?(unknown?'application-writer-registry-enrollment-unknown':'application-writer-registry-enrollment-refused'):
command==='restore-profile-enroll'?(unknown?'application-writer-restore-profile-enrollment-unknown':'application-writer-restore-profile-enrollment-refused'):
command==='source-catalogue-inspect'?'application-writer-source-catalogue-inspection-refused':
command==='marker-fence-enroll'?(unknown?'application-writer-marker-fence-enrollment-unknown':'application-writer-marker-fence-enrollment-refused'):
command==='legacy-export'?'database-legacy-export-refused':
command==='legacy-custody'?'database-legacy-custody-refused':
command==='legacy-handoff-inspect'?'database-legacy-handoff-inspection-refused':
command==='legacy-cold-verify'?'database-legacy-cold-proof-refused':
command==='legacy-handoff-execute'?(unknown?'database-legacy-handoff-execution-unknown':'database-legacy-handoff-execution-refused'):
command==='legacy-handoff-adopt'?(unknown?'database-legacy-handoff-adoption-unknown':'database-legacy-handoff-adoption-refused'):
command==='legacy-handoff-release'?(unknown?'database-legacy-handoff-release-unknown':'database-legacy-handoff-release-refused'):
command==='legacy-control-reference'?'database-legacy-control-reference-refused':
command==='legacy-source58-preflight'?'database-legacy-source58-preflight-refused':
command==='source58-foundation-enroll'?(unknown?'database-source58-foundation-unknown':'database-source58-foundation-refused'):
command==='source57-admission-enroll'?(unknown?'database-source57-admission-unknown':'database-source57-admission-refused'):
['maintenance-inspect','maintenance-resolve'].includes(process.argv[2])?'database-maintenance-command-refused':process.argv[2]==='legacy-inspect'?'database-legacy-inspection-refused':process.argv[2]==='source-preflight'?'application-writer-profile-source-preflight-refused':'application-writer-profile-publication-refused',automaticReplay:false,inspectPendingBeforeRetry:true,readyAuthorized:false}));process.exitCode=unknown?2:1
}
}
@@ -0,0 +1,63 @@
/* eslint-disable @typescript-eslint/no-explicit-any -- loosely typed fixtures in an operator test */
import test from 'node:test'
import assert from 'node:assert/strict'
import {generateKeyPairSync,createHash,X509Certificate,sign} from 'node:crypto'
import {rootCertificates} from 'node:tls'
import {createRequire} from 'node:module'
import {approval,upgradeApproval,reconciliationApproval,poolOptions,STARTUP_OPTIONS,assertJsDriverEnvironment,REBIND58_APPROVAL_FORMAT} from './writer-profile/operator-contract.mjs'
import {bytes,canonical,POLICY_DOMAIN,signedEnvelope} from '../src/db/application-writer-profile-format.ts'
const hash=(v:Buffer|string)=>createHash('sha256').update(v).digest('hex')
const pinned=()=>({format:'application-writer-publication-approval-v1',manifestSha256:'a'.repeat(64),definitionSha256:'b'.repeat(64),catalogSha256:'c'.repeat(64),expectedHeadSha256:null,validityMs:86400000,expiresAt:Date.now()+60000})
test('approval is explicit, exact, future-bounded and does not learn missing expected fields',()=>{
const value=pinned();assert.deepEqual(approval(value),value)
for(const change of [(v:any)=>delete v.definitionSha256,(v:any)=>delete v.catalogSha256,(v:any)=>v.expiresAt=Date.now(),(v:any)=>v.validityMs=86400001,(v:any)=>v.expectedHeadSha256='observed',(v:any)=>v.provider=true]){const v=pinned();change(v);assert.throws(()=>approval(v))}
})
test('isolated Pool1 options require pinned CA/host/role/database and native TLS verification',()=>{
const ca=Buffer.from(rootCertificates[0]),request={roles:{publisher:{name:'installation_policy_signer'}},source:{databaseName:'owned'},tls:{serverName:'db.example.invalid',caSha256:hash(ca),spkiSha256:hash(new X509Certificate(ca).publicKey.export({format:'der',type:'spki'}))}}
const transport={host:'db.example.invalid',port:5432,user:'installation_policy_signer',password:'test-only',database:'owned'},config=poolOptions(transport,request,ca)
assert.equal(config.max,1);assert.equal(config.connectionTimeoutMillis,5000);assert.equal(config.ssl.rejectUnauthorized,true)
assert(config.ssl.checkServerIdentity('attacker.invalid',new X509Certificate(ca).toLegacyObject()) instanceof Error)
for(const changed of [{...transport,user:'admin'},{...transport,host:'attacker.invalid'},{...transport,database:'other'},{...transport,connectionString:'postgres://env-substitution'},{...transport,port:0}])assert.throws(()=>poolOptions(changed,request,ca))
assert.throws(()=>poolOptions(transport,request,Buffer.from('not approved CA')))
})
test('real production envelope parser accepts operator-key signature and rejects tamper/foreign key',()=>{
const now=Date.now(),{publicKey,privateKey}=generateKeyPairSync('ed25519'),installation={installationId:'788629ed-e00d-4af8-bc33-c705e5a7b62d',publicKeyPem:publicKey.export({type:'spki',format:'pem'}).toString(),publicKeySpkiSha256:hash(publicKey.export({type:'spki',format:'der'}))}
const body={format:'application-writer-policy-v1',installationId:installation.installationId,epoch:'1',issuedAt:now,notBefore:now,notAfter:now+86400000,source:{systemIdentifier:'7348518042453055651',databaseOid:'16384',databaseName:'owned',serverVersionNum:180004,inRecovery:false},sourceObservationSha256:'a'.repeat(64),roles:{runtime:'20001',operator:'20002',capture:'20003',publisher:'20004',control:'20005'},manifestSha256:'b'.repeat(64),definitionSha256:'c'.repeat(64),catalogSha256:'d'.repeat(64),futureLeaves:'fixed12-native-policy-v1'}
const signature=sign(null,Buffer.from(POLICY_DOMAIN+canonical(body)),privateKey).toString('base64url')
assert.equal(signedEnvelope(bytes({body,signature}),'policy',installation).body.epoch,'1')
assert.throws(()=>signedEnvelope(bytes({body:{...body,catalogSha256:'e'.repeat(64)},signature}),'policy',installation))
const other=generateKeyPairSync('ed25519').publicKey;assert.throws(()=>signedEnvelope(bytes({body,signature}),'policy',{...installation,publicKeyPem:other.export({type:'spki',format:'pem'}).toString(),publicKeySpkiSha256:hash(other.export({type:'spki',format:'der'}))}))
})
test('actual pg ConnectionParameters cannot inherit malicious PGOPTIONS and native selection refuses',()=>{
const require=createRequire(new URL('../package.json',import.meta.url)),ConnectionParameters=require('pg/lib/connection-parameters')
const original=process.env.PGOPTIONS;process.env.PGOPTIONS='-c role=attacker -c search_path=attacker'
try{
const ca=Buffer.from(rootCertificates[0]),request={roles:{publisher:{name:'installation_policy_signer'}},source:{databaseName:'owned'},tls:{serverName:'db.example.invalid',caSha256:hash(ca),spkiSha256:'a'.repeat(64)}}
const config=poolOptions({host:'db.example.invalid',port:5432,user:'installation_policy_signer',password:'test-only',database:'owned'},request,ca)
const actual=new ConnectionParameters(config);assert.equal(actual.options,STARTUP_OPTIONS);assert(!actual.options.includes('attacker'));assert.equal(actual.user,'installation_policy_signer');assert.equal(actual.ssl.rejectUnauthorized,true)
}finally{if(original===undefined)delete process.env.PGOPTIONS;else process.env.PGOPTIONS=original}
assert.doesNotThrow(()=>assertJsDriverEnvironment({}));for(const value of ['1','0','',undefined])assert.throws(()=>assertJsDriverEnvironment({NODE_PG_FORCE_NATIVE:value}))
})
test('reconciliation accepts expired original approval as copied inspection data; publication still refuses',()=>{
const value={...pinned(),expiresAt:1},copy=reconciliationApproval(value)
assert.deepEqual(copy,value);assert.notEqual(copy,value);assert.throws(()=>approval(value))
copy.catalogSha256='d'.repeat(64);assert.equal(value.catalogSha256,'c'.repeat(64))
const future=pinned();assert.deepEqual(approval(future),reconciliationApproval(future))
})
test('reconciliation keeps exact digest/head/schema and original lifetime bounds',()=>{
for(const mutate of [(v:any)=>delete v.definitionSha256,(v:any)=>v.catalogSha256='observed',(v:any)=>v.expectedHeadSha256='foreign',(v:any)=>v.extra=true,(v:any)=>v.format='other',(v:any)=>v.validityMs=0,(v:any)=>v.validityMs=86400001,(v:any)=>v.expiresAt=0,(v:any)=>v.expiresAt=-1,(v:any)=>v.expiresAt=Number.MAX_SAFE_INTEGER+1]){
const value={...pinned(),expiresAt:1};mutate(value);assert.throws(()=>reconciliationApproval(value));assert.throws(()=>approval(value))
}
})
test('publication expiry guard remains mandatory at the real clock boundary',()=>{
const value=pinned();value.expiresAt=Date.now();assert.throws(()=>approval(value));assert.deepEqual(reconciliationApproval(value),value)
})
test('restore rebind requires a protected exact old head and changed approved catalogue',()=>{
const value={...pinned(),format:REBIND58_APPROVAL_FORMAT,previousCatalogSha256:'d'.repeat(64),expectedHeadSha256:'e'.repeat(64)}
assert.deepEqual(upgradeApproval(value),value)
assert.throws(()=>approval(value))
assert.throws(()=>upgradeApproval({...value,previousCatalogSha256:value.catalogSha256}))
assert.throws(()=>upgradeApproval({...value,expectedHeadSha256:null}))
assert.throws(()=>upgradeApproval({...value,expiresAt:1}))
})
@@ -0,0 +1,355 @@
// Candidate component tests. Actual signature/current reader, modeled PG/source/catalogue.
// No genuine SQL, TLS session, filesystem ACL or generation acceptance is claimed.
import test from 'node:test'
import assert from 'node:assert/strict'
import fs from 'node:fs/promises'
import {createRequire} from 'node:module'
import {runInNewContext} from 'node:vm'
import {EventEmitter} from 'node:events'
import path from 'node:path'
import * as format from '../src/db/application-writer-profile-format.ts'
import {MIGRATION_MANIFEST,migrationManifestDigest} from '../src/db/migration-manifest.ts'
import {readCurrentApplicationWriterProfile} from '../src/db/application-writer-profile-registry-read.ts'
import {syntheticFixture,stagedModelClient} from '../src/db/application-writer-profile-registry-read-model.test.ts'
import {approvedDefinition} from './writer-profile/catalogue.mts'
import {BUILD44_PROVENANCE,FIXED44_ROLE_TEMPLATE} from './writer-profile/fixed44-build-definition.ts'
import {BUILD56_PROVENANCE} from './writer-profile/fixed56-build-definition.ts'
import {BUILD57_PROVENANCE,FIXED57_ROLE_TEMPLATE} from './writer-profile/fixed57-build-definition.ts'
import {approval as parseApproval,upgradeApproval,reconciliationApproval,isUpgradeApprovalFormat,UPGRADE_APPROVAL_FORMAT,UPGRADE57_APPROVAL_FORMAT} from './writer-profile/operator-contract.mjs'
const require=createRequire(new URL('../package.json',import.meta.url)),ts=require('typescript'),pg=require('pg')
const candidate=await fs.readFile(new URL('./writer-profile/publisher.mjs',import.meta.url),'utf8')
interface PreflightDiagnostics {state:string;headSha256:string;sourceMatched:boolean;rolesMatched:boolean;catalogueMatched:boolean;platformProtectionVerified:boolean;writerAuthorized:boolean;generationAuthenticated:boolean;readyAuthorized:boolean}
type PublishOptions={privateKeyFile:string;expectedHeadSha256:string|null;journalDirectory:string;deadlineAt:number;validityMs?:number}
interface PublisherModule {
requireCurrentCatalogueDefinition(approval:unknown,definition:unknown):string
createConnectedPublisher(installation:unknown,collect:()=>Promise<unknown>,sourceVerifier:{verify:()=>Promise<{platformProtectionVerified:boolean}>},approval:unknown,definition:unknown):{
inspectLegacyStorage(pool:unknown,options:{deadlineAt:number}):Promise<{state:string;snapshot:{rowCount:number}}>
preflightCurrentProfile(pool:unknown,options:{deadlineAt:number}):Promise<PreflightDiagnostics>
publishCurrentProfile(pool:unknown,options:PublishOptions):Promise<{outcome:string}>
upgradeCurrentProfile(pool:unknown,options:PublishOptions):Promise<{outcome:string;epoch:string;headSha256:string}>
}
}
interface OperatorModule {main(args:string[]):Promise<{event:string;outcome:string;platformProtectionVerified:boolean;writerAuthorized:boolean;readyAuthorized:boolean;sourceActivationMatched?:boolean}>}
function compile<T>(source:string,modules:Record<string,unknown>):T{
const exports:Record<string,unknown>={};const compiled=ts.transpileModule(source.split('\nif(process.argv[1]')[0].replaceAll('import.meta.url',JSON.stringify('file:///candidate-not-cli.mts')),{compilerOptions:{module:ts.ModuleKind.CommonJS,target:ts.ScriptTarget.ES2022,esModuleInterop:true}}).outputText
runInNewContext(compiled,{exports,require:(name:string)=>{assert(name in modules,'Unapproved import '+name);return modules[name]},Buffer,URL,TextDecoder,Date,setTimeout,clearTimeout,Object,Error,AggregateError,Promise,Map,Set,process:{env:{},argv:[]}}, {filename:'candidate.mjs'})
// The isolated VM loads the exact source above; tests exercise this narrow export contract.
return exports as T
}
const frozenDefinition=approvedDefinition({format:'application-writer-source44-definition-approval-v1',provenance:BUILD44_PROVENANCE,template:FIXED44_ROLE_TEMPLATE})
const frozenMigrations=MIGRATION_MANIFEST.slice(0,44),markerIds=frozenMigrations.map(pin=>pin.file.slice(0,-4))
const markerSql='SELECT id FROM public.schema_migrations ORDER BY id COLLATE "C" LIMIT $1'
function fixture(fault='',forPublication=false,upgradeMode?:'upgrade'|'upgrade57'|'ordinary-renewal'){
const f=syntheticFixture('1'),security={sha256:'e'.repeat(64),roles:{...f.policy.roles}}
const predecessor=upgradeMode==='upgrade57'?BUILD56_PROVENANCE:BUILD44_PROVENANCE
const successor=upgradeMode==='upgrade57'?BUILD57_PROVENANCE:BUILD56_PROVENANCE
const target=upgradeMode?{definitionSha256:successor.templateSha256,manifestSha256:successor.manifestSha256}:frozenDefinition
const targetMigrations=MIGRATION_MANIFEST.slice(0,upgradeMode==='upgrade57'?57:56)
const catalogue={...target,catalogSha256:'f'.repeat(64),leafCount:117}
const currentBuild=fault==='current-build'||fault==='current-approved-frozen-definition'
f.policy.manifestSha256=predecessor.manifestSha256;f.policy.definitionSha256=predecessor.templateSha256
f.policy.catalogSha256=upgradeMode?'d'.repeat(64):format.sha(format.canonical({base:catalogue.catalogSha256,security:security.sha256}))
if(fault==='signed-source')f.policy.source.databaseOid='999'
if(fault==='signed-role')f.policy.roles.operator='99'
if(fault==='signed-catalogue')f.policy.catalogSha256='1'.repeat(64)
if(fault==='signed-definition')f.policy.definitionSha256='1'.repeat(64)
if(fault==='signed-manifest')f.policy.manifestSha256='1'.repeat(64)
if(fault==='expired'){f.policy.issuedAt=Date.now()-60000;f.policy.notBefore=f.policy.issuedAt;f.policy.notAfter=Date.now()-1000}
const raw=f.signBody(f.policy,format.POLICY_DOMAIN),hash=format.sha(raw),top=f.state.latest[0]
top.policy_sha256=hash;top.head_bytes=f.signBody({format:'application-writer-head-v1',installationId:f.subject.installationId,epoch:'1',policySha256:hash,previousHeadSha256:null},format.HEAD_DOMAIN);top.head_sha256=format.sha(top.head_bytes)
f.state.current[0].head_sha256=top.head_sha256;f.state.policy=[{policy_sha256:hash,policy_bytes:raw}]
const priorHead=top.head_sha256
if(fault==='tamper')f.state.policy[0].policy_bytes=Buffer.from(raw.toString().replace('synthetic','tampered'))
if(fault==='empty'){f.state.current=[];f.state.latest=[];f.state.policy=[]}
const model=stagedModelClient(f.state),queries:string[]=[],events:string[]=[],markerLimits:unknown[][]=[]
const writes:{sql:string;values?:unknown[]}[]=[]
let clocks=0,checkouts=0
const client=Object.assign(new EventEmitter(),{query:async(sql:string,values?:unknown[])=>{
queries.push(sql)
if(/^(INSERT|UPDATE) INTO |^UPDATE mmapp_control\./.test(sql))writes.push({sql,values})
if(sql.includes('clock_timestamp()')&&++clocks===2&&fault==='late-clock')return {rows:[{now:'1'}]}
if(sql===markerSql){
markerLimits.push(values??[])
const expected=upgradeMode?targetMigrations.map(pin=>pin.file.slice(0,-4)):markerIds
const ids=fault==='marker-missing'?expected.slice(0,-1):fault==='marker-extra'||fault==='marker-drift'&&markerLimits.length===2?[...expected,'9999_foreign']:expected
return {rows:ids.map(id=>({id}))}
}
if(sql.startsWith('WITH')||sql.includes('clock_timestamp()'))return model.client.query(sql,values)
if(sql.startsWith('SELECT pg_catalog.pg_advisory_unlock'))return {rows:[{released:true}]}
if(sql==='ROLLBACK')return {rows:[],command:fault==='rollback-ack'?'':'ROLLBACK'}
if(sql.startsWith('INSERT INTO mmapp_control.current')||sql.startsWith('UPDATE mmapp_control.current'))return {rows:[{id:1}]}
return {rows:[],command:sql.split(' ')[0]}
},release:(bad:boolean)=>{events.push('release:'+bad)}})
const pool=new pg.Pool();pool.connect=async()=>{checkouts++;return client}
const source={systemIdentifier:'123',databaseOid:'456',databaseName:'synthetic',serverVersionNum:180004,inRecovery:false}
let contexts=0,verifications=0
const installation={...f.subject,expectedSource:source,roleBindings:{publisher:{name:'publisher',signedOid:'14'}}}
const pinned={expiresAt:Date.now()+60000,manifestSha256:['approval-manifest','current-approved-frozen-definition'].includes(fault)?migrationManifestDigest():target.manifestSha256,definitionSha256:catalogue.definitionSha256,catalogSha256:format.sha(format.canonical({base:catalogue.catalogSha256,security:security.sha256})),...(upgradeMode==='upgrade'||upgradeMode==='upgrade57'?{
format:upgradeMode==='upgrade57'&&fault!=='wrong-transition'?UPGRADE57_APPROVAL_FORMAT:UPGRADE_APPROVAL_FORMAT,expectedHeadSha256:fault==='wrong-head'?'0'.repeat(64):priorHead,previousCatalogSha256:fault==='wrong-previous-catalogue'?'0'.repeat(64):'d'.repeat(64),validityMs:60000,
}:{})}
const definition=fault==='static-definition'?{...target,definitionSha256:'0'.repeat(64)}:target
const privateKeyFile=path.resolve('modeled-fixture-key.pem'),journalDirectory=path.resolve('modeled-journal'),files=new Map<string,Buffer>()
if(forPublication)files.set(privateKeyFile,Buffer.from(f.privateKey.export({format:'pem',type:'pkcs8'})))
function file(filename:string){const value=files.get(filename);if(!value)throw Object.assign(Error('Fixture file absent'),{code:'ENOENT'});return value}
function stat(filename:string){return {size:file(filename).length,dev:1,ino:1,isFile:()=>true,isSymbolicLink:()=>false}}
const modules={
pg,'node:crypto':require('node:crypto'),'node:fs/promises':{realpath:async(file:string)=>file,lstat:async(filename:string)=>stat(filename),unlink:async(filename:string)=>files.delete(filename),open:async(filename:string)=>({stat:async()=>stat(filename),readFile:async()=>Buffer.from(file(filename)),close:async()=>{}})},'node:path':path,
'./fixed44-build-definition.js':{BUILD44_PROVENANCE},'./fixed56-build-definition.js':{BUILD56_PROVENANCE},'./fixed57-build-definition.js':{BUILD57_PROVENANCE},
'./operator-contract.mjs':{UPGRADE_APPROVAL_FORMAT,UPGRADE57_APPROVAL_FORMAT,isUpgradeApprovalFormat,upgradeApproval,retainPublisherTlsPeer:(actual:unknown)=>{
assert.equal(actual,client)
if(fault==='initial-tls')throw Error('TLS peer refused')
return ()=>{if(fault==='lost-tls'&&verifications===2)throw Error('Retained TLS lost')}
}},
'./operator-files.mjs':{withJournalLease:async(_directory:string,work:()=>Promise<unknown>)=>{assert(forPublication);events.push('journal-lease');return work()},writeFile:async(filename:string,value:Buffer)=>{assert(forPublication);files.set(filename,value)}},
'../../src/db/database-source-observation.js':{captureDatabaseSourceObservation:async()=>{events.push('capture-source');return {}},assertDatabaseSourceObservationCurrent:async()=>{events.push('fresh-source');if(fault==='late-source')throw Error('Source changed')}},
'../../src/services/server-probe-legacy-source-inventory.js':{captureLegacyProbeStorage:async()=>{events.push('capture-legacy');return {rowCount:1}}},
'../../src/db/migration-manifest.js':{MIGRATION_MANIFEST:currentBuild?MIGRATION_MANIFEST:upgradeMode?targetMigrations:frozenMigrations,migrationManifestDigest:()=>currentBuild?migrationManifestDigest():target.manifestSha256},
'../../src/db/application-writer-profile-format.js':format,
'./context.mjs':{ownedContext:async()=>{contexts++;if(fault==='late-actor'&&contexts===2)throw Error('Actor changed');return {pid:1,source}},sameOwned:(a:unknown,b:unknown)=>assert.deepEqual(a,b),sourceCoordinates:(v:{source:typeof source})=>v.source,protectedStructure:async()=>{events.push('roles');if(fault==='role-structure')throw Error('Unsafe role');return security},readCurrent:forPublication&&!upgradeMode?async()=>null:readCurrentApplicationWriterProfile,HISTORY:'history',CURRENT:'current'},
}
const api=compile<PublisherModule>(candidate,modules).createConnectedPublisher(installation,async()=>{events.push('catalogue');if(fault==='catalogue-eof')throw Error('Incomplete EOF');return fault==='catalogue-manifest'?{...catalogue,manifestSha256:migrationManifestDigest()}:fault==='catalogue-definition'?{...catalogue,definitionSha256:'0'.repeat(64)}:catalogue},{verify:async()=>{verifications++;events.push('verify-source');if(fault==='source-verifier')throw Error('Unsigned source');if(fault==='late-expiry'&&verifications===2)pinned.expiresAt=1;return {platformProtectionVerified:!(fault==='initial-unprotected'&&verifications===1||fault==='terminal-unprotected'&&verifications===2)}}},pinned,definition)
const publication=()=>({privateKeyFile,journalDirectory,expectedHeadSha256:upgradeMode?pinned.expectedHeadSha256??priorHead:null,deadlineAt:Date.now()+30000,validityMs:60000})
return {run:()=>api.preflightCurrentProfile(pool,{deadlineAt:Date.now()+30000}),inspectLegacy:()=>api.inspectLegacyStorage(pool,{deadlineAt:Date.now()+30000}),publish:()=>api.publishCurrentProfile(pool,publication()),upgrade:()=>api.upgradeCurrentProfile(pool,publication()),queries,events,markerLimits,files,writes,priorHead,pinned,get checkouts(){return checkouts},f}
}
test('legacy inspection requires retained TLS, terminal source proof and acknowledged rollback',async()=>{
const valid=fixture(),result=await valid.inspectLegacy()
assert.equal(result.state,'observed');assert.equal(result.snapshot.rowCount,1);assert.equal(valid.checkouts,1)
assert.deepEqual(valid.events,['capture-source','verify-source','capture-legacy','verify-source','fresh-source','release:false'])
assert.equal(valid.queries.filter(q=>q==='ROLLBACK').length,1);assert(!valid.queries.includes('COMMIT'))
for(const fault of ['initial-tls','lost-tls','late-source','rollback-ack'])await assert.rejects(fixture(fault).inspectLegacy())
})
test('current44 preflight authenticates signed current and actual component inputs on one read-only lease',async()=>{
const f=fixture(),result=await f.run();assert.equal(result.state,'matched');assert.equal(result.headSha256,f.f.state.current[0].head_sha256)
assert.equal(result.sourceMatched,true);assert.equal(result.rolesMatched,true);assert.equal(result.catalogueMatched,true)
assert.equal(result.platformProtectionVerified,true)
assert.equal(result.writerAuthorized,false);assert.equal(result.generationAuthenticated,false);assert.equal(result.readyAuthorized,false)
assert.equal(f.queries.filter(q=>q.startsWith('BEGIN')).length,1);assert(f.queries.includes('BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY'));assert.equal(f.queries.filter(q=>q==='ROLLBACK').length,1)
assert.equal(format.canonical(f.markerLimits),'[[45]]')
assert(!f.queries.some(q=>/^(?:INSERT|UPDATE|DELETE|COMMIT|CREATE|ALTER|DROP)\b/.test(q)));assert.equal(f.events.at(-1),'release:false')
assert.deepEqual(f.events.slice(0,6),['capture-source','verify-source','roles','catalogue','verify-source','fresh-source'])
})
test('publisher refuses before SQL on a wrong peer and stops SQL when the retained TLS socket is lost',async()=>{
const initial=fixture('initial-tls')
await assert.rejects(initial.run());assert.deepEqual(initial.queries,[]);assert.deepEqual(initial.events,['release:true'])
const lost=fixture('lost-tls')
await assert.rejects(lost.run());assert.equal(lost.events.at(-1),'release:true')
assert(!lost.queries.some(q=>q==='ROLLBACK'||q.includes('pg_advisory_unlock')))
})
for(const fault of ['initial-unprotected','terminal-unprotected'])test('preflight cannot promote '+fault+' into protected evidence',async()=>{
const f=fixture(fault),result=await f.run();assert.equal(result.platformProtectionVerified,false)
assert.equal(result.writerAuthorized,false);assert.equal(result.readyAuthorized,false)
assert(!f.queries.some(q=>/^(?:INSERT|UPDATE|DELETE|COMMIT|CREATE|ALTER|DROP)\b/.test(q)))
})
for(const fault of ['tamper','empty','expired','signed-source','signed-role','signed-catalogue','signed-definition','signed-manifest','role-structure','catalogue-eof','source-verifier','late-source','late-actor','late-expiry','late-clock','rollback-ack'])test('preflight refuses '+fault+' without any write or replay',async()=>{
const f=fixture(fault);await assert.rejects(f.run());assert(!f.queries.some(q=>/^(?:INSERT|UPDATE|DELETE|COMMIT|CREATE|ALTER|DROP)\b/.test(q)));assert.equal(f.queries.filter(q=>q.startsWith('BEGIN')).length,1)
})
for(const fault of ['current-build','current-approved-frozen-definition','approval-manifest','static-definition'])test('incompatible static manifest/definition refuses '+fault+' before either pool checkout or publication lease',async()=>{
const f=fixture(fault,true)
await assert.rejects(f.run(),/Approved catalogue definition is not bound/)
await assert.rejects(f.publish(),/Approved catalogue definition is not bound/)
assert.equal(f.checkouts,0);assert.deepEqual(f.events,[]);assert.deepEqual(f.queries,[])
})
for(const fault of ['catalogue-manifest','catalogue-definition','marker-missing','marker-extra'])test('retained preflight refuses '+fault+' and rolls back',async()=>{
const f=fixture(fault)
await assert.rejects(f.run(),error=>error instanceof Error&&/Catalogue evidence differs|Exact current migration marker/.test((error.cause as Error).message))
assert.equal(f.checkouts,1);assert.equal(f.queries.filter(q=>q==='ROLLBACK').length,1)
assert(!f.queries.some(q=>/^(?:INSERT|UPDATE|DELETE|COMMIT)\b/.test(q)))
if(fault.startsWith('marker-'))assert.equal(format.canonical(f.markerLimits),'[[45]]')
else assert.deepEqual(f.markerLimits,[])
})
test('publication checks exact marker set on both retained transactions and refuses late marker drift before registry writes',async()=>{
const positive=fixture('',true)
assert.equal((await positive.publish()).outcome,'committed')
assert.equal(positive.checkouts,1);assert.equal(format.canonical(positive.markerLimits),'[[45],[45]]')
assert.equal(positive.queries.filter(q=>q==='COMMIT').length,2)
const drift=fixture('marker-drift',true)
await assert.rejects(drift.publish(),error=>error instanceof Error&&/Exact current migration marker/.test((error.cause as Error).message))
assert.equal(drift.checkouts,1);assert.equal(format.canonical(drift.markerLimits),'[[45],[45]]')
assert.equal(drift.queries.filter(q=>q==='COMMIT').length,1);assert.equal(drift.queries.filter(q=>q==='ROLLBACK').length,1)
assert(!drift.queries.some(q=>/^(?:INSERT|UPDATE|DELETE)\b/.test(q)))
assert([...drift.files.keys()].some(filename=>filename.endsWith('pending-publication.json')))
})
test('explicit 44 to 56 upgrade pins the signed predecessor, exact catalogue and CAS',async()=>{
const f=fixture('',true,'upgrade')
const approved=upgradeApproval(f.pinned)
assert.equal(approved.expectedHeadSha256,f.priorHead)
assert.throws(()=>parseApproval(approved))
for(const changed of [{expectedHeadSha256:null},{previousCatalogSha256:undefined},{expiresAt:1}]) {
assert.throws(()=>upgradeApproval({...approved,...changed}))
}
assert.equal(reconciliationApproval({...approved,expiresAt:1}).expectedHeadSha256,f.priorHead)
const result=await f.upgrade()
assert.equal(result.outcome,'committed');assert.equal(result.epoch,'2')
assert.equal(format.canonical(f.markerLimits),'[[57],[57]]')
assert.equal(f.writes.length,2)
const [history,current]=f.writes
assert.match(history.sql,/INSERT INTO mmapp_control.history/)
assert.equal(format.canonical(history.values?.slice(0,6)),format.canonical([f.f.subject.installationId,'2',result.headSha256,f.f.subject.installationId,'1',f.priorHead]))
const policy=format.signedEnvelope(history.values![7] as Buffer,'policy',f.f.subject).body
const head=format.signedEnvelope(history.values![8] as Buffer,'head',f.f.subject).body
assert.deepEqual(policy.source,f.f.policy.source);assert.deepEqual(policy.roles,f.f.policy.roles)
assert.equal(policy.manifestSha256,BUILD56_PROVENANCE.manifestSha256);assert.equal(policy.definitionSha256,BUILD56_PROVENANCE.templateSha256)
assert.equal(head.previousHeadSha256,f.priorHead)
assert.equal(format.canonical(current.values),format.canonical(['2',result.headSha256,f.f.subject.installationId,'1',f.priorHead]))
})
test('upgrade refuses drift and ordinary renewal cannot perform the 44 to 56 transition',async()=>{
for(const fault of ['wrong-head','wrong-previous-catalogue','signed-manifest','signed-definition','signed-role','signed-source','marker-drift','initial-unprotected','terminal-unprotected']){
const f=fixture(fault,true,'upgrade')
await assert.rejects(f.upgrade())
assert.deepEqual(f.writes,[],fault)
}
const explicit=fixture('',true,'upgrade')
await assert.rejects(explicit.publish(),/explicit upgrade command/)
assert.equal(explicit.checkouts,0)
const renewal=fixture('',true,'ordinary-renewal')
await assert.rejects(renewal.publish(),error=>error instanceof Error&&/Renewal cannot adopt/.test((error.cause as Error).message))
assert.deepEqual(renewal.writes,[])
})
test('explicit 56 to 57 upgrade signs only the exact reviewed successor',async()=>{
const definition=approvedDefinition({format:'application-writer-source57-definition-approval-v1',provenance:BUILD57_PROVENANCE,template:FIXED57_ROLE_TEMPLATE})
assert.equal(definition.manifestSha256,migrationManifestDigest())
assert.throws(()=>approvedDefinition({format:'application-writer-source57-definition-approval-v1',provenance:BUILD57_PROVENANCE,template:FIXED44_ROLE_TEMPLATE}))
const f=fixture('',true,'upgrade57')
await assert.rejects(f.publish(),/explicit upgrade command/)
const result=await f.upgrade()
assert.equal(result.outcome,'committed')
assert.equal(format.canonical(f.markerLimits),'[[58],[58]]')
const history=f.writes[0]
const policy=format.signedEnvelope(history.values![7] as Buffer,'policy',f.f.subject).body
const head=format.signedEnvelope(history.values![8] as Buffer,'head',f.f.subject).body
assert.equal(policy.manifestSha256,definition.manifestSha256)
assert.equal(policy.definitionSha256,definition.definitionSha256)
assert.equal(head.previousHeadSha256,f.priorHead)
for(const fault of ['wrong-transition','wrong-previous-catalogue','signed-manifest','signed-definition']){
const refused=fixture(fault,true,'upgrade57')
await assert.rejects(refused.upgrade())
assert.deepEqual(refused.writes,[],fault)
}
})
test('operator requires the protected source verifier for preflight, publication and reconciliation',async()=>{
const cli=await fs.readFile(new URL('./database-writer-profile-operator.mts',import.meta.url),'utf8'),f=syntheticFixture('1'),reads:string[]=[],events:string[]=[]
const request={installationId:f.subject.installationId,roles:Object.fromEntries(['runtime','operator','capture','publisher','control'].map((k,i)=>[k,{name:k,signedOid:String(i+11)}])),source:f.policy.source,sourceHeadSha256:'a'.repeat(64),tls:{caSha256:'a'.repeat(64),spkiSha256:'d'.repeat(64)}}
const files:Record<string,unknown>={bootstrap:{format:'application-writer-bootstrap-v1',publicationAuthorized:false,readyAuthorized:false,deploymentRequest:request,verificationSubject:f.subject},approval:{expiresAt:Date.now()+60000,definitionSha256:'b'.repeat(64)},definition:{},transport:{},ca:'fixture','signing-private.pem':'fixture'}
let failEnd=false,asyncError=false,protectedRefused=false,activationRefused=false,incompatibleDefinition=false
let catalogueEdition:44|56|57=44
const catalogueCalls:number[]=[]
const retainedPools:Pool[]=[]
class Pool extends EventEmitter{constructor(){super();retainedPools.push(this)}async end(){events.push('pool-end');if(failEnd)throw Error('Pool cleanup unconfirmed')}}
const result={state:'matched',epoch:'1',headSha256:'c'.repeat(64),notAfter:Date.now()+60000,platformProtectionVerified:true}
const protectedVerifier={verify:async()=>({platformProtectionVerified:true})}
const guardedVerifier={...protectedVerifier}
const modules={
'node:assert/strict':assert,'node:path':require('node:path'),'node:url':require('node:url'),'node:crypto':require('node:crypto'),
'./writer-profile/operator-files.mjs':{readProtected:async(file:string)=>{reads.push(file);assert(file in files);return Buffer.from(JSON.stringify(files[file]))},protectedDirectory:async()=>{},sha:format.sha},
'./writer-profile/operator-contract.mjs':{approval:(value:unknown)=>{events.push('approval');return value},upgradeApproval:(value:unknown)=>{events.push('upgrade-approval');return upgradeApproval(value)},reconciliationApproval:(value:unknown)=>{events.push('reconciliation-approval');return value},poolOptions:()=>{events.push('transport');return {}},roleKeys:['runtime','operator','capture','publisher','control'],assertJsDriverEnvironment:()=>{}},
'./database-writer-bootstrap-operator.mjs':{deploymentRequest:(value:unknown)=>value},
'./writer-profile/protected-source-registry.mjs':{createProtectedDatabaseSourceBindingVerifier:(pin:{registryDirectory:string;currentHeadSha256:string})=>{events.push('protected-source');assert.equal(pin.registryDirectory,'source-registry');assert.equal(pin.currentHeadSha256,request.sourceHeadSha256);if(protectedRefused)throw Error('Protected source refused');return protectedVerifier}},
'./writer-profile/source-activation.mjs':{withActiveSource:async(input:{directory:string;expectedSha256:string;sourcePin:{currentHeadSha256:string};expectedSource:unknown;tlsSpkiSha256:string;deadlineAt:number},verifier:unknown,work:(verifier:unknown)=>Promise<unknown>)=>{
assert.equal(verifier,protectedVerifier);assert.equal(input.directory,'active-registry');assert.equal(input.expectedSha256,'e'.repeat(64))
assert.equal(input.sourcePin.currentHeadSha256,request.sourceHeadSha256);assert.equal(input.tlsSpkiSha256,request.tls.spkiSha256);assert(input.deadlineAt>Date.now())
assert.equal(format.canonical(input.expectedSource),format.canonical({...request.source,databaseOid:Number(request.source.databaseOid),sessionUser:'publisher',currentUser:'publisher'}))
events.push('activation-admission');if(activationRefused)throw Error('Activation refused')
const value=await work(guardedVerifier);events.push('activation-terminal');return value
}},
pg:{Pool},'./writer-profile/publisher.mjs':{requireCurrentCatalogueDefinition:()=>{if(incompatibleDefinition)throw Error('Build manifest refused')},createConnectedPublisher:(_installation:unknown,collect:(session:unknown)=>Promise<unknown>,verifier:unknown)=>{
assert.equal(verifier,events.includes('activation-admission')?guardedVerifier:protectedVerifier)
return {
inspectLegacyStorage:async()=>{events.push('legacy-inspect');return {state:'observed',snapshot:{rowCount:1,sha256:'9'.repeat(64)},source:{platformProtectionVerified:true,generation:'source-generation'}}},
preflightCurrentProfile:async()=>{await collect({});events.push('preflight');if(asyncError){const activePool=retainedPools.at(-1);assert(activePool);activePool.emit('error',Error('Async pool fault'))}return result},
publishCurrentProfile:async()=>{await collect({});events.push('publish');return {...result,outcome:'committed'}},
upgradeCurrentProfile:async()=>{await collect({});events.push('upgrade');return {...result,outcome:'committed'}},
reconcilePendingPublication:async()=>{events.push('reconcile');return {state:'confirmed',headSha256:result.headSha256}},
}
}},
'./writer-profile/catalogue.mjs':{
collectSource44Catalogue:async()=>{catalogueCalls.push(44)},
collectSource56Catalogue:async()=>{catalogueCalls.push(56)},
collectSource57Catalogue:async()=>{catalogueCalls.push(57)},
approvedDefinition:()=>({...(catalogueEdition===44?{}:{edition:catalogueEdition}),definitionSha256:'b'.repeat(64),manifestSha256:frozenDefinition.manifestSha256}),
},
}
const flags=['bootstrap','bootstrap-sha256','approval','approval-sha256','definition','definition-sha256','transport','transport-sha256','ca','source-registry','source-spki-sha256','journal'],args=['preflight',...flags.flatMap(flag=>['--'+flag,flag.endsWith('sha256')?'a'.repeat(64):flag])]
const output=await compile<OperatorModule>(cli,modules).main(args)
assert.equal(output.event,'application-writer-profile-preflight');assert.equal(output.outcome,'matched');assert.equal(output.platformProtectionVerified,true);assert.equal(output.writerAuthorized,false);assert.equal(output.readyAuthorized,false)
assert.deepEqual(reads,['bootstrap','approval','definition','transport','ca']);assert.deepEqual(events,['approval','transport','protected-source','preflight','pool-end'])
failEnd=true;await assert.rejects(compile<OperatorModule>(cli,modules).main(args),/Pool cleanup unconfirmed/)
failEnd=false;asyncError=true;await assert.rejects(compile<OperatorModule>(cli,modules).main(args),/Publisher terminal unconfirmed/)
asyncError=false;result.platformProtectionVerified=false;await assert.rejects(compile<OperatorModule>(cli,modules).main(args),/Publisher terminal unconfirmed/)
result.platformProtectionVerified=true;protectedRefused=true;const before=retainedPools.length
await assert.rejects(compile<OperatorModule>(cli,modules).main(args),/Protected source refused/);assert.equal(retainedPools.length,before,'Protection failure must not open a Pool or fall back')
const activationFlags=['--activation-directory','active-registry','--activation-sha256','e'.repeat(64)]
const activated=(command:string)=>[command,...args.slice(1),...activationFlags]
for(const [command,approvalEvent,outcome] of [['publish','approval','committed'],['reconcile','reconciliation-approval','confirmed']]){
const commandArgs=command==='publish'?activated(command):[command,...args.slice(1)]
const poolCount=retainedPools.length
await assert.rejects(compile<OperatorModule>(cli,modules).main(commandArgs),/Protected source refused/)
assert.equal(retainedPools.length,poolCount)
protectedRefused=false
events.length=0;reads.length=0
const terminal=await compile<OperatorModule>(cli,modules).main(commandArgs)
assert.equal(terminal.outcome,outcome);assert.equal(terminal.readyAuthorized,false)
assert.deepEqual(events,command==='publish'
?[approvalEvent,'transport','protected-source','activation-admission',command,'pool-end','activation-terminal']
:[approvalEvent,'transport','protected-source',command,'pool-end'])
assert.equal(reads.includes('signing-private.pem'),command==='publish')
protectedRefused=true
}
const activeArgs=activated('source-preflight')
protectedRefused=false;activationRefused=true;events.length=0;reads.length=0
const beforeActivation=retainedPools.length
await assert.rejects(compile<OperatorModule>(cli,modules).main(activeArgs),/Activation refused/)
assert.equal(retainedPools.length,beforeActivation)
assert.deepEqual(events,['approval','transport','protected-source','activation-admission'])
await assert.rejects(compile<OperatorModule>(cli,modules).main(['publish',...args.slice(1)]))
await assert.rejects(compile<OperatorModule>(cli,modules).main(activated('publish')), /Activation refused/)
assert.equal(retainedPools.length,beforeActivation)
activationRefused=false;events.length=0;reads.length=0
const active=await compile<OperatorModule>(cli,modules).main(activeArgs)
assert.equal(active.event,'application-writer-profile-source-preflight');assert.equal(active.sourceActivationMatched,true)
assert.equal(active.readyAuthorized,false);assert.equal(active.writerAuthorized,false)
assert.deepEqual(events,['approval','transport','protected-source','activation-admission','preflight','pool-end','activation-terminal'])
assert.deepEqual(reads,['bootstrap','approval','definition','transport','ca'])
const legacyFlags=['bootstrap','bootstrap-sha256','transport','transport-sha256','ca','source-registry','source-spki-sha256']
const legacyArgs=['legacy-inspect',...legacyFlags.flatMap(flag=>['--'+flag,flag.endsWith('sha256')?'a'.repeat(64):flag]),...activationFlags]
events.length=0;reads.length=0
const legacy=await compile<OperatorModule>(cli,modules).main(legacyArgs)
assert.equal(legacy.event,'database-legacy-inspection');assert.equal(legacy.outcome,'observed');assert.equal(legacy.readyAuthorized,false)
assert.deepEqual(events,['transport','protected-source','activation-admission','legacy-inspect','pool-end','activation-terminal'])
assert.deepEqual(reads,['bootstrap','transport','ca'])
await assert.rejects(compile<OperatorModule>(cli,modules).main(legacyArgs.slice(0,-2)))
activationRefused=true;await assert.rejects(compile<OperatorModule>(cli,modules).main(legacyArgs),/Activation refused/);activationRefused=false
assert(catalogueCalls.length>0&&catalogueCalls.every(edition=>edition===44))
catalogueEdition=56;catalogueCalls.length=0
for(const commandArgs of [args,activeArgs,activated('publish')]){
events.length=0
await compile<OperatorModule>(cli,modules).main(commandArgs)
}
assert.deepEqual(catalogueCalls,[56,56,56])
const publicationApproval=files.approval
files.approval={...fixture('',true,'upgrade').pinned,definitionSha256:'b'.repeat(64)}
events.length=0;reads.length=0;catalogueCalls.length=0
const upgraded=await compile<OperatorModule>(cli,modules).main(activated('upgrade'))
assert.equal(upgraded.outcome,'committed')
assert.deepEqual(events,['upgrade-approval','transport','protected-source','activation-admission','upgrade','pool-end','activation-terminal'])
assert.deepEqual(catalogueCalls,[56])
assert(reads.includes('signing-private.pem'))
catalogueEdition=57
files.approval={...fixture('',true,'upgrade57').pinned,definitionSha256:'b'.repeat(64)}
events.length=0;reads.length=0;catalogueCalls.length=0
assert.equal((await compile<OperatorModule>(cli,modules).main(activated('upgrade'))).outcome,'committed')
assert.deepEqual(catalogueCalls,[57])
files.approval=publicationApproval
catalogueEdition=44;catalogueCalls.length=0
incompatibleDefinition=true;events.length=0;reads.length=0
const beforeBuildRefusal=retainedPools.length
for(const incompatibleArgs of [args,activeArgs,activated('publish')])await assert.rejects(compile<OperatorModule>(cli,modules).main(incompatibleArgs),/Build manifest refused/)
assert.equal(retainedPools.length,beforeBuildRefusal)
assert(!events.some(event=>event==='protected-source'||event==='activation-admission'))
const historical=await compile<OperatorModule>(cli,modules).main(['reconcile',...args.slice(1)])
assert.equal(historical.outcome,'confirmed');assert.equal(historical.readyAuthorized,false)
assert.deepEqual(catalogueCalls,[])
})
@@ -0,0 +1,45 @@
if(!process.send)throw Error('Owned profile load client requires an IPC parent')
function requireLoopback(value){const url=new URL(value);if(url.protocol!=='http:'||!['127.0.0.1','localhost','[::1]'].includes(url.hostname)||url.username||url.password)throw Error('Owned test client only permits localhost HTTP')}
import {Client,StreamableHTTPClientTransport} from '@modelcontextprotocol/client'
import {getHeapStatistics} from 'node:v8'
const clients=new Map(),requests=new Map(),streams=new Map();let serial=0,peakRss=0,peakHeap=0
const sample=()=>{const m=process.memoryUsage();peakRss=Math.max(peakRss,m.rss);peakHeap=Math.max(peakHeap,m.heapUsed);return{memory:m,peakRss,peakHeap,maxRssBytes:process.resourceUsage().maxRSS*1024,effectiveHeapSizeLimitBytes:getHeapStatistics().heap_size_limit}}
const timer=setInterval(sample,20)
const summarize=result=>{const row=result.structuredContent?.result??{};return{isError:result.isError===true,structuredContent:{result:{dataQuality:row.dataQuality??null,collectorQualityUnavailableReason:row.collectorQualityUnavailableReason,retainedRowsBytes:row.retainedRowsBytes,analyticsBytes:row.analytics?.bytes,eventBytes:row.events?.map(x=>x.data?.bytes),exporters:row.exporters?.length,clients:row.clients?.length,month:row.month}},hiddenSourceLeaked:JSON.stringify(result).includes('HIDDEN_MEMORY_SOURCE'),encodedBytes:Buffer.byteLength(JSON.stringify(result)),error:result.isError?result.content:undefined}}
async function run(message){
const p=message.payload??{}
if(p.endpoint)requireLoopback(p.endpoint);if(p.url)requireLoopback(p.url)
if(message.operation==='connect'){
const client=new Client({name:'external-owned-memory-client',version:'1'});await client.connect(new StreamableHTTPClientTransport(new URL(p.endpoint+'/mcp'),{requestInit:{headers:{authorization:'Bearer '+p.token}}}));const id=++serial;clients.set(id,client);return{id}
}
if(message.operation==='tool')return summarize(await clients.get(p.clientId).callTool(p.input))
if(message.operation==='close'){await clients.get(p.clientId)?.close();clients.delete(p.clientId);return{closed:true}}
if(message.operation==='sse-open'){
const abort=new AbortController(),response=await fetch(p.url,{signal:abort.signal});if(response.status!==200)throw Error('SSE status '+response.status)
const reader=response.body.getReader(),decoder=new TextDecoder();let text=''
for(;;){const chunk=await reader.read();if(chunk.done)throw Error('SSE closed before sample');text+=decoder.decode(chunk.value,{stream:true});const match=/event: (sample|error)\ndata: ([^\n]+)\n\n/.exec(text);if(match){if(match[1]!=='sample')throw Error('SSE source unavailable');const body=JSON.parse(match[2]),id=++serial;streams.set(id,{abort,reader});return{id,bytes:body.bytes}}}
}
if(message.operation==='sse-close'){const stream=streams.get(p.streamId);stream?.abort.abort();await stream?.reader.cancel().catch(()=>{});streams.delete(p.streamId);return{closed:true}}
if(message.operation==='http'){
const abort=new AbortController();requests.set(message.id,abort)
try{const response=await fetch(p.url,{...p.options,signal:abort.signal}),body=await response.arrayBuffer();let summary;if(response.status===200&&(p.url.includes('/exporters')||p.url.includes('/analytics'))){const parsed=JSON.parse(new TextDecoder().decode(body));summary={exporters:parsed.exporters?.length,bytes:parsed.bytes}}return{status:response.status,bytes:body.byteLength,summary,text:body.byteLength<4096?new TextDecoder().decode(body):''}}
finally{requests.delete(message.id)}
}
if(message.operation==='batch'){
const response=await fetch(p.endpoint+'/mcp',{method:'POST',headers:{authorization:'Bearer '+p.token,'content-type':'application/json',accept:'application/json, text/event-stream','mcp-protocol-version':'2025-06-18'},body:JSON.stringify([1,2].map(id=>({jsonrpc:'2.0',id,method:'tools/call',params:{name:'mm_netflow_analytics_report_get',arguments:{serverId:1,minutes:5}}})))})
if(response.status!==200)throw Error('Batch HTTP status '+response.status)
const reader=response.body.getReader(),decoder=new TextDecoder(),messages=[];let text=''
for(;;){const chunk=await reader.read();if(chunk.done)break;text+=decoder.decode(chunk.value,{stream:true});let end;while((end=text.indexOf('\n\n'))>=0){const frame=text.slice(0,end);text=text.slice(end+2);for(const line of frame.split('\n'))if(line.startsWith('data: ')){const row=JSON.parse(line.slice(6));messages.push(row);if(row.error||row.result?.isError)process.send?.({type:'client-progress',id:message.id,refusals:1})}}}
if(messages.length!==2)throw Error('Expected two batch replies')
return{successes:messages.filter(row=>!row.error&&!row.result?.isError).length,refusals:messages.filter(row=>row.error||row.result?.isError).length}
}
throw Error('Unknown external client operation')
}
process.on('message',async message=>{
if(message.type==='abort'){requests.get(message.requestId)?.abort();return}
if(message.type==='shutdown'){for(const abort of requests.values())abort.abort();for(const stream of streams.values())stream.abort.abort();for(const client of clients.values())try{await client.close()}catch{}clearInterval(timer);process.send?.({type:'client-report',report:sample()},()=>process.exit(0));return}
if(message.type!=='client-command')return
try{const result=await run(message);process.send?.({type:'client-result',id:message.id,ok:true,result})}
catch(error){process.send?.({type:'client-result',id:message.id,ok:false,error:error instanceof Error?error.message:String(error)})}
})
process.send?.({type:'client-ready',report:sample()})
Loaded 100 of 2762 files, more files were not shown because too many files have changed in this diff. Show more