Compare commits

..
61 Commits
Author SHA1 Message Date
DenozordecandCursor ec65249bf1 fix(ci): satisfy golangci-lint in api keys code
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 41s
CI / bird2 (push) Successful in 16s
CI / changes (push) Successful in 7s
CI / release (push) Successful in 3m18s
Правки errcheck в тестах httpapi и gofmt полей APIKeyPatch в store.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 11:33:19 +07:00
Denozordec 6329a4df27 feat(api): implement API key management and authentication enhancements
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 25s
CI / web (push) Successful in 28s
CI / go (push) Failing after 24s
CI / bird2 (push) Has been skipped
CI / release (push) Has been skipped
- Added endpoints for managing API keys, including creation, retrieval, updating, and revocation.
- Introduced a new Auth session endpoint to retrieve current tenant and role information.
- Updated the authentication middleware to support API key-based authentication and track last used timestamps.
- Enhanced documentation to reflect new API key functionalities and usage guidelines.
- Improved logging for demo authentication scenarios.
2026-05-21 11:26:17 +07:00
Denozordec 880d77810a refactor(httpapi): standardize output formatting in job report response
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 37s
CI / go (push) Successful in 50s
CI / bird2 (push) Successful in 18s
CI / changes (push) Successful in 7s
CI / release (push) Successful in 3m51s
- Adjusted the order of fields in the job report response for consistency.
- Ensured that the output structure remains clear and maintainable.
2026-05-21 10:52:08 +07:00
DenozordecandCursor 8ebce28e34 perf: wire job_audit terminal persistence hook
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Failing after 35s
CI / go (push) Failing after 30s
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 8s
CI / release (push) Has been skipped
- TerminalHook в Registry для записи статуса job в PostgreSQL job_audit
- Подключение через BootstrapWorkers при наличии pool

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 10:46:13 +07:00
DenozordecandCursor dd7d43c2c2 perf: incremental render scale, SQL pagination and observability
- BIRD filter chunking (500 CIDR); bounded job worker pool
- ListModulesPage SQL push-down; revision diff limit; batch revision prune
- DB pool tuning; Prometheus pipeline/job metrics
- Coalesce module_refresh via idempotency; GET /jobs/{id}/report
- JobAuditWriter foundation for job_audit persistence

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 10:45:46 +07:00
DenozordecandCursor a8c5e9701f perf: optimize data path, indexes and frontend virtualization
- Убран latestCDNRowsBySource; expanded BIRD preview генерируется on-demand
- Batch AS meta updates; миграция perf-индексов 000012
- VirtualPrefixList для preview префиксов; метрики pipeline refresh
- PolitePause для RIPEstat после cache miss

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 10:45:39 +07:00
DenozordecandCursor be3d73f374 perf: quick wins for pipeline, jobs, httpapi and web ops
- CDN snapshot: batch merge после parallel fetch, без race на persist
- ListRevisionPrefixes: лёгкая проверка revision вместо GetRevision
- Jobs: timeout/cancel context для pipeline и deploy
- HTTP server timeouts; кэш birdc для GET /peers
- ListModules: batch DoH profiles одним запросом
- Web: tab-scoped load на /operations, debounce job search, меньше over-fetch на dashboard

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-21 10:42:13 +07:00
Denozordec 5fca165c69 refactor(settings): deprecate settingsKnownSchema and integrate bird and revision settings
CI / changes (push) Successful in 10s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 25s
CI / web (push) Successful in 36s
CI / go (push) Successful in 49s
CI / bird2 (push) Successful in 17s
CI / release (push) Successful in 3m59s
- Merged birdSettingsSchema and revisionSettingsSchema into settingsKnownSchema, marking the previous schema as deprecated.
- Updated emptySettingsKnownForm to utilize emptyBirdSettingsForm and emptyRevisionSettingsForm.
- Refactored layout and page components to streamline theme management and improve tab synchronization in network and operations pages.
- Introduced new system settings tab in operations and updated settings page to manage theme preferences.
2026-05-21 10:11:04 +07:00
Denozordec 293115e0e1 fix(nginx): update proxy_pass configuration to use full request URI
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Successful in 52s
CI / bird2 (push) Successful in 18s
CI / release (push) Successful in 4m33s
- Modified the NGINX configuration to pass the full request URI in the proxy_pass directive, ensuring proper handling of requests.
- Added a comment to clarify the limitation of using a variable in proxy_pass for URI replacement.
2026-05-20 16:05:54 +07:00
Denozordec 6d2051f813 fix(web): improve error handling and API health check logic
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 31s
CI / go (push) Successful in 41s
CI / bird2 (push) Successful in 16s
CI / changes (push) Successful in 7s
CI / release (push) Successful in 3m25s
- Refactored the API health check to handle errors more gracefully, providing clearer feedback on loading issues.
- Updated the NGINX configuration to use a dynamic upstream variable for better resilience against IP changes in Docker.
- Enhanced user notifications for API availability and data loading errors, including instructions for local demo setup.
2026-05-20 15:48:18 +07:00
DenozordecandCursor b51a9ae3b3 ci: add ci type to release configuration and update documentation
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 31s
CI / web (push) Successful in 41s
CI / go (push) Successful in 47s
CI / bird2 (push) Successful in 19s
CI / changes (push) Successful in 7s
CI / release (push) Successful in 3m47s
- Introduced `ci` type in `.releaserc.json` for patch releases.
- Updated conventional commits documentation to reflect the new `ci` type and its implications for versioning.
- Clarified the role of `ci` in the context of patch releases in the releasing guide.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 15:24:21 +07:00
DenozordecandCursor 4d4cd2301f ci: update golangci-lint version and installation mode
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 23s
CI / web (push) Successful in 30s
CI / go (push) Successful in 1m15s
CI / bird2 (push) Successful in 15s
CI / release (push) Successful in 18s
- Upgraded golangci-lint from v1.62 to v1.64.8 to ensure compatibility with Go 1.24.
- Changed installation mode to 'goinstall' for improved setup.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 15:18:53 +07:00
DenozordecandCursor d687881eaa chore: update golangci configuration and improve resource cleanup
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 23s
CI / web (push) Successful in 31s
CI / go (push) Failing after 19s
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 7s
CI / release (push) Has been skipped
- Disabled all linters in .golangci.yml to streamline linting process.
- Updated resource cleanup in multiple files to use deferred functions for closing response bodies, ensuring proper error handling and resource management.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 15:13:02 +07:00
DenozordecandCursor 87e756f34f ci: expand changes detection to rewire all pipeline nodes
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 28s
CI / web (push) Successful in 40s
CI / go (push) Failing after 21s
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 7s
CI / release (push) Has been skipped
Полный прогон при scripts/*, workflows, golangci, pre-commit; openapi в full_pipeline; миграции и deploy-пути; исправлен fallback пустого diff.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 15:08:28 +07:00
DenozordecandCursor 27864fad58 fix(ci): make check-migrations-pair.sh POSIX sh compatible
CI / changes (push) Successful in 8s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / release (push) Has been cancelled
Убрана bash process substitution; скрипт работает под sh в Gitea runner (DEP-03).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:57:18 +07:00
DenozordecandCursor dc5e777d07 style(web): format settings-known schema for prettier
CI / openapi (push) Has been skipped
CI / commitlint (push) Has been skipped
CI / go (push) Has been cancelled
CI / bird2 (push) Has been cancelled
CI / web (push) Has been cancelled
CI / changes (push) Successful in 7s
CI / release (push) Has been cancelled
Исправление падения CI job web: prettier --check на settings-known.schema.ts.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:56:20 +07:00
DenozordecandCursor 23910c3393 docs(go): add godoc for pipeline and store exports
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 23s
CI / web (push) Failing after 34s
CI / go (push) Failing after 41s
CI / bird2 (push) Has been skipped
CI / release (push) Has been skipped
Уточнение godoc ExtractCIDRs; тесты repository (interface + optional integration).

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:53:50 +07:00
DenozordecandCursor 0e90bbee4e docs: clarify dev-only flags and openapi implementation gaps
DOC-03: X-Tenant-Id не реализован в handlers; лимиты CDN preview; SEC-02 пометки в compose; sync endpoints в access.md.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:53:20 +07:00
DenozordecandCursor 55bc87cbc4 ci: add golangci, migration checks, and pre-commit
DEP-03: check-migrations-pair.sh; migrations/* в path filter; golangci-lint в CI; pre-commit; обновление engineering.mdc (TEST-04, enforcement).
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:53:01 +07:00
DenozordecandCursor e68f034966 refactor(web): extract shared kpi metrics grid pattern
WEB-18: KpiMetricsGrid в ui/patterns; рефакторинг overview, network, modules, directories, operations, schedule, monitoring.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:52:44 +07:00
DenozordecandCursor e06ee880b2 fix(web): align settings with shadcn notify and superforms
WEB-07: notify вместо svelte-sonner; ui/core imports; WEB-05: zod + superforms + FormField для полей настроек.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:50:58 +07:00
DenozordecandCursor 33fe8fdd18 refactor(httpapi): readiness ping via store backend
ARCH-09: Ping на store.Backend; readiness без прямого pgxpool.Ping в handler.
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:49:26 +07:00
DenozordecandCursor 6fa693156d fix(httpapi): stable problem details for client errors
Стабильные detail для 4xx (store, CDN preview, CSV); логирование на сервере. Расширен lint-httpapi для ERR-01 в 4xx.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:48:57 +07:00
DenozordecandCursor 5858d0889e fix(httpapi): paginate module nested list endpoints
Пагинация cursor/limit для CDN, AS, domain и IP range list; ответы с next_cursor и has_more по OpenAPI. Добавлен writePaginatedListJSON и тест.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 14:48:32 +07:00
Denozordec bfe20c1fe0 feat(web): enhance overview page with improved state management and UI components
CI / openapi (push) Has been skipped
CI / commitlint (push) Has been skipped
CI / web (push) Successful in 42s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 9s
CI / release (push) Successful in 3m57s
- Refactored state management for modules, revisions, peers, speakers, and jobs, enhancing loading and error handling.
- Updated imports to utilize core UI components for better maintainability and consistency.
- Introduced derived states for KPI cards, providing real-time insights into system metrics.
- Improved loading function to fetch data efficiently and handle errors gracefully.
- Enhanced UI layout with new icons and dynamic descriptions based on last updated timestamps.
2026-05-20 14:27:21 +07:00
Denozordec c66cc9317d feat(web): refactor directories and network pages with improved state management and UI components
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 31s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 6s
CI / release (push) Successful in 3m42s
- Updated imports to utilize core UI components for better maintainability.
- Refactored state management for communities and DoH profiles, enhancing loading and error handling.
- Introduced derived states for KPI cards, providing better insights into community and DoH profile counts.
- Streamlined loading functions to improve data fetching efficiency and user experience.
- Enhanced UI layout with new icons and improved component structure for clarity.
2026-05-20 14:13:32 +07:00
Denozordec 9ea4a68ccf docs: update AGENTS, rules, and skills documentation for Conventional Commits
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Has been skipped
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 7s
CI / release (push) Successful in 17s
- Enhanced AGENTS.md to clarify the use of `/commit-message` and the exclusion of the Source Control button for generating commit messages.
- Updated conventional-commits.mdc to include new triggers and guidelines for commit message types, emphasizing the importance of semver impact.
- Revised commit-message SKILL.md to specify the mandatory execution of scripts and the correct usage of types and scopes in commit messages.
- Improved README documentation to reflect the updated guidelines and workflows for generating commit messages.
2026-05-20 12:52:31 +07:00
Denozordec 8204105fd6 feat(web): enhance module entry dialogs and selection handling
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 38s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 8s
CI / release (push) Successful in 4m1s
- Refactored module entry dialogs to reset forms based on edit state and improve state management.
- Updated selection logic in various components to utilize derived states for active selections, enhancing bulk operations.
- Improved dialog bindings for better state synchronization and user experience.
- Streamlined component structure for maintainability and clarity.
2026-05-20 12:35:45 +07:00
Denozordec 0c11ecfa48 feat(web): enhance module management UI with new localization and state handling
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 31s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / release (push) Successful in 3m30s
- Added new utility functions for module state representation in Russian, improving localization.
- Introduced derived states for counting enabled and disabled modules, enhancing user insights.
- Updated module management components to display last updated timestamps and improved descriptions.
- Refactored imports to utilize core UI components for better maintainability and consistency.
2026-05-20 12:23:16 +07:00
Denozordec ab8660ac42 feat(web): enhance operations UI with new job status handling and component updates
- Added a new function for determining job status badge variants to improve UI consistency.
- Refactored imports to utilize the core UI library for better maintainability.
- Updated job report and filters components to enhance user experience and streamline functionality.
- Improved layout and responsiveness across operations-related components.
2026-05-20 12:14:33 +07:00
Denozordec ce747673fd feat(web): refactor module display logic and enhance UI components
- Introduced new utility functions for date formatting and module interval labeling.
- Replaced inline badge variant logic with a dedicated function for improved readability.
- Updated imports to streamline component usage and enhance maintainability.
- Enhanced the overall structure of the module display for better clarity and user experience.
2026-05-20 12:10:11 +07:00
Denozordec e558570967 feat(web): enhance monitoring page with new status tracking and UI components
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 28s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 8s
CI / release (push) Successful in 3m36s
- Refactored monitoring page to utilize new UI components from the core library.
- Added detailed job status tracking and error handling features.
- Improved overall layout and responsiveness of the monitoring interface.
- Introduced new derived states for better management of health and job statuses.
- Updated imports to streamline component usage and enhance maintainability.
2026-05-20 11:41:58 +07:00
DenozordecandCursor b6b081b04f style(web): format app-version.svelte for prettier
CI / openapi (push) Has been skipped
CI / commitlint (push) Has been skipped
CI / web (push) Successful in 28s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / changes (push) Successful in 7s
CI / release (push) Successful in 3m45s
Импорт в одну строку — иначе падает npm run lint в CI.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 11:21:13 +07:00
DenozordecandCursor 13b4c20148 fix(web): show semver on monitoring from legacy /v1/version
CI / changes (push) Successful in 7s
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Failing after 35s
CI / go (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / release (push) Has been skipped
На проде API отдаёт только api_version и git_sha=unknown без поля version;
карточка показывала SHA вместо semver. Общий хелпер version-info.ts:
version → api_version, placeholder SHA не в заголовке.

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 11:13:15 +07:00
Denozordec 700d415422 chore: update CI workflow and release process
CI / commitlint (push) Has been skipped
CI / openapi (push) Has been skipped
CI / web (push) Successful in 30s
CI / go (push) Successful in 25s
CI / bird2 (push) Successful in 15s
CI / changes (push) Successful in 7s
CI / release (push) Successful in 19s
- Replaced `@semantic-release/git` with `@semantic-release/exec` for handling release commands.
- Updated `.releaserc.json` to reflect changes in release command execution.
- Modified CI workflow to streamline the release process, including tagging and Docker image publishing.
- Removed obsolete publish workflow and integrated its functionality into the main CI workflow.
- Enhanced documentation to clarify the new release process and CI pipeline structure.
2026-05-20 01:05:22 +07:00
semantic-release-bot 6ca34b9008 chore(release): 1.0.0 [skip ci]
## 1.0.0 (2026-05-19)

* feat: add @tanstack/table-core dependency and integrate module name mapping in operations page. Upda ([b140dcb](http://192.168.100.56:3000/denozord/EvoBGP/commit/b140dcb))
* feat: add @types/node and undici-types to devDependencies in package.json and package-lock.json for  ([d92a06c](http://192.168.100.56:3000/denozord/EvoBGP/commit/d92a06c))
* feat: add aggregated router-lists catalog endpoint and update module listing filters ([c811c43](http://192.168.100.56:3000/denozord/EvoBGP/commit/c811c43))
* feat: add auto-updater service for Docker stack with configurable settings ([ae9fbe3](http://192.168.100.56:3000/denozord/EvoBGP/commit/ae9fbe3))
* feat: add bird configuration preview functionality with tabbed interface for fragments and prefixes. ([1969a00](http://192.168.100.56:3000/denozord/EvoBGP/commit/1969a00))
* feat: add BIRD status endpoint and integrate birdc checks into job processing. Enhance UI to display ([94a4c6a](http://192.168.100.56:3000/denozord/EvoBGP/commit/94a4c6a))
* feat: add CSV import and export functionality for module entries. Implement endpoints for exporting  ([bae2d68](http://192.168.100.56:3000/denozord/EvoBGP/commit/bae2d68))
* feat: add debug logging and optimize database queries in Postgres repository ([6d0e214](http://192.168.100.56:3000/denozord/EvoBGP/commit/6d0e214))
* feat: add diagnostic log endpoint for revisions and implement download functionality ([2260ecf](http://192.168.100.56:3000/denozord/EvoBGP/commit/2260ecf))
* feat: add enabled state to BGP peer types and UI. Implement peer enabling toggle in the network page ([80965ad](http://192.168.100.56:3000/denozord/EvoBGP/commit/80965ad))
* feat: add last refreshed timestamp to modules and CDN sources ([1e13ef9](http://192.168.100.56:3000/denozord/EvoBGP/commit/1e13ef9))
* feat: add live BGP protocol state tracking to peer listing ([8a50ba4](http://192.168.100.56:3000/denozord/EvoBGP/commit/8a50ba4))
* feat: add optional name field to BGP peer creation and update UI ([1a61ae2](http://192.168.100.56:3000/denozord/EvoBGP/commit/1a61ae2))
* feat: add optional name field to DoH profile schema and types ([6bee5ad](http://192.168.100.56:3000/denozord/EvoBGP/commit/6bee5ad))
* feat: add smart aggregation for IPv4 CIDR prefixes in refresh pipeline. Implement functionality to a ([0428b83](http://192.168.100.56:3000/denozord/EvoBGP/commit/0428b83))
* feat: add strict bind configuration to BGP templates. Enhance BGP template rendering by including 's ([07f2ce5](http://192.168.100.56:3000/denozord/EvoBGP/commit/07f2ce5))
* feat: add tenant refresh functionality to HTTP API and job processing ([7a2b015](http://192.168.100.56:3000/denozord/EvoBGP/commit/7a2b015))
* feat: auto-refresh non-IP module updates and add full project manual ([1e77e1a](http://192.168.100.56:3000/denozord/EvoBGP/commit/1e77e1a))
* feat: enhance API client with pagination and new data fetching functions. Implement `apiPageAll` for ([9f0b128](http://192.168.100.56:3000/denozord/EvoBGP/commit/9f0b128))
* feat: enhance BGP template and peer configuration handling. Introduce policies_json parameter for lo ([be8e7b5](http://192.168.100.56:3000/denozord/EvoBGP/commit/be8e7b5))
* feat: enhance CDN source management with last refreshed timestamp ([4776434](http://192.168.100.56:3000/denozord/EvoBGP/commit/4776434))
* feat: enhance community attribute handling in BIRD format. Add validation for community route bodies ([b541794](http://192.168.100.56:3000/denozord/EvoBGP/commit/b541794))
* feat: enhance community label handling and improve UI localization across operations pages. Introduc ([53fd527](http://192.168.100.56:3000/denozord/EvoBGP/commit/53fd527))
* feat: enhance deployment configuration for secure Web UI with Traefik integration ([4acd685](http://192.168.100.56:3000/denozord/EvoBGP/commit/4acd685))
* feat: enhance DoH profile management and resolver policy in modules ([34ecc5c](http://192.168.100.56:3000/denozord/EvoBGP/commit/34ecc5c))
* feat: enhance EvoBGP with new command-line options for the evobgp-agent, including a watch command f ([6a55f72](http://192.168.100.56:3000/denozord/EvoBGP/commit/6a55f72))
* feat: enhance evobgp with new command-line tools for bundle management, including pull, verify, and  ([bf52b21](http://192.168.100.56:3000/denozord/EvoBGP/commit/bf52b21))
* feat: enhance job detail handling in UI with caching and loading states. Introduce SvelteMap for job ([05bc5c4](http://192.168.100.56:3000/denozord/EvoBGP/commit/05bc5c4))
* feat: enhance job detail UI with improved overflow handling and layout adjustments. Update card comp ([67ce135](http://192.168.100.56:3000/denozord/EvoBGP/commit/67ce135))
* feat: enhance job filtering and display functionality in operations page. Introduce job search and f ([e1c4cbc](http://192.168.100.56:3000/denozord/EvoBGP/commit/e1c4cbc))
* feat: enhance job processing with detailed logging and UI improvements. Introduce revision log entri ([c48cc4e](http://192.168.100.56:3000/denozord/EvoBGP/commit/c48cc4e))
* feat: enhance monitoring page with detailed status tracking and error handling ([458a885](http://192.168.100.56:3000/denozord/EvoBGP/commit/458a885))
* feat: enhance observability with prefix aggregation metrics ([07b97bd](http://192.168.100.56:3000/denozord/EvoBGP/commit/07b97bd))
* feat: enhance peer neighbor handling by introducing NormalizePeerNeighborString function. Update Cre ([80f3a8d](http://192.168.100.56:3000/denozord/EvoBGP/commit/80f3a8d))
* feat: enhance RefreshModule to aggregate prefixes from all enabled tenant modules. Update documentat ([431c6a9](http://192.168.100.56:3000/denozord/EvoBGP/commit/431c6a9))
* feat: enhance RefreshModule to support legacy ASN resolution and improve prefix handling. Introduce  ([e1aaa18](http://192.168.100.56:3000/denozord/EvoBGP/commit/e1aaa18))
* feat: enhance settings page layout and section management ([d91667f](http://192.168.100.56:3000/denozord/EvoBGP/commit/d91667f))
* feat: enhance UI components with improved styling and layout adjustments. Update button sizes, text  ([6a72842](http://192.168.100.56:3000/denozord/EvoBGP/commit/6a72842))
* feat: enhance worker functionality by adding job queuing for deploy_apply. Introduce a Registry fiel ([42d956d](http://192.168.100.56:3000/denozord/EvoBGP/commit/42d956d))
* feat: implement AS holder name resolution and enhance ASEntry structure. Add ASHolderName function t ([c958d5a](http://192.168.100.56:3000/denozord/EvoBGP/commit/c958d5a))
* feat: implement bulk selection and deletion for modules and entries. Add functionality for selecting ([e816d17](http://192.168.100.56:3000/denozord/EvoBGP/commit/e816d17))
* feat: implement CDN source preview functionality and enhance data handling. Add a new endpoint for p ([e20c9f3](http://192.168.100.56:3000/denozord/EvoBGP/commit/e20c9f3))
* feat: implement countAll function for API data retrieval ([40c8a8d](http://192.168.100.56:3000/denozord/EvoBGP/commit/40c8a8d))
* feat: implement domain resolution via DoH in refresh pipeline. Enhance the collectModulePrefixRows f ([678bd27](http://192.168.100.56:3000/denozord/EvoBGP/commit/678bd27))
* feat: implement effective peer enabling logic during peer creation. Update CreatePeer methods in Pos ([3e85431](http://192.168.100.56:3000/denozord/EvoBGP/commit/3e85431))
* feat: implement module refresh job management with concurrency control. Add CountOtherActiveModuleRe ([149fb0a](http://192.168.100.56:3000/denozord/EvoBGP/commit/149fb0a))
* feat: implement peer reconciliation job for BGP peers ([3b17228](http://192.168.100.56:3000/denozord/EvoBGP/commit/3b17228))
* feat: implement revision retention settings and pruning logic ([4f50350](http://192.168.100.56:3000/denozord/EvoBGP/commit/4f50350))
* feat: implement theme selection and automatic theme application based on user preference. Update App ([b70a643](http://192.168.100.56:3000/denozord/EvoBGP/commit/b70a643))
* feat: implement versioning and release management in the project ([bb334b1](http://192.168.100.56:3000/denozord/EvoBGP/commit/bb334b1))
* feat: improve BGP protocol summary handling by introducing isBGPProtocolSummaryRow function. Update  ([399622c](http://192.168.100.56:3000/denozord/EvoBGP/commit/399622c))
* feat: improve job detail UI with enhanced spacing and layout adjustments. Update table cell classes  ([97908b5](http://192.168.100.56:3000/denozord/EvoBGP/commit/97908b5))
* feat: improve peer speaker assignment UX in network settings ([f877ec8](http://192.168.100.56:3000/denozord/EvoBGP/commit/f877ec8))
* feat: improve UI components with enhanced overflow handling and layout adjustments. Update card and  ([0ca5542](http://192.168.100.56:3000/denozord/EvoBGP/commit/0ca5542))
* feat: improve UI layout and responsiveness in module and operations pages. Refactor component struct ([233f29b](http://192.168.100.56:3000/denozord/EvoBGP/commit/233f29b))
* feat: redesign settings page with structured controls ([2da6d49](http://192.168.100.56:3000/denozord/EvoBGP/commit/2da6d49))
* feat: refactor materialization logic to group static routes by community in the refresh pipeline. In ([1c5a9c9](http://192.168.100.56:3000/denozord/EvoBGP/commit/1c5a9c9))
* feat: refactor operations page to enhance UI and component structure. Remove unused card components  ([88226f6](http://192.168.100.56:3000/denozord/EvoBGP/commit/88226f6))
* feat: refine BGP template handling and documentation updates. Remove local IP requirements in BGP te ([6266a39](http://192.168.100.56:3000/denozord/EvoBGP/commit/6266a39))
* feat: refine job detail UI with improved layout and scrolling capabilities. Enhance spacing and over ([57c09ac](http://192.168.100.56:3000/denozord/EvoBGP/commit/57c09ac))
* feat: update BGP community structure and related documentation. Refactor BGP community handling by r ([a2cbfac](http://192.168.100.56:3000/denozord/EvoBGP/commit/a2cbfac))
* feat: update BGP template configuration and OpenAPI documentation. Change import settings from 'all' ([133ea93](http://192.168.100.56:3000/denozord/EvoBGP/commit/133ea93))
* feat: update BIRD installation process to build from source. Modify CI workflow to ensure BIRD 2.14  ([e2002ba](http://192.168.100.56:3000/denozord/EvoBGP/commit/e2002ba))
* feat: update CI configuration and Dockerfiles for improved build efficiency ([80229b3](http://192.168.100.56:3000/denozord/EvoBGP/commit/80229b3))
* feat: update collectModulePrefixRows to support prior snapshots and enhance CDN/domain prefix collec ([ee364c8](http://192.168.100.56:3000/denozord/EvoBGP/commit/ee364c8))
* feat: update Go module dependencies and enhance DNS resolution in refresh pipeline. Upgrade Go versi ([655d750](http://192.168.100.56:3000/denozord/EvoBGP/commit/655d750))
* feat: update OpenAPI documentation and internal logic for BIRD settings. Enhance the description of  ([62bf987](http://192.168.100.56:3000/denozord/EvoBGP/commit/62bf987))
* feat: update RevisionDiff type and enhance OperationsDiffTab component for better diff handling. Int ([4aaaa30](http://192.168.100.56:3000/denozord/EvoBGP/commit/4aaaa30))
* feat: update Traefik deployment configuration for Let's Encrypt and logging ([39ba976](http://192.168.100.56:3000/denozord/EvoBGP/commit/39ba976))
* docs: add router-lists-ui integration documentation to README ([b7a8aab](http://192.168.100.56:3000/denozord/EvoBGP/commit/b7a8aab))
* docs: add Web UI section to EvoBGP architecture plan, detailing Svelte and shadcn-svelte integration ([272542b](http://192.168.100.56:3000/denozord/EvoBGP/commit/272542b))
* docs: enhance EvoBGP architecture plan with new risk mitigation strategies, testing matrix for BIRD2 ([3992d01](http://192.168.100.56:3000/denozord/EvoBGP/commit/3992d01))
* docs: enhance README and CI workflow documentation for Docker image building and publishing process, ([9eb3e33](http://192.168.100.56:3000/denozord/EvoBGP/commit/9eb3e33))
* docs: enhance README and CI workflow to clarify Docker image build conditions and Gitea Container Re ([c6687b2](http://192.168.100.56:3000/denozord/EvoBGP/commit/c6687b2))
* docs: openapi.html со встроенным Redoc; инструкция для Gitea и скрипт сборки ([a754e1d](http://192.168.100.56:3000/denozord/EvoBGP/commit/a754e1d))
* docs: update AGENTS and README to include guidelines for Conventional Commits ([4c23232](http://192.168.100.56:3000/denozord/EvoBGP/commit/4c23232))
* docs: update AGENTS and README with engineering rules and guidelines ([aff27e8](http://192.168.100.56:3000/denozord/EvoBGP/commit/aff27e8))
* docs: update architecture and API documentation to clarify job processing, in-memory registry usage, ([acc0608](http://192.168.100.56:3000/denozord/EvoBGP/commit/acc0608))
* docs: update EvoBGP architecture plan with detailed REST API section, including general agreements,  ([1a86d6c](http://192.168.100.56:3000/denozord/EvoBGP/commit/1a86d6c))
* docs: update EvoBGP architecture plan with new repository structure, refined sections, and detailed  ([4480d64](http://192.168.100.56:3000/denozord/EvoBGP/commit/4480d64))
* docs: update README and CI workflow to clarify Docker workspace mounting for bird2 job ([6be46db](http://192.168.100.56:3000/denozord/EvoBGP/commit/6be46db))
* docs: update README and CI workflow to clarify Gitea Registry login process and improve Docker image ([f1b33b9](http://192.168.100.56:3000/denozord/EvoBGP/commit/f1b33b9))
* docs: update README and Docker Compose configurations to clarify usage of pre-built images from the  ([7fac79c](http://192.168.100.56:3000/denozord/EvoBGP/commit/7fac79c))
* docs: update README to include information about the EvoBGP web interface, linking to quickstart and ([5d21f01](http://192.168.100.56:3000/denozord/EvoBGP/commit/5d21f01))
* docs: статическая Redoc-страница OpenAPI (для просмотра в Gitea без встроенного рендера) ([5c9d2e2](http://192.168.100.56:3000/denozord/EvoBGP/commit/5c9d2e2))
* docs: убрать типографское тире и невидимый Unicode из текстов API ([fbe9cd3](http://192.168.100.56:3000/denozord/EvoBGP/commit/fbe9cd3))
* refactor: adjust BootstrapWorkers function call in server.go to change the order of returned values, ([032fa99](http://192.168.100.56:3000/denozord/EvoBGP/commit/032fa99))
* refactor: consolidate module interval display logic and improve edit form handling ([1f85654](http://192.168.100.56:3000/denozord/EvoBGP/commit/1f85654))
* refactor: enhance bird metrics polling with context support ([f6b94a4](http://192.168.100.56:3000/denozord/EvoBGP/commit/f6b94a4))
* refactor: enhance CI workflow for granular change detection by module and improve idempotency key ge ([54eaad3](http://192.168.100.56:3000/denozord/EvoBGP/commit/54eaad3))
* refactor: enhance job status checks in worker tests ([6107c3e](http://192.168.100.56:3000/denozord/EvoBGP/commit/6107c3e))
* refactor: enhance JSON struct tags for various data models in backend and memory store. Update field ([4a860a0](http://192.168.100.56:3000/denozord/EvoBGP/commit/4a860a0))
* refactor: enhance module display and input handling in module pages ([25374a6](http://192.168.100.56:3000/denozord/EvoBGP/commit/25374a6))
* refactor: enhance module interval label logic and add reset buttons in edit forms ([b99fc17](http://192.168.100.56:3000/denozord/EvoBGP/commit/b99fc17))
* refactor: improve code consistency and formatting across components ([c263fd5](http://192.168.100.56:3000/denozord/EvoBGP/commit/c263fd5))
* refactor: improve handling of deployable fragments in ApplyRevision and buildPreviewFragments functi ([6932271](http://192.168.100.56:3000/denozord/EvoBGP/commit/6932271))
* refactor: improve module save functionality and nullable value handling ([05ec176](http://192.168.100.56:3000/denozord/EvoBGP/commit/05ec176))
* refactor: improve PATCH handling for module updates with nullable field support ([e700f90](http://192.168.100.56:3000/denozord/EvoBGP/commit/e700f90))
* refactor: optimize job listing and counting logic for concurrency safety. Separate tenant filtering  ([d83efe2](http://192.168.100.56:3000/denozord/EvoBGP/commit/d83efe2))
* refactor: phase 2 structural alignment — reports, importer, nodecli, pagination ([250cdea](http://192.168.100.56:3000/denozord/EvoBGP/commit/250cdea))
* refactor: remove strict bind configuration from BGP templates and update related logic. Eliminate 's ([482eeb1](http://192.168.100.56:3000/denozord/EvoBGP/commit/482eeb1))
* refactor: streamline Docker build process in CI configuration and documentation ([6025f38](http://192.168.100.56:3000/denozord/EvoBGP/commit/6025f38))
* refactor: streamline edit form handling for community and DoH profile selections ([1765ab8](http://192.168.100.56:3000/denozord/EvoBGP/commit/1765ab8))
* refactor: streamline module refresh process and enhance revision handling ([10e2415](http://192.168.100.56:3000/denozord/EvoBGP/commit/10e2415))
* refactor: switch BIRD installation from source to Ubuntu package. Update CI workflow and Dockerfiles ([0b3ceaa](http://192.168.100.56:3000/denozord/EvoBGP/commit/0b3ceaa))
* refactor: update AS entry handling in API and database to support ASN-only entries. Remove prefix fi ([1db6b5b](http://192.168.100.56:3000/denozord/EvoBGP/commit/1db6b5b))
* refactor: update Docker build context and paths in CI configuration and documentation ([7d52a4a](http://192.168.100.56:3000/denozord/EvoBGP/commit/7d52a4a))
* refactor: update Docker build variable syntax in CI configuration and documentation ([27e01d0](http://192.168.100.56:3000/denozord/EvoBGP/commit/27e01d0))
* refactor: update job kind display logic in schedule page ([dc1f4a5](http://192.168.100.56:3000/denozord/EvoBGP/commit/dc1f4a5))
* refactor: update ListModules and GetModule methods in Postgres repository to include cron expression ([73d0003](http://192.168.100.56:3000/denozord/EvoBGP/commit/73d0003))
* refactor: update Switch component styles and structure for improved accessibility and responsiveness ([d4282c2](http://192.168.100.56:3000/denozord/EvoBGP/commit/d4282c2))
* refactor: update worker processes in EvoBGP to accept dependencies for shared store and job registry ([b7968db](http://192.168.100.56:3000/denozord/EvoBGP/commit/b7968db))
* ci: add conditional job for OpenAPI linting based on file changes; update README with workflow detai ([3adfc49](http://192.168.100.56:3000/denozord/EvoBGP/commit/3adfc49))
* ci: phase 4 automation — web job, lint-httpapi, golangci ([64516c3](http://192.168.100.56:3000/denozord/EvoBGP/commit/64516c3))
* ci: update README and CI workflow to improve bird2 job installation process and clarify Docker usage ([eddc6b8](http://192.168.100.56:3000/denozord/EvoBGP/commit/eddc6b8))
* chore: phase 3 code quality — pipeline prefixes, godoc baseline ([ea65451](http://192.168.100.56:3000/denozord/EvoBGP/commit/ea65451))
* chore: update CI configuration to disable filesystem entitlement for Docker builds ([9ab7de6](http://192.168.100.56:3000/denozord/EvoBGP/commit/9ab7de6))
* chore: update package dependencies and enhance AppShell component with new navigation items and tool ([7d85fef](http://192.168.100.56:3000/denozord/EvoBGP/commit/7d85fef))
* chore: upgrade Go version to 1.24 in Dockerfiles for evobgp-agent and gobinary. This change ensures  ([4ba5b03](http://192.168.100.56:3000/denozord/EvoBGP/commit/4ba5b03))
* fix: adjust Postgres resource limits and healthcheck parameters. Increase memory limit to 256M and m ([20e674c](http://192.168.100.56:3000/denozord/EvoBGP/commit/20e674c))
* fix: improve input handling and validation for settings fields ([1386b63](http://192.168.100.56:3000/denozord/EvoBGP/commit/1386b63))
* fix: improve sidebar and navigation layout in AppShell component ([9f02128](http://192.168.100.56:3000/denozord/EvoBGP/commit/9f02128))
* fix: update BGP configuration in test scenarios to use new IPv6 addresses ([9fd9745](http://192.168.100.56:3000/denozord/EvoBGP/commit/9fd9745))
* fix: update bird build script and Dockerfile to include libncurses dependency ([0cb5a7d](http://192.168.100.56:3000/denozord/EvoBGP/commit/0cb5a7d))
* fix: update bird build script and Dockerfile to include readline dependency ([35ed79b](http://192.168.100.56:3000/denozord/EvoBGP/commit/35ed79b))
* fix: update bird build script and Dockerfile to remove readline dependency ([fefadac](http://192.168.100.56:3000/denozord/EvoBGP/commit/fefadac))
* fix: update deployment configuration for Traefik and secure Web UI ([00bd037](http://192.168.100.56:3000/denozord/EvoBGP/commit/00bd037))
* fix: update Docker build commands in CI configuration and documentation ([b5061a5](http://192.168.100.56:3000/denozord/EvoBGP/commit/b5061a5))
* fix(httpapi): phase 1 compliance — ERR-01, SEC-04, CDN client, scheduler docs ([1dd7135](http://192.168.100.56:3000/denozord/EvoBGP/commit/1dd7135))
* test: add CDN prefetch tests for 304 responses and snapshot handling ([a536a2d](http://192.168.100.56:3000/denozord/EvoBGP/commit/a536a2d))
* test: add validation for 'strict bind on' in BGP rendering tests. Enhance BGP test cases to ensure ' ([dc5e20e](http://192.168.100.56:3000/denozord/EvoBGP/commit/dc5e20e))
* docker: refactor cache handling in Docker bake configuration ([5e59a1c](http://192.168.100.56:3000/denozord/EvoBGP/commit/5e59a1c))
* docker: simplify web image targets in Docker bake configuration ([765a805](http://192.168.100.56:3000/denozord/EvoBGP/commit/765a805))
* docker: update web image build process and README ([92f6de8](http://192.168.100.56:3000/denozord/EvoBGP/commit/92f6de8))
* web(ui): app layout — sidebar, mobile sheet, topbar, PageHeader ([2d78c81](http://192.168.100.56:3000/denozord/EvoBGP/commit/2d78c81))
* web(ui): AppDataTable pattern and modules pilot ([b992301](http://192.168.100.56:3000/denozord/EvoBGP/commit/b992301))
* web(ui): design tokens — spacing and component scale ([ac635e9](http://192.168.100.56:3000/denozord/EvoBGP/commit/ac635e9))
* web(ui): feedback — confirm dialog, skeletons, README ([87309cf](http://192.168.100.56:3000/denozord/EvoBGP/commit/87309cf))
* web(ui): form patterns — FormField, formsnap, directories pilot ([77a2cfd](http://192.168.100.56:3000/denozord/EvoBGP/commit/77a2cfd))
* web(ui): foundation — ui/core layout, shadcn primitives, notify ([00a2a44](http://192.168.100.56:3000/denozord/EvoBGP/commit/00a2a44))
* web(ui): migrate dashboard, schedule, monitoring to ui patterns ([57dcf30](http://192.168.100.56:3000/denozord/EvoBGP/commit/57dcf30))
* web(ui): migrate network to ui patterns ([3a3056b](http://192.168.100.56:3000/denozord/EvoBGP/commit/3a3056b))
* web(ui): migrate operations and module detail to ui patterns ([1f9930d](http://192.168.100.56:3000/denozord/EvoBGP/commit/1f9930d))
* web(ui): migrate settings to ui patterns ([9808aca](http://192.168.100.56:3000/denozord/EvoBGP/commit/9808aca))
* Enhance EvoBGP architecture plan by refining the overview, updating todos with detailed descriptions ([30efdbf](http://192.168.100.56:3000/denozord/EvoBGP/commit/30efdbf))
* init ([cbf3aaa](http://192.168.100.56:3000/denozord/EvoBGP/commit/cbf3aaa))
* Refine EvoBGP architecture plan with updated database schema details, improved descriptions for comp ([431ef3e](http://192.168.100.56:3000/denozord/EvoBGP/commit/431ef3e))
* Update EvoBGP architecture plan with new overview and todos. Refined descriptions for control-plane  ([31ff5bb](http://192.168.100.56:3000/denozord/EvoBGP/commit/31ff5bb))
2026-05-19 17:58:24 +00:00
Denozordec bb334b10f5 feat: implement versioning and release management in the project
CI / commitlint (push) Has been skipped
CI / openapi (push) Successful in 23s
CI / web (push) Successful in 28s
CI / go (push) Successful in 26s
CI / bird2 (push) Successful in 15s
CI / changes (push) Successful in 7s
- Added automatic versioning based on git tags using semantic-release in Gitea Actions.
- Introduced new API endpoints `GET /version` and `GET /v1/version` to expose build metadata.
- Updated Docker build process to include version and build time information in Go binaries.
- Enhanced documentation with details on release processes and versioning guidelines.
- Integrated version display in the web application for better user visibility.
2026-05-20 00:56:02 +07:00
Denozordec 4c23232c4e docs: update AGENTS and README to include guidelines for Conventional Commits
- Added a section on Conventional Commits in AGENTS.md, detailing the process for generating commit messages.
- Enhanced README.md with references to the Conventional Commits rules and the necessary scripts for generating commit messages.
- Clarified the format for commit messages, specifying the language requirements for headers and bodies.
2026-05-20 00:48:21 +07:00
Denozordec c263fd5c7e refactor: improve code consistency and formatting across components
CI / openapi (push) Has been skipped
CI / web (push) Successful in 29s
CI / go (push) Has been skipped
CI / docker-web (push) Successful in 1m22s
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / docker-go (push) Has been skipped
CI / changes (push) Successful in 7s
- Standardized formatting in various Svelte components for better readability.
- Updated README.md and .prettierrc for consistent styling and improved documentation.
- Refactored class attributes in UI components to enhance clarity and maintainability.
- Adjusted layout files to ensure proper alignment and spacing in the application interface.
2026-05-20 00:37:00 +07:00
Denozordec 6bee5adc34 feat: add optional name field to DoH profile schema and types
CI / openapi (push) Successful in 29s
CI / web (push) Failing after 45s
CI / go (push) Has been skipped
CI / docker-web (push) Successful in 1m37s
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / docker-go (push) Has been skipped
CI / changes (push) Successful in 10s
- Introduced an optional `name` property in the OpenAPI schema for DoH profiles, providing a display name for profiles.
- Updated TypeScript types to reflect the new optional `name` field in `DohProfile` and `DohProfileCreate` types.
2026-05-20 00:26:58 +07:00
DenozordecandCursor 64516c316d ci: phase 4 automation — web job, lint-httpapi, golangci
CI / openapi (push) Has been skipped
CI / web (push) Failing after 41s
CI / go (push) Successful in 52s
CI / docker-web (push) Successful in 2m14s
CI / docker-bird (push) Successful in 1m2s
CI / bird2 (push) Successful in 21s
CI / docker-go (push) Successful in 3m34s
CI / changes (push) Successful in 10s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 00:23:14 +07:00
DenozordecandCursor ea65451160 chore: phase 3 code quality — pipeline prefixes, godoc baseline
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 00:23:14 +07:00
DenozordecandCursor 250cdea35d refactor: phase 2 structural alignment — reports, importer, nodecli, pagination
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 00:22:39 +07:00
DenozordecandCursor 1dd713514c fix(httpapi): phase 1 compliance — ERR-01, SEC-04, CDN client, scheduler docs
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-20 00:21:16 +07:00
Denozordec aff27e8f7b docs: update AGENTS and README with engineering rules and guidelines
CI / openapi (push) Has been skipped
CI / go (push) Successful in 43s
CI / docker-web (push) Has been skipped
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 21s
CI / docker-go (push) Successful in 3m41s
CI / changes (push) Successful in 11s
- Added engineering rules references in AGENTS.md for code changes and specific areas (web, BIRD).
- Enhanced README.md to include links to engineering rules for different development areas.
- Updated birdfmt documentation to specify engineering rules for BIRD/BGP/IP.
- Clarified UI development guidelines in web/README.md to follow shadcn-svelte documentation and repository rules.
2026-05-20 00:14:14 +07:00
Denozordec 07b97bddc1 feat: enhance observability with prefix aggregation metrics
CI / changes (push) Successful in 7s
CI / openapi (push) Has been skipped
CI / go (push) Successful in 33s
CI / docker-web (push) Has been skipped
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 16s
CI / docker-go (push) Successful in 8m30s
- Added new histograms to track prefix aggregation duration, raw count, and aggregated count during tenant rendering.
- Implemented `RecordPrefixAggregation` function to record metrics for aggregation performance.
- Updated `RenderTenantRevision` and `RenderTenantRevisionFromPrefixes` functions to log aggregation statistics.
- Refactored `smartAggregatePrefixRows` to support both IPv4 and IPv6 aggregation, improving overall prefix handling.
2026-05-19 15:38:33 +07:00
Denozordec 34ecc5c235 feat: enhance DoH profile management and resolver policy in modules
CI / changes (push) Successful in 6s
CI / openapi (push) Successful in 1m2s
CI / go (push) Successful in 27s
CI / docker-web (push) Successful in 1m27s
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 16s
CI / docker-go (push) Successful in 8m5s
- Added `DohResolverPolicy` schema to OpenAPI documentation, defining policies for domain resolution.
- Updated module handling to support multiple DoH profiles via `doh_profile_ids` and introduced `doh_resolver_policy` in the API.
- Refactored related functions to accommodate the new DoH profile structure, ensuring backward compatibility with existing `doh_profile_id`.
- Enhanced UI components to allow selection and management of DoH profiles and policies in the web interface.
- Updated database interactions to handle new fields and ensure proper data normalization.
2026-05-19 14:57:50 +07:00
Denozordec a536a2d5eb test: add CDN prefetch tests for 304 responses and snapshot handling
CI / openapi (push) Has been skipped
CI / go (push) Successful in 29s
CI / docker-web (push) Has been skipped
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 15s
CI / docker-go (push) Successful in 8m32s
CI / changes (push) Successful in 8s
- Introduced `TestRefreshModuleIngest_CDN304UsesStoredSnapshot` to verify that the module ingest correctly uses stored snapshots when receiving a 304 Not Modified response.
- Added `TestCollectModulePrefixRows_CDN304RetriesWithoutETag` to ensure that the system retries without the If-None-Match header after a 304 response, and correctly collects prefixes on subsequent successful requests.
- Updated `cachedCDNPrefixRows` function to improve handling of cached prefixes during CDN source processing.
- Modified `RefreshModuleIngest` to utilize prior snapshots when collecting module prefix rows.
2026-05-19 14:26:49 +07:00
Denozordec 765a8052da docker: simplify web image targets in Docker bake configuration
CI / openapi (push) Has been skipped
CI / go (push) Has been skipped
CI / docker-web (push) Successful in 1m20s
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 15s
CI / docker-go (push) Successful in 7m50s
CI / changes (push) Successful in 8s
- Removed the "web-deps" and "web-build" targets from the web-images group to streamline the build process.
- Updated the configuration to focus on the primary targets "evobgp-web" and "evobgp-web-all".
2026-05-19 14:13:40 +07:00
Denozordec 5e59a1c334 docker: refactor cache handling in Docker bake configuration
CI / changes (push) Successful in 7s
CI / openapi (push) Has been skipped
CI / go (push) Has been skipped
CI / docker-web (push) Failing after 15s
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 14s
CI / docker-go (push) Successful in 2m36s
- Renamed cache functions for Go and web targets to clarify their purpose and prevent race conditions during parallel builds.
- Updated targets to use the new cache export functions, ensuring proper cache management.
- Revised README to reflect changes in cache usage and provide guidance on resolving potential CI issues related to cache schema changes.
2026-05-19 14:12:10 +07:00
Denozordec 92f6de8695 docker: update web image build process and README
CI / openapi (push) Has been skipped
CI / go (push) Has been skipped
CI / docker-web (push) Failing after 16s
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Successful in 22s
CI / docker-go (push) Failing after 51s
CI / changes (push) Successful in 8s
- Added a new target "web-build" to streamline the build process for web artifacts.
- Updated "evobgp-web" and "evobgp-web-all" targets to utilize the new "web-build" context.
- Modified Dockerfile to reflect changes in the build stages and context for web artifacts.
- Revised README to clarify the new build process and dependencies.
2026-05-19 13:37:32 +07:00
DenozordecandCursor 87309cfcec web(ui): feedback — confirm dialog, skeletons, README
CI / openapi (push) Has been skipped
CI / go (push) Has been skipped
CI / docker-web (push) Failing after 1m21s
CI / docker-bird (push) Has been skipped
CI / bird2 (push) Has been skipped
CI / docker-go (push) Has been skipped
CI / changes (push) Successful in 7s
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 12:01:15 +07:00
DenozordecandCursor 1f9930d467 web(ui): migrate operations and module detail to ui patterns
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 12:01:15 +07:00
DenozordecandCursor 9808aca44f web(ui): migrate settings to ui patterns
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 12:01:15 +07:00
DenozordecandCursor 3a3056ba76 web(ui): migrate network to ui patterns
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 12:01:15 +07:00
DenozordecandCursor 57dcf306f3 web(ui): migrate dashboard, schedule, monitoring to ui patterns
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 12:01:11 +07:00
DenozordecandCursor b992301c99 web(ui): AppDataTable pattern and modules pilot
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 11:59:34 +07:00
DenozordecandCursor 77a2cfd426 web(ui): form patterns — FormField, formsnap, directories pilot
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 11:59:25 +07:00
DenozordecandCursor ac635e91eb web(ui): design tokens — spacing and component scale
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 11:56:38 +07:00
DenozordecandCursor 2d78c8156f web(ui): app layout — sidebar, mobile sheet, topbar, PageHeader
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 11:56:24 +07:00
DenozordecandCursor 00a2a44630 web(ui): foundation — ui/core layout, shadcn primitives, notify
Co-authored-by: Cursor <cursoragent@cursor.com>
2026-05-19 11:54:29 +07:00
384 changed files with 25358 additions and 6783 deletions

No files matched your search

+3
View File
@@ -0,0 +1,3 @@
{
"extends": ["@commitlint/config-conventional"]
}
+37
View File
@@ -0,0 +1,37 @@
# Сгенерировать commit message (EvoBGP)
Сгенерируй сообщение коммита для **текущих staged-изменений**. Не выполняй `git commit`, если пользователь явно не просил закоммитить.
## Обязательный workflow (MUST)
1. Прочитай скилл [`.cursor/skills/commit-message/SKILL.md`](../skills/commit-message/SKILL.md).
2. Следуй правилу [`.cursor/rules/conventional-commits.mdc`](../rules/conventional-commits.mdc).
3. **Первым вызовом Shell** из корня репозитория:
```powershell
powershell -NoProfile -File scripts/commit/staged-context.ps1
```
4. Строй текст **только** по JSON stdout (`groups`, `stat`, `diff_excerpt`). Exit `1` → index пуст, сообщи пользователю.
5. **Запрещено** обходить скрипт через один `git diff --cached`.
## Формат вывода
Для каждого коммита (при auto-split — по одному блоку):
```
<type>(<scope>): <summary in English>
<тело на русском>
```
Плюс пояснение (RU): semver impact (`minor`|`patch`|`none`|`major`), почему выбран type, был ли split.
## Semver (кратко)
- Новая пользовательская возможность → `feat` (minor)
- Починка ожидаемого поведения / баг → `fix` (patch)
- Follow-up баги после недавнего `feat` в том же scope → **`fix`**, не `feat`
- Только перестройка без нового поведения → `refactor` (patch, без новых функций)
Заголовок — EN, императив, ≤72 символов. Тело — RU.
+178
View File
@@ -0,0 +1,178 @@
---
description: >-
Conventional Commits EvoBGP. Триггеры commit/коммит/закоммить/staged/commit message —
ОБЯЗАТЕЛЬНО сначала Shell scripts/commit/staged-context.ps1, скилл commit-message.
alwaysApply: false
---
# Conventional Commits (EvoBGP)
Полное описание релизного пайплайна: [docs/releasing.md](../../docs/releasing.md).
## Триггеры (применить правило + скилл)
Любой запрос на коммит или сообщение коммита: `commit`, `коммит`, `закоммить`, `git commit`, `commit message`, `conventional commit`, `staged`, «сгенерируй коммит», **`/commit-message`** — в т.ч. если это указано в плане или [AGENTS.md](../../AGENTS.md).
### Не путать с кнопкой ✨ в Source Control
Команда **`cursor.generateGitCommitMessage`** (sparkle в поле commit message) **не** читает Rules, Skills и `staged-context.ps1` — только staged diff и история коммитов ([ограничение Cursor](https://forum.cursor.com/t/how-to-set-prompt-for-generate-commit-message/148606)).
**Замена для EvoBGP:** Agent → `/commit-message` или команда [`.cursor/commands/commit-message.md`](../commands/commit-message.md).
## Обязательный запуск скрипта (MUST)
1. Прочитать скилл [`.cursor/skills/commit-message/SKILL.md`](../skills/commit-message/SKILL.md).
2. **Первым действием** выполнить Shell (из корня репо):
```powershell
powershell -NoProfile -File scripts/commit/staged-context.ps1
```
3. Сообщение коммита строить **только** по JSON из stdout скрипта (`groups`, `diff_excerpt`, `stat`).
4. **Запрещено** генерировать commit message без успешного (exit 0) запуска скрипта; не заменять скрипт одним `git diff --cached`.
## Формат (строго)
```
<type>(<scope>): <short summary in English>
<тело на русском: что изменено и зачем>
```
- **Заголовок** — только английский; императив, без точки в конце; ≤72 символов.
- **Тело** — только русский; полные предложения; пустая строка после заголовка.
- Запрещены vague-сообщения: `fix bug`, `update code`, `wip`, `misc`.
## Типы (semantic-release)
| type | Когда | Версия |
|------|--------|--------|
| `feat` | **новая** пользовательская возможность (раньше нельзя было) | minor |
| `fix` | восстановление **ожидаемого** поведения; баг, регрессия, падение UI | patch |
| `perf` | ускорение без смены API | patch |
| `refactor` | реструктуризация **без** новой возможности и **без** исправления бага | patch |
| `docs` | только документация | — |
| `test` | тесты | — |
| `ci` | CI/CD (`.gitea/`, workflows); правки, из‑за которых нужны новые образы | patch |
| `chore` | обслуживание, deps, `.cursor/` | — |
### Выбор type: semver, а не «красивые слова»
**Главный вопрос:** что изменится для пользователя после релиза?
1. Появилось **новое** действие / экран / API / настройка, которых не было → `feat`
2. То, что **должно было работать**, не работало (кнопки, диалоги, сохранение, 500) → `fix`
3. Только перестройка кода или UI на другой паттерн, поведение для пользователя то же → `refactor` (patch, без новых функций)
4. Ускорение без изменения контракта → `perf`
**Не путать с формулировкой diff:**
| В diff / задаче часто пишут | Неверный type | Верный type, если… |
|-----------------------------|---------------|---------------------|
| enhance, improve, polish UI | `feat` | …только чиним сломанное после прошлого PR → `fix` |
| refactor pages, unify tables | `feat` | …новой возможности нет, лишь перенос на AppDataTable → `refactor` |
| follow-up после feat(web) | `feat` | …исправляем баги того же экрана → `fix` |
**Follow-up rule:** коммит сразу после `feat` в той же области, который **не добавляет** новую возможность, а устраняет дефект (effect loop, не открывается dialog, confirm не срабатывает) — **`fix`**, не `feat`.
**Split при смешанном diff:** новая страница/flow → `feat`; отдельным коммитом правки багов → `fix`. Не объединять в один `feat`.
**Breaking changes** — только `feat!` / `fix!` / `BREAKING CHANGE:` когда пользователь **обязан** менять конфиг, API или привычный workflow.
### Обязательно в пояснении агенту
При каждом предложении коммита указать:
- **Semver impact:** `minor` | `patch` | `none` | `major`
- **Почему не другой type** (одно предложение), если diff большой или формулировка двусмысленная
Пример неправильно / правильно:
```
# Плохо — patch-фикс, minor-bump
feat(web): enhance module entry dialogs and selection handling
# Хорошо
fix(web): stop effect loop breaking module action buttons
Исправлен effect_update_depth_exceeded и bind:open у Dialog; кнопки редактирования/удаления снова работают.
```
```
# Плохо — рефакторинг без новой фичи
feat(web): migrate modules list to AppDataTable
# Хорошо — если не было нового user-facing
refactor(web): migrate modules list to AppDataTable
Единый паттерн таблиц; поведение списка модулей без изменений. Semver: patch.
```
```
# Хорошо feat — действительно новое
feat(web): add module create dialog on /modules
Диалог создания модуля с POST /v1/modules; раньше создание было только через API.
```
## Breaking changes
- Заголовок: `feat!` / `fix!` **или** в теле строка `BREAKING CHANGE:` (на английском ключевое слово) + описание impact **на русском**.
## Scope (EvoBGP)
Выбирать по доминирующему пути из staged diff:
| Префикс | scope |
|---------|-------|
| `internal/httpapi/` | `httpapi` |
| `internal/store/`, `internal/repository/`, `internal/db/` | `store` |
| `internal/jobs/` | `jobs` |
| `internal/pipeline/` | `pipeline` |
| `internal/birdfmt/` | `birdfmt` |
| `internal/birddeploy/` | `birddeploy` |
| `internal/bundle/`, `internal/signing/` | `bundle` |
| `cmd/` | `cmd` |
| `web/` | `web` |
| `docs/openapi.yaml`, `redocly.yaml` | `openapi` |
| `docs/` (остальное) | `docs` |
| `migrations/` | `db` |
| `.gitea/` | `ci` |
| `deploy/` | `deploy` |
| `.cursor/` | `chore` |
| прочее в корне | `chore` |
`type` определять по **содержимому diff**, не только по пути.
## Multi-change
- Несвязанные области → **отдельные коммиты** (auto-split по скиллу).
- Одна логическая фича через несколько scope (OpenAPI + httpapi + web) → **один** коммит, scope по главной области.
## Обязательный вывод агенту
Для каждого коммита:
**1. Готовое сообщение** (копировать в `git commit -m` / HEREDOC):
```
<type>(<scope>): <summary>
<тело RU>
```
**2. Пояснение (RU):** semver impact (`minor`|`patch`|`none`|`major`); почему выбран type; риск/impact; был ли split.
## Примеры
```
feat(httpapi): add version endpoint and wire web footer
Добавлен GET /v1/version. В Web UI версия в footer берётся из API вместо захардкоженного значения.
```
```
fix(auth): correct token validation edge case
Исправлена ложная 401 при истёкшем refresh-токене с валидной сессией.
```
+243
View File
@@ -0,0 +1,243 @@
---
description: Инженерные правила EvoBGP — Go, API, migrations, общие стандарты
alwaysApply: true
---
# Engineering Guidelines — EvoBGP
Специализированные правила: **Web UI** → `.cursor/rules/web-shadcn.mdc`; **BIRD2 / BGP / IP** → `.cursor/rules/networking-bird.mdc`.
Карта репозитория: `AGENTS.md`, `docs/architecture.md`. HTTP-контракт: `docs/openapi.yaml`.
---
## Architecture
**ARCH-01** | MUST | Новая persistence-логика — метод `store.Backend` + реализации в `repository` и `store.Memory`; SQL не в `httpapi`.
*Rationale:* единая абстракция данных.
*Проверка:* CI `scripts/lint-httpapi.sh`; grep SQL в `internal/httpapi` — отсутствие.
**ARCH-02** | MUST | HTTP-маршруты только в `internal/httpapi`; регистрация через `http.ServeMux` с паттернами `METHOD /v1/...`.
*Rationale:* один слой REST.
*Проверка:* маршруты только в `routes.go`, `routes_crud.go`.
**ARCH-03** | MUST | Долгие операции (refresh, apply, rollback) — `jobs.Registry`; ответ `202` + `job_id` где задано OpenAPI.
*Rationale:* не блокировать HTTP worker.
*Проверка:* OpenAPI + handlers.
**ARCH-04** | MUST | Кросс-процессные воркеры — HTTP к API или общая БД; не shared memory (кроме `evobgp-all`).
*Rationale:* `jobs.Registry` in-process only.
*Проверка:* `docs/architecture.md`; review.
**ARCH-05** | MUST | `birdfmt` не импортирует `httpapi` / `store`.
*Rationale:* направление зависимостей вниз.
*Проверка:* `go list -deps` / review imports.
**ARCH-06** | MUST | Точки входа `cmd/*` — тонкий `main`: config, wiring, signal/shutdown; без бизнес-логики.
*Rationale:* тестируемость `internal/`.
*Проверка:* review `main.go`.
**ARCH-07** | MUST | Инициализация store + jobs для API и воркеров — `httpapi.BootstrapWorkers`.
*Rationale:* единый wiring.
*Проверка:* `bootstrap.go`.
**ARCH-08** | MUST | Ingest+render префиксов — `internal/pipeline`; генерация BIRD-текста — `internal/birdfmt`.
*Rationale:* разделение data plane / control plane.
*Проверка:* review пакетов.
**ARCH-09** | NEVER | Прямой доступ handler'ов к `pgxpool` для CRUD; только `store.Backend`.
*Rationale:* абстракция бэкенда.
*Проверка:* review `httpapi`.
**ARCH-10** | MUST | Подпись бандлов — `internal/bundle` + `internal/signing`; не дублировать crypto в handlers.
*Rationale:* единая криптография артефактов.
*Проверка:* review.
---
## Code Style
**STYLE-01** | MUST | Go-код после `gofmt`; перед PR — `go vet ./...`.
*Rationale:* единый стиль.
*Проверка:* CI job `go`.
**STYLE-02** | MUST | Экспортируемые типы/функции публичных пакетов — godoc-комментарий.
*Rationale:* навигация по API пакетов.
*Проверка:* review.
**STYLE-03** | NEVER | `panic` в `internal/*` вне `init` и тестов.
*Rationale:* предсказуемые ошибки.
*Проверка:* grep `panic(`.
**STYLE-04** | MUST | Ошибки пакетов с префиксом (`birdfmt:`, `db:`, `httpapi:`) и `%w` при оборачивании.
*Rationale:* трассировка.
*Проверка:* review.
**STYLE-05** | MUST | HTTP-ошибки — `writeProblem` / `writeJSON` (`application/problem+json` для 4xx/5xx).
*Rationale:* RFC 9457, OpenAPI.
*Проверка:* `problem.go`; CI `scripts/lint-httpapi.sh` (5xx и 4xx store/cdn/csv).
**STYLE-06** | MUST | JSON полей HTTP DTO согласованы с `docs/openapi.yaml`.
*Rationale:* контракт API.
*Проверка:* OpenAPI diff + review.
---
## Dependency Management
**DEP-01** | MUST | Go-зависимости — через `go get` / `go.mod`; версия Go как в `go.mod` и CI (1.24).
*Проверка:* `go.mod`, `.gitea/workflows/ci.yaml`.
**DEP-02** | NEVER | Vendor-копирование без явного решения в репозитории.
*Проверка:* review.
**DEP-03** | MUST | Миграции схемы — пары `.up.sql`/`.down.sql` для **postgres** и **sqlite**, синхронная нумерация.
*Проверка:* `migrations/postgres/`, `migrations/sqlite/`.
**DEP-04** | MUST | Web UI-библиотеки — только экосистема shadcn-svelte/bits-ui (см. `web-shadcn.mdc`).
*Проверка:* `web/package.json` review.
---
## Error Handling
**ERR-01** | MUST | HTTP 5xx — без сырого `err.Error()` клиенту; detail через `writeProblem`.
*Проверка:* review handlers.
**ERR-02** | MUST | Публичные функции I/O — `(T, error)`; `errors.Is`/`errors.As` для sentinel.
*Проверка:* review.
**ERR-03** | MUST | `context.Context` — первый аргумент для I/O; таймауты на внешние HTTP (CDN, DoH, RIPEstat).
*Проверка:* `pipeline`, `bootstrap.go`.
---
## Testing
**TEST-01** | MUST | Перед PR: `go test ./... -race -count=1`.
*Проверка:* CI job `go`.
**TEST-02** | MUST | Табличные тесты — эталон для `birdfmt`, `pipeline`, `bundle`.
*Проверка:* `*_test.go`.
**TEST-03** | MUST | Новые BIRD-сценарии в `internal/birdfmt/testdata/scenarios/*/bird.conf` + `bird -p`.
*Проверка:* CI job `bird2`.
**TEST-04** | MUST | Изменения `web/` — локально `npm run check` и `npm run lint`; CI job `web` в `.gitea/workflows/ci.yaml`.
*Проверка:* локальные команды.
**TEST-05** | MUST | Изменения OpenAPI — `npx @redocly/cli lint docs/openapi.yaml`.
*Проверка:* CI job `openapi`.
---
## Documentation
**DOC-01** | MUST | Пользовательская документация в `docs/` — на русском.
*Проверка:* review.
**DOC-02** | MUST | Изменение HTTP API: сначала `docs/openapi.yaml`, затем handlers/store, затем `docs/api.md` при необходимости.
*Проверка:* PR diff order.
**DOC-03** | MUST | Расхождение spec/код — явно в `docs/access.md` или description операции OpenAPI.
*Проверка:* review (напр. `X-Tenant-Id`).
**DOC-04** | NEVER | Дублировать длинные фрагменты OpenAPI в комментариях; ссылка на путь/operationId.
*Проверка:* review.
---
## Naming Conventions
**NAME-01** | MUST | Бинарники: `cmd/evobgp-<role>/main.go`.
*Проверка:* `cmd/`.
**NAME-02** | MUST | Env-переменные: префикс `EVOBGP_`.
*Проверка:* `internal/config`, `docs/access.md`.
**NAME-03** | MUST | Job kinds — константы в `internal/jobs/worker.go` (`module_refresh`, …).
*Проверка:* grep `Kind`.
**NAME-04** | MUST | ID сущностей — UUID-строки; tenant только из API-ключа (auth context), не из недокументированных заголовков.
*Проверка:* `auth.go`, `docs/access.md`.
---
## Performance
**PERF-01** | MUST | Списки API — пагинация `cursor` + `limit`; не unbounded выборки в handlers.
*Проверка:* OpenAPI + store methods.
**PERF-02** | MUST | CDN HTTP — переиспользуемый `http.Client` с таймаутом (45s в bootstrap).
*Проверка:* `bootstrap.go`.
**PERF-03** | MUST | Новые метрики — `internal/observability`, экспорт `/metrics`.
*Проверка:* review.
---
## Security
**SEC-01** | NEVER | Секреты, API-ключи, токены в Git.
*Проверка:* review; `docs/access.md`.
**SEC-02** | NEVER | `EVOBGP_DEV_INSECURE=1` в production.
*Проверка:* ops review.
**SEC-03** | MUST | Роль `node` — только node API; CRUD — `403` для `node`.
*Проверка:* `auth.go`, handlers.
**SEC-04** | MUST | apply/rollback/settings — operator (или выше по `roleLevel`).
*Проверка:* review handlers.
**SEC-05** | MUST | Бандл на ноде — `verify-bundle` перед `apply-bundle`.
*Проверка:* `docs/access.md`.
**SEC-06** | MUST | CORS — явный whitelist `EVOBGP_CORS_ORIGINS`.
*Проверка:* `cors.go`.
---
## Documentation Sync Rules
| Область | Источник |
|---------|----------|
| Go / net/http | https://go.dev/doc/ |
| pgx v5 | https://pkg.go.dev/github.com/jackc/pgx/v5 |
| OpenAPI / problem+json | `docs/openapi.yaml`, RFC 9457 |
| Svelte / Kit | https://svelte.dev/docs , https://kit.svelte.dev/docs |
| shadcn-svelte | https://shadcn-svelte.com/docs |
| BIRD 2 | https://bird.network.cz/?get_doc |
| Prometheus Go | https://pkg.go.dev/github.com/prometheus/client_golang |
**DOC-SYNC-01** | MUST | Новый API библиотеки — сверка версии в `go.mod`/`package.json` с официальной документацией.
**DOC-SYNC-02** | NEVER | Устаревшие примеры (Svelte 4 `export let`, deprecated pgx).
**DOC-SYNC-03** | MUST | Конфликт docs: **OpenAPI (HTTP)** → **код** → обзорные `docs/`; `.cursor/plans/` не контракт.
**DOC-SYNC-04** | MUST | Сомнения по Svelte — Svelte MCP / `npm run check`.
**DOC-SYNC-05** | MUST | BIRD — официальная документация BIRD2 + `networking-bird.mdc` + `go test ./internal/birdfmt/...`.
Приоритет при сомнениях — **официальные источники**, не блоги и не «память модели».
---
## Enforcement Strategy
**CI (Gitea):** OpenAPI lint; `go vet`, `go test -race`, build `cmd/*`; `bird -p` на scenarios; Docker bake на push.
**Локально перед PR:**
```powershell
go vet ./...
go test ./... -race -count=1
npx @redocly/cli lint docs/openapi.yaml
# web: cd web; npm run check; npm run lint
# birdfmt: go test ./internal/birdfmt/... -count=1
```
**CI (Gitea):** job `web` (check + lint); job `go`: `go vet`, `scripts/lint-httpapi.sh` (ARCH-01, ERR-01), `scripts/check-migrations-pair.sh` (DEP-03), `golangci-lint`, `go test -race`, build `cmd/*`.
**Рекомендуется локально:** `.golangci.yml`; `.pre-commit-config.yaml` (gofmt + prettier web).
**Только code review:** слои SQL; роли; idempotency; OpenAPI bodies; secrets в compose.
**Известные ограничения:** `X-Tenant-Id` в OpenAPI не реализован в handlers; `jobs.Registry` не shared между процессами API и отдельными воркерами без HTTP.
+219
View File
@@ -0,0 +1,219 @@
---
description: EvoBGP — BIRD2, BGP policy, IP/CIDR, маршрутизация, валидация конфигов
globs:
- internal/birdfmt/**
- internal/birddeploy/**
- internal/pipeline/**
- deploy/bird/**
- "**/testdata/scenarios/**"
alwaysApply: false
---
# Networking & Routing Rules
Общие правила: `.cursor/rules/engineering.mdc`. Пакет BIRD: `internal/birdfmt/doc.go`.
## Single source of truth
| Домен | SSOT |
|-------|------|
| Префиксы для анонса | Модули tenant → materialized snapshot → ревизия (`store`, `pipeline`, `CreateRenderRevision`) |
| Текст BIRD-фрагментов | `internal/birdfmt` — только генератор `evobgp_*.conf` |
| Каркас `bird.conf` | Operator skeleton + `include` (`StandardIncludeFragments`) |
| Параметры BIRD tenant | Global settings: `bird_router_id`, `bird_local_asn`, … (`docs/manual.md`) |
| BGP peers | `BGPPeer` + `ParsePeerNeighbor` |
**BIRD2 docs:** https://bird.network.cz/?get_doc
---
## A. BIRD2 Configuration Rules
**BIRD-01** | MUST | Имена фрагментов — константы `Fragment*` (`evobgp_prefixes_v4.conf`, …); include `bird.d/<name>`.
*Rationale:* совместимость deploy.
*Проверка:* `layout.go`; `go test ./internal/birdfmt/...`
**BIRD-02** | MUST | Порядок include: filters v4/v6 → BGP template → prefixes v4/v6 → peers (`StandardIncludeFragments`).
*Rationale:* фильтры до ссылок.
*Проверка:* scenario `standard_layout`; `bird -p`
**BIRD-03** | MUST | `bird.conf` (router id, device, direct, includes) отдельно от `bird.d/evobgp_*` (политика, префиксы).
*Rationale:* атомарная подмена фрагментов.
*Проверка:* review layout.
**BIRD-04** | MUST | Новая логика фильтрации — функция в `birdfmt` + unit-тест; NEVER дублировать filter-блок в два `.conf` вручную.
*Rationale:* DRY v4/v6.
*Проверка:* `filter_test.go`; review.
**BIRD-05** | MUST | Filter: префикс `evobgp_`, суффикс `_v4`/`_v6`; templates `bgp_template` / `bgp_template_v6`.
*Проверка:* `bgp.go`, `layout.go`.
**BIRD-06** | MUST | `router id` — валидный IPv4 dotted-quad (`RenderMainBirdConf`).
*Проверка:* `birdfmt` error; `bird -p`
**BIRD-07** | MUST | Communities — `RouteCommunityAttrs`; синтаксис `((a,b))` в `add()`, не `add(65000,1)`.
*Проверка:* `community_bird_test.go`
**BIRD-08** | SHOULD | Комментарии `#` в preamble; в фрагментах — revision/module id где уместно.
*Проверка:* review.
**BIRD-09** | NEVER | Ручное редактирование `evobgp_*.conf` на ноде без ревизии/бандла в control plane.
*Rationale:* drift.
*Проверка:* `evobgp-deploy` logs.
**BIRD-10** | MUST | Новый scenario: `testdata/scenarios/<name>/bird.conf` (+ `bird.d/`) проходит `bird -p`.
*Проверка:* CI `bird2`
**BIRD-11** | MUST | Только BIRD **2.x**, не 1.6.
*Проверка:* `bird --version`
**BIRD-12** | MUST | Apply: staging → `bird -p` → atomic swap → `birdc configure`; NEVER configure без parse check.
*Проверка:* `birdctl.go`; `birddeploy` review.
---
## B. Routing Policy Rules
**RTE-01** | MUST | BGP template: `import none` на каждый AFI, если задача явно не добавляет import policy.
*Rationale:* deny-by-default.
*Проверка:* `bgp.go`
**RTE-02** | MUST | Export filter завершается `reject;`.
*Проверка:* `filter.go`; `filter_export` scenario.
**RTE-03** | MUST | Анонс только префиксов materialized revision; NEVER static `route` вне pipeline/birdfmt без operator approval.
*Проверка:* store snapshot; review.
**RTE-04** | NEVER | Анонс `0.0.0.0/0`, `::/0`, RFC1918/ULA, loopback, link-local, multicast, reserved без documented exception.
*Проверка:* ingest validation; review.
**RTE-05** | NEVER | Префиксы чужих ASN без авторизации (IRR/RPKI — ops review).
*Проверка:* ops review.
**RTE-06** | MUST | Отдельные export filters IPv4 и IPv6.
*Проверка:* `RenderExportFilterIPv4/IPv6`
**RTE-07** | SHOULD | AS_PATH match — ASN из tenant modules; дедуп `filterUniqueASNs`.
*Проверка:* unit tests.
**RTE-08** | MUST | Peer import/export overrides — расширение `birdfmt`, не ad-hoc в operator `bird.conf`.
*Проверка:* review.
**RTE-09** | SHOULD | hold 90s / keepalive 30s — менять только с обоснованием в settings/docs.
*Проверка:* review.
**RTE-10** | MUST | После apply — post-check `birdc`/метрики (`mergeBirdPostApplyMeta`, `EVOBGP_BIRDC_SOCKET`).
*Проверка:* jobs meta; observability poller.
---
## C. IP Addressing Rules
**IP-01** | MUST | CIDR в Go — `netip.Prefix`; строка — `Masked().String()`.
*Проверка:* `pipeline/parse.go`
**IP-02** | MUST | IPv4/IPv6 — отдельные fragments/filters.
*Проверка:* `layout.go`
**IP-03** | MUST | BGP neighbor — `ParsePeerNeighbor`; `/32`/`/128` нормализуются в host addr.
*Проверка:* `peer_neighbor_test.go`
**IP-04** | NEVER | Hardcoded production CIDR вне testdata/fixtures.
*Проверка:* grep; review.
**IP-05** | MUST | Fixtures/docs — TEST-NET (RFC 5737, 3849): `203.0.113.0/24`, `198.51.100.0/24`, `2001:db8::/32`.
*Проверка:* scenarios.
**IP-06** | MUST | Domain→IP: host routes `/32`, `/128` via `ipToHostPrefix`; invalid отбрасывается.
*Проверка:* `collect_parallel.go`
**IP-07** | MUST | IP range entries — валидный CIDR на границе API/store.
*Проверка:* httpapi; review.
**IP-08** | SHOULD | Множество префиксов — без дубликатов; агрегация — policy decision.
*Проверка:* snapshot dedup.
**IP-09** | MUST | `router id`, `local as` — из tenant settings, не hardcode в callers `birdfmt`.
*Проверка:* `docs/manual.md`
**IP-10** | NEVER | Reserved blocks (`0.0.0.0/8`, `127.0.0.0/8`, `169.254.0.0/16`, `224.0.0.0/4`, IPv6 analogs) в materialized prefixes.
*Проверка:* review; future validator.
---
## D. Internet Compliance
**COMP-01** | SHOULD | Policy changes — сверка RFC 4271, 4760, 7454 (обзор).
*Проверка:* review.
**COMP-02** | MUST | No excessive deaggregation без need.
*Проверка:* review prefix list.
**COMP-03** | MUST | Standard community 0..65535; large — schema store.
*Проверка:* `community_bird_test.go`
**COMP-04** | NEVER | AS_PATH/MED manipulation без product spec.
*Проверка:* review output.
**COMP-05** | SHOULD | Revision/job meta: source, counts, samples (`revisionLogEntry`).
*Проверка:* jobs.
**COMP-06** | MUST | Node: `evobgp-node verify-bundle` перед apply.
*Проверка:* `docs/access.md`
**COMP-07** | SHOULD | `bird_local_asn` = `local as` в templates/peers.
*Проверка:* settings + birdfmt.
**COMP-08** | NEVER | Default route leak без named `default-originate` policy.
*Проверка:* export filter review.
---
## Validation & Observability
**VAL-01** | MUST | Перед деплоем: `bird -c <bird.conf> -p` (main + includes).
*Проверка:* CI `bird2`; `ParseCheck`
**VAL-02** | MUST | Preview ревизии read-only; apply — jobs/apply/bundle only.
*Проверка:* OpenAPI + httpapi.
**VAL-03** | MUST | Изменения birdfmt — `go test ./internal/birdfmt/...`.
*Проверка:* CI `go`
**VAL-04** | SHOULD | Dry-run apply где API поддерживает `dry_run`.
*Проверка:* routes.go.
**VAL-05** | MUST | `/metrics` Prometheus; `EVOBGP_BIRDC_SOCKET` — BGP session poller.
*Проверка:* `observability`
**VAL-06** | MUST | Логировать job_id, revision_id, speaker_id на apply/refresh/rollback.
*Проверка:* jobs API.
**VAL-07** | SHOULD | После apply — `birdc show protocols` / parsers в `birdfmt`.
*Проверка:* `protocols.go`
**VAL-08** | MUST | Мониторить drift: `last_applied_revision_id` vs published (`evobgp-deploy`).
*Проверка:* deploy worker.
---
## Documentation Sync
**DOC-SYNC-05** | MUST | BIRD — https://bird.network.cz/?get_doc
**DOC-SYNC-08** | MUST | BGP policy — RFC 4271, 4760, 7454 + BIRD docs + `birdfmt`
**DOC-SYNC-09** | MUST | CIDR — https://pkg.go.dev/net/netip ; примеры — RFC 5737, 3849
---
## Enforcement
```powershell
go test ./internal/birdfmt/... ./internal/pipeline/... -count=1
bird -c internal/birdfmt/testdata/scenarios/<name>/bird.conf -p
```
**PR checklist `birdfmt/**`, `pipeline/**`, `deploy/bird/**`:**
- [ ] filter + unit-test + `bird -p`
- [ ] нет дубли filter logic v4/v6 без общей Go-функции
- [ ] префиксы только store/revision path
- [ ] `import none` на templates сохранён
+109
View File
@@ -0,0 +1,109 @@
---
description: EvoBGP WebUI — shadcn-svelte, Svelte 5, слои ui/core|patterns|app
globs:
- web/**
alwaysApply: false
---
# Web UI — shadcn-svelte
**Источник правды:** https://shadcn-svelte.com/docs (не React shadcn/ui, не Legacy Docs).
Общие правила Go/API: `.cursor/rules/engineering.mdc`. Локальная карта: `web/README.md`.
## Слои UI
| Слой | Путь | Назначение |
|------|------|------------|
| Примитивы | `src/lib/ui/core/` | shadcn-svelte (только CLI `add`) |
| Паттерны | `src/lib/ui/patterns/` | FormField, AppDataTable, ConfirmDialog, EmptyState |
| App chrome | `src/lib/ui/app/` | Layout, PageHeader, `notify` |
| Legacy | `src/lib/components/ui/` | Re-export; **не добавлять новые файлы** |
Тема: `src/routes/layout.css`, `src/lib/ui/app/tokens.md`. CLI из `web/`: `npx shadcn-svelte@latest add <component> -y -o`.
---
## Правила
**WEB-01** | MUST | Перед новым UI — проверить https://shadcn-svelte.com/docs/components; использовать компонент, не HTML+CSS с нуля.
*Rationale:* Open Code + единый дизайн.
*Проверка:* review; нет голых `<button class=…>`.
**WEB-02** | MUST | Отсутствующий примитив — `npx shadcn-svelte@latest add <component> -y -o` → `src/lib/ui/core/`.
*Rationale:* Distribution через CLI и `components.json`.
*Проверка:* файлы только в `ui/core`.
**WEB-03** | NEVER | Альтернативные UI-kit'ы (Material, Vuetify, DaisyUI-only без shadcn-примитива).
*Проверка:* `package.json` review.
**WEB-04** | MUST | Комозиция по docs: все sub-компоненты (`DialogHeader`, `TableRow`, `Field`, …).
*Проверка:* сверка со страницей компонента в docs.
**WEB-05** | MUST | Формы — Formsnap + `sveltekit-superforms`; UI в `ui/patterns/form`, не ad-hoc валидация на странице.
*Проверка:* https://shadcn-svelte.com/docs/components/form
**WEB-06** | MUST | Таблицы — Data Table + `@tanstack/table-core`; на страницах — `AppDataTable` из patterns.
*Проверка:* https://shadcn-svelte.com/docs/components/data-table
**WEB-07** | MUST | Toast — Sonner через `notify` из `ui/app/toast.js`.
*Проверка:* https://shadcn-svelte.com/docs/components/sonner
**WEB-08** | MUST | Иконки — `@lucide/svelte` (`components.json` → `iconLibrary: lucide`).
*Проверка:* imports.
**WEB-09** | MUST | Цвета — CSS-переменные `layout.css` и токены `tokens.md`; не hex/rgb на страницах.
*Проверка:* grep `#[0-9a-f]{3,6}` в `routes/`.
**WEB-10** | SHOULD | Кастомизация — правка `ui/core` (Open Code), не `!important` поверх API.
*Проверка:* review.
**WEB-11** | MUST | `routes/**` — композиция `ui/core` + `ui/patterns` + `ui/app`; не копировать целые примитивы shadcn в route.
*Проверка:* review.
**WEB-12** | NEVER | Примеры React shadcn/ui или Svelte 4 Legacy без адаптации под https://shadcn-svelte.com/docs/migration/svelte-5
*Проверка:* `npm run check`.
**WEB-13** | MUST | Реактивность — Svelte 5 runes (`$state`, `$derived`, `$effect`); не `export let` для локального state страниц.
*Проверка:* `npm run check`; Svelte MCP.
**WEB-14** | SHOULD | Нетривиальный UI — прочитать страницу компонента (props, a11y).
*Проверка:* PR description.
**WEB-15** | MUST | Сомнения — https://shadcn-svelte.com/llms.txt , Svelte MCP, `npm run check`.
*Проверка:* локально.
**WEB-16** | MUST | Подтверждение удаления — `ConfirmDialog` из patterns, не `window.confirm`.
*Проверка:* review.
**WEB-17** | MUST | Пустые списки — `EmptyState` из patterns.
*Проверка:* review.
**WEB-18** | SHOULD | Повторяемая комбинация core (≥2 раза) — вынести в `ui/patterns/`.
*Проверка:* review.
---
## Documentation Sync (Web)
**DOC-SYNC-06** | MUST | UI — первично https://shadcn-svelte.com/docs; при конфликте с блогами/Stack Overflow побеждает официальная страница компонента.
**DOC-SYNC-07** | MUST | Перед `add` — сверить Installation/Theming с `web/components.json` и `src/routes/layout.css`.
Tailwind v4: https://shadcn-svelte.com/docs/migration/tailwind-v4
---
## Enforcement
```powershell
cd web
npm run check
npm run lint
```
**PR checklist `web/**`:**
- [ ] `ui/core` / `ui/patterns`, не дубли примитивов
- [ ] Новые примитивы через shadcn CLI
- [ ] Ссылка на docs компонента (если новый паттерн)
**CI:** job `web` рекомендован; пока обязательно локально.
+143
View File
@@ -0,0 +1,143 @@
---
name: commit-message
description: >-
ОБЯЗАТЕЛЬНО при commit, коммит, закоммить, commit message, conventional commit,
staged, semantic-release, «сгенерируй коммит», git commit, /commit-message: ПЕРВЫМ
делом Shell — scripts/commit/staged-context.ps1; затем Conventional Commit (заголовок EN, тело RU).
---
> **Кнопка ✨ Generate commit message в Source Control** не использует этот скилл и Rules.
> Эквивалент: Agent Chat → **`/commit-message`** или «сгенерируй коммит по staged».
> См. [docs/README.md](../../docs/README.md#сообщения-коммитов-cursor).
# Commit message (EvoBGP)
## Когда применять (сразу читать этот скилл)
Триггеры в промпте пользователя или в плане/AGENTS.md:
- `commit`, `коммит`, `закоммить`, `commit message`, `conventional commit`
- `git commit`, `staged`, `сообщение коммита`, `сгенерируй коммит`
- агент собирается выполнить `git commit` или предложить текст коммита
## Шаг 0 — ОБЯЗАТЕЛЬНО (до любого текста коммита)
**Первый вызов инструментов** в этой задаче — Shell из корня репозитория:
```powershell
powershell -NoProfile -File scripts/commit/staged-context.ps1
```
| Правило | Деталь |
|---------|--------|
| **MUST** | Запустить скрипт до генерации заголовка/тела коммита |
| **MUST NOT** | Строить сообщение только по `git diff --cached` / `git status` без скрипта |
| **MUST NOT** | Пропускать скрипт, даже если diff «и так понятен» |
| Exit `1` | Index пуст — сообщить пользователю, **не коммитить** |
| Exit `0` | Разобрать JSON stdout: `staged_count`, `groups[]` (`scope`, `files`, `stat`, `diff_excerpt`) |
Дополнительно: [.cursor/rules/conventional-commits.mdc](../../rules/conventional-commits.mdc).
## Слияние групп (одна фича)
Скрипт группирует **только по путям**. Перед split проверьте логическую связность:
**Объединить в один коммит**, если это одна задача:
- `docs/openapi.yaml` + `internal/httpapi/` (+ опционально `web/`) — один endpoint/контракт;
- `migrations/` + `internal/store/` / `repository/` — одна схема;
- правки теста рядом с кодом той же фичи.
При объединении: один scope (доминирующий пакет, часто `httpapi` или `openapi`), один type, одно тело RU.
**Auto-split** — если группы **не связаны** (например `web/` + `internal/birdfmt/` без общего смысла): **отдельный коммит на группу**, без вопроса пользователю.
## Порядок коммитов при split
1. `openapi`
2. `httpapi`, `store`, `jobs`
3. `pipeline`, `birdfmt`, `birddeploy`, `bundle`
4. `web`
5. `ci`, `deploy`, `db`
6. `docs`
7. `chore`, `cmd`
## Алгоритм auto-split (уровень файлов)
1. `git status` и `git diff --cached --stat` — зафиксировать полный список staged-файлов.
2. `git reset HEAD` — снять всё из index (working tree не трогать).
3. Для каждой (объединённой) группы по порядку выше:
- `git add -- <paths…>`
- Сгенерировать сообщение по правилу conventional-commits.
- Закоммитить (см. ниже).
4. После всех коммитов — `git status` для проверки.
**Ограничение:** частичный stage одного файла с разной семантикой — предупредить; split по hunk'ам не делать; предложить разнести правки по файлам.
## Создание коммита
Только если пользователь **явно** просил закоммитить. Иначе — только вывести готовые сообщения.
```powershell
git commit -m "$( @'
<type>(<scope>): <summary in English>
<Тело на русском.>
'@ )"
```
В PowerShell для многострочного тела используйте here-string как выше или `-m` для заголовка и `-m` для тела.
Перед коммитом: `git status`, `git diff --cached` для группы — убедиться, что stage соответствует сообщению.
## Генерация текста
По `groups[].diff_excerpt`, `stat`, `files`:
### Шаг A — semver (до выбора type)
| Вопрос | Если «да» → |
|--------|-------------|
| Пользователь получает **новую** возможность? | `feat` (minor) |
| Восстанавливается **ожидаемое** поведение / устранён баг? | `fix` (patch) |
| Только скорость, контракт тот же? | `perf` (patch) |
| Только структура кода/UI, поведение то же? | `refactor` (patch) |
**Follow-up:** правки сразу после `feat` в том же scope без новой возможности → **`fix`**, не `feat` (слова *enhance/improve/refactor* в задаче не делают commit `feat`).
**Запрещено** по умолчанию ставить `feat` для «большого diff» в `web/` — type по **semver impact**, не по объёму.
### Шаг B — type, scope, текст
- **type** — результат шага A, не «chore по умолчанию» и не `feat` из-за слова enhance.
- **scope** — из JSON группы или доминирующий при merge.
- **summary** — конкретный, английский, императив; для `fix` — что **починено** (`fix broken …`, `prevent … loop`).
- **body** — русский: что, зачем, edge cases, breaking impact.
## Вывод пользователю
Для **каждого** коммита:
### 1. Готовое сообщение
```
<type>(<scope>): <summary>
<тело>
```
### 2. Пояснение (RU)
- **Semver impact:** `minor` | `patch` | `none` | `major` — и почему не другой type.
- Риски и impact.
- Split: сколько коммитов и почему.
## Не смешивать
Один commit message — одна primary intent. Не объединять несвязанный `fix` и `feat` в один заголовок.
## Ссылки
- Правило: `.cursor/rules/conventional-commits.mdc`
- Скрипт: `scripts/commit/staged-context.ps1`
- Просмотр групп: `powershell -File scripts/commit/staged-context.ps1 | ConvertFrom-Json`
+29 -51
View File
@@ -1,75 +1,53 @@
# Gitea Actions # Gitea Actions
В репозитории включён workflow [workflows/ci.yaml](workflows/ci.yaml). Workflow: [workflows/ci.yaml](workflows/ci.yaml).
Job **changes** вычисляет два флага: | Job | PR | push в main |
|-----|-----|-------------|
| changes, openapi, web, go, bird2 | quality gates | quality gates |
| commitlint | да | — |
| **release** | — | semantic-release + docker push (один run) |
- **`openapi`** — в диффе есть `docs/openapi.yaml` или `redocly.yaml` → запускается **openapi** (Redocly lint). Подробнее: [docs/releasing.md](../docs/releasing.md).
- **`code`** — в диффе есть **хотя бы один файл вне «только документация»** → запускаются **go**, **bird2** и при push в main/master **docker-images**.
Если изменены **только** файлы из списка ниже, **`code=false`**: Go-тесты, bird2 и сборка образов **не** запускаются; при необходимости всё ещё выполняется **openapi**, если трогали OpenAPI или `redocly.yaml`. ## CI (quality gates)
**Список путей «только документация» (без полной сборки):** всё под `docs/`, корневой `README.md`, `web/README.md`, `.gitea/README.md`, `redocly.yaml`. Job **changes** вычисляет флаги по путям в diff. Полный прогон (все узлы openapi / web / go / bird2 в графе): `.gitea/workflows/*`, `scripts/*`, `.golangci.yml`, `.pre-commit-config.yaml`, корневой `package.json` / `.releaserc.json`. Отдельно: `migrations/*`, `docs/openapi.yaml` → `go` / `openapi` и т.д. (см. `ci.yaml`).
Любой другой путь (в т.ч. `.gitea/workflows/ci.yaml`, `go.mod`, `web/package.json`, `deploy/**`, `**/*.go`) включает полный пайплайн. На **pull request** — **commitlint** (Conventional Commits).
Требования к runner: Runner: `ubuntu-latest`, **bird2** из apt, Docker для job **release**.
- Метка `ubuntu-latest` (или измените `runs-on` под свои labels). ## Release (job в ci.yaml)
- Job `bird2` ставит пакет **bird2** из репозитория **Ubuntu** на runner (как образ `evobgp-bird2` на `ubuntu:noble`) и вызывает `bird -c … -p` по файлам в checkout (без вложенного Docker: иначе bind-mount `/workspace/...` часто не виден демону Docker на хосте).
- Job **docker-images** требует **Docker** и (для push) доступ к Container Registry Gitea с runner’а.
## Сборка и публикация образов (`docker-images`) После успешных quality gates на **push в main** job **release**:
Запускается только при **push** в ветки `main` или `master` (после успешного job `go`). Собираются образы из `deploy/docker/*` (все бинарники через `gobinary`, плюс `evobgp-web`, `evobgp-agent`, `evobgp-bird2`). 1. `npx semantic-release` — тег `vX.Y.Z` на **текущий commit** (без дополнительного commit в main).
2. Gitea Release + `CHANGELOG.md` как attachment (не в git).
3. `docker buildx bake default --push` с `VERSION=X.Y.Z` — в том же job.
### Секреты репозитория (для push в реестр) Если releasable-коммитов нет — semantic-release no-op, образы не публикуются.
Хост Container Registry в workflow **зашит**: **`git.shts.su`** (`docker/login-action` и теги образов). ### Секреты
В настройках репозитория Gitea: **Settings → Actions → Secrets** (или аналог): **`ACTIONS_PAT`**: push tags, releases, Container Registry. Fallback: **`gitea.token`**.
| Секрет | Назначение | ### Теги образов
|--------|------------|
| `ACTIONS_PAT` | (Опционально) PAT с правом **записи пакетов**. Если не задан, для `docker login` используется **`gitea.token`** (нужны права у workflow на пакеты). |
Вход в реестр в workflow:
- **registry:** `git.shts.su`;
- **username:** `${{ gitea.actor }}`;
- **password:** `${{ secrets.ACTIONS_PAT || gitea.token }}`.
### Почему шаг «Log in to Gitea Registry» мог быть skipped
Раньше в `if:` стояло **`secrets.ACTIONS_PAT != ''`**. В **Gitea act_runner** такая проверка секрета в условии шага часто **всегда ложна**, из‑за этого логин **всегда skipped**. Сейчас условие только `docker_push == 'true'` (всегда при нормальном шаге **Prepare**), пароль: **`secrets.ACTIONS_PAT || gitea.token`**.
Если после правки `docker login` падает с 401/403 — выдайте job-токену или PAT право **записи в Container Registry** (в настройках Gitea / организации / репозитория, в зависимости от версии).
Если **`gitea.token`** в выражениях недоступен, в [ci.yaml](workflows/ci.yaml) замените пароль на `${{ secrets.ACTIONS_PAT || github.token }}`.
### Имена образов и теги
Формат:
```text ```text
git.shts.su/<owner_lowercase>/<имя>:<тег> git.shts.su/<owner>/<имя>:latest
git.shts.su/<owner>/<имя>:v1.2.3
git.shts.su/<owner>/<имя>:1.2.3
git.shts.su/<owner>/<имя>:<short-sha>
git.shts.su/<owner>/<имя>:sha-<full-sha>
``` ```
где `<имя>` — одно из: `evobgp-api`, `evobgp-all`, `evobgp-scheduler`, `evobgp-ingest`, `evobgp-render`, `evobgp-deploy`, `evobgp-node`, `evobgp-web`, `evobgp-web-all` (UI с прокси на `evobgp-all` для профиля Compose `microvps-full`), `evobgp-agent`, `evobgp-bird2`. Имена образов: `evobgp-api`, `evobgp-all`, `evobgp-scheduler`, `evobgp-ingest`, `evobgp-render`, `evobgp-deploy`, `evobgp-node`, `evobgp-web`, `evobgp-web-all`, `evobgp-agent`, `evobgp-bird2`.
Теги: `latest`, короткий SHA (7 символов), `sha-<полный_commit_sha>`. Пример:
Пример pull:
```bash ```bash
docker pull git.shts.su/myuser/evobgp-api:latest docker pull git.shts.su/myuser/evobgp-api:1.2.3
``` ```
Полная таблица образов, тегов, ссылок на страницы пакетов в Gitea и минимальный `docker run` для сервера без сборки — в [docs/quickstart.md](../docs/quickstart.md) (раздел «Готовые образы без сборки»). См. [deploy/docker/README.md](../deploy/docker/README.md), [docs/quickstart.md](../docs/quickstart.md).
Убедитесь, что в **Gitea** включены пакеты (Container Registry) и у токена есть права на создание/обновление образов. Другой хост реестра — правьте **`registry:`** и префикс **`IMG=`** в [ci.yaml](workflows/ci.yaml).
### Примечания
- Сборка с платформой `linux/amd64`. При необходимости добавьте `linux/arm64` и multi-arch в `docker buildx build`.
- Для PR job **docker-images** не выполняется (только push в main/master), чтобы не публиковать образы из чужих форков без явной политики.
+164 -128
View File
@@ -8,10 +8,9 @@ on:
jobs: jobs:
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Гранулярная детекция изменений по модулям. # Детекция изменений по модулям (флаги → downstream-джобы в графе CI).
# Каждый флаг соответствует группе файлов; downstream-джобы запускаются # Полный прогон (все флаги true): .gitea/workflows/*, scripts/*, .golangci.yml,
# только когда их группа затронута. Изменение CI-конфигурации (.gitea/workflows/*) # .pre-commit-config.yaml — чтобы при правках CI/CD пересобирались все узлы.
# поднимает все флаги, чтобы гарантировать полный прогон.
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
changes: changes:
runs-on: ubuntu-latest runs-on: ubuntu-latest
@@ -40,6 +39,23 @@ jobs:
docker_web=false docker_web=false
docker_bird=false docker_bird=false
# Все флаги true → openapi, web, go, bird2 (и release на main) в графе CI.
set_all_flags_true() {
openapi=true
go=true
web=true
bird_conf=true
docker_go=true
docker_web=true
docker_bird=true
}
write_outputs() {
for v in openapi go web bird_conf docker_go docker_web docker_bird; do
eval "echo \"\$v=\$$v\"" >> "$GITHUB_OUTPUT"
done
}
if [ "${{ github.event_name }}" = "pull_request" ]; then if [ "${{ github.event_name }}" = "pull_request" ]; then
base="${{ github.event.pull_request.base.sha }}" base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}" head="${{ github.event.pull_request.head.sha }}"
@@ -52,61 +68,84 @@ jobs:
elif git rev-parse --verify HEAD~1 >/dev/null 2>&1; then elif git rev-parse --verify HEAD~1 >/dev/null 2>&1; then
FILES="$(git diff --name-only HEAD~1 HEAD)" FILES="$(git diff --name-only HEAD~1 HEAD)"
else else
openapi=true; go=true; web=true; bird_conf=true set_all_flags_true
docker_go=true; docker_web=true; docker_bird=true write_outputs
for v in openapi go web bird_conf docker_go docker_web docker_bird; do echo "No parent commit — full pipeline (all modules)"
echo "$v=true" >> "$GITHUB_OUTPUT"
done
echo "No parent commit — full pipeline"
exit 0 exit 0
fi fi
fi fi
if [ -z "$(printf '%s' "$FILES" | tr -d '[:space:]')" ]; then if [ -z "$(printf '%s' "$FILES" | tr -d '[:space:]')" ]; then
go=true; web=true set_all_flags_true
for v in openapi go web bird_conf docker_go docker_web docker_bird; do write_outputs
eval "echo \"\$v=\$$v\"" >> "$GITHUB_OUTPUT" echo "Empty diff — full pipeline fallback"
done
echo "Empty diff — safe fallback: go=true web=true"
exit 0 exit 0
fi fi
ci_changed=false full_pipeline=false
while IFS= read -r f || [ -n "${f:-}" ]; do while IFS= read -r f || [ -n "${f:-}" ]; do
[ -z "${f:-}" ] && continue [ -z "${f:-}" ] && continue
case "$f" in case "$f" in
.gitea/workflows/*) ci_changed=true ;; # CI/CD инфраструктура — все узлы quality gates
docs/openapi.yaml|redocly.yaml) openapi=true ;; .gitea/workflows/*|.golangci.yml|.pre-commit-config.yaml|scripts/*)
web/README.md) ;; # doc-only full_pipeline=true
web/*) web=true ;; ;;
deploy/bird/*) bird_conf=true ;; docs/openapi.yaml|redocly.yaml)
deploy/docker/bird/*) docker_bird=true; docker_go=true ;; openapi=true
deploy/docker/gobinary/*) docker_go=true ;; ;;
deploy/docker/docker-bake.hcl) docker_go=true; docker_web=true ;; docs/api.md|docs/access.md)
deploy/docker/evobgp-agent/*) docker_go=true ;; openapi=true
deploy/docker/evobgp-web/*) docker_web=true ;; go=true
deploy/docker/bird2/*) docker_bird=true ;; ;;
go.mod|go.sum|go.work) go=true ;; web/README.md|web/components.json)
*.go) go=true ;; ;;
cmd/*|internal/*) go=true ;; web/*)
web=true
;;
deploy/bird/*)
bird_conf=true
go=true
;;
deploy/compose/*|deploy/docker/*)
docker_go=true
docker_web=true
docker_bird=true
go=true
;;
go.mod|go.sum|go.work)
go=true
;;
migrations/*)
go=true
;;
cmd/*|internal/*|*.go)
go=true
bird_conf=true
;;
docs/*)
go=true
;;
package.json|package-lock.json|.releaserc.json)
full_pipeline=true
;;
*)
go=true
;;
esac esac
done <<< "$FILES" done <<< "$FILES"
if $ci_changed; then if $full_pipeline; then
go=true; web=true; bird_conf=true set_all_flags_true
docker_go=true; docker_web=true; docker_bird=true
fi fi
for v in openapi go web bird_conf docker_go docker_web docker_bird; do write_outputs
eval "echo \"\$v=\$$v\"" >> "$GITHUB_OUTPUT"
done
echo "Changed files (first 30):" echo "Changed files (first 30):"
printf '%s\n' "$FILES" | head -n 30 printf '%s\n' "$FILES" | head -n 30
echo "--- flags ---" echo "--- flags ---"
echo "openapi=$openapi go=$go web=$web bird_conf=$bird_conf" echo "openapi=$openapi go=$go web=$web bird_conf=$bird_conf"
echo "docker_go=$docker_go docker_web=$docker_web docker_bird=$docker_bird ci=$ci_changed" echo "docker_go=$docker_go docker_web=$docker_web docker_bird=$docker_bird full_pipeline=$full_pipeline"
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
openapi: openapi:
@@ -121,6 +160,26 @@ jobs:
- name: Lint OpenAPI (Redocly) - name: Lint OpenAPI (Redocly)
run: npx --yes @redocly/cli@1 lint docs/openapi.yaml run: npx --yes @redocly/cli@1 lint docs/openapi.yaml
# ---------------------------------------------------------------------------
web:
needs: [changes]
if: needs.changes.outputs.web == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: "20"
cache: npm
cache-dependency-path: web/package-lock.json
- name: npm ci, check, lint
run: |
set -euxo pipefail
cd web
npm ci
npm run check
npm run lint
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
go: go:
needs: [changes] needs: [changes]
@@ -135,6 +194,16 @@ jobs:
cache-dependency-path: go.sum cache-dependency-path: go.sum
- name: Vet - name: Vet
run: go vet ./... run: go vet ./...
- name: Lint httpapi (ERR-01 / ARCH-01)
run: sh scripts/lint-httpapi.sh
- name: Check migration pairs (DEP-03)
run: sh scripts/check-migrations-pair.sh
# go.mod: go 1.24 — бинарник golangci-lint < v1.64.2 (сборка на Go 1.23) не запускается.
- name: golangci-lint
uses: golangci/golangci-lint-action@v6
with:
version: v1.64.8
install-mode: goinstall
- name: Test - name: Test
run: go test ./... -race -count=1 run: go test ./... -race -count=1
- name: Build all commands - name: Build all commands
@@ -186,88 +255,99 @@ jobs:
done done
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Docker: все Go-образы одним buildx bake (один go mod download, одна компиляция cmd/*, commitlint:
# birdc собирается один раз для api/all). Registry cache: evobgp-buildcache:go-buildcache. if: github.event_name == 'pull_request'
# ---------------------------------------------------------------------------
docker-go:
needs: [changes, go]
if: >-
always() &&
needs.changes.result == 'success' &&
needs.go.result != 'failure' &&
github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') &&
(needs.changes.outputs.go == 'true' || needs.changes.outputs.docker_go == 'true')
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Prepare image metadata
id: meta
run: |
set -euo pipefail
owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT"
short_sha="$(echo '${{ github.sha }}' | cut -c1-7)"
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
- name: Log in to Gitea Registry
uses: docker/login-action@v3
with: with:
registry: git.shts.su fetch-depth: 0
username: ${{ gitea.actor }} - uses: actions/setup-node@v4
password: ${{ secrets.ACTIONS_PAT || gitea.token }} with:
- name: Build and push Go images (bake) node-version: "22"
env: cache: npm
REGISTRY: git.shts.su/${{ steps.meta.outputs.owner_lc }} cache-dependency-path: package-lock.json
IMAGE_TAG: latest - name: Lint commit messages
SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
SHA_FULL: ${{ github.sha }}
CACHE_REF_GO: git.shts.su/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:go-buildcache
CACHE_REF_WEB: ""
BUILDX_BAKE_ENTITLEMENTS_FS: "0"
run: | run: |
set -euxo pipefail set -euxo pipefail
cd "${{ github.workspace }}/deploy/docker" npm ci
sh write-bake-override.sh npx commitlint --from "${{ github.event.pull_request.base.sha }}" --to "${{ github.event.pull_request.head.sha }}"
docker buildx bake --allow=fs.read="${{ github.workspace }}" \
-f docker-bake.hcl -f docker-bake.override.hcl go-images --push
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
# Docker: Web (evobgp-web, evobgp-web-all) — один npm ci (кэш) + два nginx-тега. # Один push в main: semantic-release (тег на текущий commit, без доп. commit) + docker push.
# --------------------------------------------------------------------------- # ---------------------------------------------------------------------------
docker-web: release:
needs: [changes] needs: [changes, openapi, web, go, bird2]
if: >- if: >-
always() &&
github.event_name == 'push' && github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') && (github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') &&
(needs.changes.outputs.web == 'true' || needs.changes.outputs.docker_web == 'true') needs.changes.result == 'success' &&
(needs.openapi.result == 'success' || needs.openapi.result == 'skipped') &&
(needs.web.result == 'success' || needs.web.result == 'skipped') &&
(needs.go.result == 'success' || needs.go.result == 'skipped') &&
(needs.bird2.result == 'success' || needs.bird2.result == 'skipped')
runs-on: ubuntu-latest runs-on: ubuntu-latest
steps: steps:
- uses: actions/checkout@v4 - uses: actions/checkout@v4
with:
fetch-depth: 0
token: ${{ secrets.ACTIONS_PAT || gitea.token }}
persist-credentials: true
- uses: actions/setup-node@v4
with:
node-version: "22"
cache: npm
cache-dependency-path: package-lock.json
- name: Install release tooling
run: npm ci
- name: Semantic release
run: npx semantic-release
env:
GITEA_URL: https://git.shts.su
GITEA_TOKEN: ${{ secrets.ACTIONS_PAT || gitea.token }}
- name: Detect new release
id: rel
run: |
set -euo pipefail
if [ -f .release-version ]; then
echo "version=$(tr -d '[:space:]' < .release-version)" >> "$GITHUB_OUTPUT"
echo "released=true" >> "$GITHUB_OUTPUT"
else
echo "released=false" >> "$GITHUB_OUTPUT"
echo "No releasable commits — skipping image publish"
fi
- name: Set up Docker Buildx - name: Set up Docker Buildx
if: steps.rel.outputs.released == 'true'
uses: docker/setup-buildx-action@v3 uses: docker/setup-buildx-action@v3
- name: Prepare image metadata - name: Prepare image metadata
if: steps.rel.outputs.released == 'true'
id: meta id: meta
run: | run: |
set -euo pipefail set -euo pipefail
echo "version=${{ steps.rel.outputs.version }}" >> "$GITHUB_OUTPUT"
owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')" owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT" echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT"
short_sha="$(echo '${{ github.sha }}' | cut -c1-7)" short_sha="$(echo '${{ github.sha }}' | cut -c1-7)"
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT" echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
echo "build_time=$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$GITHUB_OUTPUT"
- name: Log in to Gitea Registry - name: Log in to Gitea Registry
if: steps.rel.outputs.released == 'true'
uses: docker/login-action@v3 uses: docker/login-action@v3
with: with:
registry: git.shts.su registry: git.shts.su
username: ${{ gitea.actor }} username: ${{ gitea.actor }}
password: ${{ secrets.ACTIONS_PAT || gitea.token }} password: ${{ secrets.ACTIONS_PAT || gitea.token }}
- name: Build and push Web images (bake) - name: Build and push images (bake)
if: steps.rel.outputs.released == 'true'
env: env:
REGISTRY: git.shts.su/${{ steps.meta.outputs.owner_lc }} REGISTRY: git.shts.su/${{ steps.meta.outputs.owner_lc }}
IMAGE_TAG: latest IMAGE_TAG: latest
VERSION: ${{ steps.meta.outputs.version }}
SHORT_SHA: ${{ steps.meta.outputs.short_sha }} SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
SHA_FULL: ${{ github.sha }} SHA_FULL: ${{ github.sha }}
CACHE_REF_GO: "" BUILD_TIME: ${{ steps.meta.outputs.build_time }}
CACHE_REF_GO: git.shts.su/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:go-buildcache
CACHE_REF_WEB: git.shts.su/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:web-buildcache CACHE_REF_WEB: git.shts.su/${{ steps.meta.outputs.owner_lc }}/evobgp-buildcache:web-buildcache
BUILDX_BAKE_ENTITLEMENTS_FS: "0" BUILDX_BAKE_ENTITLEMENTS_FS: "0"
run: | run: |
@@ -275,49 +355,5 @@ jobs:
cd "${{ github.workspace }}/deploy/docker" cd "${{ github.workspace }}/deploy/docker"
sh write-bake-override.sh sh write-bake-override.sh
docker buildx bake --allow=fs.read="${{ github.workspace }}" \ docker buildx bake --allow=fs.read="${{ github.workspace }}" \
-f docker-bake.hcl -f docker-bake.override.hcl web-images --push -f docker-bake.hcl -f docker-bake.override.hcl default --push
# ---------------------------------------------------------------------------
# Docker: BIRD2 (evobgp-bird2).
# Собирается только при изменении deploy/bird/ или Dockerfile bird2.
# ---------------------------------------------------------------------------
docker-bird:
needs: [changes]
if: >-
github.event_name == 'push' &&
(github.ref == 'refs/heads/main' || github.ref == 'refs/heads/master') &&
(needs.changes.outputs.bird_conf == 'true' || needs.changes.outputs.docker_bird == 'true')
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v3
- name: Prepare image metadata
id: meta
run: |
set -euo pipefail
owner_lc="$(echo '${{ github.repository_owner }}' | tr '[:upper:]' '[:lower:]')"
echo "owner_lc=$owner_lc" >> "$GITHUB_OUTPUT"
short_sha="$(echo '${{ github.sha }}' | cut -c1-7)"
echo "short_sha=$short_sha" >> "$GITHUB_OUTPUT"
- name: Log in to Gitea Registry
uses: docker/login-action@v3
with:
registry: git.shts.su
username: ${{ gitea.actor }}
password: ${{ secrets.ACTIONS_PAT || gitea.token }}
- name: Build and push evobgp-bird2 (bake)
env:
REGISTRY: git.shts.su/${{ steps.meta.outputs.owner_lc }}
IMAGE_TAG: latest
SHORT_SHA: ${{ steps.meta.outputs.short_sha }}
SHA_FULL: ${{ github.sha }}
CACHE_REF_GO: ""
CACHE_REF_WEB: ""
BUILDX_BAKE_ENTITLEMENTS_FS: "0"
run: |
set -euxo pipefail
cd "${{ github.workspace }}/deploy/docker"
sh write-bake-override.sh
docker buildx bake --allow=fs.read="${{ github.workspace }}" \
-f docker-bake.hcl -f docker-bake.override.hcl evobgp-bird2 --push
+18
View File
@@ -0,0 +1,18 @@
# Root npm (semantic-release, commitlint) — npm ci in CI, never commit deps
node_modules/
# Generated by deploy/docker/write-bake-override.sh (CI/local bake)
deploy/docker/docker-bake.override.hcl
# CI-only markers (not committed)
.release-version
CHANGELOG.md
# OS
.DS_Store
Thumbs.db
# Env / secrets (local only)
.env
.env.*
!.env.example
+18
View File
@@ -0,0 +1,18 @@
run:
timeout: 5m
linters:
disable-all: true
enable:
- gofmt
- govet
- errcheck
issues:
exclude-use-default: false
max-issues-per-linter: 50
max-same-issues: 20
linters-settings:
gofmt:
simplify: true
+19
View File
@@ -0,0 +1,19 @@
# Local hooks (optional): install with `pre-commit install`
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v5.0.0
hooks:
- id: trailing-whitespace
- id: end-of-file-fixer
- repo: https://github.com/dnephin/pre-commit-golang
rev: v0.5.1
hooks:
- id: go-fmt
- repo: local
hooks:
- id: prettier-web
name: prettier (web)
entry: bash -c 'cd web && npx prettier --check .'
language: system
files: ^web/
pass_filenames: false
+50
View File
@@ -0,0 +1,50 @@
{
"branches": ["main", "master"],
"tagFormat": "v${version}",
"plugins": [
[
"@semantic-release/commit-analyzer",
{
"preset": "conventionalcommits",
"releaseRules": [
{ "type": "feat", "release": "minor" },
{ "type": "fix", "release": "patch" },
{ "type": "perf", "release": "patch" },
{ "type": "ci", "release": "patch" },
{ "type": "refactor", "release": "patch" },
{ "breaking": true, "release": "major" }
]
}
],
[
"@semantic-release/release-notes-generator",
{
"preset": "conventionalcommits"
}
],
[
"@semantic-release/changelog",
{
"changelogFile": "CHANGELOG.md"
}
],
[
"@semantic-release/exec",
{
"successCmd": "echo ${nextRelease.version} > .release-version"
}
],
[
"@markwylde/semantic-release-gitea",
{
"giteaUrl": "https://git.shts.su",
"assets": [
{
"path": "CHANGELOG.md",
"label": "Changelog"
}
]
}
]
]
}
+12
View File
@@ -4,6 +4,7 @@
## С чего начать (минимум чтения) ## С чего начать (минимум чтения)
0. **Инженерные правила** — при изменении кода следовать [.cursor/rules/engineering.mdc](.cursor/rules/engineering.mdc); для `web/` — [.cursor/rules/web-shadcn.mdc](.cursor/rules/web-shadcn.mdc); для `birdfmt` / `pipeline` / BIRD — [.cursor/rules/networking-bird.mdc](.cursor/rules/networking-bird.mdc).
1. **[docs/README.md](docs/README.md)** — оглавление и роли читателя. 1. **[docs/README.md](docs/README.md)** — оглавление и роли читателя.
2. **[docs/architecture.md](docs/architecture.md)** — компоненты `cmd/`, карта `internal/`, потоки данных (одного этого файла обычно достаточно для ориентации). 2. **[docs/architecture.md](docs/architecture.md)** — компоненты `cmd/`, карта `internal/`, потоки данных (одного этого файла обычно достаточно для ориентации).
3. Задача-специфично: [docs/api.md](docs/api.md), [docs/access.md](docs/access.md), [web/README.md](web/README.md) — только если меняете API, доступ или фронт. 3. Задача-специфично: [docs/api.md](docs/api.md), [docs/access.md](docs/access.md), [web/README.md](web/README.md) — только если меняете API, доступ или фронт.
@@ -35,6 +36,17 @@
- **Повторное использование:** если [docs/architecture.md](docs/architecture.md) уже описывает поток — не пересказывайте его длинно; укажите документ и конкретный подпункт задачи. - **Повторное использование:** если [docs/architecture.md](docs/architecture.md) уже описывает поток — не пересказывайте его длинно; укажите документ и конкретный подпункт задачи.
- **Длинные планы:** `.cursor/plans/*.plan.md` — для истории решений; для навигации пользователю достаточно `docs/`; не читайте план целиком без причины. - **Длинные планы:** `.cursor/plans/*.plan.md` — для истории решений; для навигации пользователю достаточно `docs/`; не читайте план целиком без причины.
## Коммиты (Conventional Commits)
Если пользователь просит **коммит**, **commit message**, **закоммить**, **git commit**, **`/commit-message`** или это следует из плана — **сразу**:
1. Shell: `powershell -NoProfile -File scripts/commit/staged-context.ps1` (первый вызов, до текста коммита).
2. Скилл [.cursor/skills/commit-message/SKILL.md](.cursor/skills/commit-message/SKILL.md) и правило [.cursor/rules/conventional-commits.mdc](.cursor/rules/conventional-commits.mdc).
Без вывода скрипта (exit 0) **не** придумывать сообщение коммита. Заголовок — EN, тело — RU; несвязанные области — auto-split (скилл).
**Кнопка ✨ Generate commit message в Source Control** skill/rule **не** использует. Для сообщений по правилам EvoBGP — Agent Chat → **`/commit-message`** (см. [.cursor/commands/commit-message.md](.cursor/commands/commit-message.md)).
## Команды и среда ## Команды и среда
- Консоль пользователя: **PowerShell**; пути в стиле `deploy\compose`. - Консоль пользователя: **PowerShell**; пути в стиле `deploy\compose`.
+150
View File
@@ -0,0 +1,150 @@
## 1.0.0 (2026-05-19)
* feat: add @tanstack/table-core dependency and integrate module name mapping in operations page. Upda ([b140dcb](http://192.168.100.56:3000/denozord/EvoBGP/commit/b140dcb))
* feat: add @types/node and undici-types to devDependencies in package.json and package-lock.json for ([d92a06c](http://192.168.100.56:3000/denozord/EvoBGP/commit/d92a06c))
* feat: add aggregated router-lists catalog endpoint and update module listing filters ([c811c43](http://192.168.100.56:3000/denozord/EvoBGP/commit/c811c43))
* feat: add auto-updater service for Docker stack with configurable settings ([ae9fbe3](http://192.168.100.56:3000/denozord/EvoBGP/commit/ae9fbe3))
* feat: add bird configuration preview functionality with tabbed interface for fragments and prefixes. ([1969a00](http://192.168.100.56:3000/denozord/EvoBGP/commit/1969a00))
* feat: add BIRD status endpoint and integrate birdc checks into job processing. Enhance UI to display ([94a4c6a](http://192.168.100.56:3000/denozord/EvoBGP/commit/94a4c6a))
* feat: add CSV import and export functionality for module entries. Implement endpoints for exporting ([bae2d68](http://192.168.100.56:3000/denozord/EvoBGP/commit/bae2d68))
* feat: add debug logging and optimize database queries in Postgres repository ([6d0e214](http://192.168.100.56:3000/denozord/EvoBGP/commit/6d0e214))
* feat: add diagnostic log endpoint for revisions and implement download functionality ([2260ecf](http://192.168.100.56:3000/denozord/EvoBGP/commit/2260ecf))
* feat: add enabled state to BGP peer types and UI. Implement peer enabling toggle in the network page ([80965ad](http://192.168.100.56:3000/denozord/EvoBGP/commit/80965ad))
* feat: add last refreshed timestamp to modules and CDN sources ([1e13ef9](http://192.168.100.56:3000/denozord/EvoBGP/commit/1e13ef9))
* feat: add live BGP protocol state tracking to peer listing ([8a50ba4](http://192.168.100.56:3000/denozord/EvoBGP/commit/8a50ba4))
* feat: add optional name field to BGP peer creation and update UI ([1a61ae2](http://192.168.100.56:3000/denozord/EvoBGP/commit/1a61ae2))
* feat: add optional name field to DoH profile schema and types ([6bee5ad](http://192.168.100.56:3000/denozord/EvoBGP/commit/6bee5ad))
* feat: add smart aggregation for IPv4 CIDR prefixes in refresh pipeline. Implement functionality to a ([0428b83](http://192.168.100.56:3000/denozord/EvoBGP/commit/0428b83))
* feat: add strict bind configuration to BGP templates. Enhance BGP template rendering by including 's ([07f2ce5](http://192.168.100.56:3000/denozord/EvoBGP/commit/07f2ce5))
* feat: add tenant refresh functionality to HTTP API and job processing ([7a2b015](http://192.168.100.56:3000/denozord/EvoBGP/commit/7a2b015))
* feat: auto-refresh non-IP module updates and add full project manual ([1e77e1a](http://192.168.100.56:3000/denozord/EvoBGP/commit/1e77e1a))
* feat: enhance API client with pagination and new data fetching functions. Implement `apiPageAll` for ([9f0b128](http://192.168.100.56:3000/denozord/EvoBGP/commit/9f0b128))
* feat: enhance BGP template and peer configuration handling. Introduce policies_json parameter for lo ([be8e7b5](http://192.168.100.56:3000/denozord/EvoBGP/commit/be8e7b5))
* feat: enhance CDN source management with last refreshed timestamp ([4776434](http://192.168.100.56:3000/denozord/EvoBGP/commit/4776434))
* feat: enhance community attribute handling in BIRD format. Add validation for community route bodies ([b541794](http://192.168.100.56:3000/denozord/EvoBGP/commit/b541794))
* feat: enhance community label handling and improve UI localization across operations pages. Introduc ([53fd527](http://192.168.100.56:3000/denozord/EvoBGP/commit/53fd527))
* feat: enhance deployment configuration for secure Web UI with Traefik integration ([4acd685](http://192.168.100.56:3000/denozord/EvoBGP/commit/4acd685))
* feat: enhance DoH profile management and resolver policy in modules ([34ecc5c](http://192.168.100.56:3000/denozord/EvoBGP/commit/34ecc5c))
* feat: enhance EvoBGP with new command-line options for the evobgp-agent, including a watch command f ([6a55f72](http://192.168.100.56:3000/denozord/EvoBGP/commit/6a55f72))
* feat: enhance evobgp with new command-line tools for bundle management, including pull, verify, and ([bf52b21](http://192.168.100.56:3000/denozord/EvoBGP/commit/bf52b21))
* feat: enhance job detail handling in UI with caching and loading states. Introduce SvelteMap for job ([05bc5c4](http://192.168.100.56:3000/denozord/EvoBGP/commit/05bc5c4))
* feat: enhance job detail UI with improved overflow handling and layout adjustments. Update card comp ([67ce135](http://192.168.100.56:3000/denozord/EvoBGP/commit/67ce135))
* feat: enhance job filtering and display functionality in operations page. Introduce job search and f ([e1c4cbc](http://192.168.100.56:3000/denozord/EvoBGP/commit/e1c4cbc))
* feat: enhance job processing with detailed logging and UI improvements. Introduce revision log entri ([c48cc4e](http://192.168.100.56:3000/denozord/EvoBGP/commit/c48cc4e))
* feat: enhance monitoring page with detailed status tracking and error handling ([458a885](http://192.168.100.56:3000/denozord/EvoBGP/commit/458a885))
* feat: enhance observability with prefix aggregation metrics ([07b97bd](http://192.168.100.56:3000/denozord/EvoBGP/commit/07b97bd))
* feat: enhance peer neighbor handling by introducing NormalizePeerNeighborString function. Update Cre ([80f3a8d](http://192.168.100.56:3000/denozord/EvoBGP/commit/80f3a8d))
* feat: enhance RefreshModule to aggregate prefixes from all enabled tenant modules. Update documentat ([431c6a9](http://192.168.100.56:3000/denozord/EvoBGP/commit/431c6a9))
* feat: enhance RefreshModule to support legacy ASN resolution and improve prefix handling. Introduce ([e1aaa18](http://192.168.100.56:3000/denozord/EvoBGP/commit/e1aaa18))
* feat: enhance settings page layout and section management ([d91667f](http://192.168.100.56:3000/denozord/EvoBGP/commit/d91667f))
* feat: enhance UI components with improved styling and layout adjustments. Update button sizes, text ([6a72842](http://192.168.100.56:3000/denozord/EvoBGP/commit/6a72842))
* feat: enhance worker functionality by adding job queuing for deploy_apply. Introduce a Registry fiel ([42d956d](http://192.168.100.56:3000/denozord/EvoBGP/commit/42d956d))
* feat: implement AS holder name resolution and enhance ASEntry structure. Add ASHolderName function t ([c958d5a](http://192.168.100.56:3000/denozord/EvoBGP/commit/c958d5a))
* feat: implement bulk selection and deletion for modules and entries. Add functionality for selecting ([e816d17](http://192.168.100.56:3000/denozord/EvoBGP/commit/e816d17))
* feat: implement CDN source preview functionality and enhance data handling. Add a new endpoint for p ([e20c9f3](http://192.168.100.56:3000/denozord/EvoBGP/commit/e20c9f3))
* feat: implement countAll function for API data retrieval ([40c8a8d](http://192.168.100.56:3000/denozord/EvoBGP/commit/40c8a8d))
* feat: implement domain resolution via DoH in refresh pipeline. Enhance the collectModulePrefixRows f ([678bd27](http://192.168.100.56:3000/denozord/EvoBGP/commit/678bd27))
* feat: implement effective peer enabling logic during peer creation. Update CreatePeer methods in Pos ([3e85431](http://192.168.100.56:3000/denozord/EvoBGP/commit/3e85431))
* feat: implement module refresh job management with concurrency control. Add CountOtherActiveModuleRe ([149fb0a](http://192.168.100.56:3000/denozord/EvoBGP/commit/149fb0a))
* feat: implement peer reconciliation job for BGP peers ([3b17228](http://192.168.100.56:3000/denozord/EvoBGP/commit/3b17228))
* feat: implement revision retention settings and pruning logic ([4f50350](http://192.168.100.56:3000/denozord/EvoBGP/commit/4f50350))
* feat: implement theme selection and automatic theme application based on user preference. Update App ([b70a643](http://192.168.100.56:3000/denozord/EvoBGP/commit/b70a643))
* feat: implement versioning and release management in the project ([bb334b1](http://192.168.100.56:3000/denozord/EvoBGP/commit/bb334b1))
* feat: improve BGP protocol summary handling by introducing isBGPProtocolSummaryRow function. Update ([399622c](http://192.168.100.56:3000/denozord/EvoBGP/commit/399622c))
* feat: improve job detail UI with enhanced spacing and layout adjustments. Update table cell classes ([97908b5](http://192.168.100.56:3000/denozord/EvoBGP/commit/97908b5))
* feat: improve peer speaker assignment UX in network settings ([f877ec8](http://192.168.100.56:3000/denozord/EvoBGP/commit/f877ec8))
* feat: improve UI components with enhanced overflow handling and layout adjustments. Update card and ([0ca5542](http://192.168.100.56:3000/denozord/EvoBGP/commit/0ca5542))
* feat: improve UI layout and responsiveness in module and operations pages. Refactor component struct ([233f29b](http://192.168.100.56:3000/denozord/EvoBGP/commit/233f29b))
* feat: redesign settings page with structured controls ([2da6d49](http://192.168.100.56:3000/denozord/EvoBGP/commit/2da6d49))
* feat: refactor materialization logic to group static routes by community in the refresh pipeline. In ([1c5a9c9](http://192.168.100.56:3000/denozord/EvoBGP/commit/1c5a9c9))
* feat: refactor operations page to enhance UI and component structure. Remove unused card components ([88226f6](http://192.168.100.56:3000/denozord/EvoBGP/commit/88226f6))
* feat: refine BGP template handling and documentation updates. Remove local IP requirements in BGP te ([6266a39](http://192.168.100.56:3000/denozord/EvoBGP/commit/6266a39))
* feat: refine job detail UI with improved layout and scrolling capabilities. Enhance spacing and over ([57c09ac](http://192.168.100.56:3000/denozord/EvoBGP/commit/57c09ac))
* feat: update BGP community structure and related documentation. Refactor BGP community handling by r ([a2cbfac](http://192.168.100.56:3000/denozord/EvoBGP/commit/a2cbfac))
* feat: update BGP template configuration and OpenAPI documentation. Change import settings from 'all' ([133ea93](http://192.168.100.56:3000/denozord/EvoBGP/commit/133ea93))
* feat: update BIRD installation process to build from source. Modify CI workflow to ensure BIRD 2.14 ([e2002ba](http://192.168.100.56:3000/denozord/EvoBGP/commit/e2002ba))
* feat: update CI configuration and Dockerfiles for improved build efficiency ([80229b3](http://192.168.100.56:3000/denozord/EvoBGP/commit/80229b3))
* feat: update collectModulePrefixRows to support prior snapshots and enhance CDN/domain prefix collec ([ee364c8](http://192.168.100.56:3000/denozord/EvoBGP/commit/ee364c8))
* feat: update Go module dependencies and enhance DNS resolution in refresh pipeline. Upgrade Go versi ([655d750](http://192.168.100.56:3000/denozord/EvoBGP/commit/655d750))
* feat: update OpenAPI documentation and internal logic for BIRD settings. Enhance the description of ([62bf987](http://192.168.100.56:3000/denozord/EvoBGP/commit/62bf987))
* feat: update RevisionDiff type and enhance OperationsDiffTab component for better diff handling. Int ([4aaaa30](http://192.168.100.56:3000/denozord/EvoBGP/commit/4aaaa30))
* feat: update Traefik deployment configuration for Let's Encrypt and logging ([39ba976](http://192.168.100.56:3000/denozord/EvoBGP/commit/39ba976))
* docs: add router-lists-ui integration documentation to README ([b7a8aab](http://192.168.100.56:3000/denozord/EvoBGP/commit/b7a8aab))
* docs: add Web UI section to EvoBGP architecture plan, detailing Svelte and shadcn-svelte integration ([272542b](http://192.168.100.56:3000/denozord/EvoBGP/commit/272542b))
* docs: enhance EvoBGP architecture plan with new risk mitigation strategies, testing matrix for BIRD2 ([3992d01](http://192.168.100.56:3000/denozord/EvoBGP/commit/3992d01))
* docs: enhance README and CI workflow documentation for Docker image building and publishing process, ([9eb3e33](http://192.168.100.56:3000/denozord/EvoBGP/commit/9eb3e33))
* docs: enhance README and CI workflow to clarify Docker image build conditions and Gitea Container Re ([c6687b2](http://192.168.100.56:3000/denozord/EvoBGP/commit/c6687b2))
* docs: openapi.html со встроенным Redoc; инструкция для Gitea и скрипт сборки ([a754e1d](http://192.168.100.56:3000/denozord/EvoBGP/commit/a754e1d))
* docs: update AGENTS and README to include guidelines for Conventional Commits ([4c23232](http://192.168.100.56:3000/denozord/EvoBGP/commit/4c23232))
* docs: update AGENTS and README with engineering rules and guidelines ([aff27e8](http://192.168.100.56:3000/denozord/EvoBGP/commit/aff27e8))
* docs: update architecture and API documentation to clarify job processing, in-memory registry usage, ([acc0608](http://192.168.100.56:3000/denozord/EvoBGP/commit/acc0608))
* docs: update EvoBGP architecture plan with detailed REST API section, including general agreements, ([1a86d6c](http://192.168.100.56:3000/denozord/EvoBGP/commit/1a86d6c))
* docs: update EvoBGP architecture plan with new repository structure, refined sections, and detailed ([4480d64](http://192.168.100.56:3000/denozord/EvoBGP/commit/4480d64))
* docs: update README and CI workflow to clarify Docker workspace mounting for bird2 job ([6be46db](http://192.168.100.56:3000/denozord/EvoBGP/commit/6be46db))
* docs: update README and CI workflow to clarify Gitea Registry login process and improve Docker image ([f1b33b9](http://192.168.100.56:3000/denozord/EvoBGP/commit/f1b33b9))
* docs: update README and Docker Compose configurations to clarify usage of pre-built images from the ([7fac79c](http://192.168.100.56:3000/denozord/EvoBGP/commit/7fac79c))
* docs: update README to include information about the EvoBGP web interface, linking to quickstart and ([5d21f01](http://192.168.100.56:3000/denozord/EvoBGP/commit/5d21f01))
* docs: статическая Redoc-страница OpenAPI (для просмотра в Gitea без встроенного рендера) ([5c9d2e2](http://192.168.100.56:3000/denozord/EvoBGP/commit/5c9d2e2))
* docs: убрать типографское тире и невидимый Unicode из текстов API ([fbe9cd3](http://192.168.100.56:3000/denozord/EvoBGP/commit/fbe9cd3))
* refactor: adjust BootstrapWorkers function call in server.go to change the order of returned values, ([032fa99](http://192.168.100.56:3000/denozord/EvoBGP/commit/032fa99))
* refactor: consolidate module interval display logic and improve edit form handling ([1f85654](http://192.168.100.56:3000/denozord/EvoBGP/commit/1f85654))
* refactor: enhance bird metrics polling with context support ([f6b94a4](http://192.168.100.56:3000/denozord/EvoBGP/commit/f6b94a4))
* refactor: enhance CI workflow for granular change detection by module and improve idempotency key ge ([54eaad3](http://192.168.100.56:3000/denozord/EvoBGP/commit/54eaad3))
* refactor: enhance job status checks in worker tests ([6107c3e](http://192.168.100.56:3000/denozord/EvoBGP/commit/6107c3e))
* refactor: enhance JSON struct tags for various data models in backend and memory store. Update field ([4a860a0](http://192.168.100.56:3000/denozord/EvoBGP/commit/4a860a0))
* refactor: enhance module display and input handling in module pages ([25374a6](http://192.168.100.56:3000/denozord/EvoBGP/commit/25374a6))
* refactor: enhance module interval label logic and add reset buttons in edit forms ([b99fc17](http://192.168.100.56:3000/denozord/EvoBGP/commit/b99fc17))
* refactor: improve code consistency and formatting across components ([c263fd5](http://192.168.100.56:3000/denozord/EvoBGP/commit/c263fd5))
* refactor: improve handling of deployable fragments in ApplyRevision and buildPreviewFragments functi ([6932271](http://192.168.100.56:3000/denozord/EvoBGP/commit/6932271))
* refactor: improve module save functionality and nullable value handling ([05ec176](http://192.168.100.56:3000/denozord/EvoBGP/commit/05ec176))
* refactor: improve PATCH handling for module updates with nullable field support ([e700f90](http://192.168.100.56:3000/denozord/EvoBGP/commit/e700f90))
* refactor: optimize job listing and counting logic for concurrency safety. Separate tenant filtering ([d83efe2](http://192.168.100.56:3000/denozord/EvoBGP/commit/d83efe2))
* refactor: phase 2 structural alignment — reports, importer, nodecli, pagination ([250cdea](http://192.168.100.56:3000/denozord/EvoBGP/commit/250cdea))
* refactor: remove strict bind configuration from BGP templates and update related logic. Eliminate 's ([482eeb1](http://192.168.100.56:3000/denozord/EvoBGP/commit/482eeb1))
* refactor: streamline Docker build process in CI configuration and documentation ([6025f38](http://192.168.100.56:3000/denozord/EvoBGP/commit/6025f38))
* refactor: streamline edit form handling for community and DoH profile selections ([1765ab8](http://192.168.100.56:3000/denozord/EvoBGP/commit/1765ab8))
* refactor: streamline module refresh process and enhance revision handling ([10e2415](http://192.168.100.56:3000/denozord/EvoBGP/commit/10e2415))
* refactor: switch BIRD installation from source to Ubuntu package. Update CI workflow and Dockerfiles ([0b3ceaa](http://192.168.100.56:3000/denozord/EvoBGP/commit/0b3ceaa))
* refactor: update AS entry handling in API and database to support ASN-only entries. Remove prefix fi ([1db6b5b](http://192.168.100.56:3000/denozord/EvoBGP/commit/1db6b5b))
* refactor: update Docker build context and paths in CI configuration and documentation ([7d52a4a](http://192.168.100.56:3000/denozord/EvoBGP/commit/7d52a4a))
* refactor: update Docker build variable syntax in CI configuration and documentation ([27e01d0](http://192.168.100.56:3000/denozord/EvoBGP/commit/27e01d0))
* refactor: update job kind display logic in schedule page ([dc1f4a5](http://192.168.100.56:3000/denozord/EvoBGP/commit/dc1f4a5))
* refactor: update ListModules and GetModule methods in Postgres repository to include cron expression ([73d0003](http://192.168.100.56:3000/denozord/EvoBGP/commit/73d0003))
* refactor: update Switch component styles and structure for improved accessibility and responsiveness ([d4282c2](http://192.168.100.56:3000/denozord/EvoBGP/commit/d4282c2))
* refactor: update worker processes in EvoBGP to accept dependencies for shared store and job registry ([b7968db](http://192.168.100.56:3000/denozord/EvoBGP/commit/b7968db))
* ci: add conditional job for OpenAPI linting based on file changes; update README with workflow detai ([3adfc49](http://192.168.100.56:3000/denozord/EvoBGP/commit/3adfc49))
* ci: phase 4 automation — web job, lint-httpapi, golangci ([64516c3](http://192.168.100.56:3000/denozord/EvoBGP/commit/64516c3))
* ci: update README and CI workflow to improve bird2 job installation process and clarify Docker usage ([eddc6b8](http://192.168.100.56:3000/denozord/EvoBGP/commit/eddc6b8))
* chore: phase 3 code quality — pipeline prefixes, godoc baseline ([ea65451](http://192.168.100.56:3000/denozord/EvoBGP/commit/ea65451))
* chore: update CI configuration to disable filesystem entitlement for Docker builds ([9ab7de6](http://192.168.100.56:3000/denozord/EvoBGP/commit/9ab7de6))
* chore: update package dependencies and enhance AppShell component with new navigation items and tool ([7d85fef](http://192.168.100.56:3000/denozord/EvoBGP/commit/7d85fef))
* chore: upgrade Go version to 1.24 in Dockerfiles for evobgp-agent and gobinary. This change ensures ([4ba5b03](http://192.168.100.56:3000/denozord/EvoBGP/commit/4ba5b03))
* fix: adjust Postgres resource limits and healthcheck parameters. Increase memory limit to 256M and m ([20e674c](http://192.168.100.56:3000/denozord/EvoBGP/commit/20e674c))
* fix: improve input handling and validation for settings fields ([1386b63](http://192.168.100.56:3000/denozord/EvoBGP/commit/1386b63))
* fix: improve sidebar and navigation layout in AppShell component ([9f02128](http://192.168.100.56:3000/denozord/EvoBGP/commit/9f02128))
* fix: update BGP configuration in test scenarios to use new IPv6 addresses ([9fd9745](http://192.168.100.56:3000/denozord/EvoBGP/commit/9fd9745))
* fix: update bird build script and Dockerfile to include libncurses dependency ([0cb5a7d](http://192.168.100.56:3000/denozord/EvoBGP/commit/0cb5a7d))
* fix: update bird build script and Dockerfile to include readline dependency ([35ed79b](http://192.168.100.56:3000/denozord/EvoBGP/commit/35ed79b))
* fix: update bird build script and Dockerfile to remove readline dependency ([fefadac](http://192.168.100.56:3000/denozord/EvoBGP/commit/fefadac))
* fix: update deployment configuration for Traefik and secure Web UI ([00bd037](http://192.168.100.56:3000/denozord/EvoBGP/commit/00bd037))
* fix: update Docker build commands in CI configuration and documentation ([b5061a5](http://192.168.100.56:3000/denozord/EvoBGP/commit/b5061a5))
* fix(httpapi): phase 1 compliance — ERR-01, SEC-04, CDN client, scheduler docs ([1dd7135](http://192.168.100.56:3000/denozord/EvoBGP/commit/1dd7135))
* test: add CDN prefetch tests for 304 responses and snapshot handling ([a536a2d](http://192.168.100.56:3000/denozord/EvoBGP/commit/a536a2d))
* test: add validation for 'strict bind on' in BGP rendering tests. Enhance BGP test cases to ensure ' ([dc5e20e](http://192.168.100.56:3000/denozord/EvoBGP/commit/dc5e20e))
* docker: refactor cache handling in Docker bake configuration ([5e59a1c](http://192.168.100.56:3000/denozord/EvoBGP/commit/5e59a1c))
* docker: simplify web image targets in Docker bake configuration ([765a805](http://192.168.100.56:3000/denozord/EvoBGP/commit/765a805))
* docker: update web image build process and README ([92f6de8](http://192.168.100.56:3000/denozord/EvoBGP/commit/92f6de8))
* web(ui): app layout — sidebar, mobile sheet, topbar, PageHeader ([2d78c81](http://192.168.100.56:3000/denozord/EvoBGP/commit/2d78c81))
* web(ui): AppDataTable pattern and modules pilot ([b992301](http://192.168.100.56:3000/denozord/EvoBGP/commit/b992301))
* web(ui): design tokens — spacing and component scale ([ac635e9](http://192.168.100.56:3000/denozord/EvoBGP/commit/ac635e9))
* web(ui): feedback — confirm dialog, skeletons, README ([87309cf](http://192.168.100.56:3000/denozord/EvoBGP/commit/87309cf))
* web(ui): form patterns — FormField, formsnap, directories pilot ([77a2cfd](http://192.168.100.56:3000/denozord/EvoBGP/commit/77a2cfd))
* web(ui): foundation — ui/core layout, shadcn primitives, notify ([00a2a44](http://192.168.100.56:3000/denozord/EvoBGP/commit/00a2a44))
* web(ui): migrate dashboard, schedule, monitoring to ui patterns ([57dcf30](http://192.168.100.56:3000/denozord/EvoBGP/commit/57dcf30))
* web(ui): migrate network to ui patterns ([3a3056b](http://192.168.100.56:3000/denozord/EvoBGP/commit/3a3056b))
* web(ui): migrate operations and module detail to ui patterns ([1f9930d](http://192.168.100.56:3000/denozord/EvoBGP/commit/1f9930d))
* web(ui): migrate settings to ui patterns ([9808aca](http://192.168.100.56:3000/denozord/EvoBGP/commit/9808aca))
* Enhance EvoBGP architecture plan by refining the overview, updating todos with detailed descriptions ([30efdbf](http://192.168.100.56:3000/denozord/EvoBGP/commit/30efdbf))
* init ([cbf3aaa](http://192.168.100.56:3000/denozord/EvoBGP/commit/cbf3aaa))
* Refine EvoBGP architecture plan with updated database schema details, improved descriptions for comp ([431ef3e](http://192.168.100.56:3000/denozord/EvoBGP/commit/431ef3e))
* Update EvoBGP architecture plan with new overview and todos. Refined descriptions for control-plane ([31ff5bb](http://192.168.100.56:3000/denozord/EvoBGP/commit/31ff5bb))
+8
View File
@@ -44,4 +44,12 @@ docker compose --profile reference up -d
- **Go:** модуль `evobgp`, точки входа в `cmd/`. - **Go:** модуль `evobgp`, точки входа в `cmd/`.
- **Веб:** SvelteKit в каталоге `web/` (см. [web/README.md](web/README.md)). - **Веб:** SvelteKit в каталоге `web/` (см. [web/README.md](web/README.md)).
## Версионирование и релизы
Версии определяются автоматически из Conventional Commits при merge в `main` (semantic-release на Gitea Actions). Первая версия — **1.0.0**; ручной bump не нужен.
- [docs/releasing.md](docs/releasing.md) — пайплайн, commit conventions, секреты CI
- API: `GET /version` и `GET /v1/version` (поле `version`)
- Docker-образы: теги `latest`, `vX.Y.Z`, `X.Y.Z` — в **том же CI run**, что и релиз (job `release`)
Лицензия и условия использования — по политике владельца репозитория. Лицензия и условия использования — по политике владельца репозитория.
+7 -1
View File
@@ -19,6 +19,7 @@ import (
"evobgp/internal/platform" "evobgp/internal/platform"
"evobgp/internal/render" "evobgp/internal/render"
"evobgp/internal/scheduler" "evobgp/internal/scheduler"
"evobgp/internal/version"
) )
// microVPS entrypoint: один процесс — HTTP API и фоновые воркеры scheduler, ingest, render, deploy (общий store и jobs.Registry). // microVPS entrypoint: один процесс — HTTP API и фоновые воркеры scheduler, ingest, render, deploy (общий store и jobs.Registry).
@@ -37,7 +38,7 @@ func main() {
log.Fatal(err) log.Fatal(err)
} }
defer srv.Close() defer srv.Close()
observability.SetBuildInfo("0.1.0", strings.TrimSpace(os.Getenv("EVOBGP_GIT_SHA"))) observability.SetBuildInfo(version.Version, version.GitSHA)
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop() defer stop()
@@ -56,6 +57,10 @@ func main() {
httpSrv := &http.Server{ httpSrv := &http.Server{
Addr: cfg.HTTPAddr, Addr: cfg.HTTPAddr,
Handler: srv.Handler(), Handler: srv.Handler(),
ReadHeaderTimeout: 10 * time.Second,
ReadTimeout: 60 * time.Second,
WriteTimeout: 120 * time.Second,
IdleTimeout: 120 * time.Second,
} }
go func() { go func() {
svc := platform.ServiceName("evobgp-all") svc := platform.ServiceName("evobgp-all")
@@ -90,6 +95,7 @@ func startBirdMetricsPoller(ctx context.Context) {
return birdfmt.ShowProtocols(ctx, socket, birdcBin) return birdfmt.ShowProtocols(ctx, socket, birdcBin)
}, },
birdfmt.CountEstablishedBGPSessions, birdfmt.CountEstablishedBGPSessions,
birdfmt.ParseBGPProtocolStates,
) )
log.Printf("birdc protocols poller enabled (socket=%s interval=%s)", sock, interval) log.Printf("birdc protocols poller enabled (socket=%s interval=%s)", sock, interval)
} }
+8 -2
View File
@@ -15,6 +15,7 @@ import (
"evobgp/internal/httpapi" "evobgp/internal/httpapi"
"evobgp/internal/observability" "evobgp/internal/observability"
"evobgp/internal/platform" "evobgp/internal/platform"
"evobgp/internal/version"
) )
func main() { func main() {
@@ -34,7 +35,7 @@ func main() {
} }
defer srv.Close() defer srv.Close()
observability.SetBuildInfo("0.1.0", strings.TrimSpace(os.Getenv("EVOBGP_GIT_SHA"))) observability.SetBuildInfo(version.Version, version.GitSHA)
ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM) ctx, stop := signal.NotifyContext(context.Background(), os.Interrupt, syscall.SIGTERM)
defer stop() defer stop()
@@ -43,6 +44,10 @@ func main() {
httpSrv := &http.Server{ httpSrv := &http.Server{
Addr: cfg.HTTPAddr, Addr: cfg.HTTPAddr,
Handler: srv.Handler(), Handler: srv.Handler(),
ReadHeaderTimeout: 10 * time.Second,
ReadTimeout: 60 * time.Second,
WriteTimeout: 120 * time.Second,
IdleTimeout: 120 * time.Second,
} }
go func() { go func() {
svc := platform.ServiceName("evobgp-api") svc := platform.ServiceName("evobgp-api")
@@ -52,7 +57,7 @@ func main() {
tid, mCDN, mIP, rev, sp := srv.Store().DemoIDs() tid, mCDN, mIP, rev, sp := srv.Store().DemoIDs()
log.Printf("demo tenant=%s module_cdn=%s module_ip_ranges=%s revision=%s speaker=%s", tid, mCDN, mIP, rev, sp) log.Printf("demo tenant=%s module_cdn=%s module_ip_ranges=%s revision=%s speaker=%s", tid, mCDN, mIP, rev, sp)
log.Printf("example: EVOBGP_API_KEYS=op|%s|operator,node|%s|node", tid, tid) log.Printf("example: EVOBGP_API_KEYS=op|%s|operator,node|%s|node", tid, tid)
log.Printf("with EVOBGP_DEV_INSECURE=1 use Authorization: Bearer dev (operator, demo tenant only)") log.Printf("demo auth: Authorization: Bearer dev (operator, demo tenant only)")
} }
if err := httpSrv.ListenAndServe(); err != nil && err != http.ErrServerClosed { if err := httpSrv.ListenAndServe(); err != nil && err != http.ErrServerClosed {
log.Fatal(err) log.Fatal(err)
@@ -83,6 +88,7 @@ func startBirdMetricsPoller(ctx context.Context) {
return birdfmt.ShowProtocols(ctx, socket, birdcBin) return birdfmt.ShowProtocols(ctx, socket, birdcBin)
}, },
birdfmt.CountEstablishedBGPSessions, birdfmt.CountEstablishedBGPSessions,
birdfmt.ParseBGPProtocolStates,
) )
log.Printf("birdc protocols poller enabled (socket=%s interval=%s)", sock, interval) log.Printf("birdc protocols poller enabled (socket=%s interval=%s)", sock, interval)
} }
+3 -228
View File
@@ -1,240 +1,15 @@
package main package main
import ( import (
"context"
"crypto/ed25519"
"encoding/base64"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"os" "os"
"path/filepath"
"strings"
"time"
"evobgp/internal/birdfmt" "evobgp/internal/nodecli"
"evobgp/internal/bundle"
"evobgp/internal/signing"
) )
func main() { func main() {
if len(os.Args) < 2 { if len(os.Args) < 2 {
usage() nodecli.Usage(os.Stderr)
os.Exit(2)
}
switch os.Args[1] {
case "pull-bundle":
os.Exit(cmdPullBundle(os.Args[2:]))
case "verify-bundle":
os.Exit(cmdVerifyBundle(os.Args[2:]))
case "apply-bundle":
os.Exit(cmdApplyBundle(os.Args[2:]))
default:
usage()
os.Exit(2) os.Exit(2)
} }
} os.Exit(nodecli.Run(os.Args[1:]))
func usage() {
fmt.Fprintf(os.Stderr, `Usage:
%s pull-bundle -base-url URL -token TOKEN -speaker-id ID [-revision-id ID] [-o path]
%s verify-bundle -f bundle.tar.gz (-pubkey-base64 B64 | -pubkey-hex HEX)
%s apply-bundle -f bundle.tar.gz -extract-dir DIR (-pubkey-base64 B64 | -pubkey-hex HEX)
[-bird PATH] [-birdc PATH] [-socket PATH] [-timeout DURATION]
apply-bundle verifies, extracts, runs bird -p on main bird.conf, then birdc configure.
`, os.Args[0], os.Args[0], os.Args[0])
}
func cmdPullBundle(args []string) int {
fs := flag.NewFlagSet("pull-bundle", flag.ExitOnError)
base := fs.String("base-url", "", "control plane base URL, e.g. http://localhost:8080")
token := fs.String("token", "", "Bearer token (node role)")
speaker := fs.String("speaker-id", "", "bgp_speaker id")
revision := fs.String("revision-id", "", "revision to fetch (empty = latest pointer)")
out := fs.String("o", "bundle.tar.gz", "output file")
_ = fs.Parse(args)
if strings.TrimSpace(*base) == "" || *token == "" || *speaker == "" {
fmt.Fprintln(os.Stderr, "pull-bundle: -base-url, -token, -speaker-id are required")
return 2
}
rev := strings.TrimSpace(*revision)
if rev == "" {
var err error
rev, err = fetchLatestRevision(*base, *token, *speaker)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
}
body, err := fetchBundle(*base, *token, *speaker, rev)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
if err := os.WriteFile(*out, body, 0o644); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
fmt.Fprintf(os.Stderr, "wrote %s (revision %s)\n", *out, rev)
return 0
}
func fetchLatestRevision(base, token, speaker string) (string, error) {
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/revisions/latest"
req, err := http.NewRequest(http.MethodGet, u, nil)
if err != nil {
return "", err
}
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return "", err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body)
return "", fmt.Errorf("latest revision: %s: %s", resp.Status, strings.TrimSpace(string(b)))
}
var out struct {
RevisionID string `json:"revision_id"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return "", err
}
if out.RevisionID == "" {
return "", fmt.Errorf("empty revision_id in response")
}
return out.RevisionID, nil
}
func fetchBundle(base, token, speaker, revision string) ([]byte, error) {
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/bundle/" + revision
req, err := http.NewRequest(http.MethodGet, u, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer resp.Body.Close()
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("bundle: %s: %s", resp.Status, strings.TrimSpace(string(b)))
}
return io.ReadAll(resp.Body)
}
func loadPubKey(pubB64, pubHex string) (ed25519.PublicKey, error) {
switch {
case strings.TrimSpace(pubB64) != "":
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(pubB64))
if err != nil {
return nil, err
}
if len(raw) != ed25519.PublicKeySize {
return nil, fmt.Errorf("pubkey-base64 must decode to %d bytes", ed25519.PublicKeySize)
}
return ed25519.PublicKey(raw), nil
case strings.TrimSpace(pubHex) != "":
return bundle.ParsePublicKeyHex(pubHex)
default:
return nil, fmt.Errorf("public key required")
}
}
func cmdVerifyBundle(args []string) int {
fs := flag.NewFlagSet("verify-bundle", flag.ExitOnError)
path := fs.String("f", "", "path to bundle.tar.gz")
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
_ = fs.Parse(args)
if *path == "" {
fmt.Fprintln(os.Stderr, "verify-bundle: -f required")
return 2
}
pub, err := loadPubKey(*pubB64, *pubHex)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 2
}
raw, err := os.ReadFile(*path)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
v, err := signing.VerifyGzippedTar(raw, pub)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
fmt.Fprintf(os.Stderr, "ok: revision %s, %d payload files\n", v.Manifest.RevisionID, len(v.Files))
return 0
}
func cmdApplyBundle(args []string) int {
fs := flag.NewFlagSet("apply-bundle", flag.ExitOnError)
path := fs.String("f", "", "path to bundle.tar.gz")
dir := fs.String("extract-dir", "", "directory to extract into")
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
bird := fs.String("bird", "", "bird binary (default PATH)")
birdc := fs.String("birdc", "", "birdc binary (default PATH)")
socket := fs.String("socket", "", "birdc -s socket path")
timeout := fs.Duration("timeout", 30*time.Second, "timeout for bird/birdc")
_ = fs.Parse(args)
if *path == "" || *dir == "" {
fmt.Fprintln(os.Stderr, "apply-bundle: -f and -extract-dir required")
return 2
}
pub, err := loadPubKey(*pubB64, *pubHex)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 2
}
raw, err := os.ReadFile(*path)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
v, err := signing.VerifyGzippedTar(raw, pub)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
root := filepath.Clean(*dir)
if err := os.MkdirAll(root, 0o755); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
if err := bundle.WriteExtractedFiles(root, v); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
mainRel := v.FindMainBirdConf()
if mainRel == "" {
fmt.Fprintln(os.Stderr, "bundle has no bird.conf path in manifest")
return 1
}
mainPath := filepath.Join(root, filepath.FromSlash(strings.TrimPrefix(mainRel, "/")))
ctx, cancel := context.WithTimeout(context.Background(), *timeout)
defer cancel()
ctl := &birdfmt.BirdCtl{Bird: *bird, Birdc: *birdc, Socket: *socket}
if err := ctl.ParseCheck(ctx, mainPath); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
if err := ctl.Configure(ctx); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
fmt.Fprintf(os.Stderr, "applied revision %s (main config %s)\n", v.Manifest.RevisionID, mainPath)
return 0
} }
+3
View File
@@ -41,6 +41,9 @@ func main() {
apiBase := strings.TrimSpace(os.Getenv("EVOBGP_CONTROL_PLANE_URL")) apiBase := strings.TrimSpace(os.Getenv("EVOBGP_CONTROL_PLANE_URL"))
apiTok := strings.TrimSpace(os.Getenv("EVOBGP_SCHEDULER_BEARER")) apiTok := strings.TrimSpace(os.Getenv("EVOBGP_SCHEDULER_BEARER"))
if os.Getenv("EVOBGP_SCHEDULER_STANDALONE") == "1" && apiBase == "" {
log.Fatal("evobgp-scheduler: EVOBGP_SCHEDULER_STANDALONE=1 requires EVOBGP_CONTROL_PLANE_URL and EVOBGP_SCHEDULER_BEARER (split deploy must not use in-process jobs.Registry)")
}
deps := &scheduler.Deps{ deps := &scheduler.Deps{
Store: st, Store: st,
HTTP: &http.Client{Timeout: 45 * time.Second}, HTTP: &http.Client{Timeout: 45 * time.Second},
+3 -1
View File
@@ -96,7 +96,7 @@ services:
<<: *env-ref <<: *env-ref
EVOBGP_HTTP_ADDR: ":8080" EVOBGP_HTTP_ADDR: ":8080"
EVOBGP_SEED_DEMO: "1" EVOBGP_SEED_DEMO: "1"
# Local reference only: allows Bearer dev for scheduler HTTP client (EVOBGP_SCHEDULER_BEARER). # DEV ONLY — не для production. Bearer dev + слабые demo-секреты (см. docs/access.md).
EVOBGP_DEV_INSECURE: "1" EVOBGP_DEV_INSECURE: "1"
EVOBGP_BIRDC_SOCKET: /run/bird/bird.ctl EVOBGP_BIRDC_SOCKET: /run/bird/bird.ctl
EVOBGP_BIRDC_INTERVAL: 30s EVOBGP_BIRDC_INTERVAL: 30s
@@ -126,6 +126,8 @@ services:
condition: service_started condition: service_started
environment: environment:
<<: *env-ref <<: *env-ref
# Split deploy: scheduler must enqueue jobs via API, not in-process Registry (ARCH-04).
EVOBGP_SCHEDULER_STANDALONE: "1"
EVOBGP_CONTROL_PLANE_URL: http://evobgp-api:8080 EVOBGP_CONTROL_PLANE_URL: http://evobgp-api:8080
EVOBGP_SCHEDULER_BEARER: dev EVOBGP_SCHEDULER_BEARER: dev
logging: *default-logging logging: *default-logging
+1
View File
@@ -135,6 +135,7 @@ services:
EVOBGP_BIRDC_INTERVAL: 30s EVOBGP_BIRDC_INTERVAL: 30s
EVOBGP_BIRD_ACTIVE_DIR: /etc/bird EVOBGP_BIRD_ACTIVE_DIR: /etc/bird
EVOBGP_BIRD_STAGING_DIR: /tmp/evobgp-bird-staging EVOBGP_BIRD_STAGING_DIR: /tmp/evobgp-bird-staging
# DEV ONLY — не для production (см. docs/access.md).
EVOBGP_DEV_INSECURE: "1" EVOBGP_DEV_INSECURE: "1"
volumes: volumes:
- bird_etc:/etc/bird - bird_etc:/etc/bird
+14 -4
View File
@@ -2,22 +2,30 @@
## CI (Gitea Actions) ## CI (Gitea Actions)
Сборка образов — **`docker buildx bake`** (`deploy/docker/docker-bake.hcl`), а не матрица из восьми отдельных `docker build`. Сборка образов — **`docker buildx bake`** (`deploy/docker/docker-bake.hcl`), не отдельные `docker build`.
**Publish:** push в `main` после quality gates — job **release** в [.gitea/workflows/ci.yaml](../.gitea/workflows/ci.yaml): semantic-release + bake с `VERSION` из релиза.
Локально BuildKit также кэширует `/go/pkg/mod` и `~/.cache/go-build` через `RUN --mount=type=cache`.
| Было | Стало | | Было | Стало |
|------|--------| |------|--------|
| 8× `go mod download` + 8× `go build` (разные BIN) | 1× download + 1× компиляция всех `cmd/*` | | 8× `go mod download` + 8× `go build` (разные BIN) | 1× download + 1× компиляция всех `cmd/*` |
| 2× сборка BIRD из исходников (api, all) | 1× stage `birdc`, копируется в runtime | | 2× сборка BIRD из исходников (api, all) | 1× stage `birdc`, копируется в runtime |
| 2× `npm ci` (web, web-all) | 1× `web-deps` + два nginx-образа | | 2× `npm ci` (web, web-all) | 1× `web-deps` + 1× `web-build` + два nginx-образа (общий артефакт) |
| 8 runner'ов с checkout/login | 1 job `docker-go`, 1 job `docker-web` | | 8 runner'ов с checkout/login | 1 job `docker-go`, 1 job `docker-web` |
Кэш registry (переменные bake): Кэш registry (переменные bake):
- `git.shts.su/<owner>/evobgp-buildcache:go-buildcache` - `git.shts.su/<owner>/evobgp-buildcache:go-buildcache` — **запись** только из target `go-build-all`
- `git.shts.su/<owner>/evobgp-buildcache:web-buildcache` - `git.shts.su/<owner>/evobgp-buildcache:web-buildcache` — **запись** только из target `web-build`
Остальные bake-target’ы только `cache-from` (чтение). Параллельный `cache-to` в один ref ломает manifest в registry (`content descriptor … not found`).
Локально BuildKit также кэширует `/go/pkg/mod` и `~/.cache/go-build` через `RUN --mount=type=cache`. Локально BuildKit также кэширует `/go/pkg/mod` и `~/.cache/go-build` через `RUN --mount=type=cache`.
Если CI падает на «not found» после смены схемы кэша — один раз удалите теги `evobgp-buildcache:go-buildcache` и `:web-buildcache` в registry и пересоберите.
## Локальная сборка ## Локальная сборка
Из корня репозитория: Из корня репозитория:
@@ -27,6 +35,8 @@ cd deploy/docker
export REGISTRY=git.shts.su/<owner> export REGISTRY=git.shts.su/<owner>
export IMAGE_TAG=latest export IMAGE_TAG=latest
export SHORT_SHA=$(git rev-parse --short HEAD) export SHORT_SHA=$(git rev-parse --short HEAD)
export VERSION=dev
export BUILD_TIME=
sh write-bake-override.sh sh write-bake-override.sh
docker buildx bake --allow=fs.read=../.. -f docker-bake.hcl -f docker-bake.override.hcl go-images docker buildx bake --allow=fs.read=../.. -f docker-bake.hcl -f docker-bake.override.hcl go-images
docker buildx bake --allow=fs.read=../.. -f docker-bake.hcl -f docker-bake.override.hcl web-images docker buildx bake --allow=fs.read=../.. -f docker-bake.hcl -f docker-bake.override.hcl web-images
+50 -14
View File
@@ -19,6 +19,14 @@ variable "SHA_FULL" {
default = "" default = ""
} }
variable "VERSION" {
default = "dev"
}
variable "BUILD_TIME" {
default = ""
}
variable "CACHE_REF_GO" { variable "CACHE_REF_GO" {
default = "" default = ""
} }
@@ -32,7 +40,10 @@ function "go-cache-from" {
result = notequal("", CACHE_REF_GO) ? ["type=registry,ref=${CACHE_REF_GO}"] : [] result = notequal("", CACHE_REF_GO) ? ["type=registry,ref=${CACHE_REF_GO}"] : []
} }
function "go-cache-to" { # Экспорт кэша — только go-build-all / web-build (один writer на ref).
# Несколько target с cache-to в один ref и mode=max дают гонку в registry
# (content descriptor not found при параллельном bake).
function "go-cache-to-export" {
params = [] params = []
result = notequal("", CACHE_REF_GO) ? ["type=registry,ref=${CACHE_REF_GO},mode=max"] : [] result = notequal("", CACHE_REF_GO) ? ["type=registry,ref=${CACHE_REF_GO},mode=max"] : []
} }
@@ -42,7 +53,7 @@ function "web-cache-from" {
result = notequal("", CACHE_REF_WEB) ? ["type=registry,ref=${CACHE_REF_WEB}"] : [] result = notequal("", CACHE_REF_WEB) ? ["type=registry,ref=${CACHE_REF_WEB}"] : []
} }
function "web-cache-to" { function "web-cache-to-export" {
params = [] params = []
result = notequal("", CACHE_REF_WEB) ? ["type=registry,ref=${CACHE_REF_WEB},mode=max"] : [] result = notequal("", CACHE_REF_WEB) ? ["type=registry,ref=${CACHE_REF_WEB},mode=max"] : []
} }
@@ -64,6 +75,7 @@ group "go-images" {
] ]
} }
# web-deps / web-build — только зависимости (contexts), без tags; при --push в группе не указывать.
group "web-images" { group "web-images" {
targets = ["evobgp-web", "evobgp-web-all"] targets = ["evobgp-web", "evobgp-web-all"]
} }
@@ -76,7 +88,6 @@ target "go-deps" {
target = "deps" target = "deps"
platforms = ["linux/amd64"] platforms = ["linux/amd64"]
cache-from = go-cache-from() cache-from = go-cache-from()
cache-to = go-cache-to()
} }
target "go-build-all" { target "go-build-all" {
@@ -84,11 +95,16 @@ target "go-build-all" {
dockerfile = "deploy/docker/gobinary/Dockerfile" dockerfile = "deploy/docker/gobinary/Dockerfile"
target = "build-all" target = "build-all"
platforms = ["linux/amd64"] platforms = ["linux/amd64"]
args = {
VERSION = VERSION
GIT_SHA = notequal("", SHA_FULL) ? SHA_FULL : SHORT_SHA
BUILD_TIME = BUILD_TIME
}
contexts = { contexts = {
deps = "target:go-deps" deps = "target:go-deps"
} }
cache-from = go-cache-from() cache-from = go-cache-from()
cache-to = go-cache-to() cache-to = go-cache-to-export()
} }
target "go-birdc" { target "go-birdc" {
@@ -97,7 +113,6 @@ target "go-birdc" {
target = "birdc" target = "birdc"
platforms = ["linux/amd64"] platforms = ["linux/amd64"]
cache-from = go-cache-from() cache-from = go-cache-from()
cache-to = go-cache-to()
} }
target "_go-runtime" { target "_go-runtime" {
@@ -109,7 +124,6 @@ target "_go-runtime" {
build-all = "target:go-build-all" build-all = "target:go-build-all"
} }
cache-from = go-cache-from() cache-from = go-cache-from()
cache-to = go-cache-to()
} }
target "_go-runtime-birdc" { target "_go-runtime-birdc" {
@@ -122,19 +136,24 @@ target "_go-runtime-birdc" {
birdc = "target:go-birdc" birdc = "target:go-birdc"
} }
cache-from = go-cache-from() cache-from = go-cache-from()
cache-to = go-cache-to()
} }
function "image-tags" { function "image-tags" {
params = [name] params = [name]
result = notequal("", SHA_FULL) ? [ result = concat(
notequal("", SHA_FULL) ? [
"${REGISTRY}/${name}:${IMAGE_TAG}", "${REGISTRY}/${name}:${IMAGE_TAG}",
"${REGISTRY}/${name}:${SHORT_SHA}", "${REGISTRY}/${name}:${SHORT_SHA}",
"${REGISTRY}/${name}:sha-${SHA_FULL}", "${REGISTRY}/${name}:sha-${SHA_FULL}",
] : [ ] : [
"${REGISTRY}/${name}:${IMAGE_TAG}", "${REGISTRY}/${name}:${IMAGE_TAG}",
"${REGISTRY}/${name}:${SHORT_SHA}", "${REGISTRY}/${name}:${SHORT_SHA}",
] ],
notequal(VERSION, "dev") && notequal(VERSION, "") ? [
"${REGISTRY}/${name}:v${VERSION}",
"${REGISTRY}/${name}:${VERSION}",
] : []
)
} }
target "evobgp-api" { target "evobgp-api" {
@@ -187,7 +206,6 @@ target "evobgp-agent" {
build-all = "target:go-build-all" build-all = "target:go-build-all"
} }
cache-from = go-cache-from() cache-from = go-cache-from()
cache-to = go-cache-to()
tags = image-tags("evobgp-agent") tags = image-tags("evobgp-agent")
} }
@@ -199,25 +217,43 @@ target "web-deps" {
target = "deps" target = "deps"
platforms = ["linux/amd64"] platforms = ["linux/amd64"]
cache-from = web-cache-from() cache-from = web-cache-from()
cache-to = web-cache-to() }
target "web-build" {
context = "../.."
dockerfile = "deploy/docker/evobgp-web/Dockerfile"
target = "build"
platforms = ["linux/amd64"]
contexts = {
deps = "target:web-deps"
}
cache-from = web-cache-from()
cache-to = web-cache-to-export()
} }
target "evobgp-web" { target "evobgp-web" {
context = "../.." context = "../.."
dockerfile = "deploy/docker/evobgp-web/Dockerfile" dockerfile = "deploy/docker/evobgp-web/Dockerfile"
target = "web"
platforms = ["linux/amd64"] platforms = ["linux/amd64"]
contexts = { contexts = {
deps = "target:web-deps" web-artifacts = "target:web-build"
} }
args = { EVOBGP_UPSTREAM = "evobgp-api" } args = { EVOBGP_UPSTREAM = "evobgp-api" }
cache-from = web-cache-from() cache-from = web-cache-from()
cache-to = web-cache-to()
tags = image-tags("evobgp-web") tags = image-tags("evobgp-web")
} }
target "evobgp-web-all" { target "evobgp-web-all" {
inherits = ["evobgp-web"] context = "../.."
dockerfile = "deploy/docker/evobgp-web/Dockerfile"
target = "web"
platforms = ["linux/amd64"]
contexts = {
web-artifacts = "target:web-build"
}
args = { EVOBGP_UPSTREAM = "evobgp-all" } args = { EVOBGP_UPSTREAM = "evobgp-all" }
cache-from = web-cache-from()
tags = image-tags("evobgp-web-all") tags = image-tags("evobgp-web-all")
} }
+3 -2
View File
@@ -1,5 +1,6 @@
# syntax=docker/dockerfile:1.7 # syntax=docker/dockerfile:1.7
# Статическая панель EvoBGP (SvelteKit) + nginx. # Статическая панель EvoBGP (SvelteKit) + nginx.
# Финальный stage `web` ожидает bake-контекст web-artifacts (= target:web-build).
FROM public.ecr.aws/docker/library/node:22-alpine AS deps FROM public.ecr.aws/docker/library/node:22-alpine AS deps
WORKDIR /web WORKDIR /web
COPY web/package.json web/package-lock.json ./ COPY web/package.json web/package-lock.json ./
@@ -10,9 +11,9 @@ FROM deps AS build
COPY web/ ./ COPY web/ ./
RUN npm run build RUN npm run build
FROM public.ecr.aws/docker/library/nginx:1.27-alpine FROM public.ecr.aws/docker/library/nginx:1.27-alpine AS web
ARG EVOBGP_UPSTREAM=evobgp-api ARG EVOBGP_UPSTREAM=evobgp-api
COPY deploy/docker/evobgp-web/nginx.conf /tmp/nginx-default.conf COPY deploy/docker/evobgp-web/nginx.conf /tmp/nginx-default.conf
RUN sed -e "s/evobgp-api/${EVOBGP_UPSTREAM}/g" /tmp/nginx-default.conf > /etc/nginx/conf.d/default.conf \ RUN sed -e "s/evobgp-api/${EVOBGP_UPSTREAM}/g" /tmp/nginx-default.conf > /etc/nginx/conf.d/default.conf \
&& rm -f /tmp/nginx-default.conf && rm -f /tmp/nginx-default.conf
COPY --from=build /web/build /usr/share/nginx/html COPY --from=web-artifacts /web/build /usr/share/nginx/html
+8 -2
View File
@@ -5,8 +5,14 @@ server {
gzip on; gzip on;
gzip_types text/css application/javascript application/json image/svg+xml; gzip_types text/css application/javascript application/json image/svg+xml;
# Docker embedded DNS: без resolver nginx кэширует IP upstream при старте —
# после recreate evobgp-all остаётся 502 (connection refused на старый IP).
resolver 127.0.0.11 valid=10s ipv6=off;
set $evobgp_upstream evobgp-api;
location /v1/ { location /v1/ {
proxy_pass http://evobgp-api:8080/v1/; # С переменной в proxy_pass нельзя полагаться на замену URI — передаём $request_uri целиком.
proxy_pass http://$evobgp_upstream:8080$request_uri;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Host $host; proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Real-IP $remote_addr;
@@ -15,7 +21,7 @@ server {
} }
location = /metrics { location = /metrics {
proxy_pass http://evobgp-api:8080/metrics; proxy_pass http://$evobgp_upstream:8080/metrics;
proxy_http_version 1.1; proxy_http_version 1.1;
proxy_set_header Host $host; proxy_set_header Host $host;
} }
+18 -2
View File
@@ -10,22 +10,38 @@ RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \
FROM deps AS build-all FROM deps AS build-all
COPY . . COPY . .
ARG VERSION=dev
ARG GIT_SHA=unknown
ARG BUILD_TIME=
RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \ RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \
--mount=type=cache,target=/root/.cache/go-build,sharing=locked \ --mount=type=cache,target=/root/.cache/go-build,sharing=locked \
set -eux; \ set -eux; \
mkdir -p /out; \ mkdir -p /out; \
for d in cmd/*/; do \ for d in cmd/*/; do \
name="$(basename "$d")"; \ name="$(basename "$d")"; \
CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o "/out/${name}" "./cmd/${name}"; \ CGO_ENABLED=0 go build -trimpath \
-ldflags="-s -w \
-X evobgp/internal/version.Version=${VERSION} \
-X evobgp/internal/version.GitSHA=${GIT_SHA} \
-X evobgp/internal/version.BuildTime=${BUILD_TIME}" \
-o "/out/${name}" "./cmd/${name}"; \
done done
# Один бинарь (локальная сборка); в CI — build-all + runtime. # Один бинарь (локальная сборка); в CI — build-all + runtime.
FROM deps AS build FROM deps AS build
COPY . . COPY . .
ARG BIN=evobgp-api ARG BIN=evobgp-api
ARG VERSION=dev
ARG GIT_SHA=unknown
ARG BUILD_TIME=
RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \ RUN --mount=type=cache,target=/go/pkg/mod,sharing=locked \
--mount=type=cache,target=/root/.cache/go-build,sharing=locked \ --mount=type=cache,target=/root/.cache/go-build,sharing=locked \
CGO_ENABLED=0 go build -trimpath -ldflags="-s -w" -o /out/evobgp "./cmd/${BIN}" CGO_ENABLED=0 go build -trimpath \
-ldflags="-s -w \
-X evobgp/internal/version.Version=${VERSION} \
-X evobgp/internal/version.GitSHA=${GIT_SHA} \
-X evobgp/internal/version.BuildTime=${BUILD_TIME}" \
-o /out/evobgp "./cmd/${BIN}"
FROM public.ecr.aws/docker/library/debian:bookworm-slim AS birdc FROM public.ecr.aws/docker/library/debian:bookworm-slim AS birdc
ARG BIRD_VERSION=2.14 ARG BIRD_VERSION=2.14
+8
View File
@@ -6,6 +6,8 @@ REGISTRY="${REGISTRY:?REGISTRY required}"
IMAGE_TAG="${IMAGE_TAG:-latest}" IMAGE_TAG="${IMAGE_TAG:-latest}"
SHORT_SHA="${SHORT_SHA:-dev}" SHORT_SHA="${SHORT_SHA:-dev}"
SHA_FULL="${SHA_FULL:-}" SHA_FULL="${SHA_FULL:-}"
VERSION="${VERSION:-dev}"
BUILD_TIME="${BUILD_TIME:-}"
CACHE_REF_GO="${CACHE_REF_GO:-}" CACHE_REF_GO="${CACHE_REF_GO:-}"
CACHE_REF_WEB="${CACHE_REF_WEB:-}" CACHE_REF_WEB="${CACHE_REF_WEB:-}"
cat >"$OUT" <<EOF cat >"$OUT" <<EOF
@@ -22,6 +24,12 @@ variable "SHORT_SHA" {
variable "SHA_FULL" { variable "SHA_FULL" {
default = "${SHA_FULL}" default = "${SHA_FULL}"
} }
variable "VERSION" {
default = "${VERSION}"
}
variable "BUILD_TIME" {
default = "${BUILD_TIME}"
}
variable "CACHE_REF_GO" { variable "CACHE_REF_GO" {
default = "${CACHE_REF_GO}" default = "${CACHE_REF_GO}"
} }
+25 -1
View File
@@ -4,7 +4,7 @@
## По роли читателя ## По роли читателя
- **ИИ-агент / ассистент в репозитории** — [../AGENTS.md](../AGENTS.md): с чего начать чтение, карта `internal/` и `cmd/`, что не тащить в контекст. - **ИИ-агент / ассистент в репозитории** — [../AGENTS.md](../AGENTS.md): с чего начать чтение, карта `internal/` и `cmd/`, что не тащить в контекст. **Инженерные правила:** [../.cursor/rules/engineering.mdc](../.cursor/rules/engineering.mdc) (общие), [../.cursor/rules/web-shadcn.mdc](../.cursor/rules/web-shadcn.mdc) (Web UI), [../.cursor/rules/networking-bird.mdc](../.cursor/rules/networking-bird.mdc) (BIRD2/BGP/IP).
- **Оператор / DevOps** — [quickstart.md](quickstart.md), [architecture.md](architecture.md), [access.md](access.md), [deploy/compose/docker-compose.yaml](../deploy/compose/docker-compose.yaml). - **Оператор / DevOps** — [quickstart.md](quickstart.md), [architecture.md](architecture.md), [access.md](access.md), [deploy/compose/docker-compose.yaml](../deploy/compose/docker-compose.yaml).
- **Разработчик бэкенда или интегратор API** — [api.md](api.md), [access.md](access.md), [openapi.yaml](openapi.yaml), исходники маршрутов в `internal/httpapi/`. - **Разработчик бэкенда или интегратор API** — [api.md](api.md), [access.md](access.md), [openapi.yaml](openapi.yaml), исходники маршрутов в `internal/httpapi/`.
- **Разработчик фронтенда** — [quickstart.md](quickstart.md) (раздел про `web/` и CORS), [api.md](api.md), [../web/README.md](../web/README.md). - **Разработчик фронтенда** — [quickstart.md](quickstart.md) (раздел про `web/` и CORS), [api.md](api.md), [../web/README.md](../web/README.md).
@@ -20,10 +20,34 @@
| [api.md](api.md) | REST: префикс `/v1`, публичные маршруты, ссылки на OpenAPI | | [api.md](api.md) | REST: префикс `/v1`, публичные маршруты, ссылки на OpenAPI |
| [router-lists-ui-integration.md](router-lists-ui-integration.md) | Интеграция `router-lists-ui` с EvoBGP API (`DOMAINS/IP_RANGES/AS_PREFIXES/communities`) | | [router-lists-ui-integration.md](router-lists-ui-integration.md) | Интеграция `router-lists-ui` с EvoBGP API (`DOMAINS/IP_RANGES/AS_PREFIXES/communities`) |
| [access.md](access.md) | Выдача доступа: API-ключи, роли, нода, CORS | | [access.md](access.md) | Выдача доступа: API-ключи, роли, нода, CORS |
| [releasing.md](releasing.md) | Автоматические релизы, Conventional Commits, CI |
| [openapi.yaml](openapi.yaml) | Источник правды по контракту API | | [openapi.yaml](openapi.yaml) | Источник правды по контракту API |
| [OPENAPI-GITEA.md](OPENAPI-GITEA.md) | Как открыть HTML-документацию API (в т.ч. из Gitea) | | [OPENAPI-GITEA.md](OPENAPI-GITEA.md) | Как открыть HTML-документацию API (в т.ч. из Gitea) |
| [evobgp-api-sketches.md](evobgp-api-sketches.md) | Ранний черновик идей API (контекст, не замена OpenAPI) | | [evobgp-api-sketches.md](evobgp-api-sketches.md) | Ранний черновик идей API (контекст, не замена OpenAPI) |
## Инженерные правила (Cursor / разработка)
| Файл | Область |
|------|---------|
| [../.cursor/rules/engineering.mdc](../.cursor/rules/engineering.mdc) | Go, API, migrations, security, enforcement |
| [../.cursor/rules/web-shadcn.mdc](../.cursor/rules/web-shadcn.mdc) | SvelteKit, shadcn-svelte |
| [../.cursor/rules/networking-bird.mdc](../.cursor/rules/networking-bird.mdc) | BIRD2, BGP policy, IP/CIDR |
| [../.cursor/rules/conventional-commits.mdc](../.cursor/rules/conventional-commits.mdc) | Conventional Commits (заголовок EN, тело RU) |
### Сообщения коммитов (Cursor)
После `git add`:
| Способ | Skill + rule + `staged-context.ps1` |
|--------|-------------------------------------|
| Agent → **`/commit-message`** или «сгенерируй коммит по staged» | **Да** |
| Кнопка **✨ Generate commit message** в Source Control | **Нет** (только diff + история; [ограничение Cursor](https://forum.cursor.com/t/how-to-set-prompt-for-generate-commit-message/148606)) |
Рекомендуемый workflow: Agent Chat → **`/commit-message`** ([команда](../.cursor/commands/commit-message.md), [скилл](../.cursor/skills/commit-message/SKILL.md), [правило](../.cursor/rules/conventional-commits.mdc)). Агент **обязан первым делом** запустить `scripts/commit/staged-context.ps1` (заголовок EN, тело RU, auto-split).
Просмотр групп вручную: `powershell -NoProfile -File scripts/commit/staged-context.ps1 | ConvertFrom-Json`.
## Репозиторий и CI ## Репозиторий и CI
- [../.gitea/README.md](../.gitea/README.md) — Gitea Actions, runner, сборка образов и push в Container Registry. - [../.gitea/README.md](../.gitea/README.md) — Gitea Actions, runner, сборка образов и push в Container Registry.
- [releasing.md](releasing.md) — semantic-release, commit conventions, теги образов.
+22 -3
View File
@@ -22,6 +22,19 @@ opkey|01ARZ3NDEKTSV4RRFFQ69G5FAV|operator,nodekey|01ARZ3NDEKTSV4RRFFQ69G5FAV|nod
При включённом демо-сиде сервер при старте может вывести в лог готовую подсказку с реальным `tenant_id` из БД — см. лог `evobgp-api` / `evobgp-all`. При включённом демо-сиде сервер при старте может вывести в лог готовую подсказку с реальным `tenant_id` из БД — см. лог `evobgp-api` / `evobgp-all`.
Ключи из `EVOBGP_API_KEYS` загружаются при старте и **дополняют** ключи из таблицы `api_key` в БД (break-glass / bootstrap). После первого operator-ключа можно создавать остальные через API или веб-настройки.
### Управление через API и UI
При подключённой БД operator может:
- `GET|POST /v1/api-keys`, `GET|PATCH|DELETE /v1/api-keys/{id}`, `POST /v1/api-keys/{id}/rotate` — см. OpenAPI, тег **API keys**.
- В веб-панели: **Права доступа** (`/access`) → блок «API-ключи» (только для роли `operator`). Токен для браузера — в **Настройки** (`/settings`).
Полный токен возвращается **один раз** в ответе `201` (создание) и `200` (ротация). В списках — только `prefix` (первые 8 символов). В БД хранится SHA-256 токена, не plaintext.
`GET /v1/auth/session` — текущие `tenant_id` и `role` (для UI).
### Роли ### Роли
| Роль | Уровень | Назначение | | Роль | Уровень | Назначение |
@@ -33,11 +46,16 @@ opkey|01ARZ3NDEKTSV4RRFFQ69G5FAV|operator,nodekey|01ARZ3NDEKTSV4RRFFQ69G5FAV|nod
Обратное ограничение: для эндпоинтов ноды требуется именно роль **`node`**; остальные роли получают отказ. Обратное ограничение: для эндпоинтов ноды требуется именно роль **`node`**; остальные роли получают отказ.
### Режим разработки `EVOBGP_DEV_INSECURE` ### Токен `dev` (локальная разработка)
Если установлено `EVOBGP_DEV_INSECURE=1` и в store доступен демо-tenant (`DemoIDs`), то запрос с заголовком **`Authorization: Bearer dev`** получает контекст **`operator`** для этого tenant. Если в store доступен демо-tenant (`DemoIDs`, обычно `EVOBGP_SEED_DEMO` не равен `0`), заголовок **`Authorization: Bearer dev`** даёт роль **`operator`** для этого tenant. **Не зависит** от `EVOBGP_DEV_INSECURE`.
**Запрещено** в продакшене: любой, кто знает заголовок, получает полные права оператора на демо-данные. **Запрещено** в продакшене: не оставляйте demo-seed с известным токеном `dev` на боевых данных. Переменная `EVOBGP_DEV_INSECURE` в текущей версии **не влияет** на аутентификацию (оставлена в compose для совместимости; не включайте в production — см. SEC-02 в инженерных правилах).
### Синхронные «тяжёлые» GET (control plane)
- `POST /v1/modules/{module_id}/cdn-sources/preview` — загрузка CDN в том же HTTP-запросе (лимит тела ~8 MiB, см. OpenAPI).
- `GET /v1/bird/status` (если маршрут включён в деплое) — опрос локального `birdc`, таймаут сервера ~12 с.
### Детерминированный ключ подписи бандлов (тесты) ### Детерминированный ключ подписи бандлов (тесты)
@@ -88,6 +106,7 @@ http://localhost:5173,http://127.0.0.1:5173,https://ui.example.com
| GET модули, ревизии, peers, speakers | да | да | да | нет | | GET модули, ревизии, peers, speakers | да | да | да | нет |
| POST/PATCH/DELETE CRUD сущностей | нет | да | да | нет | | POST/PATCH/DELETE CRUD сущностей | нет | да | да | нет |
| apply, rollback, PATCH settings | нет | нет | да | нет | | apply, rollback, PATCH settings | нет | нет | да | нет |
| Управление API-ключами (`/v1/api-keys`) | нет | нет | да | нет |
| bundle, latest revision, enroll | нет | нет | нет | да | | bundle, latest revision, enroll | нет | нет | нет | да |
Точные проверки по каждому маршруту — в коде `internal/httpapi` и в схеме безопасности операций в OpenAPI. Точные проверки по каждому маршруту — в коде `internal/httpapi` и в схеме безопасности операций в OpenAPI.
+6
View File
@@ -44,6 +44,12 @@
- `GET|POST /v1/communities` - `GET|POST /v1/communities`
- `GET|PATCH|DELETE /v1/communities/{id}` - `GET|PATCH|DELETE /v1/communities/{id}`
### API keys
- `GET /v1/auth/session` — tenant и роль текущего ключа
- `GET|POST /v1/api-keys` — список и создание (operator)
- `GET|PATCH|DELETE /v1/api-keys/{id}`, `POST /v1/api-keys/{id}/rotate`
### Peers ### Peers
- `GET /v1/peers`, `POST /v1/peers` - `GET /v1/peers`, `POST /v1/peers`
+330 -4
View File
@@ -1,10 +1,12 @@
openapi: 3.1.0 openapi: 3.1.0
info: info:
title: EvoBGP Control Plane API title: EvoBGP Control Plane API
version: 0.1.0 version: 1.0.0
description: | description: |
REST API управления префиксами, модулями ingest, ревизиями конфигурации BIRD и задачами (async jobs). REST API управления префиксами, модулями ingest, ревизиями конфигурации BIRD и задачами (async jobs).
**Актуальная semver-сборка:** `GET /version` или `GET /v1/version` (поле `version`; совпадает с git-тегом `vX.Y.Z`).
**Соглашения:** префикс `/v1`; идентификаторы - UUID v7 или ULID (строки); время - ISO 8601 UTC. **Соглашения:** префикс `/v1`; идентификаторы - UUID v7 или ULID (строки); время - ISO 8601 UTC.
Ошибки - `application/problem+json` ([RFC 9457](https://www.rfc-editor.org/rfc/rfc9457)). Ошибки - `application/problem+json` ([RFC 9457](https://www.rfc-editor.org/rfc/rfc9457)).
Пагинация списков - `cursor` + `limit`; ответ содержит `items`, `next_cursor`, `has_more`. Пагинация списков - `cursor` + `limit`; ответ содержит `items`, `next_cursor`, `has_more`.
@@ -43,6 +45,10 @@ tags:
description: "API для evobgp-node (бандлы ревизий и enrollment). Отдельный ключ или mTLS, роль node." description: "API для evobgp-node (бандлы ревизий и enrollment). Отдельный ключ или mTLS, роль node."
- name: Settings - name: Settings
description: Глобальные настройки и feature flags; изменение - только operator. description: Глобальные настройки и feature flags; изменение - только operator.
- name: API keys
description: Управление API-ключами tenant (operator). Секрет возвращается только при создании и ротации.
- name: Auth
description: Сессия текущего API-ключа (tenant и роль).
security: security:
- bearerAuth: [] - bearerAuth: []
@@ -65,7 +71,9 @@ components:
required: false required: false
schema: schema:
$ref: "#/components/schemas/ResourceId" $ref: "#/components/schemas/ResourceId"
description: Явный tenant (только супер-роли). Без заголовка tenant определяется по ключу. description: >
Явный tenant (только супер-роли). Без заголовка tenant определяется по API-ключу.
**Реализация v1:** заголовок в Go handlers не обрабатывается; tenant только из Bearer-токена (см. docs/access.md).
IdempotencyKey: IdempotencyKey:
name: Idempotency-Key name: Idempotency-Key
in: header in: header
@@ -131,6 +139,12 @@ components:
required: true required: true
schema: schema:
$ref: "#/components/schemas/ResourceId" $ref: "#/components/schemas/ResourceId"
ApiKeyId:
name: id
in: path
required: true
schema:
$ref: "#/components/schemas/ResourceId"
SourceId: SourceId:
name: source_id name: source_id
in: path in: path
@@ -295,6 +309,18 @@ components:
- DOMAINS - DOMAINS
- IP_RANGES - IP_RANGES
DohResolverPolicy:
type: string
description: |
Политика резолва доменов через DoH при нескольких профилях.
`primary_only` — только первый профиль; `failover` — по порядку до первого успешного;
`union` — объединение A/AAAA со всех профилей.
enum:
- primary_only
- failover
- union
default: primary_only
JobStatus: JobStatus:
type: string type: string
description: Статус задачи; перечень может расширяться. description: Статус задачи; перечень может расширяться.
@@ -332,6 +358,14 @@ components:
type: integer type: integer
doh_profile_id: doh_profile_id:
type: ["string", "null"] type: ["string", "null"]
description: Первый DoH-профиль (legacy); предпочтительно `doh_profile_ids`.
doh_profile_ids:
type: array
items:
$ref: "#/components/schemas/ResourceId"
description: Упорядоченный список DoH-профилей для модулей `DOMAINS`.
doh_resolver_policy:
$ref: "#/components/schemas/DohResolverPolicy"
refresh_interval_sec: refresh_interval_sec:
type: ["integer", "null"] type: ["integer", "null"]
minimum: 0 minimum: 0
@@ -363,6 +397,14 @@ components:
default: 0 default: 0
doh_profile_id: doh_profile_id:
type: ["string", "null"] type: ["string", "null"]
description: Первый DoH-профиль (legacy); предпочтительно `doh_profile_ids`.
doh_profile_ids:
type: array
items:
$ref: "#/components/schemas/ResourceId"
description: Упорядоченный список DoH-профилей для модулей `DOMAINS`.
doh_resolver_policy:
$ref: "#/components/schemas/DohResolverPolicy"
refresh_interval_sec: refresh_interval_sec:
type: ["integer", "null"] type: ["integer", "null"]
last_refreshed_at: last_refreshed_at:
@@ -385,6 +427,14 @@ components:
type: integer type: integer
doh_profile_id: doh_profile_id:
type: ["string", "null"] type: ["string", "null"]
description: Первый DoH-профиль (legacy); предпочтительно `doh_profile_ids`.
doh_profile_ids:
type: array
items:
$ref: "#/components/schemas/ResourceId"
description: Упорядоченный список DoH-профилей для модулей `DOMAINS`.
doh_resolver_policy:
$ref: "#/components/schemas/DohResolverPolicy"
refresh_interval_sec: refresh_interval_sec:
type: ["integer", "null"] type: ["integer", "null"]
cron_expr: cron_expr:
@@ -570,6 +620,9 @@ components:
properties: properties:
id: id:
$ref: "#/components/schemas/ResourceId" $ref: "#/components/schemas/ResourceId"
name:
type: string
description: Отображаемое имя профиля (опционально).
url: url:
type: string type: string
format: uri format: uri
@@ -585,6 +638,9 @@ components:
required: required:
- url - url
properties: properties:
name:
type: string
description: Отображаемое имя профиля (опционально).
url: url:
type: string type: string
format: uri format: uri
@@ -593,6 +649,82 @@ components:
vault_secret_ref: vault_secret_ref:
type: ["string", "null"] type: ["string", "null"]
AuthSession:
type: object
required: [tenant_id, role]
properties:
tenant_id:
$ref: "#/components/schemas/ResourceId"
role:
type: string
enum: [viewer, editor, operator, node]
ApiKey:
type: object
required: [id, name, role, prefix, created_at, updated_at]
properties:
id:
$ref: "#/components/schemas/ResourceId"
name:
type: string
role:
type: string
enum: [viewer, editor, operator, node]
prefix:
type: string
description: Первые 8 символов токена для идентификации в UI.
created_at:
type: string
format: date-time
updated_at:
type: string
format: date-time
expires_at:
type: ["string", "null"]
format: date-time
revoked_at:
type: ["string", "null"]
format: date-time
last_used_at:
type: ["string", "null"]
format: date-time
additionalProperties: true
ApiKeyCreate:
type: object
required: [name, role]
properties:
name:
type: string
role:
type: string
enum: [viewer, editor, operator, node]
expires_at:
type: ["string", "null"]
format: date-time
ApiKeyPatch:
type: object
properties:
name:
type: string
role:
type: string
enum: [viewer, editor, operator, node]
expires_at:
type: ["string", "null"]
format: date-time
ApiKeyCreated:
allOf:
- $ref: "#/components/schemas/ApiKey"
- type: object
required: [token]
properties:
token:
type: string
description: Полный Bearer-токен; показывается один раз.
BgpCommunity: BgpCommunity:
type: object type: object
required: required:
@@ -716,8 +848,14 @@ components:
VersionInfo: VersionInfo:
type: object type: object
properties: properties:
version:
type: string
description: Semver сборки (git tag без префикса v).
example: "1.2.3"
api_version: api_version:
type: string type: string
deprecated: true
description: Alias поля `version` (сохранён для обратной совместимости).
git_sha: git_sha:
type: string type: string
build_time: build_time:
@@ -804,6 +942,8 @@ components:
DohProfilePatch: DohProfilePatch:
type: object type: object
properties: properties:
name:
type: string
url: url:
type: string type: string
format: uri format: uri
@@ -923,11 +1063,31 @@ paths:
default: default:
$ref: "#/components/responses/DefaultProblem" $ref: "#/components/responses/DefaultProblem"
/version:
get:
tags: [System]
summary: Версия сборки (корневой путь)
description: |
Аналог `GET /v1/version`. Публичный маршрут без аутентификации.
Semver в поле `version` задаётся при сборке Docker-образов из git-тега.
operationId: getVersionRoot
responses:
"200":
description: Метаданные сборки.
content:
application/json:
schema:
$ref: "#/components/schemas/VersionInfo"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/version: /v1/version:
get: get:
tags: [System] tags: [System]
summary: Версия сборки summary: Версия сборки
description: Версия API и control-plane (`git_sha`, `build_time` и др.). description: |
Semver control-plane и метаданные сборки (`git_sha`, `build_time`).
Дублирует `GET /version`.
operationId: getVersion operationId: getVersion
responses: responses:
"200": "200":
@@ -1739,7 +1899,9 @@ paths:
tags: [Modules] tags: [Modules]
summary: Предпросмотр префиксов из CDN-источника summary: Предпросмотр префиксов из CDN-источника
description: > description: >
Загружает URL, парсит как plaintext или json и возвращает список извлечённых префиксов (до 100 записей). Синхронный запрос: conditional GET к URL (до 8 MiB тела ответа), парсинг plaintext или JSON,
возврат до 100 префиксов в `items` (полный счётчик в `total`). Выполняется в HTTP worker;
при таймауте клиента используйте короткий URL или меньший payload.
operationId: previewCdnSource operationId: previewCdnSource
requestBody: requestBody:
required: true required: true
@@ -2567,6 +2729,170 @@ paths:
default: default:
$ref: "#/components/responses/DefaultProblem" $ref: "#/components/responses/DefaultProblem"
/v1/auth/session:
get:
tags: [Auth]
summary: Текущая сессия API-ключа
operationId: getAuthSession
parameters:
- $ref: "#/components/parameters/TenantId"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/AuthSession"
"401":
$ref: "#/components/responses/Unauthorized"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/api-keys:
get:
tags: [API keys]
summary: Список API-ключей tenant
description: Только роль **operator**. Секреты не возвращаются.
operationId: listApiKeys
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/Cursor"
- $ref: "#/components/parameters/Limit"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
type: object
required: [items, has_more]
properties:
items:
type: array
items:
$ref: "#/components/schemas/ApiKey"
next_cursor:
type: ["string", "null"]
has_more:
type: boolean
"403":
$ref: "#/components/responses/Forbidden"
default:
$ref: "#/components/responses/DefaultProblem"
post:
tags: [API keys]
summary: Создать API-ключ
operationId: createApiKey
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/ApiKeyCreate"
responses:
"201":
description: Ключ создан; token в ответе один раз.
content:
application/json:
schema:
$ref: "#/components/schemas/ApiKeyCreated"
"403":
$ref: "#/components/responses/Forbidden"
"422":
$ref: "#/components/responses/UnprocessableEntity"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/api-keys/{id}:
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/ApiKeyId"
get:
tags: [API keys]
summary: Получить метаданные API-ключа
operationId: getApiKey
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/ApiKey"
"403":
$ref: "#/components/responses/Forbidden"
"404":
$ref: "#/components/responses/NotFound"
default:
$ref: "#/components/responses/DefaultProblem"
patch:
tags: [API keys]
summary: Обновить API-ключ
operationId: patchApiKey
parameters:
- $ref: "#/components/parameters/IdempotencyKey"
requestBody:
required: true
content:
application/json:
schema:
$ref: "#/components/schemas/ApiKeyPatch"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/ApiKey"
"403":
$ref: "#/components/responses/Forbidden"
"404":
$ref: "#/components/responses/NotFound"
default:
$ref: "#/components/responses/DefaultProblem"
delete:
tags: [API keys]
summary: Отозвать API-ключ
operationId: revokeApiKey
parameters:
- $ref: "#/components/parameters/IdempotencyKey"
responses:
"204":
description: Отозван.
"403":
$ref: "#/components/responses/Forbidden"
"404":
$ref: "#/components/responses/NotFound"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/api-keys/{id}/rotate:
parameters:
- $ref: "#/components/parameters/TenantId"
- $ref: "#/components/parameters/ApiKeyId"
post:
tags: [API keys]
summary: Ротировать секрет API-ключа
description: Выдаёт новый token; старый перестаёт работать сразу.
operationId: rotateApiKey
parameters:
- $ref: "#/components/parameters/IdempotencyKey"
responses:
"200":
description: Успешно.
content:
application/json:
schema:
$ref: "#/components/schemas/ApiKeyCreated"
"403":
$ref: "#/components/responses/Forbidden"
"404":
$ref: "#/components/responses/NotFound"
default:
$ref: "#/components/responses/DefaultProblem"
/v1/settings: /v1/settings:
get: get:
tags: [Settings] tags: [Settings]
+78
View File
@@ -0,0 +1,78 @@
# Релизы и версионирование EvoBGP
EvoBGP использует [Conventional Commits](https://www.conventionalcommits.org/) и [semantic-release](https://semantic-release.gitbook.io/) для полностью автоматических релизов на Gitea (`git.shts.su`). Ручное повышение версии в коде не требуется.
## Как определяется версия
| Тип коммита | Bump |
|-------------|------|
| `feat` | minor (1.0.0 → 1.1.0) |
| `fix`, `perf`, `ci`, `refactor` | patch (1.5.1 → 1.5.2) |
| `feat!`, `fix!` или `BREAKING CHANGE:` в теле | major (1.0.0 → 2.0.0) |
| `docs`, `chore`, `test` | без релиза |
`refactor` — patch без новых функций: перестройка кода/UI при том же поведении для пользователя. По semver на одном уровне с `fix`, но семантически «мельче» `feat` (не minor).
Отдельного суффикса `1.x.y.fix` в semver нет: «fix» в Conventional Commits означает **patch** (третья цифра). Для починки пайплайна без смены продукта — `fix(ci):` или `ci:` (оба дают patch после настройки `.releaserc.json`).
Первый релиз при отсутствии git-тегов — **1.0.0**, если есть releasable-коммиты.
**Как не перепутать `feat` и `fix`:** см. раздел «Выбор type: semver, а не «красивые слова»» в [.cursor/rules/conventional-commits.mdc](../.cursor/rules/conventional-commits.mdc). Кратко: `feat` — новая возможность (minor); `fix` — починка ожидаемого поведения (patch); follow-up баги после недавнего `feat` — всегда `fix`, даже если diff большой.
Подробные правила сообщений коммитов: [.cursor/rules/conventional-commits.mdc](../.cursor/rules/conventional-commits.mdc).
## CI-пайплайн (один push в main)
```text
push/merge в main
→ CI: openapi, web, go, bird2 (параллельно)
→ job release (в том же workflow, после quality gates):
→ semantic-release: git tag vX.Y.Z на текущий commit (без доп. commit)
→ Gitea Release + CHANGELOG.md как attachment
→ docker buildx bake с VERSION=X.Y.Z
→ образы: latest, vX.Y.Z, X.Y.Z, sha-*, короткий SHA
```
Pull request: только quality gates + commitlint; релиз и образы **не** публикуются.
Workflow: [.gitea/workflows/ci.yaml](../.gitea/workflows/ci.yaml) (job **release**).
Конфиг semantic-release: [.releaserc.json](../.releaserc.json) — без `@semantic-release/git` (CHANGELOG не коммитится в репозиторий).
## Секреты Gitea
Один PAT — **`ACTIONS_PAT`** (Settings → Actions → Secrets).
| Право PAT | Зачем |
|-----------|--------|
| push tags | git tag `vX.Y.Z` на commit merge |
| releases | Gitea Release + notes |
| packages (Container Registry) | push образов |
Fallback: **`gitea.token`** (нужны права на releases и packages).
## Источник правды для версии в runtime
Semver из git-тега пробрасывается в Go-бинарники через `-ldflags` при сборке Docker (`deploy/docker/gobinary/Dockerfile`). Пакет [`internal/version`](../internal/version/version.go):
- локально (`go run`) — `version: "dev"`
- в образе после релиза — совпадает с тегом (например `1.2.3`)
API: `GET /version`, `GET /v1/version` — поля `version`, `git_sha`, `build_time`.
Web UI показывает версию из API (footer sidebar, страница «Мониторинг»).
## CHANGELOG
Release notes — в Gitea Release; файл `CHANGELOG.md` генерируется в CI и прикрепляется как asset, **не** попадает в git history.
## Проверка после релиза
1. Один run workflow **CI** на push в main: job **release** зелёный.
2. Gitea: тег `vX.Y.Z` на том же commit, что и merge; Release с notes.
3. Container Registry: `evobgp-api:vX.Y.Z`, `evobgp-api:X.Y.Z`, `evobgp-api:latest`.
4. `curl http://localhost:8080/version` → `"version":"X.Y.Z"`.
## Первый релиз (bootstrap)
Merge в `main` с `feat(release): ...` → **v1.0.0** в том же CI run.
+2 -2
View File
@@ -42,7 +42,7 @@ func AnnouncedPrefixes(ctx context.Context, hc *http.Client, asn int64) ([]netip
if err != nil { if err != nil {
return nil, fmt.Errorf("ripestat fetch AS%d: %w", asn, err) return nil, fmt.Errorf("ripestat fetch AS%d: %w", asn, err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, 32<<20)) body, err := io.ReadAll(io.LimitReader(resp.Body, 32<<20))
if err != nil { if err != nil {
return nil, err return nil, err
@@ -104,7 +104,7 @@ func ASHolderName(ctx context.Context, hc *http.Client, asn int64) (string, erro
if err != nil { if err != nil {
return "", fmt.Errorf("ripestat as-overview AS%d: %w", asn, err) return "", fmt.Errorf("ripestat as-overview AS%d: %w", asn, err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
body, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20)) body, err := io.ReadAll(io.LimitReader(resp.Body, 4<<20))
if err != nil { if err != nil {
return "", err return "", err
+34
View File
@@ -0,0 +1,34 @@
// Package authkey generates API tokens and derives lookup hashes (no persistence).
package authkey
import (
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"fmt"
)
const tokenPrefix = "evobgp_"
// GenerateToken returns a new bearer token (evobgp_ + 32 random bytes, base64url).
func GenerateToken() (string, error) {
b := make([]byte, 32)
if _, err := rand.Read(b); err != nil {
return "", fmt.Errorf("authkey: generate token: %w", err)
}
return tokenPrefix + base64.RawURLEncoding.EncodeToString(b), nil
}
// HashToken returns SHA-256 of the full token (32 bytes).
func HashToken(token string) []byte {
sum := sha256.Sum256([]byte(token))
return sum[:]
}
// Prefix returns the first 8 characters of the token for display.
func Prefix(token string) string {
if len(token) <= 8 {
return token
}
return token[:8]
}
+2
View File
@@ -1,3 +1,5 @@
// Engineering rules for BIRD/BGP/IP: .cursor/rules/networking-bird.mdc
//
// Package birdfmt builds BIRD 2 configuration text: main bird.conf skeleton, include // Package birdfmt builds BIRD 2 configuration text: main bird.conf skeleton, include
// fragments under bird.d/, export filters, static route protocols, BGP templates, and // fragments under bird.d/, export filters, static route protocols, BGP templates, and
// BGP peer protocols (from template or standalone). Runtime helpers parse-check configs // BGP peer protocols (from template or standalone). Runtime helpers parse-check configs
+17 -6
View File
@@ -9,6 +9,21 @@ import (
const maxBGPASN = 4294967295 const maxBGPASN = 4294967295
const filterPrefixChunkSize = 500
func writePrefixSetAcceptBlocks(b *strings.Builder, keys []string) {
for i := 0; i < len(keys); i += filterPrefixChunkSize {
end := i + filterPrefixChunkSize
if end > len(keys) {
end = len(keys)
}
chunk := keys[i:end]
b.WriteString(" if net ~ [ ")
b.WriteString(strings.Join(chunk, ", "))
b.WriteString(" ] then accept;\n")
}
}
func filterUniqueASNs(pathASNs []int64) []int64 { func filterUniqueASNs(pathASNs []int64) []int64 {
seen := make(map[int64]struct{}) seen := make(map[int64]struct{})
for _, a := range pathASNs { for _, a := range pathASNs {
@@ -52,9 +67,7 @@ func RenderExportFilterIPv4(filterName string, prefixes []netip.Prefix, pathASNs
b.WriteString(strings.TrimSpace(filterName)) b.WriteString(strings.TrimSpace(filterName))
b.WriteString(" {\n") b.WriteString(" {\n")
if len(keys) > 0 { if len(keys) > 0 {
b.WriteString(" if net ~ [ ") writePrefixSetAcceptBlocks(&b, keys)
b.WriteString(strings.Join(keys, ", "))
b.WriteString(" ] then accept;\n")
} }
for _, asn := range asns { for _, asn := range asns {
fmt.Fprintf(&b, " if bgp_path ~ [= * %d =] then accept;\n", asn) fmt.Fprintf(&b, " if bgp_path ~ [= * %d =] then accept;\n", asn)
@@ -95,9 +108,7 @@ func RenderExportFilterIPv6(filterName string, prefixes []netip.Prefix, pathASNs
b.WriteString(strings.TrimSpace(filterName)) b.WriteString(strings.TrimSpace(filterName))
b.WriteString(" {\n") b.WriteString(" {\n")
if len(keys) > 0 { if len(keys) > 0 {
b.WriteString(" if net ~ [ ") writePrefixSetAcceptBlocks(&b, keys)
b.WriteString(strings.Join(keys, ", "))
b.WriteString(" ] then accept;\n")
} }
for _, asn := range asns { for _, asn := range asns {
fmt.Fprintf(&b, " if bgp_path ~ [= * %d =] then accept;\n", asn) fmt.Fprintf(&b, " if bgp_path ~ [= * %d =] then accept;\n", asn)
+45
View File
@@ -50,3 +50,48 @@ func CountEstablishedBGPSessions(showProtocolsOutput string) int {
} }
return n return n
} }
// ParseBGPProtocolStates parses `birdc show protocols all` summary rows into protocol_name -> state.
func ParseBGPProtocolStates(output string) map[string]string {
out := make(map[string]string)
for _, raw := range strings.Split(output, "\n") {
line := strings.TrimSpace(raw)
if line == "" {
continue
}
low := strings.ToLower(line)
if strings.HasPrefix(low, "bird ") || strings.HasPrefix(low, "name ") || strings.HasPrefix(low, "table ") {
continue
}
fields := strings.Fields(line)
if len(fields) < 4 {
continue
}
if !strings.EqualFold(fields[1], "BGP") {
continue
}
state := extractBGPSessionStateLine(line)
if state == "" {
state = fields[3]
}
out[fields[0]] = state
}
return out
}
func extractBGPSessionStateLine(line string) string {
known := []string{
"Established",
"Idle",
"Connect",
"Active",
"OpenSent",
"OpenConfirm",
}
for _, st := range known {
if strings.Contains(line, st) {
return st
}
}
return ""
}
+1 -1
View File
@@ -27,7 +27,7 @@ func VerifyGzippedTar(bundle []byte, pub ed25519.PublicKey) (*VerifiedContents,
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer gr.Close() defer func() { _ = gr.Close() }()
var manifestRaw []byte var manifestRaw []byte
var sig []byte var sig []byte
+14
View File
@@ -6,9 +6,12 @@ import (
"errors" "errors"
"fmt" "fmt"
"io/fs" "io/fs"
"os"
"path" "path"
"sort" "sort"
"strconv"
"strings" "strings"
"time"
"evobgp/migrations" "evobgp/migrations"
@@ -22,6 +25,17 @@ func OpenPostgresPool(ctx context.Context, dsn string) (*pgxpool.Pool, error) {
if err != nil { if err != nil {
return nil, err return nil, err
} }
if max := os.Getenv("EVOBGP_DB_MAX_CONNS"); max != "" {
if n, err := strconv.Atoi(strings.TrimSpace(max)); err == nil && n > 0 {
cfg.MaxConns = int32(n)
}
}
if min := os.Getenv("EVOBGP_DB_MIN_CONNS"); min != "" {
if n, err := strconv.Atoi(strings.TrimSpace(min)); err == nil && n >= 0 {
cfg.MinConns = int32(n)
}
}
cfg.MaxConnLifetime = 30 * time.Minute
pool, err := pgxpool.NewWithConfig(ctx, cfg) pool, err := pgxpool.NewWithConfig(ctx, cfg)
if err != nil { if err != nil {
return nil, err return nil, err
+66
View File
@@ -0,0 +1,66 @@
package httpapi
import (
"crypto/sha256"
"encoding/hex"
"sync"
"evobgp/internal/store"
)
type apiKeyResolver struct {
mu sync.RWMutex
envByToken map[string]apiKeyRecord
byHash map[string]apiKeyRecord
}
func newAPIKeyResolver(envSpec string, st store.Backend) (*apiKeyResolver, error) {
r := &apiKeyResolver{
envByToken: make(map[string]apiKeyRecord),
byHash: make(map[string]apiKeyRecord),
}
for _, rec := range parseAPIKeysSpec(envSpec) {
r.envByToken[rec.token] = rec
}
return r, r.reloadFromStore(st)
}
func (r *apiKeyResolver) reloadFromStore(st store.Backend) error {
rows, err := st.ListActiveAPIKeyHashes()
if err != nil {
return err
}
byHash := make(map[string]apiKeyRecord, len(rows))
for _, row := range rows {
if len(row.TokenHash) != 32 {
continue
}
byHash[hex.EncodeToString(row.TokenHash)] = apiKeyRecord{
token: "",
tenantID: row.TenantID,
role: row.Role,
keyID: row.ID,
}
}
r.mu.Lock()
r.byHash = byHash
r.mu.Unlock()
return nil
}
func (r *apiKeyResolver) Reload(st store.Backend) error {
return r.reloadFromStore(st)
}
func (r *apiKeyResolver) Lookup(raw string) (apiKeyRecord, bool) {
r.mu.RLock()
defer r.mu.RUnlock()
if rec, ok := r.envByToken[raw]; ok {
return rec, true
}
sum := sha256.Sum256([]byte(raw))
key := hex.EncodeToString(sum[:])
rec, ok := r.byHash[key]
return rec, ok
}
+13 -21
View File
@@ -15,6 +15,7 @@ type Auth struct {
TenantID string TenantID string
Role string // viewer, editor, operator, node Role string // viewer, editor, operator, node
Token string Token string
APIKeyID string // non-empty for DB-managed keys
} }
func authFromContext(ctx context.Context) (Auth, bool) { func authFromContext(ctx context.Context) (Auth, bool) {
@@ -26,6 +27,7 @@ type apiKeyRecord struct {
token string token string
tenantID string tenantID string
role string role string
keyID string // set for DB-managed keys (last_used_at)
} }
func parseAPIKeysSpec(spec string) []apiKeyRecord { func parseAPIKeysSpec(spec string) []apiKeyRecord {
@@ -54,20 +56,6 @@ func parseAPIKeysSpec(spec string) []apiKeyRecord {
func (s *Server) authMiddleware(next http.Handler) http.Handler { func (s *Server) authMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) { return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if s.insecureDev {
h := r.Header.Get("Authorization")
const p = "Bearer "
if strings.HasPrefix(h, p) {
tok := strings.TrimSpace(strings.TrimPrefix(h, p))
if tok == "dev" {
if a, ok := s.devAuth(); ok {
r = r.WithContext(context.WithValue(r.Context(), authCtxKey, a))
next.ServeHTTP(w, r)
return
}
}
}
}
h := r.Header.Get("Authorization") h := r.Header.Get("Authorization")
const p = "Bearer " const p = "Bearer "
if !strings.HasPrefix(h, p) { if !strings.HasPrefix(h, p) {
@@ -75,18 +63,22 @@ func (s *Server) authMiddleware(next http.Handler) http.Handler {
return return
} }
raw := strings.TrimSpace(strings.TrimPrefix(h, p)) raw := strings.TrimSpace(strings.TrimPrefix(h, p))
var matched *apiKeyRecord if raw == "dev" {
for i := range s.apiKeys { if a, ok := s.devAuth(); ok {
if s.apiKeys[i].token == raw { r = r.WithContext(context.WithValue(r.Context(), authCtxKey, a))
matched = &s.apiKeys[i] next.ServeHTTP(w, r)
break return
} }
} }
if matched == nil { matched, ok := s.keyResolver.Lookup(raw)
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "unknown api key") writeProblem(w, http.StatusUnauthorized, "Unauthorized", "unknown api key")
return return
} }
a := Auth{TenantID: matched.tenantID, Role: matched.role, Token: raw} a := Auth{TenantID: matched.tenantID, Role: matched.role, Token: raw, APIKeyID: matched.keyID}
if matched.keyID != "" {
go func(id string) { _ = s.store.TouchAPIKeyLastUsed(id) }(matched.keyID)
}
r = r.WithContext(context.WithValue(r.Context(), authCtxKey, a)) r = r.WithContext(context.WithValue(r.Context(), authCtxKey, a))
next.ServeHTTP(w, r) next.ServeHTTP(w, r)
}) })
+21 -1
View File
@@ -15,6 +15,11 @@ import (
"github.com/jackc/pgx/v5/pgxpool" "github.com/jackc/pgx/v5/pgxpool"
) )
// NewCDNHTTPClient returns the shared HTTP client for CDN and preview fetches (PERF-02 / ERR-03).
func NewCDNHTTPClient() *http.Client {
return &http.Client{Timeout: 45 * time.Second}
}
// BootstrapWorkers opens the same store.Backend and jobs.Registry as New (without HTTP or bundle keys). // BootstrapWorkers opens the same store.Backend and jobs.Registry as New (without HTTP or bundle keys).
// Used by standalone worker binaries (scheduler, ingest, …) that share PostgreSQL with the API. // Used by standalone worker binaries (scheduler, ingest, …) that share PostgreSQL with the API.
func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.Registry, *pgxpool.Pool, error) { func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.Registry, *pgxpool.Pool, error) {
@@ -44,10 +49,25 @@ func BootstrapWorkers(ctx context.Context, opts Options) (store.Backend, *jobs.R
backend = mem backend = mem
} }
cdnHTTP := &http.Client{Timeout: 45 * time.Second} cdnHTTP := NewCDNHTTPClient()
wk := &jobs.Worker{Store: backend, HTTPClient: cdnHTTP} wk := &jobs.Worker{Store: backend, HTTPClient: cdnHTTP}
reg := jobs.NewRegistry(wk.Process) reg := jobs.NewRegistry(wk.Process)
wk.Registry = reg wk.Registry = reg
if pool != nil {
audit := repository.NewJobAuditWriter(pool)
reg.SetTerminalHook(func(j *jobs.Job) {
if j == nil {
return
}
st := j.Snapshot()
status, _ := st["status"].(string)
var errMsg *string
if e, ok := st["error"].(string); ok && e != "" {
errMsg = &e
}
audit.MarkTerminal(context.Background(), j.TenantID, j.ID, status, errMsg, time.Now().UTC())
})
}
observability.RegisterStoreBackend(backend) observability.RegisterStoreBackend(backend)
return backend, reg, pool, nil return backend, reg, pool, nil
} }
+38
View File
@@ -0,0 +1,38 @@
package httpapi
import (
"net/http"
"strconv"
"evobgp/internal/store"
)
func parseListLimit(r *http.Request) int {
limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
if limit <= 0 {
return 50
}
if limit > 500 {
return 500
}
return limit
}
func strPtrOrNull(s string) any {
if s == "" {
return nil
}
return s
}
// writePaginatedListJSON returns a cursor/limit page as OpenAPI list envelopes (items, next_cursor, has_more).
func writePaginatedListJSON[T any](w http.ResponseWriter, r *http.Request, all []T, toItem func(T) map[string]any) {
page, next, more := store.PaginateOffset(all, r.URL.Query().Get("cursor"), parseListLimit(r))
items := make([]map[string]any, 0, len(page))
for _, x := range page {
items = append(items, toItem(x))
}
writeJSON(w, http.StatusOK, map[string]any{
"items": items, "next_cursor": strPtrOrNull(next), "has_more": more,
})
}
+26
View File
@@ -2,9 +2,19 @@ package httpapi
import ( import (
"encoding/json" "encoding/json"
"log"
"net/http" "net/http"
) )
const (
internalErrorDetail = "an internal error occurred"
badGatewayDetail = "upstream request failed"
notFoundDetail = "resource not found"
invalidInputDetail = "invalid request data"
cdnExtractDetail = "could not extract prefixes from source"
csvInvalidRowDetail = "invalid row in csv file"
)
// Problem is RFC 9457 application/problem+json. // Problem is RFC 9457 application/problem+json.
type Problem struct { type Problem struct {
Type string `json:"type,omitempty"` Type string `json:"type,omitempty"`
@@ -34,3 +44,19 @@ func writeJSON(w http.ResponseWriter, status int, v any) {
func writeNoContent(w http.ResponseWriter) { func writeNoContent(w http.ResponseWriter) {
w.WriteHeader(http.StatusNoContent) w.WriteHeader(http.StatusNoContent)
} }
// writeInternalError logs err server-side and returns a generic 500 problem (ERR-01).
func writeInternalError(w http.ResponseWriter, operation string, err error) {
if err != nil {
log.Printf("httpapi: %s: %v", operation, err)
}
writeProblem(w, http.StatusInternalServerError, "Internal Error", internalErrorDetail)
}
// writeBadGateway logs err server-side and returns a generic 502 problem (ERR-01).
func writeBadGateway(w http.ResponseWriter, operation string, err error) {
if err != nil {
log.Printf("httpapi: %s: %v", operation, err)
}
writeProblem(w, http.StatusBadGateway, "Bad Gateway", badGatewayDetail)
}
+135 -332
View File
@@ -19,7 +19,9 @@ import (
"evobgp/internal/jobs" "evobgp/internal/jobs"
"evobgp/internal/observability" "evobgp/internal/observability"
"evobgp/internal/pipeline" "evobgp/internal/pipeline"
"evobgp/internal/reports"
"evobgp/internal/store" "evobgp/internal/store"
"evobgp/internal/version"
) )
// Handler returns the root HTTP handler (system routes public; rest under /v1/ authenticated). // Handler returns the root HTTP handler (system routes public; rest under /v1/ authenticated).
@@ -29,6 +31,7 @@ func (s *Server) Handler() http.Handler {
wrappedV1 := http.StripPrefix("/v1", v1) wrappedV1 := http.StripPrefix("/v1", v1)
s.mux.Handle("GET /metrics", observability.MetricsHandler()) s.mux.Handle("GET /metrics", observability.MetricsHandler())
s.mux.HandleFunc("GET /version", s.handleVersion)
s.mux.HandleFunc("GET /v1/health", s.handleHealth) s.mux.HandleFunc("GET /v1/health", s.handleHealth)
s.mux.HandleFunc("GET /v1/ready", s.handleReady) s.mux.HandleFunc("GET /v1/ready", s.handleReady)
s.mux.HandleFunc("GET /v1/version", s.handleVersion) s.mux.HandleFunc("GET /v1/version", s.handleVersion)
@@ -66,6 +69,7 @@ func (s *Server) registerV1(m *http.ServeMux) {
m.HandleFunc("GET /bird/status", s.handleBirdStatus) m.HandleFunc("GET /bird/status", s.handleBirdStatus)
m.HandleFunc("GET /jobs", s.handleListJobs) m.HandleFunc("GET /jobs", s.handleListJobs)
m.HandleFunc("GET /jobs/{job_id}", s.handleGetJob) m.HandleFunc("GET /jobs/{job_id}", s.handleGetJob)
m.HandleFunc("GET /jobs/{job_id}/report", s.handleGetJobReport)
m.HandleFunc("POST /jobs/{job_id}/cancel", s.handleCancelJob) m.HandleFunc("POST /jobs/{job_id}/cancel", s.handleCancelJob)
m.HandleFunc("GET /speakers/{speaker_id}/revisions/latest", s.handleNodeLatestRevision) m.HandleFunc("GET /speakers/{speaker_id}/revisions/latest", s.handleNodeLatestRevision)
m.HandleFunc("GET /speakers/{speaker_id}/bundle/{revision_id}", s.handleNodeBundle) m.HandleFunc("GET /speakers/{speaker_id}/bundle/{revision_id}", s.handleNodeBundle)
@@ -81,12 +85,15 @@ func (s *Server) handleReady(w http.ResponseWriter, r *http.Request) {
checks := map[string]string{"store": "ok", "jobs": "memory"} checks := map[string]string{"store": "ok", "jobs": "memory"}
ctx, cancel := context.WithTimeout(r.Context(), 2*time.Second) ctx, cancel := context.WithTimeout(r.Context(), 2*time.Second)
defer cancel() defer cancel()
if err := s.store.Ping(ctx); err != nil {
checks["store"] = "unavailable"
if s.pgPool != nil { if s.pgPool != nil {
if err := s.pgPool.Ping(ctx); err != nil { checks["postgres"] = "unavailable"
checks["postgres"] = err.Error() }
writeJSON(w, http.StatusServiceUnavailable, map[string]any{"status": "not_ready", "checks": checks}) writeJSON(w, http.StatusServiceUnavailable, map[string]any{"status": "not_ready", "checks": checks})
return return
} }
if s.pgPool != nil {
checks["postgres"] = "ok" checks["postgres"] = "ok"
} else { } else {
checks["store_backend"] = "memory" checks["store_backend"] = "memory"
@@ -95,11 +102,16 @@ func (s *Server) handleReady(w http.ResponseWriter, r *http.Request) {
} }
func (s *Server) handleVersion(w http.ResponseWriter, r *http.Request) { func (s *Server) handleVersion(w http.ResponseWriter, r *http.Request) {
sha := strings.TrimSpace(os.Getenv("EVOBGP_GIT_SHA")) ver, sha, buildTime := version.Info()
if sha == "" { resp := map[string]string{
sha = "unknown" "version": ver,
"api_version": ver,
"git_sha": sha,
} }
writeJSON(w, http.StatusOK, map[string]string{"api_version": "0.1.0", "git_sha": sha}) if buildTime != "" {
resp["build_time"] = buildTime
}
writeJSON(w, http.StatusOK, resp)
} }
func moduleJSON(mod *store.Module) map[string]any { func moduleJSON(mod *store.Module) map[string]any {
@@ -122,6 +134,13 @@ func moduleJSON(mod *store.Module) map[string]any {
} else { } else {
m["default_community_id"] = nil m["default_community_id"] = nil
} }
ids := mod.EffectiveDohProfileIDs()
if len(ids) > 0 {
m["doh_profile_ids"] = ids
} else {
m["doh_profile_ids"] = []string{}
}
m["doh_resolver_policy"] = store.NormalizeDohResolverPolicy(mod.DohResolverPolicy)
if mod.DohProfileID != nil { if mod.DohProfileID != nil {
m["doh_profile_id"] = *mod.DohProfileID m["doh_profile_id"] = *mod.DohProfileID
} else { } else {
@@ -182,19 +201,39 @@ func (s *Server) handleListModules(w http.ResponseWriter, r *http.Request) {
enabledFilter = &v enabledFilter = &v
} }
mods := s.store.ListModules(a.TenantID) filtered := make([]*store.Module, 0)
items := make([]map[string]any, 0, len(mods)) limit := parseListLimit(r)
for _, mod := range mods { cursor := r.URL.Query().Get("cursor")
if typeFilter == "" && enabledFilter == nil {
page, next, more := s.store.ListModulesPage(a.TenantID, cursor, limit)
for _, mod := range page {
filtered = append(filtered, mod)
}
items := make([]map[string]any, 0, len(filtered))
for _, mod := range filtered {
items = append(items, moduleJSON(mod))
}
writeJSON(w, http.StatusOK, map[string]any{
"items": items, "next_cursor": strPtrOrNull(next), "has_more": more,
})
return
}
for _, mod := range s.store.ListModules(a.TenantID) {
if typeFilter != "" && mod.Type != typeFilter { if typeFilter != "" && mod.Type != typeFilter {
continue continue
} }
if enabledFilter != nil && mod.Enabled != *enabledFilter { if enabledFilter != nil && mod.Enabled != *enabledFilter {
continue continue
} }
filtered = append(filtered, mod)
}
page, next, more := store.PaginateOffset(filtered, r.URL.Query().Get("cursor"), parseListLimit(r))
items := make([]map[string]any, 0, len(page))
for _, mod := range page {
items = append(items, moduleJSON(mod)) items = append(items, moduleJSON(mod))
} }
writeJSON(w, http.StatusOK, map[string]any{ writeJSON(w, http.StatusOK, map[string]any{
"items": items, "next_cursor": nil, "has_more": false, "items": items, "next_cursor": strPtrOrNull(next), "has_more": more,
}) })
} }
@@ -207,91 +246,17 @@ func (s *Server) handleRouterListsCatalog(w http.ResponseWriter, r *http.Request
if !s.requireAtLeast(w, a, "viewer") { if !s.requireAtLeast(w, a, "viewer") {
return return
} }
cat, err := reports.BuildRouterListsCatalog(s.store, a.TenantID)
mods := s.store.ListModules(a.TenantID)
moduleItems := make([]map[string]any, 0, len(mods))
domains := make([]map[string]any, 0)
asns := make([]map[string]any, 0)
ipRanges := make([]map[string]any, 0)
for _, mod := range mods {
switch mod.Type {
case "DOMAINS", "AS_PREFIXES", "IP_RANGES":
moduleItems = append(moduleItems, moduleJSON(mod))
default:
continue
}
switch mod.Type {
case "DOMAINS":
list, err := s.store.ListDomainEntries(a.TenantID, mod.ID)
if err != nil { if err != nil {
writeStoreErr(w, err) writeStoreErr(w, err)
return return
} }
for _, x := range list {
domains = append(domains, map[string]any{
"module_id": mod.ID,
"entry": domainEntryJSON(x),
})
}
case "AS_PREFIXES":
list, err := s.store.ListASEntries(a.TenantID, mod.ID)
if err != nil {
writeStoreErr(w, err)
return
}
for _, x := range list {
asns = append(asns, map[string]any{
"module_id": mod.ID,
"entry": asEntryJSON(x),
})
}
case "IP_RANGES":
list, err := s.store.ListIPRangeEntries(a.TenantID, mod.ID)
if err != nil {
writeStoreErr(w, err)
return
}
for _, x := range list {
ipRanges = append(ipRanges, map[string]any{
"module_id": mod.ID,
"entry": ipRangeJSON(x),
})
}
}
}
comms, err := s.store.ListCommunities(a.TenantID)
if err != nil {
writeStoreErr(w, err)
return
}
communityItems := make([]map[string]any, 0, len(comms))
for _, c := range comms {
communityItems = append(communityItems, map[string]any{
"id": c.ID,
"community": c.Community,
"title": c.Title,
})
}
writeJSON(w, http.StatusOK, map[string]any{ writeJSON(w, http.StatusOK, map[string]any{
"modules": map[string]any{ "modules": map[string]any{"items": cat.Modules},
"items": moduleItems, "domains": map[string]any{"items": cat.Domains},
}, "asns": map[string]any{"items": cat.ASNs},
"domains": map[string]any{ "ip_ranges": map[string]any{"items": cat.IPRanges},
"items": domains, "communities": map[string]any{"items": cat.Communities},
},
"asns": map[string]any{
"items": asns,
},
"ip_ranges": map[string]any{
"items": ipRanges,
},
"communities": map[string]any{
"items": communityItems,
},
}) })
} }
@@ -310,7 +275,7 @@ func (s *Server) handleGetModule(w http.ResponseWriter, r *http.Request) {
writeProblem(w, http.StatusNotFound, "Not Found", "module not found") writeProblem(w, http.StatusNotFound, "Not Found", "module not found")
return return
} }
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
writeJSON(w, http.StatusOK, moduleJSON(mod)) writeJSON(w, http.StatusOK, moduleJSON(mod))
@@ -325,10 +290,11 @@ func (s *Server) handleListPeers(w http.ResponseWriter, r *http.Request) {
if !s.requireAtLeast(w, a, "viewer") { if !s.requireAtLeast(w, a, "viewer") {
return return
} }
peers := s.store.ListPeers(a.TenantID) allPeers := s.store.ListPeers(a.TenantID)
liveStates := s.liveBGPProtocolStates(r.Context()) page, next, more := store.PaginateOffset(allPeers, r.URL.Query().Get("cursor"), parseListLimit(r))
items := make([]map[string]any, 0, len(peers)) liveStates := s.liveBGPProtocolStates(r)
for _, p := range peers { items := make([]map[string]any, 0, len(page))
for _, p := range page {
row := peerJSON(p) row := peerJSON(p)
if st, ok := liveStates[peerProtocolNameForID(p.ID)]; ok && strings.TrimSpace(st) != "" { if st, ok := liveStates[peerProtocolNameForID(p.ID)]; ok && strings.TrimSpace(st) != "" {
row["session_state"] = strings.TrimSpace(st) row["session_state"] = strings.TrimSpace(st)
@@ -336,11 +302,21 @@ func (s *Server) handleListPeers(w http.ResponseWriter, r *http.Request) {
items = append(items, row) items = append(items, row)
} }
writeJSON(w, http.StatusOK, map[string]any{ writeJSON(w, http.StatusOK, map[string]any{
"items": items, "next_cursor": nil, "has_more": false, "items": items, "next_cursor": strPtrOrNull(next), "has_more": more,
}) })
} }
func (s *Server) liveBGPProtocolStates(ctx context.Context) map[string]string { func (s *Server) liveBGPProtocolStates(r *http.Request) map[string]string {
if r != nil && strings.EqualFold(strings.TrimSpace(r.URL.Query().Get("live")), "1") {
return s.liveBGPProtocolStatesFresh(r.Context())
}
if cached, ok := observability.CachedBirdProtocolStates(90 * time.Second); ok {
return cached
}
return s.liveBGPProtocolStatesFresh(r.Context())
}
func (s *Server) liveBGPProtocolStatesFresh(ctx context.Context) map[string]string {
sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET")) sock := strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET"))
if sock == "" { if sock == "" {
return map[string]string{} return map[string]string{}
@@ -349,7 +325,9 @@ func (s *Server) liveBGPProtocolStates(ctx context.Context) map[string]string {
if err != nil { if err != nil {
return map[string]string{} return map[string]string{}
} }
return parseBGPProtocolStates(out) states := birdfmt.ParseBGPProtocolStates(out)
observability.SetBirdProtocolStates(states)
return states
} }
// parseBGPProtocolStates parses `birdc show protocols all` summary rows into protocol_name -> state. // parseBGPProtocolStates parses `birdc show protocols all` summary rows into protocol_name -> state.
@@ -444,7 +422,7 @@ func (s *Server) handleModuleRefresh(w http.ResponseWriter, r *http.Request) {
writeProblem(w, http.StatusNotFound, "Not Found", "module not found") writeProblem(w, http.StatusNotFound, "Not Found", "module not found")
return return
} }
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
idem := r.Header.Get("Idempotency-Key") idem := r.Header.Get("Idempotency-Key")
@@ -456,7 +434,7 @@ func (s *Server) handleModuleRefresh(w http.ResponseWriter, r *http.Request) {
mid := mod.ID mid := mod.ID
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindModuleRefresh, idemPtr, &mid, map[string]any{"module_id": moduleID}) j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindModuleRefresh, idemPtr, &mid, map[string]any{"module_id": moduleID})
if err != nil { if err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
w.Header().Set("Location", "/v1/jobs/"+j.ID) w.Header().Set("Location", "/v1/jobs/"+j.ID)
@@ -505,7 +483,7 @@ func (s *Server) handleTenantRefresh(w http.ResponseWriter, r *http.Request) {
"trigger": "api", "trigger": "api",
}) })
if err != nil { if err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
w.Header().Set("Location", "/v1/jobs/"+j.ID) w.Header().Set("Location", "/v1/jobs/"+j.ID)
@@ -563,13 +541,6 @@ func revisionJSON(rev *store.Revision) map[string]any {
return m return m
} }
func strPtrOrNull(s string) any {
if s == "" {
return nil
}
return s
}
// enqueueModuleRefreshIfEnabled queues module_refresh when the module exists and is enabled (best-effort, no HTTP error). // enqueueModuleRefreshIfEnabled queues module_refresh when the module exists and is enabled (best-effort, no HTTP error).
func (s *Server) enqueueModuleRefreshIfEnabled(tenantID, moduleID, trigger string) { func (s *Server) enqueueModuleRefreshIfEnabled(tenantID, moduleID, trigger string) {
if s.jobs == nil { if s.jobs == nil {
@@ -580,7 +551,8 @@ func (s *Server) enqueueModuleRefreshIfEnabled(tenantID, moduleID, trigger strin
return return
} }
mid := moduleID mid := moduleID
_, _, _ = s.jobs.Enqueue(tenantID, jobs.KindModuleRefresh, nil, &mid, map[string]any{ key := "module_refresh:" + moduleID
_, _, _ = s.jobs.Enqueue(tenantID, jobs.KindModuleRefresh, &key, &mid, map[string]any{
"module_id": moduleID, "module_id": moduleID,
"trigger": trigger, "trigger": trigger,
}) })
@@ -647,6 +619,9 @@ func (s *Server) handleRevisionPreview(w http.ResponseWriter, r *http.Request) {
for k, v := range rev.PreviewFragments { for k, v := range rev.PreviewFragments {
obj[k] = v obj[k] = v
} }
if expanded := pipeline.BuildExpandedBirdPreview(rev.PreviewFragments); expanded != "" {
obj[pipeline.AuxBirdFullExpandedKey()] = expanded
}
writeJSON(w, http.StatusOK, obj) writeJSON(w, http.StatusOK, obj)
} }
@@ -665,225 +640,15 @@ func (s *Server) handleRevisionDiagnosticLog(w http.ResponseWriter, r *http.Requ
writeProblem(w, http.StatusNotFound, "Not Found", "revision not found") writeProblem(w, http.StatusNotFound, "Not Found", "revision not found")
return return
} }
logOut, err := reports.BuildRevisionDiagnosticLog(s.store, a.TenantID, rev)
moduleType := "" if err != nil {
moduleName := "" writeInternalError(w, "revision diagnostic log", err)
if strings.TrimSpace(rev.ModuleID) != "" { return
if mod, modErr := s.store.GetModule(a.TenantID, rev.ModuleID); modErr == nil && mod != nil {
moduleType = strings.TrimSpace(mod.Type)
moduleName = strings.TrimSpace(mod.Name)
} }
}
communityLabels := map[string]string{}
if communities, listErr := s.store.ListCommunities(a.TenantID); listErr == nil {
for _, c := range communities {
if c == nil {
continue
}
label := strings.TrimSpace(c.Title)
if label == "" {
label = strings.TrimSpace(c.Community)
}
if label == "" {
label = c.ID
}
communityLabels[c.ID] = label
}
}
type rawRow struct {
Prefix string
Source string
Kind string
SourceName string
SourceDetail string
CommunityID string
CommunityLabel string
}
type summary struct {
Kind string
Source string
SourceDetail string
CommunityID string
CommunityLabel string
Count int
Sample []string
}
cdnSourceURLByID := map[string]string{}
if moduleType == "CDN_CIDRS" && strings.TrimSpace(rev.ModuleID) != "" {
if sources, srcErr := s.store.ListCDNSources(a.TenantID, rev.ModuleID); srcErr == nil {
for _, src := range sources {
if src == nil {
continue
}
id := strings.TrimSpace(src.ID)
url := strings.TrimSpace(src.URL)
if id != "" && url != "" {
cdnSourceURLByID[id] = url
}
}
}
}
rows := make([]rawRow, 0, rev.MaterializedPrefixCount)
byGroup := map[string]*summary{}
cursor := ""
for {
page, next, more := s.store.ListRevisionPrefixes(a.TenantID, revID, cursor, 2000)
for _, p := range page {
kind, sourceName := classifyRevisionSource(p.Source)
sourceDetail := sourceName
if kind == "cdn" {
if url, ok := cdnSourceURLByID[sourceName]; ok && strings.TrimSpace(url) != "" {
sourceDetail = url
}
}
communityID := "none"
if p.CommunityID != nil && strings.TrimSpace(*p.CommunityID) != "" {
communityID = strings.TrimSpace(*p.CommunityID)
}
communityLabel := "без community"
if communityID != "none" {
if lbl, ok := communityLabels[communityID]; ok && strings.TrimSpace(lbl) != "" {
communityLabel = lbl
} else {
communityLabel = communityID
}
}
rows = append(rows, rawRow{
Prefix: p.Prefix,
Source: p.Source,
Kind: kind,
SourceName: sourceName,
SourceDetail: sourceDetail,
CommunityID: communityID,
CommunityLabel: communityLabel,
})
groupKey := kind + "|" + sourceName + "|" + communityID
g, ok := byGroup[groupKey]
if !ok {
g = &summary{
Kind: kind,
Source: sourceName,
SourceDetail: sourceDetail,
CommunityID: communityID,
CommunityLabel: communityLabel,
Sample: make([]string, 0, 5),
}
byGroup[groupKey] = g
}
g.Count++
if len(g.Sample) < 5 {
g.Sample = append(g.Sample, p.Prefix)
}
}
if !more || strings.TrimSpace(next) == "" {
break
}
cursor = next
}
summaryKeys := make([]string, 0, len(byGroup))
for k := range byGroup {
summaryKeys = append(summaryKeys, k)
}
sort.Strings(summaryKeys)
sort.Slice(rows, func(i, j int) bool {
if rows[i].Kind != rows[j].Kind {
return rows[i].Kind < rows[j].Kind
}
if rows[i].SourceName != rows[j].SourceName {
return rows[i].SourceName < rows[j].SourceName
}
if rows[i].CommunityID != rows[j].CommunityID {
return rows[i].CommunityID < rows[j].CommunityID
}
return rows[i].Prefix < rows[j].Prefix
})
var b strings.Builder
b.WriteString("EvoBGP revision diagnostic log\n")
b.WriteString("generated_at=" + time.Now().UTC().Format(time.RFC3339Nano) + "\n")
b.WriteString("tenant_id=" + a.TenantID + "\n")
b.WriteString("revision_id=" + rev.ID + "\n")
b.WriteString("revision_created_at=" + rev.CreatedAt.UTC().Format(time.RFC3339Nano) + "\n")
b.WriteString("content_hash=" + rev.ContentHash + "\n")
b.WriteString("materialized_prefix_count=" + strconv.Itoa(rev.MaterializedPrefixCount) + "\n")
if rev.ModuleID != "" {
b.WriteString("module_id=" + rev.ModuleID + "\n")
} else {
b.WriteString("module_id=\n")
}
b.WriteString("module_type=" + moduleType + "\n")
b.WriteString("module_name=" + moduleName + "\n")
b.WriteString("fetched_rows=" + strconv.Itoa(len(rows)) + "\n\n")
b.WriteString("## Aggregation by source and community\n")
for _, k := range summaryKeys {
g := byGroup[k]
b.WriteString("- kind=" + g.Kind +
" source=" + g.Source +
" source_detail=" + g.SourceDetail +
" community_id=" + g.CommunityID +
" community_label=" + g.CommunityLabel +
" count=" + strconv.Itoa(g.Count))
if len(g.Sample) > 0 {
b.WriteString(" sample=" + strings.Join(g.Sample, ","))
}
b.WriteByte('\n')
}
b.WriteString("\n## Raw rows\n")
b.WriteString("prefix\tkind\tsource\tsource_detail\tcommunity_id\tcommunity_label\n")
for _, row := range rows {
b.WriteString(row.Prefix)
b.WriteByte('\t')
b.WriteString(row.Kind)
b.WriteByte('\t')
b.WriteString(row.Source)
b.WriteByte('\t')
b.WriteString(strings.ReplaceAll(row.SourceDetail, "\t", " "))
b.WriteByte('\t')
b.WriteString(row.CommunityID)
b.WriteByte('\t')
b.WriteString(strings.ReplaceAll(row.CommunityLabel, "\t", " "))
b.WriteByte('\n')
}
filename := "revision-" + shortRevisionID(rev.ID) + "-diagnostic.log"
w.Header().Set("Content-Type", "text/plain; charset=utf-8") w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.Header().Set("Content-Disposition", `attachment; filename="`+filename+`"`) w.Header().Set("Content-Disposition", `attachment; filename="`+logOut.Filename+`"`)
w.WriteHeader(http.StatusOK) w.WriteHeader(http.StatusOK)
_, _ = w.Write([]byte(b.String())) _, _ = w.Write(logOut.Body)
}
func classifyRevisionSource(src string) (kind, sourceName string) {
switch {
case strings.HasPrefix(src, "as:"):
return "asn", strings.TrimPrefix(src, "as:")
case strings.HasPrefix(src, "domain:"):
return "domain", strings.TrimPrefix(src, "domain:")
case strings.HasPrefix(src, "cdn:"):
return "cdn", strings.TrimPrefix(src, "cdn:")
case src == "ip_range":
return "ip_range", "manual_ranges"
default:
if strings.TrimSpace(src) == "" {
return "unknown", "unknown"
}
return "source", strings.TrimSpace(src)
}
}
func shortRevisionID(id string) string {
s := strings.TrimSpace(id)
if len(s) <= 8 {
return s
}
return s[:8]
} }
func (s *Server) handleRevisionDiff(w http.ResponseWriter, r *http.Request) { func (s *Server) handleRevisionDiff(w http.ResponseWriter, r *http.Request) {
@@ -909,7 +674,7 @@ func (s *Server) handleRevisionRollback(w http.ResponseWriter, r *http.Request)
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth") writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return return
} }
if !s.requireAtLeast(w, a, "editor") { if !s.requireAtLeast(w, a, "operator") {
return return
} }
revID := r.PathValue("revision_id") revID := r.PathValue("revision_id")
@@ -926,7 +691,7 @@ func (s *Server) handleRevisionRollback(w http.ResponseWriter, r *http.Request)
"source_revision_id": revID, "source_revision_id": revID,
}) })
if err != nil { if err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
w.Header().Set("Location", "/v1/jobs/"+j.ID) w.Header().Set("Location", "/v1/jobs/"+j.ID)
@@ -973,7 +738,7 @@ func (s *Server) handleApply(w http.ResponseWriter, r *http.Request) {
"strategy": body.Strategy, "strategy": body.Strategy,
}) })
if err != nil { if err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
w.Header().Set("Location", "/v1/jobs/"+j.ID) w.Header().Set("Location", "/v1/jobs/"+j.ID)
@@ -1024,7 +789,7 @@ func (s *Server) handleSpeakerApply(w http.ResponseWriter, r *http.Request) {
"speaker_id": spkID, "speaker_id": spkID,
}) })
if err != nil { if err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
w.Header().Set("Location", "/v1/jobs/"+j.ID) w.Header().Set("Location", "/v1/jobs/"+j.ID)
@@ -1049,7 +814,7 @@ func (s *Server) handleBirdReload(w http.ResponseWriter, r *http.Request) {
} }
j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindBirdReload, idemPtr, nil, nil) j, _, err := s.jobs.Enqueue(a.TenantID, jobs.KindBirdReload, idemPtr, nil, nil)
if err != nil { if err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
w.Header().Set("Location", "/v1/jobs/"+j.ID) w.Header().Set("Location", "/v1/jobs/"+j.ID)
@@ -1111,6 +876,44 @@ func (s *Server) handleGetJob(w http.ResponseWriter, r *http.Request) {
writeJSON(w, http.StatusOK, j.Snapshot()) writeJSON(w, http.StatusOK, j.Snapshot())
} }
func (s *Server) handleGetJobReport(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok {
writeProblem(w, http.StatusUnauthorized, "Unauthorized", "missing auth")
return
}
if !s.requireAtLeast(w, a, "viewer") {
return
}
j, err := s.jobs.Get(a.TenantID, r.PathValue("job_id"))
if err != nil {
writeProblem(w, http.StatusNotFound, "Not Found", "job not found")
return
}
snap := j.Snapshot()
meta, _ := snap["meta"].(map[string]any)
out := map[string]any{
"job_id": snap["job_id"],
"kind": snap["kind"],
"status": snap["status"],
"meta": meta,
"error": snap["error"],
"created_at": snap["created_at"],
}
if meta != nil {
if v, ok := meta["log_entries"]; ok {
out["log_entries"] = v
}
if v, ok := meta["log_total"]; ok {
out["log_total"] = v
}
if v, ok := meta["revision_id"]; ok {
out["revision_id"] = v
}
}
writeJSON(w, http.StatusOK, out)
}
func (s *Server) handleCancelJob(w http.ResponseWriter, r *http.Request) { func (s *Server) handleCancelJob(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context()) a, ok := authFromContext(r.Context())
if !ok { if !ok {
@@ -1184,7 +987,7 @@ func (s *Server) handleNodeBundle(w http.ResponseWriter, r *http.Request) {
} }
tgz, err := bundle.BuildGzippedTar(rid, sid, rev.PreviewFragments, s.bundlePriv) tgz, err := bundle.BuildGzippedTar(rid, sid, rev.PreviewFragments, s.bundlePriv)
if err != nil { if err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
w.Header().Set("Content-Type", "application/gzip") w.Header().Set("Content-Type", "application/gzip")
@@ -1234,7 +1037,7 @@ func (s *Server) handleNodeEnroll(w http.ResponseWriter, r *http.Request) {
} }
patch := &store.SpeakerPatch{MetaJSON: &meta} patch := &store.SpeakerPatch{MetaJSON: &meta}
if _, err := s.store.UpdateSpeaker(a.TenantID, sid, patch); err != nil { if _, err := s.store.UpdateSpeaker(a.TenantID, sid, patch); err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "internal", err)
return return
} }
writeJSON(w, http.StatusOK, map[string]any{ writeJSON(w, http.StatusOK, map[string]any{
+217
View File
@@ -0,0 +1,217 @@
package httpapi
import (
"encoding/json"
"net/http"
"strings"
"time"
"evobgp/internal/store"
)
func (s *Server) registerAPIKeyRoutes(m *http.ServeMux) {
m.HandleFunc("GET /auth/session", s.handleAuthSession)
m.HandleFunc("GET /api-keys", s.handleListAPIKeys)
m.HandleFunc("POST /api-keys", s.handlePostAPIKey)
m.HandleFunc("GET /api-keys/{id}", s.handleGetAPIKey)
m.HandleFunc("PATCH /api-keys/{id}", s.handlePatchAPIKey)
m.HandleFunc("DELETE /api-keys/{id}", s.handleDeleteAPIKey)
m.HandleFunc("POST /api-keys/{id}/rotate", s.handleRotateAPIKey)
}
func (s *Server) handleAuthSession(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "viewer") {
return
}
writeJSON(w, http.StatusOK, map[string]any{
"tenant_id": a.TenantID,
"role": a.Role,
})
}
func apiKeyJSON(k *store.APIKey) map[string]any {
m := map[string]any{
"id": k.ID,
"name": k.Name,
"role": k.Role,
"prefix": k.Prefix,
"created_at": k.CreatedAt.UTC().Format(time.RFC3339),
"updated_at": k.UpdatedAt.UTC().Format(time.RFC3339),
}
if k.ExpiresAt != nil {
m["expires_at"] = k.ExpiresAt.UTC().Format(time.RFC3339)
} else {
m["expires_at"] = nil
}
if k.RevokedAt != nil {
m["revoked_at"] = k.RevokedAt.UTC().Format(time.RFC3339)
} else {
m["revoked_at"] = nil
}
if k.LastUsedAt != nil {
m["last_used_at"] = k.LastUsedAt.UTC().Format(time.RFC3339)
} else {
m["last_used_at"] = nil
}
return m
}
func (s *Server) handleListAPIKeys(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "operator") {
return
}
list, err := s.store.ListAPIKeys(a.TenantID)
if err != nil {
writeStoreErr(w, err)
return
}
writePaginatedListJSON(w, r, list, func(k *store.APIKey) map[string]any {
return apiKeyJSON(k)
})
}
func (s *Server) handleGetAPIKey(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "operator") {
return
}
k, err := s.store.GetAPIKey(a.TenantID, r.PathValue("id"))
if err != nil {
writeStoreErr(w, err)
return
}
writeJSON(w, http.StatusOK, apiKeyJSON(k))
}
func (s *Server) handlePostAPIKey(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "operator") {
return
}
var body struct {
Name string `json:"name"`
Role string `json:"role"`
ExpiresAt *string `json:"expires_at"`
}
if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
return
}
in := &store.APIKeyCreate{
Name: strings.TrimSpace(body.Name),
Role: strings.TrimSpace(body.Role),
}
if body.ExpiresAt != nil && strings.TrimSpace(*body.ExpiresAt) != "" {
t, err := time.Parse(time.RFC3339, strings.TrimSpace(*body.ExpiresAt))
if err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "invalid expires_at")
return
}
in.ExpiresAt = &t
}
created, err := s.store.CreateAPIKey(a.TenantID, in)
if err != nil {
writeStoreErr(w, err)
return
}
if err := s.keyResolver.Reload(s.store); err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Server Error", "failed to reload api keys")
return
}
out := apiKeyJSON(&created.APIKey)
out["token"] = created.Token
writeJSON(w, http.StatusCreated, out)
}
func (s *Server) handlePatchAPIKey(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "operator") {
return
}
var raw map[string]json.RawMessage
if err := json.NewDecoder(r.Body).Decode(&raw); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
return
}
patch := &store.APIKeyPatch{}
if v, ok := raw["name"]; ok {
var name string
if err := json.Unmarshal(v, &name); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid name")
return
}
patch.Name = &name
}
if v, ok := raw["role"]; ok {
var role string
if err := json.Unmarshal(v, &role); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid role")
return
}
patch.Role = &role
}
if v, ok := raw["expires_at"]; ok {
if string(v) == "null" {
patch.ClearExpiresAt = true
} else {
var s string
if err := json.Unmarshal(v, &s); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid expires_at")
return
}
t, err := time.Parse(time.RFC3339, strings.TrimSpace(s))
if err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "invalid expires_at")
return
}
patch.ExpiresAt = &t
}
}
k, err := s.store.UpdateAPIKey(a.TenantID, r.PathValue("id"), patch)
if err != nil {
writeStoreErr(w, err)
return
}
if err := s.keyResolver.Reload(s.store); err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Server Error", "failed to reload api keys")
return
}
writeJSON(w, http.StatusOK, apiKeyJSON(k))
}
func (s *Server) handleDeleteAPIKey(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "operator") {
return
}
if err := s.store.RevokeAPIKey(a.TenantID, r.PathValue("id")); err != nil {
writeStoreErr(w, err)
return
}
if err := s.keyResolver.Reload(s.store); err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Server Error", "failed to reload api keys")
return
}
w.WriteHeader(http.StatusNoContent)
}
func (s *Server) handleRotateAPIKey(w http.ResponseWriter, r *http.Request) {
a, ok := authFromContext(r.Context())
if !ok || !s.requireAtLeast(w, a, "operator") {
return
}
rotated, err := s.store.RotateAPIKey(a.TenantID, r.PathValue("id"))
if err != nil {
writeStoreErr(w, err)
return
}
if err := s.keyResolver.Reload(s.store); err != nil {
writeProblem(w, http.StatusInternalServerError, "Internal Server Error", "failed to reload api keys")
return
}
out := apiKeyJSON(&rotated.APIKey)
out["token"] = rotated.Token
writeJSON(w, http.StatusOK, out)
}
+119
View File
@@ -0,0 +1,119 @@
package httpapi
import (
"encoding/json"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestBearerDevWithoutInsecureDev(t *testing.T) {
srv, err := New(Options{SeedDemo: true, BundleSeedHex: testBundleSeed})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
ts := httptest.NewServer(srv.Handler())
defer ts.Close()
req, _ := http.NewRequest(http.MethodGet, ts.URL+"/v1/modules?limit=1", nil)
req.Header.Set("Authorization", "Bearer dev")
resp, err := ts.Client().Do(req)
if err != nil {
t.Fatal(err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body)
t.Fatalf("status=%d body=%s", resp.StatusCode, b)
}
}
func TestAPIKeysCRUDAndAuth(t *testing.T) {
srv, err := New(Options{SeedDemo: true, BundleSeedHex: testBundleSeed})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
tenant, _, _, _, _ := srv.Store().DemoIDs()
mustSetTestAPIKeys(t, srv, "opkey|"+tenant+"|operator")
ts := httptest.NewServer(srv.Handler())
defer ts.Close()
client := ts.Client()
base := ts.URL
reqCreate, _ := http.NewRequest(http.MethodPost, base+"/v1/api-keys", strings.NewReader(`{"name":"ci","role":"editor"}`))
reqCreate.Header.Set("Authorization", "Bearer opkey")
reqCreate.Header.Set("Content-Type", "application/json")
respCreate, err := client.Do(reqCreate)
if err != nil {
t.Fatal(err)
}
defer func() { _ = respCreate.Body.Close() }()
if respCreate.StatusCode != http.StatusCreated {
b, _ := io.ReadAll(respCreate.Body)
t.Fatalf("create status=%d body=%s", respCreate.StatusCode, b)
}
var created map[string]any
if err := json.NewDecoder(respCreate.Body).Decode(&created); err != nil {
t.Fatal(err)
}
token, _ := created["token"].(string)
if token == "" {
t.Fatal("missing token in create response")
}
id, _ := created["id"].(string)
if id == "" {
t.Fatal("missing id")
}
reqMod, _ := http.NewRequest(http.MethodGet, base+"/v1/modules?limit=1", nil)
reqMod.Header.Set("Authorization", "Bearer "+token)
respMod, err := client.Do(reqMod)
if err != nil {
t.Fatal(err)
}
defer func() { _ = respMod.Body.Close() }()
if respMod.StatusCode != http.StatusOK {
b, _ := io.ReadAll(respMod.Body)
t.Fatalf("modules status=%d body=%s", respMod.StatusCode, b)
}
reqDel, _ := http.NewRequest(http.MethodDelete, base+"/v1/api-keys/"+id, nil)
reqDel.Header.Set("Authorization", "Bearer opkey")
respDel, err := client.Do(reqDel)
if err != nil {
t.Fatal(err)
}
defer func() { _ = respDel.Body.Close() }()
if respDel.StatusCode != http.StatusNoContent {
t.Fatalf("delete status=%d", respDel.StatusCode)
}
reqAfter, _ := http.NewRequest(http.MethodGet, base+"/v1/modules?limit=1", nil)
reqAfter.Header.Set("Authorization", "Bearer "+token)
respAfter, err := client.Do(reqAfter)
if err != nil {
t.Fatal(err)
}
defer func() { _ = respAfter.Body.Close() }()
if respAfter.StatusCode != http.StatusUnauthorized {
t.Fatalf("expected 401 after revoke, got %d", respAfter.StatusCode)
}
mustSetTestAPIKeys(t, srv, "nodekey|"+tenant+"|node,opkey|"+tenant+"|operator")
reqNode2, _ := http.NewRequest(http.MethodGet, base+"/v1/api-keys", nil)
reqNode2.Header.Set("Authorization", "Bearer nodekey")
respNode, err := client.Do(reqNode2)
if err != nil {
t.Fatal(err)
}
defer func() { _ = respNode.Body.Close() }()
if respNode.StatusCode != http.StatusForbidden {
t.Fatalf("node list api-keys status=%d want 403", respNode.StatusCode)
}
}
+56 -174
View File
@@ -3,13 +3,16 @@ package httpapi
import ( import (
"encoding/csv" "encoding/csv"
"encoding/json" "encoding/json"
"errors"
"fmt" "fmt"
"io" "io"
"log"
"net/http" "net/http"
"strconv" "strconv"
"strings" "strings"
"time" "time"
"evobgp/internal/importer"
"evobgp/internal/pipeline" "evobgp/internal/pipeline"
"evobgp/internal/store" "evobgp/internal/store"
) )
@@ -67,6 +70,8 @@ func (s *Server) registerCRUDRoutes(m *http.ServeMux) {
m.HandleFunc("GET /settings", s.handleGetSettings) m.HandleFunc("GET /settings", s.handleGetSettings)
m.HandleFunc("PATCH /settings", s.handlePatchSettings) m.HandleFunc("PATCH /settings", s.handlePatchSettings)
s.registerAPIKeyRoutes(m)
} }
func (s *Server) handlePostModule(w http.ResponseWriter, r *http.Request) { func (s *Server) handlePostModule(w http.ResponseWriter, r *http.Request) {
@@ -83,6 +88,8 @@ func (s *Server) handlePostModule(w http.ResponseWriter, r *http.Request) {
CronExpr string `json:"cron_expr"` CronExpr string `json:"cron_expr"`
DefaultCommunityID *string `json:"default_community_id"` DefaultCommunityID *string `json:"default_community_id"`
DohProfileID *string `json:"doh_profile_id"` DohProfileID *string `json:"doh_profile_id"`
DohProfileIDs []string `json:"doh_profile_ids"`
DohResolverPolicy string `json:"doh_resolver_policy"`
} }
if err := json.NewDecoder(r.Body).Decode(&body); err != nil { if err := json.NewDecoder(r.Body).Decode(&body); err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json") writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid json")
@@ -92,6 +99,7 @@ func (s *Server) handlePostModule(w http.ResponseWriter, r *http.Request) {
Type: body.Type, Name: body.Name, Enabled: body.Enabled, Priority: body.Priority, Type: body.Type, Name: body.Name, Enabled: body.Enabled, Priority: body.Priority,
RefreshIntervalSec: body.RefreshIntervalSec, CronExpr: body.CronExpr, RefreshIntervalSec: body.RefreshIntervalSec, CronExpr: body.CronExpr,
DefaultCommunityID: body.DefaultCommunityID, DohProfileID: body.DohProfileID, DefaultCommunityID: body.DefaultCommunityID, DohProfileID: body.DohProfileID,
DohProfileIDs: body.DohProfileIDs, DohResolverPolicy: body.DohResolverPolicy,
}) })
if err != nil { if err != nil {
writeStoreErr(w, err) writeStoreErr(w, err)
@@ -129,6 +137,14 @@ func (s *Server) handlePatchModule(w http.ResponseWriter, r *http.Request) {
empty := "" empty := ""
body.DohProfileID = &empty body.DohProfileID = &empty
} }
if v, ok := raw["doh_profile_ids"]; ok && string(v) == "null" {
empty := []string{}
body.DohProfileIDs = &empty
}
if v, ok := raw["doh_resolver_policy"]; ok && string(v) == "null" {
p := store.DohPolicyPrimaryOnly
body.DohResolverPolicy = &p
}
if v, ok := raw["cron_expr"]; ok && string(v) == "null" { if v, ok := raw["cron_expr"]; ok && string(v) == "null" {
empty := "" empty := ""
body.CronExpr = &empty body.CronExpr = &empty
@@ -159,15 +175,18 @@ func (s *Server) handleDeleteModule(w http.ResponseWriter, r *http.Request) {
} }
func writeStoreErr(w http.ResponseWriter, err error) { func writeStoreErr(w http.ResponseWriter, err error) {
if err != nil {
log.Printf("httpapi: store: %v", err)
}
if err == store.ErrNotFound || err == store.ErrTenantScope { if err == store.ErrNotFound || err == store.ErrTenantScope {
writeProblem(w, http.StatusNotFound, "Not Found", err.Error()) writeProblem(w, http.StatusNotFound, "Not Found", notFoundDetail)
return return
} }
if err == store.ErrInvalidInput { if err == store.ErrInvalidInput {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", err.Error()) writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", invalidInputDetail)
return return
} }
writeProblem(w, http.StatusInternalServerError, "Internal Error", err.Error()) writeInternalError(w, "store", err)
} }
func (s *Server) handleListCDNSources(w http.ResponseWriter, r *http.Request) { func (s *Server) handleListCDNSources(w http.ResponseWriter, r *http.Request) {
@@ -180,11 +199,7 @@ func (s *Server) handleListCDNSources(w http.ResponseWriter, r *http.Request) {
writeStoreErr(w, err) writeStoreErr(w, err)
return return
} }
items := make([]map[string]any, 0, len(list)) writePaginatedListJSON(w, r, list, cdnSourceJSON)
for _, x := range list {
items = append(items, cdnSourceJSON(x))
}
writeJSON(w, http.StatusOK, map[string]any{"items": items, "next_cursor": nil, "has_more": false})
} }
func cdnSourceJSON(x *store.CDNSource) map[string]any { func cdnSourceJSON(x *store.CDNSource) map[string]any {
@@ -240,25 +255,26 @@ func (s *Server) handlePreviewCDNSource(w http.ResponseWriter, r *http.Request)
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "invalid url") writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "invalid url")
return return
} }
resp, err := http.DefaultClient.Do(req) resp, err := s.cdnHTTP.Do(req)
if err != nil { if err != nil {
writeProblem(w, http.StatusBadGateway, "Bad Gateway", err.Error()) writeBadGateway(w, "cdn preview fetch", err)
return return
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
_, _ = io.Copy(io.Discard, resp.Body) _, _ = io.Copy(io.Discard, resp.Body)
writeProblem(w, http.StatusBadGateway, "Bad Gateway", fmt.Sprintf("upstream status: %s", resp.Status)) writeBadGateway(w, "cdn preview fetch", fmt.Errorf("upstream status: %s", resp.Status))
return return
} }
raw, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20)) raw, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if err != nil { if err != nil {
writeProblem(w, http.StatusBadGateway, "Bad Gateway", err.Error()) writeBadGateway(w, "cdn preview read body", err)
return return
} }
pfxs, err := pipeline.ExtractCIDRs(string(raw), body.SourceKind, body.PrefixPath) pfxs, err := pipeline.ExtractCIDRs(string(raw), body.SourceKind, body.PrefixPath)
if err != nil { if err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", err.Error()) log.Printf("httpapi: cdn preview extract: %v", err)
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", cdnExtractDetail)
return return
} }
items := make([]string, 0, len(pfxs)) items := make([]string, 0, len(pfxs))
@@ -341,11 +357,7 @@ func (s *Server) handleListAS(w http.ResponseWriter, r *http.Request) {
writeStoreErr(w, err) writeStoreErr(w, err)
return return
} }
items := make([]map[string]any, 0, len(list)) writePaginatedListJSON(w, r, list, asEntryJSON)
for _, x := range list {
items = append(items, asEntryJSON(x))
}
writeJSON(w, http.StatusOK, map[string]any{"items": items})
} }
func asEntryJSON(x *store.ASEntry) map[string]any { func asEntryJSON(x *store.ASEntry) map[string]any {
@@ -437,11 +449,7 @@ func (s *Server) handleListDomain(w http.ResponseWriter, r *http.Request) {
writeStoreErr(w, err) writeStoreErr(w, err)
return return
} }
items := make([]map[string]any, 0, len(list)) writePaginatedListJSON(w, r, list, domainEntryJSON)
for _, x := range list {
items = append(items, domainEntryJSON(x))
}
writeJSON(w, http.StatusOK, map[string]any{"items": items})
} }
func domainEntryJSON(x *store.DomainEntry) map[string]any { func domainEntryJSON(x *store.DomainEntry) map[string]any {
@@ -518,11 +526,7 @@ func (s *Server) handleListIPRange(w http.ResponseWriter, r *http.Request) {
writeStoreErr(w, err) writeStoreErr(w, err)
return return
} }
items := make([]map[string]any, 0, len(list)) writePaginatedListJSON(w, r, list, ipRangeJSON)
for _, x := range list {
items = append(items, ipRangeJSON(x))
}
writeJSON(w, http.StatusOK, map[string]any{"items": items})
} }
func ipRangeJSON(x *store.IPRangeEntry) map[string]any { func ipRangeJSON(x *store.IPRangeEntry) map[string]any {
@@ -678,163 +682,41 @@ func (s *Server) handleImportModuleEntriesCSV(w http.ResponseWriter, r *http.Req
return return
} }
moduleID := r.PathValue("module_id") moduleID := r.PathValue("module_id")
mod, err := s.store.GetModule(a.TenantID, moduleID) res, err := importer.ImportModuleEntriesCSV(s.store, a.TenantID, moduleID, r.Body)
if err != nil { if err != nil {
writeStoreErr(w, err) if errors.Is(err, store.ErrInvalidInput) {
return
}
cr := csv.NewReader(io.LimitReader(r.Body, 8<<20))
cr.TrimLeadingSpace = true
cr.FieldsPerRecord = -1
rows, err := cr.ReadAll()
if err != nil {
writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid csv")
return
}
if len(rows) == 0 {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "csv is empty") writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "csv is empty")
return return
} }
if strings.Contains(err.Error(), "importer: invalid csv") {
communities, err := s.store.ListCommunities(a.TenantID) writeProblem(w, http.StatusBadRequest, "Bad Request", "invalid csv")
if err != nil {
writeStoreErr(w, err)
return return
} }
communityIDByID := make(map[string]string, len(communities)) if strings.Contains(err.Error(), "importer: line") {
communityIDByValue := make(map[string]string, len(communities)) log.Printf("httpapi: csv import: %v", err)
for _, c := range communities { writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", csvInvalidRowDetail)
communityIDByID[c.ID] = c.ID
communityIDByValue[strings.TrimSpace(c.Community)] = c.ID
}
resolveCommunity := func(raw string, required bool) (*string, error) {
v := strings.TrimSpace(raw)
if v == "" {
if required {
return nil, fmt.Errorf("community is required")
}
return nil, nil
}
if id, ok := communityIDByID[v]; ok {
return &id, nil
}
if id, ok := communityIDByValue[v]; ok {
return &id, nil
}
return nil, fmt.Errorf("unknown community %q", v)
}
start := 0
if len(rows[0]) >= 2 {
key := strings.ToLower(strings.TrimSpace(rows[0][0]))
switch key {
case "asn", "domain", "iprange":
start = 1
}
}
imported := 0
switch mod.Type {
case "AS_PREFIXES":
for i := start; i < len(rows); i++ {
rec := rows[i]
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
continue
}
if len(rec) < 2 {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: expected 2 columns", i+1))
return return
} }
asn, err := strconv.ParseInt(strings.TrimSpace(rec[0]), 10, 64) if strings.Contains(err.Error(), "importer: csv import/export") {
if err != nil || asn <= 0 {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: invalid asn", i+1))
return
}
cid, err := resolveCommunity(rec[1], false)
if err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: %v", i+1, err))
return
}
_, err = s.store.CreateASEntry(a.TenantID, moduleID, &store.ASEntry{ASN: asn, CommunityID: cid})
if err != nil {
writeStoreErr(w, err)
return
}
imported++
}
if imported > 0 {
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "as_entry_import_csv")
}
case "DOMAINS":
for i := start; i < len(rows); i++ {
rec := rows[i]
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
continue
}
if len(rec) < 2 {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: expected 2 columns", i+1))
return
}
fqdn := strings.TrimSpace(rec[0])
if fqdn == "" {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: domain is required", i+1))
return
}
cid, err := resolveCommunity(rec[1], false)
if err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: %v", i+1, err))
return
}
_, err = s.store.CreateDomainEntry(a.TenantID, moduleID, &store.DomainEntry{FQDN: fqdn, CommunityID: cid})
if err != nil {
writeStoreErr(w, err)
return
}
imported++
}
if imported > 0 {
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "domain_entry_import_csv")
}
case "IP_RANGES":
for i := start; i < len(rows); i++ {
rec := rows[i]
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
continue
}
if len(rec) < 2 {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: expected 2 columns", i+1))
return
}
prefix := strings.TrimSpace(rec[0])
if prefix == "" {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: ipRange is required", i+1))
return
}
cid, err := resolveCommunity(rec[1], true)
if err != nil {
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", fmt.Sprintf("line %d: %v", i+1, err))
return
}
_, err = s.store.CreateIPRangeEntry(a.TenantID, moduleID, &store.IPRangeEntry{Prefix: prefix, CommunityID: cid})
if err != nil {
writeStoreErr(w, err)
return
}
imported++
}
if imported > 0 {
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "ip_range_import_csv")
}
default:
writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "csv import/export is supported only for AS_PREFIXES, DOMAINS, IP_RANGES") writeProblem(w, http.StatusUnprocessableEntity, "Unprocessable Entity", "csv import/export is supported only for AS_PREFIXES, DOMAINS, IP_RANGES")
return return
} }
writeStoreErr(w, err)
return
}
if res.Imported > 0 {
switch res.ModuleType {
case "AS_PREFIXES":
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "as_entry_import_csv")
case "DOMAINS":
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "domain_entry_import_csv")
case "IP_RANGES":
s.enqueueModuleRefreshIfEnabled(a.TenantID, moduleID, "ip_range_import_csv")
}
}
writeJSON(w, http.StatusOK, map[string]any{ writeJSON(w, http.StatusOK, map[string]any{
"imported": imported, "imported": res.Imported,
"module_type": mod.Type, "module_type": res.ModuleType,
}) })
} }
+4 -4
View File
@@ -21,7 +21,7 @@ func TestModuleEntriesCSVImportExportIPRanges(t *testing.T) {
defer srv.Close() defer srv.Close()
tenant, _, modIP, _, _ := srv.Store().DemoIDs() tenant, _, modIP, _, _ := srv.Store().DemoIDs()
srv.apiKeys = parseAPIKeysSpec("opkey|" + tenant + "|operator") mustSetTestAPIKeys(t, srv, "opkey|"+tenant+"|operator")
ts := httptest.NewServer(srv.Handler()) ts := httptest.NewServer(srv.Handler())
defer ts.Close() defer ts.Close()
@@ -34,7 +34,7 @@ func TestModuleEntriesCSVImportExportIPRanges(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer respList.Body.Close() defer func() { _ = respList.Body.Close() }()
if respList.StatusCode != http.StatusOK { if respList.StatusCode != http.StatusOK {
b, _ := io.ReadAll(respList.Body) b, _ := io.ReadAll(respList.Body)
t.Fatalf("communities status %d: %s", respList.StatusCode, b) t.Fatalf("communities status %d: %s", respList.StatusCode, b)
@@ -59,7 +59,7 @@ func TestModuleEntriesCSVImportExportIPRanges(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer respImport.Body.Close() defer func() { _ = respImport.Body.Close() }()
if respImport.StatusCode != http.StatusOK { if respImport.StatusCode != http.StatusOK {
b, _ := io.ReadAll(respImport.Body) b, _ := io.ReadAll(respImport.Body)
t.Fatalf("import status %d: %s", respImport.StatusCode, b) t.Fatalf("import status %d: %s", respImport.StatusCode, b)
@@ -80,7 +80,7 @@ func TestModuleEntriesCSVImportExportIPRanges(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer respExport.Body.Close() defer func() { _ = respExport.Body.Close() }()
if respExport.StatusCode != http.StatusOK { if respExport.StatusCode != http.StatusOK {
b, _ := io.ReadAll(respExport.Body) b, _ := io.ReadAll(respExport.Body)
t.Fatalf("export status %d: %s", respExport.StatusCode, b) t.Fatalf("export status %d: %s", respExport.StatusCode, b)
+87
View File
@@ -0,0 +1,87 @@
package httpapi
import (
"encoding/json"
"fmt"
"io"
"net/http"
"net/http/httptest"
"strings"
"testing"
)
func TestNestedModuleListPagination(t *testing.T) {
srv, err := New(Options{InsecureDev: true, SeedDemo: true, BundleSeedHex: testBundleSeed})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
tenant, _, modIP, _, _ := srv.Store().DemoIDs()
mustSetTestAPIKeys(t, srv, "edkey|"+tenant+"|editor")
ts := httptest.NewServer(srv.Handler())
defer ts.Close()
client := ts.Client()
base := ts.URL
mid := modIP
for i := 0; i < 3; i++ {
body := strings.NewReader(fmt.Sprintf(`{"prefix":"10.%d.0.0/24"}`, 200+i))
req, _ := http.NewRequest(http.MethodPost, base+"/v1/modules/"+mid+"/ip-range-entries", body)
req.Header.Set("Authorization", "Bearer edkey")
req.Header.Set("Content-Type", "application/json")
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
_, _ = io.Copy(io.Discard, resp.Body)
_ = resp.Body.Close()
if resp.StatusCode != http.StatusCreated {
t.Fatalf("create entry %d: status %d", i, resp.StatusCode)
}
}
req, _ := http.NewRequest(http.MethodGet, base+"/v1/modules/"+mid+"/ip-range-entries?limit=2", nil)
req.Header.Set("Authorization", "Bearer edkey")
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body)
t.Fatalf("list status %d: %s", resp.StatusCode, b)
}
var page1 struct {
Items []map[string]any `json:"items"`
NextCursor *string `json:"next_cursor"`
HasMore bool `json:"has_more"`
}
if err := json.NewDecoder(resp.Body).Decode(&page1); err != nil {
t.Fatal(err)
}
if len(page1.Items) != 2 {
t.Fatalf("page1 items: got %d want 2", len(page1.Items))
}
if !page1.HasMore || page1.NextCursor == nil || *page1.NextCursor == "" {
t.Fatalf("page1: has_more=%v next_cursor=%v", page1.HasMore, page1.NextCursor)
}
req2, _ := http.NewRequest(http.MethodGet, base+"/v1/modules/"+mid+"/ip-range-entries?limit=2&cursor="+*page1.NextCursor, nil)
req2.Header.Set("Authorization", "Bearer edkey")
resp2, err := client.Do(req2)
if err != nil {
t.Fatal(err)
}
defer func() { _ = resp2.Body.Close() }()
var page2 struct {
Items []map[string]any `json:"items"`
HasMore bool `json:"has_more"`
}
if err := json.NewDecoder(resp2.Body).Decode(&page2); err != nil {
t.Fatal(err)
}
if len(page1.Items)+len(page2.Items) < 3 {
t.Fatalf("expected at least 3 entries across pages, got %d+%d", len(page1.Items), len(page2.Items))
}
}
+8 -4
View File
@@ -21,9 +21,9 @@ type Server struct {
pgPool *pgxpool.Pool pgPool *pgxpool.Pool
jobs *jobs.Registry jobs *jobs.Registry
bundlePriv ed25519.PrivateKey bundlePriv ed25519.PrivateKey
apiKeys []apiKeyRecord keyResolver *apiKeyResolver
insecureDev bool
corsOrigins []string corsOrigins []string
cdnHTTP *http.Client
mux *http.ServeMux mux *http.ServeMux
} }
@@ -59,14 +59,18 @@ func New(opts Options) (*Server, error) {
_, priv, _ = ed25519.GenerateKey(rand.Reader) _, priv, _ = ed25519.GenerateKey(rand.Reader)
} }
resolver, err := newAPIKeyResolver(opts.APIKeys, backend)
if err != nil {
return nil, err
}
s := &Server{ s := &Server{
store: backend, store: backend,
pgPool: pool, pgPool: pool,
jobs: reg, jobs: reg,
bundlePriv: priv, bundlePriv: priv,
apiKeys: parseAPIKeysSpec(opts.APIKeys), keyResolver: resolver,
insecureDev: opts.InsecureDev && opts.SeedDemo,
corsOrigins: parseCORSOrigins(opts.CORSAllowedOrigins), corsOrigins: parseCORSOrigins(opts.CORSAllowedOrigins),
cdnHTTP: NewCDNHTTPClient(),
} }
s.mux = http.NewServeMux() s.mux = http.NewServeMux()
s.registerRoutes() s.registerRoutes()
+85 -13
View File
@@ -13,6 +13,7 @@ import (
"evobgp/internal/jobs" "evobgp/internal/jobs"
"evobgp/internal/signing" "evobgp/internal/signing"
"evobgp/internal/version"
) )
const testBundleSeed = "0101010101010101010101010101010101010101010101010101010101010101" const testBundleSeed = "0101010101010101010101010101010101010101010101010101010101010101"
@@ -28,7 +29,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
} }
defer srv.Close() defer srv.Close()
tenant, modCDN, modIP, rev, speaker := srv.Store().DemoIDs() tenant, modCDN, modIP, rev, speaker := srv.Store().DemoIDs()
srv.apiKeys = parseAPIKeysSpec("nodekey|" + tenant + "|node,opkey|" + tenant + "|operator") mustSetTestAPIKeys(t, srv, "nodekey|"+tenant+"|node,opkey|"+tenant+"|operator,edkey|"+tenant+"|editor")
ts := httptest.NewServer(srv.Handler()) ts := httptest.NewServer(srv.Handler())
defer ts.Close() defer ts.Close()
@@ -50,7 +51,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -75,7 +76,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusAccepted { if resp.StatusCode != http.StatusAccepted {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -96,7 +97,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusAccepted { if resp.StatusCode != http.StatusAccepted {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -117,7 +118,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -131,7 +132,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -161,7 +162,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
t.Fatal(err) t.Fatal(err)
} }
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
resp.Body.Close() _ = resp.Body.Close()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
t.Fatalf("%s status %d: %s", path, resp.StatusCode, b) t.Fatalf("%s status %d: %s", path, resp.StatusCode, b)
} }
@@ -184,7 +185,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -214,7 +215,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -247,6 +248,20 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
} }
}) })
t.Run("rollback forbidden for editor", func(t *testing.T) {
req, _ := http.NewRequest(http.MethodPost, base+"/v1/revisions/"+rev+"/rollback", nil)
req.Header.Set("Authorization", "Bearer edkey")
resp, err := client.Do(req)
if err != nil {
t.Fatal(err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusForbidden {
b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d want 403: %s", resp.StatusCode, b)
}
})
t.Run("rollback queues job", func(t *testing.T) { t.Run("rollback queues job", func(t *testing.T) {
req, _ := http.NewRequest(http.MethodPost, base+"/v1/revisions/"+rev+"/rollback", nil) req, _ := http.NewRequest(http.MethodPost, base+"/v1/revisions/"+rev+"/rollback", nil)
req.Header.Set("Authorization", "Bearer opkey") req.Header.Set("Authorization", "Bearer opkey")
@@ -254,7 +269,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusAccepted { if resp.StatusCode != http.StatusAccepted {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -274,7 +289,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusAccepted { if resp.StatusCode != http.StatusAccepted {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -299,7 +314,7 @@ func TestAPIRefreshApplyJobsBundle(t *testing.T) {
if err != nil { if err != nil {
t.Fatal(err) t.Fatal(err)
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b) t.Fatalf("status %d: %s", resp.StatusCode, b)
@@ -326,7 +341,7 @@ func waitJob(t *testing.T, client *http.Client, base, token, jobID string) {
t.Fatal(err) t.Fatal(err)
} }
b, _ := io.ReadAll(resp.Body) b, _ := io.ReadAll(resp.Body)
resp.Body.Close() _ = resp.Body.Close()
var body struct { var body struct {
Status string `json:"status"` Status string `json:"status"`
} }
@@ -357,3 +372,60 @@ evobgp_p_01234567 BGP master start 2026-04-09 Connect
t.Fatalf("expected protocol state Connect, got %q", got["evobgp_p_01234567"]) t.Fatalf("expected protocol state Connect, got %q", got["evobgp_p_01234567"])
} }
} }
func TestVersionEndpoints(t *testing.T) {
srv, err := New(Options{
InsecureDev: true,
SeedDemo: true,
BundleSeedHex: testBundleSeed,
})
if err != nil {
t.Fatal(err)
}
defer srv.Close()
oldVer := version.Version
oldSHA := version.GitSHA
oldTime := version.BuildTime
version.Version = "1.2.3"
version.GitSHA = "abc123def456"
version.BuildTime = "2026-05-20T12:00:00Z"
t.Cleanup(func() {
version.Version = oldVer
version.GitSHA = oldSHA
version.BuildTime = oldTime
})
ts := httptest.NewServer(srv.Handler())
defer ts.Close()
for _, path := range []string{"/version", "/v1/version"} {
t.Run(path, func(t *testing.T) {
resp, err := http.Get(ts.URL + path)
if err != nil {
t.Fatal(err)
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body)
t.Fatalf("status %d: %s", resp.StatusCode, b)
}
var body map[string]string
if err := json.NewDecoder(resp.Body).Decode(&body); err != nil {
t.Fatal(err)
}
if body["version"] != "1.2.3" {
t.Fatalf("version=%q want 1.2.3", body["version"])
}
if body["api_version"] != "1.2.3" {
t.Fatalf("api_version=%q want 1.2.3", body["api_version"])
}
if body["git_sha"] != "abc123def456" {
t.Fatalf("git_sha=%q", body["git_sha"])
}
if body["build_time"] != "2026-05-20T12:00:00Z" {
t.Fatalf("build_time=%q", body["build_time"])
}
})
}
}
+12
View File
@@ -0,0 +1,12 @@
package httpapi
import "testing"
func mustSetTestAPIKeys(t *testing.T, srv *Server, spec string) {
t.Helper()
resolver, err := newAPIKeyResolver(spec, srv.store)
if err != nil {
t.Fatal(err)
}
srv.keyResolver = resolver
}
+147
View File
@@ -0,0 +1,147 @@
package importer
import (
"encoding/csv"
"fmt"
"io"
"strconv"
"strings"
"evobgp/internal/store"
)
// Result of importing module entries from CSV.
type Result struct {
Imported int
ModuleType string
}
// ImportModuleEntriesCSV parses and creates entries for AS_PREFIXES, DOMAINS, or IP_RANGES modules.
func ImportModuleEntriesCSV(st store.Backend, tenantID, moduleID string, body io.Reader) (*Result, error) {
mod, err := st.GetModule(tenantID, moduleID)
if err != nil {
return nil, err
}
cr := csv.NewReader(io.LimitReader(body, 8<<20))
cr.TrimLeadingSpace = true
cr.FieldsPerRecord = -1
rows, err := cr.ReadAll()
if err != nil {
return nil, fmt.Errorf("importer: invalid csv: %w", err)
}
if len(rows) == 0 {
return nil, fmt.Errorf("importer: %w", store.ErrInvalidInput)
}
communities, err := st.ListCommunities(tenantID)
if err != nil {
return nil, err
}
communityIDByID := make(map[string]string, len(communities))
communityIDByValue := make(map[string]string, len(communities))
for _, c := range communities {
communityIDByID[c.ID] = c.ID
communityIDByValue[strings.TrimSpace(c.Community)] = c.ID
}
resolveCommunity := func(raw string, required bool) (*string, error) {
v := strings.TrimSpace(raw)
if v == "" {
if required {
return nil, fmt.Errorf("community is required")
}
return nil, nil
}
if id, ok := communityIDByID[v]; ok {
return &id, nil
}
if id, ok := communityIDByValue[v]; ok {
return &id, nil
}
return nil, fmt.Errorf("unknown community %q", v)
}
start := 0
if len(rows[0]) >= 2 {
key := strings.ToLower(strings.TrimSpace(rows[0][0]))
switch key {
case "asn", "domain", "iprange":
start = 1
}
}
imported := 0
switch mod.Type {
case "AS_PREFIXES":
for i := start; i < len(rows); i++ {
rec := rows[i]
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
continue
}
if len(rec) < 2 {
return nil, fmt.Errorf("importer: line %d: expected 2 columns", i+1)
}
asn, err := strconv.ParseInt(strings.TrimSpace(rec[0]), 10, 64)
if err != nil || asn <= 0 {
return nil, fmt.Errorf("importer: line %d: invalid asn", i+1)
}
cid, err := resolveCommunity(rec[1], false)
if err != nil {
return nil, fmt.Errorf("importer: line %d: %w", i+1, err)
}
if _, err := st.CreateASEntry(tenantID, moduleID, &store.ASEntry{ASN: asn, CommunityID: cid}); err != nil {
return nil, err
}
imported++
}
case "DOMAINS":
for i := start; i < len(rows); i++ {
rec := rows[i]
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
continue
}
if len(rec) < 2 {
return nil, fmt.Errorf("importer: line %d: expected 2 columns", i+1)
}
fqdn := strings.TrimSpace(rec[0])
if fqdn == "" {
return nil, fmt.Errorf("importer: line %d: domain is required", i+1)
}
cid, err := resolveCommunity(rec[1], false)
if err != nil {
return nil, fmt.Errorf("importer: line %d: %w", i+1, err)
}
if _, err := st.CreateDomainEntry(tenantID, moduleID, &store.DomainEntry{FQDN: fqdn, CommunityID: cid}); err != nil {
return nil, err
}
imported++
}
case "IP_RANGES":
for i := start; i < len(rows); i++ {
rec := rows[i]
if len(rec) == 0 || (strings.TrimSpace(rec[0]) == "" && (len(rec) < 2 || strings.TrimSpace(rec[1]) == "")) {
continue
}
if len(rec) < 2 {
return nil, fmt.Errorf("importer: line %d: expected 2 columns", i+1)
}
prefix := strings.TrimSpace(rec[0])
if prefix == "" {
return nil, fmt.Errorf("importer: line %d: ipRange is required", i+1)
}
cid, err := resolveCommunity(rec[1], true)
if err != nil {
return nil, fmt.Errorf("importer: line %d: %w", i+1, err)
}
if _, err := st.CreateIPRangeEntry(tenantID, moduleID, &store.IPRangeEntry{Prefix: prefix, CommunityID: cid}); err != nil {
return nil, err
}
imported++
}
default:
return nil, fmt.Errorf("importer: csv import/export is supported only for AS_PREFIXES, DOMAINS, IP_RANGES")
}
return &Result{Imported: imported, ModuleType: mod.Type}, nil
}
+77
View File
@@ -0,0 +1,77 @@
package jobs
import (
"context"
"os"
"strconv"
"time"
)
const (
defaultJobTimeoutModuleRefresh = 10 * time.Minute
defaultJobTimeoutTenantRefresh = 15 * time.Minute
defaultJobTimeoutDeployApply = 5 * time.Minute
defaultJobTimeoutPeerReconcile = 10 * time.Minute
defaultJobTimeoutRollback = 5 * time.Minute
defaultJobTimeoutBirdReload = 2 * time.Minute
)
func jobTimeout(kind string) time.Duration {
envKey := map[string]string{
KindModuleRefresh: "EVOBGP_JOB_TIMEOUT_MODULE_REFRESH",
KindTenantRefresh: "EVOBGP_JOB_TIMEOUT_TENANT_REFRESH",
KindDeployApply: "EVOBGP_JOB_TIMEOUT_DEPLOY_APPLY",
KindPeerReconcile: "EVOBGP_JOB_TIMEOUT_PEER_RECONCILE",
KindRevisionRollback: "EVOBGP_JOB_TIMEOUT_ROLLBACK",
KindBirdReload: "EVOBGP_JOB_TIMEOUT_BIRD_RELOAD",
}[kind]
if envKey != "" {
if d, err := time.ParseDuration(os.Getenv(envKey)); err == nil && d > 0 {
return d
}
}
switch kind {
case KindModuleRefresh:
return defaultJobTimeoutModuleRefresh
case KindTenantRefresh:
return defaultJobTimeoutTenantRefresh
case KindDeployApply:
return defaultJobTimeoutDeployApply
case KindPeerReconcile:
return defaultJobTimeoutPeerReconcile
case KindRevisionRollback:
return defaultJobTimeoutRollback
case KindBirdReload:
return defaultJobTimeoutBirdReload
default:
if n, err := strconv.Atoi(os.Getenv("EVOBGP_JOB_TIMEOUT_SEC")); err == nil && n > 0 {
return time.Duration(n) * time.Second
}
return defaultJobTimeoutModuleRefresh
}
}
// workContext returns a timeout context that also cancels when the job is cancelled.
func (j *Job) workContext() (context.Context, context.CancelFunc) {
if j == nil {
return context.Background(), func() {}
}
timeout := jobTimeout(j.Kind)
ctx, cancel := context.WithTimeout(context.Background(), timeout)
go func() {
ticker := time.NewTicker(500 * time.Millisecond)
defer ticker.Stop()
for {
select {
case <-ctx.Done():
return
case <-ticker.C:
if j.IsCancelRequested() {
cancel()
return
}
}
}
}()
return ctx, cancel
}
+50 -1
View File
@@ -9,6 +9,8 @@ import (
"sync" "sync"
"time" "time"
"evobgp/internal/observability"
"github.com/google/uuid" "github.com/google/uuid"
) )
@@ -178,6 +180,8 @@ type Registry struct {
byID map[string]*Job byID map[string]*Job
byIdempo map[idempoKey]*Job byIdempo map[idempoKey]*Job
workerStart func(j *Job) workerStart func(j *Job)
workerSem chan struct{}
onTerminal func(j *Job)
} }
type idempoKey struct { type idempoKey struct {
@@ -186,13 +190,44 @@ type idempoKey struct {
} }
func NewRegistry(workerStart func(j *Job)) *Registry { func NewRegistry(workerStart func(j *Job)) *Registry {
maxWorkers := registryMaxConcurrentJobs()
return &Registry{ return &Registry{
byID: make(map[string]*Job), byID: make(map[string]*Job),
byIdempo: make(map[idempoKey]*Job), byIdempo: make(map[idempoKey]*Job),
workerStart: workerStart, workerStart: workerStart,
workerSem: make(chan struct{}, maxWorkers),
} }
} }
// SetTerminalHook registers a best-effort callback when jobs reach a terminal state.
func (r *Registry) SetTerminalHook(fn func(j *Job)) {
if r == nil {
return
}
r.mu.Lock()
defer r.mu.Unlock()
r.onTerminal = fn
}
func (r *Registry) fireTerminal(j *Job) {
if r == nil || j == nil {
return
}
r.mu.RLock()
fn := r.onTerminal
r.mu.RUnlock()
if fn != nil {
fn(j)
}
}
func registryMaxConcurrentJobs() int {
if n, err := strconv.Atoi(strings.TrimSpace(os.Getenv("EVOBGP_JOB_MAX_CONCURRENT"))); err == nil && n > 0 {
return n
}
return 8
}
// pruneTerminalIfOver удаляет самые старые завершённые джобы (succeeded/failed/cancelled), пока len(byID) > maxJobs. // pruneTerminalIfOver удаляет самые старые завершённые джобы (succeeded/failed/cancelled), пока len(byID) > maxJobs.
func (r *Registry) pruneTerminalIfOver(maxJobs int) { func (r *Registry) pruneTerminalIfOver(maxJobs int) {
if r == nil || maxJobs <= 0 || len(r.byID) <= maxJobs { if r == nil || maxJobs <= 0 || len(r.byID) <= maxJobs {
@@ -244,8 +279,12 @@ func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, module
if idempotencyKey != nil && *idempotencyKey != "" { if idempotencyKey != nil && *idempotencyKey != "" {
k := idempoKey{tenant: tenantID, key: *idempotencyKey} k := idempoKey{tenant: tenantID, key: *idempotencyKey}
if existing, ok := r.byIdempo[k]; ok { if existing, ok := r.byIdempo[k]; ok {
st := existing.statusLocked()
if st == StatusQueued || st == StatusRunning {
return existing, false, nil return existing, false, nil
} }
delete(r.byIdempo, k)
}
} }
j := &Job{ j := &Job{
@@ -265,7 +304,17 @@ func (r *Registry) Enqueue(tenantID, kind string, idempotencyKey *string, module
r.pruneTerminalIfOver(maxJobs) r.pruneTerminalIfOver(maxJobs)
if r.workerStart != nil { if r.workerStart != nil {
go r.workerStart(j) go func() {
r.workerSem <- struct{}{}
active := len(r.workerSem)
capacity := cap(r.workerSem)
observability.RecordJobQueueDepth(active, capacity)
defer func() {
<-r.workerSem
observability.RecordJobQueueDepth(len(r.workerSem), capacity)
}()
r.workerStart(j)
}()
} }
return j, true, nil return j, true, nil
} }
+56 -8
View File
@@ -82,6 +82,9 @@ func (w *Worker) httpClient() *http.Client {
func (w *Worker) Process(j *Job) { func (w *Worker) Process(j *Job) {
defer func() { defer func() {
observability.RecordJobTerminal(j.Kind, j.statusLocked()) observability.RecordJobTerminal(j.Kind, j.statusLocked())
if w != nil && w.Registry != nil {
w.Registry.fireTerminal(j)
}
}() }()
if w == nil || w.Store == nil { if w == nil || w.Store == nil {
@@ -102,7 +105,17 @@ func (w *Worker) Process(j *Job) {
j.Fail("missing module_id in job meta") j.Fail("missing module_id in job meta")
return return
} }
if err := pipeline.RefreshModuleIngest(context.Background(), w.Store, w.httpClient(), j.TenantID, mid); err != nil { ctx, cancel := j.workContext()
defer cancel()
if ctx.Err() != nil {
j.MarkCancelled()
return
}
if err := pipeline.RefreshModuleIngest(ctx, w.Store, w.httpClient(), j.TenantID, mid); err != nil {
if ctx.Err() != nil {
j.MarkCancelled()
return
}
j.Fail(err.Error()) j.Fail(err.Error())
return return
} }
@@ -121,11 +134,17 @@ func (w *Worker) Process(j *Job) {
j.Succeed() j.Succeed()
return return
} }
ctx, cancel := j.workContext()
defer cancel()
ctl := &birdfmt.BirdCtl{ ctl := &birdfmt.BirdCtl{
Socket: sock, Socket: sock,
Birdc: strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN")), Birdc: strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_BIN")),
} }
if err := ctl.Configure(context.Background()); err != nil { if err := ctl.Configure(ctx); err != nil {
if ctx.Err() != nil {
j.MarkCancelled()
return
}
j.Fail(err.Error()) j.Fail(err.Error())
return return
} }
@@ -152,9 +171,14 @@ func (w *Worker) runPeerReconcile(j *Job) {
return return
} }
if len(latest) == 0 { if len(latest) == 0 {
// First run fallback: render full tenant state once if no baseline revision exists yet. ctx, cancel := j.workContext()
rid, err := pipeline.RenderTenantRevision(context.Background(), w.Store, w.httpClient(), j.TenantID, triggerModuleID) defer cancel()
rid, err := pipeline.RenderTenantRevision(ctx, w.Store, w.httpClient(), j.TenantID, triggerModuleID)
if err != nil { if err != nil {
if ctx.Err() != nil {
j.MarkCancelled()
return
}
j.Fail(err.Error()) j.Fail(err.Error())
return return
} }
@@ -171,8 +195,14 @@ func (w *Worker) runPeerReconcile(j *Job) {
} }
cursor = next cursor = next
} }
rid, err := pipeline.RenderTenantRevisionFromPrefixes(context.Background(), w.Store, w.httpClient(), j.TenantID, triggerModuleID, rows) ctx, cancel := j.workContext()
defer cancel()
rid, err := pipeline.RenderTenantRevisionFromPrefixes(ctx, w.Store, w.httpClient(), j.TenantID, triggerModuleID, rows)
if err != nil { if err != nil {
if ctx.Err() != nil {
j.MarkCancelled()
return
}
j.Fail(err.Error()) j.Fail(err.Error())
return return
} }
@@ -230,7 +260,13 @@ func (w *Worker) runTenantRefresh(j *Job) {
j.Fail("missing module_ids in job meta") j.Fail("missing module_ids in job meta")
return return
} }
if err := pipeline.RefreshTenantModules(context.Background(), w.Store, w.httpClient(), j.TenantID, moduleIDs); err != nil { ctx, cancel := j.workContext()
defer cancel()
if err := pipeline.RefreshTenantModules(ctx, w.Store, w.httpClient(), j.TenantID, moduleIDs); err != nil {
if ctx.Err() != nil {
j.MarkCancelled()
return
}
j.Fail(err.Error()) j.Fail(err.Error())
return return
} }
@@ -275,6 +311,8 @@ func (w *Worker) finishModuleRefreshSuccess(j *Job, triggerModuleID string) {
mu := w.tenantRefreshMu(j.TenantID) mu := w.tenantRefreshMu(j.TenantID)
mu.Lock() mu.Lock()
defer mu.Unlock() defer mu.Unlock()
ctx, cancel := j.workContext()
defer cancel()
deferDeploy := false deferDeploy := false
if w.Registry != nil { if w.Registry != nil {
deferDeploy = w.Registry.CountOtherActiveRefresh(j.TenantID, j.ID) > 0 deferDeploy = w.Registry.CountOtherActiveRefresh(j.TenantID, j.ID) > 0
@@ -288,8 +326,12 @@ func (w *Worker) finishModuleRefreshSuccess(j *Job, triggerModuleID string) {
return return
} }
rev, err := pipeline.RenderTenantRevision(context.Background(), w.Store, w.httpClient(), j.TenantID, triggerModuleID) rev, err := pipeline.RenderTenantRevision(ctx, w.Store, w.httpClient(), j.TenantID, triggerModuleID)
if err != nil { if err != nil {
if ctx.Err() != nil {
j.MarkCancelled()
return
}
j.Fail(err.Error()) j.Fail(err.Error())
return return
} }
@@ -335,6 +377,8 @@ func (w *Worker) runDeployApply(j *Job) {
j.Fail("missing revision_id in job meta") j.Fail("missing revision_id in job meta")
return return
} }
ctx, cancel := j.workContext()
defer cancel()
activeDir := strings.TrimSpace(os.Getenv("EVOBGP_BIRD_ACTIVE_DIR")) activeDir := strings.TrimSpace(os.Getenv("EVOBGP_BIRD_ACTIVE_DIR"))
if activeDir != "" { if activeDir != "" {
revObj, err := w.Store.GetRevision(j.TenantID, revID) revObj, err := w.Store.GetRevision(j.TenantID, revID)
@@ -354,7 +398,11 @@ func (w *Worker) runDeployApply(j *Job) {
Socket: strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET")), Socket: strings.TrimSpace(os.Getenv("EVOBGP_BIRDC_SOCKET")),
} }
ctl := &birdfmt.BirdCtl{Bird: cfg.BirdBin, Birdc: cfg.BirdcBin, Socket: cfg.Socket} ctl := &birdfmt.BirdCtl{Bird: cfg.BirdBin, Birdc: cfg.BirdcBin, Socket: cfg.Socket}
if err := birddeploy.ApplyRevision(context.Background(), ctl, revObj, cfg); err != nil { if err := birddeploy.ApplyRevision(ctx, ctl, revObj, cfg); err != nil {
if ctx.Err() != nil {
j.MarkCancelled()
return
}
j.Fail(err.Error()) j.Fail(err.Error())
return return
} }
+214
View File
@@ -0,0 +1,214 @@
package nodecli
import (
"context"
"crypto/ed25519"
"encoding/base64"
"encoding/json"
"flag"
"fmt"
"io"
"net/http"
"os"
"path/filepath"
"strings"
"time"
"evobgp/internal/birdfmt"
"evobgp/internal/bundle"
"evobgp/internal/signing"
)
// CmdPullBundle downloads a signed bundle from the control plane API.
func CmdPullBundle(args []string) int {
fs := flag.NewFlagSet("pull-bundle", flag.ExitOnError)
base := fs.String("base-url", "", "control plane base URL, e.g. http://localhost:8080")
token := fs.String("token", "", "Bearer token (node role)")
speaker := fs.String("speaker-id", "", "bgp_speaker id")
revision := fs.String("revision-id", "", "revision to fetch (empty = latest pointer)")
out := fs.String("o", "bundle.tar.gz", "output file")
_ = fs.Parse(args)
if strings.TrimSpace(*base) == "" || *token == "" || *speaker == "" {
fmt.Fprintln(os.Stderr, "pull-bundle: -base-url, -token, -speaker-id are required")
return 2
}
rev := strings.TrimSpace(*revision)
if rev == "" {
var err error
rev, err = fetchLatestRevision(*base, *token, *speaker)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
}
body, err := fetchBundle(*base, *token, *speaker, rev)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
if err := os.WriteFile(*out, body, 0o644); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
fmt.Fprintf(os.Stderr, "wrote %s (revision %s)\n", *out, rev)
return 0
}
func fetchLatestRevision(base, token, speaker string) (string, error) {
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/revisions/latest"
req, err := http.NewRequest(http.MethodGet, u, nil)
if err != nil {
return "", err
}
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return "", err
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body)
return "", fmt.Errorf("latest revision: %s: %s", resp.Status, strings.TrimSpace(string(b)))
}
var out struct {
RevisionID string `json:"revision_id"`
}
if err := json.NewDecoder(resp.Body).Decode(&out); err != nil {
return "", err
}
if out.RevisionID == "" {
return "", fmt.Errorf("empty revision_id in response")
}
return out.RevisionID, nil
}
func fetchBundle(base, token, speaker, revision string) ([]byte, error) {
u := strings.TrimRight(base, "/") + "/v1/speakers/" + speaker + "/bundle/" + revision
req, err := http.NewRequest(http.MethodGet, u, nil)
if err != nil {
return nil, err
}
req.Header.Set("Authorization", "Bearer "+token)
resp, err := http.DefaultClient.Do(req)
if err != nil {
return nil, err
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
b, _ := io.ReadAll(resp.Body)
return nil, fmt.Errorf("bundle: %s: %s", resp.Status, strings.TrimSpace(string(b)))
}
return io.ReadAll(resp.Body)
}
func loadPubKey(pubB64, pubHex string) (ed25519.PublicKey, error) {
switch {
case strings.TrimSpace(pubB64) != "":
raw, err := base64.StdEncoding.DecodeString(strings.TrimSpace(pubB64))
if err != nil {
return nil, err
}
if len(raw) != ed25519.PublicKeySize {
return nil, fmt.Errorf("pubkey-base64 must decode to %d bytes", ed25519.PublicKeySize)
}
return ed25519.PublicKey(raw), nil
case strings.TrimSpace(pubHex) != "":
return bundle.ParsePublicKeyHex(pubHex)
default:
return nil, fmt.Errorf("public key required")
}
}
// CmdVerifyBundle checks bundle signature and manifest.
func CmdVerifyBundle(args []string) int {
fs := flag.NewFlagSet("verify-bundle", flag.ExitOnError)
path := fs.String("f", "", "path to bundle.tar.gz")
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
_ = fs.Parse(args)
if *path == "" {
fmt.Fprintln(os.Stderr, "verify-bundle: -f required")
return 2
}
pub, err := loadPubKey(*pubB64, *pubHex)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 2
}
raw, err := os.ReadFile(*path)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
v, err := signing.VerifyGzippedTar(raw, pub)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
fmt.Fprintf(os.Stderr, "ok: revision %s, %d payload files\n", v.Manifest.RevisionID, len(v.Files))
return 0
}
// CmdApplyBundle verifies, extracts, parse-checks, and configures BIRD from a bundle.
func CmdApplyBundle(args []string) int {
fs := flag.NewFlagSet("apply-bundle", flag.ExitOnError)
path := fs.String("f", "", "path to bundle.tar.gz")
dir := fs.String("extract-dir", "", "directory to extract into")
pubB64 := fs.String("pubkey-base64", "", "Ed25519 public key (base64)")
pubHex := fs.String("pubkey-hex", "", "Ed25519 public key (64 hex chars)")
bird := fs.String("bird", "", "bird binary (default PATH)")
birdc := fs.String("birdc", "", "birdc binary (default PATH)")
socket := fs.String("socket", "", "birdc -s socket path")
timeout := fs.Duration("timeout", 30*time.Second, "timeout for bird/birdc")
_ = fs.Parse(args)
if *path == "" || *dir == "" {
fmt.Fprintln(os.Stderr, "apply-bundle: -f and -extract-dir required")
return 2
}
pub, err := loadPubKey(*pubB64, *pubHex)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 2
}
raw, err := os.ReadFile(*path)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
v, err := signing.VerifyGzippedTar(raw, pub)
if err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
root := filepath.Clean(*dir)
if err := os.MkdirAll(root, 0o755); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
if err := bundle.WriteExtractedFiles(root, v); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
mainRel := v.FindMainBirdConf()
if mainRel == "" {
fmt.Fprintln(os.Stderr, "bundle has no bird.conf path in manifest")
return 1
}
mainPath := filepath.Join(root, filepath.FromSlash(strings.TrimPrefix(mainRel, "/")))
ctx, cancel := context.WithTimeout(context.Background(), *timeout)
defer cancel()
ctl := &birdfmt.BirdCtl{Bird: *bird, Birdc: *birdc, Socket: *socket}
if err := ctl.ParseCheck(ctx, mainPath); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
if err := ctl.Configure(ctx); err != nil {
fmt.Fprintln(os.Stderr, err)
return 1
}
fmt.Fprintf(os.Stderr, "applied revision %s (main config %s)\n", v.Manifest.RevisionID, mainPath)
return 0
}
+38
View File
@@ -0,0 +1,38 @@
// Package nodecli implements evobgp-node subcommands (pull/verify/apply bundle).
package nodecli
import (
"fmt"
"os"
)
// Run executes a subcommand from args (without program name). Returns exit code.
func Run(args []string) int {
if len(args) < 1 {
Usage(os.Stderr)
return 2
}
switch args[0] {
case "pull-bundle":
return CmdPullBundle(args[1:])
case "verify-bundle":
return CmdVerifyBundle(args[1:])
case "apply-bundle":
return CmdApplyBundle(args[1:])
default:
Usage(os.Stderr)
return 2
}
}
// Usage prints CLI help to w.
func Usage(w interface{ Write([]byte) (int, error) }) {
_, _ = fmt.Fprintf(w, `Usage:
evobgp-node pull-bundle -base-url URL -token TOKEN -speaker-id ID [-revision-id ID] [-o path]
evobgp-node verify-bundle -f bundle.tar.gz (-pubkey-base64 B64 | -pubkey-hex HEX)
evobgp-node apply-bundle -f bundle.tar.gz -extract-dir DIR (-pubkey-base64 B64 | -pubkey-hex HEX)
[-bird PATH] [-birdc PATH] [-socket PATH] [-timeout DURATION]
apply-bundle verifies, extracts, runs bird -p on main bird.conf, then birdc configure.
`)
}
+120 -1
View File
@@ -59,8 +59,95 @@ var (
Name: "build_info", Name: "build_info",
Help: "Build metadata (value always 1).", Help: "Build metadata (value always 1).",
}, []string{"version", "git_sha"}) }, []string{"version", "git_sha"})
prefixAggregationDuration = promauto.NewHistogram(prometheus.HistogramOpts{
Namespace: namespace,
Name: "prefix_aggregation_duration_seconds",
Help: "Time spent in smartAggregatePrefixRows during tenant render.",
Buckets: prometheus.ExponentialBuckets(0.0001, 2, 16),
})
prefixAggregationRawCount = promauto.NewHistogram(prometheus.HistogramOpts{
Namespace: namespace,
Name: "prefix_aggregation_raw_count",
Help: "Prefix row count before CIDR aggregation on tenant render.",
Buckets: prometheus.ExponentialBuckets(1, 2, 16),
})
prefixAggregationAggregatedCount = promauto.NewHistogram(prometheus.HistogramOpts{
Namespace: namespace,
Name: "prefix_aggregation_aggregated_count",
Help: "Prefix row count after CIDR aggregation on tenant render.",
Buckets: prometheus.ExponentialBuckets(1, 2, 16),
})
pipelineRefreshDuration = promauto.NewHistogramVec(prometheus.HistogramOpts{
Namespace: namespace,
Name: "pipeline_refresh_duration_seconds",
Help: "Module refresh ingest duration by module type.",
Buckets: prometheus.ExponentialBuckets(0.05, 2, 14),
}, []string{"module_type"})
renderPrefixCount = promauto.NewHistogram(prometheus.HistogramOpts{
Namespace: namespace,
Name: "render_prefix_count",
Help: "Materialized prefix count per tenant render.",
Buckets: prometheus.ExponentialBuckets(10, 2, 16),
})
jobQueueActive = promauto.NewGauge(prometheus.GaugeOpts{
Namespace: namespace,
Name: "job_queue_active",
Help: "Currently running in-process async jobs.",
})
jobQueueCapacity = promauto.NewGauge(prometheus.GaugeOpts{
Namespace: namespace,
Name: "job_queue_capacity",
Help: "Maximum concurrent in-process async jobs.",
})
) )
var (
birdProtocolStatesMu sync.RWMutex
birdProtocolStates map[string]string
birdProtocolStatesAt time.Time
)
// RecordPrefixAggregation records tenant render CIDR aggregation stats.
func RecordPrefixAggregation(rawCount, aggregatedCount int, duration time.Duration) {
if rawCount < 0 {
rawCount = 0
}
if aggregatedCount < 0 {
aggregatedCount = 0
}
prefixAggregationDuration.Observe(duration.Seconds())
prefixAggregationRawCount.Observe(float64(rawCount))
prefixAggregationAggregatedCount.Observe(float64(aggregatedCount))
renderPrefixCount.Observe(float64(aggregatedCount))
}
// RecordPipelineRefresh records module ingest duration.
func RecordPipelineRefresh(moduleType string, duration time.Duration) {
if moduleType == "" {
moduleType = "unknown"
}
pipelineRefreshDuration.WithLabelValues(moduleType).Observe(duration.Seconds())
}
// RecordJobQueueDepth updates in-process job worker utilization gauges.
func RecordJobQueueDepth(active, capacity int) {
if active < 0 {
active = 0
}
if capacity < 0 {
capacity = 0
}
jobQueueActive.Set(float64(active))
jobQueueCapacity.Set(float64(capacity))
}
// RecordJobTerminal increments jobs_finished_total for terminal statuses. // RecordJobTerminal increments jobs_finished_total for terminal statuses.
func RecordJobTerminal(kind, status string) { func RecordJobTerminal(kind, status string) {
switch status { switch status {
@@ -171,6 +258,35 @@ func SetBirdSessionMetrics(established int, scrapeOK bool) {
} }
} }
// SetBirdProtocolStates caches parsed BGP protocol states from the last birdc scrape.
func SetBirdProtocolStates(states map[string]string) {
birdProtocolStatesMu.Lock()
defer birdProtocolStatesMu.Unlock()
if states == nil {
birdProtocolStates = map[string]string{}
} else {
birdProtocolStates = states
}
birdProtocolStatesAt = time.Now()
}
// CachedBirdProtocolStates returns cached protocol states if younger than maxAge.
func CachedBirdProtocolStates(maxAge time.Duration) (map[string]string, bool) {
if maxAge <= 0 {
maxAge = 60 * time.Second
}
birdProtocolStatesMu.RLock()
defer birdProtocolStatesMu.RUnlock()
if birdProtocolStates == nil || time.Since(birdProtocolStatesAt) > maxAge {
return nil, false
}
out := make(map[string]string, len(birdProtocolStates))
for k, v := range birdProtocolStates {
out[k] = v
}
return out, true
}
// MetricsHandler returns the Prometheus scrape handler. // MetricsHandler returns the Prometheus scrape handler.
func MetricsHandler() http.Handler { func MetricsHandler() http.Handler {
return promhttp.HandlerFor(prometheus.DefaultGatherer, promhttp.HandlerOpts{}) return promhttp.HandlerFor(prometheus.DefaultGatherer, promhttp.HandlerOpts{})
@@ -197,7 +313,7 @@ func (s *statusRecorder) WriteHeader(code int) {
// StartBirdProtocolsPoller runs birdc "show protocols" on interval when socket is non-empty. // StartBirdProtocolsPoller runs birdc "show protocols" on interval when socket is non-empty.
// Горутина завершается при отмене ctx (корректное завершение вместе с процессом API). // Горутина завершается при отмене ctx (корректное завершение вместе с процессом API).
func StartBirdProtocolsPoller(ctx context.Context, socket string, birdcPath string, interval time.Duration, showFn func(ctx context.Context, socket, birdcBin string) (string, error), countFn func(output string) int) { func StartBirdProtocolsPoller(ctx context.Context, socket string, birdcPath string, interval time.Duration, showFn func(ctx context.Context, socket, birdcBin string) (string, error), countFn func(output string) int, parseFn func(output string) map[string]string) {
socket = trimSpace(socket) socket = trimSpace(socket)
if ctx == nil || socket == "" || interval <= 0 || showFn == nil || countFn == nil { if ctx == nil || socket == "" || interval <= 0 || showFn == nil || countFn == nil {
return return
@@ -211,6 +327,9 @@ func StartBirdProtocolsPoller(ctx context.Context, socket string, birdcPath stri
return return
} }
SetBirdSessionMetrics(countFn(out), true) SetBirdSessionMetrics(countFn(out), true)
if parseFn != nil {
SetBirdProtocolStates(parseFn(out))
}
} }
go func() { go func() {
scrape() scrape()
+1
View File
@@ -44,6 +44,7 @@ func resolveASNForEntry(ctx context.Context, st store.Backend, hc *http.Client,
if err != nil { if err != nil {
return nil, "", err return nil, "", err
} }
asnresolve.PolitePause()
holder, _ := asnresolve.ASHolderName(ctx, hc, asn) holder, _ := asnresolve.ASHolderName(ctx, hc, asn)
if st != nil { if st != nil {
strs := make([]string, len(pfxs)) strs := make([]string, len(pfxs))
+93
View File
@@ -97,3 +97,96 @@ func TestCollectModulePrefixRows_CDN304UsesSnapshot(t *testing.T) {
t.Fatalf("want cached prefix on 304, got %+v", collected) t.Fatalf("want cached prefix on 304, got %+v", collected)
} }
} }
func TestRefreshModuleIngest_CDN304UsesStoredSnapshot(t *testing.T) {
m := store.NewMemory()
m.SeedDemo()
tenant, _, _, _, _ := m.DemoIDs()
mod, err := m.CreateModule(tenant, &store.Module{
Type: "CDN_CIDRS",
Name: "cdn-304-ingest",
Enabled: true,
})
if err != nil {
t.Fatal(err)
}
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
w.WriteHeader(http.StatusNotModified)
}))
defer srv.Close()
etag := "etag-stable"
src, err := m.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
SourceKind: "txt",
URL: srv.URL,
Etag: etag,
})
if err != nil {
t.Fatal(err)
}
prior := []store.PrefixRow{{
Prefix: "203.0.113.0/24",
Source: cdnSourceKey(src.ID),
}}
if err := mergeCDNSourceIntoModuleSnapshot(m, tenant, mod, src.ID, prior); err != nil {
t.Fatal(err)
}
if err := RefreshModuleIngest(context.Background(), m, srv.Client(), tenant, mod.ID); err != nil {
t.Fatal(err)
}
}
func TestCollectModulePrefixRows_CDN304RetriesWithoutETag(t *testing.T) {
m := store.NewMemory()
m.SeedDemo()
tenant, _, _, _, _ := m.DemoIDs()
mod, err := m.CreateModule(tenant, &store.Module{
Type: "CDN_CIDRS",
Name: "cdn-304-retry",
Enabled: true,
})
if err != nil {
t.Fatal(err)
}
var calls int
srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
if calls == 1 {
if got := strings.TrimSpace(r.Header.Get("If-None-Match")); got != "etag-stable" {
t.Fatalf("first request want If-None-Match etag-stable, got %q", got)
}
w.WriteHeader(http.StatusNotModified)
return
}
if got := strings.TrimSpace(r.Header.Get("If-None-Match")); got != "" {
t.Fatalf("retry must omit If-None-Match, got %q", got)
}
w.Header().Set("ETag", "etag-stable")
_, _ = w.Write([]byte("198.51.100.0/24\n"))
}))
defer srv.Close()
if _, err := m.CreateCDNSource(tenant, mod.ID, &store.CDNSource{
SourceKind: "txt",
URL: srv.URL,
Etag: "etag-stable",
}); err != nil {
t.Fatal(err)
}
collected, err := collectModulePrefixRows(context.Background(), m, srv.Client(), tenant, mod, nil)
if err != nil {
t.Fatal(err)
}
if calls != 2 {
t.Fatalf("want 2 HTTP calls (304 then 200), got %d", calls)
}
if len(collected) != 1 || collected[0].Prefix != "198.51.100.0/24" {
t.Fatalf("unexpected collected rows: %+v", collected)
}
}
+125 -2
View File
@@ -15,6 +15,20 @@ func cdnSourceKey(sourceID string) string {
return "cdn:" + strings.TrimSpace(sourceID) return "cdn:" + strings.TrimSpace(sourceID)
} }
func cachedCDNPrefixRows(st store.Backend, tenantID, moduleID string, priorSnapshot []store.PrefixRow, sourceKey string) []store.PrefixRow {
if cached := prefixRowsForSource(priorSnapshot, sourceKey); len(cached) > 0 {
return cached
}
if st != nil {
if snap, ok, _ := st.GetModulePrefixSnapshot(tenantID, moduleID); ok && snap != nil {
if cached := prefixRowsForSource(snap.Prefixes, sourceKey); len(cached) > 0 {
return cached
}
}
}
return nil
}
func mergeSnapshotDropSource(rows []store.PrefixRow, sourceKey string) []store.PrefixRow { func mergeSnapshotDropSource(rows []store.PrefixRow, sourceKey string) []store.PrefixRow {
if len(rows) == 0 { if len(rows) == 0 {
return nil return nil
@@ -28,6 +42,35 @@ func mergeSnapshotDropSource(rows []store.PrefixRow, sourceKey string) []store.P
return out return out
} }
// mergeSnapshotDropCDNSources removes all cdn:* rows (used before batch CDN merge).
func mergeSnapshotDropCDNSources(rows []store.PrefixRow) []store.PrefixRow {
if len(rows) == 0 {
return nil
}
out := make([]store.PrefixRow, 0, len(rows))
for _, row := range rows {
if !strings.HasPrefix(strings.TrimSpace(row.Source), "cdn:") {
out = append(out, row)
}
}
return out
}
// mergeAllCDNSourcesIntoModuleSnapshot replaces all CDN rows in one write (avoids parallel read-modify-write races).
func mergeAllCDNSourcesIntoModuleSnapshot(st store.Backend, tenantID string, mod *store.Module, priorSnapshot []store.PrefixRow, cdnRows []store.PrefixRow) error {
if st == nil || mod == nil {
return nil
}
var base []store.PrefixRow
if len(priorSnapshot) > 0 {
base = mergeSnapshotDropCDNSources(priorSnapshot)
} else if snap, ok, _ := st.GetModulePrefixSnapshot(tenantID, mod.ID); ok && snap != nil {
base = mergeSnapshotDropCDNSources(snap.Prefixes)
}
merged := append(base, cdnRows...)
return persistModuleSnapshot(st, tenantID, mod, merged)
}
func cdnRowsFromParsed(mod *store.Module, src *store.CDNSource, pfxStrings []string) []store.PrefixRow { func cdnRowsFromParsed(mod *store.Module, src *store.CDNSource, pfxStrings []string) []store.PrefixRow {
var rows []store.PrefixRow var rows []store.PrefixRow
for _, p := range pfxStrings { for _, p := range pfxStrings {
@@ -88,14 +131,29 @@ func applyCDNSourceHTTPResult(ctx context.Context, st store.Backend, hc *http.Cl
if err != nil { if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err) return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
} }
defer resp.Body.Close()
if resp.StatusCode == http.StatusNotModified { if resp.StatusCode == http.StatusNotModified {
if cached := prefixRowsForSource(priorSnapshot, sourceKey); len(cached) > 0 { if cached := cachedCDNPrefixRows(st, tenantID, moduleID, priorSnapshot, sourceKey); len(cached) > 0 {
_ = resp.Body.Close()
return cached, nil return cached, nil
} }
// ETag is known but local snapshot is empty — force a full download.
_ = resp.Body.Close()
req2, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
return nil, err
}
resp, err = hc.Do(req2)
if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
}
if resp.StatusCode == http.StatusNotModified {
_ = resp.Body.Close()
return nil, fmt.Errorf("cdn url %s: 304 without cached prefixes", u) return nil, fmt.Errorf("cdn url %s: 304 without cached prefixes", u)
} }
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
_, _ = io.Copy(io.Discard, resp.Body) _, _ = io.Copy(io.Discard, resp.Body)
return nil, fmt.Errorf("cdn url %s: %s", u, resp.Status) return nil, fmt.Errorf("cdn url %s: %s", u, resp.Status)
@@ -124,3 +182,68 @@ func applyCDNSourceHTTPResult(ctx context.Context, st store.Backend, hc *http.Cl
} }
return rows, nil return rows, nil
} }
// fetchCDNSourceRows loads CDN prefixes without persisting the module snapshot (caller merges once).
func fetchCDNSourceRows(ctx context.Context, st store.Backend, hc *http.Client, tenantID, moduleID string, mod *store.Module, src *store.CDNSource, priorSnapshot []store.PrefixRow, now time.Time) ([]store.PrefixRow, error) {
u := strings.TrimSpace(src.URL)
if u == "" {
return nil, nil
}
sourceKey := cdnSourceKey(src.ID)
req, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
return nil, err
}
if etag := strings.TrimSpace(src.Etag); etag != "" {
req.Header.Set("If-None-Match", etag)
}
resp, err := hc.Do(req)
if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
}
if resp.StatusCode == http.StatusNotModified {
if cached := cachedCDNPrefixRows(st, tenantID, moduleID, priorSnapshot, sourceKey); len(cached) > 0 {
_ = resp.Body.Close()
return cached, nil
}
_ = resp.Body.Close()
req2, err := http.NewRequestWithContext(ctx, http.MethodGet, u, nil)
if err != nil {
return nil, err
}
resp, err = hc.Do(req2)
if err != nil {
return nil, fmt.Errorf("cdn fetch %s: %w", u, err)
}
if resp.StatusCode == http.StatusNotModified {
_ = resp.Body.Close()
return nil, fmt.Errorf("cdn url %s: 304 without cached prefixes", u)
}
}
defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK {
_, _ = io.Copy(io.Discard, resp.Body)
return nil, fmt.Errorf("cdn url %s: %s", u, resp.Status)
}
body, err := io.ReadAll(io.LimitReader(resp.Body, 8<<20))
if err != nil {
return nil, err
}
prefixStrs, err := parseCDNBody(string(body), src)
if err != nil {
return nil, fmt.Errorf("cdn parse %s: %w", u, err)
}
etag := strings.TrimSpace(resp.Header.Get("ETag"))
patch := &store.CDNSourcePatch{}
if etag != "" && etag != strings.TrimSpace(src.Etag) {
e := etag
patch.Etag = &e
}
refreshedAt := now
patch.LastRefreshedAt = &refreshedAt
_, _ = st.UpdateCDNSource(tenantID, moduleID, src.ID, patch)
return cdnRowsFromParsed(mod, src, prefixStrs), nil
}
+22 -7
View File
@@ -97,15 +97,18 @@ func collectASPrefixRows(ctx context.Context, st store.Backend, hc *http.Client,
seenPfx := make(map[string]struct{}) seenPfx := make(map[string]struct{})
var out []store.PrefixRow var out []store.PrefixRow
var metaUpdates []store.ASEntryResolveMetaUpdate
now := time.Now().UTC() now := time.Now().UTC()
for _, r := range results { for _, r := range results {
if r.err != nil { if r.err != nil {
return nil, r.err return nil, r.err
} }
if r.metaID != "" { if r.metaID != "" {
if err := st.UpdateASEntryResolveMeta(tenantID, moduleID, r.metaID, r.holder, r.count, now); err != nil { metaUpdates = append(metaUpdates, store.ASEntryResolveMetaUpdate{
return nil, fmt.Errorf("as entry meta AS%d: %w", r.asn, err) EntryID: r.metaID,
} ASNName: r.holder,
PrefixCount: r.count,
})
} }
for _, row := range r.rows { for _, row := range r.rows {
k := row.Prefix k := row.Prefix
@@ -116,6 +119,11 @@ func collectASPrefixRows(ctx context.Context, st store.Backend, hc *http.Client,
out = append(out, row) out = append(out, row)
} }
} }
if len(metaUpdates) > 0 {
if err := st.UpdateASEntryResolveMetaBatch(tenantID, moduleID, metaUpdates, now); err != nil {
return nil, fmt.Errorf("as entry meta batch: %w", err)
}
}
return out, nil return out, nil
} }
@@ -150,12 +158,14 @@ func collectCDNPrefixRows(ctx context.Context, st store.Backend, hc *http.Client
results[idx] = srcResult{rows: cached} results[idx] = srcResult{rows: cached}
return return
} }
if cached := latestCDNRowsBySource(st, tenantID)[sourceKey]; len(cached) > 0 { if snap, ok, _ := st.GetModulePrefixSnapshot(tenantID, moduleID); ok && snap != nil {
if cached := prefixRowsForSource(snap.Prefixes, sourceKey); len(cached) > 0 {
results[idx] = srcResult{rows: cached} results[idx] = srcResult{rows: cached}
return return
} }
} }
rows, err := applyCDNSourceHTTPResult(ctx, st, hc, tenantID, moduleID, mod, src, priorSnapshot, now) }
rows, err := fetchCDNSourceRows(ctx, st, hc, tenantID, moduleID, mod, src, priorSnapshot, now)
if err != nil { if err != nil {
results[idx] = srcResult{err: err} results[idx] = srcResult{err: err}
return return
@@ -172,10 +182,15 @@ func collectCDNPrefixRows(ctx context.Context, st store.Backend, hc *http.Client
} }
out = append(out, r.rows...) out = append(out, r.rows...)
} }
if len(valid) > 0 {
if err := mergeAllCDNSourcesIntoModuleSnapshot(st, tenantID, mod, priorSnapshot, out); err != nil {
return nil, err
}
}
return out, nil return out, nil
} }
func collectDomainPrefixRows(ctx context.Context, hc *http.Client, mod *store.Module, profile *store.DohProfile, entries []*store.DomainEntry) ([]store.PrefixRow, error) { func collectDomainPrefixRows(ctx context.Context, hc *http.Client, mod *store.Module, profiles []*store.DohProfile, policy string, entries []*store.DomainEntry) ([]store.PrefixRow, error) {
var validDom []*store.DomainEntry var validDom []*store.DomainEntry
for _, e := range entries { for _, e := range entries {
if e != nil { if e != nil {
@@ -202,7 +217,7 @@ func collectDomainPrefixRows(ctx context.Context, hc *http.Client, mod *store.Mo
c := *mod.DefaultCommunityID c := *mod.DefaultCommunityID
comm = &c comm = &c
} }
addrs, err := resolveDomainIPs(ctx, hc, profile, entry.FQDN) addrs, err := resolveDomainIPsWithPolicy(ctx, hc, profiles, policy, entry.FQDN)
if err != nil { if err != nil {
results[idx] = domResult{err: fmt.Errorf("resolve domain %q: %w", entry.FQDN, err)} results[idx] = domResult{err: fmt.Errorf("resolve domain %q: %w", entry.FQDN, err)}
return return
+3
View File
@@ -0,0 +1,3 @@
// Package pipeline ingests module inputs (CDN, ASN, domains, IP ranges), materializes
// prefixes, and creates BIRD preview revisions via store.
package pipeline
+131
View File
@@ -0,0 +1,131 @@
package pipeline
import (
"context"
"fmt"
"net/http"
"net/netip"
"strings"
"time"
"evobgp/internal/store"
"github.com/miekg/dns"
)
func loadModuleDohProfiles(st store.Backend, tenantID string, mod *store.Module) ([]*store.DohProfile, string, error) {
if mod == nil {
return nil, store.DohPolicyPrimaryOnly, nil
}
policy := store.NormalizeDohResolverPolicy(mod.DohResolverPolicy)
var profiles []*store.DohProfile
for _, id := range mod.EffectiveDohProfileIDs() {
prof, err := st.GetDohProfile(tenantID, id)
if err != nil {
return nil, "", fmt.Errorf("get doh profile %s: %w", id, err)
}
profiles = append(profiles, prof)
}
return profiles, policy, nil
}
func resolveDomainIPsWithPolicy(ctx context.Context, hc *http.Client, profiles []*store.DohProfile, policy, fqdn string) ([]netip.Addr, error) {
policy = store.NormalizeDohResolverPolicy(policy)
if len(profiles) == 0 {
return resolveDomainIPs(ctx, hc, nil, fqdn)
}
if len(profiles) == 1 {
return resolveDomainIPs(ctx, hc, profiles[0], fqdn)
}
switch policy {
case store.DohPolicyUnion:
return resolveDomainIPsUnion(ctx, hc, profiles, fqdn)
case store.DohPolicyFailover:
return resolveDomainIPsFailover(ctx, hc, profiles, fqdn)
default:
return resolveDomainIPs(ctx, hc, profiles[0], fqdn)
}
}
func resolveDomainIPsUnion(ctx context.Context, hc *http.Client, profiles []*store.DohProfile, fqdn string) ([]netip.Addr, error) {
var merged []netip.Addr
var errs []error
for _, prof := range profiles {
if prof == nil {
continue
}
ips, err := resolveDomainIPsNoSystemFallback(ctx, hc, prof, fqdn)
if err != nil {
errs = append(errs, fmt.Errorf("%s: %w", strings.TrimSpace(prof.URL), err))
continue
}
merged = append(merged, ips...)
}
merged = uniqAddrs(merged)
if len(merged) > 0 {
return merged, nil
}
if len(errs) > 0 {
return nil, fmt.Errorf("doh union failed: %v", errs)
}
return nil, nil
}
func resolveDomainIPsFailover(ctx context.Context, hc *http.Client, profiles []*store.DohProfile, fqdn string) ([]netip.Addr, error) {
var lastErr error
for _, prof := range profiles {
if prof == nil {
continue
}
ips, err := resolveDomainIPsNoSystemFallback(ctx, hc, prof, fqdn)
if err != nil {
lastErr = err
continue
}
if len(ips) > 0 {
return ips, nil
}
}
if lastErr != nil {
return nil, lastErr
}
return resolveDomainIPs(ctx, hc, nil, fqdn)
}
// resolveDomainIPsNoSystemFallback queries one DoH profile without falling back to OS resolver.
func resolveDomainIPsNoSystemFallback(ctx context.Context, hc *http.Client, profile *store.DohProfile, fqdn string) ([]netip.Addr, error) {
host := strings.TrimSpace(strings.TrimSuffix(fqdn, "."))
if host == "" {
return nil, nil
}
if profile == nil || strings.TrimSpace(profile.URL) == "" {
return nil, fmt.Errorf("empty doh profile")
}
timeout := dohProfileTimeout(profile)
dctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
baseURL := strings.TrimSpace(profile.URL)
v4, err4 := resolveDomainWithDOHMessage(dctx, hc, baseURL, host, dns.TypeA)
v6, err6 := resolveDomainWithDOHMessage(dctx, hc, baseURL, host, dns.TypeAAAA)
if err4 != nil {
v4, err4 = resolveDomainWithDOHJSON(dctx, hc, baseURL, host, "A")
}
if err6 != nil {
v6, err6 = resolveDomainWithDOHJSON(dctx, hc, baseURL, host, "AAAA")
}
if err4 != nil && err6 != nil {
return nil, fmt.Errorf("doh failed for A and AAAA: %v; %v", err4, err6)
}
return uniqAddrs(append(v4, v6...)), nil
}
func dohProfileTimeout(profile *store.DohProfile) time.Duration {
timeout := 10 * time.Second
if profile != nil && profile.TimeoutMs != nil && *profile.TimeoutMs > 0 {
timeout = time.Duration(*profile.TimeoutMs) * time.Millisecond
}
return timeout
}
+148
View File
@@ -0,0 +1,148 @@
package pipeline
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"evobgp/internal/store"
)
func TestResolveDomainIPsWithPolicy_Union(t *testing.T) {
srvRU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.1"}]}`))
}))
defer srvRU.Close()
srvEU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"203.0.113.1"}]}`))
}))
defer srvEU.Close()
profiles := []*store.DohProfile{
{URL: srvRU.URL},
{URL: srvEU.URL},
}
ips, err := resolveDomainIPsWithPolicy(context.Background(), srvRU.Client(), profiles, store.DohPolicyUnion, "example.com")
if err != nil {
t.Fatal(err)
}
if len(ips) != 2 {
t.Fatalf("want 2 ips, got %v", ips)
}
seen := map[string]bool{ips[0].String(): true, ips[1].String(): true}
if !seen["198.51.100.1"] || !seen["203.0.113.1"] {
t.Fatalf("unexpected ips: %v", ips)
}
}
func TestResolveDomainIPsWithPolicy_Failover(t *testing.T) {
var calls int
srvBad := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
http.Error(w, "fail", http.StatusBadGateway)
}))
defer srvBad.Close()
srvOK := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
calls++
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.5"}]}`))
}))
defer srvOK.Close()
profiles := []*store.DohProfile{
{URL: srvBad.URL},
{URL: srvOK.URL},
}
ips, err := resolveDomainIPsWithPolicy(context.Background(), srvBad.Client(), profiles, store.DohPolicyFailover, "example.com")
if err != nil {
t.Fatal(err)
}
if len(ips) != 1 || ips[0].String() != "198.51.100.5" {
t.Fatalf("unexpected ips: %v", ips)
}
if calls < 2 {
t.Fatalf("want at least 2 resolver calls, got %d", calls)
}
}
func TestResolveDomainIPsWithPolicy_PrimaryOnly(t *testing.T) {
var secondCalled bool
srv1 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.9"}]}`))
}))
defer srv1.Close()
srv2 := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
secondCalled = true
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"203.0.113.9"}]}`))
}))
defer srv2.Close()
profiles := []*store.DohProfile{
{URL: srv1.URL},
{URL: srv2.URL},
}
ips, err := resolveDomainIPsWithPolicy(context.Background(), srv1.Client(), profiles, store.DohPolicyPrimaryOnly, "example.com")
if err != nil {
t.Fatal(err)
}
if len(ips) != 1 || ips[0].String() != "198.51.100.9" {
t.Fatalf("unexpected ips: %v", ips)
}
if secondCalled {
t.Fatal("secondary resolver must not be queried in primary_only mode")
}
}
func TestCollectModulePrefixRows_DohUnion(t *testing.T) {
m := store.NewMemory()
m.SeedDemo()
tenant, _, _, _, _ := m.DemoIDs()
mod, err := m.CreateModule(tenant, &store.Module{
Type: "DOMAINS",
Name: "domains-union",
Enabled: true,
DohResolverPolicy: store.DohPolicyUnion,
})
if err != nil {
t.Fatal(err)
}
srvRU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"198.51.100.2"}]}`))
}))
defer srvRU.Close()
srvEU := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
_, _ = w.Write([]byte(`{"Answer":[{"type":1,"data":"203.0.113.2"}]}`))
}))
defer srvEU.Close()
ru, err := m.CreateDohProfile(tenant, &store.DohProfile{Name: "ru", URL: srvRU.URL})
if err != nil {
t.Fatal(err)
}
eu, err := m.CreateDohProfile(tenant, &store.DohProfile{Name: "eu", URL: srvEU.URL})
if err != nil {
t.Fatal(err)
}
if _, err := m.UpdateModule(tenant, mod.ID, &store.ModulePatch{
DohProfileIDs: &[]string{ru.ID, eu.ID},
}); err != nil {
t.Fatal(err)
}
mod, err = m.GetModule(tenant, mod.ID)
if err != nil {
t.Fatal(err)
}
if _, err := m.CreateDomainEntry(tenant, mod.ID, &store.DomainEntry{FQDN: "svc.example.com"}); err != nil {
t.Fatal(err)
}
rows, err := collectModulePrefixRows(context.Background(), m, srvRU.Client(), tenant, mod, nil)
if err != nil {
t.Fatal(err)
}
if len(rows) != 2 {
t.Fatalf("want 2 prefix rows, got %+v", rows)
}
}
@@ -46,4 +46,3 @@ func TestBuildPreviewFragments_SamePrefixDifferentCommunity(t *testing.T) {
t.Fatalf("expected deterministic static preview text, got first:\n%s\nsecond:\n%s", staticV4, staticV4Second) t.Fatalf("expected deterministic static preview text, got first:\n%s\nsecond:\n%s", staticV4, staticV4Second)
} }
} }
+4 -5
View File
@@ -13,7 +13,7 @@ import (
// moduleIngestInputHash fingerprints module config and child entries so snapshots invalidate on CRUD. // moduleIngestInputHash fingerprints module config and child entries so snapshots invalidate on CRUD.
func moduleIngestInputHash(st store.Backend, tenantID string, mod *store.Module) (string, error) { func moduleIngestInputHash(st store.Backend, tenantID string, mod *store.Module) (string, error) {
if st == nil || mod == nil { if st == nil || mod == nil {
return "", fmt.Errorf("module hash: missing store or module") return "", fmt.Errorf("pipeline: module hash: missing store or module")
} }
h := sha256.New() h := sha256.New()
_, _ = fmt.Fprintf(h, "type=%s\n", strings.TrimSpace(mod.Type)) _, _ = fmt.Fprintf(h, "type=%s\n", strings.TrimSpace(mod.Type))
@@ -21,9 +21,9 @@ func moduleIngestInputHash(st store.Backend, tenantID string, mod *store.Module)
if mod.DefaultCommunityID != nil { if mod.DefaultCommunityID != nil {
_, _ = fmt.Fprintf(h, "default_community=%s\n", strings.TrimSpace(*mod.DefaultCommunityID)) _, _ = fmt.Fprintf(h, "default_community=%s\n", strings.TrimSpace(*mod.DefaultCommunityID))
} }
if mod.DohProfileID != nil { _, _ = fmt.Fprintf(h, "doh_policy=%s\n", store.NormalizeDohResolverPolicy(mod.DohResolverPolicy))
_, _ = fmt.Fprintf(h, "doh_profile=%s\n", strings.TrimSpace(*mod.DohProfileID)) for _, pid := range mod.EffectiveDohProfileIDs() {
if pid := strings.TrimSpace(*mod.DohProfileID); pid != "" { _, _ = fmt.Fprintf(h, "doh_profile=%s\n", pid)
if prof, err := st.GetDohProfile(tenantID, pid); err == nil && prof != nil { if prof, err := st.GetDohProfile(tenantID, pid); err == nil && prof != nil {
_, _ = fmt.Fprintf(h, "doh_url=%s\n", strings.TrimSpace(prof.URL)) _, _ = fmt.Fprintf(h, "doh_url=%s\n", strings.TrimSpace(prof.URL))
if prof.TimeoutMs != nil { if prof.TimeoutMs != nil {
@@ -31,7 +31,6 @@ func moduleIngestInputHash(st store.Backend, tenantID string, mod *store.Module)
} }
} }
} }
}
switch mod.Type { switch mod.Type {
case "IP_RANGES": case "IP_RANGES":
+1 -1
View File
@@ -39,7 +39,7 @@ func ParseCIDRLines(body string) []netip.Prefix {
return out return out
} }
// ExtractCIDRs parses CIDRs from either plaintext lines or JSON payload. // ExtractCIDRs parses CIDR prefixes from plaintext lines or a JSON payload (see sourceKind and prefixPath).
// For sourceKind="json", prefixPath supports dotted traversal, with [] for arrays: // For sourceKind="json", prefixPath supports dotted traversal, with [] for arrays:
// e.g. "prefixes[]", "data.items[].cidr". // e.g. "prefixes[]", "data.items[].cidr".
func ExtractCIDRs(body, sourceKind, prefixPath string) ([]netip.Prefix, error) { func ExtractCIDRs(body, sourceKind, prefixPath string) ([]netip.Prefix, error) {
+199 -96
View File
@@ -8,6 +8,7 @@ import (
"encoding/json" "encoding/json"
"fmt" "fmt"
"io" "io"
"math/big"
"net" "net"
"net/http" "net/http"
"net/netip" "net/netip"
@@ -18,6 +19,7 @@ import (
"time" "time"
"evobgp/internal/birdfmt" "evobgp/internal/birdfmt"
"evobgp/internal/observability"
"evobgp/internal/store" "evobgp/internal/store"
"github.com/google/uuid" "github.com/google/uuid"
@@ -34,6 +36,11 @@ const (
revisionDefaultTTL = 30 * 24 * time.Hour revisionDefaultTTL = 30 * 24 * time.Hour
) )
// AuxBirdFullExpandedKey returns the preview map key for the expanded BIRD config (generated on demand).
func AuxBirdFullExpandedKey() string {
return auxBirdFullExpanded
}
// MaterializedASPrefixKey returns the revision snapshot key for an AS-only entry (not a CIDR). // MaterializedASPrefixKey returns the revision snapshot key for an AS-only entry (not a CIDR).
func MaterializedASPrefixKey(asn int64) string { func MaterializedASPrefixKey(asn int64) string {
return fmt.Sprintf("as:%d", asn) return fmt.Sprintf("as:%d", asn)
@@ -45,15 +52,23 @@ func RefreshModuleIngest(ctx context.Context, st store.Backend, hc *http.Client,
if hc == nil { if hc == nil {
hc = http.DefaultClient hc = http.DefaultClient
} }
start := time.Now()
mod, err := st.GetModule(tenantID, moduleID) mod, err := st.GetModule(tenantID, moduleID)
if err != nil { if err != nil {
return err return err
} }
defer func() {
observability.RecordPipelineRefresh(mod.Type, time.Since(start))
}()
if !mod.Enabled { if !mod.Enabled {
return fmt.Errorf("module disabled") return fmt.Errorf("pipeline: module disabled")
} }
rows, err := collectModulePrefixRows(ctx, st, hc, tenantID, mod, nil) var prior []store.PrefixRow
if snap, ok, _ := st.GetModulePrefixSnapshot(tenantID, moduleID); ok && snap != nil {
prior = snap.Prefixes
}
rows, err := collectModulePrefixRows(ctx, st, hc, tenantID, mod, prior)
if err != nil { if err != nil {
return err return err
} }
@@ -75,11 +90,14 @@ func RenderTenantRevision(ctx context.Context, st store.Backend, hc *http.Client
if err != nil { if err != nil {
return "", err return "", err
} }
rawCount := len(agg)
aggStart := time.Now()
agg = smartAggregatePrefixRows(agg)
observability.RecordPrefixAggregation(rawCount, len(agg), time.Since(aggStart))
hash := hashAggregatedMaterializationWithPeers(st, tenantID, agg) hash := hashAggregatedMaterializationWithPeers(st, tenantID, agg)
if prev := latestTenantRevision(st, tenantID); prev != nil && prev.ContentHash == hash { if prev := latestTenantRevision(st, tenantID); prev != nil && prev.ContentHash == hash {
return prev.ID, nil return prev.ID, nil
} }
agg = smartAggregatePrefixRows(agg)
revisionID = uuid.NewString() revisionID = uuid.NewString()
parent := parentRevision(st, tenantID, triggerModuleID) parent := parentRevision(st, tenantID, triggerModuleID)
@@ -102,11 +120,14 @@ func RenderTenantRevisionFromPrefixes(ctx context.Context, st store.Backend, hc
hc = http.DefaultClient hc = http.DefaultClient
} }
agg := append([]store.PrefixRow(nil), rows...) agg := append([]store.PrefixRow(nil), rows...)
rawCount := len(agg)
aggStart := time.Now()
agg = smartAggregatePrefixRows(agg)
observability.RecordPrefixAggregation(rawCount, len(agg), time.Since(aggStart))
hash := hashAggregatedMaterializationWithPeers(st, tenantID, agg) hash := hashAggregatedMaterializationWithPeers(st, tenantID, agg)
if prev := latestTenantRevision(st, tenantID); prev != nil && prev.ContentHash == hash { if prev := latestTenantRevision(st, tenantID); prev != nil && prev.ContentHash == hash {
return prev.ID, nil return prev.ID, nil
} }
agg = smartAggregatePrefixRows(agg)
revisionID = uuid.NewString() revisionID = uuid.NewString()
parent := parentRevision(st, tenantID, triggerModuleID) parent := parentRevision(st, tenantID, triggerModuleID)
@@ -167,16 +188,13 @@ func collectModulePrefixRows(ctx context.Context, st store.Backend, hc *http.Cli
if err != nil { if err != nil {
return nil, err return nil, err
} }
var profile *store.DohProfile profiles, policy, err := loadModuleDohProfiles(st, tenantID, mod)
if mod.DohProfileID != nil && strings.TrimSpace(*mod.DohProfileID) != "" {
profile, err = st.GetDohProfile(tenantID, strings.TrimSpace(*mod.DohProfileID))
if err != nil { if err != nil {
return nil, fmt.Errorf("get doh profile: %w", err) return nil, err
} }
} return collectDomainPrefixRows(ctx, hc, mod, profiles, policy, entries)
return collectDomainPrefixRows(ctx, hc, mod, profile, entries)
default: default:
return nil, fmt.Errorf("unknown module type %q", mod.Type) return nil, fmt.Errorf("pipeline: unknown module type %q", mod.Type)
} }
} }
@@ -188,33 +206,6 @@ func shouldSkipCDNSourceFetch(src *store.CDNSource, now time.Time) bool {
return now.UTC().Before(nextRefreshAt) return now.UTC().Before(nextRefreshAt)
} }
func latestCDNRowsBySource(st store.Backend, tenantID string) map[string][]store.PrefixRow {
out := make(map[string][]store.PrefixRow)
if st == nil {
return out
}
revs, _, _ := st.ListRevisions(tenantID, "", "", 1)
if len(revs) == 0 || strings.TrimSpace(revs[0].ID) == "" {
return out
}
revID := strings.TrimSpace(revs[0].ID)
cursor := ""
for {
page, next, more := st.ListRevisionPrefixes(tenantID, revID, cursor, 2000)
for _, row := range page {
if !strings.HasPrefix(strings.TrimSpace(row.Source), "cdn:") {
continue
}
out[row.Source] = append(out[row.Source], row)
}
if !more || strings.TrimSpace(next) == "" {
break
}
cursor = next
}
return out
}
type dohJSONAnswer struct { type dohJSONAnswer struct {
Type int `json:"type"` Type int `json:"type"`
Data string `json:"data"` Data string `json:"data"`
@@ -297,7 +288,7 @@ func resolveDomainWithDOHMessage(ctx context.Context, hc *http.Client, baseURL,
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1024)) body, _ := io.ReadAll(io.LimitReader(resp.Body, 1024))
return nil, fmt.Errorf("doh dns-message status %s: %s", resp.Status, strings.TrimSpace(string(body))) return nil, fmt.Errorf("doh dns-message status %s: %s", resp.Status, strings.TrimSpace(string(body)))
@@ -369,7 +360,7 @@ func resolveDomainWithDOHJSON(ctx context.Context, hc *http.Client, baseURL, hos
if err != nil { if err != nil {
return nil, err return nil, err
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode != http.StatusOK { if resp.StatusCode != http.StatusOK {
body, _ := io.ReadAll(io.LimitReader(resp.Body, 1024)) body, _ := io.ReadAll(io.LimitReader(resp.Body, 1024))
return nil, fmt.Errorf("doh status %s: %s", resp.Status, strings.TrimSpace(string(body))) return nil, fmt.Errorf("doh status %s: %s", resp.Status, strings.TrimSpace(string(body)))
@@ -425,14 +416,20 @@ type prefixGroupKey struct {
source string source string
} }
// smartAggregatePrefixRows performs "safe" IPv4 CIDR aggregation after full tenant materialization. // smartAggregatePrefixRows performs "safe" IPv4/IPv6 CIDR aggregation after full tenant materialization.
// We aggregate only inside identical community/source groups to preserve BIRD attributes semantics. // We aggregate only inside identical community/source groups to preserve BIRD attributes semantics.
func smartAggregatePrefixRows(rows []store.PrefixRow) []store.PrefixRow { func smartAggregatePrefixRows(rows []store.PrefixRow) []store.PrefixRow {
grouped := make(map[prefixGroupKey][]store.PrefixRow) groupedV4 := make(map[prefixGroupKey][]store.PrefixRow)
groupedV6 := make(map[prefixGroupKey][]store.PrefixRow)
var passthrough []store.PrefixRow var passthrough []store.PrefixRow
for _, row := range rows { for _, row := range rows {
pfx, err := netip.ParsePrefix(strings.TrimSpace(row.Prefix)) p := strings.TrimSpace(row.Prefix)
if err != nil || !pfx.Addr().Is4() { if strings.HasPrefix(p, "as:") {
passthrough = append(passthrough, row)
continue
}
pfx, err := netip.ParsePrefix(p)
if err != nil {
passthrough = append(passthrough, row) passthrough = append(passthrough, row)
continue continue
} }
@@ -442,17 +439,73 @@ func smartAggregatePrefixRows(rows []store.PrefixRow) []store.PrefixRow {
} }
r := row r := row
r.Prefix = pfx.Masked().String() r.Prefix = pfx.Masked().String()
grouped[k] = append(grouped[k], r) switch {
case pfx.Addr().Is4():
groupedV4[k] = append(groupedV4[k], r)
case pfx.Addr().Is6():
groupedV6[k] = append(groupedV6[k], r)
default:
passthrough = append(passthrough, row)
}
} }
out := append([]store.PrefixRow{}, passthrough...) out := append([]store.PrefixRow{}, passthrough...)
for _, grp := range grouped { out = append(out, aggregateGroupedRows(groupedV4, aggregateIPv4Group)...)
out = append(out, aggregateIPv4Group(grp)...) out = append(out, aggregateGroupedRows(groupedV6, aggregateIPv6Group)...)
sortPrefixRows(out)
return out
}
func aggregateGroupedRows(grouped map[prefixGroupKey][]store.PrefixRow, aggregateFn func([]store.PrefixRow) []store.PrefixRow) []store.PrefixRow {
if len(grouped) == 0 {
return nil
}
keys := make([]prefixGroupKey, 0, len(grouped))
for k := range grouped {
keys = append(keys, k)
}
sort.Slice(keys, func(i, j int) bool {
if keys[i].community != keys[j].community {
return keys[i].community < keys[j].community
}
return keys[i].source < keys[j].source
})
var out []store.PrefixRow
for _, k := range keys {
out = append(out, aggregateFn(grouped[k])...)
} }
return out return out
} }
func sortPrefixRows(rows []store.PrefixRow) {
sort.Slice(rows, func(i, j int) bool {
if rows[i].Prefix != rows[j].Prefix {
return rows[i].Prefix < rows[j].Prefix
}
ci, cj := prefixRowCommunity(rows[i]), prefixRowCommunity(rows[j])
if ci != cj {
return ci < cj
}
return rows[i].Source < rows[j].Source
})
}
func prefixRowCommunity(r store.PrefixRow) string {
if r.CommunityID != nil {
return *r.CommunityID
}
return ""
}
func aggregateIPv4Group(rows []store.PrefixRow) []store.PrefixRow { func aggregateIPv4Group(rows []store.PrefixRow) []store.PrefixRow {
return aggregateCIDRGroup(rows, mergeSiblingPrefixesIPv4)
}
func aggregateIPv6Group(rows []store.PrefixRow) []store.PrefixRow {
return aggregateCIDRGroup(rows, mergeSiblingPrefixesIPv6)
}
func aggregateCIDRGroup(rows []store.PrefixRow, mergeFn func(map[string]store.PrefixRow) bool) []store.PrefixRow {
if len(rows) <= 1 { if len(rows) <= 1 {
return rows return rows
} }
@@ -462,7 +515,7 @@ func aggregateIPv4Group(rows []store.PrefixRow) []store.PrefixRow {
} }
pruneCoveredPrefixes(set) pruneCoveredPrefixes(set)
for { for {
if !mergeSiblingPrefixes(set) { if !mergeFn(set) {
break break
} }
pruneCoveredPrefixes(set) pruneCoveredPrefixes(set)
@@ -471,6 +524,7 @@ func aggregateIPv4Group(rows []store.PrefixRow) []store.PrefixRow {
for _, row := range set { for _, row := range set {
out = append(out, row) out = append(out, row)
} }
sortPrefixRows(out)
return out return out
} }
@@ -483,7 +537,7 @@ func pruneCoveredPrefixes(set map[string]store.PrefixRow) {
items := make([]item, 0, len(set)) items := make([]item, 0, len(set))
for k := range set { for k := range set {
p, err := netip.ParsePrefix(k) p, err := netip.ParsePrefix(k)
if err != nil || !p.Addr().Is4() { if err != nil {
continue continue
} }
items = append(items, item{key: k, pfx: p, bits: p.Bits()}) items = append(items, item{key: k, pfx: p, bits: p.Bits()})
@@ -506,7 +560,7 @@ func pruneCoveredPrefixes(set map[string]store.PrefixRow) {
} }
} }
func mergeSiblingPrefixes(set map[string]store.PrefixRow) bool { func mergeSiblingPrefixesIPv4(set map[string]store.PrefixRow) bool {
merged := false merged := false
seen := make(map[string]struct{}, len(set)) seen := make(map[string]struct{}, len(set))
for key, row := range set { for key, row := range set {
@@ -554,6 +608,60 @@ func u32ToIPv4(v uint32) netip.Addr {
return netip.AddrFrom4([4]byte{byte(v >> 24), byte(v >> 16), byte(v >> 8), byte(v)}) return netip.AddrFrom4([4]byte{byte(v >> 24), byte(v >> 16), byte(v >> 8), byte(v)})
} }
func mergeSiblingPrefixesIPv6(set map[string]store.PrefixRow) bool {
merged := false
seen := make(map[string]struct{}, len(set))
for key, row := range set {
if _, done := seen[key]; done {
continue
}
pfx, err := netip.ParsePrefix(key)
if err != nil || !pfx.Addr().Is6() {
continue
}
bits := pfx.Bits()
if bits <= 16 {
continue
}
netNum := ipv6PrefixNetwork(pfx)
blockSize := new(big.Int).Lsh(big.NewInt(1), uint(128-bits))
siblingNet := new(big.Int).Xor(netNum, blockSize)
siblingPfx := ipv6PrefixFromBigInt(siblingNet, bits).String()
if _, ok := set[siblingPfx]; !ok {
continue
}
parentBits := bits - 1
parentBlock := new(big.Int).Lsh(big.NewInt(1), uint(128-parentBits))
mask := new(big.Int).Sub(parentBlock, big.NewInt(1))
mask.Not(mask)
parentNet := new(big.Int).And(netNum, mask)
parentPfx := ipv6PrefixFromBigInt(parentNet, parentBits).String()
delete(set, key)
delete(set, siblingPfx)
parentRow := row
parentRow.Prefix = parentPfx
set[parentPfx] = parentRow
seen[key] = struct{}{}
seen[siblingPfx] = struct{}{}
merged = true
}
return merged
}
func ipv6PrefixNetwork(p netip.Prefix) *big.Int {
a := p.Masked().Addr().As16()
n := new(big.Int)
n.SetBytes(a[:])
return n
}
func ipv6PrefixFromBigInt(n *big.Int, bits int) netip.Prefix {
b := n.Bytes()
var a [16]byte
copy(a[16-len(b):], b)
return netip.PrefixFrom(netip.AddrFrom16(a), bits).Masked()
}
func parentRevision(st store.Backend, tenantID, moduleID string) *string { func parentRevision(st store.Backend, tenantID, moduleID string) *string {
items, _, _ := st.ListRevisions(tenantID, moduleID, "", 1) items, _, _ := st.ListRevisions(tenantID, moduleID, "", 1)
if len(items) == 0 { if len(items) == 0 {
@@ -572,16 +680,19 @@ func latestTenantRevision(st store.Backend, tenantID string) *store.Revision {
return items[0] return items[0]
} }
// hashAggregatedMaterialization hashes the full tenant-wide prefix set used for BIRD (all enabled modules). type prefixHashLine struct{ p, c, s string }
func hashAggregatedMaterialization(tenantID string, rows []store.PrefixRow) string {
type line struct{ p, c, s string } func dedupeSortedPrefixLines(rows []store.PrefixRow) []prefixHashLine {
var lines []line seen := make(map[string]struct{}, len(rows))
lines := make([]prefixHashLine, 0, len(rows))
for _, r := range rows { for _, r := range rows {
c := "" c := prefixRowCommunity(r)
if r.CommunityID != nil { key := r.Prefix + "\x00" + c + "\x00" + r.Source
c = *r.CommunityID if _, ok := seen[key]; ok {
continue
} }
lines = append(lines, line{r.Prefix, c, r.Source}) seen[key] = struct{}{}
lines = append(lines, prefixHashLine{r.Prefix, c, r.Source})
} }
sort.Slice(lines, func(i, j int) bool { sort.Slice(lines, func(i, j int) bool {
if lines[i].p != lines[j].p { if lines[i].p != lines[j].p {
@@ -592,50 +703,34 @@ func hashAggregatedMaterialization(tenantID string, rows []store.PrefixRow) stri
} }
return lines[i].s < lines[j].s return lines[i].s < lines[j].s
}) })
h := sha256.New() return lines
h.Write([]byte(strings.TrimSpace(tenantID))) }
h.Write([]byte{0})
func writePrefixLinesHash(h interface{ Write([]byte) (int, error) }, tenantID string, lines []prefixHashLine) {
_, _ = h.Write([]byte(strings.TrimSpace(tenantID)))
_, _ = h.Write([]byte{0})
for _, l := range lines { for _, l := range lines {
h.Write([]byte(l.p)) _, _ = h.Write([]byte(l.p))
h.Write([]byte{1}) _, _ = h.Write([]byte{1})
h.Write([]byte(l.c)) _, _ = h.Write([]byte(l.c))
h.Write([]byte{1}) _, _ = h.Write([]byte{1})
h.Write([]byte(l.s)) _, _ = h.Write([]byte(l.s))
h.Write([]byte{0}) _, _ = h.Write([]byte{0})
} }
}
// hashAggregatedMaterialization hashes the post-aggregation tenant-wide prefix set used for BIRD.
func hashAggregatedMaterialization(tenantID string, rows []store.PrefixRow) string {
lines := dedupeSortedPrefixLines(rows)
h := sha256.New()
writePrefixLinesHash(h, tenantID, lines)
return fmt.Sprintf("sha256:%x", h.Sum(nil)) return fmt.Sprintf("sha256:%x", h.Sum(nil))
} }
func hashAggregatedMaterializationWithPeers(st store.Backend, tenantID string, rows []store.PrefixRow) string { func hashAggregatedMaterializationWithPeers(st store.Backend, tenantID string, rows []store.PrefixRow) string {
type line struct{ p, c, s string } lines := dedupeSortedPrefixLines(rows)
var lines []line
for _, r := range rows {
c := ""
if r.CommunityID != nil {
c = *r.CommunityID
}
lines = append(lines, line{r.Prefix, c, r.Source})
}
sort.Slice(lines, func(i, j int) bool {
if lines[i].p != lines[j].p {
return lines[i].p < lines[j].p
}
if lines[i].c != lines[j].c {
return lines[i].c < lines[j].c
}
return lines[i].s < lines[j].s
})
h := sha256.New() h := sha256.New()
h.Write([]byte(strings.TrimSpace(tenantID))) writePrefixLinesHash(h, tenantID, lines)
h.Write([]byte{0})
for _, l := range lines {
h.Write([]byte(l.p))
h.Write([]byte{1})
h.Write([]byte(l.c))
h.Write([]byte{1})
h.Write([]byte(l.s))
h.Write([]byte{0})
}
h.Write([]byte("peers")) h.Write([]byte("peers"))
h.Write([]byte{0}) h.Write([]byte{0})
peers := st.ListPeers(tenantID) peers := st.ListPeers(tenantID)
@@ -723,10 +818,18 @@ func buildPreviewFragments(st store.Backend, tenantID, moduleID, revisionID stri
px6: birdfmt.JoinFragments(birdfmt.ManagedBanner(revisionID), staticV6), px6: birdfmt.JoinFragments(birdfmt.ManagedBanner(revisionID), staticV6),
pPeers: peersBody, pPeers: peersBody,
} }
out[auxBirdFullExpanded] = buildExpandedBirdText(main, out)
return out, nil return out, nil
} }
// BuildExpandedBirdPreview concatenates bird.conf and deployable includes for UI preview (not persisted in revision).
func BuildExpandedBirdPreview(frags map[string]string) string {
if frags == nil {
return ""
}
main := frags["bird.conf"]
return buildExpandedBirdText(main, frags)
}
func renderStaticProtocolsByCommunity(groups []staticCommunityRoutes) (string, string) { func renderStaticProtocolsByCommunity(groups []staticCommunityRoutes) (string, string) {
var b4 strings.Builder var b4 strings.Builder
var b6 strings.Builder var b6 strings.Builder
+150
View File
@@ -8,6 +8,15 @@ import (
"evobgp/internal/store" "evobgp/internal/store"
) )
func prefixRowSet(rows []store.PrefixRow) map[string]struct{} {
got := make(map[string]struct{}, len(rows))
for _, r := range rows {
c := prefixRowCommunity(r)
got[r.Prefix+"|"+c+"|"+r.Source] = struct{}{}
}
return got
}
func TestRefreshModule_AggregatesAllEnabledModules(t *testing.T) { func TestRefreshModule_AggregatesAllEnabledModules(t *testing.T) {
t.Setenv("EVOBGP_ASN_RESOLVE", "0") t.Setenv("EVOBGP_ASN_RESOLVE", "0")
@@ -119,3 +128,144 @@ func TestSmartAggregatePrefixRows_RespectsCommunityAndSource(t *testing.T) {
t.Fatalf("expected 3 resulting rows, got %d: %+v", len(out), out) t.Fatalf("expected 3 resulting rows, got %d: %+v", len(out), out)
} }
} }
func TestSmartAggregatePrefixRows_PruneCoveredPrefixes(t *testing.T) {
rows := []store.PrefixRow{
{Prefix: "10.0.0.0/16", Source: "ip_range"},
{Prefix: "10.0.0.0/24", Source: "ip_range"},
{Prefix: "10.0.1.0/24", Source: "ip_range"},
}
out := smartAggregatePrefixRows(rows)
got := prefixRowSet(out)
if len(out) != 1 {
t.Fatalf("expected 1 row after covered prune, got %d: %+v", len(out), out)
}
if _, ok := got["10.0.0.0/16||ip_range"]; !ok {
t.Fatalf("expected /16 only, got: %+v", out)
}
}
func TestSmartAggregatePrefixRows_MergeChainFourSlash24(t *testing.T) {
rows := []store.PrefixRow{
{Prefix: "10.0.0.0/24", Source: "ip_range"},
{Prefix: "10.0.1.0/24", Source: "ip_range"},
{Prefix: "10.0.2.0/24", Source: "ip_range"},
{Prefix: "10.0.3.0/24", Source: "ip_range"},
}
out := smartAggregatePrefixRows(rows)
got := prefixRowSet(out)
if _, ok := got["10.0.0.0/22||ip_range"]; !ok {
t.Fatalf("expected merged /22, got: %+v", out)
}
if len(out) != 1 {
t.Fatalf("expected 1 row, got %d: %+v", len(out), out)
}
}
func TestSmartAggregatePrefixRows_DoesNotMergeAdjacentSlash8(t *testing.T) {
rows := []store.PrefixRow{
{Prefix: "10.0.0.0/8", Source: "ip_range"},
{Prefix: "11.0.0.0/8", Source: "ip_range"},
}
out := smartAggregatePrefixRows(rows)
if len(out) != 2 {
t.Fatalf("expected two /8 prefixes, got %d: %+v", len(out), out)
}
}
func TestSmartAggregatePrefixRows_MergesIPv6Slash64(t *testing.T) {
rows := []store.PrefixRow{
{Prefix: "2001:db8:0:0::/64", Source: "ip_range"},
{Prefix: "2001:db8:0:1::/64", Source: "ip_range"},
}
out := smartAggregatePrefixRows(rows)
got := prefixRowSet(out)
if _, ok := got["2001:db8::/63||ip_range"]; !ok {
t.Fatalf("expected merged IPv6 /63, got: %+v", out)
}
if len(out) != 1 {
t.Fatalf("expected 1 row, got %d: %+v", len(out), out)
}
}
func TestSmartAggregatePrefixRows_StableOutputOrder(t *testing.T) {
rows := []store.PrefixRow{
{Prefix: "192.168.0.0/24", Source: "cdn:b"},
{Prefix: "10.0.0.0/24", Source: "ip_range"},
{Prefix: "10.0.1.0/24", Source: "ip_range"},
}
out1 := smartAggregatePrefixRows(rows)
out2 := smartAggregatePrefixRows(rows)
if len(out1) != len(out2) {
t.Fatalf("length mismatch: %d vs %d", len(out1), len(out2))
}
for i := range out1 {
if out1[i].Prefix != out2[i].Prefix || out1[i].Source != out2[i].Source {
t.Fatalf("order not stable at %d: %+v vs %+v", i, out1[i], out2[i])
}
}
}
func TestRenderTenantRevisionFromPrefixes_SkipsWhenRedundantRawRows(t *testing.T) {
t.Setenv("EVOBGP_ASN_RESOLVE", "0")
m := store.NewMemory()
m.SeedDemo()
tenant, _, modIP, _, _ := m.DemoIDs()
for _, mod := range m.ListModules(tenant) {
if mod == nil || mod.ID == modIP {
continue
}
disabled := false
if _, err := m.UpdateModule(tenant, mod.ID, &store.ModulePatch{Enabled: &disabled}); err != nil {
t.Fatal(err)
}
}
ctx := context.Background()
baseRows := []store.PrefixRow{{Prefix: "10.0.0.0/16", Source: "ip_range"}}
rev1, err := RenderTenantRevisionFromPrefixes(ctx, m, http.DefaultClient, tenant, modIP, baseRows)
if err != nil {
t.Fatal(err)
}
rev2, err := RenderTenantRevisionFromPrefixes(ctx, m, http.DefaultClient, tenant, modIP, append(baseRows,
store.PrefixRow{Prefix: "10.0.0.0/24", Source: "ip_range"},
))
if err != nil {
t.Fatal(err)
}
if rev2 != rev1 {
t.Fatalf("expected hash skip for redundant covered prefix, got rev1=%s rev2=%s", rev1, rev2)
}
before, _, _ := m.ListRevisions(tenant, "", "", 200)
rev3, err := RenderTenantRevisionFromPrefixes(ctx, m, http.DefaultClient, tenant, modIP, append(baseRows,
store.PrefixRow{Prefix: "10.0.0.0/24", Source: "ip_range"},
store.PrefixRow{Prefix: "10.0.0.0/24", Source: "ip_range"},
))
if err != nil {
t.Fatal(err)
}
if rev3 != rev1 {
t.Fatalf("expected hash skip for duplicate raw rows, got rev1=%s rev3=%s", rev1, rev3)
}
after, _, _ := m.ListRevisions(tenant, "", "", 200)
if len(after) != len(before) {
t.Fatalf("duplicate raw rows should not create revision, before=%d after=%d", len(before), len(after))
}
}
func TestHashAggregatedMaterialization_DedupesIdenticalRows(t *testing.T) {
rows := []store.PrefixRow{
{Prefix: "10.0.0.0/24", Source: "ip_range"},
{Prefix: "10.0.0.0/24", Source: "ip_range"},
}
h1 := hashAggregatedMaterialization("tenant-a", rows)
h2 := hashAggregatedMaterialization("tenant-a", []store.PrefixRow{
{Prefix: "10.0.0.0/24", Source: "ip_range"},
})
if h1 != h2 {
t.Fatalf("expected deduped hash to match, got %s vs %s", h1, h2)
}
}
+1 -1
View File
@@ -76,5 +76,5 @@ func PickTenantRefreshTriggerModule(st store.Backend, tenantID string, moduleIDs
return mod.ID, nil return mod.ID, nil
} }
} }
return "", fmt.Errorf("no enabled module for tenant refresh") return "", fmt.Errorf("pipeline: no enabled module for tenant refresh")
} }
+151
View File
@@ -0,0 +1,151 @@
package reports
import (
"time"
"evobgp/internal/store"
)
// RouterListsCatalog is the aggregated router-lists payload for GET /router-lists/catalog.
type RouterListsCatalog struct {
Modules []map[string]any
Domains []map[string]any
ASNs []map[string]any
IPRanges []map[string]any
Communities []map[string]any
}
// BuildRouterListsCatalog loads modules and entries for the tenant catalog endpoint.
func BuildRouterListsCatalog(st store.Backend, tenantID string) (*RouterListsCatalog, error) {
mods := st.ListModules(tenantID)
out := &RouterListsCatalog{
Modules: make([]map[string]any, 0),
Domains: make([]map[string]any, 0),
ASNs: make([]map[string]any, 0),
IPRanges: make([]map[string]any, 0),
Communities: make([]map[string]any, 0),
}
for _, mod := range mods {
switch mod.Type {
case "DOMAINS", "AS_PREFIXES", "IP_RANGES":
out.Modules = append(out.Modules, moduleMap(mod))
default:
continue
}
switch mod.Type {
case "DOMAINS":
list, err := st.ListDomainEntries(tenantID, mod.ID)
if err != nil {
return nil, err
}
for _, x := range list {
out.Domains = append(out.Domains, map[string]any{
"module_id": mod.ID,
"entry": domainEntryMap(x),
})
}
case "AS_PREFIXES":
list, err := st.ListASEntries(tenantID, mod.ID)
if err != nil {
return nil, err
}
for _, x := range list {
out.ASNs = append(out.ASNs, map[string]any{
"module_id": mod.ID,
"entry": asEntryMap(x),
})
}
case "IP_RANGES":
list, err := st.ListIPRangeEntries(tenantID, mod.ID)
if err != nil {
return nil, err
}
for _, x := range list {
out.IPRanges = append(out.IPRanges, map[string]any{
"module_id": mod.ID,
"entry": ipRangeMap(x),
})
}
}
}
comms, err := st.ListCommunities(tenantID)
if err != nil {
return nil, err
}
for _, c := range comms {
out.Communities = append(out.Communities, map[string]any{
"id": c.ID,
"community": c.Community,
"title": c.Title,
})
}
return out, nil
}
func moduleMap(mod *store.Module) map[string]any {
m := map[string]any{
"id": mod.ID,
"type": mod.Type,
"name": mod.Name,
"enabled": mod.Enabled,
"priority": mod.Priority,
"refresh_interval_sec": mod.RefreshIntervalSec,
"cron_expr": mod.CronExpr,
}
if mod.LastRefreshedAt != nil {
m["last_refreshed_at"] = mod.LastRefreshedAt.UTC().Format(time.RFC3339Nano)
} else {
m["last_refreshed_at"] = nil
}
if mod.DefaultCommunityID != nil {
m["default_community_id"] = *mod.DefaultCommunityID
} else {
m["default_community_id"] = nil
}
ids := mod.EffectiveDohProfileIDs()
if len(ids) > 0 {
m["doh_profile_ids"] = ids
} else {
m["doh_profile_ids"] = []string{}
}
m["doh_resolver_policy"] = store.NormalizeDohResolverPolicy(mod.DohResolverPolicy)
if mod.DohProfileID != nil {
m["doh_profile_id"] = *mod.DohProfileID
} else {
m["doh_profile_id"] = nil
}
return m
}
func domainEntryMap(x *store.DomainEntry) map[string]any {
m := map[string]any{"id": x.ID, "fqdn": x.FQDN}
if x.CommunityID != nil {
m["community_id"] = *x.CommunityID
} else {
m["community_id"] = nil
}
return m
}
func asEntryMap(x *store.ASEntry) map[string]any {
m := map[string]any{"id": x.ID, "asn": x.ASN}
if x.CommunityID != nil {
m["community_id"] = *x.CommunityID
} else {
m["community_id"] = nil
}
return m
}
func ipRangeMap(x *store.IPRangeEntry) map[string]any {
m := map[string]any{"id": x.ID, "prefix": x.Prefix}
if x.CommunityID != nil {
m["community_id"] = *x.CommunityID
} else {
m["community_id"] = nil
}
return m
}
+236
View File
@@ -0,0 +1,236 @@
package reports
import (
"fmt"
"sort"
"strconv"
"strings"
"time"
"evobgp/internal/store"
)
// RevisionDiagnosticLog is plain-text diagnostic output for a revision.
type RevisionDiagnosticLog struct {
Body []byte
Filename string
}
// BuildRevisionDiagnosticLog aggregates revision prefixes by source and community.
func BuildRevisionDiagnosticLog(st store.Backend, tenantID string, rev *store.Revision) (*RevisionDiagnosticLog, error) {
moduleType := ""
moduleName := ""
if strings.TrimSpace(rev.ModuleID) != "" {
if mod, modErr := st.GetModule(tenantID, rev.ModuleID); modErr == nil && mod != nil {
moduleType = strings.TrimSpace(mod.Type)
moduleName = strings.TrimSpace(mod.Name)
}
}
communityLabels := map[string]string{}
if communities, listErr := st.ListCommunities(tenantID); listErr == nil {
for _, c := range communities {
if c == nil {
continue
}
label := strings.TrimSpace(c.Title)
if label == "" {
label = strings.TrimSpace(c.Community)
}
if label == "" {
label = c.ID
}
communityLabels[c.ID] = label
}
}
type rawRow struct {
Prefix string
Source string
Kind string
SourceName string
SourceDetail string
CommunityID string
CommunityLabel string
}
type summary struct {
Kind string
Source string
SourceDetail string
CommunityID string
CommunityLabel string
Count int
Sample []string
}
cdnSourceURLByID := map[string]string{}
if moduleType == "CDN_CIDRS" && strings.TrimSpace(rev.ModuleID) != "" {
if sources, srcErr := st.ListCDNSources(tenantID, rev.ModuleID); srcErr == nil {
for _, src := range sources {
if src == nil {
continue
}
id := strings.TrimSpace(src.ID)
url := strings.TrimSpace(src.URL)
if id != "" && url != "" {
cdnSourceURLByID[id] = url
}
}
}
}
rows := make([]rawRow, 0, rev.MaterializedPrefixCount)
byGroup := map[string]*summary{}
cursor := ""
for {
page, next, more := st.ListRevisionPrefixes(tenantID, rev.ID, cursor, 2000)
for _, p := range page {
kind, sourceName := classifyRevisionSource(p.Source)
sourceDetail := sourceName
if kind == "cdn" {
if url, ok := cdnSourceURLByID[sourceName]; ok && strings.TrimSpace(url) != "" {
sourceDetail = url
}
}
communityID := "none"
if p.CommunityID != nil && strings.TrimSpace(*p.CommunityID) != "" {
communityID = strings.TrimSpace(*p.CommunityID)
}
communityLabel := "без community"
if communityID != "none" {
if lbl, ok := communityLabels[communityID]; ok && strings.TrimSpace(lbl) != "" {
communityLabel = lbl
} else {
communityLabel = communityID
}
}
rows = append(rows, rawRow{
Prefix: p.Prefix,
Source: p.Source,
Kind: kind,
SourceName: sourceName,
SourceDetail: sourceDetail,
CommunityID: communityID,
CommunityLabel: communityLabel,
})
groupKey := kind + "|" + sourceName + "|" + communityID
g, ok := byGroup[groupKey]
if !ok {
g = &summary{
Kind: kind,
Source: sourceName,
SourceDetail: sourceDetail,
CommunityID: communityID,
CommunityLabel: communityLabel,
Sample: make([]string, 0, 5),
}
byGroup[groupKey] = g
}
g.Count++
if len(g.Sample) < 5 {
g.Sample = append(g.Sample, p.Prefix)
}
}
if !more || strings.TrimSpace(next) == "" {
break
}
cursor = next
}
summaryKeys := make([]string, 0, len(byGroup))
for k := range byGroup {
summaryKeys = append(summaryKeys, k)
}
sort.Strings(summaryKeys)
sort.Slice(rows, func(i, j int) bool {
if rows[i].Kind != rows[j].Kind {
return rows[i].Kind < rows[j].Kind
}
if rows[i].SourceName != rows[j].SourceName {
return rows[i].SourceName < rows[j].SourceName
}
if rows[i].CommunityID != rows[j].CommunityID {
return rows[i].CommunityID < rows[j].CommunityID
}
return rows[i].Prefix < rows[j].Prefix
})
var b strings.Builder
b.WriteString("EvoBGP revision diagnostic log\n")
b.WriteString("generated_at=" + time.Now().UTC().Format(time.RFC3339Nano) + "\n")
b.WriteString("tenant_id=" + tenantID + "\n")
b.WriteString("revision_id=" + rev.ID + "\n")
b.WriteString("revision_created_at=" + rev.CreatedAt.UTC().Format(time.RFC3339Nano) + "\n")
b.WriteString("content_hash=" + rev.ContentHash + "\n")
b.WriteString("materialized_prefix_count=" + strconv.Itoa(rev.MaterializedPrefixCount) + "\n")
if rev.ModuleID != "" {
b.WriteString("module_id=" + rev.ModuleID + "\n")
} else {
b.WriteString("module_id=\n")
}
b.WriteString("module_type=" + moduleType + "\n")
b.WriteString("module_name=" + moduleName + "\n")
b.WriteString("fetched_rows=" + strconv.Itoa(len(rows)) + "\n\n")
b.WriteString("## Aggregation by source and community\n")
for _, k := range summaryKeys {
g := byGroup[k]
b.WriteString("- kind=" + g.Kind +
" source=" + g.Source +
" source_detail=" + g.SourceDetail +
" community_id=" + g.CommunityID +
" community_label=" + g.CommunityLabel +
" count=" + strconv.Itoa(g.Count))
if len(g.Sample) > 0 {
b.WriteString(" sample=" + strings.Join(g.Sample, ","))
}
b.WriteByte('\n')
}
b.WriteString("\n## Raw rows\n")
b.WriteString("prefix\tkind\tsource\tsource_detail\tcommunity_id\tcommunity_label\n")
for _, row := range rows {
b.WriteString(row.Prefix)
b.WriteByte('\t')
b.WriteString(row.Kind)
b.WriteByte('\t')
b.WriteString(row.Source)
b.WriteByte('\t')
b.WriteString(strings.ReplaceAll(row.SourceDetail, "\t", " "))
b.WriteByte('\t')
b.WriteString(row.CommunityID)
b.WriteByte('\t')
b.WriteString(strings.ReplaceAll(row.CommunityLabel, "\t", " "))
b.WriteByte('\n')
}
filename := fmt.Sprintf("revision-%s-diagnostic.log", shortRevisionID(rev.ID))
return &RevisionDiagnosticLog{Body: []byte(b.String()), Filename: filename}, nil
}
func classifyRevisionSource(src string) (kind, sourceName string) {
switch {
case strings.HasPrefix(src, "as:"):
return "asn", strings.TrimPrefix(src, "as:")
case strings.HasPrefix(src, "domain:"):
return "domain", strings.TrimPrefix(src, "domain:")
case strings.HasPrefix(src, "cdn:"):
return "cdn", strings.TrimPrefix(src, "cdn:")
case src == "ip_range":
return "ip_range", "manual_ranges"
default:
if strings.TrimSpace(src) == "" {
return "unknown", "unknown"
}
return "source", strings.TrimSpace(src)
}
}
func shortRevisionID(id string) string {
s := strings.TrimSpace(id)
if len(s) <= 8 {
return s
}
return s[:8]
}
+70
View File
@@ -0,0 +1,70 @@
package repository
import (
"context"
"os"
"testing"
"evobgp/internal/db"
"evobgp/internal/store"
)
func TestPostgresImplementsBackend(t *testing.T) {
var _ store.Backend = (*Postgres)(nil)
}
func TestPostgresPingIntegration(t *testing.T) {
dsn := os.Getenv("EVOBGP_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("EVOBGP_TEST_DATABASE_URL not set")
}
ctx := context.Background()
pool, err := db.OpenPostgresPool(ctx, dsn)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
pg, err := NewPostgres(ctx, pool, false)
if err != nil {
t.Fatal(err)
}
if err := pg.Ping(ctx); err != nil {
t.Fatalf("ping: %v", err)
}
}
func TestPostgresModuleCRUDIntegration(t *testing.T) {
dsn := os.Getenv("EVOBGP_TEST_DATABASE_URL")
if dsn == "" {
t.Skip("EVOBGP_TEST_DATABASE_URL not set")
}
ctx := context.Background()
pool, err := db.OpenPostgresPool(ctx, dsn)
if err != nil {
t.Fatal(err)
}
defer pool.Close()
pg, err := NewPostgres(ctx, pool, false)
if err != nil {
t.Fatal(err)
}
tenant := "01TESTTENANT00000000000001"
mod, err := pg.CreateModule(tenant, &store.Module{
Name: "audit-test",
Type: "IP_RANGES",
Enabled: true,
})
if err != nil {
t.Fatal(err)
}
got, err := pg.GetModule(tenant, mod.ID)
if err != nil {
t.Fatal(err)
}
if got.Name != "audit-test" {
t.Fatalf("name: got %q", got.Name)
}
if err := pg.SoftDeleteModule(tenant, mod.ID); err != nil {
t.Fatal(err)
}
}
+50
View File
@@ -0,0 +1,50 @@
package repository
import (
"context"
"encoding/json"
"time"
"github.com/jackc/pgx/v5/pgxpool"
)
// JobAuditWriter persists async job lifecycle rows to job_audit (optional cross-process queue foundation).
type JobAuditWriter struct {
pool *pgxpool.Pool
}
func NewJobAuditWriter(pool *pgxpool.Pool) *JobAuditWriter {
if pool == nil {
return nil
}
return &JobAuditWriter{pool: pool}
}
// UpsertRunning inserts or updates a running job row (best-effort).
func (w *JobAuditWriter) UpsertRunning(ctx context.Context, tenantID, jobID, kind string, idempotencyKey *string, meta map[string]any) {
if w == nil || w.pool == nil {
return
}
metaJSON, _ := json.Marshal(meta)
var idem any
if idempotencyKey != nil && *idempotencyKey != "" {
idem = *idempotencyKey
}
_, _ = w.pool.Exec(ctx, `
INSERT INTO job_audit (id, tenant_id, kind, status, idempotency_key, meta_json, created_at, started_at)
VALUES ($1::uuid, $2::uuid, $3, 'running', $4, $5::jsonb, now(), now())
ON CONFLICT (tenant_id, idempotency_key) WHERE idempotency_key IS NOT NULL
DO UPDATE SET status='running', started_at=now(), meta_json=EXCLUDED.meta_json`,
jobID, tenantID, kind, idem, metaJSON)
}
// MarkTerminal updates job_audit terminal state (best-effort).
func (w *JobAuditWriter) MarkTerminal(ctx context.Context, tenantID, jobID, status string, errMsg *string, finishedAt time.Time) {
if w == nil || w.pool == nil {
return
}
_, _ = w.pool.Exec(ctx, `
UPDATE job_audit SET status=$3, error_message=$4, finished_at=$5
WHERE id=$1::uuid AND tenant_id=$2::uuid`,
jobID, tenantID, status, errMsg, finishedAt.UTC())
}
+178 -60
View File
@@ -28,7 +28,7 @@ func agentDebugNDJSON3214(hypothesisID, location, message string, data map[strin
if err != nil { if err != nil {
return return
} }
defer f.Close() defer func() { _ = f.Close() }()
var ms runtime.MemStats var ms runtime.MemStats
runtime.ReadMemStats(&ms) runtime.ReadMemStats(&ms)
payload := map[string]any{ payload := map[string]any{
@@ -71,6 +71,11 @@ func (p *Postgres) DemoIDs() (tenant, moduleCDN, moduleIP, revision, speaker str
return p.demoTenant, p.demoCDN, p.demoIP, p.demoRev, p.demoSpk return p.demoTenant, p.demoCDN, p.demoIP, p.demoRev, p.demoSpk
} }
// Ping checks PostgreSQL connectivity.
func (p *Postgres) Ping(ctx context.Context) error {
return p.pool.Ping(ctx)
}
func (p *Postgres) MaterializedPrefixStats() (max int, sum int) { func (p *Postgres) MaterializedPrefixStats() (max int, sum int) {
ctx := context.Background() ctx := context.Background()
// Агрегация в БД — не тащим все строки config_revision в память. // Агрегация в БД — не тащим все строки config_revision в память.
@@ -113,22 +118,25 @@ func (p *Postgres) PeerSessionCountsByState() map[string]int {
func (p *Postgres) ListModules(tenantID string) []*store.Module { func (p *Postgres) ListModules(tenantID string) []*store.Module {
ctx := context.Background() ctx := context.Background()
rows, err := p.pool.Query(ctx, ` rows, err := p.pool.Query(ctx, `
SELECT id, type, name, enabled, priority, doh_profile_id::text, refresh_interval_sec, cron_expr, default_community_id::text, last_refreshed_at SELECT id, type, name, enabled, priority, doh_profile_id::text, doh_resolver_policy,
refresh_interval_sec, cron_expr, default_community_id::text, last_refreshed_at
FROM module WHERE tenant_id = $1 AND deleted_at IS NULL ORDER BY priority, name`, tenantID) FROM module WHERE tenant_id = $1 AND deleted_at IS NULL ORDER BY priority, name`, tenantID)
if err != nil { if err != nil {
return nil return nil
} }
defer rows.Close() defer rows.Close()
var out []*store.Module var out []*store.Module
moduleByID := make(map[string]*store.Module)
for rows.Next() { for rows.Next() {
var m store.Module var m store.Module
m.TenantID = tenantID m.TenantID = tenantID
var doh, dc, cron *string var doh, dc, cron *string
var refresh *int32 var refresh *int32
var last *time.Time var last *time.Time
if err := rows.Scan(&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &refresh, &cron, &dc, &last); err != nil { if err := rows.Scan(&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &m.DohResolverPolicy, &refresh, &cron, &dc, &last); err != nil {
continue continue
} }
m.DohResolverPolicy = store.NormalizeDohResolverPolicy(m.DohResolverPolicy)
if refresh != nil { if refresh != nil {
m.RefreshIntervalSec = int(*refresh) m.RefreshIntervalSec = int(*refresh)
} }
@@ -146,10 +154,83 @@ func (p *Postgres) ListModules(tenantID string) []*store.Module {
m.LastRefreshedAt = &t m.LastRefreshedAt = &t
} }
out = append(out, &m) out = append(out, &m)
moduleByID[m.ID] = &m
}
if err := p.batchFillModuleDohFields(ctx, moduleByID); err != nil {
return nil
} }
return out return out
} }
func (p *Postgres) ListModulesPage(tenantID, cursor string, limit int) ([]*store.Module, string, bool) {
if limit <= 0 {
limit = 50
}
off := 0
if cursor != "" {
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
off = n
}
}
ctx := context.Background()
rows, err := p.pool.Query(ctx, `
SELECT id, type, name, enabled, priority, doh_profile_id::text, doh_resolver_policy,
refresh_interval_sec, cron_expr, default_community_id::text, last_refreshed_at
FROM module WHERE tenant_id = $1 AND deleted_at IS NULL
ORDER BY priority, name
LIMIT $2 OFFSET $3`, tenantID, limit+1, off)
if err != nil {
return nil, "", false
}
defer rows.Close()
var out []*store.Module
moduleByID := make(map[string]*store.Module)
for rows.Next() {
var m store.Module
m.TenantID = tenantID
var doh, dc, cron *string
var refresh *int32
var last *time.Time
if err := rows.Scan(&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &m.DohResolverPolicy, &refresh, &cron, &dc, &last); err != nil {
continue
}
m.DohResolverPolicy = store.NormalizeDohResolverPolicy(m.DohResolverPolicy)
if refresh != nil {
m.RefreshIntervalSec = int(*refresh)
}
if cron != nil {
m.CronExpr = *cron
}
if doh != nil && *doh != "" {
m.DohProfileID = doh
}
if dc != nil && *dc != "" {
m.DefaultCommunityID = dc
}
if last != nil {
t := last.UTC()
m.LastRefreshedAt = &t
}
out = append(out, &m)
moduleByID[m.ID] = &m
}
if err := p.batchFillModuleDohFields(ctx, moduleByID); err != nil {
return nil, "", false
}
more := len(out) > limit
if more {
out = out[:limit]
}
next := ""
if more {
next = fmt.Sprintf("%d", off+limit)
}
if len(out) == 0 {
return nil, "", false
}
return out, next, more
}
func (p *Postgres) GetModule(tenantID, moduleID string) (*store.Module, error) { func (p *Postgres) GetModule(tenantID, moduleID string) (*store.Module, error) {
ctx := context.Background() ctx := context.Background()
var m store.Module var m store.Module
@@ -158,9 +239,10 @@ func (p *Postgres) GetModule(tenantID, moduleID string) (*store.Module, error) {
var refresh *int32 var refresh *int32
var last *time.Time var last *time.Time
err := p.pool.QueryRow(ctx, ` err := p.pool.QueryRow(ctx, `
SELECT id, type, name, enabled, priority, doh_profile_id::text, refresh_interval_sec, cron_expr, default_community_id::text, last_refreshed_at SELECT id, type, name, enabled, priority, doh_profile_id::text, doh_resolver_policy,
refresh_interval_sec, cron_expr, default_community_id::text, last_refreshed_at
FROM module WHERE id = $1 AND tenant_id = $2 AND deleted_at IS NULL`, moduleID, tenantID).Scan( FROM module WHERE id = $1 AND tenant_id = $2 AND deleted_at IS NULL`, moduleID, tenantID).Scan(
&m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &refresh, &cron, &dc, &last) &m.ID, &m.Type, &m.Name, &m.Enabled, &m.Priority, &doh, &m.DohResolverPolicy, &refresh, &cron, &dc, &last)
if err != nil { if err != nil {
if errors.Is(err, pgx.ErrNoRows) { if errors.Is(err, pgx.ErrNoRows) {
return nil, store.ErrNotFound return nil, store.ErrNotFound
@@ -183,6 +265,10 @@ func (p *Postgres) GetModule(tenantID, moduleID string) (*store.Module, error) {
t := last.UTC() t := last.UTC()
m.LastRefreshedAt = &t m.LastRefreshedAt = &t
} }
m.DohResolverPolicy = store.NormalizeDohResolverPolicy(m.DohResolverPolicy)
if err := p.fillModuleDohFields(ctx, &m); err != nil {
return nil, err
}
return &m, nil return &m, nil
} }
@@ -190,6 +276,7 @@ func (p *Postgres) CreateModule(tenantID string, in *store.Module) (*store.Modul
if in == nil { if in == nil {
return nil, store.ErrInvalidInput return nil, store.ErrInvalidInput
} }
store.NormalizeModuleDoh(in)
ctx := context.Background() ctx := context.Background()
id := uuid.NewString() id := uuid.NewString()
var doh, dc any var doh, dc any
@@ -211,13 +298,17 @@ func (p *Postgres) CreateModule(tenantID string, in *store.Module) (*store.Modul
if in.LastRefreshedAt != nil { if in.LastRefreshedAt != nil {
lastArg = in.LastRefreshedAt.UTC() lastArg = in.LastRefreshedAt.UTC()
} }
policy := store.NormalizeDohResolverPolicy(in.DohResolverPolicy)
_, err := p.pool.Exec(ctx, ` _, err := p.pool.Exec(ctx, `
INSERT INTO module (id, tenant_id, type, name, enabled, priority, doh_profile_id, refresh_interval_sec, cron_expr, default_community_id, last_refreshed_at) INSERT INTO module (id, tenant_id, type, name, enabled, priority, doh_profile_id, doh_resolver_policy, refresh_interval_sec, cron_expr, default_community_id, last_refreshed_at)
VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11)`, VALUES ($1,$2,$3,$4,$5,$6,$7,$8,$9,$10,$11,$12)`,
id, tenantID, in.Type, in.Name, in.Enabled, in.Priority, doh, ri, cronArg, dc, lastArg) id, tenantID, in.Type, in.Name, in.Enabled, in.Priority, doh, policy, ri, cronArg, dc, lastArg)
if err != nil { if err != nil {
return nil, err return nil, err
} }
if err := p.setModuleDohProfiles(ctx, id, in.DohProfileIDs); err != nil {
return nil, err
}
return p.GetModule(tenantID, id) return p.GetModule(tenantID, id)
} }
@@ -230,77 +321,68 @@ func (p *Postgres) UpdateModule(tenantID, moduleID string, patch *store.ModulePa
if err != nil { if err != nil {
return nil, err return nil, err
} }
name := base.Name work := *base
en := base.Enabled
pr := base.Priority
ri := base.RefreshIntervalSec
cron := base.CronExpr
var dc, doh *string
dc = base.DefaultCommunityID
doh = base.DohProfileID
last := base.LastRefreshedAt
if patch.Name != nil { if patch.Name != nil {
name = strings.TrimSpace(*patch.Name) work.Name = strings.TrimSpace(*patch.Name)
} }
if patch.Enabled != nil { if patch.Enabled != nil {
en = *patch.Enabled work.Enabled = *patch.Enabled
} }
if patch.Priority != nil { if patch.Priority != nil {
pr = *patch.Priority work.Priority = *patch.Priority
} }
if patch.RefreshIntervalSec != nil { if patch.RefreshIntervalSec != nil {
ri = *patch.RefreshIntervalSec work.RefreshIntervalSec = *patch.RefreshIntervalSec
} }
if patch.CronExpr != nil { if patch.CronExpr != nil {
cron = *patch.CronExpr work.CronExpr = *patch.CronExpr
} }
if patch.DefaultCommunityID != nil { if patch.DefaultCommunityID != nil {
v := strings.TrimSpace(*patch.DefaultCommunityID) v := strings.TrimSpace(*patch.DefaultCommunityID)
if v == "" { if v == "" {
dc = nil work.DefaultCommunityID = nil
} else { } else {
dc = &v work.DefaultCommunityID = &v
}
}
if patch.DohProfileID != nil {
v := strings.TrimSpace(*patch.DohProfileID)
if v == "" {
doh = nil
} else {
doh = &v
} }
} }
store.ApplyModuleDohPatch(&work, patch)
if patch.LastRefreshedAt != nil { if patch.LastRefreshedAt != nil {
t := patch.LastRefreshedAt.UTC() t := patch.LastRefreshedAt.UTC()
last = &t work.LastRefreshedAt = &t
} }
var dcArg, dohArg any var dcArg, dohArg any
if dc != nil { if work.DefaultCommunityID != nil {
dcArg = *dc dcArg = *work.DefaultCommunityID
} }
if doh != nil { if work.DohProfileID != nil {
dohArg = *doh dohArg = *work.DohProfileID
} }
var riArg any var riArg any
if ri != 0 { if work.RefreshIntervalSec != 0 {
riArg = ri riArg = work.RefreshIntervalSec
} }
var cronArg any var cronArg any
if strings.TrimSpace(cron) != "" { if strings.TrimSpace(work.CronExpr) != "" {
cronArg = strings.TrimSpace(cron) cronArg = strings.TrimSpace(work.CronExpr)
} }
var lastArg any var lastArg any
if last != nil { if work.LastRefreshedAt != nil {
lastArg = last.UTC() lastArg = work.LastRefreshedAt.UTC()
} }
policy := store.NormalizeDohResolverPolicy(work.DohResolverPolicy)
_, err = p.pool.Exec(ctx, ` _, err = p.pool.Exec(ctx, `
UPDATE module SET name=$3, enabled=$4, priority=$5, refresh_interval_sec=$6, cron_expr=$7, UPDATE module SET name=$3, enabled=$4, priority=$5, refresh_interval_sec=$6, cron_expr=$7,
default_community_id=$8, doh_profile_id=$9, last_refreshed_at=$10, updated_at=now() default_community_id=$8, doh_profile_id=$9, doh_resolver_policy=$10, last_refreshed_at=$11, updated_at=now()
WHERE id=$1 AND tenant_id=$2 AND deleted_at IS NULL`, WHERE id=$1 AND tenant_id=$2 AND deleted_at IS NULL`,
moduleID, tenantID, name, en, pr, riArg, cronArg, dcArg, dohArg, lastArg) moduleID, tenantID, work.Name, work.Enabled, work.Priority, riArg, cronArg, dcArg, dohArg, policy, lastArg)
if err != nil { if err != nil {
return nil, err return nil, err
} }
if patch.DohProfileIDs != nil || patch.DohProfileID != nil {
if err := p.setModuleDohProfiles(ctx, moduleID, work.DohProfileIDs); err != nil {
return nil, err
}
}
return p.GetModule(tenantID, moduleID) return p.GetModule(tenantID, moduleID)
} }
@@ -674,7 +756,13 @@ func (p *Postgres) ListRevisionPrefixes(tenantID, revisionID string, cursor stri
} }
} }
ctx := context.Background() ctx := context.Background()
if _, err := p.GetRevision(tenantID, revisionID); err != nil { var one int
if err := p.pool.QueryRow(ctx, `
SELECT 1 FROM config_revision WHERE id = $1::uuid AND tenant_id = $2::uuid`,
revisionID, tenantID).Scan(&one); err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, "", false
}
return nil, "", false return nil, "", false
} }
rows, err := p.pool.Query(ctx, ` rows, err := p.pool.Query(ctx, `
@@ -743,6 +831,8 @@ func (p *Postgres) CreateRollbackRevision(tenantID, sourceRevisionID string) (st
return newID, nil return newID, nil
} }
const maxRevisionDiffRows = 5000
func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, error) { func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, error) {
if _, err := p.GetRevision(tenantID, aID); err != nil { if _, err := p.GetRevision(tenantID, aID); err != nil {
return nil, err return nil, err
@@ -767,7 +857,7 @@ func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, erro
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
EXCEPT EXCEPT
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
) s ORDER BY 1`, bID, aID) ) s ORDER BY 1 LIMIT $3`, bID, aID, maxRevisionDiffRows+1)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -779,13 +869,18 @@ func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, erro
continue continue
} }
added = append(added, s) added = append(added, s)
if len(added) > maxRevisionDiffRows {
added = added[:maxRevisionDiffRows]
break
} }
}
addedTruncated := len(added) >= maxRevisionDiffRows
rowsRem, err := p.pool.Query(ctx, ` rowsRem, err := p.pool.Query(ctx, `
SELECT prefix::text FROM ( SELECT prefix::text FROM (
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$1::uuid
EXCEPT EXCEPT
SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid SELECT prefix FROM revision_materialized_prefix WHERE revision_id=$2::uuid
) s ORDER BY 1`, aID, bID) ) s ORDER BY 1 LIMIT $3`, aID, bID, maxRevisionDiffRows+1)
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -797,40 +892,56 @@ func (p *Postgres) RevisionDiff(tenantID, aID, bID string) (map[string]any, erro
continue continue
} }
removed = append(removed, s) removed = append(removed, s)
if len(removed) > maxRevisionDiffRows {
removed = removed[:maxRevisionDiffRows]
break
}
} }
return map[string]any{ return map[string]any{
"revision_a": aID, "revision_a": aID,
"revision_b": bID, "revision_b": bID,
"prefixes": map[string]any{ "prefixes": map[string]any{
"added": added, "removed": removed, "unchanged_count": unchanged, "added": added, "removed": removed, "unchanged_count": unchanged,
"truncated": addedTruncated || len(removed) >= maxRevisionDiffRows,
}, },
}, nil }, nil
} }
func (p *Postgres) PruneRevisionsBefore(tenantID string, cutoff time.Time) (int, error) { func (p *Postgres) PruneRevisionsBefore(tenantID string, cutoff time.Time) (int, error) {
ctx := context.Background() ctx := context.Background()
total := 0
const batchSize = 50
for {
cmd, err := p.pool.Exec(ctx, ` cmd, err := p.pool.Exec(ctx, `
DELETE FROM config_revision AS cr DELETE FROM config_revision AS cr
WHERE cr.tenant_id = $1 WHERE cr.id IN (
AND cr.created_at < $2 SELECT id FROM config_revision
AND cr.id <> ( WHERE tenant_id = $1
SELECT id AND created_at < $2
FROM config_revision AND id <> (
SELECT id FROM config_revision
WHERE tenant_id = $1 WHERE tenant_id = $1
ORDER BY created_at DESC ORDER BY created_at DESC
LIMIT 1 LIMIT 1
) )
AND NOT EXISTS ( AND NOT EXISTS (
SELECT 1 SELECT 1 FROM bgp_speaker AS sp
FROM bgp_speaker AS sp
WHERE sp.tenant_id = $1 WHERE sp.tenant_id = $1
AND (sp.last_applied_revision_id = cr.id OR sp.published_revision_id = cr.id) AND (sp.last_applied_revision_id = config_revision.id OR sp.published_revision_id = config_revision.id)
)`, )
tenantID, cutoff.UTC()) ORDER BY created_at ASC
LIMIT $3
)`, tenantID, cutoff.UTC(), batchSize)
if err != nil { if err != nil {
return 0, err return total, err
} }
return int(cmd.RowsAffected()), nil n := int(cmd.RowsAffected())
total += n
if n < batchSize {
break
}
}
return total, nil
} }
func (p *Postgres) SetLastAppliedRevision(tenantID, speakerID, revisionID string) error { func (p *Postgres) SetLastAppliedRevision(tenantID, speakerID, revisionID string) error {
@@ -1041,6 +1152,13 @@ func (p *Postgres) UpdateDohProfile(tenantID, id string, patch *store.DohProfile
func (p *Postgres) DeleteDohProfile(tenantID, id string) error { func (p *Postgres) DeleteDohProfile(tenantID, id string) error {
ctx := context.Background() ctx := context.Background()
inUse, err := p.moduleDohProfileInUse(ctx, id)
if err != nil {
return err
}
if inUse {
return store.ErrInvalidInput
}
var n int var n int
_ = p.pool.QueryRow(ctx, `SELECT COUNT(*) FROM module WHERE doh_profile_id=$1::uuid AND deleted_at IS NULL`, id).Scan(&n) _ = p.pool.QueryRow(ctx, `SELECT COUNT(*) FROM module WHERE doh_profile_id=$1::uuid AND deleted_at IS NULL`, id).Scan(&n)
if n > 0 { if n > 0 {
+203
View File
@@ -0,0 +1,203 @@
package repository
import (
"context"
"errors"
"strings"
"time"
"evobgp/internal/authkey"
"evobgp/internal/store"
"github.com/google/uuid"
"github.com/jackc/pgx/v5"
)
func (p *Postgres) ListAPIKeys(tenantID string) ([]*store.APIKey, error) {
ctx := context.Background()
rows, err := p.pool.Query(ctx, `
SELECT id::text, name, role, token_prefix, created_at, updated_at, expires_at, revoked_at, last_used_at
FROM api_key WHERE tenant_id=$1 ORDER BY created_at DESC`, tenantID)
if err != nil {
return nil, err
}
defer rows.Close()
var out []*store.APIKey
for rows.Next() {
k, err := scanAPIKeyRow(rows.Scan, tenantID)
if err != nil {
return nil, err
}
out = append(out, k)
}
return out, rows.Err()
}
func (p *Postgres) GetAPIKey(tenantID, id string) (*store.APIKey, error) {
ctx := context.Background()
row := p.pool.QueryRow(ctx, `
SELECT id::text, name, role, token_prefix, created_at, updated_at, expires_at, revoked_at, last_used_at
FROM api_key WHERE id=$1 AND tenant_id=$2`, id, tenantID)
k, err := scanAPIKeyRow(row.Scan, tenantID)
if err != nil {
if errors.Is(err, pgx.ErrNoRows) {
return nil, store.ErrNotFound
}
return nil, err
}
return k, nil
}
func (p *Postgres) CreateAPIKey(tenantID string, in *store.APIKeyCreate) (*store.APIKeyWithSecret, error) {
if in == nil || strings.TrimSpace(in.Name) == "" || !store.ValidAPIKeyRole(in.Role) {
return nil, store.ErrInvalidInput
}
tok, err := authkey.GenerateToken()
if err != nil {
return nil, err
}
id := uuid.NewString()
hash := authkey.HashToken(tok)
prefix := authkey.Prefix(tok)
role := strings.ToLower(strings.TrimSpace(in.Role))
ctx := context.Background()
_, err = p.pool.Exec(ctx, `
INSERT INTO api_key (id, tenant_id, name, role, token_prefix, token_hash, expires_at)
VALUES ($1,$2,$3,$4,$5,$6,$7)`,
id, tenantID, strings.TrimSpace(in.Name), role, prefix, hash, in.ExpiresAt)
if err != nil {
return nil, err
}
k, err := p.GetAPIKey(tenantID, id)
if err != nil {
return nil, err
}
return &store.APIKeyWithSecret{APIKey: *k, Token: tok}, nil
}
func (p *Postgres) UpdateAPIKey(tenantID, id string, patch *store.APIKeyPatch) (*store.APIKey, error) {
cur, err := p.GetAPIKey(tenantID, id)
if err != nil {
return nil, err
}
if cur.RevokedAt != nil {
return nil, store.ErrInvalidInput
}
if patch == nil {
return nil, store.ErrInvalidInput
}
if patch.Name != nil {
n := strings.TrimSpace(*patch.Name)
if n == "" {
return nil, store.ErrInvalidInput
}
cur.Name = n
}
if patch.Role != nil {
if !store.ValidAPIKeyRole(*patch.Role) {
return nil, store.ErrInvalidInput
}
cur.Role = strings.ToLower(strings.TrimSpace(*patch.Role))
}
if patch.ClearExpiresAt {
cur.ExpiresAt = nil
} else if patch.ExpiresAt != nil {
cur.ExpiresAt = patch.ExpiresAt
}
ctx := context.Background()
_, err = p.pool.Exec(ctx, `
UPDATE api_key SET name=$3, role=$4, expires_at=$5, updated_at=now()
WHERE id=$1 AND tenant_id=$2 AND revoked_at IS NULL`,
id, tenantID, cur.Name, cur.Role, cur.ExpiresAt)
if err != nil {
return nil, err
}
return p.GetAPIKey(tenantID, id)
}
func (p *Postgres) RevokeAPIKey(tenantID, id string) error {
ctx := context.Background()
tag, err := p.pool.Exec(ctx, `
UPDATE api_key SET revoked_at=now(), updated_at=now()
WHERE id=$1 AND tenant_id=$2 AND revoked_at IS NULL`, id, tenantID)
if err != nil {
return err
}
if tag.RowsAffected() == 0 {
return store.ErrNotFound
}
return nil
}
func (p *Postgres) RotateAPIKey(tenantID, id string) (*store.APIKeyWithSecret, error) {
cur, err := p.GetAPIKey(tenantID, id)
if err != nil {
return nil, err
}
if cur.RevokedAt != nil {
return nil, store.ErrInvalidInput
}
tok, err := authkey.GenerateToken()
if err != nil {
return nil, err
}
hash := authkey.HashToken(tok)
prefix := authkey.Prefix(tok)
ctx := context.Background()
_, err = p.pool.Exec(ctx, `
UPDATE api_key SET token_hash=$3, token_prefix=$4, updated_at=now()
WHERE id=$1 AND tenant_id=$2 AND revoked_at IS NULL`,
id, tenantID, hash, prefix)
if err != nil {
return nil, err
}
k, err := p.GetAPIKey(tenantID, id)
if err != nil {
return nil, err
}
return &store.APIKeyWithSecret{APIKey: *k, Token: tok}, nil
}
func (p *Postgres) ListActiveAPIKeyHashes() ([]store.APIKeyAuthRow, error) {
ctx := context.Background()
rows, err := p.pool.Query(ctx, `
SELECT id::text, tenant_id::text, role, token_hash
FROM api_key
WHERE revoked_at IS NULL AND (expires_at IS NULL OR expires_at > now())`)
if err != nil {
return nil, err
}
defer rows.Close()
var out []store.APIKeyAuthRow
for rows.Next() {
var row store.APIKeyAuthRow
var hash []byte
if err := rows.Scan(&row.ID, &row.TenantID, &row.Role, &hash); err != nil {
return nil, err
}
row.TokenHash = append([]byte(nil), hash...)
out = append(out, row)
}
return out, rows.Err()
}
func (p *Postgres) TouchAPIKeyLastUsed(id string) error {
ctx := context.Background()
_, err := p.pool.Exec(ctx, `UPDATE api_key SET last_used_at=now() WHERE id=$1`, id)
return err
}
type scanFn func(dest ...any) error
func scanAPIKeyRow(scan scanFn, tenantID string) (*store.APIKey, error) {
var k store.APIKey
k.TenantID = tenantID
var expires, revoked, lastUsed *time.Time
if err := scan(&k.ID, &k.Name, &k.Role, &k.Prefix, &k.CreatedAt, &k.UpdatedAt, &expires, &revoked, &lastUsed); err != nil {
return nil, err
}
k.ExpiresAt = expires
k.RevokedAt = revoked
k.LastUsedAt = lastUsed
return &k, nil
}
+32
View File
@@ -325,6 +325,38 @@ func (p *Postgres) UpdateASEntryResolveMeta(tenantID, moduleID, entryID string,
return nil return nil
} }
func (p *Postgres) UpdateASEntryResolveMetaBatch(tenantID, moduleID string, updates []store.ASEntryResolveMetaUpdate, resolvedAt time.Time) error {
if len(updates) == 0 {
return nil
}
if _, err := p.GetModule(tenantID, moduleID); err != nil {
return err
}
ctx := context.Background()
batch := &pgx.Batch{}
for _, u := range updates {
var nameArg any
sn := strings.TrimSpace(u.ASNName)
if sn == "" {
nameArg = nil
} else {
nameArg = sn
}
batch.Queue(`
UPDATE module_as_entry SET asn_name=$3, prefix_count=$4, asn_resolved_at=$5, updated_at=now()
WHERE id=$1 AND module_id=$2`,
u.EntryID, moduleID, nameArg, u.PrefixCount, resolvedAt.UTC())
}
br := p.pool.SendBatch(ctx, batch)
defer func() { _ = br.Close() }()
for range updates {
if _, err := br.Exec(); err != nil {
return err
}
}
return nil
}
func (p *Postgres) DeleteASEntry(tenantID, moduleID, entryID string) error { func (p *Postgres) DeleteASEntry(tenantID, moduleID, entryID string) error {
if _, err := p.GetModule(tenantID, moduleID); err != nil { if _, err := p.GetModule(tenantID, moduleID); err != nil {
return err return err
@@ -0,0 +1,80 @@
package repository
import (
"context"
"evobgp/internal/store"
)
func (p *Postgres) fillModuleDohFields(ctx context.Context, m *store.Module) error {
if m == nil {
return nil
}
return p.batchFillModuleDohFields(ctx, map[string]*store.Module{m.ID: m})
}
func (p *Postgres) batchFillModuleDohFields(ctx context.Context, modules map[string]*store.Module) error {
if len(modules) == 0 {
return nil
}
ids := make([]string, 0, len(modules))
for id := range modules {
ids = append(ids, id)
}
rows, err := p.pool.Query(ctx, `
SELECT module_id::text, doh_profile_id::text
FROM module_doh_profile
WHERE module_id = ANY($1::uuid[])
ORDER BY module_id, sort_order, doh_profile_id`, ids)
if err != nil {
return err
}
defer rows.Close()
byModule := make(map[string][]string, len(modules))
for rows.Next() {
var moduleID, profileID string
if err := rows.Scan(&moduleID, &profileID); err != nil {
return err
}
byModule[moduleID] = append(byModule[moduleID], profileID)
}
if err := rows.Err(); err != nil {
return err
}
for id, m := range modules {
m.DohProfileIDs = store.NormalizeDohProfileIDList(byModule[id])
m.SyncLegacyDohProfileID()
}
return nil
}
func (p *Postgres) setModuleDohProfiles(ctx context.Context, moduleID string, ids []string) error {
ids = store.NormalizeDohProfileIDList(ids)
if _, err := p.pool.Exec(ctx, `DELETE FROM module_doh_profile WHERE module_id = $1`, moduleID); err != nil {
return err
}
for i, id := range ids {
if _, err := p.pool.Exec(ctx, `
INSERT INTO module_doh_profile (module_id, doh_profile_id, sort_order)
VALUES ($1, $2, $3)`, moduleID, id, i); err != nil {
return err
}
}
var dohArg any
if len(ids) > 0 {
dohArg = ids[0]
}
_, err := p.pool.Exec(ctx, `UPDATE module SET doh_profile_id = $2, updated_at = now() WHERE id = $1`, moduleID, dohArg)
return err
}
func (p *Postgres) moduleDohProfileInUse(ctx context.Context, dohProfileID string) (bool, error) {
var n int
if err := p.pool.QueryRow(ctx, `
SELECT COUNT(*) FROM module_doh_profile mdp
JOIN module m ON m.id = mdp.module_id
WHERE mdp.doh_profile_id = $1::uuid AND m.deleted_at IS NULL`, dohProfileID).Scan(&n); err != nil {
return false, err
}
return n > 0, nil
}
+1 -1
View File
@@ -121,7 +121,7 @@ func postTenantRefresh(ctx context.Context, deps *Deps, moduleIDs []string, idem
if err != nil { if err != nil {
return err return err
} }
defer resp.Body.Close() defer func() { _ = resp.Body.Close() }()
if resp.StatusCode == http.StatusNoContent || resp.StatusCode == http.StatusAccepted { if resp.StatusCode == http.StatusNoContent || resp.StatusCode == http.StatusAccepted {
return nil return nil
} }
+8
View File
@@ -0,0 +1,8 @@
package store
// ASEntryResolveMetaUpdate is one row for batch AS resolve metadata writes.
type ASEntryResolveMetaUpdate struct {
EntryID string
ASNName string
PrefixCount int64
}
+77 -1
View File
@@ -1,6 +1,10 @@
package store package store
import "time" import (
"context"
"strings"
"time"
)
// Backend is the persistence abstraction for the control plane (memory, PostgreSQL, SQLite). // Backend is the persistence abstraction for the control plane (memory, PostgreSQL, SQLite).
type Backend interface { type Backend interface {
@@ -13,7 +17,10 @@ type Backend interface {
// ListTenantIDs returns distinct tenant identifiers (for background workers). // ListTenantIDs returns distinct tenant identifiers (for background workers).
ListTenantIDs() ([]string, error) ListTenantIDs() ([]string, error)
// ListModules returns all modules for a tenant (control plane may paginate in httpapi).
ListModules(tenantID string) []*Module ListModules(tenantID string) []*Module
// ListModulesPage returns one page of modules (limit capped by caller).
ListModulesPage(tenantID, cursor string, limit int) ([]*Module, string, bool)
GetModule(tenantID, moduleID string) (*Module, error) GetModule(tenantID, moduleID string) (*Module, error)
CreateModule(tenantID string, in *Module) (*Module, error) CreateModule(tenantID string, in *Module) (*Module, error)
UpdateModule(tenantID, moduleID string, patch *ModulePatch) (*Module, error) UpdateModule(tenantID, moduleID string, patch *ModulePatch) (*Module, error)
@@ -30,6 +37,7 @@ type Backend interface {
DeleteASEntry(tenantID, moduleID, entryID string) error DeleteASEntry(tenantID, moduleID, entryID string) error
// UpdateASEntryResolveMeta записывает имя AS, число объявленных префиксов и время успешного резолва (pipeline). // UpdateASEntryResolveMeta записывает имя AS, число объявленных префиксов и время успешного резолва (pipeline).
UpdateASEntryResolveMeta(tenantID, moduleID, entryID string, asnName string, prefixCount int64, resolvedAt time.Time) error UpdateASEntryResolveMeta(tenantID, moduleID, entryID string, asnName string, prefixCount int64, resolvedAt time.Time) error
UpdateASEntryResolveMetaBatch(tenantID, moduleID string, updates []ASEntryResolveMetaUpdate, resolvedAt time.Time) error
ListDomainEntries(tenantID, moduleID string) ([]*DomainEntry, error) ListDomainEntries(tenantID, moduleID string) ([]*DomainEntry, error)
CreateDomainEntry(tenantID, moduleID string, in *DomainEntry) (*DomainEntry, error) CreateDomainEntry(tenantID, moduleID string, in *DomainEntry) (*DomainEntry, error)
@@ -53,6 +61,7 @@ type Backend interface {
UpdateCommunity(tenantID, id string, patch *CommunityPatch) (*Community, error) UpdateCommunity(tenantID, id string, patch *CommunityPatch) (*Community, error)
DeleteCommunity(tenantID, id string) error DeleteCommunity(tenantID, id string) error
// ListPeers returns all BGP peers for a tenant (control plane may paginate in httpapi).
ListPeers(tenantID string) []*BGPPeer ListPeers(tenantID string) []*BGPPeer
GetPeer(tenantID, id string) (*BGPPeer, error) GetPeer(tenantID, id string) (*BGPPeer, error)
CreatePeer(tenantID string, in *BGPPeer) (*BGPPeer, error) CreatePeer(tenantID string, in *BGPPeer) (*BGPPeer, error)
@@ -80,6 +89,15 @@ type Backend interface {
ListGlobalSettings(tenantID string) (map[string]any, error) ListGlobalSettings(tenantID string) (map[string]any, error)
PatchGlobalSettings(tenantID string, patch map[string]any) error PatchGlobalSettings(tenantID string, patch map[string]any) error
ListAPIKeys(tenantID string) ([]*APIKey, error)
GetAPIKey(tenantID, id string) (*APIKey, error)
CreateAPIKey(tenantID string, in *APIKeyCreate) (*APIKeyWithSecret, error)
UpdateAPIKey(tenantID, id string, patch *APIKeyPatch) (*APIKey, error)
RevokeAPIKey(tenantID, id string) error
RotateAPIKey(tenantID, id string) (*APIKeyWithSecret, error)
ListActiveAPIKeyHashes() ([]APIKeyAuthRow, error)
TouchAPIKeyLastUsed(id string) error
// Module prefix snapshots cache last successful collect per module (pipeline ingest/render). // Module prefix snapshots cache last successful collect per module (pipeline ingest/render).
GetModulePrefixSnapshot(tenantID, moduleID string) (*ModulePrefixSnapshot, bool, error) GetModulePrefixSnapshot(tenantID, moduleID string) (*ModulePrefixSnapshot, bool, error)
SetModulePrefixSnapshot(tenantID, moduleID, inputHash string, prefixes []PrefixRow) error SetModulePrefixSnapshot(tenantID, moduleID, inputHash string, prefixes []PrefixRow) error
@@ -88,6 +106,9 @@ type Backend interface {
// ASNPrefixCache stores RIPEstat announced-prefixes per ASN (global TTL cache). // ASNPrefixCache stores RIPEstat announced-prefixes per ASN (global TTL cache).
GetASNPrefixCache(asn int64) (*ASNPrefixCacheEntry, bool, error) GetASNPrefixCache(asn int64) (*ASNPrefixCacheEntry, bool, error)
SetASNPrefixCache(asn int64, holder string, prefixes []string) error SetASNPrefixCache(asn int64, holder string, prefixes []string) error
// Ping verifies backend connectivity (no-op for in-memory).
Ping(ctx context.Context) error
} }
// ASNPrefixCacheEntry is a cached RIPEstat response for one ASN. // ASNPrefixCacheEntry is a cached RIPEstat response for one ASN.
@@ -114,6 +135,8 @@ type ModulePatch struct {
CronExpr *string `json:"cron_expr,omitempty"` CronExpr *string `json:"cron_expr,omitempty"`
DefaultCommunityID *string `json:"default_community_id,omitempty"` DefaultCommunityID *string `json:"default_community_id,omitempty"`
DohProfileID *string `json:"doh_profile_id,omitempty"` DohProfileID *string `json:"doh_profile_id,omitempty"`
DohProfileIDs *[]string `json:"doh_profile_ids,omitempty"`
DohResolverPolicy *string `json:"doh_resolver_policy,omitempty"`
LastRefreshedAt *time.Time `json:"-"` LastRefreshedAt *time.Time `json:"-"`
} }
@@ -214,6 +237,59 @@ type CommunityPatch struct {
ValueJSON *string `json:"value_json,omitempty"` ValueJSON *string `json:"value_json,omitempty"`
} }
// APIKey is tenant-scoped API key metadata (secret never stored in plaintext).
type APIKey struct {
ID string `json:"id"`
TenantID string `json:"tenant_id,omitempty"`
Name string `json:"name"`
Role string `json:"role"`
Prefix string `json:"prefix"`
CreatedAt time.Time `json:"created_at"`
UpdatedAt time.Time `json:"updated_at"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
RevokedAt *time.Time `json:"revoked_at,omitempty"`
LastUsedAt *time.Time `json:"last_used_at,omitempty"`
}
// APIKeyCreate is input for issuing a new key.
type APIKeyCreate struct {
Name string `json:"name"`
Role string `json:"role"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
}
// APIKeyPatch is a partial update (role change affects auth after resolver reload).
type APIKeyPatch struct {
Name *string `json:"name,omitempty"`
Role *string `json:"role,omitempty"`
ExpiresAt *time.Time `json:"expires_at,omitempty"`
ClearExpiresAt bool `json:"-"`
}
// APIKeyWithSecret is returned only on create/rotate.
type APIKeyWithSecret struct {
APIKey
Token string `json:"token"`
}
// APIKeyAuthRow is used to build the in-process auth index.
type APIKeyAuthRow struct {
ID string
TenantID string
Role string
TokenHash []byte
}
// ValidAPIKeyRole reports whether role is allowed for API keys.
func ValidAPIKeyRole(role string) bool {
switch strings.ToLower(strings.TrimSpace(role)) {
case "viewer", "editor", "operator", "node":
return true
default:
return false
}
}
type PeerPatch struct { type PeerPatch struct {
Neighbor *string `json:"neighbor,omitempty"` Neighbor *string `json:"neighbor,omitempty"`
RemoteASN *int64 `json:"remote_asn,omitempty"` RemoteASN *int64 `json:"remote_asn,omitempty"`
+110
View File
@@ -0,0 +1,110 @@
package store
import "strings"
const (
DohPolicyPrimaryOnly = "primary_only"
DohPolicyFailover = "failover"
DohPolicyUnion = "union"
)
// NormalizeDohResolverPolicy returns a supported resolver policy name.
func NormalizeDohResolverPolicy(policy string) string {
switch strings.TrimSpace(policy) {
case DohPolicyFailover, DohPolicyUnion:
return strings.TrimSpace(policy)
default:
return DohPolicyPrimaryOnly
}
}
// NormalizeDohProfileIDList deduplicates profile ids preserving order.
func NormalizeDohProfileIDList(ids []string) []string {
if len(ids) == 0 {
return nil
}
seen := make(map[string]struct{}, len(ids))
out := make([]string, 0, len(ids))
for _, id := range ids {
id = strings.TrimSpace(id)
if id == "" {
continue
}
if _, ok := seen[id]; ok {
continue
}
seen[id] = struct{}{}
out = append(out, id)
}
if len(out) == 0 {
return nil
}
return out
}
// EffectiveDohProfileIDs returns ordered DoH profile ids for a module.
func (m *Module) EffectiveDohProfileIDs() []string {
if m == nil {
return nil
}
if ids := NormalizeDohProfileIDList(m.DohProfileIDs); len(ids) > 0 {
return ids
}
if m.DohProfileID != nil {
if id := strings.TrimSpace(*m.DohProfileID); id != "" {
return []string{id}
}
}
return nil
}
// SyncLegacyDohProfileID keeps deprecated doh_profile_id aligned with the first profile.
func (m *Module) SyncLegacyDohProfileID() {
if m == nil {
return
}
ids := NormalizeDohProfileIDList(m.DohProfileIDs)
if len(ids) == 0 {
m.DohProfileID = nil
return
}
first := ids[0]
m.DohProfileID = &first
}
// NormalizeModuleDoh fills doh_profile_ids, policy and legacy id from module input.
func NormalizeModuleDoh(m *Module) {
if m == nil {
return
}
ids := NormalizeDohProfileIDList(m.DohProfileIDs)
if len(ids) == 0 && m.DohProfileID != nil {
if id := strings.TrimSpace(*m.DohProfileID); id != "" {
ids = []string{id}
}
}
m.DohProfileIDs = ids
m.DohResolverPolicy = NormalizeDohResolverPolicy(m.DohResolverPolicy)
m.SyncLegacyDohProfileID()
}
// ApplyModuleDohPatch merges DoH-related fields from patch into mod.
func ApplyModuleDohPatch(mod *Module, patch *ModulePatch) {
if mod == nil || patch == nil {
return
}
if patch.DohProfileIDs != nil {
mod.DohProfileIDs = NormalizeDohProfileIDList(*patch.DohProfileIDs)
} else if patch.DohProfileID != nil {
v := strings.TrimSpace(*patch.DohProfileID)
if v == "" {
mod.DohProfileIDs = nil
} else {
mod.DohProfileIDs = []string{v}
}
}
if patch.DohResolverPolicy != nil {
mod.DohResolverPolicy = NormalizeDohResolverPolicy(*patch.DohResolverPolicy)
}
mod.SyncLegacyDohProfileID()
}
+32
View File
@@ -0,0 +1,32 @@
package store
import "testing"
func TestNormalizeDohProfileIDList(t *testing.T) {
got := NormalizeDohProfileIDList([]string{" a ", "b", "a", "", "b"})
if len(got) != 2 || got[0] != "a" || got[1] != "b" {
t.Fatalf("unexpected: %v", got)
}
}
func TestApplyModuleDohPatch(t *testing.T) {
mod := &Module{DohProfileIDs: []string{"one"}, DohResolverPolicy: DohPolicyPrimaryOnly}
ids := []string{"ru", "eu"}
policy := DohPolicyUnion
ApplyModuleDohPatch(mod, &ModulePatch{DohProfileIDs: &ids, DohResolverPolicy: &policy})
if len(mod.DohProfileIDs) != 2 || mod.DohResolverPolicy != DohPolicyUnion {
t.Fatalf("unexpected module doh fields: %+v", mod)
}
if mod.DohProfileID == nil || *mod.DohProfileID != "ru" {
t.Fatalf("legacy id not synced: %+v", mod.DohProfileID)
}
}
func TestEffectiveDohProfileIDs_LegacyField(t *testing.T) {
id := "legacy-id"
mod := &Module{DohProfileID: &id}
got := mod.EffectiveDohProfileIDs()
if len(got) != 1 || got[0] != "legacy-id" {
t.Fatalf("unexpected: %v", got)
}
}
+35 -2
View File
@@ -1,6 +1,7 @@
package store package store
import ( import (
"context"
"errors" "errors"
"fmt" "fmt"
"sort" "sort"
@@ -42,6 +43,7 @@ type Memory struct {
revPrefixes map[string][]PrefixRow revPrefixes map[string][]PrefixRow
moduleSnapshots map[string]*moduleSnapshotRec moduleSnapshots map[string]*moduleSnapshotRec
asnPrefixCache map[int64]*ASNPrefixCacheEntry asnPrefixCache map[int64]*ASNPrefixCacheEntry
apiKeys map[string]*apiKeyRec
// DemoIDs valid after SeedDemo() // DemoIDs valid after SeedDemo()
demoTenantID string demoTenantID string
@@ -56,6 +58,11 @@ type publishedInfo struct {
PublishedAt time.Time PublishedAt time.Time
} }
type apiKeyRec struct {
APIKey
TokenHash []byte
}
type Tenant struct { type Tenant struct {
ID string ID string
Name string Name string
@@ -72,7 +79,9 @@ type Module struct {
CronExpr string // optional cron for scheduler (display / future use) CronExpr string // optional cron for scheduler (display / future use)
Priority int Priority int
DefaultCommunityID *string DefaultCommunityID *string
DohProfileID *string DohProfileID *string // deprecated: first id in DohProfileIDs
DohProfileIDs []string
DohResolverPolicy string
LastRefreshedAt *time.Time LastRefreshedAt *time.Time
DeletedAt *time.Time DeletedAt *time.Time
} }
@@ -130,6 +139,7 @@ func NewMemory() *Memory {
revPrefixes: make(map[string][]PrefixRow), revPrefixes: make(map[string][]PrefixRow),
moduleSnapshots: make(map[string]*moduleSnapshotRec), moduleSnapshots: make(map[string]*moduleSnapshotRec),
asnPrefixCache: make(map[int64]*ASNPrefixCacheEntry), asnPrefixCache: make(map[int64]*ASNPrefixCacheEntry),
apiKeys: make(map[string]*apiKeyRec),
} }
} }
@@ -289,6 +299,12 @@ func (m *Memory) DemoIDs() (tenant, moduleCDN, moduleIP, revision, speaker strin
return m.demoTenantID, m.demoModuleCDN, m.demoModuleIP, m.demoRevisionID, m.demoSpeakerID return m.demoTenantID, m.demoModuleCDN, m.demoModuleIP, m.demoRevisionID, m.demoSpeakerID
} }
// Ping is a no-op for the in-memory backend.
func (m *Memory) Ping(ctx context.Context) error {
_ = ctx
return nil
}
// ListTenantIDs returns tenant ids sorted lexicographically. // ListTenantIDs returns tenant ids sorted lexicographically.
func (m *Memory) ListTenantIDs() ([]string, error) { func (m *Memory) ListTenantIDs() ([]string, error) {
m.mu.RLock() m.mu.RLock()
@@ -325,13 +341,22 @@ func (m *Memory) CreateRenderRevision(revisionID, tenantID, moduleID string, par
frag[k] = v frag[k] = v
} }
parent := parentRevisionID parent := parentRevisionID
createdAt := time.Now().UTC()
for _, r := range m.revisions {
if r == nil || r.TenantID != tenantID {
continue
}
if !r.CreatedAt.Before(createdAt) {
createdAt = r.CreatedAt.Add(time.Microsecond)
}
}
m.revisions[revisionID] = &Revision{ m.revisions[revisionID] = &Revision{
ID: revisionID, ID: revisionID,
TenantID: tenantID, TenantID: tenantID,
ModuleID: moduleID, ModuleID: moduleID,
ContentHash: contentHash, ContentHash: contentHash,
ParentRevisionID: parent, ParentRevisionID: parent,
CreatedAt: time.Now().UTC(), CreatedAt: createdAt,
MaterializedPrefixCount: len(prefixes), MaterializedPrefixCount: len(prefixes),
PreviewFragments: frag, PreviewFragments: frag,
} }
@@ -364,6 +389,11 @@ func (m *Memory) ListModules(tenantID string) []*Module {
return out return out
} }
func (m *Memory) ListModulesPage(tenantID, cursor string, limit int) ([]*Module, string, bool) {
all := m.ListModules(tenantID)
return PaginateOffset(all, cursor, limit)
}
// ListPeers returns BGP peers for a tenant (sorted by name). // ListPeers returns BGP peers for a tenant (sorted by name).
func (m *Memory) ListPeers(tenantID string) []*BGPPeer { func (m *Memory) ListPeers(tenantID string) []*BGPPeer {
m.mu.RLock() m.mu.RLock()
@@ -655,6 +685,9 @@ func (m *Memory) ListRevisions(tenantID, moduleID string, cursor string, limit i
all = append(all, r) all = append(all, r)
} }
sort.Slice(all, func(i, j int) bool { sort.Slice(all, func(i, j int) bool {
if all[i].CreatedAt.Equal(all[j].CreatedAt) {
return all[i].ID > all[j].ID
}
return all[i].CreatedAt.After(all[j].CreatedAt) return all[i].CreatedAt.After(all[j].CreatedAt)
}) })
off := 0 off := 0
+184
View File
@@ -0,0 +1,184 @@
package store
import (
"strings"
"time"
"evobgp/internal/authkey"
"github.com/google/uuid"
)
func (m *Memory) ListAPIKeys(tenantID string) ([]*APIKey, error) {
m.mu.RLock()
defer m.mu.RUnlock()
var out []*APIKey
for _, rec := range m.apiKeys {
if rec.TenantID == tenantID {
out = append(out, apiKeyCopy(&rec.APIKey))
}
}
return out, nil
}
func (m *Memory) GetAPIKey(tenantID, id string) (*APIKey, error) {
m.mu.RLock()
defer m.mu.RUnlock()
rec, ok := m.apiKeys[id]
if !ok || rec.TenantID != tenantID {
return nil, ErrNotFound
}
return apiKeyCopy(&rec.APIKey), nil
}
func (m *Memory) CreateAPIKey(tenantID string, in *APIKeyCreate) (*APIKeyWithSecret, error) {
if in == nil || strings.TrimSpace(in.Name) == "" || !ValidAPIKeyRole(in.Role) {
return nil, ErrInvalidInput
}
tok, err := authkey.GenerateToken()
if err != nil {
return nil, err
}
now := time.Now().UTC()
m.mu.Lock()
defer m.mu.Unlock()
if _, ok := m.tenants[tenantID]; !ok {
return nil, ErrTenantScope
}
id := uuid.NewString()
k := &apiKeyRec{
APIKey: APIKey{
ID: id,
TenantID: tenantID,
Name: strings.TrimSpace(in.Name),
Role: strings.ToLower(strings.TrimSpace(in.Role)),
Prefix: authkey.Prefix(tok),
CreatedAt: now,
UpdatedAt: now,
ExpiresAt: in.ExpiresAt,
},
TokenHash: authkey.HashToken(tok),
}
m.apiKeys[id] = k
return &APIKeyWithSecret{APIKey: *apiKeyCopy(&k.APIKey), Token: tok}, nil
}
func (m *Memory) UpdateAPIKey(tenantID, id string, patch *APIKeyPatch) (*APIKey, error) {
if patch == nil {
return nil, ErrInvalidInput
}
m.mu.Lock()
defer m.mu.Unlock()
rec, ok := m.apiKeys[id]
if !ok || rec.TenantID != tenantID {
return nil, ErrNotFound
}
if rec.RevokedAt != nil {
return nil, ErrInvalidInput
}
if patch.Name != nil {
n := strings.TrimSpace(*patch.Name)
if n == "" {
return nil, ErrInvalidInput
}
rec.Name = n
}
if patch.Role != nil {
if !ValidAPIKeyRole(*patch.Role) {
return nil, ErrInvalidInput
}
rec.Role = strings.ToLower(strings.TrimSpace(*patch.Role))
}
if patch.ClearExpiresAt {
rec.ExpiresAt = nil
} else if patch.ExpiresAt != nil {
rec.ExpiresAt = patch.ExpiresAt
}
rec.UpdatedAt = time.Now().UTC()
return apiKeyCopy(&rec.APIKey), nil
}
func (m *Memory) RevokeAPIKey(tenantID, id string) error {
m.mu.Lock()
defer m.mu.Unlock()
rec, ok := m.apiKeys[id]
if !ok || rec.TenantID != tenantID {
return ErrNotFound
}
now := time.Now().UTC()
rec.RevokedAt = &now
rec.UpdatedAt = now
return nil
}
func (m *Memory) RotateAPIKey(tenantID, id string) (*APIKeyWithSecret, error) {
m.mu.Lock()
defer m.mu.Unlock()
rec, ok := m.apiKeys[id]
if !ok || rec.TenantID != tenantID {
return nil, ErrNotFound
}
if rec.RevokedAt != nil {
return nil, ErrInvalidInput
}
tok, err := authkey.GenerateToken()
if err != nil {
return nil, err
}
now := time.Now().UTC()
rec.TokenHash = authkey.HashToken(tok)
rec.Prefix = authkey.Prefix(tok)
rec.UpdatedAt = now
return &APIKeyWithSecret{APIKey: *apiKeyCopy(&rec.APIKey), Token: tok}, nil
}
func (m *Memory) ListActiveAPIKeyHashes() ([]APIKeyAuthRow, error) {
m.mu.RLock()
defer m.mu.RUnlock()
now := time.Now().UTC()
var out []APIKeyAuthRow
for _, rec := range m.apiKeys {
if rec.RevokedAt != nil {
continue
}
if rec.ExpiresAt != nil && !rec.ExpiresAt.After(now) {
continue
}
out = append(out, APIKeyAuthRow{
ID: rec.ID,
TenantID: rec.TenantID,
Role: rec.Role,
TokenHash: append([]byte(nil), rec.TokenHash...),
})
}
return out, nil
}
func (m *Memory) TouchAPIKeyLastUsed(id string) error {
m.mu.Lock()
defer m.mu.Unlock()
rec, ok := m.apiKeys[id]
if !ok {
return ErrNotFound
}
now := time.Now().UTC()
rec.LastUsedAt = &now
return nil
}
func apiKeyCopy(k *APIKey) *APIKey {
cp := *k
if k.ExpiresAt != nil {
t := *k.ExpiresAt
cp.ExpiresAt = &t
}
if k.RevokedAt != nil {
t := *k.RevokedAt
cp.RevokedAt = &t
}
if k.LastUsedAt != nil {
t := *k.LastUsedAt
cp.LastUsedAt = &t
}
return &cp
}
+19 -10
View File
@@ -31,9 +31,11 @@ func (m *Memory) CreateModule(tenantID string, in *Module) (*Module, error) {
RefreshIntervalSec: in.RefreshIntervalSec, RefreshIntervalSec: in.RefreshIntervalSec,
CronExpr: in.CronExpr, CronExpr: in.CronExpr,
DefaultCommunityID: in.DefaultCommunityID, DefaultCommunityID: in.DefaultCommunityID,
DohProfileID: in.DohProfileID, DohProfileIDs: append([]string(nil), in.DohProfileIDs...),
DohResolverPolicy: in.DohResolverPolicy,
LastRefreshedAt: in.LastRefreshedAt, LastRefreshedAt: in.LastRefreshedAt,
} }
NormalizeModuleDoh(mod)
m.modules[id] = mod m.modules[id] = mod
return mod, nil return mod, nil
} }
@@ -71,14 +73,7 @@ func (m *Memory) UpdateModule(tenantID, moduleID string, patch *ModulePatch) (*M
mod.DefaultCommunityID = &v mod.DefaultCommunityID = &v
} }
} }
if patch.DohProfileID != nil { ApplyModuleDohPatch(mod, patch)
v := strings.TrimSpace(*patch.DohProfileID)
if v == "" {
mod.DohProfileID = nil
} else {
mod.DohProfileID = &v
}
}
if patch.LastRefreshedAt != nil { if patch.LastRefreshedAt != nil {
t := patch.LastRefreshedAt.UTC() t := patch.LastRefreshedAt.UTC()
mod.LastRefreshedAt = &t mod.LastRefreshedAt = &t
@@ -306,6 +301,15 @@ func (m *Memory) UpdateASEntryResolveMeta(tenantID, moduleID, entryID string, as
return nil return nil
} }
func (m *Memory) UpdateASEntryResolveMetaBatch(tenantID, moduleID string, updates []ASEntryResolveMetaUpdate, resolvedAt time.Time) error {
for _, u := range updates {
if err := m.UpdateASEntryResolveMeta(tenantID, moduleID, u.EntryID, u.ASNName, u.PrefixCount, resolvedAt); err != nil {
return err
}
}
return nil
}
func (m *Memory) DeleteASEntry(tenantID, moduleID, entryID string) error { func (m *Memory) DeleteASEntry(tenantID, moduleID, entryID string) error {
m.mu.Lock() m.mu.Lock()
defer m.mu.Unlock() defer m.mu.Unlock()
@@ -555,10 +559,15 @@ func (m *Memory) DeleteDohProfile(tenantID, id string) error {
return ErrNotFound return ErrNotFound
} }
for _, mod := range m.modules { for _, mod := range m.modules {
if mod.DohProfileID != nil && *mod.DohProfileID == id { if mod == nil || mod.DeletedAt != nil {
continue
}
for _, pid := range mod.EffectiveDohProfileIDs() {
if pid == id {
return ErrInvalidInput return ErrInvalidInput
} }
} }
}
delete(m.dohProfiles, id) delete(m.dohProfiles, id)
return nil return nil
} }
+29
View File
@@ -0,0 +1,29 @@
package store
import "strconv"
// PaginateOffset returns a page from a pre-sorted slice using numeric string cursors (same scheme as ListRevisions).
func PaginateOffset[T any](all []T, cursor string, limit int) (page []T, nextCursor string, hasMore bool) {
if limit <= 0 {
limit = 50
}
off := 0
if cursor != "" {
if n, err := strconv.Atoi(cursor); err == nil && n >= 0 {
off = n
}
}
if off > len(all) {
off = len(all)
}
end := off + limit
if end > len(all) {
end = len(all)
}
page = all[off:end]
if end < len(all) {
nextCursor = strconv.Itoa(end)
hasMore = true
}
return page, nextCursor, hasMore
}
+23
View File
@@ -0,0 +1,23 @@
// Package version holds build metadata injected at link time via -ldflags.
package version
// Link-time variables (defaults for local development).
var (
Version = "dev"
GitSHA = "unknown"
BuildTime = ""
)
// Info returns build metadata, substituting safe defaults for empty values.
func Info() (ver, gitSHA, buildTime string) {
ver = Version
if ver == "" {
ver = "dev"
}
gitSHA = GitSHA
if gitSHA == "" {
gitSHA = "unknown"
}
buildTime = BuildTime
return ver, gitSHA, buildTime
}
@@ -0,0 +1,4 @@
DROP TABLE IF EXISTS module_doh_profile;
ALTER TABLE module DROP CONSTRAINT IF EXISTS module_doh_resolver_policy_chk;
ALTER TABLE module DROP COLUMN IF EXISTS doh_resolver_policy;
@@ -0,0 +1,21 @@
-- Module DoH resolver policy and ordered profile list.
ALTER TABLE module
ADD COLUMN doh_resolver_policy TEXT NOT NULL DEFAULT 'primary_only'
CONSTRAINT module_doh_resolver_policy_chk CHECK (
doh_resolver_policy IN ('primary_only', 'failover', 'union')
);
CREATE TABLE module_doh_profile (
module_id UUID NOT NULL REFERENCES module (id) ON DELETE CASCADE,
doh_profile_id UUID NOT NULL REFERENCES doh_profile (id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (module_id, doh_profile_id)
);
CREATE INDEX idx_module_doh_profile_doh ON module_doh_profile (doh_profile_id);
INSERT INTO module_doh_profile (module_id, doh_profile_id, sort_order)
SELECT id, doh_profile_id, 0
FROM module
WHERE doh_profile_id IS NOT NULL
AND deleted_at IS NULL;
@@ -0,0 +1,6 @@
DROP INDEX IF EXISTS idx_rev_mat_prefix_rev_id;
DROP INDEX IF EXISTS idx_config_revision_tenant_module_created;
DROP INDEX IF EXISTS idx_bgp_speaker_published;
DROP INDEX IF EXISTS idx_bgp_speaker_last_applied;
DROP INDEX IF EXISTS idx_module_default_community;
DROP INDEX IF EXISTS idx_module_doh_profile_id;
@@ -0,0 +1,17 @@
CREATE INDEX IF NOT EXISTS idx_module_doh_profile_id
ON module (doh_profile_id) WHERE deleted_at IS NULL AND doh_profile_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_module_default_community
ON module (default_community_id) WHERE deleted_at IS NULL AND default_community_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_bgp_speaker_last_applied
ON bgp_speaker (last_applied_revision_id) WHERE last_applied_revision_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_bgp_speaker_published
ON bgp_speaker (published_revision_id) WHERE published_revision_id IS NOT NULL;
CREATE INDEX IF NOT EXISTS idx_config_revision_tenant_module_created
ON config_revision (tenant_id, module_id, created_at DESC);
CREATE INDEX IF NOT EXISTS idx_rev_mat_prefix_rev_id
ON revision_materialized_prefix (revision_id, id);
@@ -0,0 +1,3 @@
DROP INDEX IF EXISTS idx_api_key_tenant_active;
DROP INDEX IF EXISTS idx_api_key_token_hash;
DROP TABLE IF EXISTS api_key;
+19
View File
@@ -0,0 +1,19 @@
CREATE TABLE api_key (
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
tenant_id UUID NOT NULL REFERENCES tenant (id) ON DELETE CASCADE,
name TEXT NOT NULL,
role TEXT NOT NULL,
token_prefix TEXT NOT NULL,
token_hash BYTEA NOT NULL,
created_at TIMESTAMPTZ NOT NULL DEFAULT now(),
updated_at TIMESTAMPTZ NOT NULL DEFAULT now(),
expires_at TIMESTAMPTZ,
revoked_at TIMESTAMPTZ,
last_used_at TIMESTAMPTZ,
CONSTRAINT api_key_role_chk CHECK (role IN ('viewer', 'editor', 'operator', 'node')),
CONSTRAINT api_key_name_chk CHECK (length(trim(name)) > 0),
CONSTRAINT api_key_token_hash_len_chk CHECK (octet_length(token_hash) = 32)
);
CREATE UNIQUE INDEX idx_api_key_token_hash ON api_key (token_hash);
CREATE INDEX idx_api_key_tenant_active ON api_key (tenant_id) WHERE revoked_at IS NULL;
@@ -0,0 +1,3 @@
DROP TABLE IF EXISTS module_doh_profile;
ALTER TABLE module DROP COLUMN doh_resolver_policy;
@@ -0,0 +1,18 @@
ALTER TABLE module
ADD COLUMN doh_resolver_policy TEXT NOT NULL DEFAULT 'primary_only'
CHECK (doh_resolver_policy IN ('primary_only', 'failover', 'union'));
CREATE TABLE module_doh_profile (
module_id TEXT NOT NULL REFERENCES module (id) ON DELETE CASCADE,
doh_profile_id TEXT NOT NULL REFERENCES doh_profile (id) ON DELETE CASCADE,
sort_order INTEGER NOT NULL DEFAULT 0,
PRIMARY KEY (module_id, doh_profile_id)
);
CREATE INDEX idx_module_doh_profile_doh ON module_doh_profile (doh_profile_id);
INSERT INTO module_doh_profile (module_id, doh_profile_id, sort_order)
SELECT id, doh_profile_id, 0
FROM module
WHERE doh_profile_id IS NOT NULL
AND deleted_at IS NULL;
Loaded 100 of 384 files, more files were not shown because too many files have changed in this diff. Show more