57 Commits

Include renames

Author	SHA1	Message	Date
Denozordec	07b97bddc1	feat: enhance observability with prefix aggregation metrics ... - Added new histograms to track prefix aggregation duration, raw count, and aggregated count during tenant rendering. - Implemented `RecordPrefixAggregation` function to record metrics for aggregation performance. - Updated `RenderTenantRevision` and `RenderTenantRevisionFromPrefixes` functions to log aggregation statistics. - Refactored `smartAggregatePrefixRows` to support both IPv4 and IPv6 aggregation, improving overall prefix handling.	2026-05-19 15:38:33 +07:00
Denozordec	34ecc5c235	feat: enhance DoH profile management and resolver policy in modules ... - Added `DohResolverPolicy` schema to OpenAPI documentation, defining policies for domain resolution. - Updated module handling to support multiple DoH profiles via `doh_profile_ids` and introduced `doh_resolver_policy` in the API. - Refactored related functions to accommodate the new DoH profile structure, ensuring backward compatibility with existing `doh_profile_id`. - Enhanced UI components to allow selection and management of DoH profiles and policies in the web interface. - Updated database interactions to handle new fields and ensure proper data normalization.	2026-05-19 14:57:50 +07:00
Denozordec	a536a2d5eb	test: add CDN prefetch tests for 304 responses and snapshot handling ... - Introduced `TestRefreshModuleIngest_CDN304UsesStoredSnapshot` to verify that the module ingest correctly uses stored snapshots when receiving a 304 Not Modified response. - Added `TestCollectModulePrefixRows_CDN304RetriesWithoutETag` to ensure that the system retries without the If-None-Match header after a 304 response, and correctly collects prefixes on subsequent successful requests. - Updated `cachedCDNPrefixRows` function to improve handling of cached prefixes during CDN source processing. - Modified `RefreshModuleIngest` to utilize prior snapshots when collecting module prefix rows.	2026-05-19 14:26:49 +07:00
Denozordec	7a2b015b12	feat: add tenant refresh functionality to HTTP API and job processing ... Implemented a new endpoint for tenant refresh in the HTTP API, allowing for the refresh of modules associated with a tenant. Enhanced job processing to handle tenant refresh jobs, including logic for managing module IDs and job status updates. Updated the scheduler to enqueue tenant refresh jobs based on module due dates, improving the overall efficiency of module management. Additionally, introduced caching for ASN prefix data to optimize performance during refresh operations.	2026-05-19 10:35:20 +07:00
Denozordec	ee364c8b6d	feat: update collectModulePrefixRows to support prior snapshots and enhance CDN/domain prefix collection ... Modified the collectModulePrefixRows function to accept an optional priorSnapshot parameter, allowing for more efficient data retrieval by skipping unnecessary CDN fetches. Refactored the logic for collecting prefix rows from AS, CDN, and domain sources to utilize dedicated functions, improving code organization and maintainability. Additionally, introduced caching for module prefix snapshots to optimize performance during refresh operations.	2026-05-19 10:22:33 +07:00
Denozordec	1e13ef9e70	feat: add last refreshed timestamp to modules and CDN sources ... Introduced a new field, last_refreshed_at, to track the last successful update time for modules and CDN sources. Updated the database schema, API responses, and internal logic to support this feature. Enhanced the frontend to display the last refreshed timestamp, improving visibility into the data update status for users.	2026-04-09 16:10:00 +07:00
Denozordec	47764345f6	feat: enhance CDN source management with last refreshed timestamp ... Added functionality to track the last refreshed timestamp for CDN sources. Updated the database schema and relevant methods to include the last refreshed timestamp during creation and updates. Implemented logic to skip fetching CDN sources based on their refresh interval, improving efficiency in the module prefix collection process. Enhanced the data retrieval methods to support the new timestamp field, ensuring accurate state management for CDN sources.	2026-04-09 16:04:53 +07:00
Denozordec	8a50ba44b3	feat: add live BGP protocol state tracking to peer listing ... Enhanced the peer listing API to include live BGP session states by integrating a new function to retrieve protocol states from the BIRD socket. Added parsing logic for BGP protocol states and updated the peer JSON response to include session state information. Introduced unit tests for the new parsing functionality to ensure accuracy in state retrieval.	2026-04-09 15:39:01 +07:00
Denozordec	6107c3eb87	refactor: enhance job status checks in worker tests ... Updated the worker test cases to improve the handling of job status checks. Added nil checks for job instances and refined the logic to verify job statuses directly from snapshots, ensuring more robust test coverage and preventing potential nil pointer dereferences.	2026-04-09 15:14:19 +07:00
Denozordec	3b17228ef2	feat: implement peer reconciliation job for BGP peers ... Added a new job type `peer_reconcile` to handle fast reconciliation of BGP peers without module ingestion. Updated API documentation to reflect the new job's functionality and its automatic application to speakers post-reconciliation. Enhanced the HTTP API to enqueue reconciliation jobs during peer creation, updates, and deletions, ensuring efficient state management for BGP configurations.	2026-04-09 15:07:21 +07:00
Denozordec	e700f90c47	refactor: improve PATCH handling for module updates with nullable field support ... Enhanced the handlePatchModule function to read the request body more robustly and handle nullable fields explicitly. This allows clients to differentiate between omitted fields and fields set to null, improving data integrity during module updates. The changes streamline the JSON unmarshalling process and ensure proper handling of nullable values for community and DoH profile selections.	2026-04-08 21:08:09 +07:00
Denozordec	4f50350990	feat: implement revision retention settings and pruning logic ... Added functionality to handle revision retention minutes in global settings, including validation for input values. Introduced a new method to prune old revisions based on the specified retention period, ensuring that only the most recent revisions are kept. Updated the frontend to allow users to configure the revision retention setting, enhancing the management of revision lifecycles.	2026-04-08 14:08:23 +07:00
Denozordec	10e2415cfa	refactor: streamline module refresh process and enhance revision handling ... Updated the worker's job processing to utilize a new `RefreshModuleIngest` function, which simplifies the module refresh logic by separating ingestion from revision rendering. The `finishModuleRefreshSuccess` method now handles the aggregation of revision logs and enqueues deploy actions more efficiently. Additionally, refactored the `RefreshModule` function to maintain backward compatibility while improving clarity and functionality in the pipeline's revision management.	2026-04-08 13:42:53 +07:00
Denozordec	2260ecf74a	feat: add diagnostic log endpoint for revisions and implement download functionality ... Introduced a new endpoint `GET /revisions/{revision_id}/diagnostic-log` to retrieve detailed diagnostic logs for specific revisions. Implemented the corresponding handler to generate and return the log in a structured format. Additionally, updated the frontend to include a download button for the diagnostic log, enhancing user accessibility to revision data.	2026-04-08 13:12:30 +07:00
Denozordec	6d0e214655	feat: add debug logging and optimize database queries in Postgres repository ... Introduced a new debug logging function to capture detailed information during specific operations, controlled by an environment variable. Optimized the MaterializedPrefixStats and ListRevisions methods to reduce memory usage by leveraging SQL aggregation and limiting result sets. Updated the ListRevisionPrefixes method to include pagination support, enhancing performance and efficiency in data retrieval.	2026-04-08 12:51:15 +07:00
Denozordec	f6b94a44d0	refactor: enhance bird metrics polling with context support ... Updated the `startBirdMetricsPoller` function to accept a context parameter, allowing for better control over the polling lifecycle. This change was applied in both `evobgp-all` and `evobgp-api` main files. Additionally, modified the `StartBirdProtocolsPoller` function to handle context cancellation, ensuring graceful shutdown of the polling routine. Introduced a new service in the Docker Compose configuration for logging runtime service outputs, improving observability during deployment.	2026-04-08 12:29:07 +07:00
Denozordec	c811c43bbc	feat: add aggregated router-lists catalog endpoint and update module listing filters ... Introduced a new endpoint `GET /v1/router-lists/catalog` that returns a consolidated view of modules, domain entries, ASNs, IP ranges, and communities. Enhanced the existing module listing functionality to support filtering by type and enabled status. Updated documentation to reflect these changes and added tests for the new endpoint and filtering capabilities.	2026-04-07 23:15:40 +07:00
Denozordec	1e77e1a2b8	feat: auto-refresh non-IP module updates and add full project manual ... Queue module refresh jobs after AS, domain, and CDN source mutations (including CSV imports) to keep revisions current across all source types, and introduce a comprehensive project manual covering runtime modules, flows, and API operations. Made-with: Cursor	2026-04-07 21:09:38 +07:00
Denozordec	53fd527a6d	feat: enhance community label handling and improve UI localization across operations pages. Introduce community label mapping for better readability in job logs and update various components to reflect Russian translations, enhancing user experience and consistency in the interface.	2026-04-07 00:58:15 +07:00
Denozordec	d83efe24ef	refactor: optimize job listing and counting logic for concurrency safety. Separate tenant filtering and status checks to reduce lock contention, improving performance and ensuring accurate job status retrieval.	2026-04-07 00:03:46 +07:00
Denozordec	149fb0a18c	feat: implement module refresh job management with concurrency control. Add CountOtherActiveModuleRefresh method to track active jobs and enhance finishModuleRefreshSuccess to defer deploy_apply when parallel refreshes are detected, improving job processing efficiency.	2026-04-06 23:59:26 +07:00
Denozordec	b541794d84	feat: enhance community attribute handling in BIRD format. Add validation for community route bodies to ensure correct tuple structure and introduce tests for single-element standard arrays, improving error handling and robustness in community attribute processing.	2026-04-06 23:33:54 +07:00
Denozordec	1c5a9c91bb	feat: refactor materialization logic to group static routes by community in the refresh pipeline. Introduce a new structure for managing static routes, enhancing the handling of IPv4 and IPv6 routes while preserving community associations. Update related functions to support the new grouping mechanism.	2026-04-06 20:20:52 +07:00
Denozordec	0428b83eb5	feat: add smart aggregation for IPv4 CIDR prefixes in refresh pipeline. Implement functionality to aggregate prefixes based on community and source attributes, ensuring preservation of BIRD semantics. Enhance test coverage for aggregation logic and module refresh behavior.	2026-04-06 19:53:17 +07:00
Denozordec	bae2d68803	feat: add CSV import and export functionality for module entries. Implement endpoints for exporting and importing module entries in CSV format, supporting types AS_PREFIXES, DOMAINS, and IP_RANGES. Enhance UI with buttons for CSV operations, improving user experience in managing module data.	2026-04-06 18:58:52 +07:00
Denozordec	655d7507e5	feat: update Go module dependencies and enhance DNS resolution in refresh pipeline. Upgrade Go version to 1.24.0, add support for DNS over HTTPS using RFC8484 format, and improve error handling for domain resolution. Update UI components for better log entry presentation in Svelte.	2026-04-06 17:59:52 +07:00
Denozordec	c48cc4e2d4	feat: enhance job processing with detailed logging and UI improvements. Introduce revision log entries in the Worker to capture job metadata, including source and community details. Update the Svelte UI to display job logs with expandable details, improving user experience and job tracking capabilities.	2026-04-06 17:10:29 +07:00
Denozordec	678bd27085	feat: implement domain resolution via DoH in refresh pipeline. Enhance the collectModulePrefixRows function to resolve domain IPs using DNS over HTTPS (DoH) profiles, adding support for both A and AAAA record types. Introduce new helper functions for handling DoH requests and processing responses, improving domain management and IP prefix aggregation.	2026-04-06 16:59:59 +07:00
Denozordec	42d956d109	feat: enhance worker functionality by adding job queuing for deploy_apply. Introduce a Registry field in the Worker struct and implement enqueueDeployAllSpeakers method to streamline job processing after refresh or rollback operations, improving deployment efficiency.	2026-04-06 16:32:23 +07:00
Denozordec	e20c9f3113	feat: implement CDN source preview functionality and enhance data handling. Add a new endpoint for previewing CDN sources, allowing users to fetch and parse CIDR prefixes from specified URLs. Update OpenAPI documentation to include new request and response schemas, and modify internal logic to support JSON parsing with prefix path traversal. Enhance UI components to accommodate new preview features, improving user experience and data management.	2026-04-06 15:08:17 +07:00
Denozordec	a2cbfacd45	feat: update BGP community structure and related documentation. Refactor BGP community handling by renaming fields from 'name' and 'kind' to 'community' and 'title' across the codebase. Update OpenAPI specifications, database interactions, and UI components to reflect these changes, enhancing clarity and consistency in community management.	2026-04-06 14:52:29 +07:00
Denozordec	482eeb122d	refactor: remove strict bind configuration from BGP templates and update related logic. Eliminate 'strict bind on' from both IPv4 and IPv6 templates to simplify BGP peer rendering. Adjust OpenAPI documentation and internal handling of BGP source addresses to reflect these changes, enhancing clarity and compatibility.	2026-04-06 14:03:25 +07:00
Denozordec	dc5e20e9b2	test: add validation for 'strict bind on' in BGP rendering tests. Enhance BGP test cases to ensure 'strict bind on' is included in the rendered output, improving test coverage for BGP template configurations.	2026-04-06 14:00:41 +07:00
Denozordec	07f2ce562e	feat: add strict bind configuration to BGP templates. Enhance BGP template rendering by including 'strict bind on' for both IPv4 and IPv6 templates, improving address binding behavior in Docker and bridge environments.	2026-04-06 13:32:23 +07:00
Denozordec	133ea93e83	feat: update BGP template configuration and OpenAPI documentation. Change import settings from 'all' to 'none' in BGP templates, and add hold and keepalive time parameters. Enhance OpenAPI description for bird_bgp_source_ipv4 to provide additional context for MikroTik configuration.	2026-04-06 12:37:15 +07:00
Denozordec	6266a39aed	feat: refine BGP template handling and documentation updates. Remove local IP requirements in BGP template options, simplifying the configuration to focus on ASN. Update OpenAPI documentation to clarify BIRD parameters and enhance user guidance on JSON settings format in the UI. Adjust tests to align with the new BGP template structure.	2026-04-06 11:51:47 +07:00
Denozordec	62bf987a19	feat: update OpenAPI documentation and internal logic for BIRD settings. Enhance the description of BIRD parameters in openapi.yaml, adding details for optional BGP source addresses. Modify refresh.go to include handling for new BGP source address fields and update peer rendering logic accordingly. Improve user interface documentation in settings page to reflect changes in BIRD configuration options.	2026-04-06 11:35:36 +07:00
Denozordec	431c6a90f1	feat: enhance RefreshModule to aggregate prefixes from all enabled tenant modules. Update documentation to clarify the behavior of ingest and revision generation, ensuring accurate representation of data sources and prefix handling.	2026-04-06 11:03:15 +07:00
Denozordec	be8e7b5742	feat: enhance BGP template and peer configuration handling. Introduce policies_json parameter for local IP and ASN overrides in BGP speakers. Implement BGP template rendering and update peer configuration to utilize templates, improving flexibility in BGP setup. Add tests for new rendering logic and ensure proper integration with existing configurations.	2026-04-06 10:18:51 +07:00
Denozordec	c958d5af0d	feat: implement AS holder name resolution and enhance ASEntry structure. Add ASHolderName function to retrieve organization names from RIPEstat, update ASEntry model to include ASN name, prefix count, and resolution timestamp. Modify database interactions and API responses to support new fields, improving ASN metadata handling.	2026-04-06 01:49:31 +07:00
Denozordec	3e8543177e	feat: implement effective peer enabling logic during peer creation. Update CreatePeer methods in Postgres and Memory stores to utilize EffectivePeerEnabledOnCreate function, ensuring new peers are enabled by default unless specified otherwise. Add tests for effective enabling behavior.	2026-04-06 01:22:09 +07:00
Denozordec	80f3a8d105	feat: enhance peer neighbor handling by introducing NormalizePeerNeighborString function. Update CreatePeer and UpdatePeer methods in Postgres and Memory stores to validate and normalize neighbor input, improving data integrity and error handling.	2026-04-06 01:01:40 +07:00
Denozordec	399622cc70	feat: improve BGP protocol summary handling by introducing isBGPProtocolSummaryRow function. Update SummarizeProtocolsOutput and related tests to ensure evobgp_* static names are not counted as BGP sessions, enhancing accuracy in protocol summaries.	2026-04-06 00:41:59 +07:00
Denozordec	94a4c6acd4	feat: add BIRD status endpoint and integrate birdc checks into job processing. Enhance UI to display BIRD runtime status and job metadata, improving user feedback on BIRD operations.	2026-04-06 00:25:43 +07:00
Denozordec	e1aaa18377	feat: enhance RefreshModule to support legacy ASN resolution and improve prefix handling. Introduce sorting of ASNs and manage announced prefixes with polite pauses for better API interaction. Update buildPreviewFragments to streamline route rendering for BIRD configurations.	2026-04-06 00:01:16 +07:00
Denozordec	693227163a	refactor: improve handling of deployable fragments in ApplyRevision and buildPreviewFragments functions. Introduce isDeployableBirdFragment to filter out UI-only preview keys, and update related logic in the API to ensure proper job queuing and response handling for module refresh operations.	2026-04-05 23:33:56 +07:00
Denozordec	4a860a088e	refactor: enhance JSON struct tags for various data models in backend and memory store. Update fields in ModulePatch, CDNSource, ASEntry, DomainEntry, IPRangeEntry, DohProfile, Community, and BGPPeer to improve API response formatting and consistency.	2026-04-05 23:13:51 +07:00
Denozordec	1db6b5b7b2	refactor: update AS entry handling in API and database to support ASN-only entries. Remove prefix field from AS entry structure and adjust related functions and tests. Enhance OpenAPI specifications and documentation to reflect changes in AS entry representation.	2026-04-05 22:38:17 +07:00
Denozordec	73d0003281	refactor: update ListModules and GetModule methods in Postgres repository to include cron expression handling. Enhance module data retrieval by adding cron field to the scan process and updating the module struct accordingly.	2026-04-05 22:24:54 +07:00
Denozordec	acc0608432	docs: update architecture and API documentation to clarify job processing, in-memory registry usage, and future broker integration plans. Enhance OpenAPI specifications for enrollment responses and refine quickstart instructions for Docker setups.	2026-04-05 18:36:32 +07:00