AIFACTORY-FEAT-174: HealthChecks.Aggregator engine library: published and reusable from a base-template service¶
Generated from the architecture model
This page is rendered from model/**/*.yaml by tools/docs_model during mkdocs build. Edit the YAML, not this page; hand-written design lives on the linked service, platform and plane pages.
A design artefact, not a tracker
This backlog is generated from model/backlog/ and validated against the architecture model. No work item exists in any tracker, and no item carries a date, an estimate or an assignee (ADR-0029).
Epic Ecosystem component extensions · Increment I2 Deliver it · State elaborated · Route backend-developer · Phase before development · Build order 28 of 185 (Templates and libraries)
Before development
Template or library work: every task of this feature changes a template or a shared library the Factory builds on, and it lands before the Factory's own repositories of I2 are developed. The mark travels with every task (tag phase:before-development).
The aggregation engine ConnectSoft.HealthChecks.Aggregator ships as a project inside the aggregator template (B-F06) with a console sample and a pack pipeline; the move to a sibling repository is deferred. The environment and SLO services of the Factory probe environments through this engine (poller, adaptive intervals, hysteresis readiness, incident state machine) rather than running one full aggregator per environment, which needs the engine on the feed with a verified version and its types carrying environment, tenant and project. Component: HealthChecks aggregator (extend), library ConnectSoft.HealthChecks.Aggregator.
| Delivers | — |
| Stubs | — |
| Realizes | components: cmp-healthchecks adrs: ADR-0018, ADR-0028 |
| Requirements | — |
| Depends on | — |
| Sources | model/components.yaml, ConnectSoft.HealthChecksAggregatorTemplate/Docs/Control Plane/b-f06-library.md |
AIFACTORY-US-267: HealthChecks.Aggregator engine: targets, samples, flips and incidents carry environment, tenant and project, and the engine passes them through unchanged¶
enabling · persona asset-maintainer
As an asset maintainer, I want the engine types to carry the environment, tenant and project a target belongs to, so that a Factory service can probe the targets of many environments through one engine without mixing their state.
The B-F06 page of the aggregator template (Docs/Control Plane/b-f06-library.md of the template) specifies the engine package ConnectSoft.HealthChecks.Aggregator: a packable facade in src/ConnectSoft.HealthChecks.Aggregator whose public surface is HealthCheckAggregatorOptions, DownstreamServiceOptions, the probe types, IHealthCheckAggregatorService, IReadinessGateEvaluator, IHealthIncidentSink with the incident and mute records, SloMath and SloRules, ILeaderElection and the registration AddHealthChecksAggregatorEngine. The locked decisions of the template keep its registry single-tenant, so none of these types knows which environment, tenant or project a target belongs to. The environment and SLO services of the Factory probe the workloads of many environments through this engine, and ADR-0015 decision 2 has every Factory contract carry the trusted tenant and project. This PBI adds three optional identifiers, environment, tenant and project, to the target options and to the sample, flip, readiness transition and incident records the engine hands to its seams. The engine never resolves or checks them: it copies the identifiers of a target onto everything it produces for that target and passes them to IHealthIncidentSink and to the history interfaces. The identifiers are plain values named as the envelope fields of ADR-0014 decision 1 name them; the engine references no Factory package and forks no scope type. A host that sets none of them, such as the sample samples/MinimalHost, behaves as before, and no existing public name changes, as the page requires for the first version. Two targets with the same name in different environments are distinct: the key of a target in the cache, in the hysteresis machine and in the incident state is the name together with the three identifiers, so the state of one environment never leaks into another. The options validation refuses a target that sets a project without a tenant and names the target.
Acceptance criteria
| AC1 | Given a target configured with an environment, a tenant and a project, when the engine polls it, flips its state, changes the readiness of its group and opens an incident, then every sample, flip, readiness transition and incident handed to the history interfaces and to IHealthIncidentSink carries the three identifiers of the target unchanged (ADR-0015 decision 2, ADR-0014 decision 1). |
| AC2 | Given two targets with the same name in two environments, when one fails and the other stays healthy, then the cache entry, the hysteresis state and the incident of the failing target are kept under its own key and the readiness of the other environment does not change (ADR-0018, model/components.yaml). |
| AC3 | Given a host that sets none of the three identifiers, when the unit tests and the contract test LibrarySpecContractTests of the library run, then the behaviour and every public name of the engine are unchanged; given a target with a project and no tenant, then the options validation refuses it and names the target (ADR-0015 decision 2, ADR-0028). |
| Traces: services | RuntimeEnvironmentService |
| Traces: adrs | ADR-0014, ADR-0015, ADR-0018, ADR-0028 |
| Sources | model/components.yaml, model/services/runtime-cloud/RuntimeEnvironmentService.yaml, docs/architecture/ecosystem-composition.md, docs/architecture/adr/0015-factory-tenancy-under-canonical-adr-0100.md |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-1214 Environment, tenant and project on the target options and the engine records |
contract | DownstreamServiceOptions and the sample, flip, readiness transition and incident records of the engine gain the optional identifiers environment, tenant and project, named as the envelope fields name them; the options validation refuses a project without a tenant; LibrarySpecContractTests lists the added members and finds no renamed one. |
ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-1215 Identifiers copied through the poll cycle and into the target key |
implementation | The poll cycle copies the identifiers of a target onto every sample, flip, readiness transition and incident, and the key of a target in the cache, the hysteresis machine and the incident state is the name together with the three identifiers; a host without identifiers keeps the key and the behaviour it has. | ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-1216 Two-environment tests of the engine state |
fixture | Unit tests poll two targets with the same name in two environments through scripted probe results, assert the identifiers on every record the sink and the history interfaces receive, and assert that a failure in one environment changes no cache entry, hysteresis state or incident of the other. | ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-1217 Verify the scoped engine records and the unchanged default behaviour |
verification | Run dotnet test -c Release --no-build on the library and its test projects; the two-environment tests, the validation test and LibrarySpecContractTests pass and the sample samples/MinimalHost runs unchanged, which proves AC1 to AC3 of AIFACTORY-US-267 (ADR-0015, ADR-0018). |
ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-US-268: HealthChecks.Aggregator engine: a service generated from the base template polls scoped targets through the package with adaptive intervals and hysteresis and without a host stack of the aggregator¶
enabling · persona asset-maintainer
As an asset maintainer, I want the engine proven inside a service that is not the aggregator host, so that the environment and SLO services reuse the poller instead of running one aggregator per environment.
The B-F06 page divides the host from the library. The ASP.NET controllers, the Blazor /graph page, the MCP tools, the Kubernetes watcher, the Redis lock implementation and the identity middleware stay in the template host. The engine holds the probe executors, the poll cycle given an ILeaderElection, the readiness strategies with the hysteresis machine, the incident model and the SLO math, and its architecture test AggregatorLibraryMustNotDependOnHostStacks keeps the facade free of ASP.NET MVC, Blazor, MCP, Kubernetes and Redis references. The only consumer proven so far is the console sample samples/MinimalHost, which uses a project reference. The Factory reuses the engine in services scaffolded from the base template, where RuntimeEnvironmentService names the component (model/services/runtime-cloud/RuntimeEnvironmentService.yaml). This PBI proves that reuse and closes what stands in its way. A test host is generated at test time from the pinned published base template package with UseFactoryProfile into a scratch folder. The generated solution is not committed, so it cannot go stale against the base template, and it stays outside the installer payload that build/Stage-InstallerPayload.ps1 stages; only the overlay files of the host, its registration, its election, its sink and its scripted targets, live among the tests of the template repository. The host references the engine as a package from a local feed, registers AddHealthChecksAggregatorEngine, supplies its own ILeaderElection and its own IHealthIncidentSink, and polls two targets of two environments, one healthy and one that fails and recovers. The adaptive intervals shorten the interval of the failing target, the hysteresis machine holds the readiness of its environment until the configured number of consecutive results, and the sink receives one incident opened and one resolved with the identifiers of the target. The page records that the engine types still live in the fork-layer projects and reach the package through private project references, which the pack already copies into it (PrivateAssets=all with CopyProjectReferencesToPackage). The copy exists; what is missing is a proof from the side of a consumer. This PBI adds a pack test that restores the package into an empty project from a local feed and fails when a type of the public surface cannot be loaded, and the test host restores the same package, so a package that lacks one of those assemblies cannot ship. The default AlwaysLeaderElection suits a single replica only: two instances of the test host with a shared election prove that only the leader polls, and the page of the library states that a service with more than one replica registers its own election.
Acceptance criteria
| AC1 | Given a test host generated at test time from the pinned published base template package, which references the engine as a package and registers AddHealthChecksAggregatorEngine, when it polls a healthy target and a target that fails and recovers in two environments, then the interval of the failing target shortens, the readiness of its environment changes only after the configured number of consecutive results, and the sink of the host receives one opened and one resolved incident with the identifiers of the target (ADR-0018, model/services/runtime-cloud/RuntimeEnvironmentService.yaml). |
| AC2 | Given the packed engine, when the pack test restores it from a local feed into an empty project and the test host restores it without any project reference into the template, then the restore and the build succeed, every type of the public surface loads from the fork-layer assemblies inside the package, and the architecture test AggregatorLibraryMustNotDependOnHostStacks finds no reference to ASP.NET MVC, Blazor, MCP, Kubernetes, Redis or a Factory package in the engine (ADR-0028, model/components.yaml). |
| AC3 | Given two instances of the test host that share one implementation of ILeaderElection, when both run, then only the leader polls and writes samples, and when the leader stops the other instance takes over within the lease of the election, so that a service with several replicas does not probe a target twice (ADR-0018, docs/architecture/ecosystem-composition.md). |
| Traces: services | RuntimeEnvironmentService |
| Traces: adrs | ADR-0018, ADR-0028 |
| Depends on | AIFACTORY-US-267, feature:component-base-template-factory-profile |
| Sources | model/components.yaml, model/services/runtime-cloud/RuntimeEnvironmentService.yaml, docs/architecture/ecosystem-composition.md, docs/architecture/adr/0028-one-repository-and-deployable-per-service.md |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-1218 Test host generated at test time from the base template that consumes the engine as a package |
fixture | A test fixture generates a host from the pinned published base template package with UseFactoryProfile into a scratch folder at test time; the generated solution is not committed and is outside the installer payload build/Stage-InstallerPayload.ps1 stages; only the overlay files of the host live among the tests of the template repository: the registration of AddHealthChecksAggregatorEngine, its own leader election, its incident sink and two scripted targets of two environments; the host restores the packed engine from a local feed. |
ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-1219 Pack test that loads the public surface of the engine from the restored package |
fixture | The pack of src/ConnectSoft.HealthChecks.Aggregator already copies the fork-layer assemblies of its private project references into the package (PrivateAssets=all with CopyProjectReferencesToPackage); the added pack test restores the package into an empty project from a local feed, asserts that the package declares only published packages as dependencies, and fails when a type of the public surface cannot be loaded. |
ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-631 Scope in the engine types and host-free reuse proven |
verification | Run the test host of AIFACTORY-TASK-1218 and the architecture test of the library: the engine references no ASP.NET host, Blazor graph, MCP, Kubernetes or Redis lock implementation, the test host built on the base template polls two targets through the engine with adaptive intervals and hysteresis, and two instances with a shared election poll once, which proves AC1 to AC3 of AIFACTORY-US-268 (ADR-0018, ADR-0028). | ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-US-269: HealthChecks.Aggregator engine: the package is built, tested and published to the ConnectSoft feed with a version its consumers pin, and the deferred repository decision is recorded¶
evidence · persona asset-maintainer
As an asset maintainer, I want the engine on the feed under a version a service can pin, so that the environment service of this increment restores the engine from the feed and not from the template repository.
This PBI is the publication proof of the feature. The B-F06 page locks the feed and the mechanism: the internal ConnectSoft feed through azure-pipelines-healthchecks-aggregator.yml, which packs and pushes on master, with the version key in Directory.Packages.props; it defers the move of the library to a sibling repository, and the status page of the control plane backlog keeps the feature of B-F06 open until the review of its implementation closes. The need the Factory records for this component is the engine on the feed (model/components.yaml). This PBI closes that review, then has the pipeline build the library, run its unit tests, the architecture test and the contract test, pack it and push it to the ConnectSoft feed, the Azure Artifacts feed the ConnectSoft libraries already publish to. The version is raised from the first version of the page, because the scope identifiers were added to the public records, and a version that is already on the feed is refused. The deferred decision is taken and recorded as a decision record of the template repository: the library stays in the template repository or moves to a sibling repository, and in both cases the package identifier ConnectSoft.HealthChecks.Aggregator and its feed do not change, so a consumer pin survives the decision. A probe proves the package from the side of a consumer: a copy of the sample samples/MinimalHost that references the published package and no project of the template restores it from the feed and prints one aggregated report. The published version is recorded on the B-F06 page, where the consuming services read their pin. This is an evidence PBI (D113): a library feature has no scenario, so the probe names what it proves through its ADRs, the requirement on release gates and its sources.
Acceptance criteria
| AC1 | Given the review of the B-F06 implementation is closed, when azure-pipelines-healthchecks-aggregator.yml runs on master, then the library is built, its unit tests, architecture test and contract test pass, and the package ConnectSoft.HealthChecks.Aggregator is pushed to the ConnectSoft feed with a version higher than the first version and recorded on the B-F06 page (FR-11, ADR-0028, model/components.yaml). |
| AC2 | Given the published package, when a copy of the sample host that references the package and no project of the template restores it from the feed and runs, then it prints one aggregated report for its configured target, which proves that a consumer outside the template repository can use the engine (ADR-0018, docs/architecture/ecosystem-composition.md). |
| AC3 | Given a failing test or a version that is already on the feed, when the pipeline reaches the push, then the push is refused with the reason; and given the decision record of the repository, when it is read, then it states whether the library stays or moves and that the package identifier and the feed do not change (FR-11, ADR-0028). |
| Traces: adrs | ADR-0018, ADR-0028 |
| Traces: requirements | FR-11 |
| Depends on | AIFACTORY-US-267, AIFACTORY-US-268 |
| Sources | model/components.yaml, model/backlog/epics.yaml, docs/architecture/ecosystem-composition.md, docs/architecture/adr/0028-one-repository-and-deployable-per-service.md |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-630 Engine package published with a verified version |
operations | azure-pipelines-healthchecks-aggregator.yml builds, tests and packs ConnectSoft.HealthChecks.Aggregator and pushes it to the ConnectSoft feed on master after the tests pass, refuses a version that is already on the feed, and the version key of Directory.Packages.props holds the raised version. |
ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-1220 Decision record of the library home, the election guidance and the published version on the B-F06 page |
documentation | A decision record of the template repository states whether the library stays in the repository or moves to a sibling repository and that the package identifier and the feed do not change; the B-F06 page names the published version, the scope identifiers of the records and the rule that a service with more than one replica registers its own ILeaderElection; the status page closes the feature of B-F06. |
ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-1221 Feed probe from a copy of the sample host |
fixture | A copy of samples/MinimalHost that references the published package and no project of the template restores it from the ConnectSoft feed, configures one target and prints one aggregated report; the probe runs as the last stage of the library pipeline. |
ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-1222 Verify the published engine by the probe and by a repeated version |
verification | One run of the library pipeline publishes the package and the probe prints its report; a second run with the same version is refused at the push; both runs prove AC1 to AC3 of AIFACTORY-US-269 (FR-11, ADR-0028). | ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-US-122: HealthChecks.Aggregator engine library: published and reusable from a base-template service is delivered and proven by its consumers¶
enabling · persona asset-maintainer
As an asset maintainer, I want the HealthChecks.Aggregator engine to be published and reusable from a base-template service, so that it serves fleet and Factory health in the environment and SLO services without later rework.
This is the integration PBI of the feature. With the scope identifiers on the engine records, the reuse proven from a service generated from the base template and the publication proof of the earlier PBIs, a Factory service probes the targets of its environments by three steps: it pins the published ConnectSoft.HealthChecks.Aggregator package, registers AddHealthChecksAggregatorEngine with its own leader election and incident sink, and configures targets with their environment, tenant and project. The poller, the adaptive intervals, the hysteresis readiness and the incident state machine then run inside the service, and every sample, flip and incident it receives names the environment it belongs to. The service runs no aggregator host: no controller, graph page, MCP tool, Kubernetes watcher or Redis lock of the template is referenced, which the architecture test of the library keeps true. The implementation lives in the tasks of the earlier PBIs; this PBI keeps the migrated verification of the feature. The feature is done when the package is on the ConnectSoft feed with the version the B-F06 page names, the probe host restores it from the feed, and the decision on the home of the library is recorded, which gives the fleet profile of the aggregator template and the environment service of the same increment the scoped engine they depend on.
Acceptance criteria
| AC1 | Given the repository ConnectSoft.HealthChecksAggregatorTemplate with the work of this feature, when the library pipeline runs, then the engine package builds, its tests pass and it publishes to the ConnectSoft feed with a verified version that the consuming repositories pin (ADR-0028, model/components.yaml). |
| AC2 | Given the work of AIFACTORY-TASK-630 (engine package published with a verified version) is done, then azure-pipelines-healthchecks-aggregator.yml packs ConnectSoft.HealthChecks.Aggregator and pushes it to the feed on master, the deferred sibling-repository decision is taken and recorded, and a copy of the MinimalHost sample restores the package from the feed (ADR-0028, docs/architecture/ecosystem-composition.md). |
| AC3 | Given the work of AIFACTORY-TASK-631 (scope in the engine types and host-free reuse proven) is done, then target, sample, flip and incident types carry environment, tenant and project, an architecture test proves the engine references no ASP.NET host, Blazor graph, MCP, Kubernetes or Redis lock implementation, and a test host built on the base template polls two targets through the engine with adaptive intervals and hysteresis (ADR-0018, ADR-0028). |
| Traces: services | RuntimeEnvironmentService |
| Traces: adrs | ADR-0018, ADR-0028 |
| Depends on | AIFACTORY-US-267, AIFACTORY-US-268, AIFACTORY-US-269 |
| Sources | model/components.yaml, docs/architecture/ecosystem-composition.md |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-1223 Consumer page of the engine for a base-template service |
documentation | The B-F06 page gains a consumer section for a service generated from the base template: the package reference, the registration with its own leader election and incident sink, targets with their environment, tenant and project, and what the service receives on a flip and on an incident. | ConnectSoft.HealthChecksAggregatorTemplate |
AIFACTORY-TASK-681 Verify healthchecks-aggregator-engine-library-delivered by running the declared command |
verification | Run dotnet build -c Release and dotnet test -c Release --no-build on the library, its tests and the test host of AIFACTORY-TASK-1218, and the feed probe of AIFACTORY-TASK-1221 against the published version; they pass and prove AC1 to AC3 of AIFACTORY-US-122. |
ConnectSoft.HealthChecksAggregatorTemplate |