Skip to content

Product Fleet Operations — workers, grains and sagas

Generated from the architecture model

This page is rendered from model/**/*.yaml by tools/docs_model during mkdocs build. Edit the YAML, not this page; hand-written design lives on the linked service, platform and plane pages.

Workers

Worker Service Trigger Output events Retry Idempotency key Concurrency
Pager FleetIncidentService event FleetIncidentService.IncidentOpened — exponential ×5 incidentId + escalationLevel 4 by ProductId
CustomerNotifier FleetIncidentService event FleetIncidentService.IncidentStateChanged — exponential ×5 incidentId + state + customerTenantId 4 by ProductId
OnCallGapChecker FleetIncidentService schedule 0 * * * * — none ×1 hour 1 by none
ProductHealthProbe FleetInventoryService schedule */2 * * * * ProductInstanceStateChanged fixed ×2 instanceId + window 16 by ProductId
WaveDriver FleetRolloutService event FleetRolloutService.RolloutPlanned RolloutWaveCompleted, RolloutHalted, RolloutCompleted fixed ×3 rolloutId + wave 4 by ProductId
ReScaffoldRequester FleetRolloutService event AgentRegistryService.AssetReleasePublished — exponential ×3 productId + targetAssetReleaseId 2 by TenantId
  • Pager (FleetIncidentService): Pages the on-call responder for Sev1 and Sev2 through the paging connector and escalates after the edition's acknowledgement target.
  • CustomerNotifier (FleetIncidentService): Sends customer notifications for opened, mitigated and resolved incidents through the product owner's chosen connectors, recording receipts.
  • OnCallGapChecker (FleetIncidentService): Alerts when a ManagedPlus product has no on-call coverage in the coming window.
  • ProductHealthProbe (FleetInventoryService): Calls each instance's health endpoint through its product gateway (GW-07) and the HealthChecks aggregator; updates health and staleness.
  • WaveDriver (FleetRolloutService): Requests the promotions of the current wave, waits for their completion, runs the bake timer against the guards, and advances, halts or completes.
  • ReScaffoldRequester (FleetRolloutService): For template upgrades, submits a Factory run that regenerates the product with the new template release (through Run Control) and waits for its release to be ready before planning the rollout.

Grains (actors)

None modelled

No grains are modelled for this item yet.

Sagas

Saga Service Correlation Starts on Steps Ends on
OffboardingSaga FleetIncidentService offboardingId FleetIncidentService.OffboardingRequested 5 OffboardingCompleted
RolloutSaga FleetRolloutService rolloutId FleetRolloutService.RolloutPlanned 4 RolloutCompleted