SloService¶

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.


Bounded context	BC-21 Operations
Platform / plane	Observability / Trust & Operations
Deployment granularity	module of host · host `ObservabilityHost`
Status	target · `ConnectSoft.Factory.Observability.Slo`
Built from	connectsoft-microservice · libraries: connectsoft-extensions-diagnostics-health-checks, connectsoft-extensions-saas-abstractions, connectsoft-extensions-messaging-model, connectsoft-extensions-persistence-model-nhibernate
Decisions	ADR-0018
Traceability	`FR-10` `FUX-04` `NFR-06` `NFR-07` `NFR-17` `NFR-18` `XL-11` `XL-16`

Purpose and responsibilities¶

SloService turns the NFR baseline (D30: interactive acknowledgement under 2 s, control-plane availability 99.5 % monthly, RPO 15 min, RTO 4 h) and the operations metrics into objectives with windows, targets, burn thresholds and error budgets — for the platform, a tenant, a project, a workflow, an agent or a generated product — and evaluates them from the overview digests and the metric backend. It also keeps the health rollup of every host from the HealthChecks aggregator and the hosts' own probes, which the Studio status page and the fleet read. Breaches and health changes are events that AlertRuleService turns into alerts.

It absorbs RuntimeHealthService. Product objectives for generated products are defined here but operated by the Product Fleet Operations platform (P12), which consumes the same breach events.

Context and boundaries¶

flowchart LR
    S["SloService"]
    AlertRuleService["AlertRuleService"] -->|event| S
    StudioBff["StudioBff"] -->|sync-http| S
    S -->|event| DashboardService["DashboardService"]
    S -->|sync-http| ext_otel_backend["ext-otel-backend"]
    S -. "SloDefined" .-> E_SloDefined(("event"))
    S -. "SloBreached" .-> E_SloBreached(("event"))
    S -. "SloRecovered" .-> E_SloRecovered(("event"))
    S -. "ServiceHealthChanged" .-> E_ServiceHealthChanged(("event"))

Hold "Alt" / "Option" to enable pan & zoom

Objectives are declarative and tightening-only: a tenant may set a stricter availability target for its project than the platform's 99.5 %, never a looser one. A measurement is computed, never entered; a host without a fresh probe is Unknown, never assumed healthy.

Domain model¶

SloDefinition — One objective for one subject (platform, tenant, workflow, agent, generated product) with target, window and error-budget policy. (catalog entry)

Field	Type	Required	Description
`sloId`	`guid`	yes
`tenantId`	`string`	yes
`subjectKind`	`string`	yes	platform
`subjectId`	`string`	yes
`kind`	`enum:ObjectiveKind`	yes
`target`	`decimal`	yes	For example 0.995 availability, 2 s p95, 0.9 success.
`window`	`string`	yes	30d rolling or calendar month
`burnAlertThresholds`	`json`	yes	Fast and slow burn thresholds.
`source`	`string`	yes	metric name or projection
`active`	`bool`	yes

Invariants:

Platform objectives are the NFR baseline (D30) and cannot be loosened per tenant; tenant and product objectives may be stricter.
A measurement is computed from projections or metrics, never entered; error budget consumption is monotonic within the window.

Events: SloDefined, SloBreached, SloRecovered · Tables: SloDefinitions, SloMeasurements · Repository: SloDefinitionRepository

ServiceHealth — The health rollup of one host (and its modules) from liveness, readiness and dependency probes collected by the HealthChecks aggregator. (catalog entry)

Field	Type	Required	Description
`hostId`	`string`	yes
`tenantId`	`string`	yes	Platform tenant for shared hosts.
`state`	`enum:HealthState`	yes
`dependencies`	`json`	yes	Per-dependency state.
`replicasReady`	`int`	yes
`observedAt`	`datetime`	yes

Invariants:

Health is observed, never declared; a host without a fresh probe is Unknown.

Events: ServiceHealthChanged · Tables: ServiceHealth · Repository: ServiceHealthRepository

erDiagram
    SloDefinition {
        guid sloId PK
        string tenantId
        string subjectKind
        string subjectId
        enum_ObjectiveKind kind
        decimal target
        string window
        json burnAlertThresholds
        string source
        bool active
    }
    ServiceHealth {
        string hostId PK
        string tenantId
        enum_HealthState state
        json dependencies
        int replicasReady
        datetime observedAt
    }
    SloMeasurement {
        datetime measuredAt

        decimal value

        decimal errorBudgetRemaining

        decimal burnRate

        enum_SloState state

    }
    SloDefinition ||--o{ SloMeasurement : contains

Hold "Alt" / "Option" to enable pan & zoom

Entities and value objects¶

SloMeasurement (entity, in SloDefinition) — One evaluation of the objective.

Field	Type	Required
`measuredAt`	`datetime`	yes
`value`	`decimal`	yes
`errorBudgetRemaining`	`decimal`	yes
`burnRate`	`decimal`	yes
`state`	`enum:SloState`	yes

Data model and storage¶

Store	Role	Engine / profile	Tables
Operations projection store	primary	azure-sql / nhibernate-sqlserver	SloDefinitions, SloMeasurements, ServiceHealth

Table	Owner	Columns	Indexes
`dbo.SloDefinitions`	SloDefinition	10	1
`dbo.SloMeasurements`	SloMeasurement	7	0
`dbo.ServiceHealth`	ServiceHealth	6	0

Full DDL: appendix.

APIs¶

Method	Path	Operation	Scope	Side effects	Idempotency	Gateway	Summary
`GET`	`/factory/slos`	ListSlos	`factory.operations.read`	read	natural	GW-03	Objectives in scope with current state, value and error budget.
`PUT`	`/factory/slos/{subjectKind}/{subjectId}/{kind}`	SetSlo	`factory.operations.admin`	shared-write	natural	GW-03	Define or tighten an objective for a subject (tenant objectives stricter than platform only).
`GET`	`/factory/health/services`	ListServiceHealth	`factory.operations.read`	read	natural	GW-03	Health rollup of every host with dependency states (Studio status page).

None modelled

No gRPC methods are modelled for this item yet.

Messages¶

Event	Topic	Scope / v	Aggregate	Consumers	Summary
SloDefined	`operations.slos.v1.slo-defined`	integration / 1	SloDefinition	StudioBff, AuditService	An objective was defined or changed.
SloBreached	`operations.slos.v1.slo-breached`	integration / 1	SloDefinition	FleetRolloutService, FleetInventoryService, AlertRuleService, StudioBff, DashboardService	An objective is breached or burning fast.
SloRecovered	`operations.slos.v1.slo-recovered`	integration / 1	SloDefinition	FleetInventoryService, AlertRuleService, StudioBff	An objective is meeting again.
ServiceHealthChanged	`operations.health.v1.service-health-changed`	integration / 1	ServiceHealth	AlertRuleService, StudioBff, DashboardService	A host's health rollup changed.

SloDefined payload:

Field	Type	Required
`sloId`	`guid`	yes
`tenantId`	`string`	yes
`subjectKind`	`string`	yes
`subjectId`	`string`	yes
`kind`	`enum:ObjectiveKind`	yes
`target`	`decimal`	yes

SloBreached payload:

Field	Type	Required
`sloId`	`guid`	yes
`tenantId`	`string`	yes
`subjectKind`	`string`	yes
`subjectId`	`string`	yes
`kind`	`enum:ObjectiveKind`	yes
`value`	`decimal`	yes
`state`	`enum:SloState`	yes

SloRecovered payload:

Field	Type	Required
`sloId`	`guid`	yes
`tenantId`	`string`	yes
`kind`	`enum:ObjectiveKind`	yes

ServiceHealthChanged payload:

Field	Type	Required
`hostId`	`string`	yes
`tenantId`	`string`	yes
`fromState`	`enum:HealthState`	yes
`toState`	`enum:HealthState`	yes

Command	Handled by	Result events	Summary
`SetSlo`	SloDefinition	SloDefined

Event	Producer	Handler	Purpose
OperationsDigestProduced	DashboardService	`EvaluateSlos`	Compute objective values from the overview digests.

Workers and jobs¶

Worker	Trigger	Output events	Retry	Idempotency key	Summary
SloEvaluator	schedule `/5 * * *`	SloBreached, SloRecovered	fixed	`sloId + window`	Evaluates every active objective from projections and metrics, computes burn rates and emits breaches and recoveries.
HealthCollector	schedule `/1 * * *`	ServiceHealthChanged	fixed	`hostId + minute`	Collects host health from the HealthChecks aggregator and the hosts' probes and updates the rollup.

Actors and grains¶

None modelled

No grains are modelled for this item yet.

Sagas and long-running flows¶

None modelled

No sagas are modelled for this item yet.

stateDiagram-v2
    [*] --> Meeting
    Meeting --> BurningFast: burn rate above fast threshold
    BurningFast --> Breached: error budget exhausted
    Meeting --> Breached: value below target at window end
    Breached --> Recovered: value back above target
    BurningFast --> Meeting: burn rate normal
    Recovered --> Meeting: next window
    Meeting --> Suspended: source unavailable
    Suspended --> Meeting: source back

Hold "Alt" / "Option" to enable pan & zoom

Options and configuration¶

Section	Key	Type	Default	Reloadable	Secret	Summary
`Observability:Slo`	`PlatformBaseline`	json	`{ availability: 0.995, interactiveLatencyP95Seconds: 2, rpoMinutes: 15, rtoMinutes: 240 }`	no	no	NFR baseline objectives (D30) that tenants may only tighten.
`Observability:Slo`	`HealthStalenessSeconds`	int	`180`	yes	no	A host without a fresh probe is Unknown.

Metrics and observability¶

Metric	Instrument	Unit	Dimensions	Summary
`connectsoft.factory.slo.state`	gauge	1	tenant, subjectKind, kind, state	Objectives by state.
`connectsoft.factory.slo.error_budget_remaining`	gauge	1	tenant, subjectKind, kind	Remaining error budget.
`connectsoft.factory.health.hosts`	updown	{host}	state	Hosts by health state.

Security¶


Authentication	client-credentials
Scopes	`factory.operations.read` (Read objectives and health.); `factory.operations.admin` (Define objectives (Operator, Platform admin).)
Roles	Operator, WorkspaceAdmin, PlatformAdmin, Observer
Tenant isolation	row-filter
Data classification	internal
Max side-effect class	shared-write
Secrets	—

Dependencies¶

Outbound

Target	Kind	Purpose	Failure mode	Required
DashboardService	event	Overview digests.	queue	yes
ext-otel-backend	sync-http	Metric queries for objectives sourced from metrics.	degrade	yes

Inbound

Target	Kind	Purpose	Failure mode	Required
AlertRuleService	event	Breaches and health changes.	—	yes
StudioBff	sync-http	SLO and status views.	—	yes

Failure and recovery¶

Scenario	Detection	Recovery	Data impact	RTO (min)
Metric backend unavailable	query errors	objectives sourced from metrics go Suspended with a note; projection-sourced ones continue	none	60

NFR targets¶

Target	Class value (pilot-projection)	Override
latency_p95_ms	500	—
availability_pct	99.5	—
throughput_rps	100	—
rpo_minutes	15	—
rto_minutes	240	—
cost_class	low	—

Deployment¶


Granularity	module of host · host `ObservabilityHost`
Topologies	saas-default, hybrid, on-prem, hybrid-compute
Runtime	—; replicas —–—
Ports	—

Current implementation¶

Every template service exposes liveness and readiness through ConnectSoft.Extensions.Diagnostics.HealthChecks, and the HealthChecks aggregator UI exists. No objective, measurement or error budget exists anywhere in the estate.

Later capabilities¶

SLO-driven routing hints to the model router (prefer a deployment whose latency objective is meeting); objective templates per edition (D34); customer-visible status pages per generated product.

Decisions and traceability¶

Family	Ids
FR	FR-10
FUX	FUX-04
NFR	NFR-06, NFR-07, NFR-17, NFR-18
XL	XL-11, XL-16

Observability overview · NFR baseline · AlertRuleService · Product fleet operations
Observability microservices · bounded contexts · APIs · events · data model
Services catalog · Ownership map

SloService¶

Purpose and responsibilities¶

Context and boundaries¶

Domain model¶

Entities and value objects¶

Data model and storage¶

APIs¶

Messages¶

Workers and jobs¶

Actors and grains¶

Sagas and long-running flows¶

Options and configuration¶

Metrics and observability¶

Security¶

Dependencies¶

Failure and recovery¶

NFR targets¶

Deployment¶

Current implementation¶

Later capabilities¶

Decisions and traceability¶

Related¶