RuntimeEnvironmentService

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.

Bounded context	BC-25 Provisioning & Reconciliation
Platform / plane	Runtime Cloud / Execution
Deployment granularity	module of host · host RuntimeCloudHost
Status	target · ConnectSoft.Factory.RuntimeCloud.Inventory
Built from	connectsoft-microservice · libraries: connectsoft-extensions-saas-abstractions, connectsoft-extensions-messaging-model, connectsoft-extensions-persistence-model-nhibernate, connectsoft-extensions-diagnostics-health-checks
Decisions	ADR-0004
Traceability	FR-09 FUX-05 NFR-06 NFR-09 XL-10 XL-14

Purpose and responsibilities

RuntimeEnvironmentService answers "what is actually running?" without trusting anyone's declaration. The Observer reads clusters and cloud APIs every five minutes and keeps a runtime inventory per environment (the runtime service catalog of the June-2026 design); an environment not observed within the staleness window is marked stale, never assumed. The DriftComparer compares each observation with the declared specification from IaCProvisioningService and raises drift findings (version, replica, configuration, missing, unmanaged, secret-binding mismatches), grouping them into reconciliation proposals with a direction (apply the declared state, or adopt what is observed) and a plan summary; non-destructive proposals outside production may be auto-accepted where policy allows, everything else waits for an Operator, and acceptance plans a provisioning run.

It absorbs RuntimeDriftDetectionService and ServiceCatalogRuntimeService.

Context and boundaries

flowchart LR
    S["RuntimeEnvironmentService"]
    EnvironmentPromotionService["EnvironmentPromotionService"] -->|grpc| S
    StudioBff["StudioBff"] -->|sync-http| S
    S -->|sync-http| ext_pulumi["ext-pulumi"]
    S -->|sync-http| ext_config_platform["ext-config-platform"]
    S -->|grpc| IaCProvisioningService["IaCProvisioningService"]
    S -->|grpc| PolicyEngineService["PolicyEngineService"]
    S -. "ObservedStateChanged" .-> E_ObservedStateChanged(("event"))
    S -. "RuntimeDriftDetected" .-> E_RuntimeDriftDetected(("event"))
    S -. "ReconciliationProposed" .-> E_ReconciliationProposed(("event"))
    S -. "ReconciliationDecided" .-> E_ReconciliationDecided(("event"))

Hold "Alt" / "Option" to enable pan & zoom

Secret bindings are references; values never enter the inventory. A finding is resolved only by a re-observation that no longer shows it. Promotion verification and the fleet inventory (P12) read the observed state through GetObservedState.

Domain model

RuntimeInventory — The observed state of one environment: every workload with its kind, image or package version, replicas, endpoints, configuration and secret bindings (by reference), health, and the observation time; the runtime service catalog of the June-2026 design. (catalog entry)

Field	Type	Required	Description
environmentId	string	yes
tenantId	string	yes
projectId	string	yes
workloads	json	yes
currentReleaseId	guid	no
specificationVersion	int	no	Declared version last applied.
observedAt	datetime	yes
stale	bool	yes

Invariants:

• Observed state is read from the cluster and cloud APIs, never declared; an environment not observed within the staleness window is marked stale.
• Secret bindings are references; values never enter the inventory.

Events: ObservedStateChanged · Tables: RuntimeInventories, ObservedWorkloads · Repository: RuntimeInventoryRepository

DriftFinding — One difference between the declared specification and the observed state, with kind, resource, declared and observed values and the reconciliation proposal it belongs to. (catalog entry)

Field	Type	Required	Description
findingId	guid	yes
tenantId	string	yes
environmentId	string	yes
kind	enum:DriftKind	yes
resource	string	yes
declared	json	no
observed	json	no
detectedAt	datetime	yes
proposalId	guid	no
resolvedAt	datetime	no

Invariants:

• A finding is resolved only by a re-observation that no longer shows it; never by hand.

Events: RuntimeDriftDetected · Tables: DriftFindings · Repository: DriftFindingRepository

ReconciliationProposal — A reviewable change that would bring observed state back to declared state (or declare what is observed): the findings it covers, the specification version it would apply, the plan summary, and the decision by an Operator; accepted proposals become provisioning runs. (catalog entry)

Field	Type	Required	Description
proposalId	guid	yes
tenantId	string	yes
environmentId	string	yes
findingIds	list:guid	yes
direction	string	yes	apply-declared
planSummary	json	yes
destructive	bool	yes
state	enum:ProposalState	yes
decidedBy	string	no
provisioningRunId	guid	no
raisedAt	datetime	yes

Invariants:

• Destructive or production proposals are always reviewed by an Operator; non-destructive ones outside production may be auto-accepted where policy allows.
• A proposal never writes to infrastructure itself; acceptance plans a provisioning run.

stateDiagram-v2
    Proposed --> Accepted : OperatorAccepts
    Proposed --> Rejected : OperatorRejects
    Accepted --> Applied : ProvisioningApplied
    Proposed --> Superseded : NewerProposal

Hold "Alt" / "Option" to enable pan & zoom

Events: ReconciliationProposed, ReconciliationDecided · Tables: ReconciliationProposals · Repository: ReconciliationProposalRepository

erDiagram
    RuntimeInventory {
        string environmentId PK
        string tenantId
        string projectId
        json workloads
        guid currentReleaseId
        int specificationVersion
        datetime observedAt
        bool stale
    }
    DriftFinding {
        guid findingId PK
        string tenantId
        string environmentId
        enum_DriftKind kind
        string resource
        json declared
        json observed
        datetime detectedAt
        guid proposalId
        datetime resolvedAt
    }
    ReconciliationProposal {
        guid proposalId PK
        string tenantId
        string environmentId
        list_guid findingIds
        string direction
        json planSummary
        bool destructive
        enum_ProposalState state
        string decidedBy
        guid provisioningRunId
        datetime raisedAt
    }
    ObservedWorkload {
        string name

        enum_WorkloadKind kind

        string version

        int replicasReady

        int replicasDesired

        list_string endpoints

        json configRefs

        json secretRefs

        string health

    }
    RuntimeInventory ||--o{ ObservedWorkload : contains

Hold "Alt" / "Option" to enable pan & zoom

Entities and value objects

ObservedWorkload (entity, in RuntimeInventory) — One observed workload.

Field	Type	Required	Description
name	string	yes
kind	enum:WorkloadKind	yes
version	string	yes
replicasReady	int	yes
replicasDesired	int	yes
endpoints	list:string	yes
configRefs	json	yes
secretRefs	json	yes
health	string	yes

Data model and storage

Store	Role	Engine / profile	Tables
Runtime Cloud store	primary	azure-sql / nhibernate-sqlserver	RuntimeInventories, ObservedWorkloads, DriftFindings, ReconciliationProposals

Table	Owner	Columns	Indexes
dbo.RuntimeInventories	RuntimeInventory	7	0
dbo.ObservedWorkloads	ObservedWorkload	11	0
dbo.DriftFindings	DriftFinding	10	1
dbo.ReconciliationProposals	ReconciliationProposal	11	1

Full DDL: appendix.

APIs

Method	Path	Operation	Scope	Side effects	Idempotency	Gateway	Summary
GET	/factory/environments/{environmentId}/inventory	GetRuntimeInventory	factory.runtime.read	read	natural	GW-03	Observed workloads, versions, replicas, endpoints and bindings by reference, with staleness.
GET	/factory/environments/{environmentId}/drift	ListDrift	factory.runtime.read	read	natural	GW-03	Open and resolved drift findings for an environment.
POST	/factory/environments/{environmentId}/observe	ObserveEnvironment	factory.runtime.operate	read	natural	GW-03	Trigger an observation now (Operator).
GET	/factory/reconciliation/proposals	ListReconciliationProposals	factory.runtime.read	read	natural	GW-03	Proposals by environment and state with plan summaries.
POST	/factory/reconciliation/proposals/{proposalId}/decide	DecideReconciliation	factory.runtime.operate	production	natural	GW-03	Accept (plans a provisioning run) or reject with a reason (Operator; production acceptance is production class).

gRPC service	Method	Streaming	Visibility	Side effects	Summary
Inventory	GetObservedState	none	internal	read	Observed state of an environment for promotion verification and fleet inventory.

Messages

Event	Topic	Scope / v	Aggregate	Consumers	Summary
ObservedStateChanged	provisioning.inventory.v1.observed-state-changed	integration / 1	RuntimeInventory	FleetInventoryService, EnvironmentService, EnvironmentPromotionService, SloService, StudioBff, DashboardService	An environment's observed state changed (versions, replicas, health, staleness).
RuntimeDriftDetected	provisioning.inventory.v1.runtime-drift-detected	integration / 1	DriftFinding	AlertRuleService, StudioBff, AuditService, DashboardService	Observed state differs from the declared specification.
ReconciliationProposed	provisioning.inventory.v1.reconciliation-proposed	integration / 1	ReconciliationProposal	StudioBff, StudioNotificationService, AuditService	A reviewable reconciliation awaits an Operator.
ReconciliationDecided	provisioning.inventory.v1.reconciliation-decided	integration / 1	ReconciliationProposal	IaCProvisioningService, StudioBff, AuditService	A proposal was accepted (provisioning run planned) or rejected.

ObservedStateChanged payload:

Field	Type	Required	Description
environmentId	string	yes
tenantId	string	yes
projectId	string	yes
currentReleaseId	guid	no
changes	json	yes
stale	bool	yes

RuntimeDriftDetected payload:

Field	Type	Required	Description
findingId	guid	yes
tenantId	string	yes
environmentId	string	yes
kind	enum:DriftKind	yes
resource	string	yes

ReconciliationProposed payload:

Field	Type	Required	Description
proposalId	guid	yes
tenantId	string	yes
environmentId	string	yes
direction	string	yes
destructive	bool	yes

ReconciliationDecided payload:

Field	Type	Required	Description
proposalId	guid	yes
tenantId	string	yes
environmentId	string	yes
state	enum:ProposalState	yes
provisioningRunId	guid	no

Command	Handled by	Result events	Summary
ObserveEnvironment	RuntimeInventory	ObservedStateChanged
DecideReconciliation	ReconciliationProposal	ReconciliationDecided

Event	Producer	Handler	Purpose
SpecificationDeclared	IaCProvisioningService	TrackDeclared	Know the declared version to compare against.
ProvisioningApplied	IaCProvisioningService	ObserveAfterApply	Re-observe right after an apply and resolve findings.
ProvisioningFailed	IaCProvisioningService	ObserveAfterFailure	Capture partial state after a failed apply.

Workers and jobs

Worker	Trigger	Output events	Retry	Idempotency key	Summary
Observer	schedule */5 * * * *	ObservedStateChanged	fixed	environmentId + observation window	Reads clusters and cloud APIs for every environment, updates the inventory, marks stale environments, and resolves findings no longer observed.
DriftComparer	event	RuntimeDriftDetected, ReconciliationProposed	fixed	environmentId + observedAt	Compares each observation with the declared specification, raises findings and groups them into proposals (auto-accepting non-destructive ones outside production where policy allows).

Actors and grains

None modelled

No grains are modelled for this item yet.

Sagas and long-running flows

None modelled

No sagas are modelled for this item yet.

flowchart LR
    OBS[Observer every 5 min · clusters and cloud APIs] --> INV[RuntimeInventory per environment]
    INV --> CMP[DriftComparer vs declared specification]
    CMP -- differences --> F[DriftFindings]
    F --> PR[ReconciliationProposal · direction · destructive?]
    PR -- non-destructive, not production, policy allows --> AUTO[auto-accepted]
    PR -- otherwise --> OP[Operator decides]
    AUTO & OP -- accepted --> PROV[IaCProvisioningService plans a run]
    PROV -- ProvisioningApplied --> OBS

Hold "Alt" / "Option" to enable pan & zoom

Options and configuration

Section	Key	Type	Default	Reloadable	Secret	Summary
RuntimeCloud:Inventory	StalenessMinutes	int	15	yes	no	An environment without a fresh observation is stale.
RuntimeCloud:Inventory	AutoAcceptNonDestructiveOutsideProduction	bool	true	yes	no	Auto-accept non-destructive proposals outside production.

Metrics and observability

Metric	Instrument	Unit	Dimensions	Summary
connectsoft.factory.runtime.environments_stale	updown	{environment}	tenant	Environments without a fresh observation.
connectsoft.factory.runtime.drift_open	updown	{finding}	tenant, kind	Open drift findings.
connectsoft.factory.runtime.reconciliation_proposals	updown	{proposal}	tenant, state, destructive	Proposals by state.

Security

Authentication	client-credentials
Scopes	factory.runtime.read (Read inventory, drift and proposals.); factory.runtime.operate (Observe now; decide proposals (Operator).)
Roles	Operator, PlatformAdmin, WorkspaceAdmin, Observer
Tenant isolation	row-filter
Data classification	confidential
Max side-effect class	production
Secrets	—

Dependencies

Outbound

Target	Kind	Purpose	Failure mode	Required
ext-pulumi	sync-http	Cluster and resource reads.	degrade	yes
ext-config-platform	sync-http	Configuration references for comparison.	degrade	no
IaCProvisioningService	grpc	Declared specification; plan accepted proposals.	fail-fast	yes
PolicyEngineService	grpc	Auto-accept decisions.	fail-fast	yes

Inbound

Target	Kind	Purpose	Failure mode	Required
EnvironmentPromotionService	grpc	Observed state for verification.	—	yes
StudioBff	sync-http	Inventory, drift and proposal views.	—	yes

Failure and recovery

Scenario	Detection	Recovery	Data impact	RTO (min)
Cluster API unreachable	observation errors	environment marked stale; alerts; no findings raised from a failed observation	none	30

NFR targets

Target	Class value (pilot-projection)	Override
latency_p95_ms	500	—
availability_pct	99.5	—
throughput_rps	100	—
rpo_minutes	15	—
rto_minutes	240	—
cost_class	low	—

Deployment

Granularity	module of host · host RuntimeCloudHost
Topologies	saas-default, hybrid, on-prem, hybrid-compute
Runtime	—; replicas —–—
Ports	—

Current implementation

The HealthChecks aggregator shows per-service health; kubectl and the Azure portal are the inventory. No observed-state store, drift comparison or proposal exists.

Later capabilities

Unmanaged-resource adoption flows that generate the IaC for what was found; cost attribution of observed workloads to projects; security posture findings (public endpoints, missing policies) as drift kinds.

Decisions and traceability

Family	Ids
FR	FR-09
FUX	FUX-05
NFR	NFR-06, NFR-09
XL	XL-10, XL-14

Related

• Overview · IaCProvisioningService · Product fleet operations · SloService

• Runtime Cloud microservices · bounded contexts · APIs · events · data model

• Services catalog · Ownership map