ObservabilityHost¶

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.


Bounded context	BC-21 Operations
Platform / plane	Observability / Trust & Operations
Deployment granularity	standalone host
Status	target · `ConnectSoft.Factory.Observability`
Built from	connectsoft-microservice · libraries: connectsoft-extensions-observability, connectsoft-extensions-telemetry, connectsoft-extensions-diagnostics-metrics, connectsoft-extensions-ai, connectsoft-extensions-saas-abstractions, connectsoft-extensions-saas-asp-net-core, connectsoft-extensions-messaging-model, connectsoft-extensions-messaging-model-orleans, connectsoft-extensions-persistence-model-nhibernate, connectsoft-extensions-query-model
Decisions	ADR-0012, ADR-0014, ADR-0016, ADR-0018, ADR-0021
Traceability	`FR-10` `FUX-04` `FUX-10` `NFR-10` `NFR-11` `NFR-17` `XL-08` `XL-16`

Purpose and responsibilities¶

ObservabilityHost composes the five Observability modules — DashboardService (operations projections), SloService (objectives and health), AlertRuleService (alerts and notifications), QualityScoreService (evaluation and regression gates) and FeedbackService (feedback, datasets, learning proposals) — into one deployable on the BaseTemplate shape. It owns the observability-sql projection store, the inbox that feeds every module from the producers' events, and the connection to the OpenTelemetry backends.

The host draws one line that the June-2026 catalog did not: traces, logs and raw metrics are not Factory data. Every service emits them with the GenAI semantic conventions through ConnectSoft.Extensions.Observability, and they land in the installation's backend (Azure Monitor by default; Jaeger or Seq for on-prem). The Factory owns what no backend can know — the attribution of every span to a run, step, attempt, model profile, tool, tenant and project, the run timeline that joins Run Control's truth with every receipt, the objectives, the alerts and the evaluation evidence. That is what this host stores and serves.

Context and boundaries¶

flowchart LR
    S["ObservabilityHost"]
    StudioBff["StudioBff"] -->|sync-http| S
    ControlPlaneApi["ControlPlaneApi"] -->|grpc| S
    S -->|sync-http| ext_otel_backend["ext-otel-backend"]
    S -->|grpc| ControlPlaneApi["ControlPlaneApi"]
    S -->|grpc| ModelGatewayHost["ModelGatewayHost"]
    S -->|grpc| GovernanceHost["GovernanceHost"]
    S -->|grpc| KnowledgeHost["KnowledgeHost"]
    S -->|sync-http| ext_identity_platform["ext-identity-platform"]

Hold "Alt" / "Option" to enable pan & zoom

The host is a consumer of nearly every platform's events and a dependency of only two: the Studio BFF (every operations, evaluation and feedback view) and Asset Publication through Control Plane API (the evaluation verdict at the Published gate, ADR-0021). The restore order of D30 puts its store last: everything in it is rebuildable from the run journal and the producers' events.

Domain model¶

None modelled

No aggregates are modelled for this item yet.

No entities modelled.

Entities and value objects¶

None modelled

No entities or value objects are modelled for this item yet.

Data model and storage¶

Store	Role	Engine / profile	Tables
Operations projection store	primary	azure-sql / nhibernate-sqlserver	—
Cache and coordination	cache	redis / azure-cache	—

None modelled

No tables are modelled for this item yet.

APIs¶

Method	Path	Operation	Scope	Side effects	Idempotency	Gateway	Summary
`GET`	`/observability/health`	ObservabilityHealth	`factory.health.read`	read	natural	GW-01	Liveness, readiness and dependency health (projection store, OpenTelemetry backends, model gateway for evaluation, inbox lag).
`GET`	`/observability/meta/backends`	ObservabilityBackends	`factory.meta.read`	read	natural	GW-03	Configured OpenTelemetry backends (traces, metrics, logs) and the GenAI semantic-convention version in use.

None modelled

No gRPC methods are modelled for this item yet.

Messages¶

None modelled

No events are modelled for this item yet.

None modelled

No commands are modelled for this item yet.

None modelled

No consumed events are modelled for this item yet.

Workers and jobs¶

None modelled

No workers are modelled for this item yet.

Actors and grains¶

None modelled

No grains are modelled for this item yet.

Sagas and long-running flows¶

None modelled

No sagas are modelled for this item yet.

flowchart LR
    RC[Run Control journal] -- events --> IN["(inbox)"]
    RT[Runtime · Model router · Tool executor · Context builder · Usage] -- events --> IN
    IN --> D[DashboardService projections]
    D --> S[SloService]
    S --> A[AlertRuleService]
    Q[QualityScoreService] --> F[FeedbackService]
    F --> Q
    D & S & A & Q & F --> ST["(observability-sql)"]
    ST --> BFF[Studio BFF]
    Q --> CP[Control Plane API · publication gate]
    OT["(OpenTelemetry backend)"] -. trace drill-down .-> D

Hold "Alt" / "Option" to enable pan & zoom

Options and configuration¶

Section	Key	Type	Default	Reloadable	Secret	Summary
`Observability:Host`	`EnabledModules`	string[]	`[dashboards, slos, alerts, evaluation, feedback]`	no	no	Modules composed into this host.
`Observability:Backends`	`TraceBackend`	string	`azure-monitor`	no	no	Trace and log backend (Azure Monitor, Jaeger, Seq per installation).
`Observability:Backends`	`GenAiConventionsVersion`	string	`1.x`	no	no	OpenTelemetry GenAI semantic conventions version the spans follow.
`OrleansMessaging`	`ServiceName`	string	`observability`	no	no	Messaging profile service name.

Metrics and observability¶

Metric	Instrument	Unit	Dimensions	Summary
`connectsoft.factory.observability.host.inbox_lag`	gauge	s	module	Age of the oldest unprocessed event per module.
`connectsoft.factory.observability.host.projection_lag`	gauge	s	projection	Lag of each projection behind its source.

Security¶


Authentication	client-credentials
Scopes	`factory.health.read` (Health probes.); `factory.meta.read` (Backend metadata.)
Roles	Operator, PlatformAdmin, Observer
Tenant isolation	row-filter
Data classification	internal
Max side-effect class	publication
Secrets	`observability-sql` (key-vault, rotate 90 d); `otel-backend` (key-vault, rotate 90 d); `messaging-transport` (key-vault, rotate 90 d)

Dependencies¶

Outbound

Target	Kind	Purpose	Failure mode	Required
ext-otel-backend	sync-http	Trace and log queries for timelines; metric storage.	degrade	yes
ControlPlaneApi	grpc	Run journal reads for projection rebuilds; scope resolution.	degrade	yes
ModelGatewayHost	grpc	Model invocations for evaluation suites.	degrade	yes
GovernanceHost	grpc	Policy for waivers; evidence for verdicts.	fail-fast	yes
KnowledgeHost	grpc	Evaluation datasets and records as artifacts.	fail-fast	yes
ext-identity-platform	sync-http	Workload identity.	fail-fast	yes

Inbound

Target	Kind	Purpose	Failure mode	Required
StudioBff	sync-http	Operations, SLO, alert, evaluation and feedback views.	—	yes
ControlPlaneApi	grpc	Evaluation verdicts for publication gates.	—	yes

Failure and recovery¶

Scenario	Detection	Recovery	Data impact	RTO (min)
Event backlog	inbox lag	projections catch up; the Studio shows projection lag; truth stays in the producers	stale views	60
OpenTelemetry backend unavailable	query errors	projections keep serving from the store; trace drill-down degraded	none	60
Projection store lost	readiness	rebuild projections from Run Control's journal and the producers' events (restore order: last)	none	240

NFR targets¶

Target	Class value (pilot-projection)	Override
latency_p95_ms	500	—
availability_pct	99.5	—
throughput_rps	100	—
rpo_minutes	15	—
rto_minutes	240	—
cost_class	low	—

Deployment¶


Granularity	standalone host
Topologies	saas-default, hybrid, on-prem, hybrid-compute
Runtime	aks; replicas 2–8
Ports	https 8100, grpc 8102, silo 12100, gateway 31100

Current implementation¶

ConnectSoft.Extensions.Observability, Telemetry and Diagnostics.Metrics give every service OpenTelemetry tracing, metrics and health endpoints, and the HealthChecks aggregator exists as a template. No Factory host composes projections, objectives, alerts or evaluation over them.

Later capabilities¶

Multi-region projection replication; a self-hosted OpenTelemetry collector profile for hybrid-compute; per-tenant projection stores for the on-prem profile.

Decisions and traceability¶

Family	Ids
FR	FR-10
FUX	FUX-04, FUX-10
NFR	NFR-10, NFR-11, NFR-17
XL	XL-08, XL-16

Observability overview · Deployment · Observability foundations · Runtime telemetry requirements
Observability microservices · bounded contexts · APIs · events · data model
Services catalog · Ownership map

ObservabilityHost¶

Purpose and responsibilities¶

Context and boundaries¶

Domain model¶

Entities and value objects¶

Data model and storage¶

APIs¶

Messages¶

Workers and jobs¶

Actors and grains¶

Sagas and long-running flows¶

Options and configuration¶

Metrics and observability¶

Security¶

Dependencies¶

Failure and recovery¶

NFR targets¶

Deployment¶

Current implementation¶

Later capabilities¶

Decisions and traceability¶

Related¶