Observability — observability of the platform¶
Target
This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.
| Metric | Service | Instrument | Dimensions |
|---|---|---|---|
connectsoft.factory.alerts.fired |
AlertRuleService | counter | tenant, sourceKind, severity |
connectsoft.factory.alerts.open |
AlertRuleService | updown | tenant, severity |
connectsoft.factory.alerts.time_to_acknowledge |
AlertRuleService | histogram | tenant, severity |
connectsoft.factory.operations.projection_lag |
DashboardService | gauge | tenant |
connectsoft.factory.operations.task_success_rate |
DashboardService | gauge | tenant, window |
connectsoft.factory.operations.human_intervention_rate |
DashboardService | gauge | tenant, window |
connectsoft.factory.operations.cost_per_task_usd |
DashboardService | gauge | tenant, window |
connectsoft.factory.learning.feedback |
FeedbackService | counter | tenant, kind |
connectsoft.factory.learning.proposals |
FeedbackService | updown | tenant, kind, state |
connectsoft.factory.learning.datasets |
FeedbackService | updown | tenant, target |
connectsoft.factory.observability.host.inbox_lag |
ObservabilityHost | gauge | module |
connectsoft.factory.observability.host.projection_lag |
ObservabilityHost | gauge | projection |
connectsoft.factory.evaluation.runs |
QualityScoreService | counter | tenant, target, verdict |
connectsoft.factory.evaluation.duration |
QualityScoreService | histogram | target |
connectsoft.factory.evaluation.cost_usd |
QualityScoreService | counter | tenant, target |
connectsoft.factory.evaluation.drift |
QualityScoreService | counter | tenant, metric |
connectsoft.factory.evaluation.waivers |
QualityScoreService | counter | tenant |
connectsoft.factory.slo.state |
SloService | gauge | tenant, subjectKind, kind, state |
connectsoft.factory.slo.error_budget_remaining |
SloService | gauge | tenant, subjectKind, kind |
connectsoft.factory.health.hosts |
SloService | updown | state |
The platform's own signals¶
| Signal | Why it matters |
|---|---|
connectsoft.factory.observability.host.inbox_lag, projection_lag |
a stale projection must be visible before anyone trusts a timeline |
connectsoft.factory.slo.state, error_budget_remaining |
the platform's own objectives (projection freshness, alert delivery time) are objectives like any other |
connectsoft.factory.alerts.open, time_to_acknowledge |
operator load and responsiveness |
connectsoft.factory.evaluation.runs, duration, cost_usd, waivers, drift |
evaluation throughput, spend and how often the gate is bypassed |
connectsoft.factory.learning.proposals by state |
whether the learning loop is flowing or piling up |
Conventions every service follows¶
Spans and metrics across the Factory carry the OpenTelemetry GenAI semantic conventions (gen_ai.system, gen_ai.request.model, gen_ai.usage.input_tokens, gen_ai.usage.output_tokens, gen_ai.operation.name, tool-call spans) plus the Factory attributes connectsoft.factory.tenant_id, project_id, run_id, step_id, attempt_id, agent_id, agent_version, model_profile, deployment_id, tool_id, side_effect_class, approval_id. The span tree of a run (§51) is: run → step → attempt → phase (context, think, act, validate) → model call | tool call | retrieval | validator. traceId is minted by Run Control at acceptance and is in every event envelope (ADR-0014), so timelines, receipts and backend traces meet on one id.
Dashboards the platform ships¶
Operations (runs, cost, interventions), Status (hosts, objectives), Alerts, Evaluation (verdicts, drift), Learning (feedback, proposals), and the platform's own health page. All are projections served by DashboardService and rendered by the Studio; none depend on the backend's dashboarding.