Skip to content

AIFACTORY-FEAT-068: DashboardService: complete delivery

Generated from the architecture model

This page is rendered from model/**/*.yaml by tools/docs_model during mkdocs build. Edit the YAML, not this page; hand-written design lives on the linked service, platform and plane pages.

A design artefact, not a tracker

This backlog is generated from model/backlog/ and validated against the architecture model. No work item exists in any tracker, and no item carries a date, an estimate or an assignee (ADR-0029).

Epic Observability (PF-13) · Increment I4 Trust and accountability · State elaborated · Route backend-developer · Build order 131 of 185 (Factory code)

DashboardService is the service of bounded context BC-21 (Operations). Read models for operating the Factory — per-run timelines that join Run Control truth with runtime, model, tool, context and usage events, tenant and project overviews (task success, cost, tokens, latency, tool failure, fallback and human-intervention rates) and live subscriptions for the Studio; projections are rebuildable and never the truth (BC-21, original design §51). Its own repository is ConnectSoft.AI.SoftwareFactory.DashboardService.

Delivers DashboardService (complete)
Stubs —
Realizes adrs: ADR-0014, ADR-0018
Requirements FR-10, FUX-04, FUX-07, FUX-08, NFR-07, NFR-10, XL-06, XL-12, XL-16
Depends on —
Sources model/services/observability/DashboardService.yaml

Scenarios to trace (8, catalog): SCN-DashboardService-001 ✓, SCN-DashboardService-002 ✓, SCN-DashboardService-003 ✓, SCN-DashboardService-004 ✓, SCN-DashboardService-005 ✓, SCN-DashboardService-006 ✓, SCN-DashboardService-007 ✓, SCN-DashboardService-008 ✓

AIFACTORY-US-045: I read the timeline of a run and the overview of a project

behavior · persona operator As an operator, I want the timeline of a run and overviews per tenant and project, built from the events of the platforms, so that I see what a run did, what it cost and where it failed, in one place.

This is the integration PBI of the feature. DashboardService is the service of bounded context BC-21 (Operations) that builds the read models for operating the Factory, and it is a projection by design and by rule: the journal of Run Control is the truth of a run, the receipts of the producers are the truth of each call, and a projection that disagrees is rebuilt, never edited (ADR-0014). It owns two aggregates in dashboard-sql: RunProjection, the joined view of one run with its TimelineEntry rows, and OverviewProjection, the rolling aggregates per tenant, project and subject for a window. Callers reach its five REST operations through the route GW-03 of the gateway ObservabilityHost, which forwards the user token (D117), under factory.runs.read, factory.operations.read and factory.operations.admin; the Studio BFF opens the internal stream Operations.Subscribe with its workload identity. Every query runs under the tenant row filter, and a run or a subject of another tenant is answered 404, the same answer as for an id that does not exist (ADR-0015, D113). The behaviour is delivered by the behaviour PBIs of this feature: the run projection from the run events (AIFACTORY-US-1123), the run list and the timeline (AIFACTORY-US-1124), the overviews and their digest (AIFACTORY-US-1125), the rebuild (AIFACTORY-US-1126), the operations stream that ends the answer that the stream is unavailable, which AIFACTORY-US-036 AC4 gives until I4 (AIFACTORY-US-1127), and the retention and deletion orders (AIFACTORY-US-1128). In I4 the model consumes 29 events, and this feature registers all of them (D131): the handlers whose events carry a run in the run event envelope and fold into RunProjection, and the three orders of RetentionResidencyService. They are the handlers ProjectRun, ProjectAttempt, ProjectLease, ProjectExecutionAttempt, ProjectReceipt, ProjectInvocationReceipt, ProjectToolInvocation, ProjectContext, ProjectWorkflow, ProjectUsage and ProjectUsageRecord, and ApplyRetentionRule, ApplyLegalHold and ExecuteDeletionOrder; the execution attempt, lease, usage record, invocation receipt and tool invocation projections their purposes name are folded into RunProjection and its timeline. The projection handlers of the other 48 events, among them ProjectAlertInstance, ProjectSloDefinition and ProjectServiceHealth, name a projection that is not about one run and that no table or aggregate of the service holds; they leave the I4 model and arrive in a later increment (D131), so this feature registers none of them. Every projection shows the time of its last update, content never enters a projection, and telemetry follows the conventions of feature component-observability-factory-conventions, taken by the version pin of its packages and never forked. Repository: ConnectSoft.AI.SoftwareFactory.DashboardService, created by the repository checklist of feature repository-and-pipeline-conventions from the base template with the Factory profile.

Acceptance criteria

AC1 Given a run whose events are consumed, when DashboardService/GetRunTimeline is called, then the DashboardService/RunProjection joins the state of Run Control with the runtime, model, tool, context and usage events of the run, entries hold references and hashes and never content, and the projection carries the time of its last update (ADR-0014, FUX-04).
AC2 Given a project with runs in the last window, when the aggregator has run and DashboardService/GetOverview is called, then the DashboardService/OverviewProjection of the project shows task success, cost per task, tokens, latency and tool failures for the window with its computedAt (FUX-08).
AC3 Given a rebuild requested through DashboardService/RebuildRunProjection, when it completes, then the projection equals the one built from the same events before (model/services/observability/DashboardService.yaml).
AC4 Given the repository ConnectSoft.AI.SoftwareFactory.DashboardService, whose first commit is the unchanged output of dotnet new connectsoft-base -n ConnectSoft.AI.SoftwareFactory.DashboardService --UseFactoryProfile true from the pinned template package, when its pipeline runs with the work of this feature, then no second scaffold was run, the solution builds, the tests pass, a container image is published and each of the eight scenarios of DashboardService has a passing test that names its id (model/scenarios/DashboardService.yaml, ADR-0028).
AC5 Given the service starts, when liveness and readiness are probed, then readiness reports dashboard-sql and the messaging transport with the ready tag, and the telemetry conformance test of the pinned observability packages passes with the five metrics of model/services/observability/DashboardService.yaml tagged as modelled: connectsoft.factory.operations.projection_lag by tenant, connectsoft.factory.operations.task_success_rate, connectsoft.factory.operations.human_intervention_rate and connectsoft.factory.operations.cost_per_task_usd by tenant and window, and connectsoft.factory.operations.inbox_lag by service (ADR-0018).
Traces: services DashboardService, RunControlService, StudioRealtimeGateway, ObservabilityHost
Traces: aggregates DashboardService/RunProjection, DashboardService/OverviewProjection
Traces: apis DashboardService/ListRunProjections, DashboardService/GetRunTimeline, DashboardService/GetOverview, DashboardService/GetSubjectOverview, DashboardService/RebuildRunProjection, DashboardService/Operations.Subscribe
Traces: events DashboardService/RunProjectionUpdated, DashboardService/OperationsDigestProduced
Traces: adrs ADR-0014, ADR-0015, ADR-0018, ADR-0028
Traces: requirements FR-10, FUX-04, FUX-07, FUX-08, XL-06, XL-12, XL-16
Relates: screens run-list, run-timeline, overview
Relates: nfrs NFR-07, NFR-10
Depends on AIFACTORY-US-1123, AIFACTORY-US-1124, AIFACTORY-US-1125, AIFACTORY-US-1126, AIFACTORY-US-1127, AIFACTORY-US-1128, feature:observability-host, feature:studio-realtime-gateway, feature:repository-and-pipeline-conventions, feature:component-base-template-factory-profile, feature:contracts-package, feature:component-observability-factory-conventions
Sources model/services/observability/DashboardService.yaml, model/scenarios/DashboardService.yaml, docs/planes/trust-operations/observability/services/dashboard-service/index.md#purpose-and-responsibilities

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-769 Verify operations-read-models by running the declared command verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario~SCN-DashboardService" in the pipeline of ConnectSoft.AI.SoftwareFactory.DashboardService; a filter that discovers no test counts as a failure; it passes and proves AC1 to AC5 of AIFACTORY-US-045, with a test for each of SCN-DashboardService-001 to SCN-DashboardService-008. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-479 Create the repository ConnectSoft.AI.SoftwareFactory.DashboardService and scaffold it from the ConnectSoft.BaseTemplate template with the Factory profile implementation The repository exists by the repository checklist of feature repository-and-pipeline-conventions, the output of dotnet new connectsoft-base -n ConnectSoft.AI.SoftwareFactory.DashboardService --UseFactoryProfile true from the pinned template package is committed unchanged with the messaging service name dashboard, the repository records the template version, the solution builds and the template tests pass in the repository pipeline, which publishes the container image. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-480 Domain model: RunProjection, OverviewProjection implementation The aggregates hold the fields and the invariants of the model: a projection is changed only from events or by a rebuild, usage amounts carry their status and uncertain amounts are never folded into totals, and an entry holds references and hashes and never content; unit tests cover every invariant. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-481 Persistence in dashboard-sql: RunProjections, TimelineEntries, ProjectionCheckpoints, OverviewProjections implementation Mappings and migrations for the four modelled tables with the tenant filter, the indexes on tenant, state and update time and on project, and the unique index of an overview per tenant, kind, subject and window; mapping tests run against an isolated test database, never a shared one. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-482 API hosting skeleton: 5 REST and 1 gRPC operations with scope enforcement, problem responses and contract documents implementation ListRunProjections, GetRunTimeline, GetOverview, GetSubjectOverview and RebuildRunProjection are hosted with JWT validation of the token the gateway forwards, scope enforcement per route and problem responses for every modelled refusal code, and Operations.Subscribe with the workload identity check, with the OpenAPI and proto documents; the behaviour of each operation is delivered by AIFACTORY-US-1124 to AIFACTORY-US-1127, not here. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-483 Messaging skeleton: outbox and inbox of the dashboard profile for 2 published and 29 consumed events implementation The outbox writer enlists in the transaction of each projection change and the dispatcher publishes RunProjectionUpdated and OperationsDigestProduced under the canonical envelope; the inbox registers, with idempotency on the event id, the 26 run events of the handlers ProjectRun, ProjectAttempt, ProjectLease, ProjectExecutionAttempt, ProjectReceipt, ProjectInvocationReceipt, ProjectToolInvocation, ProjectContext, ProjectWorkflow, ProjectUsage and ProjectUsageRecord and the three orders of RetentionResidencyService, and no other handler of the model; what each handler does is delivered by the behaviour PBIs of this feature. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-484 Worker hosting skeleton: triggers, partitioning and retry policies of ProjectionRebuilder and OverviewAggregator implementation The two workers are hosted with their modelled triggers (on request or restore, a schedule every five minutes), retry policies, partitioning by tenant and idempotency keys (runId plus lastEventSequence, tenantId plus window plus computedAt), and a duplicate trigger has no second effect; what each worker does is delivered by AIFACTORY-US-1125 and AIFACTORY-US-1126. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-485 Security, tenancy and telemetry of DashboardService implementation The three scopes and six roles of the model are enforced, every query is tenant-filtered, the messaging transport and telemetry backend secrets come from secret references, the host makes the one call of the registration of the pinned observability packages, creates the five modelled metrics through its instrument helper and references the conformance test package from its architecture tests. ConnectSoft.AI.SoftwareFactory.DashboardService

AIFACTORY-US-1123: Build the projection and the timeline of a run from the events of Run Control, the runtime, the model router, the tool adapter, context and usage

behavior · persona operator As an operator, I want the projection of a run kept current from the events of every platform that touches it, so that one view shows the state, the steps, the attempts, the receipts and the cost of a run without asking each service.

The inbox handlers of the run projection each consume one family of events, idempotent on the event id, and join an event to its run by the runId and tenantId of the run event envelope (ADR-0014). runId is optional in docs/reference/schemas/run-event-envelope.schema.json: an event that arrives without it is recorded once in the inbox, projected onto no run, logged with its event id and type and counted, and nothing else changes; which producers may omit it is reported to the owner. ProjectRun opens the RunProjection on RunAccepted with the tenant, project and workflow and follows RunStateChanged, RunCompleted and RunFailed for the state in the vocabulary of Run Control, the duration and the failure class; approvalsWaiting is derived from RunStateChanged into and out of the state WaitingForApproval, since the service consumes no approval event of Run Control. ProjectAttempt records the attempts from AttemptDispatched, AttemptOutcomeRecorded and AttemptPhaseChanged, ProjectLease the expired leases from LeaseExpired, and ProjectExecutionAttempt the attempt start, checkpoints, decisions requested, cancellations observed, reported outcomes and delegations of AgentRuntimeService. ProjectReceipt records model receipts with tokens and cost from ModelInvoked and tool receipts from ToolInvoked, and ProjectInvocationReceipt marks the model receipt of ModelFallbackUsed as answered by a fallback; ProjectToolInvocation records the denied invocations from ToolInvocationDenied, counts the uncertain effects from ToolEffectUncertain and records the sandbox lease and release of an invocation from SandboxLeased and SandboxReleased. ProjectContext records the snapshots of ContextAssembled, ProjectWorkflow the step states of StepRunStateChanged, and ProjectUsage and ProjectUsageRecord the usage by status from UsageRecorded, ReservationSettled and UsageReconciled. Each handler appends a TimelineEntry with its kind, a receipt, snapshot, approval or decision reference and a summary, updates the counts, the tokens, the cost by status and lastEventSequence, sets updatedAt, and writes RunProjectionUpdated with runId, tenantId, state and lastEventSequence in the same transaction. The envelope has no sequence field: the projection assigns the sequence of each entry in commit order and lastEventSequence is the sequence of the latest entry, while the order of the run itself is read from aggregate.version of the envelope on the events whose aggregate is the run, RunAccepted, RunStateChanged, RunCompleted and RunFailed; events of other producers take no part in gap detection, and the source of a sequence across producers is reported to the owner. Uncertain amounts are kept by status and never folded into totals (FUX-08), and no content enters a projection. A gap in the run versions is the failure mode out-of-order events: the projection waits for the gap within a window and then rebuilds the run through the rebuilder of AIFACTORY-US-1126. A backlog shows as connectsoft.factory.operations.projection_lag and connectsoft.factory.operations.inbox_lag, while the truth stays in the producers. The archival to artifacts that Observability:Operations:TimelineMaxEntries names has no dependency on ArtifactService in the model and is reported to the owner. StudioBff and StudioRealtimeGateway consume RunProjectionUpdated in their own PBIs.

Acceptance criteria

AC1 Given RunControlService/RunAccepted of a run, then RunControlService/AttemptDispatched, ModelRouterService/ModelInvoked, ToolAdapterService/ToolInvoked, ContextBuilderService/ContextAssembled, CostUsageService/UsageRecorded and RunControlService/RunCompleted of the same run in sequence, when the handlers consume them, then one DashboardService/RunProjection holds the state Completed, the counts of steps, attempts, model calls and tool calls, the tokens, the cost by status and the duration, with one TimelineEntry per event in sequence order holding references and hashes and no content; a redelivered event id changes nothing (SCN-DashboardService-007, ADR-0014).
AC2 Given ToolAdapterService/ToolEffectUncertain and a CostUsageService/UsageRecorded with an uncertain amount for a run, when they are projected, then uncertainEffects counts the effect and the uncertain amount is kept under its status in the cost by status and is not part of any total; given RunControlService/RunStateChanged into WaitingForApproval and later out of it, then approvalsWaiting rises and falls with it; given a ModelRouterService/ModelFallbackUsed of an invocation of the run, then its model receipt is marked as a fallback; given an event without runId in its envelope, then it is recorded once in the inbox and no projection changes (FUX-08).
AC3 Given a RunControlService/RunStateChanged whose aggregate.version leaves a gap in the versions of its run, when the gap is not filled within the window, then the run is rebuilt through the rebuilder and the projection afterwards equals the projection built from the complete sequence; while events wait in the inbox, connectsoft.factory.operations.projection_lag reports the lag (model/services/observability/DashboardService.yaml).
AC4 Given the projection of a run changed, when the outbox dispatcher publishes DashboardService/RunProjectionUpdated on operations.projections.v1.run-projection-updated, then the outbox record was written in the transaction of the change, the event is sent under the canonical envelope with runId, tenantId, state and lastEventSequence, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-DashboardService-007, ADR-0012).
Traces: services DashboardService, RunControlService, AgentRuntimeService, ModelRouterService, ToolAdapterService, ContextBuilderService, CostUsageService, WorkflowEngineService
Traces: aggregates DashboardService/RunProjection
Traces: events DashboardService/RunProjectionUpdated, RunControlService/RunAccepted, RunControlService/RunStateChanged, RunControlService/RunCompleted, RunControlService/RunFailed, RunControlService/AttemptDispatched, RunControlService/AttemptOutcomeRecorded, RunControlService/LeaseExpired, AgentRuntimeService/AttemptPhaseChanged, ModelRouterService/ModelInvoked, ModelRouterService/ModelFallbackUsed, ToolAdapterService/ToolInvoked, ToolAdapterService/ToolInvocationDenied, ToolAdapterService/ToolEffectUncertain, ToolAdapterService/SandboxLeased, ToolAdapterService/SandboxReleased, ContextBuilderService/ContextAssembled, CostUsageService/UsageRecorded, CostUsageService/ReservationSettled, CostUsageService/UsageReconciled, WorkflowEngineService/StepRunStateChanged
Traces: adrs ADR-0012, ADR-0014
Traces: requirements FR-10, FUX-07, FUX-08, XL-06
Traces: scenarios SCN-DashboardService-007
Relates: nfrs NFR-07
Depends on feature:run-control-service, feature:agent-runtime-service, feature:model-router-service, feature:tool-adapter-service, feature:context-builder-service-thin, feature:cost-usage-service-complete, feature:workflow-engine-service
Sources model/services/observability/DashboardService.yaml, docs/reference/schemas/run-event-envelope.schema.json, docs/planes/trust-operations/observability/services/dashboard-service/index.md#sagas-and-long-running-flows

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-549 Run projection from the consumed events implementation The handlers ProjectRun, ProjectAttempt, ProjectLease, ProjectExecutionAttempt, ProjectReceipt, ProjectInvocationReceipt, ProjectToolInvocation, ProjectContext, ProjectWorkflow, ProjectUsage and ProjectUsageRecord update the RunProjection of the runId of the envelope and append its TimelineEntry rows in commit order idempotently on the event id, derive approvalsWaiting from RunStateChanged into and out of WaitingForApproval, keep usage by status without folding uncertain amounts, write RunProjectionUpdated in the transaction of the change, record an event without runId in the inbox only, and hand a run whose aggregate versions show a gap that outlasts the window to the rebuilder. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4489 Run event sequences built from the pinned contracts package fixture A fixture builds the consumed events of a run from the pinned contracts package under the run event envelope in a complete sequence, with a redelivered event id, with an uncertain usage amount, with a run that waits for an approval, with a gap in the run versions and with an event without runId, so that the suites of the projection and the rebuild replay the same runs. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4490 Verify run-projection-built-from-run-events through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-DashboardService-007"; a zero-match filter counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-1123 with the sequences of AIFACTORY-TASK-4489, including the redelivered event, the uncertain amount kept apart, the gap that leads to a rebuild and the payload of RunProjectionUpdated; consumer handling is proven by the suites of StudioBff and StudioRealtimeGateway. ConnectSoft.AI.SoftwareFactory.DashboardService

AIFACTORY-US-1124: List the runs in my scope and read the timeline of one run with its freshness

behavior · persona requester As a requester, I want the runs of my projects with state, counts, cost by status and waits, and the timeline of one run, so that I follow my runs in the Studio and see where one spent its time and money.

A caller whose scope carries factory.runs.read, a Requester, Reviewer, Operator or Observer, lists the run projections of the trusted tenant and the projects in scope through DashboardService/ListRunProjections, the run list of the Studio: each run with its state in the vocabulary of Run Control, the counts of steps, attempts, model and tool calls, the approvals waiting, the uncertain effects, the tokens, the cost by status and updatedAt. The read runs under the tenant row filter and the project scope, so runs of another tenant never appear. The same caller reads one run through DashboardService/GetRunTimeline: the projection with its TimelineEntry rows in sequence order, the original design trace tree as rows of steps, attempts, model and tool receipts, context snapshots, approvals, decisions and usage, each entry with its reference, summary, time, duration and cost and never content, and the traceId of the run. Both responses carry the time of the last update and the projection lag, so the Studio shows how fresh the view is. When the telemetry backend answers, the timeline carries a drill-down link for the trace; when it is unavailable, the timeline is served from the store and the link is absent, which is the modelled degradation. The drill-down is built from the model of the service (D140): the option Observability:Backends:TraceBackend names the trace and log backend of the installation, Azure Monitor by default and Jaeger or Seq where an installation runs one, and Observability:Backends:GenAiConventionsVersion names the version of the OpenTelemetry GenAI semantic conventions the spans follow, 1.x by default; both are bound at startup and neither is reloadable. The link is built for the configured backend from the traceId of the run and states the conventions version beside it. The backend is reached as the optional dependency ext-otel-backend with the failure mode degrade, and its credential is the secret otel-backend, read from its key-vault reference, rotated every 90 days and never written to a log record, an activity, a metric or a response. A backend that does not answer is the failure mode OpenTelemetry backend unavailable, detected by query errors: the projections keep serving from the store, the link is left out and readiness does not change. A caller without the scope is refused with 403; a run of another tenant or outside the projects in scope is answered 404, the same answer as for a runId that does not exist (ADR-0015, D113). Nothing is written by these reads.

Acceptance criteria

AC1 Given run projections in two projects of my tenant and in another tenant, when DashboardService/ListRunProjections is called with factory.runs.read for one project in scope, then the response is 200 with the runs of that project only, each with state, counts, approvals waiting, uncertain effects, tokens, cost by status and updatedAt, and the projection lag is stated; nothing is written; a caller without the scope is refused with 403 (SCN-DashboardService-001, ADR-0015).
AC2 Given a run of my tenant and project with timeline entries, when DashboardService/GetRunTimeline is called, then the response is 200 with the DashboardService/RunProjection and its entries in sequence order, each with its kind, reference, summary, time, duration and cost and no content, the traceId and a drill-down link while the telemetry backend answers, and updatedAt with the projection lag (SCN-DashboardService-002, FUX-04).
AC3 Given the telemetry backend is unavailable, when DashboardService/GetRunTimeline is called, then the response is 200 from the store without the drill-down link; given a caller without factory.runs.read, a runId of another tenant or a runId that does not exist, then the response is 403 for the missing scope and 404 for the other two alike, with a problem response and no state change (SCN-DashboardService-002, ADR-0015).
AC4 Given Observability:Backends:TraceBackend set in turn to azure-monitor, jaeger and seq and Observability:Backends:GenAiConventionsVersion at its default, when DashboardService/GetRunTimeline is called for a run with a traceId, then the drill-down link has the form of the configured backend for that traceId and states the conventions version; the credential of ext-otel-backend is read from the key-vault reference of the secret otel-backend and appears in no log record, activity, metric or response; given the backend answers with query errors, then the timeline is served without the link and readiness does not change (model/services/observability/DashboardService.yaml, SCN-DashboardService-002).
Traces: services DashboardService
Traces: aggregates DashboardService/RunProjection
Traces: apis DashboardService/ListRunProjections, DashboardService/GetRunTimeline
Traces: adrs ADR-0014, ADR-0015
Traces: requirements FUX-04, FUX-07, FUX-08
Traces: scenarios SCN-DashboardService-001, SCN-DashboardService-002
Relates: screens run-list, run-timeline, run-detail
Relates: nfrs NFR-07
Depends on AIFACTORY-US-1123
Sources model/services/observability/DashboardService.yaml, docs/planes/trust-operations/observability/services/dashboard-service/index.md#apis

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-4491 ListRunProjections and GetRunTimeline with project scope, freshness and the degraded drill-down implementation ListRunProjections returns the projections of the trusted tenant and the projects in scope with counts, cost by status, updatedAt and the projection lag; GetRunTimeline returns the projection with its entries in sequence order, the traceId and a drill-down link from a typed client of the telemetry backend that is left out when the backend fails; a missing scope is 403 and a foreign, out-of-scope or unknown run is 404. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-6606 Trace drill-down: Observability:Backends TraceBackend and GenAiConventionsVersion, the secret otel-backend and the dependency ext-otel-backend implementation The service binds Observability:Backends:TraceBackend (default azure-monitor; azure-monitor, jaeger or seq) and Observability:Backends:GenAiConventionsVersion (default 1.x) at startup, validates them and fails startup with the value named for a backend it does not know, reloads neither, reads the credential of the backend from the key-vault reference of the secret otel-backend with its rotation of 90 days, and calls ext-otel-backend through a typed client registered as an optional dependency with the failure mode degrade; GetRunTimeline builds the drill-down link from the traceId of the run in the form of the configured backend and states the conventions version beside it, leaves the link out when the backend answers with query errors and logs the failure with the correlation id, and the readiness probe does not report the backend; the credential appears in no log record, activity, metric or response. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4492 Verify run-list-and-run-timeline-read through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-DashboardService-001\|Scenario=SCN-DashboardService-002)"; a zero-match filter counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-1124 over a two-tenant fixture with a fake telemetry backend that can fail, including the 404 for a foreign run, the link built for each of the three backends with the conventions version, and a scan of the captured telemetry for the marker credential of otel-backend. ConnectSoft.AI.SoftwareFactory.DashboardService

AIFACTORY-US-1125: Aggregate tenant, project and subject overviews per window, serve them and announce each digest

behavior · persona workspace-admin As a workspace admin, I want success, cost, tokens, latency and tool failures of my tenant and projects over a window, so that I see whether the Factory serves my teams well and what it costs, without counting runs by hand.

The OverviewAggregator worker runs every five minutes, partitioned by tenant with at most two in parallel, and recomputes the OverviewProjection of every tenant and project for each window of Observability:Operations:Windows (1h, 24h, 7d, 30d by default) from the run projections updated in the window; aggregates are never written by hand. It computes the metrics whose inputs the run projection holds: task success, cost per task from reconciled amounts, latency per task, tokens per task, tool failure rate, model fallback rate and human intervention rate. Each overview is written once per tenant, kind, subject and window under the unique index, with computedAt, and OperationsDigestProduced with tenantId, window and computedAt is written in the same transaction; the worker is idempotent on tenant, window and computedAt and attempts a failed pass three times in all with the modelled fixed backoff of 60 seconds. The gauges connectsoft.factory.operations.task_success_rate, human_intervention_rate and cost_per_task_usd follow the tenant and window. A caller whose scope carries factory.operations.read reads the tenant and project overviews per window through DashboardService/GetOverview, and the overview of one subject through DashboardService/GetSubjectOverview on /factory/operations/overview/{kind}/{subjectId}. In I4 an overview is kept only of the kinds ProjectOverview and TenantOverview (D131): GetSubjectOverview serves a ProjectOverview with a project id of the trusted tenant as subjectId and a TenantOverview with the id of the trusted tenant, and answers 404 for any other kind, among them AgentPerformance, ToolReliability, ModelUsage and RunTimeline. The overviews AgentPerformance, ToolReliability and ModelUsage, and the metrics agent success, sandbox failures, context retrieval quality and evaluation score for which the run projection holds no input, leave the I4 model and arrive in a later increment. A caller without the scope is refused with 403; a subject of another tenant is answered 404, the same answer as for one that does not exist (ADR-0015, D113). StudioBff and SloService consume OperationsDigestProduced in their own PBIs.

Acceptance criteria

AC1 Given run projections of two projects updated within the last hour, when the OverviewAggregator runs, then one DashboardService/OverviewProjection per tenant and project and per window holds task success, cost per task from reconciled amounts, latency, tokens per task, tool failure rate, model fallback rate and human intervention rate with computedAt, and the DashboardService/OperationsDigestProduced outbox record is written in the same transaction; a duplicate trigger for the same tenant, window and computedAt writes nothing more (SCN-DashboardService-008).
AC2 Given the overviews of my tenant, when DashboardService/GetOverview is called with factory.operations.read, then the response is 200 with the tenant and project overviews per window, each with computedAt, and nothing is written; a caller without the scope is refused with 403 (SCN-DashboardService-003, FUX-08).
AC3 Given the aggregator has written the ProjectOverview of a project of my tenant, when DashboardService/GetSubjectOverview is called with the kind ProjectOverview and the project id, then the response is 200 with the metrics of each window and computedAt; given the kind AgentPerformance, ToolReliability or ModelUsage with any subject id, a project of another tenant or a project without an overview, then the response is 404 alike, and a caller without the scope is refused with 403, with a problem response and no state change (SCN-DashboardService-004, ADR-0015).
AC4 Given an overview was recomputed, when the outbox dispatcher publishes DashboardService/OperationsDigestProduced on operations.projections.v1.operations-digest-produced, then the event is sent under the canonical envelope with tenantId, window and computedAt, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-DashboardService-008, ADR-0012).
Traces: services DashboardService
Traces: aggregates DashboardService/OverviewProjection, DashboardService/RunProjection
Traces: apis DashboardService/GetOverview, DashboardService/GetSubjectOverview
Traces: events DashboardService/OperationsDigestProduced
Traces: adrs ADR-0012, ADR-0015
Traces: requirements FR-10, FUX-04, FUX-08
Traces: scenarios SCN-DashboardService-003, SCN-DashboardService-004, SCN-DashboardService-008
Relates: screens overview
Relates: nfrs NFR-10
Depends on AIFACTORY-US-1123
Sources model/services/observability/DashboardService.yaml, docs/planes/trust-operations/observability/services/dashboard-service/index.md#workers-and-jobs

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-550 Overview aggregation per tenant, project and subject implementation The OverviewAggregator recomputes the tenant and project overviews for every configured window from the run projections updated in it, writes each overview under the unique index with computedAt and the OperationsDigestProduced outbox record in one transaction, is idempotent on tenantId, window and computedAt, attempts a failed pass three times in all with a 60 second backoff and sets the three overview gauges. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4493 GetOverview and GetSubjectOverview under the tenant row filter implementation GetOverview returns the tenant and project overviews of the trusted tenant per window with computedAt; GetSubjectOverview returns the ProjectOverview of a project or the TenantOverview of the trusted tenant addressed by kind and subject id and answers 404 for the kinds AgentPerformance, ToolReliability, ModelUsage and RunTimeline, for a foreign subject and for a subject without an overview alike; a missing scope is 403. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4494 Verify overviews-aggregated-per-window through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-DashboardService-003\|Scenario=SCN-DashboardService-004\|Scenario=SCN-DashboardService-008)"; a zero-match filter counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-1125 over a two-tenant fixture, including the duplicate trigger, the 200 for a ProjectOverview, the 404 for the kinds AgentPerformance, ToolReliability and ModelUsage and for a foreign subject, and the payload of OperationsDigestProduced; consumer handling is proven by the suites of StudioBff and SloService. ConnectSoft.AI.SoftwareFactory.DashboardService

AIFACTORY-US-1126: Rebuild the projection of one run from the journal of Run Control and the events of the producers

behavior · persona operator As an operator, I want to rebuild the projection of a run that looks wrong, so that the view returns to what the truth of the run says, without anyone editing a projection by hand.

An Operator whose scope carries factory.operations.admin asks for the rebuild of one run of the trusted tenant through DashboardService/RebuildRunProjection on /factory/operations/runs/{runId}/rebuild. The service answers 202 and the ProjectionRebuilder worker rebuilds the RunProjection and its TimelineEntry rows from the journal of Run Control and the events of the producers, replaces the stored rows in one transaction with the RunProjectionUpdated outbox record of the rebuilt state, and the rebuilt projection equals the one built from the same events before. The worker is idempotent on the run id and lastEventSequence, so a repeated request while a rebuild of the same state is pending answers 202 and rebuilds once; a failed rebuild is attempted five times in all with the modelled exponential backoff from 60 seconds, and the existing projection keeps serving until the rebuild commits. The same worker rebuilds a run whose sequence gap outlasts its window (AIFACTORY-US-1123) and, after a restore, every projection when the projection store was lost, which readiness detects and which comes last in the restore order. The model gives the service a required gRPC dependency on RunControlService for journal reads with the failure mode degrade, but RunControlService models no gRPC method that returns its journal: its one stream, RunControl.WatchRun, is a live server stream of run and approval state changes from the moment of the call, not a replay of the journal from the start of the run, so it cannot rebuild a projection; and the model does not say where the events of the producers are read again; both are reported to the owner, and this PBI builds the rebuilder behind a journal port that the acceptance suite binds to a fake. A caller without the scope is refused with 403; a run of another tenant is answered 404, the same answer as for a runId that does not exist (ADR-0015, D113).

Acceptance criteria

AC1 Given a run of my tenant whose projection was built from a sequence of events, when DashboardService/RebuildRunProjection is called with factory.operations.admin, then the response is 202, the ProjectionRebuilder replaces the DashboardService/RunProjection and its entries with the state rebuilt from the journal port and the same events, the result equals the projection built before, and DashboardService/RunProjectionUpdated is written in the same transaction (SCN-DashboardService-005).
AC2 Given a rebuild of the same run and lastEventSequence already pending, when the request is repeated, then the response is 202 and one rebuild runs; given the journal port fails, then the rebuild is retried with exponential backoff and the existing projection keeps serving DashboardService/GetRunTimeline until a rebuild commits (model/services/observability/DashboardService.yaml).
AC3 Given a caller without factory.operations.admin, a runId of another tenant or a runId that does not exist, when DashboardService/RebuildRunProjection is called, then the response is 403 for the missing scope and 404 for the other two alike, with a problem response, and no rebuild is started (SCN-DashboardService-005, ADR-0015).
AC4 Given the projection store was restored empty, when readiness reports it and the restore trigger runs, then the ProjectionRebuilder rebuilds every run projection from the journal port and the events, partitioned by tenant with at most four in parallel, and each rebuilt run equals its projection before the loss (ADR-0018).
Traces: services DashboardService, RunControlService
Traces: aggregates DashboardService/RunProjection
Traces: apis DashboardService/RebuildRunProjection, DashboardService/GetRunTimeline
Traces: events DashboardService/RunProjectionUpdated
Traces: adrs ADR-0014, ADR-0015, ADR-0018
Traces: requirements FR-10, XL-12
Traces: scenarios SCN-DashboardService-005
Relates: screens run-list
Depends on AIFACTORY-US-1123
Sources model/services/observability/DashboardService.yaml, docs/planes/trust-operations/observability/services/dashboard-service/index.md#failure-and-recovery

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-4495 RebuildRunProjection and the ProjectionRebuilder behind a journal port implementation RebuildRunProjection answers 202 and queues the rebuild of a run of the trusted tenant, 403 without the scope and 404 for a foreign or unknown run; the ProjectionRebuilder reads the run through a journal port, replays the events of the run through the handlers of AIFACTORY-US-1123 into a fresh projection, replaces the stored rows with the RunProjectionUpdated outbox record in one transaction, is idempotent on runId plus lastEventSequence, attempts a failed rebuild five times in all with a 60 second exponential backoff and rebuilds every projection on the restore trigger. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4496 Fake journal port for the rebuild suite fixture A fake of the journal port returns the journal of the runs of the fixture of AIFACTORY-TASK-4489 and can fail, so that the suite proves the rebuild, the retry and the restore without a deployed Run Control. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4497 Verify run-projection-rebuilt-on-request through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-DashboardService-005"; a zero-match filter counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-1126 against the fake journal port, including the equality of the rebuilt projection, the repeated request, the failing port and the restore of an empty store. ConnectSoft.AI.SoftwareFactory.DashboardService

AIFACTORY-US-1127: Stream the projection updates of the runs in scope to the Studio BFF

behavior · persona requester As a requester, I want the run pages of the Studio to move as the run moves, so that I see each change of my run without reloading the page.

The Studio BFF opens the internal server stream DashboardService/Operations.Subscribe with its workload identity and an execution-scope-v1 that names the tenant and the projects of the user it serves. The service reads that tenant and project only when the workload token holds the service scope that names DashboardService (D114), and then streams one run-event-envelope-v1 per committed change of a run projection in scope, carrying the run id, the tenant, the state and lastEventSequence of the change, in commit order, until the caller cancels. The stream reads the store under the tenant row filter, so a run of another tenant is never sent, and it writes nothing. A caller without a workload identity is refused with UNAUTHENTICATED, a token without the service scope with PERMISSION_DENIED and a request that fails the validation of execution-scope-v1 with INVALID_ARGUMENT; no stream is opened. With this operation the service ends the reason for the answer that the operations stream is unavailable, which AIFACTORY-US-036 AC4 of feature studio-realtime-gateway gives until the operations projections are delivered. Switching the stream bridge of ConnectSoft.AI.SoftwareFactory.StudioRealtimeGateway from that answer to this stream is a change of that repository and is left to feature studio-realtime-gateway, which is reported to the owner; StudioRealtimeGateway consumes RunProjectionUpdated through its handler PushRun in its own PBI.

Acceptance criteria

AC1 Given the Studio BFF with a workload token that holds the service scope of DashboardService and an execution-scope-v1 naming one tenant and project, when DashboardService/Operations.Subscribe is called and two runs of that project change, then the stream delivers one run-event-envelope-v1 per committed change with run id, tenant, state and lastEventSequence in commit order, and nothing is written (SCN-DashboardService-006).
AC2 Given a run of another tenant and a run of a project outside the scope change while the stream is open, when the changes commit, then neither is delivered on the stream (SCN-DashboardService-006, ADR-0015).
AC3 Given a caller without workload identity, a token without the service scope of DashboardService or a request that fails validation, when DashboardService/Operations.Subscribe is called, then the call is refused with UNAUTHENTICATED, PERMISSION_DENIED or INVALID_ARGUMENT respectively and no stream is opened (SCN-DashboardService-006, ADR-0015).
Traces: services DashboardService, StudioBff, StudioRealtimeGateway
Traces: aggregates DashboardService/RunProjection
Traces: apis DashboardService/Operations.Subscribe
Traces: descriptors execution-scope-v1, run-event-envelope-v1
Traces: adrs ADR-0014, ADR-0015
Traces: requirements FUX-04, FUX-07
Traces: scenarios SCN-DashboardService-006
Relates: screens run-list, run-timeline
Depends on AIFACTORY-US-1123
Sources model/services/observability/DashboardService.yaml, docs/reference/schemas/run-event-envelope.schema.json, docs/reference/schemas/execution-scope.schema.json

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-551 The operations stream to the Studio BFF implementation Operations.Subscribe streams one run-event-envelope-v1 per committed change of a run projection of the tenant and projects of the execution-scope-v1 the caller forwards, in commit order and until the caller cancels, reading the forwarded scope only when the workload token holds the service scope of DashboardService; a caller without identity is UNAUTHENTICATED, a token without the scope PERMISSION_DENIED and an invalid request INVALID_ARGUMENT. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4498 Verify operations-stream-for-the-studio through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-DashboardService-006"; a zero-match filter counts as a failure; the tests prove AC1 to AC3 of AIFACTORY-US-1127 with an in-process gRPC client, including the run of another tenant that is never streamed and the three refusals. ConnectSoft.AI.SoftwareFactory.DashboardService

AIFACTORY-US-1128: Apply retention rules, legal holds and deletion orders to the run projections and overviews of a tenant

behavior · persona workspace-admin As a workspace admin, I want the projections of my tenant kept no longer than the retention of the runs, held on a legal hold and deleted on order, so that a deleted run or project leaves no copy behind in the operations views.

Three inbox handlers apply the orders of RetentionResidencyService to dashboard-sql, each idempotent on the event id. ApplyRetentionRule handles RetentionRuleChanged and applies the effective retention period of the rule to the records of its store class; run projections follow the retention of the run, one year by default, and the retention job of the service deletes the projections and timeline entries older than the effective period. ApplyLegalHold handles LegalHoldChanged and suspends deletion and expiry for the held scope while the hold is active. ExecuteDeletionOrder handles DeletionOrderIssued: an order of tenant scope deletes every row of the tenant in RunProjections, TimelineEntries, ProjectionCheckpoints and OverviewProjections; an order of project scope deletes the run projections of the project with their timeline entries and the overviews whose subject is the project; a run purge deletes the projection and the timeline of the run. A held scope is not deleted and the receipt names the hold. The handler acknowledges its task through RetentionResidencyService/AcknowledgeOrderTask with the workload token of the service, which holds factory.data.acknowledge, and a receipt that names the order, the service, the tables and the row counts; a redelivered order deletes nothing more and acknowledges with the same receipt, and an answer 403 or 404 or a failure before the acknowledgement leaves the task unacknowledged, is logged with the order id and is retried by the inbox. A deleted projection is not rebuilt for a scope the order removed.

Acceptance criteria

AC1 Given RetentionResidencyService/DeletionOrderIssued of project scope for one of two projects, when ExecuteDeletionOrder handles it, then the DashboardService/RunProjection rows of that project with their timeline entries and the DashboardService/OverviewProjection rows whose subject is that project are deleted, the other project and other tenants are untouched, and the task is acknowledged through RetentionResidencyService/AcknowledgeOrderTask with a receipt that names the tables and row counts (ADR-0019, XL-14).
AC2 Given an order of tenant scope and a run purge, when they are handled, then every row of the tenant in the four tables, and the projection and timeline of the purged run, are deleted; a redelivered order deletes nothing more and acknowledges with the same receipt; an acknowledgement answered 403 or 404 leaves the task unacknowledged and the inbox retries it (model/services/data-event/RetentionResidencyService.yaml).
AC3 Given RetentionResidencyService/LegalHoldChanged that holds a project and RetentionResidencyService/RetentionRuleChanged that shortens the retention of the store class of dashboard-sql, when the retention job runs, then no projection of the held project is deleted, and after the hold is lifted the projections older than the effective period are deleted (ADR-0019).
Traces: services DashboardService, RetentionResidencyService
Traces: aggregates DashboardService/RunProjection, DashboardService/OverviewProjection
Traces: apis RetentionResidencyService/AcknowledgeOrderTask
Traces: events RetentionResidencyService/RetentionRuleChanged, RetentionResidencyService/LegalHoldChanged, RetentionResidencyService/DeletionOrderIssued
Traces: adrs ADR-0019
Traces: requirements XL-14
Depends on AIFACTORY-US-1123, feature:retention-residency-service
Sources model/services/observability/DashboardService.yaml, model/services/data-event/RetentionResidencyService.yaml, docs/architecture/adr/0019-data-governance.md

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-4499 ApplyRetentionRule, ApplyLegalHold and ExecuteDeletionOrder over the four projection tables implementation The three handlers apply the effective retention period through a retention job of the service, suspend and resume expiry and deletion for a held scope, delete the rows of a tenant order, a project order and a run purge in RunProjections, TimelineEntries, ProjectionCheckpoints and OverviewProjections, acknowledge through a typed client of AcknowledgeOrderTask with the workload token that holds factory.data.acknowledge, and are idempotent on the order id. ConnectSoft.AI.SoftwareFactory.DashboardService
AIFACTORY-TASK-4500 Verify projections-follow-retention-and-deletion-orders through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Feature=InboxHandlers"; a zero-match filter counts as a failure; the tests prove AC1 to AC3 of AIFACTORY-US-1128 over a two-tenant fixture with a fake of AcknowledgeOrderTask that records the receipt and can answer 403 and 404, including the redelivered order and the hold that blocks expiry. ConnectSoft.AI.SoftwareFactory.DashboardService