Skip to content

Increment I4: Trust and accountability

Generated from the architecture model

This page is rendered from model/**/*.yaml by tools/docs_model during mkdocs build. Edit the YAML, not this page; hand-written design lives on the linked service, platform and plane pages.

A design artefact, not a tracker

This backlog is generated from model/backlog/ and validated against the architecture model. No work item exists in any tracker, and no item carries a date, an estimate or an assignee (ADR-0029).

Outcome. A release publishes only with evaluation evidence, approvals follow the rules of the project, and a run recovers after a worker is killed.

Depends on: I1 · Elaboration: stories and tasks · Azure DevOps CSV of this increment

Demonstrations

  • A workflow or agent release without evaluation evidence cannot be published.
  • An approval is routed to the reviewers the rule names, and the requester can never decide it.
  • A killed worker is replaced and the run resumes from its journal.
  • A breached service level objective raises its alert and is visible on the operations views.

Services delivered in this increment

Service Role Depth Scope in this increment Repository
AgentRegistryService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.AgentRegistryService
DataEventFoundations platform-gateway complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.DataEventFoundations
EnvelopeRegistryService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService
RetentionResidencyService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.RetentionResidencyService
StoreProfileService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.StoreProfileService
ApprovalPolicyService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.ApprovalPolicyService
AlertRuleService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.AlertRuleService
DashboardService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.DashboardService
FeedbackService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.FeedbackService
QualityScoreService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.QualityScoreService
SloService service complete the whole service as modelled ConnectSoft.AI.SoftwareFactory.SloService

Stub register

No stub is declared in this increment.

Before development

Template and library work of this increment, done first: these features change the templates and shared libraries the Factory scaffolds from and builds on, and none delivers a Factory service. Every task under them carries the tag phase:before-development.

Feature Repositories State Tasks
AIFACTORY-FEAT-164 SaaS Metering template: idempotent usage, windows and the quota contract before usage export ConnectSoft.Saas.MeteringTemplate elaborated 31

Features by epic

Control Plane (PF-01)

Feature Phase State Route PBIs Tasks
AIFACTORY-FEAT-007 AgentRegistryService: complete delivery
AgentRegistryService is the service of bounded context BC-04 (Asset Publication). Publishes immutable releases of agents, skills, tools, workflows, policies and model profiles with their full dependency closure, hashes and evaluation evidence; serves release resolution to Run Control and release loading to the runtime (BC-04). This feature completes the service after its thin delivery in I1: every aggregate, operation, event and worker of the model. Repository ConnectSoft.AI.SoftwareFactory.AgentRegistryService.
development elaborated backend-developer 8 26

Data & Event Platform (PF-11)

Feature Phase State Route PBIs Tasks
AIFACTORY-FEAT-055 DataEventFoundations: gateway delivery
DataEventFoundations is the platform API gateway of bounded context BC-42 (Data & Event Foundations). The API gateway of the Data & Event platform (ADR-0028): routes public REST and internal gRPC calls to the 3 services it fronts, validates tokens and scopes, and owns no business state. Its own repository is ConnectSoft.AI.SoftwareFactory.DataEventFoundations.
development elaborated backend-developer 6 17
AIFACTORY-FEAT-056 EnvelopeRegistryService: complete delivery
EnvelopeRegistryService is the service of bounded context BC-42 (Data & Event Foundations). The registry of every integration topic (context.area.vN.event-name), its producer, consumers and payload schema versions, the port-level envelope (ADR-0014) with its mapping per messaging profile, the outbox and inbox table contracts per profile, and compatibility checks that fail a release which would break a consumer (BC-42). Its own repository is ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService.
development elaborated backend-developer 7 24
AIFACTORY-FEAT-057 RetentionResidencyService: complete delivery
RetentionResidencyService is the service of bounded context BC-42 (Data & Event Foundations). Retention rules per store class and classification (traces 30 days, audit 7 years redacted, runs and artifacts 1 year tenant-configurable), residency rules per tenant (selected region, allowed engines), deletion orders that fan out to every owning service and complete within 30 days with acknowledgements, and export orders in open formats — data governance (D31, ADR-0019) as a service (BC-42). Its own repository is ConnectSoft.AI.SoftwareFactory.RetentionResidencyService.
development elaborated backend-developer 15 81
AIFACTORY-FEAT-058 StoreProfileService: complete delivery
StoreProfileService is the service of bounded context BC-42 (Data & Event Foundations). The catalogue of store classes (operational relational, worker durable, cache, artifact, vector, graph, event transport, audit, analytics), the engine profile each class runs per topology (ADR-0006 NHibernate, ADR-0012 transport, ADR-0013 vector), and the binding of every logical store to its owning service, region and backup class (BC-42, original design §54). Its own repository is ConnectSoft.AI.SoftwareFactory.StoreProfileService.
development elaborated backend-developer 6 21

Governance (PF-12)

Feature Phase State Route PBIs Tasks
AIFACTORY-FEAT-063 ApprovalPolicyService: complete delivery
ApprovalPolicyService is the service of bounded context BC-20 (Approval Policy). The rules of human decisions — which roles may approve which action class in which scope, separation of duties (never the requester, never the author), quorum, expiry, break-glass conditions — and the routing evaluations that tell Run Control who may decide a given request and whether a given decision is valid; instances and decisions stay in BC-03 (BC-20, ADR-0015). Its own repository is ConnectSoft.AI.SoftwareFactory.ApprovalPolicyService.
development elaborated backend-developer 6 25

Observability (PF-13)

Feature Phase State Route PBIs Tasks
AIFACTORY-FEAT-067 AlertRuleService: complete delivery
AlertRuleService is the service of bounded context BC-21 (Operations). Alert rules over SLO breaches, host health, uncertain tool effects, audit findings and metric conditions, with deduplication, severity, acknowledgement and routing to the Studio, chat and email connectors and, later, incidents (BC-21). Its own repository is ConnectSoft.AI.SoftwareFactory.AlertRuleService.
development elaborated backend-developer 8 27
AIFACTORY-FEAT-068 DashboardService: complete delivery
DashboardService is the service of bounded context BC-21 (Operations). Read models for operating the Factory — per-run timelines that join Run Control truth with runtime, model, tool, context and usage events, tenant and project overviews (task success, cost, tokens, latency, tool failure, fallback and human-intervention rates) and live subscriptions for the Studio; projections are rebuildable and never the truth (BC-21, original design §51). Its own repository is ConnectSoft.AI.SoftwareFactory.DashboardService.
development elaborated backend-developer 7 24
AIFACTORY-FEAT-069 FeedbackService: complete delivery
FeedbackService is the service of bounded context BC-22 (Evaluation & Learning). Human feedback on runs and artifacts, evaluation datasets built from historical tasks and completed runs, and learning proposals (prompt, skill and context-strategy improvements, threshold changes, dataset additions) raised from feedback, drift and reinforced memory for Asset maintainers to review; nothing changes an asset without a release (BC-22, original design §32). Its own repository is ConnectSoft.AI.SoftwareFactory.FeedbackService.
development elaborated backend-developer 7 23
AIFACTORY-FEAT-070 QualityScoreService: complete delivery
QualityScoreService is the service of bounded context BC-22 (Evaluation & Learning). Evaluation as a core capability: suites per target (models, agents, skills, tools, workflows, context and memory strategies, policies) with the quality-gate metrics, evaluation runs over datasets of historical tasks, verdicts with baseline comparison, waivers, drift monitoring and the evidence records publication reads (BC-22, D32, original design §47–49). This feature completes the service after its thin delivery in I1: every aggregate, operation, event and worker of the model. Repository ConnectSoft.AI.SoftwareFactory.QualityScoreService.
development elaborated backend-developer 10 33
AIFACTORY-FEAT-071 SloService: complete delivery
SloService is the service of bounded context BC-21 (Operations). SLO definitions per tenant, workflow, agent and platform (success rate, latency, cost per task, availability, human intervention), their measurements and error budgets, and the service health rollup from every host's probes; breaches feed alert rules and the fleet (BC-21, D30). Its own repository is ConnectSoft.AI.SoftwareFactory.SloService.
development elaborated backend-developer 5 19

Ecosystem component extensions

Feature Phase State Route PBIs Tasks
AIFACTORY-FEAT-164 SaaS Metering template: idempotent usage, windows and the quota contract before usage export
CostUsageService exports usage to the SaaS Metering service and the export must be repeatable without a duplicate meter (FEAT-102); budgets are capped per period and invoices read the same measurement. The template's own gap backlog (saas-EPIC-MET) is not started: a usage record has no idempotency key, counters are flat with a boolean reset instead of windows, the quota events name Dimension where Billing expects MeterKey, and the inbound usage topic is not registered. FEAT-102 depends on this feature. Component: SaaS Metering service (extend).
before development elaborated backend-developer 7 31
AIFACTORY-FEAT-102 Metering integration for usage accounting
Usage that CostUsageService records is exported to the SaaS Metering service in its units, so that budgets and invoices use one measurement. Component: SaaS Metering service (extend).
development elaborated backend-developer 7 23

Factory assets: workflow definitions, agents and skills

Feature Phase State Route PBIs Tasks
AIFACTORY-FEAT-117 Workflow definition: Compliance check
The workflow definition compliance-check as a published asset: Checks generated code and infrastructure against the selected compliance rule sets, raises findings to audit and blocks on critical findings. It has 7 steps, names the agents privacy-compliance, security-engineer and the tools none, and is executed by WorkflowEngineService under Run Control.
development elaborated solution-architect 4 13
AIFACTORY-FEAT-118 Workflow definition: Technical debt review
The workflow definition debt-review as a published asset: Collects reviewer comments and analyzer results of a pipeline run, classifies the shortcuts as debt with owners and severity, and raises them as findings. It has 3 steps, names the agents code-reviewer, tech-lead and the tools none, and is executed by WorkflowEngineService under Run Control.
development elaborated solution-architect 3 10
AIFACTORY-FEAT-119 Workflow definition: Behaviour simulation
The workflow definition behaviour-simulation as a published asset: Runs the acceptance scenarios of a generated product end to end in the sandbox with agents, scores the results into evaluation records and stores the validated results. It has 4 steps, names the agents e2e-testing, qa-engineer and the tools shell-sandbox, and is executed by WorkflowEngineService under Run Control.
development elaborated solution-architect 3 10
AIFACTORY-FEAT-120 Workflow definition: Test plan
The workflow definition test-plan as a published asset: Builds the Test Blueprint from the service and module blueprints: scenarios from features and invariants, a coverage matrix and the quality constraints, validated against the DSL. It has 6 steps, names the agents qa-engineer, test-generator and the tools blueprint-submit-version, blueprint-validate, and is executed by WorkflowEngineService under Run Control.
development elaborated solution-architect 4 13
AIFACTORY-FEAT-121 Workflow definition: Plan a run
The workflow definition plan-run as a published asset: Turns an accepted run submission into a workflow-definition instance: classifies the intent, selects a Published definition or composes a plan validated against the DSL, and hands the first steps to the engine. It has 5 steps, names the agents factory-workflow-orchestrator and the tools knowledge-search, and is executed by WorkflowEngineService under Run Control.
development elaborated solution-architect 4 13
AIFACTORY-FEAT-148 Workflow definition: Pipeline blueprint
The workflow definition pipeline-blueprint as a published asset: Builds the Pipeline Blueprint from the Module, Test and Security Blueprints: build, test, scan and packaging stages per module with their gates, and the artefacts a release takes; deployment stays a promotion and is not a pipeline stage. Validated against the DSL. It has 5 steps, names the agents devops-engineer and the tools blueprint-submit-version, blueprint-validate, and is executed by WorkflowEngineService under Run Control.
development elaborated solution-architect 4 13

Outcome verification

Feature Phase State Route PBIs Tasks
AIFACTORY-FEAT-142 Demonstrate I4: Trust and accountability
The outcome of increment I4 is demonstrated end to end and the evidence is recorded: A release publishes only with evaluation evidence, approvals follow the rules of the project, and a run recovers after a worker is killed. Demonstrations: A workflow or agent release without evaluation evidence cannot be published. An approval is routed to the reviewers the rule names, and the requester can never decide it. A killed worker is replaced and the run resumes from its journal. A breached service level objective raises its alert and is visible on the operations views.
development elaborated qa-engineer 4 13

Requirements this increment touches

FR-02, FR-03, FR-04, FR-05, FR-06, FR-08, FR-10, FR-11, FR-12, FUX-02, FUX-03, FUX-04, FUX-05, FUX-07, FUX-08, FUX-09, FUX-10, FUX-11, FUX-12, NFR-02, NFR-04, NFR-05, NFR-06, NFR-07, NFR-10, NFR-11, NFR-13, NFR-14, NFR-15, NFR-16, NFR-17, NFR-18, XL-02, XL-03, XL-05, XL-06, XL-08, XL-09, XL-10, XL-11, XL-12, XL-13, XL-15, XL-16