Skip to content

AIFACTORY-FEAT-008: CostUsageService: complete delivery

Generated from the architecture model

This page is rendered from model/**/*.yaml by tools/docs_model during mkdocs build. Edit the YAML, not this page; hand-written design lives on the linked service, platform and plane pages.

A design artefact, not a tracker

This backlog is generated from model/backlog/ and validated against the architecture model. No work item exists in any tracker, and no item carries a date, an estimate or an assignee (ADR-0029).

Epic Control Plane (PF-01) ยท Increment I1 Generate a microservice ยท State elaborated ยท Route backend-developer ยท Build order 56 of 185 (Factory code)

CostUsageService is the service of bounded context BC-05 (Usage Accounting). Owns budgets, reservations, usage records, pricing revisions and reconciliation; reserves before dispatch, settles observed usage, keeps unknown usage uncertain and feeds meters, budgets and billing (BC-05). It is delivered complete in the first outcome by owner decision (D80), so that every run reserves before it spends and stops at its budget cap. Its own repository is ConnectSoft.AI.SoftwareFactory.CostUsageService.

Delivers CostUsageService (complete)
Stubs โ€”
Realizes adrs: ADR-0020, ADR-0014
Requirements FR-09, FUX-08, NFR-08, NFR-09, NFR-12, XL-12
Depends on โ€”
Sources model/services/control-plane/CostUsageService.yaml

Scenarios to trace (14, catalog): SCN-CostUsageService-001 โœ“, SCN-CostUsageService-002 โœ“, SCN-CostUsageService-003 โœ“, SCN-CostUsageService-014 โœ“, SCN-CostUsageService-004 โœ“, SCN-CostUsageService-005 โœ“, SCN-CostUsageService-006 โœ“, SCN-CostUsageService-007 โœ“, SCN-CostUsageService-012 โœ“, SCN-CostUsageService-008 โœ“, SCN-CostUsageService-009 โœ“, SCN-CostUsageService-010 โœ“, SCN-CostUsageService-011 โœ“, SCN-CostUsageService-013 โœ“

AIFACTORY-US-020: A run reserves before it spends and stops at its cap

behavior ยท persona workspace-admin As a workspace administrator, I want to set a budget for a project and have every run reserve before it spends, so that a run stops at its cap instead of overspending, and I can trust the usage I am shown.

This is the integration PBI of the feature: it proves on a deployed instance that the behaviours of CostUsageService work together along the life of a run, as the first outcome demands (D80). ProjectService announces a new project and the default budget of the project is opened; a Workspace admin sets its limit and its approval threshold, and a monthly budget follows the calendar month in UTC and carries the reservations that are still open at the month boundary into the new period with their estimates (AIFACTORY-US-452, D119); a Platform admin sets the budget of the tenant, and no project budget may exceed the budget of its tenant nor a tenant budget the platform limit (AIFACTORY-US-1495, D137). RunControlService accepts a run and CostUsageService reserves the estimated cost before anything is dispatched (AIFACTORY-US-453): CostUsageService/BudgetReserved lets RunControlService dispatch the first attempt, and CostUsageService/BudgetReservationDenied makes it fail an Accepted or a Held run as policy-denied. A run whose estimate is above the approval threshold gets no reservation at first: CostUsageService/SpendApprovalRequired makes RunControlService open a run-level spend approval, the run waits in WaitingForApproval, the reservation is made when the approval is granted, and a rejected or expired approval fails the run as policy-denied (D117); for a run that is Held by a frozen environment the approval is opened at once and can be decided while the run is Held, an approved one is reserved at once, and the run proceeds when the approval is granted and the environment is active (D119). That is the first half of what the title means: the cap is enforced at reservation, before the first dispatch, so a run whose estimate the budget cannot cover stops before it spends. The second half is the overrun: when the observed usage of a running run takes the committed amount of the budget above its limit, CostUsageService/BudgetExceeded is published, RunControlService records a cancellation intent for that run with the reason budget-exceeded, the run ends Cancelled with that reason and is no failed run (D123), and the next reservation against the budget is denied (AIFACTORY-US-549, D117); the event names the run whose usage crossed the limit and is published once per run, and only a named run is stopped (D119); every later run whose observed usage raises the committed amount of a budget that is above its limit is named once and stopped the same way, and the runs that stay within their estimate finish and settle (D121). A child run has no reservation of its own: nothing is reserved for it, its usage is recorded with the child run id and counts on the reservation of its parent run, and the run an overrun names may be a child run, which ends Cancelled like every stopped run and which only its parent treats as failed (AIFACTORY-US-737, D121, D123). While the run works, ModelRouterService and ToolAdapterService report what each call consumed and every receipt becomes exactly one usage record per meter, the input tokens and the output tokens of a model call on two meters, with unknown or unpriced usage kept uncertain and never refused (AIFACTORY-US-454). The usage of a model invocation that names no run, a model call outside a run, is recorded against the budget of its project without a reservation and counts in the committed amount of that budget (AIFACTORY-US-881, D123). When the run ends, the reservation is settled to the observed amount or released when the run was cancelled before dispatch (AIFACTORY-US-455); a reservation whose run never dispatched expires (AIFACTORY-US-459); estimated and uncertain records are confirmed later from late receipts, and unpriced ones are priced once a price exists (AIFACTORY-US-456); and the usage of the project and of the run is read through the gateway and the command-line client, while the Studio screens follow with StudioBff in I3 (AIFACTORY-US-457). The repository and the store come from AIFACTORY-US-450 and the hosted surface from AIFACTORY-US-451. The service is deployed as its own workload with its own store cost-usage-sql and its own messaging service name cost-usage (ADR-0028); its eight events are written to the outbox in the transaction of the change they announce and published in the run event envelope of the contracts package. A resource of another tenant is answered 404, over gRPC NOT_FOUND, and a missing scope inside the tenant of the caller 403. At this increment every usage fact is kept in cost-usage-sql. The model names the Factory Metering service as the meter store behind a typed client; that service arrives in I4 and is not required before it (D115, D117, feature component-saas-metering), so nothing here exports usage or references its client. The delivery states its other limits where they belong. The handlers of events whose producers arrive later are not built: the three retention events of RetentionResidencyService (I4), whose legal-hold and deletion-order handlers are built in this repository in I4 by feature retention-residency-service (D137), SubscriptionChanged of FactoryCommercialService (I7), and QuotaExceeded, CapacityLeaseGranted and CapacityLeaseReleased, which the complete ExecutionCapacityService publishes from I2. Four things remain inputs of the owner and are never invented here: the rule that estimates a run and its values, the limit and the period of the default budget of a new project, the platform limit no tenant budget may exceed (D137), and the unit, the factor and the prices of each meter. The values of Usage:Budgets:DefaultLimitUsd, Usage:Budgets:DefaultPeriod and Usage:Budgets:PlatformLimitUsd, the section Usage:Estimates and the pricing revisions of the meters are entries of the I1 precondition register of AIFACTORY-US-1339 that the owner supplies before the I1 run (D140); the unit and the factor of each meter stay open rows of the register of AIFACTORY-US-379. Repository: ConnectSoft.AI.SoftwareFactory.CostUsageService.

Acceptance criteria

AC1 Given a project whose budget was set through CostUsageService/SetProjectBudget and a run accepted by RunControlService whose estimate, computed by the configured estimator, fits the budget and the threshold, when the acceptance reaches CostUsageService, then a CostUsageService/Reservation is Reserved, CostUsageService/BudgetReserved is published and RunControlService dispatches the first attempt; given a second run whose estimate the budget cannot cover, then CostUsageService/BudgetReservationDenied is published and that run ends Failed with the failure class policy-denied without any attempt dispatched; given a third run whose estimate fits the budget and is above the threshold, then CostUsageService/SpendApprovalRequired is published, nothing is reserved and the run waits in WaitingForApproval, and after a reviewer who is not its requester approved the spend the reservation is Reserved, CostUsageService/BudgetReserved is published and the first attempt is dispatched (ADR-0014, XL-12).
AC2 Given a model call and a tool call of the running run report their receipts, when the receipts reach CostUsageService through CostUsageService/Usage.RecordUsage or the receipt events, then one CostUsageService/UsageRecord per receipt and meter exists, the input tokens and the output tokens of the model call on their two meters, and CostUsageService/UsageRecorded is published once per record; when the run ends, then the reservation is Settled to the observed amount and CostUsageService/ReservationSettled is published; given a running run whose recorded usage takes committedUsd of the budget above limitUsd, then CostUsageService/BudgetExceeded is published once with its runId, that run ends Cancelled with the reason budget-exceeded, a second running run of the same budget that stays within its estimate is not stopped and settles, a third whose observed usage raises committedUsd further while it is above limitUsd is named once and stopped the same way, and the next reservation against the budget is Denied (FR-09, XL-12, D119, D121).
AC3 Given a provider reports no usage for one call of the run, when the receipt is recorded and the run ends, then that record is Uncertain with no amount invented and CostUsageService/ReservationSettled carries uncertain true; when a later receipt for the same source reports the usage and the ReconciliationWorker applies it, then CostUsageService/UsageReconciled is published and the record is Reconciled, on the deployed instance and without a fixture source (FR-09, NFR-12).
AC4 Given a finished run without uncertain records, when CostUsageService/GetRunUsage and CostUsageService/GetProjectUsage are read through the gateway and through the command-line client, then the usage of the run equals the sum of its receipts and the settledUsd of its reservation, and the project figures show it under the observed state (FR-09, XL-12).
AC5 Given the acceptance suite of the repository, when it runs against the deployed instance, then each of the fourteen scenarios of model/scenarios/CostUsageService.yaml is executed by at least one test that carries its scenario id, a filter that matches no test counts as a failure, and the suite passes without a Metering service present.
Traces: services CostUsageService, RunControlService, ModelRouterService, ToolAdapterService
Traces: aggregates CostUsageService/Budget, CostUsageService/Reservation, CostUsageService/UsageRecord
Traces: apis CostUsageService/SetProjectBudget, CostUsageService/GetProjectUsage, CostUsageService/GetRunUsage, CostUsageService/Usage.Reserve, CostUsageService/Usage.RecordUsage
Traces: events CostUsageService/BudgetReserved, CostUsageService/BudgetReservationDenied, CostUsageService/ReservationSettled, CostUsageService/UsageRecorded, CostUsageService/UsageReconciled, CostUsageService/BudgetUpdated, CostUsageService/SpendApprovalRequired, CostUsageService/BudgetExceeded
Traces: descriptors usage-reservation-v1, usage-record-v1
Traces: adrs ADR-0014, ADR-0020, ADR-0028
Traces: requirements FR-09, XL-12, NFR-12
Relates: screens usage-cost, project-overview
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-452, AIFACTORY-US-453, AIFACTORY-US-454, AIFACTORY-US-455, AIFACTORY-US-456, AIFACTORY-US-457, AIFACTORY-US-459, AIFACTORY-US-549, AIFACTORY-US-737, AIFACTORY-US-881, AIFACTORY-US-1495, feature:run-control-service, feature:project-service, feature:model-router-service, feature:tool-adapter-service, feature:control-plane-api, feature:command-line-client
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, docs/architecture/adr/0020-commercial-model.md, docs/planes/control/services/cost-usage-service/index.md

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-313 Budget and usage commands in the command-line client implementation The client sets the budget of a project through SetProjectBudget and shows the usage of a project and of a run through GetProjectUsage and GetRunUsage, by meter and by the five states with uncertain usage shown as uncertain, using the typed gateway client and the trusted scope of the signed-in user. ConnectSoft.AI.SoftwareFactory.Cli
AIFACTORY-TASK-1826 Deploy CostUsageService as its own workload with its store, its messaging service name, its owner-supplied configuration and its gateway routes operations CostUsageService runs as its own workload from the manifests the template generates, with its own relational store cost-usage-sql, the messaging service name cost-usage and the transport credential read from the secret reference messaging-transport; the deployment sets the values the owner supplies for Usage:Budgets:DefaultLimitUsd, Usage:Budgets:DefaultPeriod, Usage:Budgets:PlatformLimitUsd (D137) and the section Usage:Estimates and loads the pricing revisions of the owner, and readiness answers ready only after the store is migrated, the messaging profile has started and each of the meters ModelInputTokens, ModelOutputTokens and ToolInvocations has a pricing revision valid at that moment, so a deployment without prices takes no traffic; the four REST routes are reachable through the platform gateway; no Metering service is configured, since that service arrives in I4 and is not required before it (ADR-0028, D117). ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-699 Verify run-budget-cap through the end-to-end chain of the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance" against the deployed instance with RunControlService, ModelRouterService and ToolAdapterService of the same environment; a filter that matches no test counts as a failure; the chain budget, reservation, denial at the cap, spend approval above the threshold, receipts, a running run stopped by a budget overrun, settlement, reconciliation from a late receipt and read passes and every scenario id of model/scenarios/CostUsageService.yaml is carried by at least one passing test, which proves AC1 to AC5 of AIFACTORY-US-020. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-450: CostUsageService repository: scaffolded from the base template with the Factory profile, with the Budget, Reservation and UsageRecord aggregates and the store cost-usage-sql

enabling The repository ConnectSoft.AI.SoftwareFactory.CostUsageService is created and scaffolded in I0 by AIFACTORY-TASK-238 of feature repository-and-pipeline-conventions (D140), and this feature starts from that scaffold, made with the scaffold command of ADR-0028 and its amendment on the Factory profile switch (D114): dotnet new connectsoft-base -n ConnectSoft.AI.SoftwareFactory.CostUsageService --UseFactoryProfile true, from the template package at the version the gate of feature component-base-template-factory-profile published (AIFACTORY-US-230), after the findings of feature component-base-template-findings are closed (AIFACTORY-US-235). The generated output is committed unchanged and the repository records the version of the template package it was generated from. What the profile generates, that is the Orleans messaging profile with an outbox and an inbox of the service, the propagation of the trusted tenant and project scope, the role policies, the REST and gRPC hosts from a service descriptor, health and telemetry, is the work of the template feature; it is consumed by that version pin and is neither rewritten nor forked here, and the run event envelope and the descriptors usage-reservation-v1 and usage-record-v1 come from the contracts package by a version pin. On that scaffold this PBI adds the domain and the store. The Budget holds a limit for a project in a period (monthly or total), the start of the open period, the amount committed, which the model describes as reserved plus settled in the period, and the approval threshold. Its invariants are those of the model: committedUsd never exceeds limitUsd at reservation, where a reservation that would exceed it is Denied; only observed usage, recorded or settled, may take it above limitUsd, which publishes BudgetExceeded, and after such an overrun the next reservation of the budget is Denied (AIFACTORY-US-549); a monthly budget follows the calendar month in UTC, with a rollover that opens the new period and keeps the closed one (AIFACTORY-US-452); and the approval threshold of a project budget is never higher than the platform threshold. The Reservation holds the estimate for one run with its state, an optional approvalId, expiresAt and the time it was settled; it is created in Reserved or recorded as Denied, has exactly three transitions, Reserved to Settled (RunTerminal), Reserved to Released (RunCancelledBeforeDispatch) and Reserved to Expired (ExpiryWithoutRun), and at most one exists per run; for an estimate above the spend threshold none is written until the spend approval is granted, and it is then created in Reserved with its approvalId (AIFACTORY-US-453). The UsageRecord holds one metered quantity on one of the eight meters, the input tokens and the output tokens of a model call being two of them, with its dimensions, its status (Estimated, Reserved, Observed, Reconciled, Uncertain), the pricing revision it was priced with, and the receipt it came from; a record is created from a receipt exactly once per meter, uncertain usage is never zeroed, and a record whose quantity is unknown or for which no pricing revision is valid is Uncertain and holds no quantity or no revision, and no cost, until it is priced or reconciled. The store cost-usage-sql is owned by this service alone. This PBI migrates four of the six tables of the model, Budgets, Reservations, UsageRecords and PricingRevisions, with the unique index over tenant, project and period, the unique index over the run, the unique index over meter and source receipt and the index over tenant, project and time, and with the nullable quantity and pricing revision of a usage record; UsageRecords is retained seven years (D31) and partitioned by tenant and month as the service page states; PricingRevisions is platform-wide and carries no tenant filter, the three other tables carry it in every mapping. The two other tables are migrated where their behaviour is delivered: BudgetPeriods, the closed periods of a monthly budget, by AIFACTORY-US-452, and LateReceipts, the kept late receipts, together with the note on a reservation that an attempt outcome of its run was recorded, by AIFACTORY-US-454. The table ChildRuns, in which a child run is noted under the reservation its usage counts on, is migrated by AIFACTORY-US-737 (D121). The model names the Factory Metering service as the meter store behind a typed client and lists the metering library of the SaaS extensions; that service arrives in I4 and is not required before it (D115, D117), so this PBI references neither the client nor a meter store and binds no library as one.

Acceptance criteria

AC1 Given the repository created and scaffolded in I0 by AIFACTORY-TASK-238 of feature repository-and-pipeline-conventions with dotnet new connectsoft-base -n ConnectSoft.AI.SoftwareFactory.CostUsageService --UseFactoryProfile true from the base template package at the version the gate of feature component-base-template-factory-profile published, when its pipeline runs, then the committed output equals the generated output, the solution builds in Release, the generated tests pass, a container image is published and the repository records the template version (ADR-0028, model/template-parameters/connectsoft-base.yaml).
AC2 Given the CostUsageService/Budget, CostUsageService/Reservation and CostUsageService/UsageRecord aggregates, when their unit tests run, then a reservation that would take committedUsd above limitUsd is Denied and leaves the budget unchanged, a threshold above the platform threshold is refused, a second reservation for one run and a second record for one meter and source receipt are refused, the reservation accepts only the transitions Reserved to Settled, Reserved to Released and Reserved to Expired, and a transition out of Settled, Released, Expired or Denied is refused, as model/services/control-plane/CostUsageService.yaml states.
AC3 Given cost-usage-sql created by the migrations of the repository on an isolated test database, when the mapping tests run, then the tables Budgets, Reservations, UsageRecords and PricingRevisions exist with their keys and indexes, a usage record is stored and read back without a quantity and without a pricing revision, a second reservation for one run and a second usage record for one meter and source receipt each violate their unique index, a query made for one tenant never returns a budget, reservation or usage record of another, and a pricing revision is readable for every tenant (ADR-0015, NFR-01).
Traces: services CostUsageService
Traces: aggregates CostUsageService/Budget, CostUsageService/Reservation, CostUsageService/UsageRecord
Traces: descriptors usage-reservation-v1, usage-record-v1
Traces: adrs ADR-0015, ADR-0020, ADR-0028
Traces: requirements NFR-01, NFR-12
Depends on feature:repository-and-pipeline-conventions, feature:component-base-template-factory-profile, feature:component-base-template-findings, feature:contracts-package
Sources model/services/control-plane/CostUsageService.yaml, model/template-parameters/connectsoft-base.yaml, model/stores.yaml, docs/architecture/adr/0028-one-repository-and-deployable-per-service.md, docs/planes/control/services/cost-usage-service/index.md#domain-model, docs/reference/schemas/usage-record.schema.json

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-239 Domain model: Budget, Reservation, UsageRecord and PricingRevision implementation The three aggregates and the PricingRevision entity enforce the invariants of the model as it words them; a reservation that would take committedUsd above limitUsd is Denied; the Reservation is created in Reserved or recorded as Denied and accepts only RunTerminal, RunCancelledBeforeDispatch and ExpiryWithoutRun out of Reserved; unit tests cover every invariant, each of the three transitions and the refusal of every other one. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-240 Persistence in cost-usage-sql: Budgets, Reservations, UsageRecords, PricingRevisions migration Mappings and migrations for the tables Budgets, Reservations, UsageRecords and PricingRevisions with the columns the model lists, among them the start of the open period of a budget, the settlement time of a reservation and the nullable quantity and pricing revision of a usage record, and except the attempt outcome note of a reservation, which AIFACTORY-TASK-1849 adds; with the unique indexes UX_Budgets_Tenant_Project_Period, UX_Reservations_Run and UX_Usage_Meter_Receipt, the index IX_Usage_Tenant_Project_OccurredAt, the row version of Budgets, the partitioning of UsageRecords by tenant and month and the tenant filter in the mappings of the three tenant-scoped tables; PricingRevisions is platform-wide; BudgetPeriods and LateReceipts are migrated by AIFACTORY-TASK-2201 and AIFACTORY-TASK-1849, and ChildRuns by AIFACTORY-TASK-2974 (D121); mapping tests run against an isolated test database, never a shared one. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1827 Verify the scaffold, the aggregates and the store of CostUsageService verification Run dotnet build -c Release and dotnet test -c Release --no-build in the repository pipeline; the generated tests, the unit tests of the three aggregates and the mapping tests on an isolated database pass, a diff of the committed tree against a fresh generation at the recorded template version shows only the files this PBI added, and the solution references no metering client, which proves AC1 to AC3 of AIFACTORY-US-450 (ADR-0028). ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-451: CostUsageService host: four REST routes and two gRPC methods behind scope policies, the cost-usage messaging profile, tenancy and telemetry

enabling This PBI turns the scaffold of AIFACTORY-US-450 into a host on which the behaviour PBIs of the feature land. From the service descriptor the host exposes the four REST operations of the model for the platform gateway (GW-03), SetTenantBudget among them (D137), and the internal gRPC methods Usage.Reserve and Usage.RecordUsage, with their contract documents, an OpenAPI document and a proto. Tokens are validated through the resource-server library the profile wires, and each route requires its scope: factory.usage.read for the two reads and factory.usage.admin for the two budget routes. The gRPC methods require a workload identity whose token holds the service scope that names CostUsageService: RunControlService for both methods, ModelRouterService and ToolAdapterService for Usage.RecordUsage. A workload token carries no tenant claim, and the service reads the tenant and project a workload forwards only because the token holds that service scope (ADR-0015 amendment of D114). The tenant of a user call comes from the token. Two routes address a project by projectId, one addresses a run and SetTenantBudget addresses the tenant of the token; a route that addresses a project or a run admits the call only when the role claims of the token name that project, or the project of that run, or carry a tenant-wide role. A resource of another tenant is invisible under the tenant row filter and is answered 404, over gRPC NOT_FOUND, while a missing scope inside the tenant of the caller is answered 403 (D113); the filter is wired here and the 404 is proven by the behaviour PBIs. Every modelled refusal code has a problem response. The messaging profile of the service is named cost-usage: the outbox writer enlists in the unit of work of a command, the dispatcher publishes the eight events of the model in the run event envelope of the contracts package with their routes declared in the OrleansMessaging section, and the inbox subscribes, with deduplication on the event id, to the consumed events whose publication is delivered up to this increment: RunAccepted, RunCompleted, RunFailed, RunCancelled, AttemptOutcomeRecorded and ApprovalDecided of RunControlService, ProjectCreated of ProjectService, ModelInvoked of ModelRouterService and ToolInvoked of ToolAdapterService. What each handler does is delivered by AIFACTORY-US-452 to AIFACTORY-US-455. The seven options of the model are bound and validated at start: Usage:Thresholds:PlatformApprovalThresholdUsd, Usage:Reservations:ExpiryMinutes and OrleansMessaging:ServiceName, which ship with a default, and Usage:Budgets:DefaultLimitUsd, Usage:Budgets:DefaultPeriod, Usage:Budgets:PlatformLimitUsd and the section Usage:Estimates, which are inputs of the owner without a shipped value, so that a start without one of them is refused with the key named (D117, D137). The transport credential is read from the secret reference messaging-transport, and the metrics connectsoft.factory.usage.cost_usd, connectsoft.factory.usage.uncertain_records and connectsoft.factory.usage.reservations_denied are registered with their dimensions. Readiness reports the store, the messaging profile and the prices: it is not ready while one of the meters the receipts of this increment feed, ModelInputTokens, ModelOutputTokens and ToolInvocations, has no pricing revision valid at that moment, so a deployment without prices takes no traffic; a receipt that still finds no valid revision on a ready instance is stored as Uncertain and priced later, and is never refused (AIFACTORY-US-454). The delivery states its limits. The handlers of RetentionRuleChanged, DeletionOrderIssued and LegalHoldChanged (RetentionResidencyService, first delivered in I4, the deletion-order and legal-hold handlers being built in this repository by feature retention-residency-service, D137) and of SubscriptionChanged (FactoryCommercialService, I7) are not part of it. Neither are the handlers of QuotaExceeded, CapacityLeaseGranted and CapacityLeaseReleased: the thin ExecutionCapacityService of I1 publishes none of the three, and their publication is delivered with its complete delivery in I2. Readiness does not report a Metering service, which arrives in I4 and is not required before it (D115, D117).

Acceptance criteria

AC1 Given the hosted service, when each of the four REST operations of model/services/control-plane/CostUsageService.yaml is called without a token and with a token that lacks the scope of the route, then the answers are 401 and 403 with a problem response; when one of the two operations that address a projectId is called with a token whose role claims do not admit that project, or the run operation for a run whose project they do not admit, then the answer is 403; a gRPC call without a workload identity or without the service scope is refused with Unauthenticated or PermissionDenied, and ModelRouterService and ToolAdapterService are admitted to Usage.RecordUsage only; no state changes (ADR-0015).
AC2 Given a handler that changes a CostUsageService/Reservation and publishes one of the eight declared events, when its unit of work commits, then one outbox record in run-event-envelope-v1 with event id, tenant id, project id, run id where the event has one, correlation id and payload is committed in the same transaction; when the unit of work rolls back, neither the change nor the record exists; a publish of an event the OrleansMessaging section does not declare throws (ADR-0012, ADR-0014).
AC3 Given the same event id delivered twice to the inbox of the cost-usage messaging profile, when it is dispatched, then the handler runs once and the second delivery has no effect; a handler that keeps failing is retried with backoff and never applied twice (ADR-0012).
AC4 Given the service starts without any Metering service in its environment, when liveness and readiness are probed, then readiness answers ready only after cost-usage-sql is migrated, the messaging profile has started and each of the meters ModelInputTokens, ModelOutputTokens and ToolInvocations has a pricing revision valid at that moment, and it names what is missing while it is unready; the seven options of model/services/control-plane/CostUsageService.yaml are bound and a missing or invalid value, among them an absent default budget key, an absent platform limit or an absent section Usage:Estimates, stops the start with the key named; and traces and logs carry the tenant, project and run scope of the call (ADR-0015).
Traces: services CostUsageService
Traces: aggregates CostUsageService/Reservation
Traces: descriptors run-event-envelope-v1, usage-reservation-v1, usage-record-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0015, ADR-0028
Traces: requirements NFR-01, NFR-12
Depends on AIFACTORY-US-450, feature:component-base-template-factory-profile, feature:contracts-package
Sources model/services/control-plane/CostUsageService.yaml, model/gateways.yaml, docs/architecture/adr/0015-factory-tenancy-under-canonical-adr-0100.md, docs/architecture/adr/0014-run-and-lifecycle-envelopes.md, docs/planes/control/services/cost-usage-service/index.md#apis, docs/planes/control/services/cost-usage-service/index.md#security

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-241 API hosting skeleton: REST and gRPC hosts, scope enforcement, problem responses and contract documents for the 4 REST operations and 2 gRPC methods implementation The four REST routes and the gRPC methods Usage.Reserve and Usage.RecordUsage are hosted from the service descriptor with token validation, scope enforcement per route, the workload identity and service scope check per gRPC method (RunControlService for both, ModelRouterService and ToolAdapterService for Usage.RecordUsage), problem responses for every modelled refusal code, and the OpenAPI and proto documents; which fields of usage-reservation-v1 and usage-record-v1 a caller sends and which the service assigns is fixed by the contract tasks AIFACTORY-TASK-1846 and AIFACTORY-TASK-1848, and the behaviour of each operation is delivered by AIFACTORY-US-452 to AIFACTORY-US-457 and AIFACTORY-US-1495, not here. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-242 Messaging skeleton: outbox and inbox wiring of the cost-usage profile for 8 published and 9 consumed events implementation The outbox writer enlists in the unit of work and the dispatcher of the cost-usage messaging profile publishes BudgetReserved, BudgetReservationDenied, SpendApprovalRequired, ReservationSettled, UsageRecorded, UsageReconciled, BudgetUpdated and BudgetExceeded in the run event envelope of the pinned contracts package, each declared with its route in the OrleansMessaging section; the inbox registers RunAccepted, RunCompleted, RunFailed, RunCancelled, AttemptOutcomeRecorded, ApprovalDecided, ProjectCreated, ModelInvoked and ToolInvoked with idempotency on the event id; the retention events, SubscriptionChanged, QuotaExceeded, CapacityLeaseGranted and CapacityLeaseReleased are not subscribed in this delivery; what each handler does is delivered by the behaviour PBIs of this feature. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-244 Security, tenancy and telemetry of CostUsageService implementation The two scopes of the model are enforced, a route that addresses a project or a run admits a call only when the role claims of the token admit that project, every query of a tenant-scoped table is tenant-filtered, the transport credential comes from the secret reference messaging-transport, the seven options of the model are bound and validated at start, the two default budget keys, the platform limit Usage:Budgets:PlatformLimitUsd and the section Usage:Estimates having no shipped value, readiness reports the store, the messaging profile and a valid pricing revision for the meters ModelInputTokens, ModelOutputTokens and ToolInvocations and no Metering service, and the metrics connectsoft.factory.usage.cost_usd, connectsoft.factory.usage.uncertain_records and connectsoft.factory.usage.reservations_denied are registered with their dimensions under the observability conventions; the 404 that follows from the tenant filter is proven by the behaviour PBIs. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1828 Verify the hosted surface of CostUsageService with a two-tenant fixture verification Run dotnet test -c Release --no-build --filter "Category=Acceptance&Area=Hosting"; a filter that matches no test counts as a failure; the tests call every route without a token and without its scope, call the project routes and the run route with a token that does not admit the project, call both gRPC methods with each workload identity, commit and roll back a publishing handler, deliver one event id twice, and probe readiness with the store stopped and with a meter that has no valid pricing revision, which proves AC1 to AC4 of AIFACTORY-US-451 (ADR-0015, ADR-0012). ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-452: Open the default budget of a new project, set its limit and approval threshold, announce every change, and roll a monthly budget over at the calendar month in UTC

behavior ยท persona workspace-admin As a workspace admin, I want every project to have a budget from the moment it exists, and to set its limit and the amount above which a run needs approval, so that the runs of my project cannot commit more than I allowed.

A budget is the spending limit of a project with the amount already committed and the spend-approval threshold. At this increment a project has exactly one budget. It comes to exist without anyone acting: when ProjectService announces a new project through ProjectService/ProjectCreated, the inbox handler OpenProjectBudget creates the default Budget of the project, idempotently on the event id and on the rule that a project has one budget, with committedUsd zero, the approval threshold set to Usage:Thresholds:PlatformApprovalThresholdUsd and periodStart set to the first instant of the current calendar month in UTC for a monthly budget and to the instant of its creation for a total one, and writes BudgetUpdated in the same transaction. The limit and the period of that default budget are read from the two options of the model Usage:Budgets:DefaultLimitUsd and Usage:Budgets:DefaultPeriod (monthly or total). Their values are an input of the owner: none is shipped, a start without them is refused with the key named, and the deployment of AIFACTORY-TASK-1826 sets the values the owner supplies. A Workspace admin whose scope carries factory.usage.admin then sets the budget through CostUsageService/SetProjectBudget, a PUT whose body states the period (monthly or total), limitUsd and approvalThresholdUsd. The budget is addressed by the projectId of the path inside the trusted tenant, never by a tenant of the body (ADR-0015). The PUT changes the one budget of the project: a body that states another period than the stored one replaces the period of that budget and creates no second budget, so Usage.Reserve always reserves against the same row; a change to monthly sets periodStart to the first day of the current calendar month in UTC. The stored budget and the BudgetUpdated outbox record with budgetId, tenantId, projectId and limitUsd are committed in one transaction and the answer is 200 with the budget; a PUT that states the stored values changes nothing and publishes nothing. A new limit and a new threshold apply to reservations made from then on; reservations already Reserved keep their amount. The period is stored and returned. A monthly budget follows the calendar month in UTC (D117). The BudgetRolloverWorker runs at the first instant of every month, two tenants in parallel, with five attempts and exponential backoff from 60 seconds, over the monthly budgets whose periodStart lies before the first day of the current calendar month in UTC. For each of them it writes the closed period to the table BudgetPeriods, with its start, its end, its limitUsd, the committedUsd it closed with and the time of the closing, sets periodStart to the first day of the new month and resets committedUsd, all in one transaction and idempotent on budgetId plus periodStart, so a repeated trigger closes a period once and a worker that missed its trigger still opens the current month on its next run. The closed period is kept for reports and is never changed again, a total budget is never rolled over, and the worker publishes no event, as the model gives it none. Reservations that are still Reserved at the month boundary are carried into the new period with their estimates (D119): each stays Reserved with its estimate, and the reset leaves committedUsd of the new period at the sum of those estimates, and at zero when there is none. A reservation settled after the rollover is then corrected in the new period as always (AIFACTORY-US-455), and the settledAt of a reservation tells the period its settlement belongs to. The route answers 409 for the three requests that would break an invariant of the Budget: a threshold above the platform threshold, which a project may only lower (D21), a limit below the amount already committed, and a limit above the limit of the budget of its tenant, or above Usage:Budgets:PlatformLimitUsd when the tenant has no budget, because the owner decided that a project budget never exceeds the budget of its tenant (D137, AIFACTORY-US-1495); nothing is changed by a refused PUT. Who may call is decided from the token: the route needs the scope, and the role claims of the token must name the caller Workspace admin of the project or Platform admin. A caller without the scope, or a member of the tenant who by those claims is neither, is refused with 403; a project of another tenant is answered 404, the same answer as for a project without a budget in the tenant of the caller (ADR-0015 amendment of D113). BudgetUpdated is published on usage-accounting.budgets.v1.budget-updated in the run event envelope; StudioBff refreshes its budget view and AuditService records it as evidence in their own PBIs. The tenant-wide budget without a project is set through CostUsageService/SetTenantBudget and delivered by AIFACTORY-US-1495, which also holds the default budget of a new project to the ceiling of its tenant. The handler ApplyEditionLimits, which adjusts budget ceilings to the edition of a subscription, is not part of this delivery, because SubscriptionChanged is published by FactoryCommercialService from I7.

Acceptance criteria

AC1 Given ProjectService/ProjectCreated for a new project of a tenant, when the inbox handler OpenProjectBudget runs, then one CostUsageService/Budget exists for tenant and project with the period of Usage:Budgets:DefaultPeriod, the limit of Usage:Budgets:DefaultLimitUsd, committedUsd zero and the approval threshold of Usage:Thresholds:PlatformApprovalThresholdUsd, the CostUsageService/BudgetUpdated outbox record is written in the same transaction, and a redelivery of the event creates no second budget and no second record (SCN-CostUsageService-011).
AC2 Given a Workspace admin whose scope carries factory.usage.admin and the budget of a project of the trusted tenant, when CostUsageService/SetProjectBudget states a limit above the committed amount and a threshold not above the platform threshold, then the response is 200 with the budget, the budget and the CostUsageService/BudgetUpdated outbox record are committed in one transaction, and the same PUT repeated changes nothing and writes no record; when the PUT states another period than the stored one, then the period of that budget is replaced and the project still has exactly one budget (SCN-CostUsageService-003).
AC3 Given a threshold above Usage:Thresholds:PlatformApprovalThresholdUsd, a limit below the committedUsd of the budget, a limit above the limit of the budget of the tenant or, for a tenant without a budget, above Usage:Budgets:PlatformLimitUsd (D137), a caller without factory.usage.admin or who by the role claims of the token is neither Workspace admin of the project nor Platform admin, or a project of another tenant, when CostUsageService/SetProjectBudget is called, then the answer is 409, 409, 409, 403 or 404 respectively with a problem response, the CostUsageService/Budget is unchanged and nothing is published (SCN-CostUsageService-003, ADR-0015).
AC4 Given a budget was opened or changed, when the outbox dispatcher of the cost-usage messaging profile publishes CostUsageService/BudgetUpdated on usage-accounting.budgets.v1.budget-updated, then the event is sent in run-event-envelope-v1 with budgetId, tenantId, projectId and limitUsd, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-CostUsageService-011, ADR-0012).
AC5 Given a monthly CostUsageService/Budget whose periodStart lies in the previous calendar month in UTC, with settled usage and one reservation that is still Reserved, and a total budget, when the BudgetRolloverWorker runs at the first instant of the month, then the table BudgetPeriods holds one row for the closed month with its start, its end, its limitUsd, the committedUsd it closed with and the time of the closing, periodStart of the budget is the first day of the new month, the reservation is still Reserved with its estimate and is carried into the new period, so committedUsd holds that estimate only, the total budget is unchanged, nothing is published, and a second trigger for the same budgetId and periodStart changes nothing, as the worker and the rollover invariants of the Budget and of the CostUsageService/Reservation in model/services/control-plane/CostUsageService.yaml state (XL-12, D119).
Traces: services CostUsageService, ProjectService
Traces: aggregates CostUsageService/Budget, CostUsageService/Reservation
Traces: apis CostUsageService/SetProjectBudget
Traces: events CostUsageService/BudgetUpdated, ProjectService/ProjectCreated
Traces: descriptors run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0015, ADR-0020
Traces: requirements FR-09, XL-12, NFR-12
Traces: scenarios SCN-CostUsageService-003, SCN-CostUsageService-011
Relates: screens project-settings, usage-cost
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, feature:project-service
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, model/services/control-plane/ProjectService.yaml, docs/planes/control/services/cost-usage-service/index.md#context-and-boundaries

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-1829 Contract of the budget body and the two configuration keys of the default budget contract The OpenAPI document of the service holds the budget with period, periodStart, limitUsd, committedUsd and approvalThresholdUsd as model/services/control-plane/CostUsageService.yaml names them; the options Usage:Budgets:DefaultLimitUsd (decimal) and Usage:Budgets:DefaultPeriod (monthly or total) of the model are documented on the options page of the repository without a shipped value and validated at start, where a missing or invalid value stops the start with the key named. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1847 Confirm the two default budget keys in the options of the CostUsageService model and on its generated service page documentation model/services/control-plane/CostUsageService.yaml lists Usage:Budgets:DefaultLimitUsd and Usage:Budgets:DefaultPeriod under options with their type, no default and a summary that their values are an owner input, as it does since D117; the generated options section of the service page shows both keys, and python -I tools/docs_model validate passes. ConnectSoft.AI.SoftwareFactory.Documentation
AIFACTORY-TASK-1830 OpenProjectBudget and SetProjectBudget with BudgetUpdated in the transaction of the change implementation OpenProjectBudget creates the one Budget of a project on ProjectCreated, idempotently on the event id and on the rule that a project has one budget, with the limit and the period of the two default budget keys, the threshold of Usage:Thresholds:PlatformApprovalThresholdUsd and periodStart at the first instant of the current calendar month in UTC for a monthly budget and at its creation for a total one; SetProjectBudget updates period, limit and threshold of that budget in the trusted tenant, replacing its period when the body states another one, setting periodStart to the first day of the current calendar month in UTC on a change to monthly and never creating a second budget; both write the BudgetUpdated outbox record in the same transaction; SetProjectBudget answers 200, changes nothing for the stored values, 409 for a threshold above the platform threshold, a limit below committedUsd or a limit above the limit of the tenant budget, or above Usage:Budgets:PlatformLimitUsd for a tenant without one (D137), 403 for a missing scope or a caller who by the role claims of the token is neither Workspace admin of the project nor Platform admin, and 404 for a foreign project or one without a budget. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2201 Migration of the closed periods of a monthly budget migration A migration of cost-usage-sql adds the tenant-filtered table BudgetPeriods of the model, keyed by budget and period start, with the end of the period, its limit, the committed amount it closed with and the time of the closing; mapping tests on an isolated test database prove the table, its key, its reference to Budgets and its tenant filter. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2202 BudgetRolloverWorker opening the new period of a monthly budget and keeping the closed one implementation The worker runs on the schedule of the model at the first instant of every month over the monthly budgets whose periodStart lies before the first day of the current calendar month in UTC, partitioned by tenant with two partitions in parallel and five attempts with exponential backoff from 60 seconds; under the row version of the Budget it writes the closed period with its start, end, limit, committed amount and closing time to BudgetPeriods, sets periodStart to the first day of the new month and resets committedUsd to the sum of the estimates of the reservations of the budget that are still Reserved, which stay Reserved and are carried into the new period with their estimates (D119), in one transaction and idempotent on budgetId plus periodStart; a total budget is skipped, no event is published, a duplicate trigger has no second effect and a run after a missed trigger opens the current month. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1831 Verify set-a-project-budget through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-CostUsageService-003\|Scenario=SCN-CostUsageService-011)"; a filter that matches no test counts as a failure; the tests prove AC1 to AC5 of AIFACTORY-US-452 with a fixture publisher of ProjectCreated and a controlled clock, including the redelivered event, the unchanged repeat, a PUT that replaces the period, the 403, 404 and 409 refusals, the 409 of a limit above the tenant limit and above the platform limit among them, the publication and payload of BudgetUpdated, and the rollover of a monthly budget across a month boundary in UTC with the closed period kept in BudgetPeriods, a Reserved reservation carried into the new period, a total budget left alone and a repeated trigger; consumer handling is proven by the suites of the consumers. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-1495: Set the budget of a tenant under the platform limit and hold every project budget of the tenant to it

behavior ยท persona platform-admin As a platform admin, I want to set one budget for my whole tenant that no project budget may exceed, so that the projects of the tenant together stay inside what the tenant may spend, and the tenant inside what the platform allows.

The owner decided that a project budget may not exceed the budget of its tenant and that a tenant budget may not exceed the budget of the platform, and that CostUsageService enforces both (D137). The platform limit is the option Usage:Budgets:PlatformLimitUsd, an input of the owner without a shipped value, so a start without it is refused with the key named. The ceiling of a tenant budget is the lower of that limit and the edition limit of the subscription of the tenant (D139); the edition limit is kept by ApplyEditionLimits from SubscriptionChanged of FactoryCommercialService, which arrives in I7, so up to then the ceiling is the platform limit, and this delivery proves that one. A caller whose scope carries factory.usage.admin and whose role claims name the Platform admin of the tenant sets the tenant-wide budget through CostUsageService/SetTenantBudget, a PUT on /factory/budget whose body states the period (monthly or total) and limitUsd. The tenant is the one the token names and never one of the body (ADR-0015); the route addresses no project and therefore answers no 404. The PUT creates the one Budget of the tenant without a project when none exists and changes it otherwise, never a second one; a monthly tenant budget has its periodStart at the first day of the current calendar month in UTC, as for a project budget. The budget and the BudgetUpdated outbox record with budgetId, tenantId, limitUsd and committedUsd and without projectId are committed in one transaction and the answer is 200 with the budget; a PUT that states the stored values changes nothing and publishes nothing. The route answers 409 and changes nothing for a limit above Usage:Budgets:PlatformLimitUsd and for a limit below the limit of a project budget of the tenant, so lowering the tenant limit never leaves a project above it; the problem response names the rule that was broken. The same ceiling holds on the project side: CostUsageService/SetProjectBudget answers 409 for a project limit above the limit of the tenant budget (AIFACTORY-US-452), and a project budget of a tenant without a tenant budget is held to the platform limit. Both checks read the other budget in the transaction of the change under its row version, so two concurrent changes, one to the tenant budget and one to a project budget, never leave a project limit above the limit of its tenant. As a design choice of this PBI the default budget that OpenProjectBudget opens for a new project takes the smaller of Usage:Budgets:DefaultLimitUsd and the ceiling of the tenant, its tenant budget or, without one, the platform limit, so a default never breaks the rule. The tenant budget is a ceiling of the limits of the project budgets: reservations and usage records count on the budget of their project, as the model gives a reservation one budget, so the committed amount of the tenant budget stays zero; that reading is reported to the owner. A caller without the scope, or with the scope and without the Platform admin role, such as the Workspace admin of a project, is refused with 403. BudgetUpdated is published on usage-accounting.budgets.v1.budget-updated in the run event envelope; StudioBff and AuditService handle it in their own PBIs.

Acceptance criteria

AC1 Given a Platform admin whose scope carries factory.usage.admin and a tenant without a budget, when CostUsageService/SetTenantBudget states a period and a limit at or below Usage:Budgets:PlatformLimitUsd and at or above the limit of every project budget of the tenant, then the response is 200 with the budget, one CostUsageService/Budget of the tenant without a project exists, the CostUsageService/BudgetUpdated outbox record without projectId is written in the same transaction, a second PUT with other values changes that budget and creates no second one, and the same PUT repeated changes nothing and writes no record (SCN-CostUsageService-014, D137).
AC2 Given a limit above Usage:Budgets:PlatformLimitUsd, or a limit below the limit of a project budget of the tenant, when CostUsageService/SetTenantBudget is called, then the answer is 409 with a problem response that names the rule, the tenant budget is unchanged and nothing is published; given a caller without factory.usage.admin, or with it and without the Platform admin role, then the answer is 403 and nothing changes (SCN-CostUsageService-014, ADR-0015).
AC3 Given a tenant budget, when CostUsageService/SetProjectBudget states a project limit above its limit, then the answer is 409 and the project budget is unchanged; given a tenant without a budget, when it states a project limit above Usage:Budgets:PlatformLimitUsd, then the answer is 409; given a tenant budget change and a project budget change of the same tenant that run concurrently, when both have committed or been refused, then no project limit is above the tenant limit (SCN-CostUsageService-003, D137).
AC4 Given a tenant budget whose limit is below Usage:Budgets:DefaultLimitUsd, when ProjectService/ProjectCreated announces a new project of the tenant and OpenProjectBudget opens its default budget, then the limit of that budget is the limit of the tenant budget; given a tenant without a budget and a default above the platform limit, then the limit is the platform limit (SCN-CostUsageService-011, D137).
AC5 Given the service starts without Usage:Budgets:PlatformLimitUsd, when the options are validated, then the start is refused with the key named; given a committed tenant budget change, when the dispatcher of the cost-usage messaging profile publishes CostUsageService/BudgetUpdated, then it is sent in run-event-envelope-v1 with budgetId, tenantId, limitUsd and committedUsd and without projectId, and a transport outage is retried with backoff so it is neither lost nor duplicated under its event id (SCN-CostUsageService-014, ADR-0012).
Traces: services CostUsageService, ProjectService
Traces: aggregates CostUsageService/Budget
Traces: apis CostUsageService/SetTenantBudget, CostUsageService/SetProjectBudget
Traces: events CostUsageService/BudgetUpdated, ProjectService/ProjectCreated
Traces: descriptors run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0015, ADR-0020
Traces: requirements FR-09, NFR-12
Traces: scenarios SCN-CostUsageService-003, SCN-CostUsageService-011, SCN-CostUsageService-014
Relates: screens usage-cost
Relates: nfrs NFR-12
Depends on AIFACTORY-US-451, AIFACTORY-US-452, feature:project-service
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, docs/planes/control/services/cost-usage-service/index.md#apis

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-6076 Contract of SetTenantBudget and of the platform limit option contract The OpenAPI document of the service holds PUT /factory/budget under factory.usage.admin with the side-effect class shared-write, natural idempotency, a body with period and limitUsd and the codes 200, 403 and 409, and the options page of the repository documents Usage:Budgets:PlatformLimitUsd (decimal) without a shipped value, validated at start so that a missing or invalid value stops the start with the key named (D137). ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-6077 SetTenantBudget and the ceiling checks of the tenant and the project budget implementation SetTenantBudget admits a caller with factory.usage.admin and the Platform admin role, creates or changes the one Budget of the tenant of the token without a project, sets periodStart as for a project budget, writes BudgetUpdated without projectId in the same transaction, changes nothing on a repeat and answers 409 for a limit above Usage:Budgets:PlatformLimitUsd or below the limit of a project budget of the tenant and 403 otherwise as AIFACTORY-US-1495 states; SetProjectBudget answers 409 for a limit above the tenant limit, or above the platform limit for a tenant without a budget; each check reads the other budget in the transaction of the change under its row version, so concurrent changes never leave a project limit above its tenant limit (D137). ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-6078 Default budget of a new project held to the ceiling of its tenant implementation OpenProjectBudget opens the default budget of a new project with the smaller of Usage:Budgets:DefaultLimitUsd and the limit of the tenant budget, or the platform limit for a tenant without one, so that a default budget never exceeds the ceiling of its tenant (D137). ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-6079 Verify set-the-tenant-budget-under-the-platform-limit through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Feature=TenantBudget&(Scenario=SCN-CostUsageService-014\|Scenario=SCN-CostUsageService-003\|Scenario=SCN-CostUsageService-011)"; a filter that matches no test counts as a failure; the tests prove AC1 to AC5 of AIFACTORY-US-1495: the tenant budget created and changed once with BudgetUpdated without projectId, the repeat, the 409 above the platform limit and below a project limit, the 403 of a Workspace admin, the 409 of a project limit above the tenant limit and above the platform limit, two concurrent changes, the default budget held to the ceiling and the start refused without the platform limit. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-453: Reserve the estimated cost of an accepted run before dispatch, deny it at the budget limit, and above the approval threshold announce the required spend approval and reserve once it is granted

behavior Every run reserves before it spends, except a child run, which has no reservation of its own and spends on the reservation of its parent run (D121). When RunControlService announces a run through RunControlService/RunAccepted, the inbox handler ReserveForRun estimates the cost of the run and reserves it against the budget of the project. For a RunAccepted that carries parentRunId, which marks a child run, the handler estimates and reserves nothing and publishes none of the three events of this PBI; it notes the child run under the reservation of the parent run, which AIFACTORY-US-737 delivers, and everything below concerns a run without a parent. The same command is exposed as the internal gRPC method CostUsageService/Usage.Reserve, whose request and reply are usage-reservation-v1; at this increment the event is the path that runs, since no PBI of RunControlService calls the method. Both paths are one command, idempotent on runId: the unique index over the run admits one reservation per run, and a second request for a run that has a reservation returns the outcome that exists and changes nothing. The descriptor is the same in both directions and requires fields a caller cannot know, so the contract of the method states who sets what. The caller sends tenantId, projectId and runId. reservationId, budgetId, state, expiresAt and approvalId are assigned by the service, and a value sent for them is ignored. amount.estimatedCostUsd is computed by the service: an amount the caller sends is not trusted, the service reserves its own estimate and returns it, whatever the caller sent. The estimate needs a rule, and the rule and its values are an input of the owner that is never invented here: RunAccepted carries the workflowId and the modelProfileId of the run and no limits, and the model holds the section Usage:Estimates as the place the estimator reads, without a default value. Until the owner gives the rule, the estimate comes from one estimator port with one configured implementation whose inputs are named: for the workflow and the model profile of the run it reads the expected quantity per meter from the configuration section Usage:Estimates and prices it with the pricing revision valid at the moment of the reservation. When the section holds no entry for that workflow and profile, or no pricing revision is valid, the estimator returns no estimate and never zero. The command has three outcomes. When the estimate fits, that is committedUsd plus the estimate does not exceed limitUsd and the estimate does not exceed approvalThresholdUsd, the Reservation is created in Reserved with expiresAt set from Usage:Reservations:ExpiryMinutes, committedUsd grows by the estimate and the BudgetReserved outbox record is written, all in one transaction. When the estimate would take committedUsd above limitUsd, which is always the case while observed usage holds the budget above its limit (AIFACTORY-US-549), or when no estimate is available, the reservation is recorded as Denied, the budget is unchanged, BudgetReservationDenied is written in the same transaction with the reason, budget-exceeded or estimate-unavailable, and connectsoft.factory.usage.reservations_denied is incremented with that reason, so a run that cannot be covered or cannot be estimated does not dispatch. When the estimate fits the limit and exceeds the threshold, the run needs a spend approval (D21), and the owner decided how it is obtained (D117): nothing is reserved, no reservation row is written, committedUsd is unchanged, and the SpendApprovalRequired outbox record is written with runId, tenantId, projectId, estimatedUsd and thresholdUsd; the gRPC reply names the outcome approval-required. The state enumeration of usage-reservation-v1 holds no value for that outcome, so the proto reply carries the outcome beside the reservation, which is absent. A redelivered RunAccepted is absorbed by the inbox and announces nothing twice, and a repeated call of the method for such a run gives the outcome approval-required again, while RunControlService opens at most one spend approval per run. On the event RunControlService opens a run-level approval of the action class spend and the run waits in WaitingForApproval, or stays Held when a frozen environment holds it (AIFACTORY-US-546 of feature run-control-service). The estimate travels with that approval and no row waits in this service: RunControlService/ApprovalDecided carries the proposal kind spend and the estimatedUsd the decision authorizes, and the inbox handler ReserveOnSpendApproval consumes it. For an Approved spend approval the handler reserves the approved estimate like any other estimate: under the row version of the budget the limit is checked again, the Reservation is created in Reserved with its approvalId and its expiresAt, committedUsd grows and BudgetReserved is written in the same transaction, on which RunControlService dispatches the run it had returned to Accepted; when the budget no longer covers the estimate the reservation is recorded as Denied with BudgetReservationDenied and the reason budget-exceeded. The approved estimate is reserved as approved and is not estimated again, because the decision is bound to that amount. A spend approval may be decided while its run is Held by a frozen environment (D119): the handler does not look at the state of the run and reserves an Approved one at once, so the reservation exists when the environment is active again and the run is dispatched. For a Rejected or an Expired spend approval the handler releases nothing, because nothing was reserved, and publishes nothing; RunControlService fails that run as policy-denied on its own decision. A spend approval that ends Cancelled with its cancelled or failed run changes nothing either, and the decisions of other proposal kinds are ignored. The handler is idempotent on the event id and on runId, so a redelivered decision reserves once. Two reservations that race on one budget are serialised by the row version of the Budget: the loser is retried and the limit is never exceeded by a reservation. BudgetReserved is published on usage-accounting.reservations.v1.budget-reserved with reservationId, runId, tenantId and estimatedUsd and lets RunControlService dispatch the first attempt; BudgetReservationDenied is published on usage-accounting.reservations.v1.budget-reservation-denied with runId, tenantId and reason and makes RunControlService fail an Accepted or a Held run as policy-denied; SpendApprovalRequired is published on usage-accounting.reservations.v1.spend-approval-required. All three travel in the run event envelope and are handled by RunControlService, the first two also by StudioBff, in their own PBIs. Over gRPC a caller without a workload identity is refused with Unauthenticated, a workload token without the service scope with PermissionDenied, an invalid request with InvalidArgument, and a project or a run of another tenant is answered NOT_FOUND like an unknown one (ADR-0015 amendment of D114).

Acceptance criteria

AC1 Given a CostUsageService/Budget with room, an entry of Usage:Estimates for the workflow and the model profile of the run and a valid pricing revision, such that the estimate does not exceed the threshold, when RunControlService/RunAccepted is handled by ReserveForRun or CostUsageService/Usage.Reserve is called for the run, then a CostUsageService/Reservation is Reserved with the estimate the service computed and expiresAt from Usage:Reservations:ExpiryMinutes, committedUsd grows by the estimate, the CostUsageService/BudgetReserved outbox record is written in the same transaction, the gRPC reply is usage-reservation-v1 with the state Reserved and the service estimate even when the caller sent another amount, and a second request for the same run returns the same reservation and changes nothing (SCN-CostUsageService-004).
AC2 Given an estimate that would take committedUsd above limitUsd, when the run is reserved, then the reservation is recorded as Denied, the CostUsageService/Budget is unchanged, the CostUsageService/BudgetReservationDenied outbox record with the reason budget-exceeded is written in the same transaction and connectsoft.factory.usage.reservations_denied is incremented; given a run whose workflow and model profile have no entry in Usage:Estimates, then it is Denied the same way with the reason estimate-unavailable and never reserved with an estimate of zero; given two reservations that race on one budget whose room covers one of them, then one is Reserved, the other Denied, and committedUsd does not exceed limitUsd (SCN-CostUsageService-004, NFR-09).
AC3 Given an estimate that fits the limit and exceeds approvalThresholdUsd, when the run is reserved through ReserveForRun or CostUsageService/Usage.Reserve, then no CostUsageService/Reservation is written, committedUsd is unchanged, the CostUsageService/SpendApprovalRequired outbox record with runId, tenantId, projectId, estimatedUsd and thresholdUsd is written, neither CostUsageService/BudgetReserved nor CostUsageService/BudgetReservationDenied is written, the gRPC reply names the outcome approval-required without a reservation, and a redelivery of the same RunControlService/RunAccepted writes no second record (SCN-CostUsageService-004, SCN-CostUsageService-012).
AC4 Given RunControlService/ApprovalDecided with the proposal kind spend and the state Approved for a run without a reservation, whatever the state of that run and also while it is Held (D119), when the inbox handler ReserveOnSpendApproval runs, then a CostUsageService/Reservation is Reserved with the estimatedUsd of the event, its approvalId and expiresAt from Usage:Reservations:ExpiryMinutes, committedUsd grows by that estimate and the CostUsageService/BudgetReserved outbox record is written in the same transaction, or the reservation is recorded as Denied with CostUsageService/BudgetReservationDenied and the reason budget-exceeded when the budget no longer covers the estimate; given the state Rejected, Expired or Cancelled, or another proposal kind, then no reservation is written, the CostUsageService/Budget is unchanged and nothing is published; a redelivery of the event reserves nothing twice (SCN-CostUsageService-012, SCN-CostUsageService-006).
AC5 Given a caller without a workload identity, a workload token without the service scope, a request without tenantId, projectId or runId, or a project or run of another tenant, when CostUsageService/Usage.Reserve is called, then the call is refused with Unauthenticated, PermissionDenied, InvalidArgument or NOT_FOUND respectively, no reservation is written and the budget is unchanged; and given a reservation was made or denied or a spend approval was required, when the outbox dispatcher of the cost-usage messaging profile publishes CostUsageService/BudgetReserved on usage-accounting.reservations.v1.budget-reserved, CostUsageService/BudgetReservationDenied on usage-accounting.reservations.v1.budget-reservation-denied or CostUsageService/SpendApprovalRequired on usage-accounting.reservations.v1.spend-approval-required, then the event is sent in run-event-envelope-v1 with reservationId, runId, tenantId and estimatedUsd, with runId, tenantId and reason, or with runId, tenantId, projectId, estimatedUsd and thresholdUsd, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-CostUsageService-004, SCN-CostUsageService-006, SCN-CostUsageService-007, SCN-CostUsageService-012, ADR-0012, ADR-0015).
Traces: services CostUsageService, RunControlService
Traces: aggregates CostUsageService/Budget, CostUsageService/Reservation
Traces: apis CostUsageService/Usage.Reserve
Traces: events CostUsageService/BudgetReserved, CostUsageService/BudgetReservationDenied, CostUsageService/SpendApprovalRequired, RunControlService/RunAccepted, RunControlService/ApprovalDecided
Traces: descriptors usage-reservation-v1, run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0015, ADR-0020
Traces: requirements FR-09, XL-12, NFR-09, NFR-12
Traces: scenarios SCN-CostUsageService-004, SCN-CostUsageService-006, SCN-CostUsageService-007, SCN-CostUsageService-012
Relates: nfrs NFR-09, NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-452, feature:run-control-service
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, model/services/control-plane/RunControlService.yaml, docs/reference/schemas/usage-record.schema.json, docs/reference/schemas/approval.schema.json, docs/planes/control/services/cost-usage-service/index.md#sagas-and-long-running-flows

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-1846 Contract of Usage.Reserve, input and assigned fields, the three outcomes and the denial reasons contract The contract document and the proto of Usage.Reserve state, against the UsageReservation definition of docs/reference/schemas/usage-record.schema.json, that the caller sends tenantId, projectId and runId, that reservationId, budgetId, state, expiresAt and approvalId are assigned by the service and ignored when sent, and that amount.estimatedCostUsd is computed by the service and returned whatever amount the caller sent; the reply carries the outcome Reserved, Denied or approval-required beside the reservation, which is absent for approval-required; the outcome approval-required is announced by SpendApprovalRequired with runId, tenantId, projectId, estimatedUsd and thresholdUsd, and the reservation that follows an approval carries its approvalId; the reasons budget-exceeded and estimate-unavailable of BudgetReservationDenied are named; and the options page of the repository documents the section Usage:Estimates with its keys workflow and model profile and the expected quantity per meter, its values being an input of the owner. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-310 Budgets and reservations before dispatch as one command with three outcomes and a configured estimator implementation ReserveForRun on RunAccepted and the gRPC method Usage.Reserve run one command that is idempotent on runId; a RunAccepted that carries parentRunId does not run this command and is handled as AIFACTORY-TASK-2975 states, since a child run gets no reservation of its own (D121); the estimator port has one configured implementation that reads the expected quantity per meter for the workflow and the model profile of the run from Usage:Estimates and prices it with the pricing revision valid at the reservation, and returns no estimate, never zero, when the entry or the revision is missing; under the row version of the Budget the command creates the Reservation in Reserved with expiresAt and the BudgetReserved outbox record while committedUsd grows, or records it as Denied with the BudgetReservationDenied outbox record, the reason budget-exceeded or estimate-unavailable and the denial metric, or, above the threshold, writes no reservation, leaves committedUsd unchanged, writes the SpendApprovalRequired outbox record with runId, tenantId, projectId, estimatedUsd and thresholdUsd and replies approval-required; a version conflict is retried so no reservation exceeds the limit. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2200 ReserveOnSpendApproval inbox handler reserving the approved estimate of a spend approval implementation The handler of ApprovalDecided acts on the proposal kind spend only: for the state Approved it runs the reservation command with the estimatedUsd of the event instead of a new estimate, so that under the row version of the Budget the Reservation is created in Reserved with its approvalId and expiresAt and the BudgetReserved outbox record while committedUsd grows, or is recorded as Denied with BudgetReservationDenied and the reason budget-exceeded when the budget no longer covers the estimate; for Rejected, Expired and Cancelled it writes and publishes nothing, because nothing was reserved; it does not read the state of the run, so the approval of a run that is Held is reserved at once (D119); it ignores every other proposal kind and is idempotent on the event id and on runId. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1832 Acceptance fixtures of estimator entries, pricing revisions and publishers of RunAccepted and ApprovalDecided fixture The acceptance suite configures entries of Usage:Estimates and seeds pricing revisions so that a test fixes the estimate of a run through its workflow and model profile without replacing the estimator, leaves one workflow without an entry for the estimate-unavailable outcome, and publishes RunAccepted through a fixture publisher, so that the three outcomes of the command and the race of two reservations are reproducible; a second fixture publisher sends ApprovalDecided with the proposal kind spend, a stated estimatedUsd and the state Approved, Rejected, Expired or Cancelled, and with another proposal kind, so that the reservation on a granted spend approval is reproducible without RunControlService. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1833 Verify reserve-run-budget through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-CostUsageService-004\|Scenario=SCN-CostUsageService-006\|Scenario=SCN-CostUsageService-007\|Scenario=SCN-CostUsageService-012)"; a filter that matches no test counts as a failure; the tests prove AC1 to AC5 of AIFACTORY-US-453, including the idempotent second request, an amount sent by the caller that the service overrides, the denial for a missing estimate, two racing reservations, the approval-required outcome that reserves nothing and writes SpendApprovalRequired once, the reservation with its approvalId on an Approved spend approval, the denial when the budget no longer covers it, nothing written for a Rejected, Expired or Cancelled one, the gRPC refusals and the publication and payload of BudgetReserved, BudgetReservationDenied and SpendApprovalRequired; consumer handling is proven by the suite of RunControlService. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-454: Record each metered quantity from a model or tool receipt exactly once per meter, priced by revision, keep unknown or unpriced usage uncertain and keep a late receipt as its confirmation

behavior ModelRouterService and ToolAdapterService, the only services that call model providers and execute tools, report what a call consumed, and CostUsageService turns each receipt into one usage record per meter. A receipt arrives on two paths that are one command: the internal gRPC method CostUsageService/Usage.RecordUsage, whose request is usage-record-v1 and whose reply is ack-v1, called by ModelRouterService, ToolAdapterService and RunControlService with a workload identity; and the inbox handlers RecordModelUsage for ModelRouterService/ModelInvoked and RecordToolUsage for ToolAdapterService/ToolInvoked. The descriptor holds fields a caller cannot know, so the contract of the method states who sets what. The caller sends tenantId, projectId, the runId when the usage belongs to a run, the optional stepId, attemptId and reservationId, meter, dimensions, the quantity when it is known, status (observed, estimated or uncertain), sourceReceipt and occurredAt. usageId, pricingRevision, estimatedCost, confirmedCost and reconciledAt are assigned by the service: a usageId the caller sends is ignored, and the service prices by the revision of its own table that is valid at occurredAt, so a pricingRevision the caller sends is not stored and one that differs changes nothing. A request without sourceReceipt, on which the idempotency rests, with the status reserved or reconciled, with a meter outside the enumeration, or with the status observed or estimated and no quantity is refused with InvalidArgument. The command is idempotent on the receipt, since sourceReceipt is unique per meter. The same receipt with the same content, delivered by both paths or twice by one, yields one UsageRecord per meter and an accepted acknowledgement. A later receipt is not a duplicate: when a receipt arrives for a sourceReceipt whose stored record is Estimated or Uncertain and the new receipt reports the usage, it is kept as the confirmation of that record in the table LateReceipts and acknowledged as accepted; the stored record is not rewritten by the command, and the ReconciliationWorker applies the confirmation (AIFACTORY-US-456). A receipt with other content for a record that is Observed or Reconciled is acknowledged as a conflict and changes nothing. A record attributes one quantity on one meter to the trusted tenant, the project and, where the receipt names them, the run and the attempt, with the dimensions of the receipt (profile, provider, model, tool, pool), occurredAt, and the pricing revision valid at occurredAt for the meter and the dimension, from which estimatedCostUsd is computed; a past record keeps the revision it was priced with. The input tokens and the output tokens of a model call are two meters, each with its own pricing (D117): from ModelInvoked two records are written with the receipt of the invocation as their sourceReceipt, one on the meter ModelInputTokens with inputTokens as its quantity and one on the meter ModelOutputTokens with outputTokens, each priced by the revision of its own meter, while the estimatedCostUsd the event carries is not stored. From ToolInvoked one record of quantity one is written on the meter ToolInvocations, with the tool and the state of the invocation in the dimensions. The unit and the factor of each meter and whether a failed or cancelled invocation is metered are inputs of the owner and are not decided here. The status follows the receipt: usage the provider reported is recorded as Observed, estimated usage as Estimated, and a receipt with unknown usage as Uncertain. Usage that cannot be priced or whose quantity is unknown is stored and never refused (D117). A receipt that reports no quantity, that is a usage-record-v1 with the status uncertain and no quantity or a ModelInvoked without the token count of a meter, is recorded as Uncertain without a quantity; one that carries an estimate keeps it as its quantity. A receipt whose meter and dimension have no pricing revision valid at occurredAt is recorded as Uncertain as well, whatever status it reports, with the quantity it carries and without a pricing revision. An Uncertain record has empty cost columns, no read and no event presents it as observed or as zero, and it is priced or reconciled later by the ReconciliationWorker, from a late receipt for its quantity and from a pricing revision valid at its occurredAt once one exists (AIFACTORY-US-456). No receipt is refused and no event is left unacknowledged for a missing price or a missing quantity, so ModelRouterService and ToolAdapterService need not keep and repeat a receipt. connectsoft.factory.usage.uncertain_records rises for an Uncertain record and connectsoft.factory.usage.cost_usd counts the cost by meter and status. The record and its UsageRecorded outbox record are written in one transaction, and the event is published on usage-accounting.usage.v1.usage-recorded in the run event envelope with usageId, tenantId, projectId, meter, status and the quantity, which is absent for an Uncertain record whose quantity is unknown; DashboardService projects it in its own PBI and FactoryCommercialService accumulates meters once it is delivered in I7. A record also counts against the budget where AIFACTORY-US-549, AIFACTORY-US-455 and AIFACTORY-US-459 state it: when the observed cost of a run exceeds the estimate of its Reserved reservation, and when the reservation of its run is no longer Reserved. The usage of a child run is recorded like any other usage, with the child run id as its runId, and counts on the reservation of the parent run, because a child run has no reservation of its own (D121, AIFACTORY-US-737). A record may also name no run (D123): runId is optional in usage-record-v1 and in ModelInvoked, the receipt of a model call outside a run carries the tenant and the project and neither a run nor an attempt, and its record is written like any other, without a run; how that usage counts against the budget of its project without a reservation is delivered by AIFACTORY-US-881. The model gives the handler RecordAttemptUsage the purpose to record usage from the attempt receipt and to note on the reservation of the run the time the first attempt outcome was recorded. RunControlService/AttemptOutcomeRecorded names the run, the step, the attempt, the outcome and the hash of the receipt and carries no metered quantity, so the handler creates no usage record; it sets attemptOutcomeNotedAt of the reservation, the one fact about dispatch the service receives, which the release and the expiry of a reservation read. That the first half of that purpose has no payload to act on is reported to the owner. The handler RecordCapacityUsage is not part of this delivery. CapacityLeaseGranted and CapacityLeaseReleased are published by the complete ExecutionCapacityService of I2 and not by its thin delivery of I1; they carry holderKind and holderId and no projectId, although a usage record requires one; and the meter and the factor of a lease are an owner input. Like QuotaExceeded they are not subscribed here. The receipts that reach the service at this increment feed the meters ModelInputTokens, ModelOutputTokens and ToolInvocations, and SandboxMinutes when ToolAdapterService reports it through the method; the model names no source for RunsAccepted, RunsCompleted, StorageGb and ProductCapacity, and nothing is recorded on them. The prices of PricingRevisions are an owner input, an entry of the I1 precondition register of AIFACTORY-US-1339 that the owner supplies before the I1 run (D140): the table is platform-wide, has no route in the model, and is loaded from a versioned seed that ships without production values. A deployment without prices takes no traffic, because readiness is not ready while a meter in use has no valid revision (AIFACTORY-US-451); a receipt that still finds none is stored as Uncertain, as stated above. All usage stays in cost-usage-sql; the Factory Metering service that the model names as the meter store arrives in I4 and is not required before it, feature component-saas-metering exports these records from then on, and nothing is exported here (D115, D117). Over gRPC a caller without a workload identity is refused with Unauthenticated, a workload token without the service scope with PermissionDenied, and a project of another tenant with NOT_FOUND.

Acceptance criteria

AC1 Given a model receipt with reported usage for a run of a tenant and project and pricing revisions valid at its occurredAt, when CostUsageService/Usage.RecordUsage is called by ModelRouterService with usage-record-v1, or ModelRouterService/ModelInvoked is handled, then one CostUsageService/UsageRecord exists per meter with tenant, project, run, attempt, dimensions, the status Observed, the pricing revision of the service and the estimatedCostUsd computed from it; the event yields two records with its receipt as their sourceReceipt, one on the meter ModelInputTokens with inputTokens as its quantity and one on the meter ModelOutputTokens with outputTokens, each priced by the revision of its own meter; a usageId or pricingRevision the caller sent is not stored, one CostUsageService/UsageRecorded outbox record per usage record is written in the same transaction, and the gRPC reply is ack-v1 (SCN-CostUsageService-005).
AC2 Given the receipt of AC1 was recorded, when the same receipt with the same content arrives again by either path, then no second record and no second outbox record exist and the acknowledgement is accepted, and other content for that Observed record is acknowledged as a conflict and changes nothing; given a stored record that is Uncertain and a later receipt for its sourceReceipt that reports the usage, then the receipt is kept as the confirmation of that record, the record is not rewritten and the acknowledgement is accepted; given ToolAdapterService/ToolInvoked for a tool call, then one record of quantity one exists on the meter ToolInvocations; given RunControlService/AttemptOutcomeRecorded, then no usage record is created and the reservation of the run notes that an attempt outcome was recorded (SCN-CostUsageService-005).
AC3 Given a receipt whose provider reported no usage, when it is recorded with the status uncertain and without a quantity, or ModelRouterService/ModelInvoked arrives without a token count, then the CostUsageService/UsageRecord is Uncertain and holds no quantity and empty cost columns, CostUsageService/UsageRecorded carries the status Uncertain and no quantity, and connectsoft.factory.usage.uncertain_records rises by one; given such a receipt that carries an estimate, then the record stores it as its quantity; given a receipt with estimated usage, then the status is Estimated; given a receipt with reported usage whose meter and dimension have no pricing revision valid at occurredAt, then the record is Uncertain with its quantity, without a pricing revision and without a cost, the call is acknowledged as accepted and the event is acknowledged, so that no receipt is refused for a missing price and no record is written with a quantity or a cost of zero (SCN-CostUsageService-005, FR-09).
AC4 Given a caller without a workload identity, a workload token without the service scope, a request that fails validation, names an unknown meter, has no sourceReceipt or states the status observed or estimated without a quantity, or a project of another tenant, when CostUsageService/Usage.RecordUsage is called, then the call is refused with Unauthenticated, PermissionDenied, InvalidArgument or NOT_FOUND respectively, no record is written and nothing is published; a missing pricing revision and an unknown quantity of an uncertain receipt are never among the refusals (SCN-CostUsageService-005, ADR-0015).
AC5 Given a usage record was created, when the outbox dispatcher of the cost-usage messaging profile publishes CostUsageService/UsageRecorded on usage-accounting.usage.v1.usage-recorded, then the event is sent in run-event-envelope-v1 with usageId, tenantId, projectId, meter, status and the quantity, which is absent for an Uncertain record whose quantity is unknown, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-CostUsageService-009, ADR-0012).
Traces: services CostUsageService, ModelRouterService, ToolAdapterService, RunControlService
Traces: aggregates CostUsageService/UsageRecord, CostUsageService/Reservation
Traces: apis CostUsageService/Usage.RecordUsage
Traces: events CostUsageService/UsageRecorded, ModelRouterService/ModelInvoked, ToolAdapterService/ToolInvoked, RunControlService/AttemptOutcomeRecorded
Traces: descriptors usage-record-v1, ack-v1, run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0015, ADR-0020
Traces: requirements FR-09, FUX-08, NFR-12
Traces: scenarios SCN-CostUsageService-005, SCN-CostUsageService-009
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, feature:model-router-service, feature:tool-adapter-service, feature:run-control-service
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, model/services/model-compute/ModelRouterService.yaml, model/services/tools-mcp/ToolAdapterService.yaml, model/services/runtime-cloud/ExecutionCapacityService.yaml, docs/reference/schemas/usage-record.schema.json, docs/architecture/adr/0020-commercial-model.md

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-1848 Contract of Usage.RecordUsage, input and assigned fields, statuses and the acknowledgement contract The contract document and the proto of Usage.RecordUsage state, against the UsageRecord definition of docs/reference/schemas/usage-record.schema.json, that the caller sends tenantId, projectId, the runId when the usage belongs to a run, which the descriptor makes optional (D123), the optional stepId, attemptId and reservationId, meter, dimensions, status, sourceReceipt and occurredAt and the quantity when it is known, that the quantity may be absent only with the status uncertain, that usageId, pricingRevision, estimatedCost, confirmedCost and reconciledAt are assigned by the service and ignored when sent, that only the statuses observed, estimated and uncertain are accepted, that a request without sourceReceipt is invalid, how the eight meter names of the descriptor map to the Meter enumeration, model.input-tokens and model.output-tokens being two meters, that a receipt without a valid pricing revision is accepted and stored as uncertain, and that ack-v1 answers accepted for a new record, a duplicate and a kept late receipt and conflict for other content of an Observed or Reconciled record. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1849 Migration of the kept late receipts and of the attempt outcome note of a reservation migration A migration of cost-usage-sql adds the tenant-filtered table LateReceipts of the model, keyed by usage record and receipt reference, with the reported quantity, the time it was received and the time it was applied, and adds to Reservations the nullable column AttemptOutcomeNotedAt of the model with the time the first attempt outcome of the run was recorded; mapping tests on an isolated test database prove both. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-311 Usage records from model and tool receipts, once per receipt and meter, with duplicates, late receipts and unknown usage told apart implementation Usage.RecordUsage and the handlers RecordModelUsage and RecordToolUsage run one command that creates the UsageRecord with tenant, project, the run and the attempt where the receipt names them, meter, dimensions, quantity, the status the receipt reports, the pricing revision valid at occurredAt and its estimated cost, together with the UsageRecorded outbox record in one transaction, idempotent on meter plus source receipt; from ModelInvoked two records are written under the receipt of the invocation, one on ModelInputTokens with inputTokens and one on ModelOutputTokens with outputTokens, each priced by the revision of its own meter, and the cost of the event is not stored; from ToolInvoked the quantity is one; a receipt without a quantity is stored as Uncertain without one, a receipt without a valid pricing revision is stored as Uncertain with its quantity and without a revision, both with empty cost columns, and neither is refused, left unacknowledged or written as zero; the same content again changes nothing, a later receipt that reports the usage of an Estimated or Uncertain record is written to the table LateReceipts of AIFACTORY-TASK-1849, other content for an Observed or Reconciled record is a conflict; RecordAttemptUsage sets attemptOutcomeNotedAt of the reservation of the run and creates no record; the cost and uncertain metrics follow the records. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1834 Versioned seed of PricingRevisions without production values migration A migration step loads PricingRevisions from a versioned seed file with revision, meter, dimension key, unit price and validity, refuses two revisions of one meter and dimension whose validity overlaps, and never changes a revision a usage record already names; the seed of the repository holds test revisions only, the input tokens and the output tokens of a model call having revisions of their own, and the production prices and the unit and the factor of each meter are entered by the owner. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1835 Verify record-metered-usage through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-CostUsageService-005\|Scenario=SCN-CostUsageService-009)"; a filter that matches no test counts as a failure; the tests prove AC1 to AC5 of AIFACTORY-US-454 with fixture publishers of ModelInvoked, ToolInvoked and AttemptOutcomeRecorded, including one receipt delivered by both paths, the two token records of one model call, a conflicting repeat, a late receipt kept for an Uncertain record, an unknown-usage receipt with and without an estimate, an unpriced receipt stored as Uncertain and acknowledged, the gRPC refusals and the publication and payload of UsageRecorded with and without a quantity; consumer handling is proven by the suites of the consumers. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-455: Settle the reservation of a run at its terminal state, release it when the run was cancelled before dispatch, and keep the budget true for usage that arrives afterwards

behavior A reservation is an amount held before dispatch; this PBI brings it to rest when the run ends. When RunControlService announces the terminal state of a run through RunControlService/RunCompleted or RunControlService/RunFailed, the inbox handler SettleReservation moves the reservation from Reserved to Settled (trigger RunTerminal): settledUsd becomes the sum of the cost of the usage records of the run, where the cost of a record is its confirmed cost when it has one and its estimated cost otherwise, settledAt is set, committedUsd of the budget is corrected from the amount counted for the run, which is its estimate or its observed cost where that already exceeded the estimate (AIFACTORY-US-549), to the settled amount, and the ReservationSettled outbox record is written with reservationId, runId, tenantId, settledUsd and uncertain, all in one transaction. uncertain is true when at least one usage record of the run is Uncertain or Estimated at settlement; the settled amount then counts what is known and invents nothing for the rest, and reconciliation confirms the records later (AIFACTORY-US-456). Settlement is idempotent on the set of receipts settled: a redelivered terminal event changes nothing and publishes nothing. Settlement records what was observed, also when it exceeds the estimate. The invariant of the Budget holds the limit at reservation and lets only observed usage, recorded or settled, take committedUsd above limitUsd (D117). A run that spends more than its estimate while it runs is counted and stopped as AIFACTORY-US-549 states: the usage record that takes the budget above its limit writes BudgetExceeded, and RunControlService cancels the run. Settlement never hides observed cost: an overrun is stored unchanged, a settlement that itself raises committedUsd and leaves it above limitUsd writes BudgetExceeded with the runId of the settled run in the same transaction, unless the event was already written for that run, since it is published once per run (D119), and the next reservation against that budget is Denied. A receipt that arrives after settlement is still recorded (AIFACTORY-US-454). The settled reservation is not reopened and ReservationSettled is not published again, because the Reservation has no transition out of Settled, but the budget stays true: the cost of a usage record created for a run whose reservation is no longer Reserved is added to committedUsd of the budget in the transaction of that record. The same correction is made for a confirmation applied after settlement (AIFACTORY-US-456) and for the usage of a run whose reservation was Released or Expired (AIFACTORY-US-459). When RunControlService/RunCancelled arrives, SettleOrRelease decides from what the service holds for the run. A run with no usage record and no noted attempt outcome was cancelled before dispatch as far as the service can know, so its reservation moves from Reserved to Released (trigger RunCancelledBeforeDispatch) and the whole estimate returns to the budget; a run with a usage record or with the note of RecordAttemptUsage is settled like a terminal run and never released. A release is announced like an expiry: ReservationSettled is written with a settledUsd of zero and uncertain false in the transaction of the release. The model does not say that a release is silent, the aggregate lists ReservationSettled among the events of the reservation, and StudioBff and DashboardService learn of a reservation through events only, so a silent release would leave their views at Reserved; the state Released itself is read through GetRunUsage. committedUsd is described by the model as reserved plus settled in the period. A monthly budget is rolled over at the calendar month in UTC by the BudgetRolloverWorker (AIFACTORY-US-452, D117), which carries a reservation that is still Reserved into the new period with its estimate (D119), so a settlement after the rollover corrects the budget of the new period, and the settledAt of the reservation tells the period the settlement belongs to. A terminal event for a run whose reservation is Denied, Released, Expired or already Settled moves no reservation. The terminal events of a child run settle and release nothing, because a child run has no reservation; its usage is settled with the reservation of the parent run, whose settledUsd then counts the usage records of the parent run and of the child runs noted under that reservation (D121, AIFACTORY-US-737). ReservationSettled is published on usage-accounting.reservations.v1.reservation-settled in the run event envelope; StudioBff and DashboardService handle it in their own PBIs and FactoryCommercialService accumulates the run cost once it is delivered in I7.

Acceptance criteria

AC1 Given a Reserved CostUsageService/Reservation of a run with three Observed usage records, when RunControlService/RunCompleted or RunControlService/RunFailed is handled by SettleReservation, then the reservation is Settled with settledUsd equal to the sum of the cost of the three records, committedUsd of the CostUsageService/Budget is corrected from the estimate to that sum, the CostUsageService/ReservationSettled outbox record with uncertain false is written in the same transaction, and a redelivery of the event changes nothing and writes no record (SCN-CostUsageService-008).
AC2 Given the run of AC1 with one further CostUsageService/UsageRecord in the status Uncertain, when the run is settled, then settledUsd counts the three known records only, nothing is invented for the uncertain one, and CostUsageService/ReservationSettled carries uncertain true; given observed records whose sum exceeds the estimate so far that committedUsd is above limitUsd, then the observed sum is stored unchanged at settlement, CostUsageService/BudgetExceeded was written by the write that took the amount above the limit, and the next reservation against that budget is Denied (SCN-CostUsageService-008, FR-09).
AC3 Given RunControlService/RunCancelled for a run for which the service holds no usage record and no noted attempt outcome, when SettleOrRelease runs, then the reservation is Released, the whole estimate returns to committedUsd, and CostUsageService/ReservationSettled with a settledUsd of zero and uncertain false is written in the same transaction; given the same event for a run with a usage record or with a noted attempt outcome, then the reservation is Settled as in AC1 and never Released (SCN-CostUsageService-008).
AC4 Given a Settled reservation, when a further receipt of its run is recorded, then the CostUsageService/UsageRecord is created, the reservation stays Settled with its settledUsd, no second CostUsageService/ReservationSettled is written, and committedUsd of the CostUsageService/Budget grows by the cost of that record in the transaction of the record (SCN-CostUsageService-008, XL-12).
AC5 Given a reservation was settled or released, when the outbox dispatcher of the cost-usage messaging profile publishes CostUsageService/ReservationSettled on usage-accounting.reservations.v1.reservation-settled, then the event is sent in run-event-envelope-v1 with reservationId, runId, tenantId, settledUsd and uncertain, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-CostUsageService-008, ADR-0012).
Traces: services CostUsageService, RunControlService
Traces: aggregates CostUsageService/Reservation, CostUsageService/Budget, CostUsageService/UsageRecord
Traces: events CostUsageService/ReservationSettled, CostUsageService/BudgetExceeded, RunControlService/RunCompleted, RunControlService/RunFailed, RunControlService/RunCancelled
Traces: descriptors run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0020
Traces: requirements FR-09, XL-12, NFR-12
Traces: scenarios SCN-CostUsageService-008
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-453, AIFACTORY-US-454, feature:run-control-service
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, model/services/control-plane/RunControlService.yaml, docs/planes/control/services/cost-usage-service/index.md#domain-model, docs/planes/control/services/cost-usage-service/index.md#workers-and-jobs

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-1836 SettleReservation and SettleOrRelease on the terminal events of a run implementation SettleReservation on RunCompleted and RunFailed moves the Reserved reservation to Settled with settledUsd as the sum of the cost of the usage records of the run, confirmed cost before estimated cost, sets settledAt, corrects committedUsd of the budget from the amount counted for the run to the observed sum also when it exceeds limitUsd, in which case the BudgetExceeded record of AIFACTORY-TASK-2203 is written when the settlement itself raised the amount and none was written for the run before (D119), and writes the ReservationSettled outbox record with uncertain true while a record of the run is Uncertain or Estimated, all in one transaction and idempotent on the set of receipts settled; SettleOrRelease on RunCancelled releases the reservation, returns the estimate and writes ReservationSettled with a settledUsd of zero when the service holds no usage record and no attempt outcome note of the run, and settles it otherwise; a reservation that is not Reserved is left unchanged; the usage records of the child runs noted under a reservation count in its settledUsd and as usage of that reservation, and a terminal event of a child run moves no reservation, as AIFACTORY-TASK-2976 delivers (D121). ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1850 Budget correction for usage recorded after the reservation of its run came to rest implementation The command that creates a usage record adds the cost of the record to committedUsd of the budget of the project, in the transaction of the record and under the row version of the Budget, when the reservation of the run is Settled, Released or Expired; the reservation keeps its state and its settledUsd and no second ReservationSettled is written; a record whose run has a Reserved reservation changes the budget only by the excess over the estimate that AIFACTORY-TASK-2203 counts, and a record whose run has no reservation and is not noted as a child run changes no budget, while a record of a child run counts on the reservation of its parent run as AIFACTORY-TASK-2976 states (D121); a correction that leaves committedUsd above limitUsd writes the BudgetExceeded record of AIFACTORY-TASK-2203, once per run (D119); unit and acceptance tests cover each of the three rest states. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1837 Verify settle-a-reservation through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-CostUsageService-008&Area=Settlement"; a filter that matches no test counts as a failure; the tests prove AC1 to AC5 of AIFACTORY-US-455 with fixture publishers of RunCompleted, RunFailed and RunCancelled, including the corrected committedUsd of the budget, a settlement above the limit followed by a denied reservation, a redelivered terminal event, the release with its event, a cancelled run with a noted attempt outcome that is settled, a receipt recorded after settlement, and the publication and payload of ReservationSettled; consumer handling is proven by the suites of the consumers. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-459: Expire the reservation of a run that never dispatched, never expire one of a run that spent, and count the usage of a run that spends after its reservation came to rest

behavior A reservation whose run never dispatches would hold budget without end; the ReservationExpiryWorker frees it. The worker runs every fifteen minutes, two tenants in parallel, with three attempts at a fixed backoff of 60 seconds. The model gives its input as Reserved reservations past expiresAt without a dispatched attempt. CostUsageService does not hold that fact: it does not consume AttemptDispatched of RunControlService, and nothing else tells it that an attempt was dispatched. This PBI does not replace the missing fact by a rule presented as certain; it states what the service can know and what follows when that is not enough. What the service holds for a run is its usage records and the note of RecordAttemptUsage that an attempt outcome was recorded (AIFACTORY-US-454). A Reserved reservation of a run with a usage record or with that note is never expired and never released, however old it is: the run did dispatch, and its reservation comes to rest only through the terminal event of the run (AIFACTORY-US-455). A usage record of a child run counts on the reservation of its parent run (D121, AIFACTORY-US-737), so a reservation under which a child run has a usage record is never expired and never released either. A Reserved reservation past expiresAt of a run with neither is moved to Expired (trigger ExpiryWithoutRun): the estimate returns to committedUsd of the budget and ReservationSettled, the output event of the worker in the model, is written with a settledUsd of zero and uncertain false, in one transaction and idempotent on reservationId plus expiresAt. That judgement can be wrong in two cases the service cannot see: a run whose first attempt was dispatched and has reported neither a receipt nor an outcome when expiresAt passes, and a run that was Held after BudgetReserved for longer than Usage:Reservations:ExpiryMinutes and is dispatched later. RunControlService does not learn of the expiry either, since it does not consume ReservationSettled, so such a run goes on. The branch in which a run spends although its reservation is Expired or Released is therefore part of this PBI. The usage records of that run are recorded as always, and the cost of each is added to committedUsd of the budget of the project in the transaction of the record, exactly as for a record that arrives after settlement, so the cost of a run that really spent reaches the budget and the cap of the first outcome (D80) is not bypassed for the next reservation. When that cost takes committedUsd above limitUsd, BudgetExceeded names the run and RunControlService cancels it with the reason budget-exceeded (AIFACTORY-US-549, D117), so such a run is stopped at the limit like any other. The terminal event of such a run moves no reservation, because the Reservation has no transition out of Expired or Released, and writes no second ReservationSettled; its usage is visible through the usage records, UsageRecorded and GetRunUsage. A confirmation applied later corrects the budget the same way (AIFACTORY-US-456). The two missing signals, a dispatch fact for CostUsageService and an expiry signal for RunControlService, are reported to the owner and are not decided here. ReservationSettled is published on usage-accounting.reservations.v1.reservation-settled in the run event envelope; StudioBff and DashboardService handle it in their own PBIs.

Acceptance criteria

AC1 Given a Reserved CostUsageService/Reservation past expiresAt for whose run the service holds no usage record and no noted attempt outcome, when the ReservationExpiryWorker runs, then the reservation is Expired, the estimate returns to committedUsd of the CostUsageService/Budget, CostUsageService/ReservationSettled with a settledUsd of zero and uncertain false is written once in the same transaction under the key reservationId plus expiresAt, and a duplicate trigger has no second effect (SCN-CostUsageService-008).
AC2 Given a Reserved reservation past expiresAt of a run with a CostUsageService/UsageRecord, and one of a run whose attempt outcome was noted, when the ReservationExpiryWorker runs, then both stay Reserved with their estimate in committedUsd; when RunControlService/RunCancelled then arrives for either run, then the reservation is Settled and not Released (SCN-CostUsageService-008).
AC3 Given a reservation that is Expired, and one that is Released, when a receipt of its run is recorded, then the CostUsageService/UsageRecord and its CostUsageService/UsageRecorded record are written and committedUsd of the CostUsageService/Budget grows by the cost of the record in the same transaction; when the terminal event of that run arrives, then the reservation keeps its state, no second CostUsageService/ReservationSettled is written and committedUsd still holds the cost of those records, so the next reservation is judged against it (SCN-CostUsageService-008, XL-12).
AC4 Given the store fails while the ReservationExpiryWorker expires the reservations of two tenants, when the worker retries with three attempts at a fixed backoff of 60 seconds, then each reservation is expired once and each CostUsageService/ReservationSettled is published once on usage-accounting.reservations.v1.reservation-settled in run-event-envelope-v1, neither lost nor duplicated under its event id (SCN-CostUsageService-008, ADR-0012).
Traces: services CostUsageService, RunControlService
Traces: aggregates CostUsageService/Reservation, CostUsageService/Budget, CostUsageService/UsageRecord
Traces: events CostUsageService/ReservationSettled, CostUsageService/UsageRecorded, RunControlService/RunCancelled
Traces: descriptors run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0020
Traces: requirements FR-09, XL-12, NFR-12
Traces: scenarios SCN-CostUsageService-008
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-453, AIFACTORY-US-454, AIFACTORY-US-455, feature:run-control-service
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, model/services/control-plane/RunControlService.yaml, docs/planes/control/services/cost-usage-service/index.md#workers-and-jobs

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-243 ReservationExpiryWorker expiring the reservation of a run for which the service holds no sign of dispatch implementation The worker runs on the schedule of the model every fifteen minutes over Reserved reservations past expiresAt for whose run the service holds no usage record and no attempt outcome note, partitioned by tenant with two partitions in parallel and three attempts at a fixed backoff of 60 seconds; it moves each to Expired, returns the estimate to the budget and writes ReservationSettled with a settledUsd of zero in one transaction, idempotent on reservationId plus expiresAt; a reservation of a run with a usage record or with the note is skipped however old it is, and so is a reservation under which a child run noted in ChildRuns has a usage record (D121); a duplicate trigger has no second effect. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1851 Verify expire-a-stale-reservation through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-CostUsageService-008&Area=Expiry"; a filter that matches no test counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-459 with a controlled clock, including the expiry with its event, a reservation of a run with a usage record and one with a noted attempt outcome that stay Reserved, receipts recorded for a run whose reservation is Expired and for one whose reservation is Released with committedUsd corrected through AIFACTORY-TASK-1850, the terminal event of such a run, and a store failure during the worker; consumer handling is proven by the suites of the consumers. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-456: Confirm estimated and uncertain usage from late receipts, keep the port for provider statements, and announce each confirmation

behavior Unknown usage stays unknown until evidence confirms it (FR-09). The ReconciliationWorker runs every hour over the usage records in the status Estimated or Uncertain that are older than one hour, two tenants in parallel, with five attempts and exponential backoff from 60 seconds. For each record it asks one confirmation port, keyed by usageId and a statement reference, for the confirmation the invariant of the model names: a later receipt or a line of a provider statement. The first of the two is delivered here and works on a deployed instance. A late receipt reaches the service through CostUsageService/Usage.RecordUsage or the receipt events and is kept by that command as the confirmation of the record it belongs to (AIFACTORY-US-454); the late-receipt source of the port reads those kept receipts, takes the reference of the receipt as the statement reference and computes the confirmed cost from the reported quantity and the pricing revision the record was priced with, or, for a record that was stored without one, the revision valid at its occurredAt once it exists. The second is not delivered. The model lists no route, no table and no consumed event for a provider statement, and the service page lists the import of provider statements per provider among the later capabilities; the port is the place where such an adapter lands, none is built or invented here, and that is reported to the owner. When a confirmation is found, the record gets its confirmedCostUsd, moves to the status Reconciled and keeps its pricing revision, its quantity and its estimate, the UsageReconciled outbox record with usageId, tenantId and confirmedCostUsd is written in the same transaction, the kept receipt is marked as applied, connectsoft.factory.usage.uncertain_records falls for a record that was Uncertain, and connectsoft.factory.usage.cost_usd counts the confirmed cost under the reconciled status. An Uncertain record that was stored without a price is priced by the same worker (D117): once a pricing revision valid at its occurredAt exists for its meter and dimension and the record holds a quantity, from its receipt or from an applied late receipt, the worker sets the pricing revision, computes the cost from the quantity, sets confirmedCostUsd, moves the record to Reconciled and writes UsageReconciled in the same transaction, with the pricing revision as the reference of what confirmed it; a record that still has no quantity or no valid revision stays Uncertain. The worker is idempotent on usageId plus the statement reference, so the same confirmation applied twice confirms once. A confirmation never reopens a reservation, but it keeps the budget true: when the reservation of the run of the record is no longer Reserved, committedUsd of the budget is corrected in the same transaction by the difference between the confirmed cost and the cost counted for the record so far, which is its estimated cost or nothing for a record that was Uncertain; while the reservation is still Reserved nothing is corrected, because settlement will sum the confirmed cost (AIFACTORY-US-455). A record without a confirmation is left as it is: it is never zeroed, never guessed and never aged out, and it stays visible as uncertain in every usage read (AIFACTORY-US-457). The corrected cost travels in UsageReconciled, which is published on usage-accounting.usage.v1.usage-reconciled in the run event envelope; DashboardService projects it in its own PBI and FactoryCommercialService adjusts meters once it is delivered in I7.

Acceptance criteria

AC1 Given a CostUsageService/UsageRecord in the status Uncertain that is older than one hour and a later receipt for its sourceReceipt that reports the usage and was kept by Usage.RecordUsage, when the ReconciliationWorker runs on an instance without any fixture source, then the record has its confirmedCostUsd, computed from the reported quantity and the pricing revision of the record, the status Reconciled and its unchanged pricing revision and estimate, the CostUsageService/UsageReconciled outbox record is written in the same transaction, the kept receipt is marked as applied, and connectsoft.factory.usage.uncertain_records falls by one (SCN-CostUsageService-010).
AC2 Given an Estimated record of a run whose reservation is Settled, when a confirmation with a statement reference is applied through the confirmation port, then committedUsd of the budget is corrected in the same transaction by the difference between the confirmed and the estimated cost, the reservation keeps its state and its settledUsd, and CostUsageService/UsageReconciled is written; given the same for a record whose run still has a Reserved reservation, then no budget amount changes (SCN-CostUsageService-010, XL-12).
AC3 Given a confirmation was applied, when the same late receipt or the same statement is offered again or the worker is triggered twice, then the record is confirmed once and no second CostUsageService/UsageReconciled is written, because the worker is idempotent on usageId plus the statement reference; given an Uncertain record that holds a quantity and was stored without a pricing revision, when a revision valid at its occurredAt has been loaded and the worker runs, then the record gets that revision and its confirmedCostUsd, moves to Reconciled and CostUsageService/UsageReconciled is written once, while a record that still has no valid revision stays Uncertain (SCN-CostUsageService-010).
AC4 Given an Estimated or Uncertain record without a confirmation, a record younger than one hour, or a confirmation source that fails, when the ReconciliationWorker runs, then the record keeps its status, quantity and cost, is never written as zero, and a failing source is retried five times with exponential backoff without changing the record (SCN-CostUsageService-010, FR-09).
AC5 Given a record was reconciled, when the outbox dispatcher of the cost-usage messaging profile publishes CostUsageService/UsageReconciled on usage-accounting.usage.v1.usage-reconciled, then the event is sent in run-event-envelope-v1 with usageId, tenantId and confirmedCostUsd, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-CostUsageService-010, ADR-0012).
Traces: services CostUsageService
Traces: aggregates CostUsageService/UsageRecord, CostUsageService/Budget
Traces: events CostUsageService/UsageReconciled
Traces: descriptors run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0020
Traces: requirements FR-09, FUX-08, XL-12, NFR-12
Traces: scenarios SCN-CostUsageService-010
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-454, AIFACTORY-US-455
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, docs/planes/control/services/cost-usage-service/index.md#workers-and-jobs, docs/planes/control/services/cost-usage-service/index.md#failure-and-recovery, docs/planes/control/services/cost-usage-service/index.md#later-capabilities

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-312 ReconciliationWorker confirming estimated and uncertain usage through a confirmation port implementation The worker runs on the schedule of the model every hour over usage records in Estimated or Uncertain older than one hour, partitioned by tenant with two partitions in parallel and five attempts with exponential backoff from 60 seconds; it asks the confirmation port for each record, sets confirmedCostUsd and the status Reconciled, writes the UsageReconciled outbox record and, when the reservation of the run is Settled, Released or Expired, corrects committedUsd of the budget by the difference between the confirmed cost and the cost counted so far, all in one transaction and idempotent on usageId plus the statement reference; an Uncertain record stored without a price is priced, with the pricing revision as the reference of the confirmation, once a revision valid at its occurredAt exists and it holds a quantity; a record without a confirmation is left unchanged and never zeroed, and the uncertain and cost metrics follow each confirmation. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1852 Late-receipt source of the confirmation port implementation A source of the confirmation port that reads the kept late receipts of AIFACTORY-TASK-1849 for a usage record, returns the reference of the receipt as the statement reference and the confirmed cost computed from the reported quantity and the pricing revision the record was priced with, or the revision valid at its occurredAt for a record that was stored without one, and marks the kept receipt as applied in the transaction of the confirmation; it is registered in the deployed service, so an Uncertain record is reconciled from a late receipt without any fixture; no provider statement adapter is registered. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1838 Acceptance fixture of a statement source behind the confirmation port fixture The acceptance suite can register a second source behind the port that returns, for a usage id, a confirmed cost with a statement reference a test states, returns nothing for other records and can be made to fail, so that a statement confirmation, a repeated statement, a missing confirmation and a failing source are reproducible with a controlled clock; the tests of the late-receipt path run without it. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1839 Verify reconcile-uncertain-usage through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-CostUsageService-010"; a filter that matches no test counts as a failure; the tests prove AC1 to AC5 of AIFACTORY-US-456, including an Uncertain record reconciled from a kept late receipt without a fixture source, the budget corrected for a record of a settled run and untouched for a run still Reserved, the confirmation applied twice, an unpriced Uncertain record priced after its revision was loaded, the record left uncertain and the publication and payload of UsageReconciled; consumer handling is proven by the suites of the consumers. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-457: Read the usage of a project by meter and state and the usage records and reservation of one run

behavior ยท persona observer As an observer, I want to see what a project and a single run have reserved, used and still owe an answer on, so that I can trust the figures, including the part that is still uncertain.

A caller whose scope carries factory.usage.read reads usage through two routes. CostUsageService/GetProjectUsage returns the usage of the project the path addresses for a period: per meter, the quantity and the cost in each of the five states the service shows as such, estimated, reserved, observed, reconciled and uncertain (FUX-08), together with the limits of the project, that is the limitUsd, committedUsd and approvalThresholdUsd of its budget. The reserved figure is the estimate of the reservations that are still Reserved; the other four are sums over the usage records by status. Uncertain usage is shown as uncertain: a quantity it holds is marked as not confirmed, a record without a quantity shows none, it has no cost, and it is never folded into another state or shown as observed or as zero. CostUsageService/GetRunUsage returns the usage records of one run, each with meter, dimensions, quantity, status, pricing revision, estimated and confirmed cost and source receipt, and the reservation of the run with its state, among them Released and Expired, its estimate, its settled amount and its approval reference. Both routes read cost-usage-sql under the tenant row filter and write nothing. A caller without the scope, or a member of the tenant whose role claims do not admit the project, is refused with 403. A run of another tenant is answered 404, the same answer as for a runId the service holds nothing for, and a project of another tenant is answered 404 as well, the same answer as for a project without a budget in the tenant of the caller, so no response confirms that a foreign resource exists (ADR-0015 amendment of D113). A project of the caller without usage in the period answers 200 with empty figures. When cost-usage-sql is unavailable or the token cannot be validated the call fails fast with a problem response. At this increment the routes are read through the platform gateway and by the command-line client; the project overview and the usage and cost screens of the Studio read them once StudioBff is delivered in I3. The figures come from the usage records of this service alone at this increment, since the Metering service and its export arrive in I4.

Acceptance criteria

AC1 Given a project with a budget, one Reserved reservation and usage records in the statuses Observed, Estimated, Uncertain and Reconciled on two meters, when CostUsageService/GetProjectUsage is called for a period by a caller with factory.usage.read, then the response is 200 with, per meter, quantity and cost in each of the five states, the reserved figure taken from the Reserved CostUsageService/Reservation, the uncertain usage shown as uncertain, with a quantity it holds marked as not confirmed and none shown for a record without one, and never as observed or as zero, and the limitUsd, committedUsd and approvalThresholdUsd of the CostUsageService/Budget; nothing is written (SCN-CostUsageService-001, FUX-08).
AC2 Given a run with a settled reservation and three usage records, when CostUsageService/GetRunUsage is called, then the response is 200 with each CostUsageService/UsageRecord (meter, dimensions, quantity, status, pricing revision, estimated and confirmed cost, source receipt) and the reservation with its state, estimatedUsd, settledUsd and approvalId; for a settled run without uncertain records and without a receipt recorded after settlement the sum of the cost of its records equals settledUsd (SCN-CostUsageService-002, FR-09).
AC3 Given a caller without factory.usage.read, a member of the tenant whose role claims do not admit the project, a run of another tenant, or a project of another tenant, when CostUsageService/GetRunUsage or CostUsageService/GetProjectUsage is called, then the answer is 403, 403, 404 or 404 respectively, each 404 being the same answer as for a resource that does not exist, with a problem response and no state change; a project of the caller without usage in the period answers 200 with empty figures (SCN-CostUsageService-001, SCN-CostUsageService-002, ADR-0015).
Traces: services CostUsageService
Traces: aggregates CostUsageService/Budget, CostUsageService/Reservation, CostUsageService/UsageRecord
Traces: apis CostUsageService/GetProjectUsage, CostUsageService/GetRunUsage
Traces: adrs ADR-0015, ADR-0020
Traces: requirements FR-09, FUX-08, XL-12
Traces: scenarios SCN-CostUsageService-001, SCN-CostUsageService-002
Relates: screens project-overview, usage-cost
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-454, AIFACTORY-US-455
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, model/ux/screens.yaml, docs/planes/control/services/cost-usage-service/index.md#apis

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-1840 Contract of the project usage view and the run usage view contract The OpenAPI document of the service holds the project usage view with, per meter, quantity and cost in the states estimated, reserved, observed, reconciled and uncertain, a mark that a quantity of uncertain usage is not confirmed and may be absent, and the limit, committed amount and threshold of the budget, and the run usage view with the usage records and the reservation of the run in any of its five states, named after the fields of model/services/control-plane/CostUsageService.yaml and docs/reference/schemas/usage-record.schema.json. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1841 GetProjectUsage and GetRunUsage read model over cost-usage-sql with the tenant row filter implementation GetProjectUsage sums the usage records of the project and period by meter and status, takes the reserved figure from the reservations still Reserved and returns the limits of the budget; GetRunUsage returns the usage records and the reservation of the run; both apply the tenant row filter so a foreign run or project is answered 404 like an unknown one, answer 403 for a missing scope or a project the role claims of the token do not admit, show uncertain usage as uncertain, with a quantity it holds marked as not confirmed and none for a record without one, and write nothing. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-1842 Verify read-usage through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-CostUsageService-001\|Scenario=SCN-CostUsageService-002)"; a filter that matches no test counts as a failure; the tests prove AC1 to AC3 of AIFACTORY-US-457 with usage of two tenants, records in every status and a settled run whose record costs sum to settledUsd. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-549: Count the observed cost of a run above its estimate against the budget, announce a budget overrun so that the run is stopped, and deny the next reservation

behavior A reservation holds the estimate of a run, and the cap of a budget is enforced where a reservation is made. What a run really spends can still exceed its estimate, and the owner decided that such an overrun stops the running run and is not only recorded (D117). The invariants of the Budget in the model say it this way: committedUsd never exceeds limitUsd at reservation; only observed usage, recorded or settled, may take it above limitUsd, and that publishes BudgetExceeded and stops the run of that usage; and after such an overrun the next reservation of the budget is Denied. The owner then fixed the reach of the stop (D119, D121): BudgetExceeded names the run whose usage crossed the limit and is published once per run; only a named run is stopped; every run whose observed usage raises committedUsd of a budget that is above limitUsd, the run that crossed the limit and every later one, is named once and stopped; the runs that stay within their estimate finish and settle; and new reservations of the budget are denied. The run the event names may be a child run, whose usage counts on the reservation of its parent run (AIFACTORY-US-737): only that child run is stopped, it ends Cancelled with the reason budget-exceeded like every stopped run, and only its parent treats it as a failed child (D123). This PBI delivers that on the side of CostUsageService; RunControlService records the cancellation intent of the run in its own PBI (AIFACTORY-US-547 of feature run-control-service). For observed usage to reach the budget while a run is still running, the cost of its usage records has to count before settlement. The model describes committedUsd as reserved plus settled in the period and does not say how the usage of a run whose reservation is still Reserved counts, so this PBI states the rule it applies and reports it to the owner for confirmation: a run with a Reserved reservation counts in committedUsd with the larger of its estimate and the cost of its usage records so far. As long as the observed cost stays within the estimate a usage record changes no budget, because the estimate already holds that room. When a usage record takes the sum of the cost of the records of its run above the estimate, the part above the estimate that was not yet counted is added to committedUsd of the budget, in the transaction of the record and under the row version of the Budget. The cost of a record is its confirmed cost when it has one and its estimated cost otherwise; an Uncertain record has no cost and adds nothing until it is priced or reconciled. Two other writes raise committedUsd by observed usage and are delivered where they belong, the settlement of a reservation (AIFACTORY-US-455) and a usage record of a run whose reservation is Settled, Released or Expired (AIFACTORY-US-455, AIFACTORY-US-459); settlement corrects the amount counted for the run, its estimate or its observed cost where that already exceeded the estimate, to the settled sum. Whenever one of these writes, the excess of a running run, a settlement or a record after the reservation came to rest, raises committedUsd and leaves it above limitUsd, the BudgetExceeded outbox record is written in the same transaction with budgetId, tenantId, projectId, committedUsd, limitUsd and the runId of the run of that usage; runId is absent when the usage names no run, which is the usage of an invocation outside a run that AIFACTORY-US-881 counts against the budget of its project (D123). The event is published on usage-accounting.budgets.v1.budget-exceeded in the run event envelope. RunControlService consumes it and records a cancellation intent for the run with the reason budget-exceeded, so that the run follows the cancellation path and ends Cancelled with that reason; the stop is no run failure and the run is never Failed (D123). A run that is already terminal is left as it is there. A write that does not raise committedUsd, and a write that leaves it at or below the limit, publishes nothing, so a run that stays within its estimate is never stopped by the overrun of another run: the running runs of the budget that stay within their estimates keep their reservations, finish and settle as always. The event is published once per run (D119): a further write of observed usage for a run that already has one, as the later records of a run that is being cancelled, still raises committedUsd and writes no second event. A second run that takes its own cost above its own estimate while the budget is already above the limit is a run whose observed usage raised the committed amount of a budget that is above its limit, so it is named once and stopped too, and so is every later run of that kind (D121). The overrun itself is never hidden: the observed cost stays in committedUsd, and while the amount is above the limit every reservation against the budget is Denied with BudgetReservationDenied and the reason budget-exceeded (AIFACTORY-US-453), until the limit is raised through SetProjectBudget or the rollover of a monthly budget opens a new period (AIFACTORY-US-452). The confirmation of a record by the ReconciliationWorker corrects the budget as AIFACTORY-US-456 states and publishes UsageReconciled only, since the model lists BudgetExceeded as a result of recording and of settlement and not of reconciliation.

Acceptance criteria

AC1 Given a run with a Reserved CostUsageService/Reservation and usage records whose cost stays within its estimate, when a further CostUsageService/UsageRecord of the run is recorded and the sum is still within the estimate, then committedUsd of the CostUsageService/Budget is unchanged and no CostUsageService/BudgetExceeded is written; when a record takes the sum above the estimate, then committedUsd grows by the part above the estimate that was not yet counted, in the transaction of the record, and an Uncertain record, which has no cost, adds nothing (SCN-CostUsageService-013, XL-12).
AC2 Given that growth takes committedUsd above limitUsd, when the transaction of the record commits, then the CostUsageService/BudgetExceeded outbox record with budgetId, tenantId, projectId, the runId of the run of that record, committedUsd and limitUsd is committed with it; given a usage record of a run whose reservation is Settled, Released or Expired, or a settlement, that raises committedUsd and leaves it above limitUsd, then the record is written the same way; given a write that does not raise committedUsd or leaves it at or below limitUsd, then nothing is written; given a further write of observed usage for a run that already has its CostUsageService/BudgetExceeded record, then committedUsd grows and no second record is written for that run; and given a second run of the budget with a Reserved CostUsageService/Reservation whose usage stays within its estimate, then no record names it and it settles as always; and given a second run whose usage record takes its own cost above its own estimate while committedUsd is already above limitUsd, then a CostUsageService/BudgetExceeded record names that run too, once (SCN-CostUsageService-013, D119, D121).
AC3 Given a budget whose committedUsd is above limitUsd, when the next run of the project is reserved through ReserveForRun or CostUsageService/Usage.Reserve, then its reservation is Denied with CostUsageService/BudgetReservationDenied and the reason budget-exceeded, and the same holds for an Approved spend approval; when the limit is raised through CostUsageService/SetProjectBudget above the committed amount, or the rollover of a monthly budget opened a new period, then a reservation that fits is Reserved again (SCN-CostUsageService-013, FR-09).
AC4 Given the record was written, when the outbox dispatcher of the cost-usage messaging profile publishes CostUsageService/BudgetExceeded on usage-accounting.budgets.v1.budget-exceeded, then the event is sent in run-event-envelope-v1 with budgetId, tenantId, projectId, runId, committedUsd and limitUsd, runId being absent when the usage names no run, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-CostUsageService-013, ADR-0012).
Traces: services CostUsageService, RunControlService
Traces: aggregates CostUsageService/Budget, CostUsageService/Reservation, CostUsageService/UsageRecord
Traces: apis CostUsageService/Usage.RecordUsage, CostUsageService/Usage.Reserve, CostUsageService/SetProjectBudget
Traces: events CostUsageService/BudgetExceeded, CostUsageService/BudgetReservationDenied
Traces: descriptors run-event-envelope-v1
Traces: adrs ADR-0012, ADR-0014, ADR-0020
Traces: requirements FR-09, XL-12, NFR-12
Traces: scenarios SCN-CostUsageService-013
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-453, AIFACTORY-US-454, AIFACTORY-US-455, feature:run-control-service
Sources model/services/control-plane/CostUsageService.yaml, model/scenarios/CostUsageService.yaml, model/services/control-plane/RunControlService.yaml

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-2203 Excess of a running run counted against the budget and BudgetExceeded written by every write of observed usage that leaves the budget above its limit implementation The command that creates a usage record adds to committedUsd of the budget of the project, in the transaction of the record and under the row version of the Budget, the part of the cost of the records of the run above the estimate of its Reserved reservation that was not yet counted, so that such a run counts with the larger of its estimate and its observed cost; a record within the estimate and an Uncertain record change no budget; the three writes of observed usage, this excess, the settlement of AIFACTORY-TASK-1836 and the correction of AIFACTORY-TASK-1850, write the BudgetExceeded outbox record with budgetId, tenantId, projectId, the runId of the usage when it names a run, committedUsd and limitUsd whenever they raise committedUsd and leave it above limitUsd, and write nothing otherwise; the record is written once per run, so a later write of observed usage for a run that already has one raises committedUsd and writes none, and no write touches the reservation of another run (D119); a second run whose own excess raises committedUsd while it is above limitUsd gets its own record, once, and so does every later one, while a run within its estimate gets none (D121); for the usage of a child run the estimate compared is that of the reservation of its parent run and the record names the child run, as AIFACTORY-TASK-2976 delivers; a reservation against a budget above its limit is Denied with the reason budget-exceeded. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2204 Verify announce-a-budget-overrun through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-CostUsageService-013"; a filter that matches no test counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-549 with fixture publishers of ModelInvoked and of the terminal events of a run, including records within the estimate that change no budget, the excess counted once, the BudgetExceeded record with the runId of the run that crossed the limit, no second record for further usage of that run, a second run within its estimate that is not named and settles, a further run that goes above its own estimate while the budget is above its limit and is named once too, a record after an Expired reservation that crosses the limit, the denied next reservation, a reservation that fits again after the limit was raised, and the publication and payload of BudgetExceeded; the cancellation of the run is proven by the suite of RunControlService. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-737: Reserve nothing for a child run, record its usage with the child run id and count it on the reservation of the parent run

behavior A run may start child runs: AgentRuntimeService submits one for a delegation and WorkflowEngineService one for a step of kind SubWorkflow, both through the internal operation SubmitChildRun of RunControlService. The owner decided how such a run is paid for (D121): a child run has no reservation of its own; its usage is recorded with the child run id and counts on the reservation of the parent run; and BudgetExceeded names the run whose usage crossed the limit, which may be a child run. This PBI delivers that in CostUsageService. RunControlService marks a child run by the optional parentRunId of RunControlService/RunAccepted, which is present only for a run accepted through SubmitChildRun, and it waits for no BudgetReserved and opens no spend approval for a child run. When the inbox handler ReserveForRun receives a RunAccepted that carries parentRunId, it estimates nothing and reserves nothing: no Reservation row is written for the child run, committedUsd of the budget is unchanged, and none of BudgetReserved, BudgetReservationDenied and SpendApprovalRequired is published. The handler notes the child run in the table ChildRuns of the model instead, with childRunId, parentRunId, the tenant, notedAt and the reservationId its usage counts on. That is the reservation of the parent run, and for the child of a child run it is the reservation its parent counts on, read from the ChildRuns row of that parent, so every run of a tree counts on the one reservation of its root. The note is idempotent on childRunId, and a redelivered event writes nothing. The row requires a reservation id, so a child run is noted only under a reservation that exists. When the service holds neither a reservation nor a ChildRuns row for the run that parentRunId names, which can happen when the events of a tree arrive out of order, the handler does not acknowledge the delivery and the inbox redelivers it with backoff; the model does not state that case, and it is reported to the owner. From then on the usage of the child run is recorded as AIFACTORY-US-454 states, one UsageRecord per receipt and meter with the child run id as its runId, by Usage.RecordUsage and by the handlers of the receipt events alike, and it counts on the reservation of the parent run. Wherever the cost of the usage records of a run is compared with the estimate of its Reserved reservation (AIFACTORY-US-549), the cost compared is that of the records of the run of the reservation and of every child run noted under it, taken together. As long as that sum stays within the estimate, a record of a child run changes no budget, because the estimate of the parent run already holds that room. When a record takes the sum above the estimate, the part above the estimate that was not yet counted is added to committedUsd in the transaction of the record. When the reservation is no longer Reserved, the cost of the record is added in full, as for any record that arrives after its reservation came to rest (AIFACTORY-US-455). A usage record of a child run that arrives before its note is recorded and changes no budget, having no reservation to count on yet; the handler that writes the note then counts the cost of the records the child run already has, by the same rule and in the transaction of the note. When one of these writes raises committedUsd and leaves it above limitUsd, BudgetExceeded is written with the runId of the run of that usage, so it names the child run when the usage is that of the child run, once per run; RunControlService stops only the run the event names, which ends Cancelled with the reason budget-exceeded and is no failed run, and only the parent run treats that child as failed (D123). A write for the usage of a child run never names the parent run. The terminal events of a child run, RunCompleted, RunFailed and RunCancelled, settle and release nothing, because the child run has no reservation: the handlers acknowledge them without effect. The usage of the child run is settled with the reservation of the parent run: when the terminal event of the parent run arrives, settledUsd is the sum of the cost of the usage records of the parent run and of the child runs noted under the reservation, and uncertain is true while one of those records is Uncertain or Estimated. A usage record of a child run also counts as usage of that reservation where a release or an expiry is decided, so the reservation of a parent run whose child run has spent is neither released by SettleOrRelease nor expired by the ReservationExpiryWorker (AIFACTORY-US-455, AIFACTORY-US-459). Three points the decision and the model leave unsaid are reported to the owner and not decided here: what the handler RecordAttemptUsage notes for the attempt outcome of a child run, which has no reservation, so this delivery leaves the reservation of the parent run unchanged for it; what GetRunUsage returns as the reservation of a child run, so the read of AIFACTORY-US-457 is unchanged and returns the usage records of the child run without a reservation; and what Usage.Reserve answers for the run id of a child run, since its request names no parent. The table ChildRuns is migrated here with its index on the reservation and carries the tenant filter in its mapping.

Acceptance criteria

AC1 Given a parent run with a Reserved CostUsageService/Reservation, when RunControlService/RunAccepted with the parentRunId of that run is handled by ReserveForRun, then no reservation is written for the child run, committedUsd of the CostUsageService/Budget is unchanged, none of CostUsageService/BudgetReserved, CostUsageService/BudgetReservationDenied and CostUsageService/SpendApprovalRequired is written, and one ChildRuns row holds childRunId, parentRunId and the reservationId of the parent run; given a child of that child run, then its row holds the same reservationId; and a redelivery of either event writes nothing (D121, model/services/control-plane/CostUsageService.yaml).
AC2 Given a child run noted under the reservation of its parent run, when a receipt of the child run is recorded through CostUsageService/Usage.RecordUsage or a receipt event, then one CostUsageService/UsageRecord per meter exists with the child run id as its runId and CostUsageService/UsageRecorded is written once per record; while the cost of the records of the parent run and of its child runs together stays within the estimate of the reservation, committedUsd is unchanged; when a record of the child run takes that sum above the estimate, then committedUsd grows by the part above the estimate that was not yet counted, in the transaction of the record (D121, XL-12).
AC3 Given that growth leaves committedUsd above limitUsd, when the transaction of the record of the child run commits, then one CostUsageService/BudgetExceeded outbox record with the runId of the child run is committed with it, no record names the parent run, and a further usage record of the child run raises committedUsd and writes no second record; given a later usage record of the parent run that raises committedUsd while it is above limitUsd, then a record names the parent run, once (D121, model/services/control-plane/CostUsageService.yaml).
AC4 Given a child run with usage records, when RunControlService/RunCompleted, RunControlService/RunFailed or RunControlService/RunCancelled of the child run is handled, then no reservation changes its state and no CostUsageService/ReservationSettled is written; when the terminal event of the parent run is handled, then its reservation is Settled with settledUsd equal to the sum of the cost of the usage records of the parent run and of its child runs, and CostUsageService/ReservationSettled is written once, with uncertain true while one of those records is Uncertain or Estimated; and given a parent run without usage records of its own whose child run has one, when the ReservationExpiryWorker runs past expiresAt, then the reservation stays Reserved, and when RunControlService/RunCancelled of the parent run arrives, then it is Settled and not Released (D121, FR-09).
AC5 Given RunControlService/RunAccepted of a child run whose parent run has neither a reservation nor a ChildRuns row yet, when it is handled, then nothing is written and the delivery is not acknowledged, and after the reservation of the parent run exists the redelivered event writes the note once; given usage records of a child run that were recorded before its note, when the note is written, then their cost is counted on the reservation of the parent run in the transaction of the note by the rule of AC2 (model/services/control-plane/CostUsageService.yaml).
Traces: services CostUsageService, RunControlService
Traces: aggregates CostUsageService/Budget, CostUsageService/Reservation, CostUsageService/UsageRecord
Traces: apis CostUsageService/Usage.RecordUsage
Traces: events RunControlService/RunAccepted, RunControlService/RunCompleted, RunControlService/RunFailed, RunControlService/RunCancelled, CostUsageService/BudgetReserved, CostUsageService/BudgetReservationDenied, CostUsageService/SpendApprovalRequired, CostUsageService/UsageRecorded, CostUsageService/BudgetExceeded, CostUsageService/ReservationSettled
Traces: descriptors usage-record-v1, run-event-envelope-v1
Traces: adrs ADR-0014, ADR-0020
Traces: requirements FR-09, XL-12, NFR-12
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-453, AIFACTORY-US-454, AIFACTORY-US-455, AIFACTORY-US-459, AIFACTORY-US-549, feature:run-control-service
Sources model/services/control-plane/CostUsageService.yaml, model/services/control-plane/RunControlService.yaml, model/scenarios/CostUsageService.yaml

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-2974 Migration and mapping of the table ChildRuns in cost-usage-sql migration The table ChildRuns of model/services/control-plane/CostUsageService.yaml exists with the columns ChildRunId as key, TenantId, ParentRunId, ReservationId as a foreign key to Reservations and NotedAt, and with the index IX_ChildRuns_Reservation; its mapping carries the tenant filter of the Factory profile; mapping tests run against an isolated test database, never a shared one. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2975 ReserveForRun noting a child run under the reservation of its parent run and reserving nothing for it implementation For a RunAccepted that carries parentRunId the handler ReserveForRun runs no estimate and no reservation command, writes no Reservation row and none of BudgetReserved, BudgetReservationDenied and SpendApprovalRequired, and writes one ChildRuns row with childRunId, parentRunId, the tenant, notedAt and the reservationId of the parent run, or the reservationId of the ChildRuns row of the parent run when the parent run is itself a child run; the note is idempotent on childRunId; when the service holds neither a reservation nor a ChildRuns row for the parent run the handler writes nothing and does not acknowledge the delivery, so the inbox redelivers it with backoff; in the transaction of the note the cost of usage records the child run already has is counted by the rule of AIFACTORY-TASK-2976 (D121). ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2976 Usage of a child run counted, announced and settled on the reservation of its parent run implementation The command that creates a usage record resolves the reservation a run counts on, its own or, for a run noted in ChildRuns, the one of that row, and applies the rules of AIFACTORY-TASK-2203 and AIFACTORY-TASK-1850 to it with the cost of the records of the run of the reservation and of every child run noted under it taken together, so that a record of a child run within the estimate changes no budget, the uncounted part above the estimate is added to committedUsd in the transaction of the record, and a record under a reservation that is no longer Reserved is added in full; the BudgetExceeded outbox record carries the runId of the run of the usage, the child run for the usage of a child run, once per run; SettleReservation and SettleOrRelease move no reservation on a terminal event of a child run, sum the usage records of the parent run and of its noted child runs into settledUsd and into the uncertain flag when the parent run ends, and treat a usage record of a noted child run as usage of the reservation, as the ReservationExpiryWorker does (D121). ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2977 Acceptance fixtures of a run tree with child runs, their receipts and their terminal events fixture The acceptance suite holds a fixture publisher that sends RunAccepted with a parentRunId for a child run and for the child of a child run, in order and out of order, receipts that name the child run id through Usage.RecordUsage and through the receipt events with costs that stay within and that exceed the estimate of the reservation of the parent run, and RunCompleted, RunFailed and RunCancelled for a child run and for its parent run, so that the note, the counting, the BudgetExceeded record that names the child run and the settlement of the parent run are reproducible without RunControlService. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-2978 Verify count-child-run-usage-on-the-parent-reservation through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Feature=ChildRunUsage"; a filter that matches no test counts as a failure; the tests prove AC1 to AC5 of AIFACTORY-US-737: no reservation and none of the three reservation events for a RunAccepted with parentRunId, the ChildRuns row of a child run and of the child of a child run on the reservation of the root, the redelivery that writes nothing, usage recorded with the child run id that changes no budget within the estimate of the parent run and raises committedUsd above it, CostUsageService/BudgetExceeded naming the child run once and the parent run only for its own usage, terminal events of a child run that move no reservation, the settlement of the parent run over its own usage and that of its child runs, a parent reservation with child usage that is neither released nor expired, the unacknowledged note of a child run whose parent run is not known yet and the cost of earlier records counted when the note is written; the stop of the named run is proven by the suite of RunControlService. ConnectSoft.AI.SoftwareFactory.CostUsageService

AIFACTORY-US-881: Record the usage of an invocation that names no run against the budget of its project, without a reservation, and count it in the committed amount

behavior Not every model call belongs to a run. ModelRouterService serves the calls of the OpenAI-compatible route of the model gateway, and such a call is of one of two kinds (D123). A run-bound call carries the run and the attempt of its token, and its usage counts on the reservation of the run as AIFACTORY-US-454 and AIFACTORY-US-549 state. A call outside a run names no run and no attempt, and its receipt carries the tenant and the project the caller named. The owner decided how the second kind is paid for (D123), and the model of CostUsageService states it: the usage of an invocation that names no run is recorded against the budget of its project without a reservation, and it counts in committedUsd of that budget. This PBI delivers that. The receipt reaches the service on the two paths of AIFACTORY-US-454, which stay one command: CostUsageService/Usage.RecordUsage with a usage-record-v1 that carries no runId, which the descriptor makes optional, and the inbox handler RecordModelUsage for a ModelRouterService/ModelInvoked whose runId and attemptId are absent. The record is written as AIFACTORY-US-454 states, once per receipt and meter, the input tokens and the output tokens of the call on their two meters, each priced by the revision valid at occurredAt, with its UsageRecorded outbox record in the same transaction. It carries the tenant and the project and neither a run nor an attempt, so the column RunId of UsageRecords stays empty, and it belongs to no reservation, so the request carries no reservationId. Nothing is estimated or reserved for the call: no Reservation row is written, none of BudgetReserved, BudgetReservationDenied and SpendApprovalRequired is published, and no spend approval is asked for. The cost of the record is instead added to committedUsd of the budget of the project in the transaction of the record and under the row version of the Budget, in full, because no estimate holds room for it. The cost of a record is its confirmed cost when it has one and its estimated cost otherwise; an Uncertain record has no cost and adds nothing until it is priced or reconciled. When the ReconciliationWorker confirms or prices such a record (AIFACTORY-US-456), committedUsd is corrected in the transaction of the confirmation by the difference between the confirmed cost and the cost counted for the record so far, because no settlement will ever sum it. No terminal event of a run touches the record, no settlement counts it, and the ReservationExpiryWorker has nothing to expire for it. The rollover of a monthly budget (AIFACTORY-US-452) closes the period with the amount that holds this usage in BudgetPeriods and opens the new period with the estimates of the reservations still Reserved only, so usage outside a run counts in the period in which it was recorded. The usage is read with the figures of its project through GetProjectUsage, by meter and state like any other record (AIFACTORY-US-457), and no read of a run returns it. Usage.RecordUsage records what was consumed also when the budget is already above its limit, since the consumption has happened and no receipt is refused; this service offers no check before a call outside a run, and none is added here. The budget still holds its limit for what comes next. When a record outside a run raises committedUsd and leaves it above limitUsd, the BudgetExceeded outbox record is written in the same transaction with budgetId, tenantId, projectId, committedUsd and limitUsd and without a runId, which the model makes optional for exactly this case. The event names no run, so no run is stopped by it, and from then on every reservation against the budget is Denied with the reason budget-exceeded until the limit is raised or a new period opens (AIFACTORY-US-549, AIFACTORY-US-453). The model states the rule of one event per run for runs only. For usage outside a run this PBI writes the event when a write takes committedUsd from at or below limitUsd to above it and writes none for further usage outside a run while the amount stays above the limit, because there is no run to name; that reading is reported to the owner. A running run of the same budget is named by no such event and is stopped only by its own usage, as AIFACTORY-US-549 states. Over gRPC the refusals of AIFACTORY-US-454 hold unchanged, a project of another tenant being answered NOT_FOUND.

Acceptance criteria

AC1 Given a project with a CostUsageService/Budget and pricing revisions valid at occurredAt, when CostUsageService/Usage.RecordUsage is called by ModelRouterService with a usage-record-v1 that carries no runId, or ModelRouterService/ModelInvoked without runId and attemptId is handled, then one CostUsageService/UsageRecord per meter exists with the tenant, the project, no run and no attempt, one CostUsageService/UsageRecorded outbox record per usage record is written in the same transaction, and committedUsd of the budget of the project has grown by the cost of those records in that transaction (D123, model/services/control-plane/CostUsageService.yaml).
AC2 Given that receipt was recorded, when the tables and the outbox are inspected, then no CostUsageService/Reservation exists for it and none of CostUsageService/BudgetReserved, CostUsageService/BudgetReservationDenied and CostUsageService/SpendApprovalRequired was written; when the same receipt arrives again by either path, then no record is added and committedUsd is unchanged; given an Uncertain record outside a run, then it adds nothing to committedUsd; and when the ReconciliationWorker confirms a record outside a run, then committedUsd is corrected by the difference between the confirmed cost and the cost counted for the record so far, in the transaction of the confirmation (D123, FR-09).
AC3 Given a budget whose committedUsd is at or below limitUsd, when a usage record outside a run raises it above limitUsd, then one CostUsageService/BudgetExceeded outbox record with budgetId, tenantId, projectId, committedUsd and limitUsd and without runId is committed with the record; a running run of the same budget that stays within its estimate is named by no record and settles as always; further usage outside a run raises committedUsd and writes no further record while the amount stays above the limit; and the next reservation of the project through ReserveForRun or CostUsageService/Usage.Reserve is Denied with CostUsageService/BudgetReservationDenied and the reason budget-exceeded (D123, XL-12).
AC4 Given usage records outside a run and usage records of a run in one project, when CostUsageService/GetProjectUsage is read, then its figures by meter and state hold both and committedUsd holds the cost of the records outside a run; when CostUsageService/GetRunUsage is read for that run, then it returns the records of the run only; when the terminal event of that run is handled and when the ReservationExpiryWorker runs, then no record outside a run is settled, released or expired; and when the monthly budget rolls over, then the closed period in BudgetPeriods holds the amount with that usage and the new period starts without it (D123, model/services/control-plane/CostUsageService.yaml).
Traces: services CostUsageService, ModelRouterService
Traces: aggregates CostUsageService/Budget, CostUsageService/UsageRecord, CostUsageService/Reservation
Traces: apis CostUsageService/Usage.RecordUsage, CostUsageService/Usage.Reserve, CostUsageService/GetProjectUsage, CostUsageService/GetRunUsage
Traces: events ModelRouterService/ModelInvoked, CostUsageService/UsageRecorded, CostUsageService/BudgetExceeded, CostUsageService/BudgetReserved, CostUsageService/BudgetReservationDenied, CostUsageService/SpendApprovalRequired
Traces: descriptors usage-record-v1, run-event-envelope-v1
Traces: adrs ADR-0014, ADR-0020
Traces: requirements FR-09, XL-12, NFR-12
Relates: nfrs NFR-12
Depends on AIFACTORY-US-450, AIFACTORY-US-451, AIFACTORY-US-452, AIFACTORY-US-453, AIFACTORY-US-454, AIFACTORY-US-456, AIFACTORY-US-457, AIFACTORY-US-549, feature:model-router-service
Sources model/services/control-plane/CostUsageService.yaml, model/services/model-compute/ModelRouterService.yaml, model/scenarios/CostUsageService.yaml, docs/reference/schemas/usage-record.schema.json

Tasks

Task Kind Expected output Repository
AIFACTORY-TASK-3552 Contract of Usage.RecordUsage for a usage record that names no run contract The contract document and the proto of Usage.RecordUsage state, against the UsageRecord definition of docs/reference/schemas/usage-record.schema.json, that runId is optional (D123), that a request without runId carries no attemptId and no reservationId, that such usage is counted against the budget of its project without a reservation, and that ack-v1 answers it as it answers any record; the golden files of the contract test hold a request with a run and a request without one, and the consumer test of ModelInvoked reads an event of the pinned contracts package whose runId and attemptId are absent. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-3553 Nullable RunId of UsageRecords in the migration and the mapping migration The column RunId of UsageRecords in cost-usage-sql is nullable, as model/services/control-plane/CostUsageService.yaml lists it (D123), the mapping of UsageRecord reads and writes a record without a run and without an attempt, and the reads that select the usage of one run never return a record without a run; mapping tests on an isolated test database prove a record with a run and a record without one. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-3554 Usage outside a run counted against the budget of its project without a reservation implementation The command of AIFACTORY-TASK-311 that creates a usage record accepts a receipt without a run from Usage.RecordUsage and from the handler RecordModelUsage, writes the record with tenant, project and neither run nor attempt together with its UsageRecorded outbox record, resolves no reservation and writes none, and adds the cost of the record in full to committedUsd of the budget of the project in the same transaction and under the row version of the Budget; an Uncertain record adds nothing; the same receipt again changes nothing; the confirmation of such a record by the ReconciliationWorker corrects committedUsd by the difference between the confirmed cost and the cost counted so far, in the transaction of the confirmation; a write that takes committedUsd from at or below limitUsd to above it writes the BudgetExceeded outbox record with budgetId, tenantId, projectId, committedUsd and limitUsd and without runId, and further usage outside a run writes none while the amount stays above the limit; settlement, release and expiry never select a record without a run, and a reservation against a budget above its limit is Denied with the reason budget-exceeded. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-3555 Acceptance fixtures of receipts that name no run fixture The acceptance suite holds a fixture publisher that sends ModelInvoked without runId and attemptId for a stated tenant and project, built from the pinned contracts package, and requests of Usage.RecordUsage without runId, with reported, estimated and unknown usage and with costs that stay below and that cross the limit of the budget of the project, beside the receipts of a run of the same project, so that the counting, the BudgetExceeded record without a run and the reads are reproducible without ModelRouterService. ConnectSoft.AI.SoftwareFactory.CostUsageService
AIFACTORY-TASK-3556 Verify record-usage-outside-a-run through the acceptance suite verification Run dotnet test -c Release --filter "Category=Acceptance&Feature=UsageOutsideARun"; a filter that matches no test counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-881: a receipt without a run recorded by both paths with no run and no attempt, committedUsd grown by its cost in the transaction of the record, no reservation and none of the three reservation events, the repeated receipt that changes nothing, an Uncertain record that adds nothing and its confirmation that corrects the budget, CostUsageService/BudgetExceeded written once without a runId when the limit is crossed and not again while the amount stays above it, a running run of the same budget that is not named, the denied next reservation, GetProjectUsage holding the usage and GetRunUsage not returning it, and a rollover that keeps the usage in the closed period. ConnectSoft.AI.SoftwareFactory.CostUsageService