AgentRuntimeService¶
Target
This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.
| Bounded context | BC-06 Execution |
| Platform / plane | Agent Mesh / Agent & Orchestration runtime |
| Deployment granularity | module of host · host AgentRuntimeWorker |
| Status | target · ConnectSoft.Factory.Runtime.Execution |
| Built from | connectsoft-ai-agent · libraries: connectsoft-extensions-ai, connectsoft-extensions-ai-agents-framework, connectsoft-extensions-ai-factory, connectsoft-extensions-model-context-protocol, connectsoft-extensions-messaging-model, connectsoft-extensions-persistence-model-nhibernate |
| Decisions | ADR-0014, ADR-0021, ADR-0022, ADR-0023 |
| Traceability | FR-03 FR-05 FR-06 FR-07 FR-10 FUX-04 FUX-05 FUX-07 NFR-05 NFR-06 NFR-11 NFR-13 XL-03 XL-04 XL-05 XL-06 XL-10 |
Purpose and responsibilities¶
AgentRuntimeService is where the work happens. For every attempt Run Control dispatches it loads and verifies the pinned release, asks Knowledge for a context snapshot within the token budget, and runs the loop of the original design (resolve → build context → plan → model call → tool call → observe → update state → next step → verify → result) on the execution adapter. It writes a checkpoint before every gated call and after every observed outcome, stops for human decisions, delegates sub-agents as child runs when the task justifies it, and reports exactly one outcome with a hash and the lease's fencing token.
It absorbs the June-2026 rows AgentExecutionService (attempt records), AgentValidationService (output validation) and AgentCorrectionService (bounded self-correction): the three were phases of one loop, not services. Lifecycle authority stays with Run Control (BC-03); model routing with Model & Compute (BC-07); tool execution and sandboxes with Tools & MCP (BC-08) and Execution Capacity (BC-26).
Context and boundaries¶
flowchart LR
S["AgentRuntimeService"]
RunControlService["RunControlService"] -->|grpc| S
StudioBff["StudioBff"] -->|sync-http| S
S -->|grpc| RunControlService["RunControlService"]
S -->|grpc| AgentRegistryService["AgentRegistryService"]
S -->|grpc| ModelRouterService["ModelRouterService"]
S -->|grpc| ToolAdapterService["ToolAdapterService"]
S -->|grpc| ContextBuilderService["ContextBuilderService"]
S -->|grpc| ArtifactService["ArtifactService"]
S -->|grpc| PolicyEngineService["PolicyEngineService"]
S -->|grpc| AgentPoolManager["AgentPoolManager"]
S -. "AttemptStarted" .-> E_AttemptStarted(("event"))
S -. "AttemptPhaseChanged" .-> E_AttemptPhaseChanged(("event"))
S -. "AttemptCheckpointed" .-> E_AttemptCheckpointed(("event"))
S -. "AttemptDecisionRequested" .-> E_AttemptDecisionRequested(("event"))
S -. "AttemptCancellationObserved" .-> E_AttemptCancellationObserved(("event"))
S -. "AttemptOutcomeReported" .-> E_AttemptOutcomeReported(("event"))
S -. "SubAgentDelegated" .-> E_SubAgentDelegated(("event"))
S -. "ReleaseLoaded" .-> E_ReleaseLoaded(("event"))
S -. "ReleaseLoadRejected" .-> E_ReleaseLoadRejected(("event"))
What the runtime may never do:
- Complete, fail or cancel a run on its own authority; it reports an attempt outcome and Run Control decides.
- Call a model, a tool or an external system outside the gateway and the executor; the executor enforces the side-effect class and the policy decision (D21).
- Change the snapshot: release, policy revision, model profile and context versions are fixed at acceptance; a deprecated release is finished or drained per its in-flight policy, never swapped.
- Retry an action whose effect is unconfirmed: an in-flight call at the moment of a crash or cancellation settles as
Uncertainand is shown to an operator with its receipt.
Domain model¶
ExecutionAttempt — One try of one step under one lease; the unit the runtime owns end to end. Holds the fencing token, the immutable snapshot reference, the loop phase, checkpoints, every call record, decision requests and the single reported outcome. (catalog entry)
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | Issued by Run Control with the lease. |
runId |
guid |
yes | |
stepId |
guid |
yes | |
tenantId |
string |
yes | |
projectId |
string |
yes | |
environmentId |
string |
no | |
leaseId |
guid |
yes | |
fencingToken |
long |
yes | Monotonic token from the lease; every outbound call and the outcome carry it. |
snapshotId |
guid |
yes | Resolved execution snapshot (resolved-execution-snapshot-v1). |
releaseId |
string |
yes | |
agentId |
string |
yes | |
modelProfileId |
string |
yes | |
state |
enum:AttemptState |
yes | |
phase |
enum:AttemptPhase |
yes | |
correctionCount |
int |
yes | |
maxCorrections |
int |
yes | From the workflow step or the agent contract (agent-task-contract-v1). |
lastCheckpointId |
guid |
no | |
contextSnapshotRef |
string |
no | context-snapshot-v1 reference produced by Knowledge (BC-09). |
outcomeHash |
hash |
no | Hash of the reported outcome (attempt-outcome-v1); the report is idempotent on it. |
cancellationObservedAt |
datetime |
no | |
startedAt |
datetime |
yes | |
endedAt |
datetime |
no | |
version |
long |
yes |
Invariants:
- An attempt exists only for a lease issued by Run Control; its fencing token accompanies every outbound call and the outcome report.
- Every call of class shared-write or above goes through the Tools and MCP executor with the policy decision id; the runtime never opens a connection to an external system itself.
- A checkpoint is written before every gated call and after every observed outcome; resumption happens only from a checkpoint whose adapter and checkpoint-format versions are compatible, otherwise Run Control starts a new attempt.
- correctionCount never exceeds maxCorrections; exhaustion ends the attempt as Failed with class validation, never as a silent success.
- Exactly one outcome is reported per attempt; a repeated report with the same outcomeHash is acknowledged, a different one is rejected.
- After cancellation is observed no further model or tool call is made; a call already in flight settles as Uncertain when its effect cannot be confirmed.
stateDiagram-v2
Claimed --> Running : ReleaseLoadedAndContextBuilt
Running --> WaitingForTool : ToolCallIssued
WaitingForTool --> Running : ToolResultObserved
Running --> WaitingForDecision : DecisionRequested
WaitingForDecision --> Running : DecisionReceived
Running --> Paused : ControlPlaneUnreachable
Paused --> Running : LeaseRenewed
Running --> Succeeded : OutputVerified
Running --> Failed : CorrectionsExhaustedOrTerminalError
Running --> TimedOut : AttemptBudgetExceeded
WaitingForDecision --> Failed : DecisionRejectedOrExpired
Running --> Cancelled : CancellationObserved
WaitingForTool --> Cancelled : CancellationObserved
WaitingForDecision --> Cancelled : CancellationObserved
WaitingForTool --> Uncertain : EffectUnconfirmed
Events: AttemptStarted, AttemptPhaseChanged, AttemptCheckpointed, AttemptDecisionRequested, AttemptCancellationObserved, AttemptOutcomeReported, SubAgentDelegated · Tables: ExecutionAttempts, Checkpoints, CallRecords, DecisionRequests, SubAgentDelegations · Repository: ExecutionAttemptRepository
LoadedRelease — Runtime read model of a Published release verified against its manifest hashes and the adapter compatibility of this worker; never a write path to publication. (catalog entry)
| Field | Type | Required | Description |
|---|---|---|---|
releaseId |
string |
yes | |
tenantId |
string |
yes | |
manifestHash |
hash |
yes | |
adapterVersion |
string |
yes | Runtime adapter and checkpoint format the release was verified against. |
contentRefs |
json |
yes | Artifact references of agent instructions, skill contents, tool descriptors, workflow definition, policy revision, model profile. |
state |
enum:LoadState |
yes | |
verifiedAt |
datetime |
yes | |
inFlightPolicy |
string |
no | finish |
Invariants:
- A release is usable only in state Verified; a hash mismatch or an incompatible adapter rejects it with an explicit reason.
- Revocation applies the in-flight policy to running attempts of that release and never deletes the manifest the attempts already loaded.
Events: ReleaseLoaded, ReleaseLoadRejected · Tables: LoadedReleases · Repository: LoadedReleaseRepository
erDiagram
ExecutionAttempt {
guid attemptId PK
guid runId
guid stepId
string tenantId
string projectId
string environmentId
guid leaseId
long fencingToken
guid snapshotId
string releaseId
string agentId
string modelProfileId
enum_AttemptState state
enum_AttemptPhase phase
int correctionCount
int maxCorrections
guid lastCheckpointId
string contextSnapshotRef
hash outcomeHash
datetime cancellationObservedAt
datetime startedAt
datetime endedAt
long version
}
LoadedRelease {
string releaseId PK
string tenantId
hash manifestHash
string adapterVersion
json contentRefs
enum_LoadState state
datetime verifiedAt
string inFlightPolicy
}
Checkpoint {
guid checkpointId
int sequence
enum_AttemptPhase phase
string adapterStateRef
string adapterVersion
string contextSnapshotRef
hash hash
datetime createdAt
}
CallRecord {
guid callId
int sequence
enum_CallKind kind
string target
hash requestHash
guid policyDecisionId
string receiptRef
string status
datetime startedAt
datetime endedAt
}
DecisionRequest {
guid decisionId
enum_DecisionKind kind
guid approvalId
hash proposalHash
json options
string recommendation
string status
datetime requestedAt
datetime answeredAt
}
SubAgentDelegation {
guid delegationId
guid childRunId
string childAgentId
string purpose
int depth
string status
}
AttemptLimits {
int maxMinutes
int maxModelCalls
int maxToolCalls
int maxContextTokens
int maxSubAgentDepth
}
ExecutionAttempt ||--o{ Checkpoint : contains
ExecutionAttempt ||--o{ CallRecord : contains
ExecutionAttempt ||--o{ DecisionRequest : contains
ExecutionAttempt ||--o{ SubAgentDelegation : contains
stateDiagram-v2
[*] --> Claimed: AttemptDispatched with lease
Claimed --> Running: release verified, context built
Running --> WaitingForTool: gated or long tool call issued
WaitingForTool --> Running: result observed
Running --> WaitingForDecision: approval, input or escalation needed
WaitingForDecision --> Running: decision received
Running --> Paused: control plane unreachable
Paused --> Running: lease renewed
Running --> Succeeded: output verified
Running --> Failed: corrections exhausted or terminal error
Running --> TimedOut: attempt budget exceeded
WaitingForDecision --> Failed: rejected or expired
Running --> Cancelled: cancellation observed
WaitingForTool --> Uncertain: effect unconfirmed
Succeeded --> [*]
Failed --> [*]
TimedOut --> [*]
Cancelled --> [*]
Uncertain --> [*]
The loop phases (Resolving, LoadingRelease, BuildingContext, Planning, Calling, Observing, Verifying, Correcting, Checkpointing, Reporting) are recorded on every checkpoint so that a resumed attempt knows where it was and the Studio timeline shows where it is.
Entities and value objects¶
Checkpoint (entity, in ExecutionAttempt) — A durable point the attempt can resume from.
| Field | Type | Required | Description |
|---|---|---|---|
checkpointId |
guid |
yes | |
sequence |
int |
yes | |
phase |
enum:AttemptPhase |
yes | |
adapterStateRef |
string |
yes | Blob reference of the adapter checkpoint (opaque to the runtime). |
adapterVersion |
string |
yes | |
contextSnapshotRef |
string |
no | |
hash |
hash |
yes | |
createdAt |
datetime |
yes |
CallRecord (entity, in ExecutionAttempt) — One outbound call with its request hash and receipt reference.
| Field | Type | Required | Description |
|---|---|---|---|
callId |
guid |
yes | |
sequence |
int |
yes | |
kind |
enum:CallKind |
yes | |
target |
string |
yes | Model profile, tool id, child agent id or decision kind. |
requestHash |
hash |
yes | |
policyDecisionId |
guid |
no | |
receiptRef |
string |
no | model-invocation-v1 or tool-invocation-v1 receipt reference. |
status |
string |
yes | issued |
startedAt |
datetime |
yes | |
endedAt |
datetime |
no |
DecisionRequest (entity, in ExecutionAttempt) — A human decision the attempt waits for; approvals are owned by Run Control and referenced by approvalId.
| Field | Type | Required | Description |
|---|---|---|---|
decisionId |
guid |
yes | |
kind |
enum:DecisionKind |
yes | |
approvalId |
guid |
no | |
proposalHash |
hash |
no | |
options |
json |
no | |
recommendation |
string |
no | |
status |
string |
yes | pending |
requestedAt |
datetime |
yes | |
answeredAt |
datetime |
no |
SubAgentDelegation (entity, in ExecutionAttempt) — A child run delegated by a supervisor attempt; the child is a run of its own under the same scope and approval semantics.
| Field | Type | Required | Description |
|---|---|---|---|
delegationId |
guid |
yes | |
childRunId |
guid |
yes | |
childAgentId |
string |
yes | |
purpose |
string |
yes | |
depth |
int |
yes | |
status |
string |
yes | submitted |
AttemptLimits (value-object) — Budgets the attempt must respect.
| Field | Type | Required | Description |
|---|---|---|---|
maxMinutes |
int |
yes | |
maxModelCalls |
int |
yes | |
maxToolCalls |
int |
yes | |
maxContextTokens |
int |
yes | |
maxSubAgentDepth |
int |
yes |
Data model and storage¶
| Store | Role | Engine / profile | Tables |
|---|---|---|---|
| Runtime worker durable store | primary | azure-sql / nhibernate-sqlserver | ExecutionAttempts, Checkpoints, CallRecords, DecisionRequests, SubAgentDelegations, LoadedReleases |
| Artifact content store | blob | blob / azure-blob | — |
| Table | Owner | Columns | Indexes |
|---|---|---|---|
dbo.ExecutionAttempts |
ExecutionAttempt | 23 | 3 |
dbo.Checkpoints |
Checkpoint | 10 | 1 |
dbo.CallRecords |
CallRecord | 12 | 1 |
dbo.DecisionRequests |
DecisionRequest | 11 | 0 |
dbo.SubAgentDelegations |
SubAgentDelegation | 8 | 1 |
dbo.LoadedReleases |
LoadedRelease | 8 | 0 |
Full DDL: appendix.
Call records hold hashes and receipt references only; prompt text, model responses and tool payloads live in the artifact store under the tenant's retention and are never copied into runtime-sql (D31). The adapter checkpoint blob is opaque to the runtime: it is whatever the execution adapter needs to resume, versioned by adapterVersion.
APIs¶
| Method | Path | Operation | Scope | Side effects | Idempotency | Gateway | Summary |
|---|---|---|---|---|---|---|---|
GET |
/runtime/attempts/{attemptId} |
GetAttemptTimeline | factory.runs.read |
read | natural | GW-03 | Phases, checkpoints, calls (by hash and receipt reference, never content), decisions and the outcome of one attempt. |
GET |
/runtime/attempts/{attemptId}/checkpoints |
ListCheckpoints | factory.runs.read |
read | natural | GW-03 | Checkpoints with sequence, phase and adapter version; used by operators before a resume decision. |
GET |
/runtime/releases/{releaseId} |
GetLoadedRelease | factory.assets.read |
read | natural | GW-03 | What this worker verified for a release and against which adapter version. |
| gRPC service | Method | Streaming | Visibility | Side effects | Summary |
|---|---|---|---|---|---|
Runtime |
Interrupt |
none | internal | shared-write | Run Control asks the lease holder to stop an attempt (cancellation or revocation); the runtime records the request and reports the observed termination. |
Runtime |
AnswerDecision |
none | internal | shared-write | Deliver a human answer (input or escalation choice) to a waiting attempt; approvals arrive through ApprovalDecided instead. |
Runtime |
WatchAttempt |
server | internal | read | Stream phase changes and checkpoints of one attempt to the Studio BFF. |
Messages¶
| Event | Topic | Scope / v | Aggregate | Consumers | Summary |
|---|---|---|---|---|---|
| AttemptStarted | execution.attempts.v1.attempt-started |
integration / 1 | ExecutionAttempt | StudioBff, DashboardService, QualityScoreService | The lease holder started executing; release verified and context built. |
| AttemptPhaseChanged | execution.attempts.v1.attempt-phase-changed |
integration / 1 | ExecutionAttempt | StudioBff, DashboardService | The loop moved to another phase. |
| AttemptCheckpointed | execution.attempts.v1.attempt-checkpointed |
integration / 1 | ExecutionAttempt | DashboardService | A resumable checkpoint was written. |
| AttemptDecisionRequested | execution.attempts.v1.attempt-decision-requested |
integration / 1 | ExecutionAttempt | StudioBff, StudioNotificationService, DashboardService | The attempt waits for a human: an approval (owned by Run Control), missing input or an escalation with options. |
| AttemptCancellationObserved | execution.attempts.v1.attempt-cancellation-observed |
integration / 1 | ExecutionAttempt | StudioBff, DashboardService | The lease holder observed the interruption; in-flight calls settled or marked uncertain. |
| AttemptOutcomeReported | execution.attempts.v1.attempt-outcome-reported |
integration / 1 | ExecutionAttempt | MemoryPromotionService, DashboardService, QualityScoreService | The single outcome was reported to Run Control (which records the authoritative AttemptOutcomeRecorded). |
| SubAgentDelegated | execution.attempts.v1.sub-agent-delegated |
integration / 1 | ExecutionAttempt | StudioBff, DashboardService | A supervisor attempt submitted a child run. |
| ReleaseLoaded | execution.releases.v1.release-loaded |
integration / 1 | LoadedRelease | DashboardService | A release was verified for this adapter version. |
| ReleaseLoadRejected | execution.releases.v1.release-load-rejected |
integration / 1 | LoadedRelease | StudioBff, DashboardService, AuditService | Hash mismatch or incompatible adapter; attempts of that release fail with an explicit reason. |
AttemptStarted payload:
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | |
runId |
guid |
yes | |
stepId |
guid |
yes | |
tenantId |
string |
yes | |
releaseId |
string |
yes | |
snapshotId |
guid |
yes | |
fencingToken |
long |
yes |
AttemptPhaseChanged payload:
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | |
runId |
guid |
yes | |
tenantId |
string |
yes | |
phase |
enum:AttemptPhase |
yes | |
state |
enum:AttemptState |
yes |
AttemptCheckpointed payload:
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | |
runId |
guid |
yes | |
tenantId |
string |
yes | |
checkpointId |
guid |
yes | |
sequence |
int |
yes | |
adapterVersion |
string |
yes |
AttemptDecisionRequested payload:
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | |
runId |
guid |
yes | |
tenantId |
string |
yes | |
decisionId |
guid |
yes | |
kind |
enum:DecisionKind |
yes | |
approvalId |
guid |
no |
AttemptCancellationObserved payload:
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | |
runId |
guid |
yes | |
tenantId |
string |
yes | |
uncertainCalls |
int |
yes |
AttemptOutcomeReported payload:
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | |
runId |
guid |
yes | |
tenantId |
string |
yes | |
state |
enum:AttemptState |
yes | |
outcomeHash |
hash |
yes | |
corrections |
int |
yes | |
modelCalls |
int |
yes | |
toolCalls |
int |
yes |
SubAgentDelegated payload:
| Field | Type | Required | Description |
|---|---|---|---|
attemptId |
guid |
yes | |
runId |
guid |
yes | |
tenantId |
string |
yes | |
childRunId |
guid |
yes | |
childAgentId |
string |
yes | |
depth |
int |
yes |
ReleaseLoaded payload:
| Field | Type | Required | Description |
|---|---|---|---|
releaseId |
string |
yes | |
tenantId |
string |
yes | |
manifestHash |
hash |
yes | |
adapterVersion |
string |
yes |
ReleaseLoadRejected payload:
| Field | Type | Required | Description |
|---|---|---|---|
releaseId |
string |
yes | |
tenantId |
string |
yes | |
reason |
string |
yes | |
adapterVersion |
string |
yes |
| Command | Handled by | Result events | Summary |
|---|---|---|---|
StartAttempt |
ExecutionAttempt | AttemptStarted | Claim the dispatched attempt under its lease and start the loop. |
Interrupt |
ExecutionAttempt | AttemptCancellationObserved | Stop at the next safe point and settle in-flight calls. |
ReportOutcome |
ExecutionAttempt | AttemptOutcomeReported | Report the single outcome with its hash and fencing token. |
LoadRelease |
LoadedRelease | ReleaseLoaded, ReleaseLoadRejected | Fetch and verify a release manifest for this adapter. |
| Event | Producer | Handler | Purpose |
|---|---|---|---|
| AttemptDispatched | RunControlService | ClaimAndStart |
Claim the lease, load the release, build context and start the loop. |
| ApprovalDecided | RunControlService | ResumeAfterApproval |
Continue or fail the waiting attempt according to the decision. |
| RunCancelled | RunControlService | ObserveCancellation |
Settle attempts of a cancelled run. |
| LeaseExpired | RunControlService | AbandonAttempt |
Stop work on an attempt whose lease the Control plane reclaimed. |
| AssetReleasePublished | AgentRegistryService | WarmReleaseCache |
Pre-verify releases likely to be dispatched. |
| AssetReleaseDeprecated | AgentRegistryService | ApplyInFlightPolicy |
Apply the in-flight policy to running attempts. |
| AssetReleaseRevoked | AgentRegistryService | ApplyInFlightPolicy |
Interrupt or drain attempts of a revoked release. |
The runtime's events are its timeline for the Studio and Observability; the authoritative lifecycle events (AttemptOutcomeRecorded, run terminal events) are Run Control's, produced after the runtime's gRPC report is accepted.
Workers and jobs¶
| Worker | Trigger | Output events | Retry | Idempotency key | Summary |
|---|---|---|---|---|---|
| LeaseHeartbeat | timer every 15s |
AttemptPhaseChanged | fixed | leaseId + renewalSequence |
Renews the lease of every running attempt before expiry through Run Control; a failed renewal pauses the attempt at its next checkpoint. |
| OrphanedAttemptReconciler | schedule */2 * * * * |
AttemptOutcomeReported | exponential | attemptId + outcomeHash |
Finds attempts whose grain is inactive and whose lease is gone; settles them as Uncertain when a call was in flight, otherwise Cancelled, and reports. |
| CheckpointCompactor | schedule 0 3 * * * |
— | fixed | checkpointId |
Deletes superseded adapter checkpoints of terminal attempts after the retention window, keeping the last one and every checkpoint referenced by evidence. |
| ReleaseCacheWarmer | event | ReleaseLoaded, ReleaseLoadRejected | exponential | releaseId + adapterVersion |
Verifies newly published releases against this worker's adapter before the first dispatch. |
Actors and grains¶
| Grain | Key | State | Reminders | Reentrant | Summary |
|---|---|---|---|---|---|
| AttemptGrain | guid = attemptId | state, phase, fencingToken, lastCheckpointId, openCallId |
call-timeout, attempt-budget | no | Serialises the loop of one attempt; holds the adapter session, writes checkpoints, enforces the fencing token and the limits, and reports the outcome once. |
Sagas and long-running flows¶
None modelled
No sagas are modelled for this item yet.
sequenceDiagram
participant RC as RunControlService
participant A as AttemptGrain
participant AR as AgentRegistryService
participant CB as ContextBuilderService
participant PE as PolicyEngineService
participant MR as ModelRouterService
participant TA as ToolAdapterService
RC-)A: AttemptDispatched with lease and fencing token
A->>AR: LoadRelease and verify hashes
A->>CB: build context snapshot within budget
A->>A: checkpoint, AttemptStarted
loop until verified or budget exhausted
A->>MR: model call with profile and receipt
A->>PE: decision for the proposed tool call class
alt approval required
A->>RC: RequestApproval with proposalHash
RC-)A: ApprovalDecided
end
A->>A: checkpoint before the gated call
A->>TA: tool call with policy decision id and fencing token
TA-->>A: receipt
A->>A: observe, verify, correct within maxCorrections
end
A->>RC: ReportAttemptOutcome with outcomeHash and fencing token
Options and configuration¶
| Section | Key | Type | Default | Reloadable | Secret | Summary |
|---|---|---|---|---|---|---|
Runtime:Attempts |
DefaultMaxCorrections |
int | 2 |
yes | no | Bounded self-correction when the contract does not set it. |
Runtime:Attempts |
CheckpointEveryCalls |
int | 1 |
yes | no | Checkpoint cadence in addition to the mandatory pre-gated-call checkpoint. |
Runtime:Attempts |
MaxSubAgentDepth |
int | 2 |
yes | no | Supervisor to sub-agent delegation depth. |
Runtime:Context |
MaxContextTokens |
int | 32000 |
yes | no | Upper bound passed to context assembly (BC-09). |
Runtime:Decisions |
InputTimeoutMinutes |
int | 1440 |
yes | no | How long an attempt waits for human input before failing. |
Metrics and observability¶
| Metric | Instrument | Unit | Dimensions | Summary |
|---|---|---|---|---|
connectsoft.factory.runtime.attempts_active |
updown | {attempt} | tenant, state | Attempts by state. |
connectsoft.factory.runtime.attempt_duration |
histogram | s | tenant, agent, outcome | Attempt duration by outcome. |
connectsoft.factory.runtime.calls |
counter | {call} | tenant, kind, status | Outbound calls by kind and status. |
connectsoft.factory.runtime.corrections |
counter | {correction} | tenant, agent | Self-corrections performed. |
connectsoft.factory.runtime.checkpoints_written |
counter | {checkpoint} | tenant | Checkpoints persisted. |
connectsoft.factory.runtime.decisions_waiting |
updown | {decision} | tenant, kind | Attempts waiting for a human. |
connectsoft.factory.runtime.uncertain_outcomes |
counter | {attempt} | tenant | Attempts settled as Uncertain. |
connectsoft.factory.runtime.release_load_rejected |
counter | {release} | tenant, reason | Releases rejected at load. |
Model and tool spans follow the OpenTelemetry GenAI semantic conventions (gen_ai.* attributes) and join the run trace keyed on run.id; the runtime adds attempt.id, phase and checkpoint.sequence. Content is never put on spans.
Security¶
| Authentication | client-credentials |
| Scopes | factory.runs.read (Read attempt timelines and checkpoints.); factory.assets.read (Read loaded releases.) |
| Roles | Requester, Reviewer, Operator, WorkspaceAdmin, PlatformAdmin, Observer |
| Tenant isolation | row-filter |
| Data classification | confidential |
| Max side-effect class | production |
| Secrets | — |
Prompt injection cannot escalate: scope, release, policy and tool permissions come from the snapshot and the executor, not from model output; a model that asks for a tool outside the release's closure gets a refusal recorded as a call with status failed.
Dependencies¶
Outbound
| Target | Kind | Purpose | Failure mode | Required |
|---|---|---|---|---|
| RunControlService | grpc | AcquireLease (renewals), ReportAttemptOutcome, RequestApproval, child run submission. | fail-fast | yes |
| AgentRegistryService | grpc | LoadRelease manifests and content references. | fail-fast | yes |
| ModelRouterService | grpc | Model invocations against the pinned profile with receipts. | degrade | yes |
| ToolAdapterService | grpc | Tool and MCP invocations with policy decision ids and fencing tokens. | fail-fast | yes |
| ContextBuilderService | grpc | Context assembly into a context snapshot within the token budget. | degrade | yes |
| ArtifactService | grpc | Store adapter checkpoints and produced artifacts by hash. | fail-fast | yes |
| PolicyEngineService | grpc | Policy decision before every gated call. | fail-fast | yes |
| AgentPoolManager | grpc | Capacity registration and sandbox workspace leases (BC-26). | degrade | no |
Inbound
| Target | Kind | Purpose | Failure mode | Required |
|---|---|---|---|---|
| RunControlService | grpc | Interrupt and WatchAttempt. | — | yes |
| StudioBff | sync-http | Attempt timeline reads. | — | yes |
Failure and recovery¶
| Scenario | Detection | Recovery | Data impact | RTO (min) |
|---|---|---|---|---|
| Tool call in flight when the worker dies | orphaned attempt with an open call record | reconciler marks the attempt Uncertain and reports it; an operator confirms the effect through the receipt before any retry | none | 10 |
| Checkpoint incompatible after an adapter upgrade | adapter version mismatch at resume | attempt fails with class infrastructure; Run Control starts a new attempt from the step input | work of the attempt | 5 |
| Context assembly unavailable | ContextBuilderService errors | attempt pauses and retries within its budget; fails as transient when exhausted | none | 30 |
| Corrections exhausted | correctionCount reaches maxCorrections | attempt Failed with class validation and the validator feedback attached; the workflow decides on retry or escalation | none | 0 |
NFR targets¶
| Target | Class value (pilot-interactive) | Override |
|---|---|---|
| latency_p95_ms | 2000 | — |
| availability_pct | 99.5 | — |
| throughput_rps | 50 | — |
| rpo_minutes | 15 | — |
| rto_minutes | 240 | — |
| cost_class | low | — |
Deployment¶
| Granularity | module of host · host AgentRuntimeWorker |
| Topologies | saas-default, hybrid, on-prem, hybrid-compute |
| Runtime | —; replicas —–— |
| Ports | — |
Current implementation¶
ConnectSoft.Extensions.AI.Factory has the lifecycle skeleton (FactoryAgentBase: accept → prompt → run → validate → retry → complete or escalate) with no persistence, no checkpoint, no lease, no run identity and no Cancelled stage; ConnectSoft.Extensions.AI.AgentsFramework builds Microsoft Agent Framework agents from configuration with keyed services, explicit tools, RAG and file skills. Both are the components this service extends.
Later capabilities¶
Replay-based simulation against historical traces (BC-22, after the journal exists); warm adapter sessions; computer-use and browser agents as sandboxed tools (Tools & MCP); outbound A2A calls to partner agents as tools of class external.
Decisions and traceability¶
| Family | Ids |
|---|---|
| FR | FR-03, FR-05, FR-06, FR-07, FR-10 |
| FUX | FUX-04, FUX-05, FUX-07 |
| NFR | NFR-05, NFR-06, NFR-11, NFR-13 |
| XL | XL-03, XL-04, XL-05, XL-06, XL-10 |