Skip to content

WorkflowEngineService

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.

Bounded context BC-38 Workflow Definitions & Engine
Platform / plane Agent Mesh / Agent & Orchestration runtime
Deployment granularity module of host · host AgentRuntimeWorker
Status target · ConnectSoft.Factory.Runtime.Workflows
Built from connectsoft-ai-agent · libraries: connectsoft-extensions-ai-agents-framework, connectsoft-extensions-messaging-model, connectsoft-extensions-messaging-model-orleans, connectsoft-extensions-persistence-model-nhibernate
Decisions ADR-0014, ADR-0021
Traceability FR-02 FR-03 FR-06 FR-10 FUX-04 FUX-05 FUX-11 NFR-05 NFR-06 NFR-11 XL-05 XL-06 XL-09

Purpose and responsibilities

The original design asks for deterministic workflows next to dynamic agent reasoning: "Agent = dynamic reasoning, Workflow = deterministic orchestration, both should coexist", with DAG execution, conditions, retries, compensation, timers, approvals, events and agent, tool and human steps. WorkflowEngineService is that engine. It validates and caches workflow definitions published as assets (BC-04), opens a projection for every accepted run, computes the steps that are ready, evaluates conditions on recorded outputs, fires durable timers, plans compensation and tells Run Control what to dispatch and when the run is complete.

The engine is deliberately not an authority. Run Control owns run, step and attempt state (ADR-0014); the engine emits StepReady and Run Control creates the step and the lease; the engine consumes AttemptOutcomeRecorded and advances. This split keeps one writer of truth and lets the engine be re-projected from Run Control's journal after a loss.

Context and boundaries

flowchart LR
    S["WorkflowEngineService"]
    RunControlService["RunControlService"] -->|grpc| S
    StudioBff["StudioBff"] -->|sync-http| S
    S -->|event| RunControlService["RunControlService"]
    S -->|grpc| AgentRegistryService["AgentRegistryService"]
    S -->|grpc| PolicyEngineService["PolicyEngineService"]
    S -->|grpc| ArtifactService["ArtifactService"]
    S -. "WorkflowDefinitionLoaded" .-> E_WorkflowDefinitionLoaded(("event"))
    S -. "WorkflowRunStarted" .-> E_WorkflowRunStarted(("event"))
    S -. "StepReady" .-> E_StepReady(("event"))
    S -. "StepRunStateChanged" .-> E_StepRunStateChanged(("event"))
    S -. "TimerFired" .-> E_TimerFired(("event"))
    S -. "CompensationStarted" .-> E_CompensationStarted(("event"))
    S -. "WorkflowRunCompleted" .-> E_WorkflowRunCompleted(("event"))
    S -. "WorkflowRunFailed" .-> E_WorkflowRunFailed(("event"))
    S -. "WorkflowRunCancelled" .-> E_WorkflowRunCancelled(("event"))
Hold "Alt" / "Option" to enable pan & zoom

Step kinds and what they become:

Kind Becomes Gate
Agent an attempt of the named agent dispatched by Run Control the agent's tool calls carry their own classes
Tool an attempt that executes one tool call through the executor the declared action class (D21)
Human an approval request (exact version) or an input request always a decision
Condition evaluated by the engine on recorded outputs and variables none; never a model call
Timer a durable reminder none
SubWorkflow a child run of another Published workflow inherits scope and budget
Fanout / Join parallel branches with quorum none
Compensation the compensation step of a succeeded step, run in reverse order on failure the compensation step's own class

Domain model

WorkflowDefinitionVersion — Immutable, validated graph of a Published workflow asset — steps, edges, conditions, timers, compensation map, input and output schemas — cached in the runtime for execution. (catalog entry)

Field Type Required Description
definitionId string yes
version string yes
releaseId string yes Asset release of the workflow (BC-04).
tenantId string yes
graphHash hash yes
stepCount int yes
inputSchemaRef string yes
outputSchemaRef string yes
maxLoopIterations int yes
loadedAt datetime yes

Invariants:

  • The graph is acyclic except for explicitly declared loops with a bounded iteration count.
  • Every Agent, Tool and Human step declares its action class (D21) and its failure policy; publication is refused otherwise (validated by the engine at validate time and at load).
  • A Condition step reads only recorded step outputs and run variables; it never calls a model.
  • A definition version is immutable once loaded; a changed graph has a new release id.

Events: WorkflowDefinitionLoaded · Tables: WorkflowDefinitions, StepDefinitions, EdgeDefinitions · Repository: WorkflowDefinitionVersionRepository

WorkflowRun — The engine's state of one run — cursor over the graph, step runs, variables, timers and compensation plan — projected from Run Control's accepted run and attempt outcomes. (catalog entry)

Field Type Required Description
runId guid yes Same id as the Run in Run Control.
tenantId string yes
projectId string yes
definitionReleaseId string yes
graphHash hash yes
state enum:WorkflowRunState yes
variablesRef string yes Artifact reference of the run variables (JSON).
activeStepRuns int yes
lastOutcomeSequence long yes Last Run Control attempt outcome applied; makes Advance idempotent.
version long yes

Invariants:

  • The engine never changes a run's authoritative state; it emits StepReady for Run Control to dispatch and WorkflowRunCompleted, WorkflowRunFailed or WorkflowRunCancelled for Run Control to record.
  • Advance is idempotent on the outcome sequence; an outcome applied twice changes nothing.
  • One StepRun per step is active unless the step is inside a Fanout; a Join waits for every branch or the declared quorum.
  • Timers are durable reminders; a fired timer that finds its step already advanced is a no-op.
  • Compensation runs the declared compensation steps of succeeded steps in reverse completion order; a failed compensation ends the run Failed with an operator follow-up, never silently.
stateDiagram-v2
    Pending --> Running : RunAcceptedAndBudgetReserved
    Running --> WaitingForTimer : TimerStepReached
    WaitingForTimer --> Running : TimerFired
    Running --> WaitingForHuman : HumanStepReached
    WaitingForHuman --> Running : DecisionApplied
    Running --> Compensating : StepFailedWithCompensatePolicy
    Compensating --> Failed : CompensationCompletedOrFailed
    Running --> Completed : TerminalStepsSucceeded
    Running --> Failed : StepFailedWithFailPolicy
    Running --> Cancelled : RunCancelled
    WaitingForTimer --> Cancelled : RunCancelled
    WaitingForHuman --> Cancelled : RunCancelled
Hold "Alt" / "Option" to enable pan & zoom

Events: WorkflowRunStarted, StepReady, StepRunStateChanged, TimerFired, CompensationStarted, WorkflowRunCompleted, WorkflowRunFailed, WorkflowRunCancelled · Tables: WorkflowRuns, StepRuns, Timers, CompensationEntries · Repository: WorkflowRunRepository

erDiagram
    WorkflowDefinitionVersion {
        string definitionId PK
        string version
        string releaseId
        string tenantId
        hash graphHash
        int stepCount
        string inputSchemaRef
        string outputSchemaRef
        int maxLoopIterations
        datetime loadedAt
    }
    WorkflowRun {
        guid runId PK
        string tenantId
        string projectId
        string definitionReleaseId
        hash graphHash
        enum_WorkflowRunState state
        string variablesRef
        int activeStepRuns
        long lastOutcomeSequence
        long version
    }
    StepDefinition {
        string stepId

        enum_StepKind kind

        string name

        string agentId

        string toolId

        string actionClass

        enum_FailurePolicy failurePolicy

        int maxAttempts

        int timeoutMinutes

        string compensationStepId

        string inputMappingRef

        string conditionExpression

    }
    EdgeDefinition {
        string fromStepId

        string toStepId

        string guard

        bool isLoopBack

    }
    StepRun {
        guid stepRunId

        string stepId

        enum_StepKind kind

        enum_StepRunState state

        int iteration

        list_guid attemptIds

        string outputRef

        guid approvalId

        datetime startedAt

        datetime endedAt

    }
    TimerEntry {
        guid timerId

        guid stepRunId

        datetime dueAt

        bool fired

    }
    CompensationEntry {
        guid entryId

        guid forStepRunId

        string compensationStepId

        int order

        enum_StepRunState state

    }
    WorkflowDefinitionVersion ||--o{ StepDefinition : contains
    WorkflowDefinitionVersion ||--o{ EdgeDefinition : contains
    WorkflowRun ||--o{ StepRun : contains
    WorkflowRun ||--o{ TimerEntry : contains
    WorkflowRun ||--o{ CompensationEntry : contains
Hold "Alt" / "Option" to enable pan & zoom
stateDiagram-v2
    [*] --> Pending: RunAccepted
    Pending --> Running: budget reserved, first steps ready
    Running --> WaitingForTimer: Timer step
    WaitingForTimer --> Running: timer fired
    Running --> WaitingForHuman: Human step
    WaitingForHuman --> Running: decision applied
    Running --> Compensating: step failed with policy Compensate
    Compensating --> Failed: compensation finished or failed
    Running --> Completed: terminal steps succeeded
    Running --> Failed: step failed with policy Fail
    Running --> Cancelled: RunCancelled
    WaitingForTimer --> Cancelled: RunCancelled
    WaitingForHuman --> Cancelled: RunCancelled
    Completed --> [*]
    Failed --> [*]
    Cancelled --> [*]
Hold "Alt" / "Option" to enable pan & zoom

Entities and value objects

StepDefinition (entity, in WorkflowDefinitionVersion) — One node of the graph.

Field Type Required Description
stepId string yes
kind enum:StepKind yes
name string yes
agentId string no
toolId string no
actionClass string no read
failurePolicy enum:FailurePolicy yes
maxAttempts int yes
timeoutMinutes int yes
compensationStepId string no
inputMappingRef string no
conditionExpression string no

EdgeDefinition (entity, in WorkflowDefinitionVersion) — A directed edge with an optional guard.

Field Type Required Description
fromStepId string yes
toStepId string yes
guard string no
isLoopBack bool yes

StepRun (entity, in WorkflowRun) — One execution of a step inside a run.

Field Type Required Description
stepRunId guid yes
stepId string yes
kind enum:StepKind yes
state enum:StepRunState yes
iteration int yes
attemptIds list:guid yes
outputRef string no
approvalId guid no
startedAt datetime no
endedAt datetime no

TimerEntry (entity, in WorkflowRun) — A durable timer for a Timer step or a step timeout.

Field Type Required Description
timerId guid yes
stepRunId guid yes
dueAt datetime yes
fired bool yes

CompensationEntry (entity, in WorkflowRun) — Planned compensation of a succeeded step.

Field Type Required Description
entryId guid yes
forStepRunId guid yes
compensationStepId string yes
order int yes
state enum:StepRunState yes

Data model and storage

Store Role Engine / profile Tables
Runtime worker durable store primary azure-sql / nhibernate-sqlserver WorkflowDefinitions, StepDefinitions, EdgeDefinitions, WorkflowRuns, StepRuns, Timers, CompensationEntries
Table Owner Columns Indexes
dbo.WorkflowDefinitions WorkflowDefinitionVersion 10 1
dbo.StepDefinitions StepDefinition 14 0
dbo.EdgeDefinitions EdgeDefinition 6 0
dbo.WorkflowRuns WorkflowRun 10 1
dbo.StepRuns StepRun 12 1
dbo.Timers TimerEntry 6 1
dbo.CompensationEntries CompensationEntry 7 0

Full DDL: appendix.

APIs

Method Path Operation Scope Side effects Idempotency Gateway Summary
POST /runtime/workflow-definitions/validate ValidateWorkflowDefinition factory.assets.write sandbox natural GW-03 Validate a workflow DSL document: acyclicity, declared action classes and failure policies, schema references, bounded loops; used by authoring and by publication (BC-04) as a gate.
GET /runtime/workflow-runs/{runId} GetWorkflowRunGraph factory.runs.read read natural GW-03 The graph of a run with the state of every step run, timers and compensation entries, for the Studio run view.
GET /runtime/workflow-definitions/{definitionId}/versions ListWorkflowDefinitionVersions factory.assets.read read natural GW-03 Loaded versions of a definition with graph hashes.
gRPC service Method Streaming Visibility Side effects Summary
WorkflowEngine PlanNextSteps none internal read Given the run and the last applied outcome sequence return the steps that are ready to dispatch with their input mappings and action classes; idempotent.
WorkflowEngine GetRunGraph none internal read Current engine projection of a run for Run Control and the BFF.

Example of the workflow DSL the engine validates (the schema is registered as a descriptor; see shared contracts):

workflow: repository-assessment
version: 2.1.0
input: schemas/assessment-input.json
output: schemas/assessment-report.json
steps:
  - id: classify
    kind: Agent
    agent: triage-agent
    failurePolicy: Retry
    maxAttempts: 2
  - id: retrieve
    kind: Tool
    tool: knowledge-search
    actionClass: read
  - id: analyse
    kind: Agent
    agent: repository-assessor
    failurePolicy: Escalate
    timeoutMinutes: 45
  - id: confident
    kind: Condition
    expression: "{ \">=\": [ { \"var\": \"analyse.confidence\" }, 0.8 ] }"
  - id: review
    kind: Human
    actionClass: shared-write
  - id: propose
    kind: Agent
    agent: change-proposer
    actionClass: shared-write
    compensation: retract-proposal
edges:
  - { from: classify, to: retrieve }
  - { from: retrieve, to: analyse }
  - { from: analyse, to: confident }
  - { from: confident, to: propose, guard: "true" }
  - { from: confident, to: review, guard: "false" }
  - { from: review, to: propose }

Messages

Event Topic Scope / v Aggregate Consumers Summary
WorkflowDefinitionLoaded workflows.definitions.v1.workflow-definition-loaded integration / 1 WorkflowDefinitionVersion DashboardService A workflow release was validated and cached by the engine.
WorkflowRunStarted workflows.runs.v1.workflow-run-started integration / 1 WorkflowRun StudioBff, DashboardService The engine opened its projection of an accepted run and computed the first ready steps.
StepReady workflows.runs.v1.step-ready integration / 1 WorkflowRun RunControlService, DashboardService A step is ready; Run Control creates the step and dispatches an attempt (Agent or Tool) or an approval (Human).
StepRunStateChanged workflows.runs.v1.step-run-state-changed integration / 1 WorkflowRun StudioBff, DashboardService A step run moved state.
TimerFired workflows.runs.v1.timer-fired integration / 1 WorkflowRun DashboardService A durable timer fired and the run advanced.
CompensationStarted workflows.runs.v1.compensation-started integration / 1 WorkflowRun StudioBff, DashboardService, AuditService Compensation of succeeded steps began in reverse order.
WorkflowRunCompleted workflows.runs.v1.workflow-run-completed integration / 1 WorkflowRun RunControlService, StudioBff, DashboardService Every terminal step succeeded; Run Control records Completed.
WorkflowRunFailed workflows.runs.v1.workflow-run-failed integration / 1 WorkflowRun RunControlService, StudioBff, DashboardService A step failed with policy Fail or compensation finished; Run Control records Failed with the class.
WorkflowRunCancelled workflows.runs.v1.workflow-run-cancelled integration / 1 WorkflowRun DashboardService The engine closed its projection after a cancellation.

WorkflowDefinitionLoaded payload:

Field Type Required Description
releaseId string yes
tenantId string yes
definitionId string yes
version string yes
graphHash hash yes

WorkflowRunStarted payload:

Field Type Required Description
runId guid yes
tenantId string yes
definitionReleaseId string yes
graphHash hash yes

StepReady payload:

Field Type Required Description
runId guid yes
tenantId string yes
stepRunId guid yes
stepId string yes
kind enum:StepKind yes
actionClass string no
inputRef string no

StepRunStateChanged payload:

Field Type Required Description
runId guid yes
tenantId string yes
stepRunId guid yes
stepId string yes
fromState enum:StepRunState yes
toState enum:StepRunState yes

TimerFired payload:

Field Type Required Description
runId guid yes
tenantId string yes
timerId guid yes
stepRunId guid yes

CompensationStarted payload:

Field Type Required Description
runId guid yes
tenantId string yes
entries int yes

WorkflowRunCompleted payload:

Field Type Required Description
runId guid yes
tenantId string yes
outputRef string no

WorkflowRunFailed payload:

Field Type Required Description
runId guid yes
tenantId string yes
failedStepId string yes
failureClass string yes
compensated bool yes

WorkflowRunCancelled payload:

Field Type Required Description
runId guid yes
tenantId string yes
openStepRuns int yes
Command Handled by Result events Summary
StartWorkflowRun WorkflowRun WorkflowRunStarted, StepReady Open the projection for an accepted run and compute the first ready steps.
AdvanceWorkflowRun WorkflowRun StepRunStateChanged, StepReady, WorkflowRunCompleted, WorkflowRunFailed Apply an attempt outcome or decision and compute what is ready next.
CancelWorkflowRun WorkflowRun WorkflowRunCancelled Close the projection after Run Control cancelled the run.
LoadWorkflowDefinition WorkflowDefinitionVersion WorkflowDefinitionLoaded Validate and cache a Published workflow release.
Event Producer Handler Purpose
RunAccepted RunControlService StartWorkflowRun Open the engine projection for an accepted run.
AttemptOutcomeRecorded RunControlService AdvanceOnOutcome Apply the authoritative outcome and compute the next ready steps.
ApprovalDecided RunControlService AdvanceOnDecision Complete a Human step or fail the run per policy.
RunCancelled RunControlService CancelWorkflowRun Close the projection and skip pending timers.
AssetReleasePublished AgentRegistryService LoadWorkflowDefinition Validate and cache workflow releases.

Workers and jobs

Worker Trigger Output events Retry Idempotency key Summary
TimerScheduler reminder per timer dueAt TimerFired, StepReady exponential timerId Registers an Orleans reminder for every timer entry and applies the fired timer to the run.
StuckRunReconciler schedule */5 * * * * StepReady, StepRunStateChanged fixed stepRunId + lastOutcomeSequence Finds workflow runs whose active step runs have no corresponding open attempt or approval in Run Control and re-emits StepReady or fails the step per policy.
DefinitionCacheWarmer event WorkflowDefinitionLoaded exponential releaseId Validates and caches newly published workflow definitions.

Actors and grains

Grain Key State Reminders Reentrant Summary
WorkflowRunGrain guid = runId state, lastOutcomeSequence, activeStepRuns timer, stuck-detection no Serialises Advance for one run, holds the cursor and active step runs, registers timer reminders and commits step-run rows with the outbox in one transaction.

Sagas and long-running flows

None modelled

No sagas are modelled for this item yet.

sequenceDiagram
    participant RC as RunControlService
    participant E as WorkflowRunGrain
    participant AR as AgentRegistryService
    RC-)E: RunAccepted with definitionReleaseId
    E->>AR: LoadRelease (cached after the first load)
    E->>E: validate graph, open projection, WorkflowRunStarted
    E-)RC: StepReady classify
    RC->>RC: create step, lease, dispatch attempt
    RC-)E: AttemptOutcomeRecorded classify Succeeded
    E->>E: Advance idempotent on outcome sequence
    E-)RC: StepReady retrieve
    RC-)E: AttemptOutcomeRecorded analyse Succeeded
    E->>E: evaluate Condition confident on recorded output
    E-)RC: StepReady review as Human step
    RC-)E: ApprovalDecided approved
    E-)RC: StepReady propose with actionClass shared-write
    RC-)E: AttemptOutcomeRecorded propose Succeeded
    E-)RC: WorkflowRunCompleted with outputRef
Hold "Alt" / "Option" to enable pan & zoom

Options and configuration

Section Key Type Default Reloadable Secret Summary
Workflows:Engine MaxFanout int 16 yes no Maximum parallel branches of a Fanout step.
Workflows:Engine DefaultMaxLoopIterations int 10 yes no Loop bound when the definition does not declare one.
Workflows:Engine DefaultStepTimeoutMinutes int 60 yes no Step timeout when the definition does not declare one.
Workflows:Engine ConditionLanguage enum jsonlogic no no Expression language for conditions and guards (evaluated on recorded outputs only).

Metrics and observability

Metric Instrument Unit Dimensions Summary
connectsoft.factory.workflows.runs_active updown {run} tenant, definition, state Workflow runs by state.
connectsoft.factory.workflows.step_latency histogram s tenant, definition, stepKind Time from StepReady to a terminal step state.
connectsoft.factory.workflows.timers_pending updown {timer} tenant Timers not yet fired.
connectsoft.factory.workflows.compensations counter {run} tenant, definition, outcome Compensations started and their outcome.
connectsoft.factory.workflows.definition_rejections counter {definition} tenant, reason Definitions rejected at validation or load.

Security

Authentication client-credentials
Scopes factory.runs.read (Read run graphs.); factory.assets.read (Read loaded definitions.); factory.assets.write (Validate definitions.)
Roles Requester, Reviewer, AssetMaintainer, Operator, WorkspaceAdmin, PlatformAdmin, Observer
Tenant isolation row-filter
Data classification internal
Max side-effect class sandbox
Secrets —

Dependencies

Outbound

Target Kind Purpose Failure mode Required
RunControlService event StepReady and terminal projections for Run Control to act on; PlanNextSteps answered on request. queue yes
AgentRegistryService grpc LoadRelease for workflow definitions and their closure. fail-fast yes
PolicyEngineService grpc Validate declared action classes against policy at load. fail-fast yes
ArtifactService grpc Run variables and step outputs by reference. fail-fast yes

Inbound

Target Kind Purpose Failure mode Required
RunControlService grpc PlanNextSteps and GetRunGraph. — yes
StudioBff sync-http Run graph and definition reads. — yes

Failure and recovery

Scenario Detection Recovery Data impact RTO (min)
Engine crash between applying an outcome and emitting StepReady lastOutcomeSequence behind Run Control grain re-activation re-applies the outcome idempotently and re-emits StepReady; Run Control deduplicates on stepRunId none 5
Timer reminder lost stuck-run reconciler finds a due timer not fired reconciler fires it; a late timer whose step advanced is a no-op delay 10
Definition incompatible with the engine version load validation fails WorkflowDefinitionLoaded is not emitted; runs pinning that release fail at acceptance with an explicit reason none 0
Compensation step fails compensation entry Failed run Failed with class compensation and an operator task; no further automatic retry partially compensated effects listed on the run 60

NFR targets

Target Class value (pilot-interactive) Override
latency_p95_ms 2000 —
availability_pct 99.5 —
throughput_rps 50 —
rpo_minutes 15 —
rto_minutes 240 —
cost_class low —

Deployment

Granularity module of host · host AgentRuntimeWorker
Topologies saas-default, hybrid, on-prem, hybrid-compute
Runtime —; replicas —–—
Ports —

The engine runs inside AgentRuntimeWorker; it is a module of the worker rather than of the Control plane host because its state is runtime state (projection, timers, adapter workflow graphs), not authority, and because the Microsoft Agent Framework workflow graph executes in the same process as the attempts (durability spike).

Current implementation

Nothing executes workflows. ConnectSoft.AI.SoftwareFactory.Platform holds six placeholder workflow definitions and schemas; the coordinators (bootstrap, sprint, milestone, assembly, release) are documents. Those documents are the seed for the first workflow definitions (B1, coordinators).

Later capabilities

Workflow simulation (dry-run cost and duration against synthetic inputs, with Usage Accounting); versioned migration of in-flight runs to a newer definition on explicit operator decision; visual authoring in Workflow Studio producing the same DSL; cross-run event triggers (a workflow started by a product telemetry event, Fleet Operations).

Decisions and traceability

Family Ids
FR FR-02, FR-03, FR-06, FR-10
FUX FUX-04, FUX-05, FUX-11
NFR NFR-05, NFR-06, NFR-11
XL XL-05, XL-06, XL-09