Durability spike — execution substrate¶
Evidence (dated)
This page records observations at a stated revision and date. Re-verify before relying on it; it is not a permanent contract.
Status
Analysis of 2026-10-09 against the ConnectSoft estate as recorded in the ecosystem composition. The recommendation is written up as ADR-0024 (Accepted 2026-10-12, decision D46). No code was run; this is a design-level comparison with the criteria the Factory requires.
The question¶
A two-hour run must not disappear because a worker restarted (original handoff §3.6). Attempts need resumable checkpoints, durable timers, leases with fencing, exactly-once outcome reporting, and an engine for deterministic workflows that coexists with dynamic agent loops. Which substrate carries that durability?
Options¶
| A. Microsoft Agent Framework durable and workflow extensions | B. Orleans grains + journal (MessagingModel Orleans adapter) | C. Process manager over a broker (MassTransit saga on Service Bus) | |
|---|---|---|---|
| Durable state | Framework checkpoint store; workflow graph state persisted by the framework | Grain state in runtime-sql, journal rows committed with the outbox |
Saga state table; messages on the broker |
| Timers | Framework timers where supported | Orleans reminders (durable, persisted) | Broker scheduled messages |
| Leases and fencing | Not a framework concept; must be added | Natural: grain activation is single-threaded; fencing token in grain state | Must be added in saga logic |
| Exactly-once outcome | Not a framework concept | Grain serialises the report; idempotent on outcome hash | Idempotent consumer pattern |
| Deterministic workflows | Framework workflow graphs (sequential, concurrent, conditional, human-in-the-loop) | Engine written on grains; graph semantics are ours | Saga state machine per workflow definition (code, not data) |
| Checkpoint portability | Framework-specific format | Opaque blob per adapter version, owned by us | Saga state, framework-specific |
| Exists in ConnectSoft today | ConnectSoft.Extensions.AI.AgentsFramework wraps the framework; no durable extension in use |
ConnectSoft.Extensions.MessagingModel.Orleans (outbox/inbox), Orleans in DerechHaLev production, agent template silo |
MessagingModel.MassTransit adapter exists; MassTransit 9 licensing recorded as a constraint; saga scaffold in the agent template |
| Fit with ADR-0012 | Needs a bridge to the messaging port | Direct: the Orleans adapter is the default profile | Direct for the broker profile only |
| Fit with ADR-0014 | Run vocabulary must be mapped onto framework states | Our vocabulary, our journal | Our vocabulary in saga code |
| Lock-in | Checkpoints and workflow graphs tied to framework versions | Orleans version only; adapter checkpoints are opaque and versioned | Broker and MassTransit licence |
| Hybrid and on-prem | Framework works anywhere; durable store must be provided | Orleans clustering on SQL works anywhere; no broker required | Requires a broker on-prem |
Criteria and scoring¶
| Criterion (weight) | A | B | C |
|---|---|---|---|
| Single writer of truth stays in Run Control (must) | partial | yes | yes |
| Leases, fencing, exactly-once report (must) | add | native | add |
| Durable timers without a broker (must for on-prem) | partial | yes | no |
| Workflow definitions as data, validated at publication (must) | graph in code or data, framework-shaped | our DSL, engine on grains | state machines in code |
| Reuse of the existing estate (high) | medium | high | medium |
| Agent loop ergonomics (high) | high | medium | low |
| Portability of checkpoints across adapter versions (medium) | low | high | low |
| Operational maturity in ConnectSoft (medium) | low | high (DerechHaLev) | medium |
Recommendation¶
B as the durability substrate, A as the execution adapter inside it. Attempts are Orleans grains (AttemptGrain) that run the Microsoft Agent Framework agent in-process and write the framework's checkpoint as an opaque, versioned blob; the workflow engine is WorkflowRunGrain driving our DSL, with reminders as timers; truth stays in Run Control's journal; the messaging port's Orleans profile carries the events. The framework's own durable and workflow extensions are not the substrate: they are allowed inside a single attempt (for example a framework workflow graph used by one agent step) as long as the checkpoint they produce is stored through the attempt's checkpoint entity and the run vocabulary is ours.
Consequences:
- A second execution adapter (another agent framework) can be added by implementing the attempt adapter contract and declaring its checkpoint format version; the contract suite in Tools & MCP, Model & Compute and Knowledge does not change.
- Replay is still not promised (ADR-0014); the journal and checkpoints make recovery possible, not deterministic re-execution.
- The broker profile (C) remains available for installations that already run a broker; sagas are not used for run or workflow state.
What would change the recommendation¶
The framework's durable extension gaining first-class leases, fencing and an exportable checkpoint format; or Orleans becoming unavailable for a target topology. Either is a trigger to revisit ADR-0024.