Skip to content

Runtime Memory

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.

Reviewed 2026-10-09

Target. Memory classes per the Oct-2026 target: working/session (run state, minutes to hours), episodic (past runs), semantic (governed facts), procedural (evolves into skills); lifecycle observation → candidate → classification → evidence → confidence → policy → store → reinforce/decay → archive/delete. Only in-memory chat history exists today. The 2025 runtime state-and-memory page is merged below as an earlier design generation.

Target Architecture — Final-State Design

This page describes the final-state runtime memory of the Knowledge Platform: how the behaviour of everything the factory ships flows back in as durable, queryable knowledge. Owned by RuntimeMemoryService and KnowledgeReplayService.

Runtime memory is what makes the factory self-improving. Generated SaaS products run in production and emit telemetry; the Observability & Feedback Platform collects it; and the Knowledge Platform ingests the meaningful signals as RuntimeSignal, IncidentMemory, and FeedbackItem records, linking them back through the knowledge graph to the exact artifacts, decisions, and context that produced the running code. This closes the loop from intent → generation → runtime → learning.

Ingestion from Observability

flowchart LR
    OBS["Observability & Feedback Platform"] -->|"RuntimeSignalReceived"| RTW["RuntimeSignalWorker"]
    RTW --> RM["RuntimeMemoryService"]
    RM --> Signal["RuntimeSignal"]
    RM --> Incident["IncidentMemory"]
    RM --> Feedback["FeedbackItem"]
    Signal --> Graph["GraphProjectionWorker"]
    Incident --> Graph
    Feedback --> Graph
    Graph --> KG["('Knowledge Graph')"]
    Signal --> OTEL["('App Insights / OTEL')"]
    RM --> Conflict["ConflictDetectionWorker"]
Hold "Alt" / "Option" to enable pan & zoom

The Observability & Feedback Platform publishes runtime events in the canonical envelope. POST /knowledge/ingest/runtime-signal and the RuntimeSignalWorker persist them and link each to its producing artifact via linkedArtifactId, emitting RuntimeSignalRecorded, IncidentMemoryCaptured, or FeedbackItemRecorded.

Runtime Knowledge Types

Aggregate Captures Example
RuntimeSignal A discrete runtime observation (metric, error, saturation, cost) p95 latency regression on module-reservations-api
IncidentMemory A production incident with timeline, root cause, resolution Reservation double-booking incident, root cause + fix
FeedbackItem Explicit human/automated feedback on an artifact or output Reviewer rates a generated test suite as incomplete

Each links into the graph (GENERATED, TARGETS) so the factory can answer "which runtime problems trace back to this blueprint/decision/context?".

The Runtime Feedback Loop

flowchart LR
    Context["ContextPackage"] --> Artifact["Generated Artifact"]
    Artifact --> Deploy["Deployment"]
    Deploy --> Signal["RuntimeSignal"]
    Signal --> Attribute["Attribute to producing context/decision"]
    Attribute --> Quality["KnowledgeQualityService"]
    Attribute --> Pattern["PatternCatalogService"]
    Quality -->|down-rank weak knowledge| Context
    Pattern -->|promote proven solutions| Context
    Attribute --> Replan["Control Plane re-planning"]
Hold "Alt" / "Option" to enable pan & zoom

Because every ContextPackage records its policyDecisionId, traceId, and sources, and every artifact records the contextPackageId that produced it, a runtime signal can be attributed back to the exact context that grounded its generation. The platform uses this to:

  • Down-rank weak knowledge — knowledge that consistently produces problematic outcomes loses ranking weight in the Context Builder.
  • Promote proven solutions — knowledge correlated with healthy runtime behaviour is promoted to KnowledgePatterns for reuse.
  • Trigger re-planning — significant signals/incidents flow to the Control Plane to open corrective work.
  • Detect conflicts — the ConflictDetectionWorker flags when new runtime evidence contradicts an existing decision or pattern.

This is the concrete mechanism behind the factory's promise that each project is better than the last.

Knowledge Replay

KnowledgeReplayService and the KnowledgeReplayWorker re-drive historical envelope events to rebuild projections, recover from a bad ingestion, or analyse a timeline.

stateDiagram-v2
    [*] --> Requested
    Requested --> Loading: select window (traceId / time range / topic)
    Loading --> Replaying: stream events in order
    Replaying --> Projecting: re-run idempotent handlers
    Projecting --> Completed: projections rebuilt
    Replaying --> Failed: unrecoverable
    Completed --> [*]
    Failed --> [*]
Hold "Alt" / "Option" to enable pan & zoom
  • Deterministic — replay relies on envelopes being immutable once published and on every handler being idempotent (keyed by the idempotency key).
  • Scoped — a replay window can be a single traceId, a time range, or a topic, partitioned by tenantId.
  • Use cases — rebuilding the graph after a schema change, re-embedding after a model upgrade, reconstructing an incident timeline, or recovering dead-lettered messages after a fix.
  • Auditable — emits KnowledgeReplayStarted / KnowledgeReplayCompleted with counts of records rebuilt.

Governance of Runtime Memory

  • Runtime signals are classified like any memory; production data that may contain sensitive values is classified Confidential/Secret and redacted before inclusion in any context.
  • Raw telemetry has a shorter retention (30–90 days) than the derived IncidentMemory/FeedbackItem aggregates, which persist as durable learning.
  • Access to runtime memory in context assembly is governed by MemoryAccessPolicy (see Governance).

Earlier design generation (2025)

Runtime state and memory — merged 2026-10-09; historical, not current direction

Merged from factory/runtime/state-and-memory.md. Service names, states and technology statements below are the 2025 generation of the design and are superseded where they conflict with the sections above and the ADR review notes.

Overview

The Factory runtime manages two distinct but related concepts:

  1. Run State — Operational state for active and recent runs (ephemeral, per-run)
  2. AI Memory — Long-term knowledge stored in the Knowledge & Memory System (persistent, cross-project)

This separation enables efficient operational execution while building a persistent knowledge base that improves over time.

Run State Store

What Is Stored

The Run State Store maintains operational state for Factory runs:

Run Metadata
  • Run Identifiers — runId, tenantId, projectId, templateRecipeId
  • Run Status — Current state (Requested, Validated, Queued, Running, Succeeded, Failed, Cancelled)
  • Timestamps — requestedAt, startedAt, completedAt, updatedAt
  • Request Context — User who requested, request parameters, configuration
Step/Job Status
  • Job Identifiers — jobId, stepName, attempt number
  • Job Status — Current state (Pending, Running, Succeeded, Failed, Cancelled)
  • Job Results — Success/failure status, error messages, execution duration
  • Artifact References — Links to generated artifacts (repo URLs, pipeline IDs, etc.)
Execution Context
  • Correlation IDs — traceId, spanId for distributed tracing
  • External System IDs — Azure DevOps buildId, repoId, pipelineId, workItemId
  • Execution Metadata — Worker instance, execution environment, resource usage

Storage Choice

The Run State Store is typically implemented as:

  • Relational Database (SQL) — For structured queries, joins, and transactional consistency
    • Examples: PostgreSQL, SQL Server, Azure SQL Database
    • Benefits: ACID transactions, complex queries, referential integrity
  • Document Database — For flexible schema and horizontal scaling
    • Examples: Cosmos DB, MongoDB
    • Benefits: Schema flexibility, horizontal scaling, JSON-native storage

Considerations:

  • Query Patterns — Relational DB for complex queries (e.g., "all runs for project X in last 30 days")
  • Scale Requirements — Document DB for high-scale, multi-tenant scenarios
  • Consistency Needs — Relational DB for strong consistency requirements

Job State & Idempotency Keys

Idempotency Key Structure

Jobs use structured idempotency keys to ensure safe retries:

  • Format: {runId}:{stepName}:{attempt}
  • Example: run-abc123:generate-repo:1
  • Purpose: Uniquely identifies a job execution attempt

State Fields for Safe Retries

Job state includes fields that enable safe retries:

  • idempotencyKey — Unique key for deduplication
  • status — Current job status (Pending, Running, Succeeded, Failed)
  • attemptNumber — Current retry attempt (1, 2, 3, ...)
  • lastAttemptAt — Timestamp of last execution attempt
  • result — Execution result (success/failure, error details)
  • checkpoint — Progress checkpoint for resumable jobs

Atomic State Updates

State updates are atomic to prevent race conditions:

  • Optimistic Locking — Use version numbers or timestamps to detect concurrent updates
  • Transactional Updates — Use database transactions for multi-field updates
  • Idempotent Operations — State updates are idempotent (applying same update twice has no effect)

Artifacts & Metadata

Artifact Storage

Generated artifacts are stored in external systems, not in the Run State Store:

  • Git Repositories — Code, tests, documentation stored in Azure DevOps or GitHub
  • Azure DevOps — Pipelines, work items, artifacts stored in Azure DevOps
  • Blob Storage — Large artifacts (diagrams, models) stored in Azure Blob Storage
  • Container Registries — Docker images stored in Azure Container Registry

Artifact References

The Run State Store maintains references to artifacts:

  • Repository URLs — Links to generated Git repositories
  • Pipeline IDs — References to generated CI/CD pipelines
  • Work Item IDs — References to created Azure DevOps work items
  • Blob URLs — Links to stored artifacts in blob storage
  • Artifact Metadata — Size, type, creation timestamp, checksums

Artifact Lifecycle

Artifacts follow a lifecycle managed by the Factory:

  1. Generation — Artifacts are generated by workers
  2. Storage — Artifacts are stored in external systems (Git, Azure DevOps, etc.)
  3. Reference — Artifact references are stored in Run State Store
  4. Indexing — Selected artifacts are indexed in Knowledge & Memory System
  5. Retention — Artifacts are retained according to retention policies

AI Memory & Knowledge System Integration

Operational State vs. Long-Term Memory

The Factory maintains a clear separation:

Aspect Operational State (Run State Store) Long-Term Memory (Knowledge System)
Purpose Track active runs, enable execution Learn patterns, enable reuse
Lifetime Ephemeral (weeks/months) Persistent (years)
Scope Per-run, per-project Cross-project, cross-tenant
Query Pattern Structured queries (SQL) Semantic search (vector)
Update Frequency High (real-time) Low (batch/indexing)

How Factory Interacts with Knowledge System

During Execution
  • Pattern Retrieval — Agents query Knowledge System for similar past solutions
  • Template Lookup — Look up templates and patterns from Knowledge System
  • Context Enrichment — Retrieve relevant historical context for agents
After Execution
  • Run Summaries — Store run summaries, outcomes, and learnings
  • Pattern Extraction — Extract reusable patterns from generated artifacts
  • Failure Analysis — Store failure patterns and resolutions for future reference
  • Success Patterns — Index successful solutions for reuse

Vector Indexes

The Knowledge System uses vector indexes for semantic search:

  • Template Knowledge — Vector embeddings of templates, blueprints, and patterns
  • Past Runs — Vector embeddings of run summaries, decisions, and outcomes
  • Code Patterns — Vector embeddings of code snippets and architectural patterns
  • Domain Knowledge — Vector embeddings of domain-specific solutions

Example Query: 

Query: "multi-tenant user management with role-based access"
→ Vector search finds:
  - Past runs that generated similar solutions
  - Templates for multi-tenant patterns
  - Code patterns for RBAC implementation

State & Memory Architecture

graph TD
    RunStore["(Run State DB)"]
    Queue["(Job Queue)"]
    Artifacts["(Repos, Pipelines, Docs)"]
    Memory[Knowledge & Memory System<br/>Vector DB, search]

    Worker --> RunStore
    Worker --> Queue
    Worker --> Artifacts
    Orchestrator --> RunStore
    Orchestrator --> Queue

    RunStore --> Memory
    Artifacts --> Memory
    Worker --> Memory
    Orchestrator --> Memory
Hold "Alt" / "Option" to enable pan & zoom

Data Flows:

  1. Operational Flow — Workers update RunStore with execution state
  2. Artifact Flow — Workers create artifacts in external systems, store references in RunStore
  3. Indexing Flow — Selected artifacts and run summaries are indexed in Memory System
  4. Query Flow — Agents query Memory System for patterns and context

State Retention and Cleanup

Run State Retention

Run state is retained for operational and audit purposes:

  • Active Runs — Retained indefinitely while run is active
  • Completed Runs — Retained for configurable period (e.g., 90 days, 1 year)
  • Failed Runs — Retained longer for debugging and analysis (e.g., 1 year)
  • Archived Runs — Old runs can be archived to cold storage

Memory System Retention

Knowledge & Memory System retains data indefinitely:

  • Patterns — Retained permanently for pattern reuse
  • Run Summaries — Retained for historical context and learning
  • Artifact Indexes — Retained for semantic search and retrieval
  • Failure Patterns — Retained for failure analysis and prevention