Skip to content

Data Architecture

The factory uses polyglot persistence: each store is chosen for its access pattern, and every store has exactly one owning service. This page is the architectural overview; the concrete inventory is the Storage Catalog.

Store selection

Store Technology Used for
Transactional metadata Azure SQL / PostgreSQL via NHibernate Aggregates, registries, lifecycle state (DB-per-service).
Document data Cosmos DB Flexible/document-shaped data where schema varies.
Large immutable artifacts Azure Blob Storage Generated code snapshots, docs, diagrams, payloads.
Vector memory Qdrant (primary), Azure AI Search (alternative) Semantic search over blueprints, docs, patterns, code.
Source of truth (code/docs/IaC) Git (Azure DevOps) Generated repositories, source-controlled memory.
Hot cache / context Redis Hot context packages, sessions, graph traversals.
Telemetry Application Insights / OTEL backend Traces, metrics, logs, runtime signals.
Secrets Azure Key Vault Secret references only; never stored as artifacts.

Ownership rule

flowchart LR
    service["Owning Service"] -->|writes| store["Store"]
    other["Other Services"] -.->|read via API/events| service
Hold "Alt" / "Option" to enable pan & zoom

A store is written by exactly one service. Other services read through that service's API or by subscribing to its events — never by reaching into its database. This preserves bounded-context autonomy and keeps lineage clean. Each platform's storage.md page documents its owned stores.

Data flow across the lifecycle

flowchart LR
    blueprint["Blueprint<br/>(Azure SQL)"] --> context["Context Package<br/>(Redis hot, PostgreSQL durable)"]
    context --> artifact["Artifact<br/>(Blob + metadata in SQL)"]
    artifact --> vector["Embeddings<br/>(Qdrant)"]
    artifact --> git["Code/Docs<br/>(Git)"]
    git --> deployment["Deployment<br/>(SQL inventory)"]
    deployment --> telemetry["Telemetry<br/>(App Insights)"]
    telemetry --> knowledge["Runtime Memory<br/>(graph + SQL)"]
Hold "Alt" / "Option" to enable pan & zoom

Multi-tenancy and retention

  • tenantId participates in every index for tenant-scoped data (see Multi-Tenancy).
  • Retention is store- and class-specific: telemetry has rolling windows; artifacts are retained with version history; audit and compliance data follow regulatory retention. Each storage.md page states retention per data type.
  • Sensitivity classification (Public/Internal/Confidential/Secret) drives access and redaction (see Artifact Metadata).