Skip to content

Storage

Target Architecture — Final-State Design

This page describes the final-state storage architecture of the Governance, Security & Compliance Platform. Each service owns its data (database-per-service). Relational data lives in Azure SQL / PostgreSQL via NHibernate; secret references live in Azure Key Vault (never values); long-lived evidence and exports live in Azure Blob Storage.

Storage Map

Data Store Owner Service Access Pattern Retention Notes
Policy definitions & rules (versioned) Azure SQL / PostgreSQL PolicyDefinitionService Read-heavy (loaded on evaluation), low write Indefinite (versions) Published versions immutable; cached hot in Redis for the PDP.
Policy decisions Azure SQL / PostgreSQL (append-only) PolicyEngineService Write on every evaluation; indexed reads by traceId/subject ≥ 7 years Immutable; references exact policyVersion.
Approval requests & decisions Azure SQL / PostgreSQL ApprovalService Mixed; open requests indexed by expiresAt ≥ 7 years Drives ApprovalTimeoutWorker.
Audit entries Azure SQL / PostgreSQL (append-only) AuditService Write-heavy; read via projections Hot ≥ 13 months Hash-chained; exported to Blob.
Audit exports Azure Blob Storage (NDJSON) AuditService Append/write by worker; rare read Indefinite (tiered cool/archive) Long-term, queryable evidence; immutable container.
Compliance report metadata Azure SQL / PostgreSQL ComplianceReportService Low write, indexed reads Per framework (≈ 7 years) Status, control results, evidence pointer.
Compliance evidence bundles Azure Blob Storage ComplianceReportService Write on generation; read on export Per framework WORM/immutable container where required.
Security findings Azure SQL / PostgreSQL SecurityFindingService Mixed; dedup on dedupKey ≥ 2 years after resolution Indexed by severity/status.
Secret references (metadata only) Azure SQL / PostgreSQL SecretGovernanceService Low write, frequent rotation reads Secret lifetime + history Never stores secret material.
Secret values Azure Key Vault (external) referenced by SecretGovernanceService Read via managed identity at runtime by owning services Managed in Key Vault Platform holds only VaultReferences.
Tenant isolation rules Azure SQL / PostgreSQL TenantIsolationPolicyService Read-heavy (ABAC supplier) Indefinite Backed by ConnectSoft.Extensions.Saas.*.
Data classifications Azure SQL / PostgreSQL DataClassificationService Write on artifact change; read on evaluation Latest live + history ≥ 1 year One active label per (subjectRef, contentHash).
Risk scores Azure SQL / PostgreSQL RiskScoringService Frequent recompute; latest-per-subject reads Latest + history ≥ 1 year Latest-wins projection + version history.
Hot caches (policy bundles, decisions) Redis PolicyEngineService, PolicyEvaluationService Read-through cache Ephemeral (TTL) Speeds the inline PDP path; never the system of record.

Key Vault: References, Never Values

The SecretGovernanceService is deliberately built so the platform cannot leak secrets it governs: it persists only SecretReference rows (vaultUri, secretName, secretVersion, owner, rotation policy, status). Actual secret material is read directly from Azure Key Vault by the owning service at runtime via managed identity — the governance platform never reads, caches, or proxies secret values.

flowchart LR
    SecretGov["SecretGovernanceService"] -->|stores reference| SqlSec["SQL: SecretReference"]
    SecretGov -->|reads metadata only| KeyVault["Azure Key Vault"]
    OwningService["Owning factory service"] -->|reads value via managed identity| KeyVault
    SecretScan["SecretScanWorker"] -->|leak detected| Finding["SecurityFindingService"]
    Finding -->|SecurityFindingRaised| Audit["AuditService"]
Hold "Alt" / "Option" to enable pan & zoom

Blob: Evidence and Audit Exports

  • Audit exports — the AuditExportWorker streams new audit entries to Blob as NDJSON, verifies the hash chain, and tiers older blobs to cool/archive. Containers are immutable (WORM) so exports cannot be altered.
  • Compliance evidence — the ComplianceReportWorker renders the evidence bundle (decisions, findings, control results) to Blob and records the evidenceLocation on the ComplianceReport. Where a framework demands tamper-proof retention, the container uses time-based immutability policies.

Storage Principles

  • Database-per-service — no service reads another's tables; cross-context data flows via APIs/events only.
  • Append-only where it matters — AuditEntry and PolicyDecision are immutable, the backbone of provable governance.
  • Tenant isolation at the storage layer — tenantId indexed on every table; ConnectSoft.Extensions.Saas.NHibernate applies session-level tenant filters; high-isolation editions can use database-per-tenant for audit/compliance.
  • Defence in depth — encryption at rest (Azure-managed keys, customer-managed keys for restricted editions), TLS in transit, managed-identity access to Key Vault and Blob, no secrets in connection strings (resolved via Key Vault).
  • Cache is never truth — Redis accelerates the PDP path but is always reconstructable from the relational system of record.