RetentionResidencyService

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.

Bounded context	BC-42 Data & Event Foundations
Platform / plane	Data & Event Platform / Data & Event
Deployment granularity	module of host · host DataEventFoundations
Status	target · ConnectSoft.Factory.DataEvent.Retention
Built from	connectsoft-microservice · libraries: connectsoft-extensions-persistence-model-nhibernate, connectsoft-extensions-saas-abstractions, connectsoft-extensions-messaging-model, connectsoft-extensions-scheduler-model-hang-fire
Decisions	ADR-0019, ADR-0015
Traceability	FR-12 FUX-11 FUX-12 NFR-14 NFR-15 NFR-16 XL-09 XL-13

Purpose and responsibilities

RetentionResidencyService is where the data-governance decisions (D31, ADR-0019) stop being prose. Retention rules say how long each store class keeps data of each classification — traces 30 days, runs and artifacts 1 year, audit 7 years redacted, projections 13 months — as platform ceilings a tenant may shorten; residency rules record the tenant's selected region (West Europe default, Israel Central available) and the engines and regions allowed per class, and feed the model router's privacy class; governance orders carry out tenant or project deletion (production class, approval-gated, complete within 30 days with every owner's acknowledgement, derived indexes and memory deleted with the source), run purges at expiry, exports in open formats with a manifest, and legal holds that suspend expiry.

Context and boundaries

flowchart LR
    S["RetentionResidencyService"]
    StoreProfileService["StoreProfileService"] -->|grpc| S
    ModelRouterService["ModelRouterService"] -->|grpc| S
    StudioBff["StudioBff"] -->|sync-http| S
    S -->|grpc| RunControlService["RunControlService"]
    S -->|grpc| AuditService["AuditService"]
    S -->|grpc| ArtifactService["ArtifactService"]
    S -->|event| AlertRuleService["AlertRuleService"]
    S -. "RetentionRuleChanged" .-> E_RetentionRuleChanged(("event"))
    S -. "ResidencyRuleChanged" .-> E_ResidencyRuleChanged(("event"))
    S -. "DeletionOrderIssued" .-> E_DeletionOrderIssued(("event"))
    S -. "DeletionOrderCompleted" .-> E_DeletionOrderCompleted(("event"))
    S -. "ExportCompleted" .-> E_ExportCompleted(("event"))
    S -. "LegalHoldChanged" .-> E_LegalHoldChanged(("event"))

Hold "Alt" / "Option" to enable pan & zoom

Orders fan out to owning services as tasks and complete only on acknowledgement; an owner that never acknowledges leaves the order Overdue, alerts the Operator and is recorded in audit evidence. The tenant-deletion saga orders the owners: derived stores first, then artifacts and authoritative stores, then audit redaction and the commercial close.

Domain model

RetentionRule — How long a store class keeps data of a classification for a scope (platform default, tenant override that may only shorten), and what happens at expiry (delete, redact, archive). (catalog entry)

Field	Type	Required	Description
ruleId	guid	yes
tenantId	string	yes
storeClass	string	yes
classification	string	yes	public
retentionDays	int	yes
expiryAction	string	yes	delete
legalHoldExempt	bool	yes
active	bool	yes

Invariants:

• Platform defaults (D31) are the ceiling; a tenant may shorten retention, never lengthen it beyond the platform rule for the class.
• Audit evidence is never deleted before 7 years and is redacted, not purged; legal holds suspend expiry for the held scope.

Events: RetentionRuleChanged · Tables: RetentionRules · Repository: RetentionRuleRepository

ResidencyRule — The tenant's selected region and the engines and regions allowed for every store class; evaluated when a store is bound and when a model deployment or connector is selected. (catalog entry)

Field	Type	Required	Description
tenantId	string	yes
region	string	yes
allowedRegions	list:string	yes
allowedEngines	json	yes	Per store class.
modelDataResidency	string	yes	same-region
changedAt	datetime	yes

Invariants:

• Changing a tenant's region issues a migration order; data is never in two regions at once except during a tracked move.

Events: ResidencyRuleChanged · Tables: ResidencyRules · Repository: ResidencyRuleRepository

GovernanceOrder — A deletion, purge, export or legal-hold order for a scope, fanned out to every owning service as a task with an acknowledgement receipt; completes only when every owner acknowledged; overdue at 30 days (deletion) with an alert and audit evidence. (catalog entry)

Field	Type	Required	Description
orderId	guid	yes
tenantId	string	yes
kind	enum:OrderKind	yes
scope	json	yes	tenant, project, run set or classification
issuedBy	string	yes
approvalId	guid	no	Tenant deletion is a production-class action with an approval.
state	enum:OrderState	yes
tasks	json	yes	Per owning service with state and receipt.
dueAt	datetime	yes
completedAt	datetime	no
exportRef	string	no	Artifact reference of an export bundle.

Invariants:

• Deletion removes derived indexes and memory with the source (D31); the order is not complete until the vector, graph, cache and projection owners acknowledged.
• An export bundle is in open formats and includes a manifest of what was and was not exported (audit evidence stays redacted).

stateDiagram-v2
    Issued --> InProgress : TasksDispatched
    InProgress --> AwaitingAcknowledgements : AllTasksSent
    AwaitingAcknowledgements --> Completed : AllAcknowledged
    AwaitingAcknowledgements --> Overdue : DueElapsed
    Overdue --> Completed : AllAcknowledged
    Issued --> Cancelled : Cancel

Hold "Alt" / "Option" to enable pan & zoom

Events: DeletionOrderIssued, DeletionOrderCompleted, ExportCompleted, LegalHoldChanged · Tables: GovernanceOrders, GovernanceOrderTasks · Repository: GovernanceOrderRepository

erDiagram
    RetentionRule {
        guid ruleId PK
        string tenantId
        string storeClass
        string classification
        int retentionDays
        string expiryAction
        bool legalHoldExempt
        bool active
    }
    ResidencyRule {
        string tenantId PK
        string region
        list_string allowedRegions
        json allowedEngines
        string modelDataResidency
        datetime changedAt
    }
    GovernanceOrder {
        guid orderId PK
        string tenantId
        enum_OrderKind kind
        json scope
        string issuedBy
        guid approvalId
        enum_OrderState state
        json tasks
        datetime dueAt
        datetime completedAt
        string exportRef
    }

Hold "Alt" / "Option" to enable pan & zoom

Entities and value objects

None modelled

No entities or value objects are modelled for this item yet.

Data model and storage

Store	Role	Engine / profile	Tables
Data & Event foundations store	primary	azure-sql / nhibernate-sqlserver	RetentionRules, ResidencyRules, GovernanceOrders, GovernanceOrderTasks
Artifact content store	blob	blob / azure-blob	—

Table	Owner	Columns	Indexes
dbo.RetentionRules	RetentionRule	8	1
dbo.ResidencyRules	ResidencyRule	6	0
dbo.GovernanceOrders	GovernanceOrder	10	1
dbo.GovernanceOrderTasks	GovernanceOrder	6	0

Full DDL: appendix.

APIs

Method	Path	Operation	Scope	Side effects	Idempotency	Gateway	Summary
GET	/factory/data/retention	ListRetentionRules	factory.data.read	read	natural	GW-03	Effective retention per store class and classification for the tenant (platform defaults plus overrides).
PUT	/factory/data/retention/{storeClass}/{classification}	SetRetentionRule	factory.data.admin	shared-write	natural	GW-03	Shorten retention for the tenant (Workspace admin) or set a platform default (Platform admin).
GET	/factory/data/residency	GetResidency	factory.data.read	read	natural	GW-03	The tenant's region and allowed engines per class.
PUT	/factory/data/residency	SetResidency	factory.data.admin	production	natural	GW-03	Select the tenant's region (at tenant creation; later changes issue a migration order).
POST	/factory/data/orders	IssueGovernanceOrder	factory.data.admin	production	key-header	GW-03	Issue a deletion, purge, export or legal-hold order for a scope; tenant deletion is production-class and approval-gated.
GET	/factory/data/orders/{orderId}	GetGovernanceOrder	factory.data.read	read	natural	GW-03	Order state with per-owner tasks, receipts and the export bundle reference.
POST	/factory/data/orders/{orderId}/acknowledge	AcknowledgeOrderTask	factory.data.acknowledge	shared-write	natural	GW-03	An owning service acknowledges its task with a receipt (internal).

gRPC service	Method	Streaming	Visibility	Side effects	Summary
Retention	EvaluateResidency	none	internal	read	Check a candidate store binding, model deployment or connector region against the tenant's residency rule.

Messages

Event	Topic	Scope / v	Aggregate	Consumers	Summary
RetentionRuleChanged	data-foundations.retention.v1.retention-rule-changed	integration / 1	RetentionRule	StudioBff, AuditService, KnowledgeHost, ControlPlaneApi, ObservabilityHost	An effective retention rule changed.
ResidencyRuleChanged	data-foundations.residency.v1.residency-rule-changed	integration / 1	ResidencyRule	StudioBff, AuditService, ModelRouterService, StoreProfileService	A tenant's region or allowed engines changed.
DeletionOrderIssued	data-foundations.orders.v1.deletion-order-issued	integration / 1	GovernanceOrder	ControlPlaneApi, AgentRuntimeWorker, ModelGatewayHost, KnowledgeHost, ToolExecutorHost, IntegrationHost, GovernanceHost, ObservabilityHost, StudioBff	Every owning service must delete the scope and acknowledge.
DeletionOrderCompleted	data-foundations.orders.v1.deletion-order-completed	integration / 1	GovernanceOrder	FleetIncidentService, AuditService, StudioBff, FactoryCommercialService	Every owner acknowledged; the scope is gone.
ExportCompleted	data-foundations.orders.v1.export-completed	integration / 1	GovernanceOrder	FleetIncidentService, AuditService, StudioBff, StudioNotificationService	An export bundle is available.
LegalHoldChanged	data-foundations.orders.v1.legal-hold-changed	integration / 1	GovernanceOrder	AuditService, KnowledgeHost, ControlPlaneApi, ObservabilityHost, StudioBff	A legal hold was placed or lifted on a scope.

RetentionRuleChanged payload:

Field	Type	Required	Description
ruleId	guid	yes
tenantId	string	yes
storeClass	string	yes
classification	string	yes
retentionDays	int	yes

ResidencyRuleChanged payload:

Field	Type	Required	Description
tenantId	string	yes
region	string	yes
modelDataResidency	string	yes

DeletionOrderIssued payload:

Field	Type	Required	Description
orderId	guid	yes
tenantId	string	yes
kind	enum:OrderKind	yes
scope	json	yes
dueAt	datetime	yes

DeletionOrderCompleted payload:

Field	Type	Required	Description
orderId	guid	yes
tenantId	string	yes
completedAt	datetime	yes

ExportCompleted payload:

Field	Type	Required	Description
orderId	guid	yes
tenantId	string	yes
exportRef	string	yes
format	enum:ExportFormat	yes

LegalHoldChanged payload:

Field	Type	Required	Description
orderId	guid	yes
tenantId	string	yes
scope	json	yes
active	bool	yes

Command	Handled by	Result events	Summary
SetRetentionRule	RetentionRule	RetentionRuleChanged
SetResidency	ResidencyRule	ResidencyRuleChanged
IssueGovernanceOrder	GovernanceOrder	DeletionOrderIssued, LegalHoldChanged
AcknowledgeOrderTask	GovernanceOrder	DeletionOrderCompleted

Event	Producer	Handler	Purpose
StoreBindingChanged	StoreProfileService	CheckResidency	Verify every binding against the tenant's residency rule.

Workers and jobs

Worker	Trigger	Output events	Retry	Idempotency key	Summary
RetentionSweeper	schedule 0 3 * * *	DeletionOrderIssued	fixed	ruleId + day	Computes expiry per rule and issues purge tasks to owning services (delete, redact or archive) with receipts.
OrderDispatcher	event	DeletionOrderCompleted	exponential	orderId + ownerService	Fans an order out to every owning service as tasks, tracks acknowledgements and marks orders Completed or Overdue.
Exporter	event	ExportCompleted	exponential	orderId	Collects export bundles from owning services into one archive with a manifest in open formats.

Actors and grains

None modelled

No grains are modelled for this item yet.

Sagas and long-running flows

TenantDeletionSaga — correlation orderId; starts on RetentionResidencyService.DeletionOrderIssued

Step	Trigger	Do	Compensation
1	DeletionOrderIssued (TenantDeletion)	Verify the production-class approval bound to the order; record evidence.	Cancel the order; evidence of refusal.
2	ApprovalVerified	Dispatch tasks to derived-store owners first (ObservabilityHost projections, KnowledgeHost indexes and memory, cache).	Mark the order Overdue naming the failing owner; alert.
3	DerivedAcknowledged	Dispatch tasks to artifact and authoritative owners (KnowledgeHost artifacts, ControlPlaneApi, AgentRuntimeWorker, ModelGatewayHost, ToolExecutorHost, IntegrationHost).	Stop remaining dispatches; keep acknowledged deletions (irreversible); Overdue.
4	AuthoritativeAcknowledged	Ask GovernanceHost to redact (never purge) audit evidence for the scope; FactoryCommercialService closes the subscription.	Overdue with the governance task named.
5	AllAcknowledged	Record completion evidence; emit DeletionOrderCompleted.	none

sequenceDiagram
    participant WA as Workspace admin
    participant R as RetentionResidencyService
    participant RC as RunControlService (approvals)
    participant O as Owning hosts
    participant A as AuditService
    WA->>R: IssueGovernanceOrder (TenantDeletion)
    R->>RC: approval (production class) bound to the order
    RC-->>R: ApprovalDecided
    R-)O: DeletionOrderIssued (scope, dueAt = +30 d)
    O->>R: AcknowledgeOrderTask (receipt) × every owner
    R->>A: redact audit evidence for the scope (never purge)
    R-)A: DeletionOrderCompleted (evidence)

Hold "Alt" / "Option" to enable pan & zoom

Options and configuration

Section	Key	Type	Default	Reloadable	Secret	Summary
DataEvent:Retention	PlatformDefaults	json	{ traces: 30, runs: 365, artifacts: 365, audit: 2555, projections: 395 }	no	no	Platform retention days per class (D31).
DataEvent:Retention	DeletionWindowDays	int	30	no	no	Days to complete a deletion before it is Overdue (D31).

Metrics and observability

Metric	Instrument	Unit	Dimensions	Summary
connectsoft.factory.data_event.orders	updown	{order}	tenant, kind, state	Orders by kind and state.
connectsoft.factory.data_event.order_completion_days	histogram	d	kind	Time to complete an order.
connectsoft.factory.data_event.residency_violations	counter	{check}	tenant, kind	Residency checks that refused.

Security

Authentication	client-credentials
Scopes	factory.data.read (Read rules and orders.); factory.data.admin (Rules, residency, orders (Workspace admin for the tenant, Platform admin for defaults).); factory.data.acknowledge (Owning services acknowledge tasks.)
Roles	WorkspaceAdmin, PlatformAdmin, Operator, Observer
Tenant isolation	row-filter
Data classification	confidential
Max side-effect class	production
Secrets	—

Dependencies

Outbound

Target	Kind	Purpose	Failure mode	Required
RunControlService	grpc	Approval for tenant deletion (production class).	fail-fast	yes
AuditService	grpc	Classification labels; evidence of orders.	fail-fast	yes
ArtifactService	grpc	Export bundles.	degrade	yes
AlertRuleService	event	Overdue orders.	queue	no

Inbound

Target	Kind	Purpose	Failure mode	Required
StoreProfileService	grpc	EvaluateResidency on binding.	—	yes
ModelRouterService	grpc	EvaluateResidency for deployment selection.	—	yes
StudioBff	sync-http	Retention, residency and order views.	—	yes

Failure and recovery

Scenario	Detection	Recovery	Data impact	RTO (min)
Owner never acknowledges	due date elapsed	order Overdue; alert; the owner's task is retried and escalated to the Operator; evidence records the delay	delayed deletion	1440

NFR targets

Target	Class value (pilot-batch)	Override
latency_p95_ms	0	—
availability_pct	99.5	—
throughput_rps	10	—
rpo_minutes	15	—
rto_minutes	240	—
cost_class	medium	—

Deployment

Granularity	module of host · host DataEventFoundations
Topologies	saas-default, hybrid, on-prem, hybrid-compute
Runtime	—; replicas —–—
Ports	—

Current implementation

Retention and residency are decisions (D31) and a page (data governance); the SaaS Tenants template has tenant lifecycle states but no deletion fan-out; nothing enforces retention across stores.

Later capabilities

Residency migration orders that move a tenant between regions with a tracked double-write window; customer-held keys per tenant; automated export scheduling per edition (D34).

Decisions and traceability

Family	Ids
FR	FR-12
FUX	FUX-11, FUX-12
NFR	NFR-14, NFR-15, NFR-16
XL	XL-09, XL-13

Related

• Overview · Data governance · Governance security · FactoryCommercialService

• Data & Event Platform microservices · bounded contexts · APIs · events · data model

• Services catalog · Ownership map