ComputePoolService¶

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.


Bounded context	BC-41 Compute Pools & Placement
Platform / plane	Model & Compute Platform / Model & Compute
Deployment granularity	module of host · host `ModelGatewayHost`
Status	target · `ConnectSoft.Factory.ModelAccess.Compute`
Built from	connectsoft-microservice · libraries: connectsoft-extensions-saas-abstractions, connectsoft-extensions-messaging-model, connectsoft-extensions-messaging-model-orleans, connectsoft-extensions-persistence-model-nhibernate, connectsoft-extensions-actor-model-orleans
Decisions	ADR-0004, ADR-0017
Traceability	`FR-04` `FUX-08` `NFR-08` `NFR-09` `NFR-12` `NFR-18` `XL-04` `XL-16`

Purpose and responsibilities¶

The same model may run on a developer workstation, a local GPU server, a DGX-class device, a cloud GPU provider, Azure or a customer's data centre (original handoff §14). ComputePoolService describes that compute independently of models: each pool carries GPU type, VRAM, CPU, RAM, region, tenant, price, availability, utilization, queue depth, model cache, cold-start estimate and security boundary. When the router picks a self-hosted deployment, this service answers the placement question: which pool satisfies the accelerator memory, privacy, region and price requirement right now, and what will the cold start cost. Model-serving compute is distinct from agent-execution workers (BC-26 in Runtime Cloud), which run the orchestration loop and tools, not inference.

Provisioning of new cloud pools (the "compute scheduler" of the original design) is a later-stage capability routed through Runtime Cloud and Pulumi; the first design places onto registered pools only.

Context and boundaries¶

flowchart LR
    S["ComputePoolService"]
    ModelRouterService["ModelRouterService"] -->|grpc| S
    StudioBff["StudioBff"] -->|sync-http| S
    S -->|grpc| SecretBrokerService["SecretBrokerService"]
    S -->|grpc| RuntimeEnvironmentService["RuntimeEnvironmentService"]
    S -->|grpc| CostUsageService["CostUsageService"]
    S -. "ComputePoolRegistered" .-> E_ComputePoolRegistered(("event"))
    S -. "ComputePoolStateChanged" .-> E_ComputePoolStateChanged(("event"))
    S -. "ModelWarmed" .-> E_ModelWarmed(("event"))
    S -. "ModelEvicted" .-> E_ModelEvicted(("event"))
    S -. "PlacementDecided" .-> E_PlacementDecided(("event"))

Hold "Alt" / "Option" to enable pan & zoom

Domain model¶

ComputePool — One pool of accelerators with its metadata (GPU type, VRAM, CPU, RAM, region, tenant, price, availability), live utilization and queue depth, model cache and cold-start estimate. (catalog entry)

Field	Type	Required	Description
`poolId`	`string`	yes
`tenantId`	`string`	yes	Platform tenant for ConnectSoft pools; the owning tenant for hybrid-compute pools.
`kind`	`enum:PoolKind`	yes
`state`	`enum:PoolState`	yes
`region`	`string`	yes
`securityBoundary`	`enum:SecurityBoundary`	yes
`gpuType`	`string`	yes
`gpuCount`	`int`	yes
`vramGb`	`int`	yes
`cpuCores`	`int`	yes
`ramGb`	`int`	yes
`pricePerHourUsd`	`decimal`	yes
`utilization`	`decimal`	yes	0 to 1, from the collector.
`queueDepth`	`int`	yes
`coldStartSeconds`	`int`	yes	Estimated time to load a model that is not cached.
`endpointRef`	`string`	yes	Serving endpoint reference; credentials from the Secret Broker.
`version`	`long`	yes

Invariants:

A pool is placeable only in state Active and with reported utilization younger than the staleness window.
A pool's security boundary and region are facts set at registration; placement reads them and never relaxes a requirement to fit a cheaper pool.
The model cache lists what is loaded now; a placement onto a pool without the model adds the cold-start estimate to the decision.

stateDiagram-v2
    Draft --> Active : Activate
    Active --> Draining : Drain
    Draining --> Active : Resume
    Draining --> Retired : Retire

Hold "Alt" / "Option" to enable pan & zoom

Events: ComputePoolRegistered, ComputePoolStateChanged, ModelWarmed, ModelEvicted · Tables: ComputePools, PoolModelCache · Repository: ComputePoolRepository

PlacementDecision — The record of one placement request — requirements, candidates considered with the reason each was kept or excluded, and the chosen pool. (catalog entry)

Field	Type	Required	Description
`placementId`	`guid`	yes
`tenantId`	`string`	yes
`deploymentId`	`string`	yes	The self-hosted model deployment to place (BC-07).
`runId`	`guid`	no
`requirements`	`json`	yes	Accelerator memory, privacy and security boundary, region, latency class, max price.
`candidates`	`json`	yes	Pools considered with keep or exclude reasons.
`chosenPoolId`	`string`	no
`outcome`	`enum:PlacementOutcome`	yes
`decidedAt`	`datetime`	yes

Invariants:

Every candidate exclusion names its reason (boundary, region, memory, price, capacity, state) so that an operator can see why a cheaper pool was not chosen.
PolicyDenied is returned, not worked around, when no pool satisfies the privacy or region requirement.

Events: PlacementDecided · Tables: PlacementDecisions · Repository: PlacementDecisionRepository

erDiagram
    ComputePool {
        string poolId PK
        string tenantId
        enum_PoolKind kind
        enum_PoolState state
        string region
        enum_SecurityBoundary securityBoundary
        string gpuType
        int gpuCount
        int vramGb
        int cpuCores
        int ramGb
        decimal pricePerHourUsd
        decimal utilization
        int queueDepth
        int coldStartSeconds
        string endpointRef
        long version
    }
    PlacementDecision {
        guid placementId PK
        string tenantId
        string deploymentId
        guid runId
        json requirements
        json candidates
        string chosenPoolId
        enum_PlacementOutcome outcome
        datetime decidedAt
    }
    CachedModel {
        string model

        datetime loadedAt

        datetime lastUsedAt

        int vramGb

    }
    ComputePool ||--o{ CachedModel : contains

Hold "Alt" / "Option" to enable pan & zoom

Placement example (from the original design): a task needs 70 GB of accelerator memory and private execution; the local GPU is insufficient, cloud pool A is available, cloud pool B is cheaper but in another region; policy restricts the region; result: pool A, with the exclusion reasons recorded on the decision.

Entities and value objects¶

CachedModel (entity, in ComputePool) — A model currently loaded on the pool.

Field	Type	Required
`model`	`string`	yes
`loadedAt`	`datetime`	yes
`lastUsedAt`	`datetime`	yes
`vramGb`	`int`	yes

Data model and storage¶

Store	Role	Engine / profile	Tables
Model access store	primary	azure-sql / nhibernate-sqlserver	ComputePools, PoolModelCache, PlacementDecisions

Table	Owner	Columns	Indexes
`dbo.ComputePools`	ComputePool	18	1
`dbo.PoolModelCache`	CachedModel	6	0
`dbo.PlacementDecisions`	PlacementDecision	9	1

Full DDL: appendix.

APIs¶

Method	Path	Operation	Scope	Side effects	Idempotency	Gateway	Summary
`GET`	`/factory/compute-pools`	ListComputePools	`factory.compute.read`	read	natural	GW-03	Pools with kind, region, boundary, capacity, utilization, queue depth and cached models.
`POST`	`/factory/compute-pools`	RegisterComputePool	`factory.compute.admin`	shared-write	key-header	GW-03	Register a pool (Platform admin; a tenant's Platform admin for hybrid-compute pools).
`POST`	`/factory/compute-pools/{poolId}/drain`	DrainComputePool	`factory.compute.admin`	production	natural	GW-03	Stop new placements and let loaded models finish (production action on serving capacity).
`POST`	`/factory/compute-pools/{poolId}/warm`	WarmModel	`factory.compute.admin`	shared-write	key-header	GW-03	Pre-load a model on a pool before expected demand.
`GET`	`/factory/compute-pools/placements/{placementId}`	GetPlacementDecision	`factory.compute.read`	read	natural	GW-03	Why a pool was chosen or no pool fitted.

gRPC service	Method	Streaming	Visibility	Side effects	Summary
`ComputePools`	`Place`	none	internal	shared-write	Choose a pool for a self-hosted deployment under the request's privacy, region, memory, latency and price constraints; returns the decision id and the endpoint reference.
`ComputePools`	`ReportUtilization`	client	internal	shared-write	Pools and their agents report utilization, queue depth and the model cache.

Messages¶

Event	Topic	Scope / v	Aggregate	Consumers	Summary
ComputePoolRegistered	`compute-pools.pools.v1.compute-pool-registered`	integration / 1	ComputePool	StudioBff, DashboardService, AuditService	A pool exists with its metadata.
ComputePoolStateChanged	`compute-pools.pools.v1.compute-pool-state-changed`	integration / 1	ComputePool	ModelRouterService, StudioBff, DashboardService, AuditService	Activated, draining, resumed or retired.
ModelWarmed	`compute-pools.cache.v1.model-warmed`	integration / 1	ComputePool	ModelRouterService, DashboardService	A model is loaded on a pool.
ModelEvicted	`compute-pools.cache.v1.model-evicted`	integration / 1	ComputePool	ModelRouterService, DashboardService	A model was unloaded from a pool.
PlacementDecided	`compute-pools.placement.v1.placement-decided`	integration / 1	PlacementDecision	DashboardService, AuditService	A placement request was answered.

ComputePoolRegistered payload:

Field	Type	Required
`poolId`	`string`	yes
`tenantId`	`string`	yes
`kind`	`enum:PoolKind`	yes
`region`	`string`	yes
`securityBoundary`	`enum:SecurityBoundary`	yes

ComputePoolStateChanged payload:

Field	Type	Required
`poolId`	`string`	yes
`tenantId`	`string`	yes
`fromState`	`enum:PoolState`	yes
`toState`	`enum:PoolState`	yes

ModelWarmed payload:

Field	Type	Required
`poolId`	`string`	yes
`tenantId`	`string`	yes
`model`	`string`	yes
`vramGb`	`int`	yes

ModelEvicted payload:

Field	Type	Required
`poolId`	`string`	yes
`tenantId`	`string`	yes
`model`	`string`	yes
`reason`	`string`	yes

PlacementDecided payload:

Field	Type	Required
`placementId`	`guid`	yes
`tenantId`	`string`	yes
`deploymentId`	`string`	yes
`chosenPoolId`	`string`	no
`outcome`	`enum:PlacementOutcome`	yes

Command	Handled by	Result events
`RegisterComputePool`	ComputePool	ComputePoolRegistered
`DrainComputePool`	ComputePool	ComputePoolStateChanged
`WarmModel`	ComputePool	ModelWarmed
`Place`	PlacementDecision	PlacementDecided

None modelled

No consumed events are modelled for this item yet.

Workers and jobs¶

Worker	Trigger	Output events	Retry	Idempotency key	Summary
UtilizationCollector	schedule `/1 * * *`	—	fixed	`poolId + minute`	Marks pools whose utilization report is older than the staleness window as not placeable and refreshes cold-start estimates from observed loads.
WarmKeeper	schedule `/5 * * *`	ModelWarmed, ModelEvicted	exponential	`poolId + model + window`	Keeps the models of profiles with warm-keep policy loaded on their pools and evicts models unused beyond the idle window.

Actors and grains¶

Grain	Key	State	Reminders	Reentrant	Summary
PoolGrain	string = poolId	`state`, `utilization`, `queueDepth`	staleness	no	Serialises placements and cache updates of one pool so that capacity is never double-booked.

Sagas and long-running flows¶

None modelled

No sagas are modelled for this item yet.

Options and configuration¶

Section	Key	Type	Default	Reloadable	Secret	Summary
`ComputePools:Placement`	`Weights`	json	`{ warm: 0.4, price: 0.3, utilization: 0.2, latency: 0.1 }`	yes	no	Scoring weights after the hard filters (boundary, region, memory, state).
`ComputePools:Placement`	`UtilizationStalenessSeconds`	int	`120`	yes	no	A pool without a fresh report is not placeable.
`ComputePools:Warm`	`IdleEvictMinutes`	int	`30`	yes	no	Evict models unused for this long unless warm-keep applies.

Metrics and observability¶

Metric	Instrument	Unit	Dimensions	Summary
`connectsoft.factory.compute_pools.pools`	updown	{pool}	tenant, kind, state, region	Pools by state.
`connectsoft.factory.compute_pools.utilization`	gauge	1	pool	Reported utilization.
`connectsoft.factory.compute_pools.placements`	counter	{placement}	tenant, outcome	Placement outcomes.
`connectsoft.factory.compute_pools.cold_starts`	counter	{load}	pool	Model loads caused by placements.

Security¶


Authentication	client-credentials
Scopes	`factory.compute.read` (Read pools and placements.); `factory.compute.admin` (Register, drain, warm (Platform admin).)
Roles	Operator, PlatformAdmin, Observer
Tenant isolation	row-filter
Data classification	confidential
Max side-effect class	production
Secrets	`pool-endpoint-credentials` (key-vault, rotate 90 d)

Hybrid-compute pools are tenant-owned rows: the tenant's Platform admin registers them and only that tenant's deployments may be placed on them; ConnectSoft pools belong to the platform tenant with a ConnectSoftRegional or Shared boundary.

Dependencies¶

Outbound

Target	Kind	Purpose	Failure mode	Required
SecretBrokerService	grpc	Short-lived credentials for pool endpoints.	fail-fast	yes
RuntimeEnvironmentService	grpc	Provisioning of cloud pools through Pulumi programs (later-stage scheduler).	degrade	no
CostUsageService	grpc	Record pool hours as usage.	degrade	no

Inbound

Target	Kind	Purpose	Failure mode	Required
ModelRouterService	grpc	Place and warm-state reads.	—	yes
StudioBff	sync-http	Pool administration views.	—	yes

Failure and recovery¶

Scenario	Detection	Recovery	Data impact	RTO (min)
No pool satisfies privacy and region	placement returns PolicyDenied	the router excludes self-hosted candidates and fails the invocation if no provider deployment fits; never relaxes the requirement	none	0
Utilization reports stop	staleness reminder	pool not placeable until a report arrives; alert	none	10
Pool endpoint down after placement	adapter error from the router	deployment marked Degraded by the health probe; placements avoid the pool	none	10

NFR targets¶

Target	Class value (pilot-interactive)	Override
latency_p95_ms	2000	—
availability_pct	99.5	—
throughput_rps	50	—
rpo_minutes	15	—
rto_minutes	240	—
cost_class	low	—

Deployment¶


Granularity	module of host · host `ModelGatewayHost`
Topologies	saas-default, hybrid, on-prem, hybrid-compute
Runtime	—; replicas —–—
Ports	—

Current implementation¶

No compute abstraction, pool registry or scheduler exists; Ollama is the only self-hosted inference adapter and Pulumi programs exist for infrastructure. See compute pools.

Later capabilities¶

Compute scheduler that provisions pools on demand through Runtime Cloud and Pulumi; cost-aware eviction; GPU sharing across tenants with isolation evidence; pool hours as a Factory meter.

Decisions and traceability¶

Family	Ids
FR	FR-04
FUX	FUX-08
NFR	NFR-08, NFR-09, NFR-12, NFR-18
XL	XL-04, XL-16

Compute pools · ModelRouterService · Runtime Cloud · Deployment profiles
Model & Compute Platform microservices · bounded contexts · APIs · events · data model
Services catalog · Ownership map

ComputePoolService¶

Purpose and responsibilities¶

Context and boundaries¶

Domain model¶

Entities and value objects¶

Data model and storage¶

APIs¶

Messages¶

Workers and jobs¶

Actors and grains¶

Sagas and long-running flows¶

Options and configuration¶

Metrics and observability¶

Security¶

Dependencies¶

Failure and recovery¶

NFR targets¶

Deployment¶

Current implementation¶

Later capabilities¶

Decisions and traceability¶

Related¶