Compute pools¶
Target
This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.
The same model may run on a developer workstation, a local GPU server, a DGX-class device, a cloud GPU provider, Azure, AWS, another cloud, or a customer's data centre. Compute is therefore described independently of models, and model-serving compute is distinct from agent-execution workers (which run the orchestration loop and tools, not inference).
Pool metadata¶
GPU type · VRAM · CPU · RAM · region · tenant · price · availability · current utilization · queue depth · model cache (which models are loaded) · cold-start estimate · security boundary.
Placement policy (later)¶
A compute scheduler may provision or select a pool for a task from accelerator memory, privacy (private execution), region restrictions, cost and availability. Example from the Oct-2026 target: a task needing 70 GB of accelerator memory and private execution is placed on a cloud pool that satisfies a region restriction even if a cheaper pool exists elsewhere. Compute provisioning is a later-stage capability and must not block early versions.
Deployment topologies the pools must support¶
| Topology | Control plane | Runtime and compute | Data |
|---|---|---|---|
| SaaS | ConnectSoft-managed | ConnectSoft-managed | ConnectSoft-managed, tenant-isolated |
| Hybrid | ConnectSoft-managed | Customer runtime | Customer data and tools |
| Customer on-premises | Customer environment | Customer environment | Customer environment |
| Hybrid compute | Either | Local models for sensitive workloads; cloud models permitted for non-sensitive work | Per policy |
Contracts must not assume SaaS-only.
Current implementation¶
No compute abstraction, pool registry or scheduler exists. The template estate provides containerised hosts, Kubernetes and docker-compose manifests and Pulumi programs; Ollama is the only self-hosted inference adapter.