Model & Compute plane¶
Target
This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.
Models are replaceable; the Factory depends on logical model profiles, not provider names. Compute is a separate abstraction from models: the same model may run on a workstation, a local GPU server, a cloud pool or a customer's data centre.
Responsibilities¶
| Component | Target behaviour |
|---|---|
| Model gateway (GW-04) | One pipeline for every model invocation: trusted scope, pinned release and profile, policy decision, request hash, budget reservation, invocation receipt with actual provider/model/version, usage status (confirmed / estimated / unknown), redacted diagnostics. In-process first; a remote host only for central credential isolation, independent capacity or multiple consumers |
| Logical profiles | Aliases such as code-fast, code-agent-strong, reasoning-high, structured, vision, long-context, embedding, reranker, pii-local; each declares modality, tool and structured-output support, context and reliable-context limits, privacy/deployment constraints, latency class, price basis, embedding compatibility |
| Capability registry | Models evaluated against criteria (tool calling, parallel tool calls, strict JSON, schema compliance, instruction following, patch success, multi-turn consistency, file editing, shell usage), not by name or parameter count |
| Capability router | Select by capabilities, privacy, tenant policy, latency, context, cost, availability, region, loaded models, GPU capacity, historical success, evaluation score; explicit fallback chains that respect privacy policy; a confidential workload never falls back to an external provider if policy prohibits |
| Provider adapters | IChatClient / IEmbeddingGenerator adapters for approved providers; OpenAI-compatible and Anthropic-compatible protocols, embeddings, reranking, streaming, structured outputs, tool calls |
| Streaming and retry | Cancellation propagation, partial-output status, backpressure, final usage receipt; bounded retries by deadline, budget and side-effect rules; a partial stream is never spliced with another model's output |
| Compute pools | Metadata (GPU type, VRAM, region, tenant, price, utilization, queue depth, model cache, cold start, security boundary); a scheduler is a later capability |
| ModelOps | Discover → license and security review → benchmark → evaluate → quantize → deploy → warm → monitor → promote → retire; model license metadata lives in the registry |
Current implementation¶
ConnectSoft.Extensions.AI: keyedIChatClientregistrations for OpenAI, Azure OpenAI, Ollama, Azure AI Inference; embeddings; evaluator registration. No Anthropic adapter exists anywhere in the estate.ConnectSoft.Extensions.AI.AgentsFramework: keyed agent fallback tries other registered clients when a key is missing (openAI, azureOpenAI); it is configuration fallback, not provider-failure failover, and never tries ollama or azureAIInference.- No profile contract, capability registry, router, usage receipt or gateway pipeline exists. The prototype's model pickers are Claude-branded fixtures.
- Package pins: Microsoft.Extensions.AI 10.10.0; Microsoft.Agents.AI 1.21.0 (1.8.0 in the Factory messaging libraries).
Existing pages that belong to this plane¶
- Model access sections of agent runtime (
ModelRouterService,ModelPolicyService) - Provider "integration services" listed under the Integration Platform (OpenAI, Azure OpenAI, Ollama) are adapters of this plane, not separate services
- ADR-0002 and its review note
- Packet GW-04 contract in
handoffs/2026-10-08-ai-factory/analysis/documentation-drilldown-handoff.md(re-validated input for the gateway page)
New pages to be written in this plane: model gateway contract, capability registry and router, providers and profiles, compute pools.
Later capabilities¶
Compute scheduler and GPU autoscaling, provider cost comparison and local-first routing, caching and prompt compression, model batching, serverless execution, marketplace of model profiles.
Open questions¶
Q-11 (who approves profiles, providers and fallbacks), Q-09 (where data may be processed). Decided 2026-10-09 (D16): an Anthropic adapter is required for the first solution pack; see providers and profiles.
Platform design pages (2026-10-09, design plan P3)¶
- Narrative: overview · workflows · UI surfaces · security · observability · deployment · later capabilities
- Design pages: model gateway · capability registry and router · providers and profiles · compute pools · model profiles catalog
- Service design pages: ModelGatewayHost (host) · ModelRouterService · ComputePoolService
Modernization (2026-10-09, design plan M1)¶
Horizon now to next, per the technology radar: AI gateway patterns (profiles, routing, prompt caching, fallback, receipts); structured outputs; local model pools for restricted data; capability evidence from evaluation. Exists in ConnectSoft today: Keyed IChatClient registrations only. Each platform page of this plane states its horizon in its overview and later-capabilities pages.
Platforms in this plane¶
| Platform | Bounded contexts | Services | Modelled | Pages |
|---|---|---|---|---|
| Model & Compute Platform (PF-03) | BC-07, BC-41 | 7 | 3 | overview · services · APIs · events |
Bounded contexts¶
| Context | Platform | Authoritative state |
|---|---|---|
| BC-07 Model Access | Model & Compute Platform | Model profiles, route plans, invocation receipts |
| BC-41 Compute Pools & Placement | Model & Compute Platform | Model-serving compute pools, placement decisions, warm state |
Services¶
| Service | Platform | Context | Granularity | Status | Page |
|---|---|---|---|---|---|
| AzureOpenAIIntegrationService | Model & Compute Platform | BC-07 | external / canonical | unmodelled | — |
| ComputePoolService | Model & Compute Platform | BC-41 | module of host | target | design |
| ModelGatewayHost | Model & Compute Platform | BC-07 | standalone host | target | design |
| ModelPolicyService | Model & Compute Platform | BC-07 | module of host | unmodelled | — |
| ModelRouterService | Model & Compute Platform | BC-07 | module of host | target | design |
| OllamaIntegrationService | Model & Compute Platform | BC-07 | external / canonical | unmodelled | — |
| OpenAIIntegrationService | Model & Compute Platform | BC-07 | external / canonical | unmodelled | — |
Events crossing the plane boundary¶
| Direction | Event | Producer | Consumer | Topic |
|---|---|---|---|---|
| out | ComputePoolRegistered | ComputePoolService | StudioBff | compute-pools.pools.v1.compute-pool-registered |
| out | ComputePoolRegistered | ComputePoolService | DashboardService | compute-pools.pools.v1.compute-pool-registered |
| out | ComputePoolRegistered | ComputePoolService | AuditService | compute-pools.pools.v1.compute-pool-registered |
| out | ComputePoolStateChanged | ComputePoolService | StudioBff | compute-pools.pools.v1.compute-pool-state-changed |
| out | ComputePoolStateChanged | ComputePoolService | DashboardService | compute-pools.pools.v1.compute-pool-state-changed |
| out | ComputePoolStateChanged | ComputePoolService | AuditService | compute-pools.pools.v1.compute-pool-state-changed |
| out | ModelWarmed | ComputePoolService | DashboardService | compute-pools.cache.v1.model-warmed |
| out | ModelEvicted | ComputePoolService | DashboardService | compute-pools.cache.v1.model-evicted |
| out | PlacementDecided | ComputePoolService | DashboardService | compute-pools.placement.v1.placement-decided |
| out | PlacementDecided | ComputePoolService | AuditService | compute-pools.placement.v1.placement-decided |
| out | ModelProfileActivated | ModelRouterService | StudioBff | model-access.profiles.v1.model-profile-activated |
| out | ModelProfileActivated | ModelRouterService | DashboardService | model-access.profiles.v1.model-profile-activated |
| out | ModelProfileActivated | ModelRouterService | AuditService | model-access.profiles.v1.model-profile-activated |
| out | ModelProfileRetired | ModelRouterService | StudioBff | model-access.profiles.v1.model-profile-retired |
| out | ModelProfileRetired | ModelRouterService | AuditService | model-access.profiles.v1.model-profile-retired |
| out | DeploymentStateChanged | ModelRouterService | StudioBff | model-access.deployments.v1.deployment-state-changed |
| out | DeploymentStateChanged | ModelRouterService | DashboardService | model-access.deployments.v1.deployment-state-changed |
| out | DeploymentStateChanged | ModelRouterService | AuditService | model-access.deployments.v1.deployment-state-changed |
| out | CapabilityEvidenceRecorded | ModelRouterService | StudioBff | model-access.deployments.v1.capability-evidence-recorded |
| out | CapabilityEvidenceRecorded | ModelRouterService | DashboardService | model-access.deployments.v1.capability-evidence-recorded |
| out | RoutePlanResolved | ModelRouterService | DashboardService | model-access.routing.v1.route-plan-resolved |
| out | ModelInvoked | ModelRouterService | CostUsageService | model-access.invocations.v1.model-invoked |
| out | ModelInvoked | ModelRouterService | DashboardService | model-access.invocations.v1.model-invoked |
| out | ModelInvoked | ModelRouterService | QualityScoreService | model-access.invocations.v1.model-invoked |
| out | ModelInvoked | ModelRouterService | AgentTelemetryService | model-access.invocations.v1.model-invoked |
| out | ModelFallbackUsed | ModelRouterService | DashboardService | model-access.invocations.v1.model-fallback-used |
| out | ModelFallbackUsed | ModelRouterService | AuditService | model-access.invocations.v1.model-fallback-used |
| in | AssetReleasePublished | AgentRegistryService | ModelRouterService | asset-publication.releases.v1.asset-release-published |
| in | AssetReleaseDeprecated | AgentRegistryService | ModelRouterService | asset-publication.releases.v1.asset-release-deprecated |
| in | AssetReleaseRevoked | AgentRegistryService | ModelRouterService | asset-publication.releases.v1.asset-release-revoked |
| in | ResidencyRuleChanged | RetentionResidencyService | ModelRouterService | data-foundations.residency.v1.residency-rule-changed |
| in | DeletionOrderIssued | RetentionResidencyService | ModelGatewayHost | data-foundations.orders.v1.deletion-order-issued |
| in | ClassificationChanged | AuditService | ModelRouterService | audit.classification.v1.classification-changed |
| in | SecretRotated | SecretBrokerService | ModelRouterService | secret-broker.references.v1.secret-rotated |
| in | SecretRotated | SecretBrokerService | ComputePoolService | secret-broker.references.v1.secret-rotated |
| in | SecretReferenceRevoked | SecretBrokerService | ModelRouterService | secret-broker.references.v1.secret-reference-revoked |
| in | SecretReferenceRevoked | SecretBrokerService | ComputePoolService | secret-broker.references.v1.secret-reference-revoked |
| in | EvaluationCompleted | QualityScoreService | ModelRouterService | evaluation.runs.v1.evaluation-completed |
Gateways¶
| Gateway | Role | Fronts |
|---|---|---|
| GW-04 Model gateway | In-process pipeline first: profile resolution, policy, reservation, provider adapter, streaming, receipts | ModelGatewayHost |
Model coverage¶
| Count | |
|---|---|
| Services in the registry | 7 |
| Modelled (YAML exists) | 3 |
| With a design page | 3 |
| Unmodelled | 4 |