Skip to content

Model & Compute Platform — overview

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.

The Model & Compute Platform is the only path from any Factory component to a model. Agents, workflows and Knowledge name profiles; the platform resolves them to evaluated deployments under policy, invokes providers through adapters with streaming, structured outputs and tool calls, and writes a receipt for every call. Model-serving compute is a separate abstraction with its own registry and placement. The platform was added by the design plan (D22) to make the packet's gateway, capability registry, router and compute plane first-class rather than a corner of the Agent Mesh.
Platform PF-03 Model & Compute Platform (Model & Compute)
Bounded contexts BC-07, BC-41
Services 7 (1 hosts, 3 modules, 3 external, 0 later) — 3 modelled
Gateways GW-04
Status target

Purpose

  • Replaceable models, owned intelligence. Clients depend on logical profiles (code-fast, code-agent, reasoning-high, structured, vision, long-context, embedding, reranker, pii-local); providers and revisions change behind them.
  • Evidence over names. A deployment becomes routable only after the capability suite scores it against the quality-gate thresholds (D32) and a Platform admin approves it with its license metadata.
  • Policy before provider. Privacy class, region and tenant policy filter candidates before cost and latency score them; fallback is explicit and off by default.
  • Every call accounted. The receipt names the actual deployment, the policy decision, the usage status (reported, estimated, unknown) and the cost estimate; Usage Accounting settles it.
  • Compute is not a model. Pools carry accelerators, boundary, region, price and warm state; placement never relaxes a privacy or region requirement for a cheaper pool.

Bounded contexts and services

Context Owns Services (granularity)
BC-07 Model Access Profiles (read model of Published assets), deployments with capability evidence, route plans, invocation receipts ModelRouterService (module); ModelPolicyService merged; provider adapters are libraries, not services (OpenAIIntegrationService, AzureOpenAIIntegrationService, OllamaIntegrationService external-canonical)
BC-41 Compute Pools & Placement Pools, model cache, placement decisions ComputePoolService (module)
Host Protocols, adapters, credentials, composition ModelGatewayHost (standalone)
Service Context Granularity Status
AzureOpenAIIntegrationService BC-07 external / canonical unmodelled
ComputePoolService BC-41 module of host target
ModelGatewayHost BC-07 standalone host target
ModelPolicyService BC-07 module of host unmodelled
ModelRouterService BC-07 module of host target
OllamaIntegrationService BC-07 external / canonical unmodelled
OpenAIIntegrationService BC-07 external / canonical unmodelled

Role in the Factory

flowchart LR
    Mesh["Agent Mesh<br/>attempts"] -- "Invoke, ResolveRoutePlan" --> MC["Model & Compute<br/>ModelGatewayHost"]
    KM["Knowledge & Memory"] -- "Embed, Rerank" --> MC
    Clients["In-tenant clients<br/>OpenAI- and Anthropic-compatible protocols via GW-01"] --> MC
    MC -- "decisions, credentials" --> Gov["Governance"]
    MC -- "RecordUsage" --> CP["Control Plane<br/>Usage Accounting"]
    MC -- "profiles as Published assets" --> CP
    MC -- "capability evidence" --> Obs["Observability<br/>evaluation"]
    MC --> Prov["Providers: Anthropic, Azure OpenAI, OpenAI, Azure AI Inference, Ollama, self-hosted pools"]
Hold "Alt" / "Option" to enable pan & zoom

The routing flow (original handoff §71)

flowchart TB
    P["Requested profile<br/>code-agent-strong"] --> Rq["Requirements<br/>coding strong, tools strict, 64K reliable context, tenant private"]
    Rq --> C["Candidates<br/>local model A, cloud GPU model B, provider model C"]
    C --> F["Policy filters<br/>privacy class, region, tenant policy"]
    F --> S["Capability scores<br/>evidence from the evaluation suite"]
    S --> CL["Cost and latency"]
    CL --> D["Decision: route plan pinned for the run"]
    D --> I["Invoke with receipt"]
    I -. "failure, equivalent fallback only" .-> I
Hold "Alt" / "Option" to enable pan & zoom

Component view

flowchart LR
    subgraph P["Model & Compute Platform"]
        ModelGatewayHost["ModelGatewayHost"]
    end
    GW_04{"GW-04 Model gateway"} --> P
    P --> SecretBrokerService["SecretBrokerService"]
    P --> RuntimeEnvironmentService["RuntimeEnvironmentService"]
    P --> CostUsageService["CostUsageService"]
    P --> ext_model_providers["ext-model-providers"]
    P --> GovernanceHost["GovernanceHost"]
    P --> ControlPlaneApi["ControlPlaneApi"]
    P --> ext_identity_platform["ext-identity-platform"]
    P --> PolicyEngineService["PolicyEngineService"]
    P --> AgentRegistryService["AgentRegistryService"]
    P --> QualityScoreService["QualityScoreService"]
    P --> ArtifactService["ArtifactService"]
Hold "Alt" / "Option" to enable pan & zoom

Built from the ConnectSoft estate

Component Use here Status
ConnectSoft.Extensions.AI (keyed IChatClient, embeddings) provider adapters extend (Anthropic adapter to add, D16)
Microservice template (BaseTemplate) ModelGatewayHost exists
MessagingModel Orleans adapter outbox and inbox exists
Observability libraries GenAI spans and metrics exists
ConnectSoft.Extensions.RateLimiting per-tenant and per-profile limits at the gateway exists
Evaluation (Microsoft.Extensions.AI.Evaluation via BC-22) capability evidence extend

Modernization (2 to 3 year horizon)

  • AI gateway patterns: routing, caching, prompt caching, fallback and cost receipts as one pipeline rather than per-agent code.
  • OpenTelemetry GenAI conventions on every invocation span.
  • Structured outputs and tool-call reliability as measured capabilities, not assumptions.
  • Compute scheduler that provisions pools on demand (later), keeping privacy and region as hard constraints.

Current implementation

Adapters exist for OpenAI, Azure OpenAI, Ollama and Azure AI Inference; the agents framework resolves them by key with a missing-key fallback that is not provider failover. No profile, registry, router, receipt, pool or Anthropic adapter exists (providers and profiles).

Final-state summary

One gateway host, two modules, five provider families behind evaluated profiles, a receipt for every call, pools described independently of models. Models are swapped under the profiles; the Factory's evidence, receipts and policies stay.