ModelRouterService

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.

Bounded context	BC-07 Model Access
Platform / plane	Model & Compute Platform / Model & Compute
Deployment granularity	module of host · host ModelGatewayHost
Status	target · ConnectSoft.Factory.ModelAccess.Routing
Built from	connectsoft-microservice · libraries: connectsoft-extensions-ai, connectsoft-extensions-saas-abstractions, connectsoft-extensions-messaging-model, connectsoft-extensions-persistence-model-nhibernate, connectsoft-extensions-rate-limiting
Decisions	ADR-0002, ADR-0013, ADR-0014, ADR-0021
Traceability	FR-04 FR-09 FUX-08 FUX-10 NFR-03 NFR-07 NFR-11 NFR-12 NFR-13 XL-04 XL-08 XL-12 XL-16

Purpose and responsibilities

Model selection must not be based on model name or parameter count (original handoff §12). ModelRouterService makes the Factory depend on profiles (code-agent, reasoning-high, embedding, pii-local, see the model profiles catalog) and resolves them to deployments whose capabilities were evaluated against the quality-gate thresholds (D32): tool calling, parallel tool calls, strict JSON and JSON schema, instruction following, agent task success, patch success, reliable context, file editing, shell tool usage, multi-turn consistency, planning, error recovery. The router filters by policy (privacy class, region, tenant policy), scores candidates by evidence, cost, latency and history, pins a route plan per run so that every attempt routes the same way, invokes the adapter with streaming, structured outputs and tool calls, and writes a receipt that names the deployment that answered and how certain the usage is.

It absorbs the June-2026 row ModelPolicyService: profiles and their policy are Published assets (BC-04) projected here, not a separate service.

Context and boundaries

flowchart LR
    S["ModelRouterService"]
    AgentRuntimeService["AgentRuntimeService"] -->|grpc| S
    KnowledgeIngestionService["KnowledgeIngestionService"] -->|grpc| S
    KnowledgeSearchService["KnowledgeSearchService"] -->|grpc| S
    StudioBff["StudioBff"] -->|sync-http| S
    S -->|grpc| PolicyEngineService["PolicyEngineService"]
    S -->|grpc| CostUsageService["CostUsageService"]
    S -->|grpc| SecretBrokerService["SecretBrokerService"]
    S -->|event| AgentRegistryService["AgentRegistryService"]
    S -->|grpc| ComputePoolService["ComputePoolService"]
    S -->|event| QualityScoreService["QualityScoreService"]
    S -->|grpc| ArtifactService["ArtifactService"]
    S -->|sync-http| ext_model_providers["ext-model-providers"]
    S -. "ModelProfileActivated" .-> E_ModelProfileActivated(("event"))
    S -. "ModelProfileRetired" .-> E_ModelProfileRetired(("event"))
    S -. "DeploymentStateChanged" .-> E_DeploymentStateChanged(("event"))
    S -. "CapabilityEvidenceRecorded" .-> E_CapabilityEvidenceRecorded(("event"))
    S -. "RoutePlanResolved" .-> E_RoutePlanResolved(("event"))
    S -. "ModelInvoked" .-> E_ModelInvoked(("event"))
    S -. "ModelFallbackUsed" .-> E_ModelFallbackUsed(("event"))

Hold "Alt" / "Option" to enable pan & zoom

Rules that hold at every invocation:

• Fallback is explicit per profile and disabled unless every fallback satisfies the same privacy, tool and output guarantees and passed its tests; a confidential workload never falls back to an external provider.
• The decision and the actual deployment are on the receipt; a timeout or cancellation does not prove that no billable work happened, so usage is Unknown until reconciled.
• A partially emitted stream is never spliced with another model's output: the receipt says Partial and the runtime decides.
• Prompt and response content never enters this service's store or logs (D31); outputs are artifacts by hash.

Domain model

ModelProfile — Runtime read model of a Published model-profile asset (BC-04, D32) — the logical capability, privacy, limit and fallback requirement that agents and workflows reference instead of model names. (catalog entry)

Field	Type	Required	Description
profileId	string	yes	Logical name such as code-agent, reasoning-high, embedding, pii-local.
version	string	yes
releaseId	string	yes	Published asset release of the profile.
tenantId	string	yes	Platform tenant for ConnectSoft profiles; a tenant may publish its own.
modality	enum:Modality	yes
requiredCapabilities	json	yes	Capability thresholds (tool calling, structured output, JSON schema, reliable context, patch success, multi-turn…) from the quality gates.
privacyClass	enum:PrivacyClass	yes
latencyClass	enum:LatencyClass	yes
maxContextTokens	int	yes
maxOutputTokens	int	yes
allowedRegions	list:string	yes
allowedDeployments	list:string	yes	Deployment ids the router may choose, in preference order.
fallbackChain	list:string	yes	Explicit profile ids to fall back to; each must satisfy the same privacy and output guarantees.
fallbackEnabled	bool	yes	Runtime fallback is off unless an equivalent-constraint alternative exists and passed its tests.
embeddingCompatibility	string	no	Embedding space identifier for embedding profiles.
priceBasis	string	yes	Pricing revision reference used for estimates.

Invariants:

• A profile is usable only when its release is Published and Platform-admin approved (D32); runs pin the profile version at acceptance.
• Every entry of fallbackChain has the same or a stricter privacy class and the same tool and structured-output guarantees.
• Clients reference profiles, never provider model names; a provider model appears only on receipts.

Events: ModelProfileActivated, ModelProfileRetired · Tables: ModelProfiles · Repository: ModelProfileRepository

ModelDeployment — One concrete provider, model revision and endpoint in a region with its capability evidence, license metadata, health and ModelOps state. (catalog entry)

Field	Type	Required	Description
deploymentId	string	yes
tenantId	string	yes
provider	string	yes	openai
model	string	yes	Provider model identifier and revision.
region	string	yes
privacyClass	enum:PrivacyClass	yes
endpointRef	string	yes	Connection or pool reference; credentials come from the Secret Broker.
computePoolId	string	no	For self-hosted deployments the pool that serves it (BC-41).
capabilityEvidenceRef	string	no	Evaluation record reference (BC-22) with scores per capability.
licenseMetadata	json	yes	Commercial use, redistribution, fine-tuning, customer deployment.
state	enum:DeploymentState	yes
healthy	bool	yes
version	long	yes

Invariants:

• A deployment is routable only in state Approved and healthy; Degraded deployments are skipped by the router without changing the route plan's constraints.
• Approval requires capability evidence that meets the thresholds of every profile that lists the deployment (quality gates).
• License metadata is recorded before approval; a deployment whose license forbids the intended use is never Approved.

stateDiagram-v2
    Candidate --> Evaluated : CapabilityEvidenceRecorded
    Evaluated --> Approved : PlatformAdminApproval
    Approved --> Degraded : HealthProbeFailed
    Degraded --> Approved : HealthProbeRecovered
    Approved --> Retired : Retire
    Degraded --> Retired : Retire

Hold "Alt" / "Option" to enable pan & zoom

Events: DeploymentStateChanged, CapabilityEvidenceRecorded · Tables: ModelDeployments, CapabilityScores · Repository: ModelDeploymentRepository

RoutePlan — The ordered candidates resolved for one run and profile under the policy revision in force; immutable for the run so that every attempt routes the same way. (catalog entry)

Field	Type	Required	Description
routePlanId	guid	yes
runId	guid	yes
tenantId	string	yes
profileId	string	yes
profileVersion	string	yes
policyRevision	string	yes
candidates	list:string	yes	Deployment ids in order after policy filters, capability scores, cost and latency.
constraints	json	yes	Privacy class, region, tool and output guarantees that every candidate satisfies.
resolvedAt	datetime	yes

Invariants:

• A route plan is resolved once per run and profile and reused by every attempt; a changed policy affects future runs only.
• Every candidate satisfies the plan's constraints; the router never adds a candidate at invocation time.

Events: RoutePlanResolved · Tables: RoutePlans · Repository: RoutePlanRepository

InvocationReceipt — The record of one model call — what was asked by hash, which deployment actually answered, how much was used and with what certainty, which policy decision allowed it and whether a fallback was taken. (catalog entry)

Field	Type	Required	Description
invocationId	guid	yes
tenantId	string	yes
projectId	string	yes
runId	guid	no
attemptId	guid	no
profileId	string	yes
deploymentId	string	yes	The deployment that produced the output.
routePlanId	guid	no
fallbackUsed	bool	yes
requestHash	hash	yes
idempotencyKey	string	yes
policyDecisionId	guid	yes
outcome	enum:InvocationOutcome	yes
usageStatus	enum:UsageStatus	yes
inputTokens	int	no
outputTokens	int	no
estimatedCostUsd	decimal	no
pricingRevision	string	yes
latencyMs	int	no
outputArtifactRef	string	no	Artifact reference of the output (content never stored here).
startedAt	datetime	yes
endedAt	datetime	no

Invariants:

• A receipt exists for every invocation that reached an adapter, including failed, denied, cancelled and partial ones.
• The same idempotency key within scope returns the existing receipt instead of a second call.
• Usage Unknown is recorded as such and reconciled by statement; it is never written as zero.

Events: ModelInvoked, ModelFallbackUsed · Tables: InvocationReceipts · Repository: InvocationReceiptRepository

erDiagram
    ModelProfile {
        string profileId PK
        string version
        string releaseId
        string tenantId
        enum_Modality modality
        json requiredCapabilities
        enum_PrivacyClass privacyClass
        enum_LatencyClass latencyClass
        int maxContextTokens
        int maxOutputTokens
        list_string allowedRegions
        list_string allowedDeployments
        list_string fallbackChain
        bool fallbackEnabled
        string embeddingCompatibility
        string priceBasis
    }
    ModelDeployment {
        string deploymentId PK
        string tenantId
        string provider
        string model
        string region
        enum_PrivacyClass privacyClass
        string endpointRef
        string computePoolId
        string capabilityEvidenceRef
        json licenseMetadata
        enum_DeploymentState state
        bool healthy
        long version
    }
    RoutePlan {
        guid routePlanId PK
        guid runId
        string tenantId
        string profileId
        string profileVersion
        string policyRevision
        list_string candidates
        json constraints
        datetime resolvedAt
    }
    InvocationReceipt {
        guid invocationId PK
        string tenantId
        string projectId
        guid runId
        guid attemptId
        string profileId
        string deploymentId
        guid routePlanId
        bool fallbackUsed
        hash requestHash
        string idempotencyKey
        guid policyDecisionId
        enum_InvocationOutcome outcome
        enum_UsageStatus usageStatus
        int inputTokens
        int outputTokens
        decimal estimatedCostUsd
        string pricingRevision
        int latencyMs
        string outputArtifactRef
        datetime startedAt
        datetime endedAt
    }
    CapabilityScore {
        string capability

        string verdict

        decimal score

        datetime evaluatedAt

        string suiteVersion

    }
    ModelDeployment ||--o{ CapabilityScore : contains

Hold "Alt" / "Option" to enable pan & zoom

stateDiagram-v2
    [*] --> Candidate: RegisterDeployment
    Candidate --> Evaluated: capability evidence recorded
    Evaluated --> Approved: Platform admin approval (license and thresholds)
    Approved --> Degraded: health probe failed
    Degraded --> Approved: health probe recovered
    Approved --> Retired: Retire
    Degraded --> Retired: Retire
    Retired --> [*]

Hold "Alt" / "Option" to enable pan & zoom

This is the ModelOps lifecycle of the capability registry: discover → license review → security review → benchmark → evaluate → deploy → warm → monitor → promote → retire, with the evaluation suite of BC-22 as the only source of capability scores.

Entities and value objects

CapabilityScore (entity, in ModelDeployment) — One evaluated capability of a deployment.

Field	Type	Required	Description
capability	string	yes	tool-calling, parallel-tool-calling, strict-json, json-schema, instruction-following, agent-task-success, patch-success, reliable-context, file-editing, shell-tool-usage, multi-turn, planning, error-recovery
verdict	string	yes	pass
score	decimal	no
evaluatedAt	datetime	yes
suiteVersion	string	yes

Data model and storage

Store	Role	Engine / profile	Tables
Model access store	primary	azure-sql / nhibernate-sqlserver	ModelProfiles, ModelDeployments, CapabilityScores, RoutePlans, InvocationReceipts
Cache and coordination	cache	redis / azure-cache	—

Table	Owner	Columns	Indexes
dbo.ModelProfiles	ModelProfile	17	0
dbo.ModelDeployments	ModelDeployment	13	1
dbo.CapabilityScores	CapabilityScore	7	0
dbo.RoutePlans	RoutePlan	9	1
dbo.InvocationReceipts	InvocationReceipt	22	3

Full DDL: appendix.

APIs

Method	Path	Operation	Scope	Side effects	Idempotency	Gateway	Summary
GET	/factory/model-profiles	ListModelProfiles	factory.models.read	read	natural	GW-03	Active profiles with capabilities, privacy class, limits and fallback chains.
GET	/factory/model-profiles/{profileId}	GetModelProfile	factory.models.read	read	natural	GW-03	A profile with its allowed deployments and their evidence summary.
GET	/factory/model-deployments	ListModelDeployments	factory.models.admin	read	natural	GW-03	Deployments with provider, region, privacy class, state, health and license metadata (Platform admin and Observer).
POST	/factory/model-deployments	RegisterDeployment	factory.models.admin	shared-write	key-header	GW-03	Register a Candidate deployment (Platform admin); credentials are brokered, never posted.
POST	/factory/model-deployments/{deploymentId}/evaluate	EvaluateDeployment	factory.models.admin	sandbox	key-header	GW-03	Run the capability suite against a deployment (sandbox action; produces evidence through BC-22).
POST	/factory/model-deployments/{deploymentId}/approve	ApproveDeployment	factory.models.admin	publication	natural	GW-03	Move an Evaluated deployment to Approved after license and evidence checks (publication-class action).
POST	/factory/model-profiles/{profileId}/route-plan/preview	PreviewRoutePlan	factory.models.read	read	natural	GW-03	Dry-run the router for a scope and profile: candidates, filters applied, reasons for exclusion.
GET	/factory/runs/{runId}/model-receipts	ListRunReceipts	factory.runs.read	read	natural	GW-03	Invocation receipts of a run (hashes, deployments, usage status, outcome).

gRPC service	Method	Streaming	Visibility	Side effects	Summary
ModelGateway	ResolveRoutePlan	none	internal	shared-write	Resolve and persist the route plan for a run and profile under the policy revision in force; idempotent per run and profile.
ModelGateway	Invoke	server	internal	spend	Chat completion with tools and structured output through the route plan; streams tokens and ends with the receipt (model-invocation-v1).
ModelGateway	Embed	none	internal	spend	Embeddings through an embedding profile with a receipt.
ModelGateway	Rerank	none	internal	spend	Reranking through a reranker profile with a receipt.

Example route-plan preview response:

{
  "profileId": "code-agent",
  "profileVersion": "3",
  "constraints": { "privacyClass": "Regional", "region": "westeurope", "tools": "strict", "structuredOutput": true },
  "candidates": [
    { "deploymentId": "anthropic/westeurope/claude-coding-approved", "score": 0.91, "kept": true },
    { "deploymentId": "azure-openai/westeurope/coding-approved", "score": 0.87, "kept": true }
  ],
  "excluded": [
    { "deploymentId": "ollama/tenant-pool-a/qwen-coder", "reason": "pool draining" },
    { "deploymentId": "openai/global/coding", "reason": "privacy class External not allowed by tenant policy" }
  ]
}

Messages

Event	Topic	Scope / v	Aggregate	Consumers	Summary
ModelProfileActivated	model-access.profiles.v1.model-profile-activated	integration / 1	ModelProfile	StudioBff, DashboardService, AuditService	A Published profile version is active in the gateway.
ModelProfileRetired	model-access.profiles.v1.model-profile-retired	integration / 1	ModelProfile	StudioBff, AuditService	A profile version is no longer selectable; pinned runs finish on their route plans.
DeploymentStateChanged	model-access.deployments.v1.deployment-state-changed	integration / 1	ModelDeployment	StudioBff, DashboardService, AuditService	Candidate, Evaluated, Approved, Degraded or Retired.
CapabilityEvidenceRecorded	model-access.deployments.v1.capability-evidence-recorded	integration / 1	ModelDeployment	StudioBff, DashboardService	Capability scores of a deployment were recorded from an evaluation record.
RoutePlanResolved	model-access.routing.v1.route-plan-resolved	integration / 1	RoutePlan	DashboardService	A run's route plan for a profile was resolved.
ModelInvoked	model-access.invocations.v1.model-invoked	integration / 1	InvocationReceipt	CostUsageService, DashboardService, QualityScoreService, AgentTelemetryService	A receipt was written (completed, partial, failed, denied or cancelled).
ModelFallbackUsed	model-access.invocations.v1.model-fallback-used	integration / 1	InvocationReceipt	DashboardService, AuditService	The first candidate failed and an equivalent-constraint fallback answered.

ModelProfileActivated payload:

Field	Type	Required	Description
profileId	string	yes
version	string	yes
tenantId	string	yes
releaseId	string	yes
privacyClass	enum:PrivacyClass	yes

ModelProfileRetired payload:

Field	Type	Required	Description
profileId	string	yes
version	string	yes
tenantId	string	yes

DeploymentStateChanged payload:

Field	Type	Required	Description
deploymentId	string	yes
tenantId	string	yes
fromState	enum:DeploymentState	yes
toState	enum:DeploymentState	yes
healthy	bool	yes

CapabilityEvidenceRecorded payload:

Field	Type	Required	Description
deploymentId	string	yes
tenantId	string	yes
suiteVersion	string	yes
evidenceRef	string	yes

RoutePlanResolved payload:

Field	Type	Required	Description
routePlanId	guid	yes
runId	guid	yes
tenantId	string	yes
profileId	string	yes
candidates	list:string	yes

ModelInvoked payload:

Field	Type	Required	Description
invocationId	guid	yes
tenantId	string	yes
projectId	string	yes
runId	guid	no
attemptId	guid	no
profileId	string	yes
deploymentId	string	yes
outcome	enum:InvocationOutcome	yes
usageStatus	enum:UsageStatus	yes
inputTokens	int	no
outputTokens	int	no
estimatedCostUsd	decimal	no

ModelFallbackUsed payload:

Field	Type	Required	Description
invocationId	guid	yes
tenantId	string	yes
profileId	string	yes
fromDeploymentId	string	yes
toDeploymentId	string	yes
reason	string	yes

Command	Handled by	Result events	Summary
ResolveRoutePlan	RoutePlan	RoutePlanResolved
Invoke	InvocationReceipt	ModelInvoked, ModelFallbackUsed
RegisterDeployment	ModelDeployment	DeploymentStateChanged
ApproveDeployment	ModelDeployment	DeploymentStateChanged

Event	Producer	Handler	Purpose
AssetReleasePublished	AgentRegistryService	ActivateProfile	Project Published model-profile assets into active profiles.
AssetReleaseDeprecated	AgentRegistryService	RetireProfile	Retire profile versions.
AssetReleaseRevoked	AgentRegistryService	RetireProfile	Retire revoked profile versions immediately.

Workers and jobs

Worker	Trigger	Output events	Retry	Idempotency key	Summary
DeploymentHealthProbe	schedule */2 * * * *	DeploymentStateChanged	fixed	deploymentId + probeWindow	Probes every Approved deployment with a minimal request and marks it Degraded or Approved; records latency.
ReceiptReconciler	schedule 30 * * * *	ModelInvoked	exponential	invocationId + statementRef	Confirms receipts with usage Unknown or Estimated from provider statements and forwards the correction to Usage Accounting.
CapabilityEvidenceImporter	event	CapabilityEvidenceRecorded, DeploymentStateChanged	exponential	deploymentId + suiteVersion + evidenceRef	Imports evaluation records of the capability suite into deployment scores and moves Candidate deployments to Evaluated.

Actors and grains

None modelled

No grains are modelled for this item yet.

Sagas and long-running flows

None modelled

No sagas are modelled for this item yet.

sequenceDiagram
    participant A as AgentRuntimeService
    participant R as ModelRouterService
    participant P as PolicyEngineService
    participant S as SecretBrokerService
    participant D as Provider adapter
    participant U as CostUsageService
    A->>R: ResolveRoutePlan (run, profile) once per run
    R->>P: decision for profile, data class, region, tools
    R->>R: filter, score, persist RoutePlan
    A->>R: Invoke (request hash, idempotency key, budget)
    R->>S: short-lived credential for the first candidate
    R->>D: call with streaming, structured output, tools
    D-->>R: tokens, final usage or none
    R->>R: receipt with actual deployment, usage status
    R->>U: RecordUsage from the receipt
    R-->>A: stream and receipt

Hold "Alt" / "Option" to enable pan & zoom

Options and configuration

Section	Key	Type	Default	Reloadable	Secret	Summary
ModelAccess:Routing	FallbackEnabledByDefault	bool	false	yes	no	Runtime fallback is disabled unless a profile declares an equivalent-constraint chain that passed its tests.
ModelAccess:Routing	Weights	json	{ capability: 0.5, cost: 0.2, latency: 0.2, history: 0.1 }	yes	no	Scoring weights of the router after policy filters.
ModelAccess:Invocation	DefaultTimeoutSeconds	int	120	yes	no	Per-invocation timeout unless the attempt budget is lower.
ModelAccess:Invocation	PromptCaching	bool	true	yes	no	Use provider prompt caching where the adapter supports it; cache keys are hashes, never content.
ModelAccess:Privacy	LogContent	bool	false	no	no	Never true in production (D31); outputs go to the artifact store under tenant retention.

Metrics and observability

Metric	Instrument	Unit	Dimensions	Summary
connectsoft.factory.model_gateway.invocations	counter	{invocation}	tenant, profile, provider, outcome	Invocations by outcome.
connectsoft.factory.model_gateway.latency	histogram	s	profile, provider	Time to first token and completion.
connectsoft.factory.model_gateway.tokens	counter	{token}	tenant, profile, provider, direction	Input and output tokens.
connectsoft.factory.model_gateway.fallbacks	counter	{invocation}	profile, reason	Fallbacks taken.
connectsoft.factory.model_gateway.denials	counter	{invocation}	tenant, reason	Invocations denied by policy.
connectsoft.factory.model_gateway.usage_unknown	updown	{receipt}	provider	Receipts awaiting usage reconciliation.
connectsoft.factory.model_gateway.provider_errors	counter	{error}	provider, class	Adapter errors by class.

Spans carry the OpenTelemetry GenAI attributes with the Factory's model.profile, model.deployment, receipt.id and policy.decision.id; content is never on spans.

Security

Authentication	client-credentials
Scopes	factory.models.read (Read profiles and preview routes.); factory.models.admin (Register, evaluate, approve and retire deployments (Platform admin).); factory.runs.read (Read receipts of a run.)
Roles	Requester, Operator, PlatformAdmin, Observer
Tenant isolation	row-filter
Data classification	confidential
Max side-effect class	spend
Secrets	provider-credentials (key-vault, rotate 90 d)

Dependencies

Outbound

Target	Kind	Purpose	Failure mode	Required
PolicyEngineService	grpc	Decision per invocation (profile, data class, region, tools).	fail-fast	yes
CostUsageService	grpc	RecordUsage per receipt and reservation checks against the run budget.	fail-fast	yes
SecretBrokerService	grpc	Short-lived provider credentials per deployment.	fail-fast	yes
AgentRegistryService	event	Profile assets.	queue	yes
ComputePoolService	grpc	Placement and warm state for self-hosted deployments.	degrade	no
QualityScoreService	event	Capability evidence from the evaluation suite.	queue	yes
ArtifactService	grpc	Store outputs by hash under tenant retention.	degrade	yes
ext-model-providers	sync-http	Provider endpoints.	degrade	yes

Inbound

Target	Kind	Purpose	Failure mode	Required
AgentRuntimeService	grpc	Invoke and ResolveRoutePlan.	—	yes
KnowledgeIngestionService	grpc	Embed.	—	yes
KnowledgeSearchService	grpc	Rerank and Embed.	—	yes
StudioBff	sync-http	Profile, deployment and receipt views.	—	yes

Failure and recovery

Scenario	Detection	Recovery	Data impact	RTO (min)
Provider reports no usage	adapter response without usage	receipt written with usageStatus Unknown; reconciler confirms from statements	cost uncertain until reconciled	1440
All candidates degraded	router finds no healthy candidate	invocation fails with class transient and reason; the attempt retries within budget; operators see the degraded deployments	none	30
Stream broken after partial output	adapter stream exception	receipt outcome Partial with the emitted tokens counted; the runtime decides, never splicing another model's output	partial output flagged	0
Evaluation suite unavailable	no evidence for a Candidate	the deployment stays Candidate and is never routed	none	0

NFR targets

Target	Class value (pilot-interactive)	Override
latency_p95_ms	2000	—
availability_pct	99.5	—
throughput_rps	50	—
rpo_minutes	15	—
rto_minutes	240	—
cost_class	low	—

Deployment

Granularity	module of host · host ModelGatewayHost
Topologies	saas-default, hybrid, on-prem, hybrid-compute
Runtime	—; replicas —–—
Ports	—

Current implementation

Keyed agent registration in ConnectSoft.Extensions.AI.AgentsFramework falls back to other registered clients when a key is missing and does nothing on provider failure; no profile, deployment registry, route plan, receipt or capability evaluation exists. The Anthropic adapter is design-only (D16). See providers and profiles.

Later capabilities

Historical-success weighting from evaluation and run outcomes; automatic promotion proposals when a candidate scores better at lower cost; model failover orchestration per policy; prompt-cache aware routing; regional capacity reservations.

Decisions and traceability

Family	Ids
FR	FR-04, FR-09
FUX	FUX-08, FUX-10
NFR	NFR-03, NFR-07, NFR-11, NFR-12, NFR-13
XL	XL-04, XL-08, XL-12, XL-16

Related

• Model gateway · Capability registry and router · Providers and profiles · Model profiles catalog

• ComputePoolService · Quality gates · FinOps

• Model & Compute Platform microservices · bounded contexts · APIs · events · data model

• Services catalog · Ownership map