Skip to content

Capability registry and router

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.

Capability registry

A model enters a profile only after evaluation against the capabilities agentic work depends on: reliable tool and function calling, parallel tool calls, strict JSON and JSON-schema compliance, instruction following, agent task success, patch success, declared vs reliable context, file editing, shell tool usage, multi-turn consistency, planning, error recovery, plus latency class, cost basis, privacy (private vs external) and provider type (self-hosted vs commercial). Thresholds are chosen empirically (evaluation); the Oct-2026 target's example figures are illustrations, not targets. License metadata (commercial use, redistribution, fine-tuning, customer deployment) lives in the registry.

Router

The router selects a provider/model/deployment for a task from: required capabilities, privacy and tenant policy, latency, context requirement, cost, provider availability, region, whether the model is already loaded, GPU capacity, historical success and evaluation score.

flowchart LR
    T["Task requires<br/>coding=strong · toolCalling=strict<br/>structuredOutput · context ≥ 64K · privacy=tenant-private"] --> R["Router"]
    R --> L["Local model (private)"]
    R --> C1["Cloud model A"]
    R --> C2["Cloud model B"]
    L -. "failure" .-> F1["Fallback 1 (same constraints)"]
    F1 -. "failure" .-> F2["Fallback 2"]
Hold "Alt" / "Option" to enable pan & zoom

Rules:

  • Fallback chains are explicit per profile; every fallback satisfies the same privacy, tool and output constraints and the same budget. A confidential workload never falls back to an external provider if tenant policy prohibits it.
  • Runtime fallback is initially disabled unless an alternative has explicitly equivalent constraints and tests; failing fast on a missing configuration beats silently switching providers.
  • The decision and the actual model used are recorded on the invocation receipt.

ModelOps

discover → license review → security review → benchmark → evaluate → quantize if necessary → deploy → warm → monitor → promote → replace/retire. A candidate that scores better at lower cost is promoted to staging through the regression gate, never directly.

Current implementation

  • Keyed agent registration falls back to other registered clients when a key is missing (openAI, azureOpenAI); it never tries ollama or azureAIInference and does nothing on provider failure.
  • No registry, router, capability evaluation, license metadata or fallback policy exists.