VectorMemoryService¶

Target

This page describes intended behaviour or structure. No implementation is claimed unless a section is explicitly marked Current implementation or Evidence.


Bounded context	BC-11 Retrieval & Indexes
Platform / plane	Knowledge Platform / Knowledge & Memory
Deployment granularity	module of host · host `KnowledgeHost`
Status	target · `ConnectSoft.Factory.Knowledge.Retrieval`
Built from	connectsoft-microservice · libraries: connectsoft-extensions-ai, connectsoft-extensions-saas-abstractions, connectsoft-extensions-messaging-model, connectsoft-extensions-persistence-model-nhibernate, connectsoft-extensions-query-model
Decisions	ADR-0005, ADR-0013
Traceability	`FR-07` `FUX-09` `NFR-03` `NFR-07` `NFR-16` `XL-07`

Purpose and responsibilities¶

Vector search alone is insufficient for enterprise intelligence (original handoff §21). VectorMemoryService keeps its June-2026 name but is the retrieval and indexes module: it builds index revisions from published chunks (vector through the embedding profile, full-text, symbol, metadata, graph projections), combines them with rank fusion and reranking under the caller's trusted scope, and returns cited candidates. It also holds the graph facts: temporal, provenance-carrying relationships in separate domains (code, architecture, infrastructure, organization, customer, business, agent experience) behind one federated query, with the trust tiers source fact, inferred fact, agent observation and unverified memory.

It absorbs KnowledgeSearchService, EmbeddingService and MetadataIndexService. The knowledge graph as a separate service (KnowledgeGraphService) stays a later row: the first design projects facts into this module.

Context and boundaries¶

flowchart LR
    S["VectorMemoryService"]
    ContextBuilderService["ContextBuilderService"] -->|grpc| S
    ToolAdapterService["ToolAdapterService"] -->|grpc| S
    StudioBff["StudioBff"] -->|sync-http| S
    S -->|grpc| ModelRouterService["ModelRouterService"]
    S -->|grpc| KnowledgeIngestionService["KnowledgeIngestionService"]
    S -->|grpc| ArtifactService["ArtifactService"]
    S -. "IndexRevisionReady" .-> E_IndexRevisionReady(("event"))
    S -. "IndexRevisionRetired" .-> E_IndexRevisionRetired(("event"))
    S -. "IndexDegraded" .-> E_IndexDegraded(("event"))
    S -. "GraphFactsUpdated" .-> E_GraphFactsUpdated(("event"))
    S -. "RetrievalExecuted" .-> E_RetrievalExecuted(("event"))

Hold "Alt" / "Option" to enable pan & zoom

The rule that defines this service: scope is enforced by the backend filter before top-k (ADR-0013); a backend that cannot filter by tenant, project and ACL before ranking is not enabled. The agent host sample's post-filtering is exactly what this design forbids.

Domain model¶

IndexRevision — One revision of one index of a collection, pinned to an embedding profile version and a vector profile; derived, rebuildable state. (catalog entry)

Field	Type	Required	Description
`indexRevisionId`	`guid`	yes
`tenantId`	`string`	yes
`collectionId`	`guid`	yes
`kind`	`enum:IndexKind`	yes
`revision`	`int`	yes
`embeddingProfileVersion`	`string`	no	For vector indexes; a profile change means a new revision.
`vectorProfile`	`string`	no	qdrant
`state`	`enum:IndexState`	yes
`itemCount`	`long`	yes
`builtFromSnapshots`	`list:guid`	yes
`builtAt`	`datetime`	no

Invariants:

Scope (tenant, project, ACL) is enforced by the backend filter before top-k, never by post-filtering results (ADR-0013).
A revision is immutable once Ready; rebuilding produces the next revision and retires the old one after the switch.
Every retrieved item cites source id, source version, chunk id, embedding profile version and index revision.

stateDiagram-v2
    Building --> Ready : BuildCompleted
    Ready --> Rebuilding : RebuildRequested
    Rebuilding --> Ready : BuildCompleted
    Ready --> Degraded : BackendUnhealthy
    Degraded --> Ready : BackendRecovered
    Ready --> Retired : NextRevisionReady

Hold "Alt" / "Option" to enable pan & zoom

Events: IndexRevisionReady, IndexRevisionRetired, IndexDegraded · Tables: IndexRevisions, IndexItems · Repository: IndexRevisionRepository

GraphFact — A temporal, provenance-carrying relationship in one graph domain (subject, predicate, object, validFrom, validTo, observedAt, sourceVersion, confidence, trust tier). (catalog entry)

Field	Type	Required	Description
`factId`	`guid`	yes
`tenantId`	`string`	yes
`domain`	`enum:GraphDomain`	yes
`subject`	`string`	yes
`predicate`	`string`	yes
`object`	`string`	yes
`validFrom`	`datetime`	no
`validTo`	`datetime`	no
`observedAt`	`datetime`	yes
`sourceId`	`guid`	no
`sourceVersion`	`string`	no
`confidence`	`decimal`	yes
`trustTier`	`string`	yes	SourceFact
`verified`	`bool`	yes

Invariants:

Historical truth is never replaced by current truth; a changed fact closes validTo and a new fact opens.
Inferred facts and agent observations never outrank source facts in retrieval ranking and are labelled in citations.
A fact without provenance is not stored.

Events: GraphFactsUpdated · Tables: GraphFacts · Repository: GraphFactRepository

RetrievalQuery — The record of one hybrid retrieval — scope, query hash, filters, indexes used, ranking signals and the cited results; kept for context snapshots and evaluation. (catalog entry)

Field	Type	Required	Description
`queryId`	`guid`	yes
`tenantId`	`string`	yes
`projectId`	`string`	yes
`runId`	`guid`	no
`queryHash`	`hash`	yes
`filters`	`json`	yes	metadata, temporal, domain, trust tier
`indexesUsed`	`list:guid`	yes
`resultCount`	`int`	yes
`citationsRef`	`string`	yes	Artifact reference of the cited results.
`latencyMs`	`int`	yes
`executedAt`	`datetime`	yes

Invariants:

Results outside the caller's scope are never present in citationsRef; the query records the scope it ran under.

Events: RetrievalExecuted · Tables: RetrievalQueries · Repository: RetrievalQueryRepository

erDiagram
    IndexRevision {
        guid indexRevisionId PK
        string tenantId
        guid collectionId
        enum_IndexKind kind
        int revision
        string embeddingProfileVersion
        string vectorProfile
        enum_IndexState state
        long itemCount
        list_guid builtFromSnapshots
        datetime builtAt
    }
    GraphFact {
        guid factId PK
        string tenantId
        enum_GraphDomain domain
        string subject
        string predicate
        string object
        datetime validFrom
        datetime validTo
        datetime observedAt
        guid sourceId
        string sourceVersion
        decimal confidence
        string trustTier
        bool verified
    }
    RetrievalQuery {
        guid queryId PK
        string tenantId
        string projectId
        guid runId
        hash queryHash
        json filters
        list_guid indexesUsed
        int resultCount
        string citationsRef
        int latencyMs
        datetime executedAt
    }

Hold "Alt" / "Option" to enable pan & zoom

Temporal facts never overwrite history: "Service A uses RabbitMQ, valid 2024-01 to 2026-05" and "Service A uses Azure Service Bus, valid from 2026-05" coexist (original handoff §24), and a query at a point in time returns what was true then.

Entities and value objects¶

None modelled

No entities or value objects are modelled for this item yet.

Data model and storage¶

Store	Role	Engine / profile	Tables
Knowledge metadata store	primary	azure-sql / nhibernate-sqlserver	IndexRevisions, IndexItems, GraphFacts, RetrievalQueries
Vector index	index	qdrant / vector-qdrant	—

Table	Owner	Columns	Indexes
`dbo.IndexRevisions`	IndexRevision	11	1
`dbo.IndexItems`	IndexRevision	7	1
`dbo.GraphFacts`	GraphFact	14	2
`dbo.RetrievalQueries`	RetrievalQuery	11	1

Full DDL: appendix.

APIs¶

Method	Path	Operation	Scope	Side effects	Idempotency	Gateway	Summary
`POST`	`/factory/knowledge/search`	Search	`factory.knowledge.read`	read	natural	GW-03	Hybrid retrieval (vector, full-text, symbol, metadata, graph, temporal) under the caller's scope with citations; the Studio and CLI knowledge search.
`POST`	`/factory/knowledge/graph/query`	QueryGraph	`factory.knowledge.read`	read	natural	GW-03	Federated query over graph domains with temporal filters; returns facts with provenance and trust tier.
`GET`	`/factory/knowledge/collections/{collectionId}/indexes`	ListIndexRevisions	`factory.knowledge.read`	read	natural	GW-03	Index revisions of a collection with state, counts and profile versions.
`POST`	`/factory/knowledge/collections/{collectionId}/indexes/rebuild`	RebuildIndexes	`factory.knowledge.admin`	shared-write	key-header	GW-03	Rebuild indexes from published snapshots (after an embedding profile change or a restore).

gRPC service	Method	Streaming	Visibility	Side effects	Summary
`Retrieval`	`Retrieve`	none	internal	read	Scoped hybrid retrieval for context assembly and the knowledge-search tool; returns cited candidates with ranking signals.
`Retrieval`	`ExpandGraph`	none	internal	read	Neighbours of a set of entities in one or more graph domains with temporal validity.

Example citation returned with every candidate:

{
  "chunkId": "9f1c…", "sourceId": "repo:ConnectSoft.CaseManagement", "sourceVersion": "a1b2c3d",
  "path": "src/Booking/CancellationHandler.cs", "indexRevision": 12, "embeddingProfileVersion": "embedding@3",
  "trustTier": "SourceFact", "score": 0.83, "signals": { "vector": 0.79, "fullText": 0.9, "graph": 0.6 }
}

Messages¶

Event	Topic	Scope / v	Aggregate	Consumers	Summary
IndexRevisionReady	`retrieval.indexes.v1.index-revision-ready`	integration / 1	IndexRevision	ContextBuilderService, StudioBff, DashboardService	An index revision serves queries.
IndexRevisionRetired	`retrieval.indexes.v1.index-revision-retired`	integration / 1	IndexRevision	DashboardService	A revision was replaced.
IndexDegraded	`retrieval.indexes.v1.index-degraded`	integration / 1	IndexRevision	ContextBuilderService, DashboardService	The backend of an index is unhealthy; retrieval degrades to the remaining indexes.
GraphFactsUpdated	`retrieval.graph.v1.graph-facts-updated`	integration / 1	GraphFact	ContextBuilderService, DashboardService	Facts were added or closed in a domain.
RetrievalExecuted	`retrieval.queries.v1.retrieval-executed`	integration / 1	RetrievalQuery	DashboardService, QualityScoreService	A scoped retrieval ran.

IndexRevisionReady payload:

Field	Type	Required
`indexRevisionId`	`guid`	yes
`tenantId`	`string`	yes
`collectionId`	`guid`	yes
`kind`	`enum:IndexKind`	yes
`revision`	`int`	yes

IndexRevisionRetired payload:

Field	Type	Required
`indexRevisionId`	`guid`	yes
`tenantId`	`string`	yes
`collectionId`	`guid`	yes

IndexDegraded payload:

Field	Type	Required
`indexRevisionId`	`guid`	yes
`tenantId`	`string`	yes
`kind`	`enum:IndexKind`	yes

GraphFactsUpdated payload:

Field	Type	Required
`tenantId`	`string`	yes
`domain`	`enum:GraphDomain`	yes
`added`	`int`	yes
`closed`	`int`	yes
`sourceId`	`guid`	no

RetrievalExecuted payload:

Field	Type	Required
`queryId`	`guid`	yes
`tenantId`	`string`	yes
`runId`	`guid`	no
`resultCount`	`int`	yes
`latencyMs`	`int`	yes

Command	Handled by	Result events	Summary
`RebuildIndexes`	IndexRevision	IndexRevisionReady, IndexRevisionRetired
`Retrieve`	RetrievalQuery	RetrievalExecuted

Event	Producer	Handler	Purpose
ChunksPublished	KnowledgeIngestionService	`IndexChunks`	Embed and index published chunks; extract entities and relationships into graph facts.
SourceDeleted	KnowledgeIngestionService	`RemoveDerivedItems`	Remove vectors, index items and facts of a deleted source.

Workers and jobs¶

Worker	Trigger	Output events	Retry	Idempotency key	Summary
IndexBuilder	event	IndexRevisionReady	exponential	`indexRevisionId + chunkId`	Embeds chunks through the embedding profile, writes vector and full-text items under the backend scope filter and flips the revision to Ready.
GraphExtractor	event	GraphFactsUpdated	exponential	`snapshotId + chunkId + extractorVersion`	Entity and relationship extraction from published chunks into temporal facts with provenance (structured profile for extraction).
BackendHealthProbe	schedule `/2 * * *`	IndexDegraded, IndexRevisionReady	fixed	`indexRevisionId + probeWindow`	Probes the vector and full-text backends and marks revisions Degraded or Ready.

Actors and grains¶

None modelled

No grains are modelled for this item yet.

Sagas and long-running flows¶

None modelled

No sagas are modelled for this item yet.

Options and configuration¶

Section	Key	Type	Default	Reloadable	Secret	Summary
`Knowledge:Retrieval`	`DefaultTopK`	int	`20`	yes	no	Candidates per index before fusion and reranking.
`Knowledge:Retrieval`	`Fusion`	enum	`rrf`	yes	no	Rank fusion across indexes (rrf
`Knowledge:Retrieval`	`RerankProfile`	string	`reranker`	yes	no	Reranker profile used for second-stage ranking.
`Knowledge:Graph`	`ExtractionProfile`	string	`structured`	yes	no	Model profile for entity and relationship extraction.

Metrics and observability¶

Metric	Instrument	Unit	Dimensions	Summary
`connectsoft.factory.knowledge.retrievals`	counter	{query}	tenant, indexKinds, outcome	Retrievals and outcomes.
`connectsoft.factory.knowledge.retrieval_latency`	histogram	s	tenant	End-to-end retrieval latency.
`connectsoft.factory.knowledge.index_items`	updown	{item}	tenant, kind	Items per index kind.
`connectsoft.factory.knowledge.graph_facts`	updown	{fact}	tenant, domain, trustTier	Facts by domain and trust tier.
`connectsoft.factory.knowledge.index_degraded`	updown	{index}	tenant, kind	Degraded index revisions.

Security¶


Authentication	client-credentials
Scopes	`factory.knowledge.read` (Scoped search and graph query.); `factory.knowledge.admin` (Rebuild indexes.)
Roles	Requester, Reviewer, AssetMaintainer, Operator, WorkspaceAdmin, PlatformAdmin, Observer
Tenant isolation	row-filter
Data classification	confidential
Max side-effect class	shared-write
Secrets	`vector-index` (key-vault, rotate 90 d)

Dependencies¶

Outbound

Target	Kind	Purpose	Failure mode	Required
ModelRouterService	grpc	Embed and Rerank through profiles with receipts.	degrade	yes
KnowledgeIngestionService	grpc	Source scope and ACL per caller.	fail-fast	yes
ArtifactService	grpc	Citation payloads by hash.	fail-fast	yes

Inbound

Target	Kind	Purpose	Failure mode	Required
ContextBuilderService	grpc	Retrieve and ExpandGraph.	—	yes
ToolAdapterService	grpc	knowledge-search tool.	—	yes
StudioBff	sync-http	Search and graph views.	—	yes

Failure and recovery¶

Scenario	Detection	Recovery	Data impact	RTO (min)
Vector backend down	health probe	vector revisions Degraded; retrieval serves full-text, symbol and metadata; context assembly notes reduced recall in the snapshot	none	60
Embedding profile version changed	collection pins a newer version	new revision built from snapshots; old revision serves until Ready	none	0
Scope filter unsupported by a backend	ADR-0013 contract test fails at startup	the backend is not enabled; fail fast	none	0

NFR targets¶

Target	Class value (pilot-interactive)	Override
latency_p95_ms	2000	—
availability_pct	99.5	—
throughput_rps	50	—
rpo_minutes	15	—
rto_minutes	240	—
cost_class	low	—

Deployment¶


Granularity	module of host · host `KnowledgeHost`
Topologies	saas-default, hybrid, on-prem, hybrid-compute
Runtime	—; replicas —–—
Ports	—

Current implementation¶

The agent host wires only the in-memory connector (Qdrant, PgVector, SQL Server and Azure AI Search throw), filters tenant and tags after search, passes options: null through the adapter and fixes dimensions at 1536; no graph facts exist. See vector memory and knowledge graph for the earlier design pages this service realises.

Later capabilities¶

Multi-hop graph reasoning and inferred-edge suggestions for human confirmation; ontology management as versioned configuration; embedding-model A/B evaluation before refresh; cold-vector tiering; a dedicated graph service when domains outgrow projections.

Decisions and traceability¶

Family	Ids
FR	FR-07
FUX	FUX-09
NFR	NFR-03, NFR-07, NFR-16
XL	XL-07

Vector memory · Knowledge graph · ContextBuilderService · ADR-0013
Knowledge Platform microservices · bounded contexts · APIs · events · data model
Services catalog · Ownership map

VectorMemoryService¶

Purpose and responsibilities¶

Context and boundaries¶

Domain model¶

Entities and value objects¶

Data model and storage¶

APIs¶

Messages¶

Workers and jobs¶

Actors and grains¶

Sagas and long-running flows¶

Options and configuration¶

Metrics and observability¶

Security¶

Dependencies¶

Failure and recovery¶

NFR targets¶

Deployment¶

Current implementation¶

Later capabilities¶

Decisions and traceability¶

Related¶