AIFACTORY-FEAT-056: EnvelopeRegistryService: complete delivery¶
Generated from the architecture model
This page is rendered from model/**/*.yaml by tools/docs_model during mkdocs build. Edit the YAML, not this page; hand-written design lives on the linked service, platform and plane pages.
A design artefact, not a tracker
This backlog is generated from model/backlog/ and validated against the architecture model. No work item exists in any tracker, and no item carries a date, an estimate or an assignee (ADR-0029).
Epic Data & Event Platform (PF-11) · Increment I4 Trust and accountability · State elaborated · Route backend-developer · Build order 126 of 185 (Factory code)
EnvelopeRegistryService is the service of bounded context BC-42 (Data & Event Foundations). The registry of every integration topic (context.area.vN.event-name), its producer, consumers and payload schema versions, the port-level envelope (ADR-0014) with its mapping per messaging profile, the outbox and inbox table contracts per profile, and compatibility checks that fail a release which would break a consumer (BC-42). Its own repository is ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService.
| Delivers | EnvelopeRegistryService (complete) |
| Stubs | — |
| Realizes | adrs: ADR-0012, ADR-0014 |
| Requirements | FR-10, FUX-11, NFR-05, NFR-14, XL-02 |
| Depends on | — |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml |
Scenarios to trace (11, catalog): SCN-EnvelopeRegistryService-001 ✓, SCN-EnvelopeRegistryService-002 ✓, SCN-EnvelopeRegistryService-003 ✓, SCN-EnvelopeRegistryService-004 ✓, SCN-EnvelopeRegistryService-005 ✓, SCN-EnvelopeRegistryService-006 ✓, SCN-EnvelopeRegistryService-007 ✓, SCN-EnvelopeRegistryService-008 ✓, SCN-EnvelopeRegistryService-009 ✓, SCN-EnvelopeRegistryService-010 ✓, SCN-EnvelopeRegistryService-011 ✓
AIFACTORY-US-047: Every topic and its schema versions are registered¶
behavior · persona platform-admin
As a platform administrator, I want one registry of the integration topics, their producers, consumers and schema versions, so that a breaking change of an event is refused before it reaches a consumer.
This is the integration PBI of the feature. EnvelopeRegistryService is the service of bounded context BC-42 that makes the event architecture of ADR-0012 and ADR-0014 enforceable at runtime. It owns two aggregates in envelope-registry-sql: TopicDefinition, one integration topic context.area.vN.event-name with its one producer, its registered consumers, its payload schema versions with their compatibility and its state Draft, Active, Deprecated or Retired, and EnvelopeProfile, the carriage of the port-level envelope on one messaging profile with its outbox and inbox table contracts and deduplication window. Topics, schema versions and envelope profiles are platform-wide: the model holds them under the platform tenant, so the registry keeps no data of a customer tenant. Callers reach its five REST operations through the gateway DataEventFoundations under factory.data.read and factory.data.admin, and the hosts of every platform call its two internal gRPC methods with a workload identity. The behaviour is delivered by the behaviour PBIs of this feature: topics are read (AIFACTORY-US-1446), registered and synchronised from the model (AIFACTORY-US-1447), receive schema versions under the compatibility check (AIFACTORY-US-1448), are deprecated and retired (AIFACTORY-US-1449), the envelope profiles are listed and activated (AIFACTORY-US-1450) and the publish-time resolution and validation answer the hosts (AIFACTORY-US-1451). Every state change and its outbox record commit in one transaction and the four events leave through the outbox of the envelope-registry messaging profile under the canonical envelope; the service consumes no event. Messaging, persistence and telemetry come from the base template with the Factory profile and the pinned observability packages, never forked. The service store is restored from backup with a recovery point of 15 minutes, and because the registrations are also in the architecture model, ModelSync can apply them again. Repository: ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService, created by the repository checklist of feature repository-and-pipeline-conventions from the base template with the Factory profile.
Acceptance criteria
| AC1 | Given the repository ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService, whose first commit is the unchanged output of dotnet new connectsoft-base -n ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService --UseFactoryProfile true from the pinned template package, when its pipeline runs with the work of this feature, then no second scaffold was run, the solution builds, the unit and acceptance tests pass and a container image is published (ADR-0028). |
| AC2 | Given the acceptance suites of AIFACTORY-US-1446 to AIFACTORY-US-1451, when they run in the pipeline, then each of the eleven scenarios of EnvelopeRegistryService has a passing test that names its id, and a filter that discovers no test counts as a failure (model/scenarios/EnvelopeRegistryService.yaml). |
| AC3 | Given the outbox of the envelope-registry messaging profile after the acceptance suites, when its records are read, then EnvelopeRegistryService/TopicRegistered, EnvelopeRegistryService/SchemaVersionPublished, EnvelopeRegistryService/TopicDeprecated and EnvelopeRegistryService/EnvelopeProfileActivated were each written in the transaction of the state change they announce and published under the canonical envelope, and the inbox registers no subscription, because the model lists no consumed event (ADR-0012, ADR-0014). |
| AC4 | Given the service starts, when liveness and readiness are probed, then readiness reports envelope-registry-sql and the messaging transport, the telemetry conformance test of the pinned observability packages passes with connectsoft.factory.data_event.topics tagged with state and connectsoft.factory.data_event.envelope_validation_failures tagged with producer and topic, and no payload appears in a metric, an activity or a log record (NFR-05, model/services/data-event/EnvelopeRegistryService.yaml). |
| AC5 | Given an empty envelope-registry-sql after a restore, when ModelSync runs on the architecture model, then every topic of the generated topic catalog is registered again with its producer, its consumers and its schema references, as the failure mode of a lost service store states (ADR-0012, model/services/data-event/EnvelopeRegistryService.yaml). |
| Traces: services | EnvelopeRegistryService, DataEventFoundations |
| Traces: aggregates | EnvelopeRegistryService/TopicDefinition, EnvelopeRegistryService/EnvelopeProfile |
| Traces: apis | EnvelopeRegistryService/ListTopics, EnvelopeRegistryService/GetTopic, EnvelopeRegistryService/RegisterTopic, EnvelopeRegistryService/PublishSchemaVersion, EnvelopeRegistryService/ListEnvelopeProfiles, EnvelopeRegistryService/Envelopes.ResolveTopic, EnvelopeRegistryService/Envelopes.ValidateEnvelope |
| Traces: events | EnvelopeRegistryService/TopicRegistered, EnvelopeRegistryService/SchemaVersionPublished, EnvelopeRegistryService/TopicDeprecated, EnvelopeRegistryService/EnvelopeProfileActivated |
| Traces: adrs | ADR-0012, ADR-0014, ADR-0028 |
| Traces: requirements | FR-10, NFR-05, XL-02 |
| Relates: screens | topic-registry |
| Relates: nfrs | NFR-14 |
| Depends on | AIFACTORY-US-1446, AIFACTORY-US-1447, AIFACTORY-US-1448, AIFACTORY-US-1449, AIFACTORY-US-1450, AIFACTORY-US-1451, feature:repository-and-pipeline-conventions, feature:component-base-template-factory-profile, feature:contracts-package, feature:component-observability-factory-conventions |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml, model/scenarios/EnvelopeRegistryService.yaml, docs/planes/data-event/services/envelope-registry-service/index.md#purpose-and-responsibilities, docs/architecture/adr/0014-run-and-lifecycle-envelopes.md |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-374 Create the repository ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService and scaffold it from the ConnectSoft.BaseTemplate template with the Factory profile |
implementation | The repository exists by the repository checklist of feature repository-and-pipeline-conventions, the output of dotnet new connectsoft-base -n ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService --UseFactoryProfile true from the pinned template package is committed unchanged with the messaging service name envelope-registry, the repository records the template version, the solution builds and the template tests pass in the repository pipeline, which publishes the container image. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-375 Domain model: TopicDefinition, EnvelopeProfile |
implementation | The aggregates hold the fields and the invariants of the model (exactly one producer per topic whose bounded-context slug is the context segment, a new schema version Active only when backward compatible or acknowledged by every registered consumer, Retired topics reject publication, identical envelope fields on every profile) and the three transitions of TopicState (Draft to Active, Active to Deprecated, Deprecated to Retired); unit tests cover every invariant and every transition and refuse every transition the model does not list. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-376 Persistence in envelope-registry-sql: Topics, SchemaVersions, TopicConsumers, EnvelopeProfiles |
implementation | Mappings and migrations for the four modelled tables with the tenant column, the index on producer and the unique index on event name; mapping tests run against an isolated test database, never a shared one. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-377 API hosting skeleton: the 5 REST and 2 gRPC operations with scope enforcement and problem responses |
implementation | ListTopics, GetTopic, RegisterTopic, PublishSchemaVersion and ListEnvelopeProfiles are hosted with JWT validation of the token the gateway forwards, scope enforcement per route, the Idempotency-Key filter of PublishSchemaVersion and problem responses for every modelled refusal code; Envelopes.ResolveTopic and Envelopes.ValidateEnvelope are hosted with workload identity checks; the OpenAPI and proto documents are published; the behaviour of each operation is delivered by AIFACTORY-US-1446 to AIFACTORY-US-1451, not here. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-378 Messaging skeleton: outbox of the envelope-registry profile for 4 published events, no consumed event |
implementation | The outbox writer enlists in the transaction of each state change and the dispatcher publishes TopicRegistered, SchemaVersionPublished, TopicDeprecated and EnvelopeProfileActivated on their modelled topics under the canonical envelope; the inbox registers no subscription because the model lists no consumed event, and a test fails when a subscription appears that the model does not list. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-379 Worker hosting skeleton: triggers, retry policies and idempotency keys of ModelSync and DeprecationSweeper |
implementation | ModelSync is hosted with its manual trigger on release of the architecture model, a fixed retry of two attempts and the idempotency key topic plus schema version; DeprecationSweeper with its daily schedule, no retry and the idempotency key topic plus day; both run one at a time and a duplicate trigger has no second effect; what each worker does is delivered by AIFACTORY-US-1447 and AIFACTORY-US-1449. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-380 Security, tenancy and telemetry of EnvelopeRegistryService |
implementation | The two scopes and three roles of the model are enforced, every query runs under the tenant filter with the platform tenant the registry rows carry, the messaging transport secret comes from a secret reference, the host makes the one call of the registration of the pinned observability packages, creates connectsoft.factory.data_event.topics and connectsoft.factory.data_event.envelope_validation_failures through its instrument helper and references the conformance test package from its architecture tests. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-704 Verify topics-and-envelopes-registered by running the declared command |
verification | Run dotnet build -c Release and dotnet test -c Release --no-build in the pipeline of ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService, then dotnet test -c Release --filter "Category=Acceptance&Scenario~SCN-EnvelopeRegistryService"; a filter that discovers no test counts as a failure; they pass and prove AC1 to AC5 of AIFACTORY-US-047, with a test for each of SCN-EnvelopeRegistryService-001 to SCN-EnvelopeRegistryService-011. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-US-1446: List the registered topics and read one topic with its schema versions and compatibility history¶
behavior · persona observer
As an observer, a platform admin or the Studio topic registry view, I want to list the integration topics with their producer, consumers, schema version and state and to read one with its history, so that I see the event architecture that runs, not only the one that was designed.
A caller whose scope carries factory.data.read lists topics through EnvelopeRegistryService/ListTopics and receives every registered topic with its producer, its registered consumers, its current schema version and its state, optionally filtered by context, the producing bounded-context slug, or by producer, paged. The same caller reads one topic through EnvelopeRegistryService/GetTopic, addressed by its name context.area.vN.event-name, and receives the topic with every schema version it ever had, each with its schema reference, its compatibility against the previous version (Backward, Forward, Full or Breaking), the moment it was published and, where it applies, the moment it was deprecated, together with the version each consumer has acknowledged. StudioBff serves both through the view topic-registry, and the DSL studio reads the topic list. Topics are platform-wide rows of the registry, held under the platform tenant as the table Topics states, so both reads return the same registry to every caller with the scope and there is no topic of another tenant to hide. A caller without the scope is refused with 403 and a topic the registry does not hold is answered 404 with a problem response. Nothing is written by either read. Through the gateway the route is a read route (AIFACTORY-US-1442 of feature data-event-foundations).
Acceptance criteria
| AC1 | Given registered topics of several producing contexts and a caller with factory.data.read, when EnvelopeRegistryService/ListTopics is called without a filter and then filtered by context and by producer, then each response is 200 with the matching topics, each with producer, consumers, current schema version and state, and nothing is written (SCN-EnvelopeRegistryService-001). |
| AC2 | Given a topic with three schema versions, one of them Breaking and acknowledged by every consumer, when EnvelopeRegistryService/GetTopic is called with its name, then the response is 200 with the three versions in order, each with schema reference, compatibility and publication moment, and the acknowledged version of each consumer (SCN-EnvelopeRegistryService-002). |
| AC3 | Given a caller without factory.data.read, or a topic name the registry does not hold, when EnvelopeRegistryService/ListTopics or EnvelopeRegistryService/GetTopic is called, then the response is 403 or 404 respectively with a problem response and no state change (SCN-EnvelopeRegistryService-001, SCN-EnvelopeRegistryService-002, ADR-0015). |
| Traces: services | EnvelopeRegistryService |
| Traces: aggregates | EnvelopeRegistryService/TopicDefinition |
| Traces: apis | EnvelopeRegistryService/ListTopics, EnvelopeRegistryService/GetTopic |
| Traces: adrs | ADR-0012, ADR-0015 |
| Traces: requirements | FR-10, FUX-11 |
| Traces: scenarios | SCN-EnvelopeRegistryService-001, SCN-EnvelopeRegistryService-002 |
| Relates: screens | topic-registry, dsl-studio |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml, docs/planes/data-event/services/envelope-registry-service/index.md#apis |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-5862 ListTopics and GetTopic read model with filters, paging and schema history |
implementation | ListTopics returns the topics of the registry with producer, consumers, current schema version and state, filtered by context or producer and paged; GetTopic returns one topic with every schema version, its schema reference, compatibility and publication and deprecation moments and the acknowledged version per consumer; a missing scope is 403 and an unknown topic 404 with a problem response; neither read writes anything. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5863 Verify the topic reads through the acceptance suite |
verification | Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-EnvelopeRegistryService-001\|Scenario=SCN-EnvelopeRegistryService-002)"; a zero-match filter counts as a failure; the tests prove AC1 to AC3 of AIFACTORY-US-1446 with topics of several contexts, a topic with a Breaking version, a caller without the scope and an unknown topic. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-US-1447: Register or update a topic from the architecture model and announce it with TopicRegistered¶
behavior · persona platform-admin
As a platform admin or the release pipeline, I want every topic of the architecture model registered with its producer and consumers, so that the registry at runtime holds the same facts as the model and drift is reported.
A Platform admin, or the release pipeline that normally applies the architecture model, calls EnvelopeRegistryService/RegisterTopic with factory.data.admin on PUT /factory/data/topics/{topic}. A user caller needs factory.data.admin and the Platform admin role; the release pipeline calls with a workload token, which carries service scopes and no user role (ADR-0015), and is admitted when its token holds the service scope factory.data.admin that names the operation. The body names the producing service, the registered consumers and the envelope profile of the topic; the context, area, version and event name are taken from the topic name context.area.vN.event-name of the path. A topic the registry does not hold is created in state Draft and the answer is 201; a topic it holds is updated in its producer or consumers and the answer is 200; a repeated PUT with the same content changes nothing and publishes nothing, which is the natural idempotency of the operation. The service refuses with 409 a topic whose context segment is not the bounded-context slug of the producing service, a second producer for a topic, an event name that another topic already carries (the unique index of Topics) and a change to a Retired topic; a user caller without the scope or without the Platform admin role, and a workload token without the scope, are refused with 403. A registration or a change of producer or consumers writes EnvelopeRegistryService/TopicRegistered with topic, producer, currentSchemaVersion and state in the same transaction; StudioBff refreshes the topic view and AuditService records the evidence in their own suites. The worker ModelSync applies the generated topic catalog of the architecture model at release: it registers or updates every topic with its producer, consumers and schema references through the same commands, so each change is announced by TopicRegistered and each new schema version goes through the compatibility check of AIFACTORY-US-1448, and it reports as drift every topic the registry holds that the catalog does not list and every difference it could not apply. It runs one at a time, retries twice with a fixed backoff and is idempotent on topic plus schema version. connectsoft.factory.data_event.topics moves with every change of state.
Acceptance criteria
| AC1 | Given a user caller with factory.data.admin and the Platform admin role, or the release pipeline with a workload token that holds the service scope factory.data.admin, when EnvelopeRegistryService/RegisterTopic is called for a new topic, then the response is 201, the EnvelopeRegistryService/TopicDefinition is in state Draft with its producer and consumers, and EnvelopeRegistryService/TopicRegistered is written in the same transaction; when it is called again with changed consumers, the response is 200 and one more event is written; when it is repeated unchanged, nothing is written (SCN-EnvelopeRegistryService-003, ADR-0012, ADR-0015). |
| AC2 | Given a topic whose context segment is not the slug of the producer, a second producer, an event name carried by another topic, or a Retired topic, when EnvelopeRegistryService/RegisterTopic is called, then the response is 409 with a problem response that names the rule; given a user caller without the scope or the Platform admin role, or a workload token without the service scope, then the response is 403, while the workload token of the release pipeline with the scope is not refused for lacking a user role; no refusal writes a row or an event (SCN-EnvelopeRegistryService-003, ADR-0015). |
| AC3 | Given a registration committed, when the outbox dispatcher of the envelope-registry messaging profile publishes EnvelopeRegistryService/TopicRegistered on data-foundations.topics.v1.topic-registered, then the event is sent under the canonical envelope with topic, producer, currentSchemaVersion and state, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-EnvelopeRegistryService-008, ADR-0014). |
| AC4 | Given a generated topic catalog of the model and a registry that holds one extra topic and misses two, when ModelSync runs, then the two topics are registered with their producers, consumers and schema references, the extra topic is reported as drift and left unchanged, and a second run of the same catalog writes nothing and publishes nothing (SCN-EnvelopeRegistryService-008, model/services/data-event/EnvelopeRegistryService.yaml). |
| Traces: services | EnvelopeRegistryService |
| Traces: aggregates | EnvelopeRegistryService/TopicDefinition |
| Traces: apis | EnvelopeRegistryService/RegisterTopic |
| Traces: events | EnvelopeRegistryService/TopicRegistered |
| Traces: adrs | ADR-0012, ADR-0014, ADR-0015 |
| Traces: requirements | FR-10, XL-02 |
| Traces: scenarios | SCN-EnvelopeRegistryService-003, SCN-EnvelopeRegistryService-008 |
| Relates: screens | topic-registry |
| Depends on | AIFACTORY-US-1446 |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml, docs/planes/data-event/services/envelope-registry-service/index.md#workers-and-jobs |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-555 Topic registry synchronised from the model by ModelSync |
implementation | ModelSync reads the generated topic catalog of the architecture model at release and applies every topic with producer, consumers and schema references through RegisterTopic and PublishSchemaVersion; topics the registry holds that the catalog does not list, and differences it could not apply, are reported as drift in the run report; a second run of the same catalog has no effect. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5864 RegisterTopic with the topic invariants and the TopicRegistered outbox record |
implementation | RegisterTopic creates a Draft topic with 201 or updates producer and consumers with 200, writes TopicRegistered in the same transaction, changes nothing on an unchanged repeat, refuses with 409 a context segment that is not the producer slug, a second producer, an event name of another topic and a Retired topic, and with 403 a user caller without factory.data.admin or the Platform admin role and a workload token without the service scope factory.data.admin; the workload token of the release pipeline with that scope is admitted without a user role, and an acceptance test covers it. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5865 Verify registration and synchronisation through the acceptance suite |
verification | Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-EnvelopeRegistryService-003\|Scenario=SCN-EnvelopeRegistryService-008)"; a zero-match filter counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-1447, including the publication and payload of TopicRegistered under a transport outage and a ModelSync run on a fixture catalog with drift; consumer handling is proven by the suites of StudioBff and AuditService. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-US-1448: Publish a payload schema version, refuse a breaking one that a consumer has not acknowledged, and announce SchemaVersionPublished¶
behavior · persona platform-admin
As a platform admin or the release pipeline, I want a new payload schema version compared with the current one and refused when it would break a consumer that has not acknowledged it, so that no release of a producer breaks a consumer at runtime.
A Platform admin or the release pipeline calls EnvelopeRegistryService/PublishSchemaVersion with factory.data.admin on POST /factory/data/topics/{topic}/schema-versions with an Idempotency-Key. The body names the version and the reference of its JSON Schema document; the service reads the document from ArtifactService and compares it with the current version of the topic, recording the result as Backward, Forward, Full or Breaking. The invariant of TopicDefinition admits a new version only when it is backward compatible with the current one or acknowledged by every registered consumer: a Backward or Full version becomes the current version at once; a Forward or Breaking version, which is not backward compatible, becomes current only when every row of TopicConsumers of the topic records that version as AcknowledgedSchemaVersion, and otherwise the publish is refused with 409 and a problem response that lists the unacknowledged consumers (the failure mode of the model). The first version of a Draft topic activates it (trigger Activate, Draft to Active). The answer is 201 with the version and its compatibility, and EnvelopeRegistryService/SchemaVersionPublished with topic, version and compatibility is written in the same transaction as the SchemaVersion row and the new current version. A repeat under the same key and body returns the same 201 and changes nothing; the same key with a different body is refused with 409; a Retired topic is refused with 409; a version that does not parse, a schema document that cannot be read as JSON Schema or a topic the registry does not hold is refused with 422; a caller without the scope or the role is refused with 403. When ArtifactService does not answer, the document cannot be compared, so no SchemaVersion row, current version or event is written, the caller receives 503 with a problem response that names ArtifactService, as D137 rules for a dependency the call cannot do without, and a repeat under the same Idempotency-Key after ArtifactService is back is applied once. The model names no operation through which a consumer records its acknowledgement of a breaking version, and this PBI does not decide one; it reads what TopicConsumers holds.
Acceptance criteria
| AC1 | Given an Active topic and a Backward compatible version, when EnvelopeRegistryService/PublishSchemaVersion is called with an Idempotency-Key, then the response is 201 with compatibility Backward, the version is the current version of the EnvelopeRegistryService/TopicDefinition and EnvelopeRegistryService/SchemaVersionPublished is written in the same transaction; given a Draft topic, its first version activates it (SCN-EnvelopeRegistryService-004). |
| AC2 | Given a Breaking version and one registered consumer that has not acknowledged it, or a Forward version that one registered consumer has not acknowledged, when it is published, then the response is 409 with a problem response that names that consumer and nothing is written; given every row of TopicConsumers acknowledges it, then it is published with compatibility Breaking or Forward respectively; given a Full version, then it becomes current at once like a Backward one (SCN-EnvelopeRegistryService-004, ADR-0012). |
| AC3 | Given the same call repeated with the same key and body, a different body under the same key, a Retired topic, an unparseable version or unreadable schema, an unknown topic, or a caller without factory.data.admin, when EnvelopeRegistryService/PublishSchemaVersion is called, then the repeat returns the first 201 and changes nothing and the others are refused with 409, 409, 422, 422 and 403 respectively, none of them writing a row or an event; given ArtifactService does not answer, then no row, current version or event is written, the caller receives 503 with a problem response that names ArtifactService, and a repeat under the same Idempotency-Key once ArtifactService answers is applied once (SCN-EnvelopeRegistryService-004, D137). |
| AC4 | Given a version published, when the outbox dispatcher publishes EnvelopeRegistryService/SchemaVersionPublished on data-foundations.topics.v1.schema-version-published, then the event is sent under the canonical envelope with topic, version and compatibility, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-EnvelopeRegistryService-009, ADR-0014). |
| Traces: services | EnvelopeRegistryService, ArtifactService |
| Traces: aggregates | EnvelopeRegistryService/TopicDefinition |
| Traces: apis | EnvelopeRegistryService/PublishSchemaVersion |
| Traces: events | EnvelopeRegistryService/SchemaVersionPublished |
| Traces: adrs | ADR-0012, ADR-0014 |
| Traces: requirements | FR-10, XL-02 |
| Traces: scenarios | SCN-EnvelopeRegistryService-004, SCN-EnvelopeRegistryService-009 |
| Relates: screens | topic-registry |
| Depends on | AIFACTORY-US-1447, feature:artifact-service |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml, docs/planes/data-event/services/envelope-registry-service/index.md#failure-and-recovery |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-556 Schema version compatibility check |
contract | A comparator reads two JSON Schema documents and returns Backward, Forward, Full or Breaking with the list of changed members; golden test pairs cover each result, including a removed required member, an added optional member, a changed type, a Forward pair (a new version that old readers can read but that cannot read data of the current version, such as a member that becomes required) and a Full pair, and the comparison result is what SchemaVersions stores. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5866 PublishSchemaVersion with the acknowledgement rule, Activate and the SchemaVersionPublished outbox record |
implementation | PublishSchemaVersion reads the schema document from ArtifactService, compares it with the current version, makes a Backward or Full version current at once and a Forward or Breaking version only when every row of TopicConsumers acknowledges it, otherwise answering 409 with the unacknowledged consumers, activates a Draft topic on its first version, writes SchemaVersionPublished in the same transaction and answers 201; the 409 with the unacknowledged consumers for a Forward and for a Breaking version, the 409 for a reused key with a different body and for a Retired topic, the 422 refusals and the 403 are covered by acceptance tests; with a fake of ArtifactService that does not answer, nothing is written, the answer is 503 with a problem response naming ArtifactService (D137) and a repeat under the same Idempotency-Key after the fake answers is applied once. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5867 Verify schema publication through the acceptance suite |
verification | Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-EnvelopeRegistryService-004\|Scenario=SCN-EnvelopeRegistryService-009)"; a zero-match filter counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-1448, including the refused breaking version, the idempotent repeat and the publication and payload of SchemaVersionPublished; consumer handling is proven by the suites of StudioBff and AuditService. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-US-1449: Deprecate a topic with its deprecation window, announce TopicDeprecated and retire it when the window elapses¶
behavior · persona platform-admin
As a platform admin, I want a deprecated topic to keep delivering for its window and then be retired, so that consumers have time to move and no host publishes on a topic that is gone.
A topic leaves service in two steps. The transition Deprecate moves an Active TopicDefinition to Deprecated, sets deprecatedAt on its current schema version and writes EnvelopeRegistryService/TopicDeprecated with topic and retiresAt, the moment of the transition plus DataEvent:Envelopes:DeprecationWindowDays (90 days by default), in the same transaction; a Deprecated topic keeps delivering for the whole window, so Envelopes.ResolveTopic still resolves it with its state and Envelopes.ValidateEnvelope still accepts envelopes on it. The worker DeprecationSweeper runs daily as the model schedules it, one at a time, without retry and idempotent on topic plus day: it moves every Deprecated topic whose retiresAt has passed to Retired through the trigger WindowElapsed, publishes no event, as the model gives the worker none, and moves connectsoft.factory.data_event.topics; a Retired topic rejects publication, which Envelopes.ValidateEnvelope and Envelopes.ResolveTopic report to the outbox writer of a host (AIFACTORY-US-1451). A second run on the same day changes nothing. Deprecate applies only to an Active topic, and no transition leaves Retired. The model names no operation that applies Deprecate to a topic; this PBI builds the transition, its event and the sweep, and which operation deprecates a topic is reported to the owner.
Acceptance criteria
| AC1 | Given an Active topic, when the transition Deprecate is applied, then the EnvelopeRegistryService/TopicDefinition is Deprecated, its current schema version carries deprecatedAt, and EnvelopeRegistryService/TopicDeprecated is written in the same transaction with retiresAt at the transition plus DataEvent:Envelopes:DeprecationWindowDays; Deprecate on a Draft or Retired topic is refused (SCN-EnvelopeRegistryService-010). |
| AC2 | Given a topic Deprecated, when the outbox dispatcher publishes EnvelopeRegistryService/TopicDeprecated on data-foundations.topics.v1.topic-deprecated, then the event is sent under the canonical envelope with topic and retiresAt, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-EnvelopeRegistryService-010, ADR-0014). |
| AC3 | Given one Deprecated topic whose window elapsed and one whose window has not, when DeprecationSweeper runs, then only the first moves to Retired, no event is published and connectsoft.factory.data_event.topics moves by state; a second run on the same day changes nothing, and EnvelopeRegistryService/Envelopes.ResolveTopic reports the Retired state (SCN-EnvelopeRegistryService-010, model/services/data-event/EnvelopeRegistryService.yaml). |
| Traces: services | EnvelopeRegistryService |
| Traces: aggregates | EnvelopeRegistryService/TopicDefinition |
| Traces: apis | EnvelopeRegistryService/Envelopes.ResolveTopic |
| Traces: events | EnvelopeRegistryService/TopicDeprecated |
| Traces: adrs | ADR-0012, ADR-0014 |
| Traces: requirements | FR-10 |
| Traces: scenarios | SCN-EnvelopeRegistryService-010 |
| Depends on | AIFACTORY-US-1447 |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml, docs/planes/data-event/services/envelope-registry-service/index.md#workers-and-jobs |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-5868 Deprecate transition, TopicDeprecated outbox record and the DeprecationSweeper worker |
implementation | The Deprecate transition of an Active topic sets deprecatedAt on its current schema version and writes TopicDeprecated with retiresAt from DataEvent:Envelopes:DeprecationWindowDays in the same transaction; DeprecationSweeper moves the Deprecated topics whose retiresAt passed to Retired through WindowElapsed once per topic and day, publishes nothing and updates connectsoft.factory.data_event.topics. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5869 Verify deprecation and retirement through the acceptance suite |
verification | Run dotnet test -c Release --filter "Category=Acceptance&Scenario=SCN-EnvelopeRegistryService-010"; a zero-match filter counts as a failure; the tests prove AC1 to AC3 of AIFACTORY-US-1449 with a controlled clock, including the publication and payload of TopicDeprecated and two sweeps on the same day. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-US-1450: List the envelope carriage per messaging profile and announce the active profile of the installation with EnvelopeProfileActivated¶
behavior · persona platform-admin
As a platform admin or an operator, I want to see how the one envelope is carried on each messaging profile and which profile this installation runs, so that every host writes the same envelope and deduplicates the same way whatever transport the installation uses.
The port-level envelope run-event-envelope-v1 of ADR-0014, whose schema docs/reference/schemas/run-event-envelope.schema.json is canonical (D137), has the same members on every messaging profile: eventId, eventType, topic, schemaVersion, occurredAt, producer (service, instance, boundedContext), tenantId, projectId, runId, stepId, attemptId, aggregate (type, id, version), correlationId, causationId, traceId, orderingKey, classification and payload, of which eventId, eventType, schemaVersion, occurredAt, producer, tenantId, correlationId and payload are required; what differs per profile is its carriage, and EnvelopeProfile records it: the body fields, the transport properties mirrored from the envelope (such as cs-event-type, cs-tenant-id and cs-schema-version), the outbox and inbox table contracts and the deduplication window in hours. The service holds one profile per value of MessagingProfile (OrleansOutboxInbox, MassTransitAzureServiceBus, NServiceBus, ADR-0012). A caller with factory.data.read lists them through EnvelopeRegistryService/ListEnvelopeProfiles and receives each profile with its carriage and contracts and the flag of the active one; a caller without the scope is refused with 403 and nothing is written. Under the reading of D137, the service activates at start-up the profile that DataEvent:Messaging:Profile names for the installation: when the active profile changes or the contracts of the active profile differ from what the service holds, it activates that profile, deactivates the others and writes EnvelopeRegistryService/EnvelopeProfileActivated with profile and envelopeSchemaVersion in the same transaction; a start with nothing changed writes nothing. A test checks the first invariant of the aggregate: the envelope members are identical across the three profiles and equal the members of the canonical schema listed above, with the same required members, and only their carriage differs. No profile promises exactly-once external effects or replay, and consumers deduplicate on eventId within the window (ADR-0014).
Acceptance criteria
| AC1 | Given the three envelope profiles and a caller with factory.data.read, when EnvelopeRegistryService/ListEnvelopeProfiles is called, then the response is 200 with each profile, its body fields, mirrored transport properties, outbox and inbox contracts and deduplication window and the flag of the active profile; a caller without the scope is refused with 403 and nothing is written (SCN-EnvelopeRegistryService-005). |
| AC2 | Given DataEvent:Messaging:Profile set to orleans-outbox-inbox and another profile active, or changed contracts of the active profile, when the service starts, then the EnvelopeRegistryService/EnvelopeProfile of that profile is active, the others are not, and EnvelopeRegistryService/EnvelopeProfileActivated is written in the same transaction; a second start with nothing changed writes nothing (SCN-EnvelopeRegistryService-011, ADR-0012, D137). |
| AC3 | Given a profile activated, when the outbox dispatcher publishes EnvelopeRegistryService/EnvelopeProfileActivated on data-foundations.envelopes.v1.envelope-profile-activated, then the event is sent under the canonical envelope with profile and envelopeSchemaVersion, and a transport outage is retried with backoff so the event is neither lost nor duplicated under its event id (SCN-EnvelopeRegistryService-011, ADR-0014). |
| AC4 | Given the three profiles, when the invariant test of EnvelopeRegistryService/EnvelopeProfile runs, then the envelope members are identical across the three profiles and equal the members and required members of docs/reference/schemas/run-event-envelope.schema.json, the canonical schema (D137), only their carriage differs, and no profile states an exactly-once or replay guarantee; a profile with a missing or extra member fails the test (ADR-0014, model/services/data-event/EnvelopeRegistryService.yaml). |
| Traces: services | EnvelopeRegistryService |
| Traces: aggregates | EnvelopeRegistryService/EnvelopeProfile |
| Traces: apis | EnvelopeRegistryService/ListEnvelopeProfiles |
| Traces: events | EnvelopeRegistryService/EnvelopeProfileActivated |
| Traces: descriptors | run-event-envelope-v1 |
| Traces: adrs | ADR-0012, ADR-0014 |
| Traces: requirements | FR-10, XL-02 |
| Traces: scenarios | SCN-EnvelopeRegistryService-005, SCN-EnvelopeRegistryService-011 |
| Relates: screens | topic-registry |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml, docs/reference/schemas/run-event-envelope.schema.json, docs/architecture/adr/0012-messaging-port-and-transport-profiles.md |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-557 Envelope profiles per messaging profile |
implementation | The three envelope profiles of MessagingProfile are held with their body fields, mirrored transport properties, outbox and inbox contracts and deduplication window; ListEnvelopeProfiles returns them with the active flag under factory.data.read; under the reading of D137, at start-up the profile of DataEvent:Messaging:Profile is applied and EnvelopeProfileActivated is written in the same transaction only when the active profile or its contracts changed. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5870 Invariant test of the envelope fields across the three envelope profiles |
fixture | A test compares the envelope members of each of the three profiles with each other and with the members and required members of docs/reference/schemas/run-event-envelope.schema.json, the canonical schema that the EnvelopeProfile summary in model/services/data-event/EnvelopeRegistryService.yaml restates (D137); it fails on a member missing from or added to any profile and on a profile that states exactly-once or replay. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5871 Verify the envelope profiles through the acceptance suite |
verification | Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-EnvelopeRegistryService-005\|Scenario=SCN-EnvelopeRegistryService-011)"; a zero-match filter counts as a failure; the tests prove AC1 to AC4 of AIFACTORY-US-1450, including two starts with and without a change and the publication and payload of EnvelopeProfileActivated. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-US-1451: Resolve an event name to its topic and validate an envelope and payload for the outbox writer of a host¶
behavior · persona platform-admin
As a platform admin who relies on the publish-time checks of every Factory host, I want the outbox writer of each host to resolve the topic, schema version and state of an event and to validate an envelope before it enters the outbox, so that no event is written on a retired topic, with a missing scope field or with a payload its consumers cannot read.
The hosts of every platform call two internal gRPC methods with a workload identity through the gateway DataEventFoundations. EnvelopeRegistryService/Envelopes.ResolveTopic resolves an event name to its topic, its current schema version and its state at publish time; its request topic-resolution-request-v1 names the event type and the producer, and its reply topic-resolution-v1 carries the event type, the topic, the producer, the current schema version and the state (D137). EnvelopeRegistryService/Envelopes.ValidateEnvelope takes envelope-validation-request-v1, which carries an envelope of run-event-envelope-v1 and the strict mode, and answers envelope-validation-v1 with the event id, the verdict and every violation, each with its path and rule (D137). These dedicated descriptors replace the borrowed run-event-v1 and ack-v1. A host caches the resolution, and while the registry is unavailable it keeps publishing with its cached topic and schema set, as the failure mode of the model states. EnvelopeRegistryService/Envelopes.ValidateEnvelope checks that every envelope field is present, the scope fields tenantId and, where the event family carries them, projectId, runId, stepId and attemptId, and the correlation fields correlationId and causationId, that the producer of the envelope is the producer of the topic, that the topic is not Retired, and that the payload is valid against the registered schema version the envelope names; the envelope is accepted or refused, each failed rule is a violation of envelope-validation-v1 with its path and rule, and every refused envelope increments connectsoft.factory.data_event.envelope_validation_failures by producer and topic. The option DataEvent:Envelopes:StrictValidation, on in non-production profiles, says whether every outbox write is validated. Both methods are of class read and change nothing in the registry. A caller without a workload identity is refused with UNAUTHENTICATED and an invalid request with INVALID_ARGUMENT; an event name or a topic the registry does not hold is answered NOT_FOUND. The outbox writer that calls these methods belongs to the messaging library of the base template with the Factory profile, consumed by version pin and never forked; which feature makes it call them is reported to the owner, and this PBI delivers the service side and the proto of the Envelopes service.
Acceptance criteria
| AC1 | Given a workload identity and the event name of an Active topic, when EnvelopeRegistryService/Envelopes.ResolveTopic is called, then the reply topic-resolution-v1 carries the topic, its current schema version and the state Active; for a Deprecated or Retired topic it carries that state; an unknown event name is answered NOT_FOUND, and nothing is written (SCN-EnvelopeRegistryService-006, D137). |
| AC2 | Given a valid envelope and payload for an Active topic, when EnvelopeRegistryService/Envelopes.ValidateEnvelope is called, then the reply envelope-validation-v1 is valid with no violation; given an envelope without tenantId or correlationId, a producer that is not the producer of the topic, a Retired topic or a payload that violates the registered schema version, then the reply is not valid, each failed rule is a violation with its path and rule, and connectsoft.factory.data_event.envelope_validation_failures is incremented by producer and topic (SCN-EnvelopeRegistryService-007, ADR-0014, D137). |
| AC3 | Given a caller without workload identity or an invalid request, when either method is called, then it is refused with UNAUTHENTICATED or INVALID_ARGUMENT and no state changes (SCN-EnvelopeRegistryService-006, SCN-EnvelopeRegistryService-007, ADR-0015). |
| Traces: services | EnvelopeRegistryService |
| Traces: apis | EnvelopeRegistryService/Envelopes.ResolveTopic, EnvelopeRegistryService/Envelopes.ValidateEnvelope |
| Traces: descriptors | topic-resolution-request-v1, topic-resolution-v1, envelope-validation-request-v1, envelope-validation-v1, run-event-envelope-v1 |
| Traces: adrs | ADR-0012, ADR-0014, ADR-0015 |
| Traces: requirements | XL-02, NFR-05 |
| Traces: scenarios | SCN-EnvelopeRegistryService-006, SCN-EnvelopeRegistryService-007 |
| Depends on | AIFACTORY-US-1448, AIFACTORY-US-1449, feature:contracts-package |
| Sources | model/services/data-event/EnvelopeRegistryService.yaml, docs/reference/schemas/run-event-envelope.schema.json, docs/reference/schemas/topic-resolution.schema.json, docs/reference/schemas/envelope-validation.schema.json, docs/planes/data-event/services/envelope-registry-service/index.md#purpose-and-responsibilities |
Tasks
| Task | Kind | Expected output | Repository |
|---|---|---|---|
AIFACTORY-TASK-5872 Proto of the Envelopes service with the methods and the dedicated descriptors of the model |
contract | The proto of the Envelopes service declares ResolveTopic with the messages of topic-resolution-request-v1 and topic-resolution-v1 and ValidateEnvelope with the messages of envelope-validation-request-v1, which carries the envelope of run-event-envelope-v1, and envelope-validation-v1 with the verdict and the violations, as model/services/data-event/EnvelopeRegistryService.yaml and docs/reference/schemas/topic-resolution.schema.json and envelope-validation.schema.json record them (D137); it defines no field the schemas do not hold and is published from the repository as the contract the outbox writers of the hosts build against. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5873 ResolveTopic and ValidateEnvelope against the registered topics and schema versions |
implementation | ResolveTopic answers topic-resolution-v1 with topic, current schema version and state for an event name and NOT_FOUND for an unknown one; ValidateEnvelope checks the envelope fields, the scope and correlation fields, the producer of the topic, the Retired state and the payload against the registered schema version and answers envelope-validation-v1 with the verdict and every failed rule as a violation with its path and rule, incrementing connectsoft.factory.data_event.envelope_validation_failures; callers without workload identity and invalid requests are refused with UNAUTHENTICATED and INVALID_ARGUMENT; neither method writes. | ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |
AIFACTORY-TASK-5874 Verify publish-time resolution and validation through the acceptance suite |
verification | Run dotnet test -c Release --filter "Category=Acceptance&(Scenario=SCN-EnvelopeRegistryService-006\|Scenario=SCN-EnvelopeRegistryService-007)"; a zero-match filter counts as a failure; the tests prove AC1 to AC3 of AIFACTORY-US-1451 over an Active, a Deprecated and a Retired topic and an envelope matrix of missing fields, a foreign producer and an invalid payload. |
ConnectSoft.AI.SoftwareFactory.EnvelopeRegistryService |