[msureshkumar88]

Pull Request

🔗 Related Issue

Closes #6571

---

📝 Summary

This PR makes ContextForge speak MCP 2026-07-28 subscriptions/listen (SEP-2575) on both sides and keeps legacy behavior unchanged:

• As a server, the gateway accepts subscriptions/listen from modern clients. It sends notifications/{tools,prompts,resources}/list_changed when its own catalog changes, on every worker.
• As a client, polling of modern upstreams is proven end to end and now meets the MCP caching spec (jitter and backoff). The optional standing listener to modern upstreams is hardened: one holder, jittered reconnects, refresh after re-listen.
• Legacy clients and servers see no behavior change. Two pre-existing bugs that broke the issue's legacy criteria are fixed: the per-server refresh interval was never saved, and debounced change notifications were dropped.

Why this needed code (and not an SDK bump)

The MCP Python SDK 2.x ships the building blocks: mcp.server.subscriptions.ListenHandler and SubscriptionBus. It registers them automatically only for the high-level MCPServer. ContextForge builds its endpoint on the low-level Server (ContextForgeMCPServer(Server[Any])), where on_subscriptions_listen defaults to None. Before this PR:

• a 2026-07-28 client got -32601 for subscriptions/listen;
• 2026-07-28 capabilities reported listChanged: false (the SDK derives the flag from whether subscriptions/listen is registered).

I checked SDK 2.2.0 (installed) and 2.3.0 (latest): both behave the same way, so an upgrade does not help. The gateway must also publish events when its catalog changes, and fan them out across workers. No SDK can do that for us.

---

✅ Acceptance criteria coverage

Criterion (#6571)	State before this PR	Delivered by
Server: accept subscriptions/listen from modern clients and send events when the gateway's own catalog changes	Missing: no handler, -32601, listChanged: false	Catalog change bus fed by registry-cache invalidation, a listen handler on /mcp and /servers/{id}/mcp, and the missing invalidations closed
Client: polling works for modern MCP servers	Code path present (v2 Client negotiates either era), never proven; no jitter or backoff	A live proof against an SDK MCPServer upstream on 2026-07-28, plus jitter and failed-refresh backoff
Client: hold listen streams to modern upstreams (listener service, optional)	Built, but every worker opened its own stream; undocumented; no e2e	One holder via primary-worker election, jittered reconnects, refresh after re-listen, flag documented in .env.example, Helm and docs, live proof. The default stays false
Legacy: unchanged; an upstream change shows within one refresh interval (default 300 s, per-server override respected)	The per-server override was silently ignored (never persisted)	Era gate keeps legacy -32601; legacy capabilities pinned byte-identical; refresh never later than today; refresh_interval_seconds now persisted

---

🏗️ Architecture

End-to-end change flow

flowchart LR
    subgraph Mutations
        A[Admin CRUD / import / gRPC] --> I
        U[Upstream refresh<br/>scheduled · manual · listener] --> I
    end
    I["RegistryCache.invalidate_*()"] -->|local, before Redis| N
    I -->|"Redis mcpgw:cache:invalidate<br/>registry:&lt;type&gt;"| S[CacheInvalidationSubscriber<br/>on peer workers]
    S --> N["notify_catalog_change()"]
    N --> C[Coalesce per kind, 0.3 s]
    C --> B[InMemorySubscriptionBus<br/>one per worker]
    B --> L[SDK ListenHandler]
    W[ContextForge wrapper:<br/>era gate · permission narrowing · lifetime] --> L
    L -->|SSE: ack, then list_changed| M[Modern client]

Loading

Server side in detail

1. Event source: mcpgateway/cache/catalog_change_bus.py (new).

   • notify_catalog_change(cache_type) maps invalidation types to SDK events:

     Invalidation	Events
     tools	ToolsListChanged
     prompts	PromptsListChanged
     resources	ResourcesListChanged
     servers	all three (a virtual server's membership changes all three of its lists)
     anything else (agents, gateways, catalog, …)	none

   • It is synchronous and never raises. It does nothing when there is no running loop or the type is unmapped.

   • It coalesces per kind and per loop in a fixed 0.3 s window, and keeps a strong reference to each publish task.

   • It imports only the stdlib and the SDK, so registry_cache can import it without a cycle.

2. Hook points: mcpgateway/cache/registry_cache.py.
   • RegistryCache.invalidate() emits right after the in-memory clear and before the Redis block. Delivery on the local worker therefore needs neither Redis nor a successful SCAN.
   • CacheInvalidationSubscriber._process_invalidation() emits on registry:<type> messages from peer workers.
   • About 37 existing invalidate_* call sites now publish events with no change at the call site.
3. Listen handler: _adapt_subscriptions_listen in mcpgateway/transports/streamablehttp_transport.py. It runs these steps in order:
   1. Era gate: anything other than 2026-07-28 gets -32601, so legacy sessions see exactly today's response.
   2. Scope: _resolve_catalog_scope() resolves server, headers and user context once. When MCP_REQUIRE_AUTH=false it also runs the oauth_enabled check once.
   3. Narrowing: each requested kind is kept only if _token_allows_catalog_read() grants the matching list permission (tools.read, prompts.read, resources.read). The handler builds a fresh SubscriptionFilter, so resourceSubscriptions and unknown or extension keys can never reach the SDK.
   4. Deny: if every requested list kind is denied, the handler raises the same PermissionError(_ACCESS_DENIED_MSG) as tools/list.
   5. Delegate: the call goes to one process-wide SDK ListenHandler, which owns the ack, subscription-ID stamping, buffering and the per-worker 1024-stream cap. No DB session is held while it runs.
   6. Lifetime: the delegate runs under anyio.move_on_after(deadline). The deadline is the earlier of token exp and MCP_LISTEN_MAX_LIFETIME_SECONDS, with a 5 s floor. When it fires, the handler returns SubscriptionsListenResult(_meta={subscriptionId}), which is the spec's graceful closure. A client disconnect still propagates as a cancellation.
4. Shared gate. _list_catalog_page now calls the same two helpers, and LIST_PERMISSIONS maps every */list method exactly. The old .get(method, "resources.read") fallback is gone, so an unexpected method raises instead of being authorized as a resources read.
5. Capabilities. ContextForgeMCPServer.get_capabilities clears resources.subscribe for modern versions, next to its existing modern-only logging removal. This also reaches server/discover. The SDK builds new capability objects on every call, so nothing leaks between requests.
6. /rpc (mcpgateway/main.py) answers subscriptions/listen with -32601 before the tool-name fallback.

Client side in detail

• Polling (mcpgateway/services/gateway_service.py):
  • _jittered_tick() sleeps interval × (1 − 0.1 × U[0,1]) between maintenance cycles.
  • The refresh throttle compares against refresh_interval × 0.9.
  • _refresh_backoff_seconds() and _refresh_failures skip a gateway after a failed scheduled refresh for min(cap, tick × 2^failures) × U[0.5, 1]. The cap is the gateway's effective refresh interval, and success resets the count.
  • Manual refresh is unaffected.
• Modern listener (mcpgateway/services/modern_listener_service.py):
  • _reconcile() starts listeners only when is_primary_worker() is true, and cancels them all when that flips to false.
  • _backoff_with_jitter() applies after both errors and graceful closes. The base backoff resets only after a stream stayed up for BACKOFF_MAX_SECONDS.
  • After a re-listen that follows a drop or close, it calls notify_list_changed once per kind.
• Startup (mcpgateway/main.py):
  • The listener now starts after start_primary_worker_elector().
  • A WARNING fires when listeners are on with CACHE_TYPE=redis and the filelock election backend.

Pre-existing bug fixes

• refresh_interval_seconds was never persisted. GatewayCreate and GatewayUpdate accepted it, but nothing in mcpgateway/ wrote it to the row. Both create paths now persist it. Update writes it only when the payload sets it (model_fields_set), and an explicit null clears it.
• Debounced list_changed was dropped (mcpgateway/services/notification_service.py).
  • Before: an event that arrived inside the 5 s window after the pending refresh had already run was counted as "debounced" and discarded, with no later refresh. A second upstream change a few seconds after the first stayed invisible until the next poll.
  • Now: the service schedules at most one trailing refresh per gateway at last_enqueued + debounce. Later events merge their flags into it, it goes through the same queue and rate limiting as a normal refresh, and shutdown and unregister_gateway cancel it.
  • Owner check: pops of _pending_refresh_flags and _trailing_refresh_tasks check entry identity, so a queue backlog cannot leave an untracked second trailing task.

---

🧭 Decisions and why

The decisions below come from the issue, the 2026-07-28 spec (re-fetched and checked line by line on 2026-10-19), the SDK source and the existing code.

D1 — Every listener gets every change kind it may read; no per-virtual-server filtering

• Chosen because:
  • The spec requires only "The server MUST NOT send notification types the client has not explicitly requested" (basic/patterns/subscriptions), and says servers declaring listChanged "SHOULD send a notification" (server/tools). Nothing requires scoping list_changed to the caller's visible set.
  • Events carry only the kind, never data. The client refetches through its own RBAC-scoped */list, which already returns cacheScope: "private".
  • SDK events (ToolsListChanged, …) have no audience field, and invalidation messages carry no scope.
• Alternative rejected: per-virtual-server or per-team delivery. It needs a new event pipeline that carries scope out of every service. That is a far larger change for a no-data signal.
• Known cost: a listener can tell that the platform catalog changed, including outside its teams. That is a timing signal with no content. A /servers/{id}/mcp listener may also refetch when nothing in its server changed. Per-server delivery can follow if this matters in practice.

D2 — Redis carries events across workers; without Redis, same-worker best effort

• Chosen because:
  • The platform already treats Redis as its cross-worker channel. .env.example calls CACHE_TYPE=database "good for single-node" and redis the mode "for multi-node deployments". Registry-cache coherence, auth-cache invalidation, session affinity and leader election already depend on it. Helm and docker-compose default to CACHE_TYPE=redis.
  • The spec makes delivery best effort. The send is a SHOULD and nothing defines replay. Its stated fallback is ttlMs: "TTL is a freshness hint, not a guarantee" (server/utilities/caching), and the gateway already sets ttlMs on every list.
  • Both SDKs advise implementing the bus over your own pub/sub, and neither ships a database fallback.
  • The existing mcpgw:cache:invalidate channel already reaches every worker, so no custom Redis SubscriptionBus is needed.
• Alternatives rejected:
  • A DB-polled fan-out needs a new table and migration, and duplicates Redis.
  • Hiding listen without Redis would disable it on single-worker deployments too, where same-worker delivery is complete.
• Documented next to CACHE_TYPE in configuration.md.

D3 — GATEWAY_MODERN_LISTENERS_ENABLED stays false

• Chosen because:
  • The spec does not recommend that clients hold listen streams; polling with ttlMs is the baseline.
  • Polling already meets the legacy freshness criterion for every era.
  • Flipping the default changes an environment default, a consumer contract under AGENTS.md blocking criterion 4. It would also open long-lived outbound connections to every eligible upstream, including ones behind idle-timeout proxies.
  • The issue calls the listener "purely optional at the moment", and the issue is labeled COULD.
• Follow-up: with the single holder and the e2e proof in place, flipping the default is a one-line change for a later release.

D4 — The listen permission gate is the matching */list gate, not stricter

• Chosen because events carry no data, so listen cannot reveal anything tools/list would not. Sharing one helper means the two gates cannot drift apart.
• Note: the gate is a token-scope ceiling. An empty scope defers to RBAC, exactly as for the list calls.

D5 — A stream ends at token exp or MCP_LISTEN_MAX_LIFETIME_SECONDS (default 3600), with a 5 s floor

• Chosen because:
  • The spec is silent on re-authentication during a long-lived stream. Without a bound, a revoked, deactivated or narrowed caller would keep receiving change pings until disconnect.
  • Ending at exp or a lifetime cap bounds that window without a per-stream revocation watcher. The client re-listens and re-authenticates.
  • The 5 s floor ensures the ack always precedes the completion result. Callers without exp, or with exp: None (basic or proxy auth, MCP_REQUIRE_AUTH=false), get the max lifetime.
• Closure: completion uses the spec's graceful closure ("SHOULD respond to the original subscriptions/listen request with a completion result before closing the stream"). No notifications/cancelled is sent: the cancellation page contradicts graceful closure (spec issue modelcontextprotocol#3348), and the schema limits server-sent cancel to stdio.
• Expired tokens never reach the handler, because the auth middleware returns 401 first.

D6 — Subtractive jitter on health-loop ticks, a 0.9 × throttle, and capped backoff on failed refresh

• Chosen because:
  • The caching spec says "Implementations that do choose to poll MUST apply jitter and backoff". Before this PR the refresh had neither, and a failed fetch retried on every tick.
  • Jitter on the refresh due-time alone does nothing at defaults: 10 % of 300 s never crosses a 60 s tick boundary. The jitter therefore sits on the health-loop sleep.
  • Shorter ticks alone would make refreshes later. The throttle needs a measured gap of at least R, so with R a multiple of T (300/60) the refresh slipped one extra tick (300 s → up to 360 s; R == T == 60 doubled the cadence). The throttle threshold is therefore scaled to 0.9 × R.
• Proof: each tick is at least 0.9T, so ⌈R/T⌉ ticks sum to at least 0.9R. The refresh fires on or before today's tick, never later, and the legacy "within one refresh interval" bound holds.
• Alternative rejected: additive jitter, which would stretch the documented bound.
• Cost: a scheduled refresh can run up to 10 % early. That is slightly more upstream load and is noted in the CHANGELOG.
• Backoff: capped at the gateway's refresh interval, so the legacy bound holds again once the upstream recovers.

D7 — <category>.* token scopes and the HTTP 403 mapping are a follow-up

• The gap: _check_scoped_permission does not honor <category>.* scopes, whereas token_scope_grants() (the AGENTS.md single policy point) does.
• The mapping: on the 2026-07-28 wire, a catalog permission denial reaches the client as -32603 with HTTP 200 (the SDK maps a raw PermissionError that way). The spec wants HTTP 403 with error="insufficient_scope".
• Deferred because: both fixes change */list behavior for existing tokens, so they belong in their own PR. The scope gap fails closed: a tools.* token is refused, never over-granted. Listen deliberately matches tools/list today.

D8 — Listen streams close abruptly on shutdown

• Chosen because:
  • Uvicorn waits for open connections before it runs lifespan shutdown, so a ListenHandler.close() in the shutdown path never runs while streams are open.
  • A graceful close would need a uvicorn exit-signal hook that nothing in the app has today.
  • The spec allows an abrupt close: the client "MAY treat [it] as a trigger to reconnect". Long-lived legacy GET /mcp streams already behave this way, and D5 bounds every stream anyway.

D9 — Omit resources.subscribe and drop resourceSubscriptions on 2026-07-28

• Chosen because:
  • The spec ties resources.subscribe to "resource-specific update notifications for resources requested through subscriptions/listen" (server/resources).
  • The ack "reflects the subset the server agreed to honor" (basic/patterns/subscriptions), and the schema says the ack "Only includes notification types the server actually supports".
  • The gateway emits no per-URI resources/updated, while the SDK would otherwise advertise subscribe: true and echo resourceSubscriptions.
• Scope: legacy resources/subscribe is untouched.
• Effect: a request that carries only resourceSubscriptions gets an ack with an empty filter, not an error.

Implementation decisions made during the build

Decision	Why	Cost if wrong
Hook events at RegistryCache.invalidate(), not in each service	One choke point already covers about 37 call sites and already crosses workers through Redis	A mutation that skips invalidation also skips the event. The audit closed every gap found (below)
Suppress a worker's own Redis echo by coalescing, not by explicit dedupe	The echo usually lands inside the 0.3 s window. A late echo causes one duplicate list_changed, which is harmless (idempotent refetch) and self-heals a stale L1 read	An occasional duplicate event
The single listener holder reuses is_primary_worker() instead of a new Redis/FileLock holder	The project already ships primary-worker election (filelock per host, Redis lease per cluster) with heartbeat and failover. Reuse removed about 150 lines of bespoke lease code and a GET-then-DEL release race	With CACHE_TYPE=redis but the default filelock election backend, the listener runs once per pod, not once per cluster. A startup WARNING, the Helm comment and the docs tell operators to set PRIMARY_WORKER_ELECTION_BACKEND=redis
Reconnect jitter 0.5×–1.5× on the listener	"Subtractive only" binds refresh freshness (the legacy criterion), not reconnect backoff	A reconnect can be up to 50 % slower than the base backoff
Listener backoff resets only after a stream stayed up for BACKOFF_MAX_SECONDS	An upstream that closes at once would otherwise be re-listened every ~5 s, each time with a full three-kind refresh	Slower recovery after a short but healthy stream
Refresh after re-listen	The spec defines no replay. Events missed while the stream was down must be caught by a refetch	One extra debounced refresh per reconnect
Fix the two pre-existing bugs here	Each breaks a #6571 criterion: "per-server override respected" and listener freshness	A larger PR
Gateway delete or transfer does not also invalidate servers	tools, resources and prompts invalidation already fires all three list_changed kinds, and listeners refetch their scoped list	The admin REST server-list cache can be stale for up to its 20 s TTL
gRPC set_service_state does not invalidate	It writes only grpc_services.enabled/updated_at; no tool row changes, and tool lists never read gRPC service state	None
MCP_LISTEN_MAX_LIFETIME_SECONDS is a string in values.schema.json	Every config-map key in that schema is a string; the pydantic Field enforces 60–86400 at startup	An invalid Helm value fails at gateway start, not at helm install
Do not add subscriptions/listen to _MCP_KNOWN_REQUEST_METHODS	That set feeds only the legacy session-less unknown-method peek. Adding it would turn today's -32601 for a legacy session-less probe into the SDK's -32600 Missing session ID	None
Live test gateways set RATE_LIMITING_ENABLED=false	The 100 rpm REST limit tripped the polling helpers	Test harness only
Cross-worker live test asserts at least 5 events (one per create)	Redis-echo duplicates are expected (see above)	The test no longer catches a duplicate-event storm

Invalidation audit

Every create, update, delete and toggle path in the tool, prompt, resource, server, gateway, import, A2A and gRPC services was checked. Every new call runs after a successful db.commit(), and a failed commit never invalidates.

Path	Before	After
ResourceService.register_resource	no invalidate	invalidate_resources
ResourceService.register_resources_bulk	none	once per call, keyed on committed chunks
PromptService.register_prompts_bulk	none	once per call (also before re-raising TemplateValidationError when an earlier chunk committed)
ServerService.register_server	none	invalidate_servers
GatewayService.update_gateway (both branches)	invalidate_gateways only	plus tools, resources and prompts (it renames tools and propagates visibility)
GatewayService.delete_gateway (async lifecycle and hard delete)	gateways and catalog only	plus tools, resources and prompts
GatewayService.transfer_gateway_ownership	none	tools, resources and prompts (owner, team and visibility change)
ImportService._assign_imported_items_to_team	none	servers, tools, resources and prompts
GrpcService register, update, delete and reflection sync	none	invalidate_tools
Tool create/update/delete/toggle/bulk, set_gateway_state, the upstream refresh (scheduled, manual, listener-triggered), A2A toggle	already invalidating	regression guards added

---

🔒 Security

• Authentication before stream establishment. Every /mcp and /servers/{id}/mcp request, of any era, passes streamable_http_auth in MCPPathRewriteMiddleware. Then come the servers.use check (403) and the server-existence check (404), all before the SDK sees the body. Modern requests carry no Mcp-Session-Id, so the 30 s session-affinity forward to /rpc never applies to them.
• Layer 1 / Layer 2. Listen uses the same token-scope gate and scoped visibility helpers as */list (D4). Identity comes only from the server-side user context set by the auth middleware; nothing in params affects authorization.
• No data in events. Events carry only the kind. Catalog content is only ever returned by the caller's own */list.
• MCP_REQUIRE_AUTH=false. An unauthenticated caller can listen with public-only scope, except on oauth_enabled virtual servers, which return 401 (security invariant).
• Resource bounds.
  • The SDK caps listen streams at 1024 per worker and ends a stream whose backlog reaches 1024 events.
  • Each worker also caps concurrent listen streams per authenticated principal (MCP_LISTEN_MAX_STREAMS_PER_PRINCIPAL, default 32) and for all anonymous callers together (MCP_LISTEN_MAX_ANONYMOUS_STREAMS, default 256; 0 refuses anonymous streams). A stream past a cap gets -32603 Subscription limit reached.
  • Coalescing keeps a burst (for example a 1,000-tool import) to about one event per kind per 0.3 s.
  • D5 bounds stream lifetime, and no DB session is held while a stream is open.
• Deny-path tests (unit and live):
  • unauthenticated → 401
  • expired token → 401
  • no servers.use on a server-scoped path → 403
  • unknown server → 404
  • invisible or disabled server → same as tools/list
  • partly denied scope → the ack narrowed to the permitted kinds
  • all kinds denied → same error as tools/list
  • oauth_enabled server unauthenticated with MCP_REQUIRE_AUTH=false → 401
  • legacy era → -32601

---

🔁 Compatibility and contract changes

Change	Who sees it	Flagged in
New env var MCP_LISTEN_MAX_LIFETIME_SECONDS (default 3600, 60–86400)	Operators	.env.example, Helm values and schema, configuration.md, CHANGELOG Added
New env vars MCP_LISTEN_MAX_STREAMS_PER_PRINCIPAL (default 32, 1–1024) and MCP_LISTEN_MAX_ANONYMOUS_STREAMS (default 256, 0–1024)	Operators, modern MCP clients	.env.example, Helm values and schema, configuration.md, CHANGELOG Added
GATEWAY_MODERN_LISTENERS_ENABLED documented (default false, unchanged); runs on the primary worker	Operators	.env.example, Helm, configuration.md, CHANGELOG Changed, startup WARNING
2026-07-28 capabilities: listChanged: true; resources.subscribe omitted	Modern MCP clients	CHANGELOG Added and Changed
Scheduled refresh jittered and may run up to 10 % early, never later	Operators, upstreams	CHANGELOG Changed
Failed scheduled refresh backs off (capped at the interval)	Operators, upstreams	CHANGELOG Changed
refresh_interval_seconds now persisted; gateways that set it refresh on that schedule for the first time	API users	CHANGELOG Fixed
A debounced upstream list_changed gets one trailing refresh instead of being dropped	Legacy and modern upstreams	CHANGELOG Fixed
Missing cache invalidations closed (lists no longer stale for up to the cache TTL)	All clients	CHANGELOG Fixed

• Unchanged: legacy-era capabilities (pinned byte-identical by a test), the legacy session behavior and -32601, JSON_RESPONSE_ENABLED (the SDK always serves listen as SSE), the DB schema (no Alembic migration) and the SDK version.

Deployment notes

• Multi-worker or multi-replica: use CACHE_TYPE=redis (the Helm and compose default) so listen events reach every worker.
• Turning on GATEWAY_MODERN_LISTENERS_ENABLED with several replicas: also set PRIMARY_WORKER_ELECTION_BACKEND=redis to get one listener per cluster rather than one per pod.
• Proxies:
  • The SDK sends an SSE keep-alive every 15 s, which resets idle-read timeouts (nginx /mcp has proxy_read_timeout 1h and proxy_buffering off).
  • Worker recycling and redeploys cut streams abruptly; clients re-listen.
  • If the Rust MCP runtime returns in front of /mcp, it would forward listen to /rpc with a 30 s timeout, so listen would need to bypass it. The compose stack no longer starts it (chore: remove Rust MCP config surface (issue #5819) #7255).

---

🧪 Verification

Check	Command	Status
Ruff	make ruff	✅ pass
Docstring coverage	make interrogate	✅ 100 %
Pylint	make pylint	10.00/10. Exits 2 on an existing E0611 at mcpgateway/middleware/rate_limit_middleware.py:25, a file this PR does not touch (it also fails on main)
Unit tests	make test	24586 passed, 3 failed, none caused by this PR. test_auth.py::TestCookieOnlySessionSmoke::test_refresh_with_cookie_only_without_csrf_rejected also fails on main (a211ab7d8). The two test_config.py CSRF-warning tests pass alone on this branch and on main, so they depend on xdist test order
Touched unit files after the last fix commits	uv run pytest <7 touched files>	✅ 945 passed
Live gateway proofs	uv run pytest tests/live_gateway/mcp/test_subscriptions_listen.py -k "not cross_worker"	✅ 20 passed in about 2 min 40 s (real gateway and SDK MCPServer upstream subprocesses)
Cross-worker live proof	test_listen_cross_worker_delivery	Needs the compose testing stack; left to CI
Coverage, e2e, secrets	make coverage diff-cover, make test-e2e, make detect-secrets-scan	Left to CI
Helm	helm lint charts/mcp-stack	✅ pass

What the live proofs cover (tests/live_gateway/mcp/test_subscriptions_listen.py)

• Delivery: the ack is the first frame and every event carries the same subscription ID, on /mcp and on /servers/{id}/mcp.
• D9: the ack never contains resourceSubscriptions, and server/discover omits resources.subscribe.
• Deny paths: all paths in the Security section.
• Legacy: capabilities equal the snapshot, and listen returns -32601.
• Polling:
  • a modern upstream change is picked up by the scheduled refresh, honoring a per-server 60 s interval against a 300 s global default;
  • manual refresh as an extra case;
  • a scheduled refresh reaches a downstream listen stream.
• Modern listener: an upstream change appears with AUTO_REFRESH_SERVERS=false within seconds, and the listener-triggered refresh reaches a downstream listen stream.
• Trailing refresh: a second upstream change inside the 5 s debounce window is still refreshed.
• Persistence: PUT /gateways/{id} stores refresh_interval_seconds.
• Cross-worker (compose): 5 creates on fresh connections spaced 0.6 s apart; at least one event per create. Most creates land on another worker, so a broken Redis path drops events.

Review history

• Design: the design was reviewed twice by a separate reviewer and checked against the official 2026-07-28 spec pages. That check found 4 gaps, all fixed before implementation: resources.subscribe, jitter that did nothing at defaults, the denial wire shape, and the close edge cases.
• Tasks: each task was reviewed for spec compliance and quality before the next started. Fix rounds:
  • Polling jitter: one round, for the "later than today" regression above.
  • Live proofs: one round, for an owner check on two pops in the trailing-refresh code.
• Whole branch: a final review returned "with fixes" and no blockers. All its Important findings are fixed in this PR:
  • the listener hammering an upstream that closes at once;
  • the gRPC invalidation gap;
  • the CHANGELOG completeness;
  • the Helm note on listener scope.

---

📁 Files

Area	Files
New	mcpgateway/cache/catalog_change_bus.py, tests/unit/mcpgateway/cache/test_catalog_change_bus.py, tests/live_gateway/mcp/test_subscriptions_listen.py
Listen server	mcpgateway/transports/streamablehttp_transport.py, mcpgateway/cache/registry_cache.py, mcpgateway/main.py (/rpc, startup order, warning)
Invalidation gaps	mcpgateway/services/{resource,prompt,server,gateway,import,grpc}_service.py
Polling	mcpgateway/services/gateway_service.py
Notifications	mcpgateway/services/notification_service.py
Modern listener	mcpgateway/services/modern_listener_service.py
Config and docs	mcpgateway/config.py, .env.example, charts/mcp-stack/values.yaml, charts/mcp-stack/values.schema.json, docs/docs/manage/configuration.md, docs/docs/architecture/get-stream-quickstart.md (new "change delivery per era" section), CHANGELOG.md

36 files changed, about 2.8k insertions, most of them tests.

---

📏 Reviewability

• This PR has one clear purpose
• The linked issue is not labeled triage
• Unrelated bugs or improvements are tracked in separate issues/PRs
• Tests are included with the code they validate
• If AI-assisted, I understand and can explain the generated changes

---

🏷️ Type of Change

• Bug fix
• Feature / Enhancement
• Documentation
• Refactor
• Chore (deps, CI, tooling)
• Other (describe below)

---

✅ Checklist

• Code formatted (make black isort pre-commit)
• Tests added/updated for changes
• Documentation updated (if applicable)
• No secrets or credentials committed

---

📓 Notes

Known limits

• Without Redis, events reach only listeners on the same worker (D2). ttlMs remains the freshness fallback.
• Listeners can infer that the catalog changed outside their teams (D1, no data).
• A listen stream on a revoked or narrowed token keeps receiving kind-only pings until exp or the max lifetime (D5).
• A refresh that raises an exception (rather than returning success: False) gets no backoff, as before.
• On a redeploy, open listen streams are cut, not closed gracefully (D8).

Follow-ups (not in this PR)

• D7: honor <category>.* scopes in the streamable HTTP list gate, and map catalog denials on the 2026-07-28 wire to HTTP 403 insufficient_scope.
• A graceful listen close on SIGTERM (D8).
• Per-virtual-server event scoping (D1), if refetch load or the timing signal matters.
• Flip GATEWAY_MODERN_LISTENERS_ENABLED to true in a later release (D3).

Commit history

Commit 5a2336125 introduced a dedicated holder lock, and b37ff1736 replaced it with primary-worker election. Squash-merge keeps the history clean.

Spec references

• Subscriptions: https://modelcontextprotocol.io/specification/2026-07-28/basic/patterns/subscriptions
• Caching: https://modelcontextprotocol.io/specification/2026-07-28/server/utilities/caching
• Tools: https://modelcontextprotocol.io/specification/2026-07-28/server/tools
• Resources: https://modelcontextprotocol.io/specification/2026-07-28/server/resources
• Authorization: https://modelcontextprotocol.io/specification/2026-07-28/basic/authorization
• Streamable HTTP: https://modelcontextprotocol.io/specification/2026-07-28/basic/transports/streamable-http
• SEP-2575: https://modelcontextprotocol.io/seps/2575
• Stream-close semantics (open): cancellation.mdx and subscriptions.mdx disagree on how a server ends a subscriptions/listen stream modelcontextprotocol/modelcontextprotocol#3348

[gandhipratik203]

1. functionally-impacting | streamablehttp_transport.py::_adapt_subscriptions_listen: With the default MCP_REQUIRE_AUTH=false, an anonymous caller gets a listen stream. The SDK cap is 1024 streams per worker, and each stream can live up to 1 hour. One client can fill the cap and block legitimate modern clients on that worker. The PR defers the per-principal cap to a follow-up.

2. functionally-impacting | catalog_change_bus.py (D1): Events carry no audience. A change to a team-private tool pings every listener, including public-only and other-team callers. This is a content-free timing side channel, and AGENTS.md treats visibility as Layer 1.

[msureshkumar88]

Thanks for the review. Both points are on the stream lifecycle, so here is where each one stands.

1. One caller can fill the 1024-stream listen cap: fixed.

• 366e7b719 adds two per-worker caps, checked before the stream opens:
  • MCP_LISTEN_MAX_STREAMS_PER_PRINCIPAL (default 32) for each authenticated principal.
  • MCP_LISTEN_MAX_ANONYMOUS_STREAMS (default 256) for all anonymous callers together. 0 refuses anonymous streams.
• A stream past a cap gets -32603 Subscription limit reached, the same error as the SDK cap. A request that the token scope denies never takes a slot, and every exit path releases the slot.
• 773878c4d counts an empty-string email as anonymous.
• Tests:
  • Unit: per-principal refusal, principal isolation, the anonymous cap, the anonymous cap at 0, release after an error, a denied request that takes no slot, and an empty email.
  • Live gateway (b9b6d0eee): test_listen_anonymous_streams_capped_without_blocking_authenticated and test_listen_streams_capped_per_principal.
• One correction to the premise: MCP_REQUIRE_AUTH inherits AUTH_REQUIRED (default true), so anonymous listen is an operator opt-in. The per-principal cap still matters, because one authenticated caller could fill the worker cap too.
• I removed the per-principal cap from the PR's follow-up list and added both settings to the Security and Compatibility sections.

2. Events carry no audience: I propose to keep this as the documented decision D1.

• The 2026-07-28 spec does not require list_changed to be scoped to the caller's visible set. D1 quotes the relevant spec text.
• Events carry only the kind. The client refetches through its own scoped */list, so no catalog data crosses a team boundary. What leaks is the fact that something changed.
• A correct fix must carry the audience out of every write path: about 70 invalidation sites across 7 services, plus a new cross-worker message. Gateway children keep their own visibility, and some paths use bulk Core update()/delete(). An audience missed on any path means a listener misses an event and keeps a stale catalog. I think that correctness risk outweighs a content-free timing signal for this PR.
• "Per-virtual-server event scoping (D1)" stays in the follow-up list. I can open a tracking issue with the design if you want it before merge.

If you see the timing signal as more serious than D1 describes, let me know and we can look at it again.

[gandhipratik203]

Thanks for the changes! LGTM.

[msureshkumar88]

Closing this PR. As requested, I split the work per sub-issue of #6571:

• feat: serve MCP 2026-07-28 subscriptions/listen to modern clients (#6573) #7414 closes Serve subscriptions/listen to modern clients #6573 (serve subscriptions/listen to modern clients). It carries all of this PR's server, client-side and legacy-refresh work, rebased on main. It adds notifications/cancelled on gateway teardown, which Serve subscriptions/listen to modern clients #6573 requires. It also adds a catch-up refresh when a new primary worker starts its listeners.
• feat: per-resource subscriptions on subscriptions/listen from modern upstreams (#6644) #7415 closes Per-resource subscriptions on modern client to modern server #6644 (per-resource subscriptions from modern upstreams). It is stacked on feat: serve MCP 2026-07-28 subscriptions/listen to modern clients (#6573) #7414.

Thanks @gandhipratik203 for the review here. Both caps from that discussion are in #7414.