Skip to content

[BUG]: cpex-rate-limiter 0.1.11 rejects ordinary prompts when by_user_per_server is enabled #202

Closed

Description

@vishu-bh

Summary

Enabling by_user_per_server in cpex-rate-limiter==0.1.11 breaks ordinary prompt retrieval.
The Rust core requires GlobalContext.server_id for both tool_pre_invoke and prompt_pre_fetch.
ContextForge ordinary prompt requests omit this optional field.
The plugin rejects valid requests with RATE_LIMIT_CONTEXT_MISSING and a 503 violation status.

Keep by_user_per_server specific to tool invocations.
Preserve existing prompt protection through by_user, by_tenant, and prompt-name overrides in by_tool.

Related work

Affected version and configuration

Affected package: cpex-rate-limiter==0.1.11.
The regression occurs with the memory backend and the shared Redis backend.
The plugin must register prompt_pre_fetch alongside tool_pre_invoke.

hooks:
  - prompt_pre_fetch
  - tool_pre_invoke
mode: enforce
config:
  backend: memory
  by_user: null
  by_user_per_server: "2/h"
  by_tenant: null
  by_tool: {}

Minimal reproduction

Run this script with cpex-rate-limiter==0.1.11 installed:

import asyncio

from cpex.framework import (
    GlobalContext,
    PluginConfig,
    PluginContext,
    PromptPrehookPayload,
)
from cpex_rate_limiter.rate_limiter import RateLimiterPlugin


async def main():
    plugin = RateLimiterPlugin(
        PluginConfig(
            name="RateLimiterPlugin",
            kind="cpex_rate_limiter.rate_limiter.RateLimiterPlugin",
            hooks=["prompt_pre_fetch", "tool_pre_invoke"],
            config={
                "backend": "memory",
                "by_user": None,
                "by_user_per_server": "2/h",
                "by_tenant": None,
                "by_tool": {},
            },
        )
    )
    context = PluginContext(
        global_context=GlobalContext(
            request_id="prompt-regression",
            user="alice@example.com",
        )
    )
    try:
        result = await plugin.prompt_pre_fetch(
            PromptPrehookPayload(prompt_id="summary", args={}), context
        )
        print("continue_processing:", result.continue_processing)
        if result.violation:
            print("code:", result.violation.code)
            print("status:", result.violation.http_status_code)
    finally:
        await plugin.shutdown()


asyncio.run(main())

Actual output:

continue_processing: False
code: RATE_LIMIT_CONTEXT_MISSING
status: 503

Control configuration: set by_user="2/h" and by_user_per_server=None.
The same prompt request succeeds.

Expected behavior

Prompt retrieval succeeds without a catalog server_id when existing prompt limits permit it.
Prompt retrieval consumes no by_user_per_server counters.
Existing global user, tenant, and prompt-name limits remain active.
Shared global counters continue to include both prompt fetches and tool invocations.

Gateway impact

ContextForge registers both hooks in its shipped limiter configuration.
Enabling per-server tool quotas therefore breaks an existing prompt API path.
REST GET returns HTTP 503 in the live reproduction.
REST POST wraps the same plugin violation as HTTP 422.

Virtual-server prompt calls also carry a different identity from tool calls.
Prompt calls use a virtual-server UUID; tool quotas use an upstream catalog UUID.
Requiring clients to supply a quota identity does not resolve this mismatch.
Disabling prompt hooks removes existing prompt protection.

Suggested correction

Exclude by_user_per_server from prompt-hook evaluation in the Rust core.
Share the backend and existing global counters between prompt and tool evaluation.
Retain catalog identity validation for tool invocations.
Document the hook-specific behavior and publish a corrected package release.

Acceptance criteria

  • Ordinary prompt fetches succeed without server_id when only by_user_per_server is configured.
  • Prompts consume no per-user catalog quota, including calls through different virtual servers.
  • by_user, by_tenant, and prompt-name overrides in by_tool continue to protect prompts.
  • Global counters remain shared between prompt and tool hooks.
  • Tool calls without a valid catalog identity still fail when by_user_per_server is configured.
  • Regression coverage exercises memory and Redis backends, REST GET/POST, and MCP prompts/get.
  • A corrected package release allows the gateway PR to update its dependency and lockfile.

Local validation of a candidate correction

A local correction gives prompt evaluation a configuration view without by_user_per_server.
The prompt and tool engines share the same backend.
Validation against upstream commit 57fc9f0c36cbcac3b1d96ea85d73f0317c8de35c reports:

  • Rust tests: 90 passed.
  • Python hook tests: 51 passed.
  • Rust formatting and Clippy with -D warnings: passed.
  • Live gateway matrix with the corrected wheel: 24 passed; 12 prompt cases skip because tool bindings activate only tool hooks.
  • Published 0.1.11: four stateless REST prompt regression cases fail; two global-only controls pass.

The live matrix covers both MCP transport modes, two replicas, regular users, explicit teams, and wildcard runtime bindings.
The corrected wheel is local. Dependency adoption awaits an upstream release.

Activity

  1. added
    bugSomething isn't working
    rustPull requests that update rust code
    on Oct 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingrustPull requests that update rust codetriage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions