Repository navigation
Conversation
Track request rates, failures, queue and processing latency, response queue latency, total latency, and in-flight requests by listener and API. Attribute Produce metrics to canonical database/table names and share the lifecycle of locally hosted table metrics. Handle partial failures, acks=0, response ordering, and cancellation without duplicate counts. Document the metric boundaries and cover request timing, table attribution, cleanup, and listener registration. Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 512/512 AI-Contributed/UT: 947/947
Share the metric-group cache and track request progress with one stage and one stage timestamp. Consolidate failure handling and remove unused construction paths without changing metric semantics. Merge API and table metric tests into one fixture, retaining core timing, failure attribution, acks=0, cancellation, cleanup, and listener wiring. Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 111/111 AI-Contributed/UT: 915/915
Wait for topic and partition metadata to report success after creating tables, since online replicas do not guarantee a synchronized Kafka metadata cache. Keep the protocol-version, alias, topic ID, and database non-recreation assertions unchanged. Co-Authored-By: Codex <noreply@openai.com> AI-Model: gpt-6 AI-Contributed/Feature: 0/0 AI-Contributed/UT: 11/11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generative AI disclosure: Yes — OpenAI Codex (GPT-6).
Purpose
Related issue: #4185.
Kafka listeners currently lack request-level throughput, failure, latency, and in-flight metrics. Add these metrics by listener and API, with an additional database/table breakdown for Produce requests.
Brief change log
requestsPerSecond,failedRequestsPerSecond,requestQueueTimeMs,requestProcessTimeMs,responseQueueTimeMs,totalTimeMs, andinFlightRequests.acks=0, and cancellation without double counting.Tests
KafkaMetadataITCasenow waits for topic and partition metadata readiness after creating tables before running the custom-default-database alias assertions. All original protocol-version, topic ID, and database non-recreation checks are retained.KafkaRequestMetricsTestuses one fixture and five test methods (nine cases) to cover timing and write completion, table aliases and partial failures,acks=0, cancellation, processing/serialization failures, and table metric cleanup. A single metrics integration test inKafkaProtocolPluginITCaseverifies both API and table registration through a real Netty listener.Validated on JDK 11, including the full
KafkaMetadataITCaseintegration suite:mvn -o -pl fluss-kafka -am verify \ '-Dtest=*Kafka*Test,ApiVersionsHandlerTest,JsonToFlussConvertersTest,TabletServerMetricGroupTest,TableMetricGroupTest,KafkaProtocolPluginITCase,KafkaMetadataITCase' \ -Dsurefire.failIfNoSpecifiedTests=falseResult: BUILD SUCCESS. The selected Kafka suite ran 348 tests, and the server suite ran 13 tests, with no failures, errors, or skips. Spotless and Checkstyle passed.
API and Format
Adds a backward-compatible default overload of
NetworkProtocolPlugin.createChannelHandlerto supply the server metric group, and a lookup for existing local table metric groups. No Kafka wire protocol or storage format changes.Documentation
Updated the Kafka compatibility guide with metric names, scopes, timing boundaries,
acks=0behavior, and per-table counting and lifecycle semantics.