Repository navigation
Conversation
Server channels started reading as soon as they connected, so a channel registering while request-queue backpressure was already active was never paused: reconcileBackpressure early-returns while the queue sits between the resume and backpressure thresholds, and pauseAllChannels only runs at the pause transition. Under overload, reconnecting clients kept feeding the unbounded request queue. Start server channels with autoRead disabled in ServerChannelInitializer and make RequestChannel.registerChannel the single point that decides reads: it aligns the channel with the current backpressure state, enabling reads when backpressure is inactive and pausing the channel when it is active, so late arrivals join the paused set and are released by the next resume transition. Inactive channels are skipped, since registration races with disconnects on the event loop. Update the registerChannel and backpressureLock documentation to match the new semantics, and cover them in RequestChannelTest.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Server channels start reading the moment they connect, so a channel that registers while request-queue backpressure is already active is never paused:
reconcileBackpressureearly-returns while the queue sits between the resume and backpressure thresholds, andpauseAllChannelsonly runs at the pause transition. Under overload, reconnecting clients (timed-out producers that reconnect into fresh connections) keep feeding the unbounded per-processor request queue — the exact memory growth backpressure exists to prevent.This changes server channels to start with reads disabled and makes
RequestChannel.registerChannelthe single point that decides reads, aligned with the current backpressure state.Linked issue: part of umbrella #4185 (Kafka Protocols Compatibility – Basic Produce), whose Future Work lists bounded Produce admission and flow control; referenced without closing it.
Brief change log
ServerChannelInitializer: accepted Fluss-protocol channels start withautoRead=false; they cannot read a single byte before their backpressure state has been decided.RequestChannel.registerChannel: aligns the channel with the current state — enables reads when backpressure is inactive, pauses the channel when backpressure is active. A channel arriving during an episode therefore joins the paused set and is released by the next resume transition once the queue drains below the resume threshold. Inactive channels are skipped (registration races with disconnects on the event loop, and a closed channel's read state no longer matters).registerChanneljavadoc previously documented the opposite behavior ("new channels are NOT immediately paused ... can cause deadlock"); it and thebackpressureLockjavadoc now describe the actual semantics, including why registration needs no lock (the add-before-flag-read ordering plus event-loop serialization of read-state changes).RequestChannelTest: new cases for registering during active backpressure, enabling reads for a start-paused channel, and skipping inactive channels; the hand-writtenTestChannelfake gained anactiveflag.Complements #4897 (which registers Kafka listener channels): once both merge, channels of any protocol that feed a queue are gated at registration. The two PRs both touch
RequestChannel.javain non-overlapping regions; whichever lands second needs a trivial rebase.Tests
RequestChannelTest.testRegisterDuringActiveBackpressureStartsPaused— a channel registering while backpressure is active stays paused and resumes with the rest when the queue drains (red against both the previous code and the intermediate "enable-only" variant).RequestChannelTest.testRegisterEnablesReadsForStartPausedChannel— a channel delivered with reads disabled gets reads enabled on registration (red against previous code).RequestChannelTest.testRegisterSkipsInactiveChannel— an inactive channel's read state is left untouched (red against the unguarded variant, which also NPEs on test doubles whoseconfig()is null, caught byMessageCodecTest).mvn test -pl fluss-rpcgreen.API and Format
No public API or storage format change;
registerChannelsemantics are internal to the server RPC layer.Documentation
No new feature or configuration option; no documentation change needed.
Generative AI disclosure: