[nherson]

Problem

TimeoutRecoveryReader#read(count) (the exp_async_timeout_recovery read path) fills values through its 8 KB line buffer: one read_nonblock(8192) per chunk, then a copy of each chunk into the result. A value of N bytes therefore costs about N/8192 recvfrom calls plus one extra copy of the whole payload. A 512 KB value takes ~65 reads; a 2 MB value ~257.

Change

After draining whatever is already buffered, if at least CHUNK_SIZE bytes are still owed, read reads them straight from the socket:

• The first read goes into the result string itself, so it needs no copy.
• Later reads ask for min(remaining, LARGE_READ_SIZE) (64 KiB) into a per-connection scratch string and append it. The cap keeps the retained scratch at 64 KiB no matter how large the values are.
• A read never asks for more than the bytes still owed. The buffer stays empty after a value, so finish_request!'s pending? check is unaffected.
• Every wait goes through wait_for, so the post-deadline poll still applies to each read.
• EOF behaves as before: a short read returns the bytes received, and a read with nothing received returns nil.

Values with less than CHUNK_SIZE still owed, and gets, are unchanged.

Results

memcached on a UNIX socket, Ruby 3.4, async 2.46. The client uses the meta protocol, raw values, socket_timeout: 0.05, socket_max_failures: 0. Syscalls per op come from strace -f -c (500 iterations minus 0 iterations, divided by 500).

• default: the default read path, without the flag
• recovery (before): exp_async_timeout_recovery: true on main
• recovery (after): exp_async_timeout_recovery: true with this change

case	default	recovery (before)	recovery (after)
get 1 KB	3.1	2.1	2.0
get 16 KB	7.2	4.2	4.2
get 128 KB	35.4	18.6	5.6
get 512 KB	131.8	66.7	11.7
get 2 MB	516.8	258.7	36.8
get_multi 40 × 4 KB	43.6	22.5	22.5
get_multi 40 × 12 KB	124.0	62.7	62.6
get_multi 20 × 64 KB	324.9	163.1	45.3

CPU time and p50 latency per op, in µs (median of 3 × 3000 iterations; default / before / after):

case	CPU	p50
get 1 KB	31.8 / 32.9 / 33.3	52.0 / 52.6 / 52.3
get 128 KB	69.8 / 73.0 / 62.0	88.6 / 85.1 / 75.1
get 512 KB	195.2 / 164.7 / 115.9	207.3 / 198.3 / 162.0
get 2 MB	698.8 / 529.1 / 333.7	704.0 / 641.0 / 455.4
get_multi 40 × 4 KB	231.3 / 268.9 / 266.3	227.5 / 255.4 / 256.1
get_multi 20 × 64 KB	528.6 / 449.3 / 358.4	514.3 / 453.3 / 388.2

The EPoll and URing selectors give the same counts, within 0.4 syscalls per op. With local memcached the data is nearly always ready, so io-event's optimistic non-blocking read completes without going through the ring.

The new async_get target in bin/benchmark (700 KB value over TCP, Ruby 3.2) shows the same direction:

	default	recovery
before	4115 i/s	4967 i/s
after	4097 i/s	6844 i/s

Not addressed: get_multi of many small values (< 8 KB each) costs ~15% more CPU with the flag than without, before and after this change. That cost comes from Ruby-level line parsing in the reader rather than syscalls.

TLS EOF fix

Randomized testing found a bug that predates this change: at EOF, a socket's read_nonblock empties the buffer it was given, but OpenSSL's leaves it untouched. fill then reset the read offset, so bytes already returned were returned again. After a TLS server closed a connection cleanly, the next request with exp_async_timeout_recovery could receive the previous reply; for example, a set reported success although the server never received it, where the default path reconnects and stores the value. fill now clears the buffer when the read hits EOF. The new large-value path was not affected: at EOF it only hands OpenSSL an empty string or a scratch buffer whose contents are discarded.

Post-timeout poll fix

Also found by the randomized tests, in CI: after a wait reports a timeout, the reader polls once with a zero timeout to see whether data arrived anyway. A zero-timeout poll can report nothing without checking the socket while the thread has a pending interrupt (from Thread#wakeup, a queue push, and the like), so the reader could fail a read whose data had already arrived. With data always present on a socket and another thread interrupting the main thread, wait_readable(0) reported nothing about 1 in 3,600 times on Ruby 3.2 and between 1 in a million and 1 in 5 million on 3.3–4.0 (never without the interrupts). When the poll finds nothing, the reader now makes one more non-blocking read and fails only if that read has nothing too. Waits and their bounds are unchanged.

Timeout behaviour

Every wait in the new path goes through the existing wait_for: a wait of up to the socket timeout through the fiber scheduler, then, if that reports a timeout, one zero-timeout poll outside it, continuing with a fresh wait if data is there, or a final non-blocking read if the poll finds nothing. The timeout stays per wait, as before: a read fails one timeout after the last byte arrives, and a value that arrives in pauses each shorter than the timeout still succeeds. With the same seed, a stalled stream failed at most 5.9 ms past the timeout with the previous reader and 5.8 ms with this one (180 stalls each).

Tests

• Large value: reads a 40 × 8 KB + 3 byte value split across two writes and asserts it takes fewer than 20 socket reads. This fails on main with 41 reads. The test also checks that the trailing \r\n and EN\r\n are still read correctly, which proves the reader never reads past the value.
• Short read at EOF: a large value cut short by EOF returns the bytes received, and the next read returns nil.
• Large-value timeouts: recovery when a wait in the middle of a value reports a timeout while data is ready; failure within one timeout once a value stops arriving; a full timeout per wait while a value arrives in pauses.
• TLS EOF: after a TLS peer sends a reply and closes, the reader returns the reply once and then nil.
• Missed poll: when the poll after a timed-out wait reports nothing although data has arrived, the reader still returns the data.
• Randomized (test/protocol/test_timeout_recovery_reader_fuzz.rb): compares the reader with IO#gets/IO#read semantics over UNIX, TCP, and TLS sockets, with no scheduler and under each available Async selector. Payloads include sizes at the 8 KB and 64 KiB boundaries, values up to 1.5 MB, embedded protocol lines, NUL bytes, and invalid UTF-8, written in random pieces and sometimes cut short by EOF. The socket injects short reads, spurious :wait_readable/:wait_writable, and waits that report a timeout although data is ready. Each case checks pending?, that large reads stop at the value's end, that every wait is either the socket timeout through the scheduler or a zero-timeout poll outside it, that a stalled stream fails within one timeout of its last byte, and that the scratch buffer stays at 64 KiB. FUZZ_SEED replays a run, FUZZ_ITERATIONS scales it, and FUZZ_STATS=1 prints coverage.
• Integration (test/integration/test_async_timeout_recovery.rb): the delayed-reply server can now send large values in parts with pauses. New tests cover a large value whose second half arrives while another fiber holds the thread (recovered with the option, IO::TimeoutError without it), pauses each shorter than the timeout, and failure within one timeout once a value stops arriving, followed by a correct read on the next request.

The randomized and integration tests pass on Ruby 3.3 and 3.4 on Linux (EPoll, Select, URing; 1,560 randomized cases per Ruby version) and Ruby 4.0 on macOS (KQueue, Select). Deliberately breaking the reader (over-reading past a value, skipping the post-timeout poll, waiting without a timeout, polling for a full timeout, or reverting the TLS fix) makes these tests fail.

bin/benchmark gains an async_get target that compares the default and recovery read paths under Sync. It runs only with BENCH_TARGET=async_get, and only that run installs async, so BENCH_TARGET=all (the Benchmarks workflow) runs exactly as on main.