[nherson]

Problem

socket_timeout is applied to sockets through IO#timeout. Under a fiber scheduler, that timeout is a scheduler timer. The timer starts when a read starts to wait, which is right after Dalli sends the request.

Fiber schedulers run an event loop, and one pass of the loop works like this:

1. Poll for ready descriptors.
2. Resume the waiting fibers one at a time.
3. Run expired timers.

Suppose a fiber that this pass resumed keeps the thread for longer than socket_timeout. A reply that arrives during that time is not seen until the next poll, but the reply's timer runs first, in step 3. The read raises IO::TimeoutError even though the reply is already in the socket.

Dalli then handles this as a network failure. It closes the socket, which discards the reply. With socket_max_failures it also marks the server down, so requests during down_retry_delay fail as well. From the caller's side, a slow neighbouring fiber looks like a server timeout.

The exposure depends on the selector backend. The tests below reproduce it with Async on every io-event backend:

The busy fiber was resumed by	URing	EPoll	KQueue	Select
its own descriptor becoming ready	fails	fails	fails	fails
the ready queue (Thread::Queue, ConditionVariable, promises)	fails	ok	ok	ok
a timer (sleep)	ok	ok	ok	ok

Rescuing IO::TimeoutError and reading again is not safe. Before IO#gets and IO#read(n) wait, they move bytes from Ruby's internal read buffer into the result string they are building. When the exception propagates, that string is discarded. A second read would then start in the middle of a response and get out of step with the protocol.

Solution

This change adds an opt-in request option, exp_async_timeout_recovery: true. It uses the existing request-options plumbing, so a single request can opt in:

client.get('key', exp_async_timeout_recovery: true)
client.get_multi('a', 'b', exp_async_timeout_recovery: true)
client.set('key', 'value', nil, exp_async_timeout_recovery: true)

• Base#request reads the option and passes it to ConnectionManager#start_request!. The flag is cleared when the request finishes or aborts, or when the connection closes.
• While the flag is set, ConnectionManager#read_line, #readline, #read and #read_exact read through the new Dalli::Protocol::TimeoutRecoveryReader:
  • It reads with read_nonblock into a connection-local binary buffer and slices lines and values out of it.
  • It waits with wait_readable(socket_timeout). With an explicit timeout, a scheduler returns false instead of raising.
  • When a wait expires, it calls Fiber.blocking { wait_readable(0) }, which polls the socket directly instead of going through the scheduler. If data is there, the read continues. If not, it raises IO::TimeoutError, which takes the same rescue path and failure policy as today.
  • Unconsumed bytes stay in the reader's buffer across waits, so continuing after an expired wait never loses data.
• If a flagged request finishes with unread bytes in the buffer, the connection is closed. A later unflagged read, which reads the socket directly, can therefore never skip those bytes.
• The reader is dropped when the connection closes, so a reconnect or a fork starts from a clean state.

The following are unchanged:

• requests without the option
• writes, which still use IO#timeout
• connection setup
• the pipelined get_multi path used with multiple servers or a block
• error classes and the retry and down-marking policy

Memory and allocations

• A connection that never serves a flagged request allocates nothing. A test covers this.
• A connection that serves a flagged request holds one 8 KiB buffer (8,233 bytes measured), and frees it when the connection closes.
  • read_nonblock does not use Ruby's internal IO read buffer. A connection that serves only flagged requests therefore never allocates Ruby's own 8 KiB read buffer.
  • A connection that serves both flagged and unflagged requests holds both buffers.
• The buffer does not grow with value size. A large value is copied into a string sized to the value, and a test checks the buffer size after a 1 MB read.
• Each line or value allocates one string, the same as IO#gets and IO#read. The read loops avoid block-based iteration, which would allocate on every call, and a test checks the per-line allocation count.

Benchmark setup: Linux, Ruby 4.0.6 with YJIT, memcached 1.6.41 on a UNIX socket, async 2.46.0, io-event 1.21.1. Each row shows the median per operation and objects allocated per operation.

Operation	Scheduler	Default	With option
get 100 B	none	39 µs, 17 objects	41 µs, 16 objects
get 700 KB	none	271 µs, 17 objects	235 µs, 21 objects
get_multi 40 × 3 KB	none	137 µs, 616 objects	138 µs, 641 objects
get 100 B	Async/EPoll	42 µs, 33 objects	45 µs, 24 objects
get 700 KB	Async/EPoll	275 µs, 393 objects	256 µs, 40 objects
get_multi 40 × 3 KB	Async/EPoll	140 µs, 692 objects	146 µs, 650 objects
get 100 B	Async/URing	44–46 µs, 33 objects	43–45 µs, 24 objects
get 700 KB	Async/URing	252 µs, 398 objects	229 µs, 41 objects
get_multi 40 × 3 KB	Async/URing	141 µs, 692 objects	144 µs, 650 objects

On macOS without a scheduler, get_multi of 40 × 3 KB values was slower with the option: 120 µs default against 159 µs. A profile shows that this time is spent in socket calls, not in the reader's Ruby code.

Tests

test/protocol/test_timeout_recovery_reader.rb (no memcached needed):

• reads lines and values that cross chunk boundaries
• matches IO#gets and IO#read results and encodings, including at EOF
• raises IO::TimeoutError when nothing arrives before the timeout
• reads data that becomes ready while the wait reports a timeout
• keeps partially read bytes across a timeout
• keeps the buffer size bounded after a 1 MB read
• allocates one string per buffered line

test/integration/test_async_timeout_recovery.rb:

• Without a fiber scheduler:
  • Flagged and unflagged requests return the same results over TCP, a UNIX socket and SSL. The operations covered are get (hit, miss and 700 KB value), get_multi with misses, get_multi_with_status, get_with_status, get_cas, gat, touch, fetch, incr, set and delete.
  • Flagged and unflagged requests share a connection without reconnecting.
  • Unflagged requests allocate no reader.
  • A reply that never arrives still fails.
• Under Async, for each io-event selector available on the host (URing, EPoll, KQueue, Select):
  • A reply that a busy fiber delays is returned when the busy fiber was resumed by descriptor readiness, by the ready queue or by a timer, and the same holds for get_multi.
  • Without the option, the readiness case fails. This control shows the harness reproduces the problem.
  • A reply that never arrives still fails.
  • Cancelling the waiting fiber closes the connection.
  • Flagged and unflagged requests return the same results.
• The delayed-reply server runs in a separately spawned process, so its replies arrive while the test process's thread is busy.

Results:

• Linux (Ruby 4.0.6, memcached 1.6.41 built with --enable-tls, IO_EVENT_SELECTOR=URing as in CI): the full suite ran 643 tests with 0 failures and 0 errors.
• macOS: the new tests pass with the KQueue and Select selectors. The SSL parity test needs a memcached built with TLS.
• CI Linux runners exercise URing, EPoll and Select. KQueue runs only on macOS.

Rollout

The option is experimental and off by default, so callers can roll it out gradually. A follow-up will either make this the default read path or remove the option.

[mrattle]

Left a couple of test comments, one which should unblock CI.

I think the actual intent of the change makes sense and the implementation is good.

[mrattle]

Kinda irrelevant for our purposes tbh but this test is hanging in the ruby 3.2 and 3.3 test matrices in CI.

My robot did some smoke testing and came back with this:

I ran the same sequence on main with default reads only (flush, set, a 700 KB set, then set_multi of 40 × 3 KB, twice on one SSL connection). It hung in the same frame in 1 of 6 runs. On this openssl version, SSLSocket#sysread doesn't apply IO#timeout, so socket_timeout never fires and the wait has no limit. I didn't find out why memcached stops replying.

It's possible that the mn was never received by the server for that one flake, so the pipelined request was never registered as finished and memcached never relied back. You could probably just make this test skip on OpenSSL::VERSION < '3.3'