Feature/io uring#264
Merged
Merged
Conversation
|
An automated preview of the documentation is available at https://264.corosio.prtest3.cppalliance.org/index.html If more commits are pushed to the pull request, the docs will rebuild at the same URL. 2026-06-02 22:23:37 UTC |
Add a speculative non-blocking syscall fast path to every socket op: read_some / write_some / submit_send / submit_recv attempt ::readv / ::sendmsg / ::recvmsg before falling through to the io_uring submit path. On success the op completes without a kernel round-trip; on EAGAIN, the io_uring path runs unchanged. Speculative ::accept4 also fires at the top of the multishot acceptor entry. Connect is left on the io_uring path because IORING_OP_CONNECT re-invokes connect(2) internally and a prior speculative ::connect leaves the fd in EINPROGRESS → EALREADY. Gate the speculative attempts on a per-socket per-op-type hint (detail::speculative_state). The hint is flipped false when speculation discovers an exhausted buffer (EAGAIN) and restored when an io_uring CQE indicates kernel readiness (res > 0). Skips the wasted speculative syscall when the kernel buffer is known empty / full. Embed the per-op slots (uring_read_op, uring_write_op, uring_connect_op, uring_dgram_send_op, uring_dgram_recv_op, file read/write ops) as members of each socket/file impl. Eliminates the per-call heap allocation on the I/O hot path and gives the speculative path stable storage to dispatch through (the embedded cont_op is always there). Batch deferred SQE submission via submit_sqes_op. The first cross-thread io_uring_submit_op in a batch wins a CAS and posts a single op that flushes the SQ ring; subsequent submitters in the same batch piggyback on the same flush rather than each issuing their own syscall. Keep do_one's submit_and_get_events + process_completions prologue so the kernel CQE pump runs on every dispatch iteration. A polling timer with 0ns expiry keeps completed_ops_ non-empty and the leader- phase kernel pass below it never runs without the prologue; CQEs accumulate in the ring forever. Misc liveness / safety: - Cap the leader's unbounded kernel wait at 1s — defense in depth against a lost wakeup (multishot poll on wakeup_eventfd_ silently terminating). - Align op destroy() with the reactor backend — do not touch the awaiter handle at shutdown; calling h.destroy() in op destroy() recurses through capy's promise dtor. - Release ring_mutex_ across the leader's kernel wait so cross-thread submitters can prep new SQEs while the leader sleeps. - Switch the wakeup poll SQE to multishot and force-wake unconditionally from interrupt_reactor in multi-thread mode (CAS- coalescing would drop wakes given the kernel waits indefinitely between CQEs). The reactor backend still speculates unconditionally and uses iovec- style syscalls; porting the speculative_state mixin and the single- buffer fast path is future work.
- Drain expired timers at the top of do_one so stopper timers fire under continuous I/O and shutdown-deadlock socket_stress tests pass. - Skip io_uring_submit_and_get_events in do_one when no SQEs are in flight, gated on an io_uring_inflight_ counter incremented at SQE submit and decremented on the terminal CQE. - Defer the eager getsockname syscall on accepted TCP sockets to a three-state lazy-resolution scheme, so accept-heavy paths skip the round trip until local_endpoint() is observed. - Place outstanding_work_ and io_uring_inflight_ on distinct cache lines via alignas(64) to eliminate false sharing on multi-thread workloads. - Latch speculative reads permanently off after a consecutive-EAGAIN streak so structurally bursty workloads (e.g. fan_out:nested/16) stop burning a wasted readv syscall per read_some. - Emit IORING_OP_RECV / IORING_OP_SEND on single-buffer reads and writes to skip the iovec-array indirection that IORING_OP_READV / IORING_OP_SENDMSG pays. - Gate timer_service::process_expired() on timer_service::empty() so the unconditional timer drain added above is free (a single relaxed-acquire load) when no timer is registered. - Add BOOST_COROSIO_BENCH_ASIO_IO_URING (default ON) so the asio bench variants build against io_uring by default for apples-to-apples comparison, and reconfigure with -DBOOST_COROSIO_BENCH_ASIO_IO_URING=OFF to revert to asio's epoll reactor without touching the source.
- Implement wait() on all six io_uring socket/acceptor types via a new uring_wait_op that emits IORING_OP_POLL_ADD with POLLIN / POLLOUT / POLLPRI|POLLERR|POLLHUP for wait_type::read / write / error. - Add stream_file_type, stream_file_service_type, random_access_file_type, and random_access_file_service_type aliases to io_uring_t. - Include the io_uring detail headers from the native_*.hpp tag-dispatch wrappers so they can instantiate against io_uring_t. - Register reactor_paths.cpp for reactor backends only via a new COROSIO_REACTOR_BACKEND_TESTS macro: testWriteEAGAIN's small-buffer (SO_SNDBUF=1024) loopback pattern triggers a kernel-level slow-path in io_uring's POLLOUT-rearm cycle that exceeds reasonable ctest timeouts; io_uring socket coverage is preserved by the other test files.
|
GCOVR code coverage report https://264.corosio.prtest3.cppalliance.org/gcovr/index.html Build time: 2026-06-02 22:37:27 UTC |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #264 +/- ##
========================================
Coverage 77.78% 77.78%
========================================
Files 96 96
Lines 7256 7256
Branches 1769 1769
========================================
Hits 5644 5644
Misses 1102 1102
Partials 510 510
Continue to review full report in Codecov by Sentry.
🚀 New features to boost your workflow:
|
mvandeberg
force-pushed
the
feature/io_uring_perf
branch
8 times, most recently
from
e6ea2ae to
fd418a3
Compare
ci.yml: add liburing-dev to the apt-get list for the package-install step. The step only runs on apt-based systems, so macOS / Windows / FreeBSD entries are unaffected. code-coverage.yml: add a dedicated install step before the coverage script runs, so io_uring code paths are included in the Linux coverage report. io_uring: PUBLIC liburing link + clang-tidy fixes b2: detect liburing and enable io_uring backend when present test: register io_uring shadow tests for all native types fix(io_context): drop unsafe scheduler downcasts cmake: emit raw -luring for install consumers Fix asan leaks
mvandeberg
force-pushed
the
feature/io_uring_perf
branch
from
fd418a3 to
5be499f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.