liburing: add api for napi busy poll
v9 cover liburing: add api for napi busy poll
This adds two new api's to set/clear the napi busy poll settings. The two new functions are called:
- io_uring_register_napi
- io_uring_unregister_napi
The patch series also contains the documentation for the two new functions and two example programs. The client program is called napi-busy-poll-client and the server program napi-busy-poll-server. The client measures the roundtrip times of requests.
There is also a kernel patch "io-uring: support napi busy poll" to enable this feature on the kernel side.
Changes:
- V9:
- Update to library V2.4 and kernel 6.4
- V8:
- Use tab in liburing-ffi.map file
- Small fix to io_uring_register_napi man page
- V7:
- Add new functions to liburing-ffi.map file.
- Fix some whitespace issues.
- Address the compile errors from CI in the two files napi-busy-poll-client.c and napi-busy-poll-server.c
- V6:
- Check return value of unregister napi call and verify that busy poll timeout and prefer busy poll return the expected values.
- V5:
- Fixes to documentation.
- Correct opcode for unregister call
- Initialize napi structure in example programs
- Address tab issues in recordRTT()
- V4:
- Modify functions to use a structure to pass the napi busy poll settings to the kernel.
- Return previous values when returning from the above functions.
- Rename the functions and remove one function (no longer needed as the data is passed as a structure)
- V3:
- Updated liburing.map file
- Moved example programs from the test directory to the example directory. The two example programs don't fit well in the test category and need to be run from separate hosts.
- Added the io_uring_register_napi_prefer_busy_poll API.
- Added the call to io_uring_register_napi_prefer_busy_poll to the example programs
- Updated the documentation
- V2:
- Updated the liburing.map file for the two new functions. (added a 2.4 section)
- Added a description of the new feature to the changelog file
- Fixed the indentation of the longopts structure
- Used defined exit constants
- Fixed encodeUserData to support 32 bit builds
*** BLURB HERE ***
Stefan Roesch (4):
liburing: add api to set napi busy poll settings
liburing: add documentation for new napi busy polling
liburing: add example programs for napi busy poll
liburing: update changelog with new feature
.gitignore | 2 + CHANGELOG | 1 + examples/Makefile | 2 + examples/napi-busy-poll-client.c | 451 +++++++++++++++++++++++++++++++ examples/napi-busy-poll-server.c | 386 ++++++++++++++++++++++++++ man/io_uring_register_napi.3 | 40 +++ man/io_uring_unregister_napi.3 | 27 ++ src/include/liburing.h | 3 + src/include/liburing/io_uring.h | 12 + src/liburing-ffi.map | 2 + src/liburing.map | 2 + src/register.c | 12 + 12 files changed, 940 insertions(+) create mode 100644 examples/napi-busy-poll-client.c create mode 100644 examples/napi-busy-poll-server.c create mode 100644 man/io_uring_register_napi.3 create mode 100644 man/io_uring_unregister_napi.3
base-commit: b7f85996a5cb290fc2ad7d2f4d7341fc54321016
Signature · 1 line
2.39.1
PATCH v9 1/4 liburing: add api to set napi busy poll settings
This adds two functions to manage the napi busy poll settings:
- io_uring_register_napi
- io_uring_unregister_napi
--- src/include/liburing.h | 3 +++ src/include/liburing/io_uring.h | 12 ++++++++++++ src/liburing-ffi.map | 2 ++ src/liburing.map | 2 ++ src/register.c | 12 ++++++++++++ 5 files changed, 31 insertions(+)
src/include/liburing.h +3 -0
@@ -241,6 +241,9 @@ int io_uring_register_sync_cancel(struct io_uring *ring,int io_uring_register_file_alloc_range (struct io_uring *ring,unsigned off,unsigned len); +int io_uring_register_napi (struct io_uring *ring,struct io_uring_napi *napi); +int io_uring_unregister_napi (struct io_uring *ring,struct io_uring_napi *napi); +int io_uring_get_events (struct io_uring *ring);int io_uring_submit_and_get_events (struct io_uring *ring);
src/include/liburing/io_uring.h +12 -0
@@ -523,6 +523,10 @@ enum {/* register a range of fixed file slots for automatic slot allocation */ IORING_REGISTER_FILE_ALLOC_RANGE =25 , +/* set/clear busy poll settings */ +IORING_REGISTER_NAPI =26 , +IORING_UNREGISTER_NAPI =27 , +/* this goes last */ IORING_REGISTER_LAST,
@@ -662,6 +666,14 @@ struct io_uring_buf_reg {__u64 resv[3 ]; }; +/* argument for IORING_(UN)REGISTER_NAPI */ +struct io_uring_napi { +__u32 busy_poll_to; +__u8 prefer_busy_poll; +__u8 pad[3 ]; +__u64 resv; +}; +/* * io_uring_restriction->opcode values */
src/liburing-ffi.map +2 -0
@@ -171,6 +171,8 @@ LIBURING_2.4 { io_uring_prep_msg_ring_fd; io_uring_prep_msg_ring_fd_alloc; io_uring_prep_sendto; + io_uring_register_napi; + io_uring_unregister_napi; local: *; };
src/liburing.map +2 -0
@@ -79,4 +79,6 @@ LIBURING_2.4 { io_uring_register_restrictions; io_uring_setup_buf_ring; io_uring_free_buf_ring; + io_uring_register_napi; + io_uring_unregister_napi; } LIBURING_2.3;
src/register.c +12 -0
@@ -337,3 +337,15 @@ int io_uring_register_file_alloc_range(struct io_uring *ring,return do_register (ring,IORING_REGISTER_FILE_ALLOC_RANGE , &range,0 ); } + +int io_uring_register_napi (struct io_uring *ring,struct io_uring_napi *napi) +{ +return __sys_io_uring_register (ring->ring_fd, +IORING_REGISTER_NAPI , napi,0 ); +} + +int io_uring_unregister_napi (struct io_uring *ring,struct io_uring_napi *napi) +{ +return __sys_io_uring_register (ring->ring_fd, +IORING_UNREGISTER_NAPI , napi,0 ); +}
Signature · 1 line
2.39.1
Re: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 10 lines · Stefan Roesch
(quoted diff) +2
-0+ +int io_uring_register_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +6
-0+{ +return __sys_io_uring_register (ring->ring_fd, +IORING_REGISTER_NAPI , napi,0 ); +} + +int io_uring_unregister_napi(struct io_uring *ring, struct
io_uring_napi*napi)
src/register.c int io_uring_register_file_alloc_range(struct io_uring *ring, · line 351
Quoted 4 lines
src/register.c +4
-0+{ +return __sys_io_uring_register (ring->ring_fd, +IORING_UNREGISTER_NAPI , napi,0 ); +}
my apologies if this is not the latest version of the patch but I think that it is...
nr_args should be 1 to match what __io_uring_register() in the current corresponding kernel patch expects:
https://git.kernel.dk/cgit/linux/commit/?h=io_uring-napi&id=787d81d3132aaf4eb4a4a5f24ff949e350e537d0
Jens Axboe
maintainerRe: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 19 lines · Olivier Langlois
Quoted 10 lines · Stefan Roesch
(quoted diff) +2
-0+ +int io_uring_register_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +6
-0+{ +return __sys_io_uring_register (ring->ring_fd, +IORING_REGISTER_NAPI , napi,0 ); +} + +int io_uring_unregister_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c int io_uring_register_file_alloc_range(struct io_uring *ring, · line 351
Quoted 4 lines
src/register.c +4
-0+{ +return __sys_io_uring_register (ring->ring_fd, +IORING_UNREGISTER_NAPI , napi,0 ); +}my apologies if this is not the latest version of the patch but I think that it is...
Quoted 4 lines
nr_argsshould be 1 to match what__io_uring_register()in the current corresponding kernel patch expects:https://git.kernel.dk/cgit/linux/commit/?h=io_uring-napi&id=787d81d3132aaf4eb4a4a5f24ff949e350e537d0
Can you send a patch I can fold in?
Signature · 1 line
Jens Axboe
Re: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 22 lines · Jens Axboe
Quoted 21 lines · Olivier Langlois
Quoted 13 lines · Stefan Roesch
(quoted diff) +2
-0+ +int io_uring_register_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +6
-0+{ +return __sys_io_uring_register (ring->ring_fd, +IORING_REGISTER_NAPI , napi,0 ); +} + +int io_uring_unregister_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +3
-0+{ + return __sys_io_uring_register(ring->ring_fd, + IORING_UNREGISTER_NAPI, napi,Quoted 2 lines
0);
+}my apologies if this is not the latest version of the patch but I think
that it is...
Quoted 8 lines
Quoted 5 lines
nr_argsshould be 1 to match what__io_uring_register()in the current
corresponding kernel patch expects:https://git.kernel.dk/cgit/linux/commit/?h=io_uring-napi&id=787d81d3132aaf4eb4a4a5f24ff949e350e537d0
Can you send a patch I can fold in?
Jens,
I am unsure of what you are asking me.
You would like me to send a patch to fix a liburing patch that has never been applied AFAIK?
if the v9 patch has never been applied. Wouldn't just editing Stefan patch by replacing the nr_args param from 0 to 1 directly do it?
Jens Axboe
maintainerRe: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 34 lines · Olivier Langlois
Quoted 22 lines · Jens Axboe
Quoted 22 lines · Olivier Langlois
Quoted 13 lines · Stefan Roesch
(quoted diff) +2
-0+ +int io_uring_register_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +6
-0+{ +return __sys_io_uring_register (ring->ring_fd, +IORING_REGISTER_NAPI , napi,0 ); +} + +int io_uring_unregister_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +3
-0+{ + return __sys_io_uring_register(ring->ring_fd, + IORING_UNREGISTER_NAPI, napi,Quoted 2 lines
0);
+}my apologies if this is not the latest version of the patch but I think
that it is...Quoted 8 lines
Quoted 5 lines
nr_argsshould be 1 to match what__io_uring_register()in the current
corresponding kernel patch expects:https://git.kernel.dk/cgit/linux/commit/?h=io_uring-napi&id=787d81d3132aaf4eb4a4a5f24ff949e350e537d0
Can you send a patch I can fold in?
Jens,
Quoted 7 lines
I am unsure of what you are asking me.
You would like me to send a patch to fix a liburing patch that has never been applied AFAIK?
if the v9 patch has never been applied. Wouldn't just editing Stefan patch by replacing the
nr_argsparam from 0 to 1 directly do it?
The current version is what is in that branch you referenced. If you see anything in there that needs changing, send a patch for that.
I can send out the series again if needed.
—
Jens Axboe
Re: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 42 lines · Jens Axboe
Quoted 40 lines · Olivier Langlois
Quoted 23 lines · Jens Axboe
Quoted 23 lines · Olivier Langlois
Quoted 9 lines · Stefan Roesch
(quoted diff) +2
-0+ +int io_uring_register_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +6
-0+{ + return __sys_io_uring_register(ring->ring_fd, + IORING_REGISTER_NAPI, napi, 0); +} + +int io_uring_unregister_napi(struct io_uring *ring, structQuoted 6 lines
io_uring_napi*napi)
+{
+ return__sys_io_uring_register(ring->ring_fd, + IORING_UNREGISTER_NAPI, napi, 0);
+}my apologies if this is not the latest version of the patch but I
think
that it is...Quoted 8 lines
Quoted 5 lines
nr_argsshould be 1 to match what__io_uring_register()in the current
corresponding kernel patch expects:https://git.kernel.dk/cgit/linux/commit/?h=io_uring-napi&id=787d81d3132aaf4eb4a4a5f24ff949e350e537d0
Can you send a patch I can fold in?
Jens,
I am unsure of what you are asking me.
You would like me to send a patch to fix a liburing patch that has never been applied AFAIK?
Quoted 9 lines
Quoted 3 lines
if the v9 patch has never been applied. Wouldn't just editing Stefan
patch by replacing thenr_argsparam from 0 to 1 directly do it?The current version is what is in that branch you referenced. If you see anything in there that needs changing, send a patch for that.
I can send out the series again if needed.
AFAIK, the kernel patch is fine.
my comment was referring the liburing patch that is calling __sys_io_uring_register() by passing a nr_args value of 0.
This is problematic because on the kernel side, io_uring_register() returns -EINVAL if nr_args is not 1.
Jens Axboe
maintainerRe: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 53 lines · Olivier Langlois
Quoted 42 lines · Jens Axboe
Quoted 42 lines
On Feb 2, 2024, at 10:41 AM, Olivier Langlois
<[email protected]> wrote:Quoted 23 lines · Jens Axboe
Quoted 23 lines · Olivier Langlois
Quoted 13 lines · Stefan Roesch
(quoted diff) +2
-0+ +int io_uring_register_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +6
-0+{ +return __sys_io_uring_register (ring->ring_fd, +IORING_REGISTER_NAPI , napi,0 ); +} + +int io_uring_unregister_napi(struct io_uring *ring, struct
io_uring_napi*napi)src/register.c +3
-0+{ + return __sys_io_uring_register(ring->ring_fd, + IORING_UNREGISTER_NAPI, napi,Quoted 2 lines
0);
+}my apologies if this is not the latest version of the patch but I
think
that it is...Quoted 8 lines
Quoted 5 lines
nr_argsshould be 1 to match what__io_uring_register()in the current
corresponding kernel patch expects:https://git.kernel.dk/cgit/linux/commit/?h=io_uring-napi&id=787d81d3132aaf4eb4a4a5f24ff949e350e537d0
Can you send a patch I can fold in?
Jens,
I am unsure of what you are asking me.
You would like me to send a patch to fix a liburing patch that has never been applied AFAIK?
Quoted 9 lines
Quoted 3 lines
if the v9 patch has never been applied. Wouldn't just editing Stefan
patch by replacing thenr_argsparam from 0 to 1 directly do it?The current version is what is in that branch you referenced. If you see anything in there that needs changing, send a patch for that.
I can send out the series again if needed.
Quoted 7 lines
AFAIK, the kernel patch is fine.
my comment was referring the liburing patch that is calling
__sys_io_uring_register()by passing anr_argsvalue of 0.This is problematic because on the kernel side,
io_uring_register()returns-EINVALifnr_argsis not 1.
Ah gotcha, yeah that’s odd and could not ever have worked. I wonder how that was tested…
I’ll setup a liburing branch as well.
—
Jens Axboe
Re: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 6 lines · Jens Axboe
Ah gotcha, yeah that’s odd and could not ever have worked. I wonder how that was tested…
I’ll setup a liburing branch as well.
It is easy. You omit to check the function return value by telling to yourself that it cannot fail...
I caught my mistake on a second pass code review...
C++ has a very useful [[nodiscard]] attribute that can help to catch this simple error... I am not sure if there is something similar to the [[nodiscard]] in the ISO C standard...
Jens Axboe
maintainerRe: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 11 lines · Olivier Langlois
Quoted 6 lines · Jens Axboe
Ah gotcha, yeah that?s odd and could not ever have worked. I wonder how that was tested?
I?ll setup a liburing branch as well.
It is easy. You omit to check the function return value by telling to yourself that it cannot fail...
I caught my mistake on a second pass code review...
Oh I can see how that can happen, but then there should be no functional changes in terms of latency... Which means that it was never tested. The test results were from the original postings, so probably just fine. It's just that later versions would've failed. Looking at the example test case, it doesn't check the return value.
Quoted 3 lines
C++ has a very useful [[nodiscard]] attribute that can help to catch this simple error... I am not sure if there is something similar to the [[nodiscard]] in the ISO C standard...
You can use __attribute__((__warn_unused_result__)) - the kernel does that, for example.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerRe: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 11 lines · Jens Axboe
Quoted 10 lines · Olivier Langlois
Quoted 6 lines · Jens Axboe
Ah gotcha, yeah that?s odd and could not ever have worked. I wonder how that was tested?
I?ll setup a liburing branch as well.
It is easy. You omit to check the function return value by telling to yourself that it cannot fail...
Quoted 7 lines
Quoted 1 line
I caught my mistake on a second pass code review...
Oh I can see how that can happen, but then there should be no functional changes in terms of latency... Which means that it was never tested. The test results were from the original postings, so probably just fine. It's just that later versions would've failed. Looking at the example test case, it doesn't check the return value.
Setup a 'napi' branch with the patches, and some fixes on top. It's a start... I'll try the example ping test here, just need to get off a plane and be able to access test boxes.
Signature · 1 line
Jens Axboe
Jens Axboe
maintainerRe: [PATCH v9 1/4] liburing: add api to set napi busy poll settings
Quoted 20 lines · Jens Axboe
Quoted 11 lines · Jens Axboe
Quoted 10 lines · Olivier Langlois
Quoted 6 lines · Jens Axboe
Ah gotcha, yeah that?s odd and could not ever have worked. I wonder how that was tested?
I?ll setup a liburing branch as well.
It is easy. You omit to check the function return value by telling to yourself that it cannot fail...
Quoted 7 lines
Quoted 1 line
I caught my mistake on a second pass code review...
Oh I can see how that can happen, but then there should be no functional changes in terms of latency... Which means that it was never tested. The test results were from the original postings, so probably just fine. It's just that later versions would've failed. Looking at the example test case, it doesn't check the return value.
Quoted 3 lines
Setup a 'napi' branch with the patches, and some fixes on top. It's a start... I'll try the example ping test here, just need to get off a plane and be able to access test boxes.
Added ipv4 support to the napi example code, and here's what I get. Stock settings, no NAPI, 100k packets:
rtt(us) min/avg/max/mdev = 31.730/37.006/87.960/0.497
and with -t10 -b enabled:
rtt(us) min/avg/max/mdev = 23.250/29.795/63.511/1.203
So it seems to work fine as-is, with the current branch. I've added some tweaks on the liburing napi branch, but mostly just little fixes.
Signature · 1 line
Jens Axboe
PATCH v9 2/4 liburing: add documentation for new napi busy polling
In reply to Stefan RoeschThis adds two man pages for the two new functions:
- io_uring_register_nap
- io_uring_unregister_napi
--- man/io_uring_register_napi.3 | 40 ++++++++++++++++++++++++++++++++++ man/io_uring_unregister_napi.3 | 27 +++++++++++++++++++++++ 2 files changed, 67 insertions(+) create mode 100644 man/io_uring_register_napi.3 create mode 100644 man/io_uring_unregister_napi.3
man/io_uring_register_napi.3 new file +40 -0
@@ -0,0 +1,40 @@ +.\" Copyright (C) 2022 Stefan Roesch <[email protected]> +.\" +.\" SPDX-License-Identifier: LGPL-2.0-or-later +.\" +.TH io_uring_register_napi 3 "November 16, 2022" "liburing-2.4" "liburing Manual" +.SH NAME +io_uring_register_napi \- register NAPI busy poll settings +.SH SYNOPSIS +.nf +.B #include <liburing.h> +.PP +.BI "int io_uring_register_napi(struct io_uring *" ring "," +.BI " struct io_uring_napi *" napi) +.PP +.fi +.SH DESCRIPTION +.PP +The +.BR io_uring_register_napi (3) +function registers the NAPI settings for subsequent operations. The NAPI +settings are specified in the structure that is passed in the +.I napi +parameter. The structure consists of the napi timeout +.I busy_poll_to +(napi busy poll timeout in us) and +.IR prefer_busy_poll . + +Registering a NAPI settings sets the mode when calling the function +napi_busy_loop and corresponds to the SO_PREFER_BUSY_POLL socket +option. + +NAPI busy poll can reduce the network roundtrip time. + + +.SH RETURN VALUE +On success +.BR io_uring_register_napi (3) +return 0. On failure they return +.BR -errno . +It also updates the napi structure with the current values.
man/io_uring_unregister_napi.3 new file +27 -0
@@ -0,0 +1,27 @@ +.\" Copyright (C) 2022 Stefan Roesch <[email protected]> +.\" +.\" SPDX-License-Identifier: LGPL-2.0-or-later +.\" +.TH io_uring_unregister_napi 3 "November 16, 2022" "liburing-2.4" "liburing Manual" +.SH NAME +io_uring_unregister_napi \- unregister NAPI busy poll settings +.SH SYNOPSIS +.nf +.B #include <liburing.h> +.PP +.BI "int io_uring_unregister_napi(struct io_uring *" ring "," +.BI " struct io_uring_napi *" napi) +.PP +.fi +.SH DESCRIPTION +.PP +The +.BR io_uring_unregister_napi (3) +function unregisters the NAPI busy poll settings for subsequent operations. + +.SH RETURN VALUE +On success +.BR io_uring_unregister_napi (3) +return 0. On failure they return +.BR -errno . +It also updates the napi structure with the current values.
Signature · 1 line
2.39.1
PATCH v9 3/4 liburing: add example programs for napi busy poll
In reply to Stefan RoeschThis adds two example programs to test the napi busy poll functionality. It consists of a client program and a server program. To get a napi id, the client and the server program need to be run on different hosts.
To test the napi busy poll timeout, the -t needs to be specified. A reasonable value for the busy poll timeout is 100. By specifying the busy poll timeout on the server and the client the best results are accomplished.
--- .gitignore | 2 + examples/Makefile | 2 + examples/napi-busy-poll-client.c | 451 +++++++++++++++++++++++++++++++ examples/napi-busy-poll-server.c | 386 ++++++++++++++++++++++++++ 4 files changed, 841 insertions(+) create mode 100644 examples/napi-busy-poll-client.c create mode 100644 examples/napi-busy-poll-server.c
.gitignore +2 -0
@@ -19,6 +19,8 @@ /examples/io_uring-test /examples/io_uring-udp /examples/link-cp +/examples/napi-busy-poll-client +/examples/napi-busy-poll-server /examples/ucontext-cp /examples/poll-bench /examples/send-zerocopy
examples/Makefile +2 -0
@@ -19,6 +19,8 @@ example_srcs := \ io_uring-test.c \ io_uring-udp.c \ link-cp.c \ + napi-busy-poll-client.c \ + napi-busy-poll-server.c \ poll-bench.c \ send-zerocopy.c \ rsrc-update-bench.c
examples/napi-busy-poll-client.c new file +451 -0
451 lines not on this page · raw message
examples/napi-busy-poll-server.c new file +386 -0
386 lines not on this page · raw message
Signature · 1 line
2.39.1
PATCH v9 4/4 liburing: update changelog with new feature
In reply to Stefan RoeschAdd a new entry to the changelog file for the napi busy poll feature.
--- CHANGELOG | 1 + 1 file changed, 1 insertion(+)
CHANGELOG +1 -0
@@ -15,6 +15,7 @@ liburing-2.4 release io_uring_prep_socket_direct() factor in being called with IORING_FILE_INDEX_ALLOC for allocating a direct descriptor. - Add io_uring_prep_sendto() function. +- Support for napi busy polling liburing-2.3 release
Signature · 1 line
2.39.1