liburing: add api for napi busy poll

This adds two new api's to set/clear the napi busy poll settings. The two new functions are called:
- io_uring_register_napi
- io_uring_unregister_napi

The patch series also contains the documentation for the two new functions and two example programs. The client program is called napi-busy-poll-client and the server program napi-busy-poll-server. The client measures the roundtrip times of requests.

There is also a kernel patch "io-uring: support napi busy poll" to enable this feature on the kernel side.

Changes:
- V6:
- Check return value of unregister napi call and verify that busy poll timeout and prefer busy poll return the expected values.
- V5:
- Fixes to documentation.
- Correct opcode for unregister call
- Initialize napi structure in example programs
- Address tab issues in recordRTT()
- V4:
- Modify functions to use a structure to pass the napi busy poll settings to the kernel.
- Return previous values when returning from the above functions.
- Rename the functions and remove one function (no longer needed as the data is passed as a structure)
- V3:
- Updated liburing.map file
- Moved example programs from the test directory to the example directory. The two example programs don't fit well in the test category and need to be run from separate hosts.
- Added the io_uring_register_napi_prefer_busy_poll API.
- Added the call to io_uring_register_napi_prefer_busy_poll to the example programs
- Updated the documentation
- V2:
- Updated the liburing.map file for the two new functions. (added a 2.4 section)
- Added a description of the new feature to the changelog file
- Fixed the indentation of the longopts structure
- Used defined exit constants
- Fixed encodeUserData to support 32 bit builds

Stefan Roesch (4):
liburing: add api to set napi busy poll settings
liburing: add documentation for new napi busy polling
liburing: add example programs for napi busy poll
liburing: update changelog with new feature

 .gitignore                       |   2 +
 CHANGELOG                        |   1 +
 examples/Makefile                |   2 +
 examples/napi-busy-poll-client.c | 451 +++++++++++++++++++++++++++++++
 examples/napi-busy-poll-server.c | 386 ++++++++++++++++++++++++++
 man/io_uring_register_napi.3     |  40 +++
 man/io_uring_unregister_napi.3   |  27 ++
 src/include/liburing.h           |   3 +
 src/include/liburing/io_uring.h  |  12 +
 src/liburing.map                 |   3 +
 src/register.c                   |  12 +
 11 files changed, 939 insertions(+)
 create mode 100644 examples/napi-busy-poll-client.c
 create mode 100644 examples/napi-busy-poll-server.c
 create mode 100644 man/io_uring_register_napi.3
 create mode 100644 man/io_uring_unregister_napi.3

base-commit: d32d53b65377d846635387ce6c1cd2ed0d700f92

2.30.2

This adds two functions to manage the napi busy poll settings:
- io_uring_register_napi
- io_uring_unregister_napi

---
 src/include/liburing.h          |  3 +++
 src/include/liburing/io_uring.h | 12 ++++++++++++
 src/liburing.map                |  3 +++
 src/register.c                  | 12 ++++++++++++
 4 files changed, 30 insertions(+)

index 1f91983..13546c4 100644
@@ -240,6 +240,9 @@ int io_uring_register_sync_cancel(struct io_uring *ring,
 int io_uring_register_file_alloc_range(struct io_uring *ring,
 					unsigned off, unsigned len);
 
+int io_uring_register_napi(struct io_uring *ring, struct io_uring_napi *napi);
+int io_uring_unregister_napi(struct io_uring *ring, struct io_uring_napi *napi);
+
 int io_uring_get_events(struct io_uring *ring);
 int io_uring_submit_and_get_events(struct io_uring *ring);
 

index 636a4c2..ab6c86e 100644
@@ -518,6 +518,10 @@ enum {
 	/* register a range of fixed file slots for automatic slot allocation */
 	IORING_REGISTER_FILE_ALLOC_RANGE	= 25,
 
+	/* set/clear busy poll settings */
+	IORING_REGISTER_NAPI			= 26,
+	IORING_UNREGISTER_NAPI			= 27,
+
 	/* this goes last */
 	IORING_REGISTER_LAST
 };

@@ -640,6 +644,14 @@ struct io_uring_buf_reg {
 	__u64	resv[3];
 };
 
+/* argument for IORING_(UN)REGISTER_NAPI */
+struct io_uring_napi {
+	__u32   busy_poll_to;
+	__u8    prefer_busy_poll;
+	__u8    pad[3];
+	__u64   resv;
+};
+
 /*
  * io_uring_restriction->opcode values
  */

index 1dbe765..013dc4a 100644
@@ -76,4 +76,7 @@ LIBURING_2.4 {
 
 		io_uring_enable_rings;
 		io_uring_register_restrictions;
+
+		io_uring_register_napi;
+		io_uring_unregister_napi;
 } LIBURING_2.3;

index ac4c9e3..966839b 100644
@@ -342,3 +342,15 @@ int io_uring_register_file_alloc_range(struct io_uring *ring,
 				       IORING_REGISTER_FILE_ALLOC_RANGE, &range,
 				       0);
 }
+
+int io_uring_register_napi(struct io_uring *ring, struct io_uring_napi *napi)
+{
+	return __sys_io_uring_register(ring->ring_fd,
+				IORING_REGISTER_NAPI, napi, 0);
+}
+
+int io_uring_unregister_napi(struct io_uring *ring, struct io_uring_napi *napi)
+{
+	return __sys_io_uring_register(ring->ring_fd,
+				IORING_UNREGISTER_NAPI, napi, 0);
+}

2.30.2

We have a new rule. Since commit:

9e2890d35e9677d8cfc7ac66cdb2d97c48a0b5a2 ("build: add liburing-ffi")

Adding a new exported function should also be reflected in liburing-ffi.map.

Ammar Faizi

I'll add it to the next version.

This adds two man pages for the two new functions:
- io_uring_register_nap
- io_uring_unregister_napi

---
 man/io_uring_register_napi.3   | 40 ++++++++++++++++++++++++++++++++++
 man/io_uring_unregister_napi.3 | 27 +++++++++++++++++++++++
 2 files changed, 67 insertions(+)
 create mode 100644 man/io_uring_register_napi.3
 create mode 100644 man/io_uring_unregister_napi.3

new file mode 100644
index 0000000..78eaa71
@@ -0,0 +1,40 @@
+.\" Copyright (C) 2022 Stefan Roesch <[email protected]>
+.\"
+.\" SPDX-License-Identifier: LGPL-2.0-or-later
+.\"
+.TH io_uring_register_napi 3 "November 16, 2022" "liburing-2.4" "liburing Manual"
+.SH NAME
+io_uring_register_napi \- register NAPI busy poll settings
+.SH SYNOPSIS
+.nf
+.B #include <liburing.h>
+.PP
+.BI "int io_uring_register_napi(struct io_uring *" ring ","
+.BI "                           struct io_uring_napi *" napi)
+.PP
+.fi
+.SH DESCRIPTION
+.PP
+The
+.BR io_uring_register_napi (3)
+function registers the NAPI settings for subsequent operations. The NAPI
+settings are specified in the structure that is passed in the
+.I napi
+parameter. The structure consists of the napi timeout
+.I busy_poll_to
+(napi busy poll timeout in us) and
+.I prefer_busy_poll.
+
+Registering a NAPI settings sets the mode when calling the function
+napi_busy_loop and corresponds to the SO_PREFER_BUSY_POLL socket
+option.
+
+NAPI busy poll can reduce the network roundtrip time.
+
+
+.SH RETURN VALUE
+On success
+.BR io_uring_register_napi (3)
+return 0. On failure they return
+.BR -errno .
+It also updates the napi structure with the current values.

new file mode 100644
index 0000000..f7087ef
@@ -0,0 +1,27 @@
+.\" Copyright (C) 2022 Stefan Roesch <[email protected]>
+.\"
+.\" SPDX-License-Identifier: LGPL-2.0-or-later
+.\"
+.TH io_uring_unregister_napi 3 "November 16, 2022" "liburing-2.4" "liburing Manual"
+.SH NAME
+io_uring_unregister_napi \- unregister NAPI busy poll settings
+.SH SYNOPSIS
+.nf
+.B #include <liburing.h>
+.PP
+.BI "int io_uring_unregister_napi(struct io_uring *" ring ","
+.BI "                             struct io_uring_napi *" napi)
+.PP
+.fi
+.SH DESCRIPTION
+.PP
+The
+.BR io_uring_unregister_napi (3)
+function unregisters the NAPI busy poll settings for subsequent operations.
+
+.SH RETURN VALUE
+On success
+.BR io_uring_unregister_napi (3)
+return 0. On failure they return
+.BR -errno .
+It also updates the napi structure with the current values.

2.30.2

This adds two example programs to test the napi busy poll functionality. It consists of a client program and a server program. To get a napi id, the client and the server program need to be run on different hosts.

To test the napi busy poll timeout, the -t needs to be specified. A reasonable value for the busy poll timeout is 100. By specifying the busy poll timeout on the server and the client the best results are accomplished.

---
 .gitignore                       |   2 +
 examples/Makefile                |   2 +
 examples/napi-busy-poll-client.c | 451 +++++++++++++++++++++++++++++++
 examples/napi-busy-poll-server.c | 386 ++++++++++++++++++++++++++
 4 files changed, 841 insertions(+)
 create mode 100644 examples/napi-busy-poll-client.c
 create mode 100644 examples/napi-busy-poll-server.c

index 0cd84dc..edccea9 100644
@@ -18,6 +18,8 @@
 /examples/io_uring-test
 /examples/io_uring-udp
 /examples/link-cp
+/examples/napi-busy-poll-client
+/examples/napi-busy-poll-server
 /examples/ucontext-cp
 /examples/poll-bench
 /examples/send-zerocopy

index e561e05..59f1260 100644
@@ -15,6 +15,8 @@ example_srcs := \
 	io_uring-test.c \
 	io_uring-udp.c \
 	link-cp.c \
+	napi-busy-poll-client.c \
+	napi-busy-poll-server.c \
 	poll-bench.c \
 	send-zerocopy.c
 

2.30.2

Those two break liburing's upstream CI. Also, it has many whitespace issues:

Applying: liburing: add example programs for napi busy poll .git/rebase-apply/patch:258: space before tab in indent.

        	avgRTT += ctx->rtt[i];
.git/rebase-apply/patch:382: space before tab in indent.
        	fprintf(stderr, "inet_pton error for %s\n", optarg);
.git/rebase-apply/patch:391: space before tab in indent.
        	fprintf(stderr, "socket() failed: (%d) %s\n", errno, strerror(errno));
.git/rebase-apply/patch:392: space before tab in indent.
        	exit(1);
.git/rebase-apply/patch:794: space before tab in indent.
        	fprintf(stderr, "inet_pton error for %s\n", optarg);

warning: squelched 2 whitespace errors
warning: 7 lines add whitespace errors.

-----------------------------------------------------------

napi-busy-poll-client.c:65:15: error: no previous extern declaration for non-static variable 'longopts' [-Werror,-Wmissing-variable-declarations] struct option longopts[] =

              ^

napi-busy-poll-client.c:65:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit struct option longopts[] =
^
napi-busy-poll-client.c:435:28: error: comparison of integers of different signs: 'int' and '__u32' (aka 'unsigned int') [-Werror,-Wsign-compare]

                if (opt.timeout          != napi.busy_poll_to ||
                    ~~~~~~~~~~~          ^  ~~~~~~~~~~~~~~~~~

napi-busy-poll-client.c:50:3: error: no previous extern declaration for non-static variable 'ctx' [-Werror,-Wmissing-variable-declarations] } ctx;
^
napi-busy-poll-client.c:33:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit struct ctx
^
napi-busy-poll-client.c:63:3: error: no previous extern declaration for non-static variable 'options' [-Werror,-Wmissing-variable-declarations] } options;
^
napi-busy-poll-client.c:52:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit struct options
^
4 errors generated.
make[1]: *** [Makefile:38: napi-busy-poll-client] Error 1
make[1]: *** Waiting for unfinished jobs....
napi-busy-poll-server.c:64:15: error: no previous extern declaration for non-static variable 'longopts' [-Werror,-Wmissing-variable-declarations] struct option longopts[] =

              ^

napi-busy-poll-server.c:64:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit struct option longopts[] =
^
napi-busy-poll-server.c:110:32: error: a function declaration without a prototype is deprecated in all versions of C [-Werror,-Wstrict-prototypes] static void setProcessScheduler()

                               ^
                                void

napi-busy-poll-server.c:48:3: error: no previous extern declaration for non-static variable 'ctx' [-Werror,-Wmissing-variable-declarations] } ctx;
^
napi-busy-poll-server.c:32:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit struct ctx
^
napi-busy-poll-server.c:62:3: error: no previous extern declaration for non-static variable 'options' [-Werror,-Wmissing-variable-declarations] } options;
^
napi-busy-poll-server.c:50:1: note: declare 'static' if the variable is not intended to be used outside of this translation unit struct options
^
4 errors generated.
make[1]: *** [Makefile:38: napi-busy-poll-server] Error 1
make[1]: Leaving directory '/home/runner/work/liburing/liburing/examples'
make: *** [Makefile:12: all] Error 2
Error: Process completed with exit code 2.

Ammar Faizi

Do you happen to know which compiler and what settings are used in the CI environment? I don't see these warnings in my local environment.

GCC and Clang, both are not happy. Here is the build result of your patch series:

https://github.com/ammarfaizi2/liburing/actions/runs/4087640126/jobs/7048391772

liburing's upstream CI config file can be found here:

https://github.com/axboe/liburing/blob/master/.github/workflows/build.yml

Ammar Faizi

Add a new entry to the changelog file for the napi busy poll feature.

---
 CHANGELOG | 1 +
 1 file changed, 1 insertion(+)

index 0722aae..7810137 100644
@@ -6,6 +6,7 @@ liburing-2.4 release
 - Add io_uring_prep_msg_ring_cqe_flags() function.
 - Deprecate --nolibc configure option.
 - CONFIG_NOLIBC is always enabled on x86-64, x86, and aarch64.
+- Support for napi busy polling
 
 liburing-2.3 release
 

2.30.2