Skip to contentSkip to conversation

pipe: only enable the extra wake_up(rd_wait) when epoll is actually used

PATCH In review Oleg Nesterov 4 participants 12 messages Jul 23 – Jul 29

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover pipe: only enable the extra wake_up(rd_wait) when epoll is actually used

Breno, Christian, Mateusz,

To be honest, I am not sure this change makes a lot of sense. But somehow the extra wakeup in anon_pipe_write() looks annoying to me.

Can you take a look?

Plus I'd like to know what sashiko.dev thinks.

I have no idea how to test it wrt performance...

Oleg.
---

 fs/pipe.c                 | 23 +++++++++++++++++++----
 include/linux/pipe_fs_i.h |  6 ++++--
 2 files changed, 23 insertions(+), 6 deletions(-)
#1 lore raw

PATCH 1/1 pipe: only enable the extra wake_up(rd_wait) when epoll is actually used

pipe_poll() unconditionally sets poll_usage on the first call, forcing anon_pipe_write() to wake up readers on every write even if the pipe was not empty. But this is only needed for epoll's "nasty semantics"; poll() and select() users pay for it for no reason.

Rename it to epoll_usage, and only set it when the caller is actually using epoll on the read side of the pipe.

---
 fs/pipe.c                 | 23 +++++++++++++++++++----
 include/linux/pipe_fs_i.h |  6 ++++--
 2 files changed, 23 insertions(+), 6 deletions(-)
fs/pipe.c +19 -4
index 32140cb00d7e..b45b4b8d0b0b 100644
@@ -357,6 +357,23 @@ static inline unsigned int pipe_update_tail(struct pipe_inode_info *pipe,
 	return tail;
 }
 
+static void pipe_set_epoll_usage(struct file *filp, struct pipe_inode_info *pipe)
+{
+#ifdef CONFIG_EPOLL
+	if ((filp->f_mode & FMODE_READ) && filp->f_ep &&
+	    unlikely(!READ_ONCE(pipe->epoll_usage)))
+		WRITE_ONCE(pipe->epoll_usage, true);
+#endif
+}
+
+static bool pipe_get_epoll_usage(struct pipe_inode_info *pipe)
+{
+#ifdef CONFIG_EPOLL
+	return pipe->epoll_usage;
+#endif
+	return false;
+}
+
 static ssize_t
 anon_pipe_read(struct kiocb *iocb, struct iov_iter *to)
 {
@@ -689,7 +706,7 @@ anon_pipe_write(struct kiocb *iocb, struct iov_iter *from)
 	 * Epoll nonsensically wants a wakeup whether the pipe
 	 * was already empty or not.
 	 */
-	if (was_empty || pipe->poll_usage)
+	if (was_empty || pipe_get_epoll_usage(pipe))
 		wake_up_interruptible_sync_poll(&pipe->rd_wait, EPOLLIN | EPOLLRDNORM);
 	kill_fasync(&pipe->fasync_readers, SIGIO, POLL_IN);
 	if (wake_next_writer)
@@ -761,9 +778,7 @@ pipe_poll(struct file *filp, poll_table *wait)
 	union pipe_index idx;
 
 	/* Epoll has some historical nasty semantics, this enables them */
-	if (unlikely(!READ_ONCE(pipe->poll_usage)))
-		WRITE_ONCE(pipe->poll_usage, true);
-
+	pipe_set_epoll_usage(filp, pipe);
 	/*
 	 * Reading pipe state only -- no need for acquiring the semaphore.
 	 *
include/linux/pipe_fs_i.h +4 -2
index 7f6a92ac9704..d06da9bad3a8 100644
@@ -74,7 +74,7 @@ union pipe_index {
  *	@files: number of struct file referring this pipe (protected by ->i_lock)
  *	@r_counter: reader counter
  *	@w_counter: writer counter
- *	@poll_usage: is this pipe used for epoll, which has crazy wakeups?
+ *	@epoll_usage: is this pipe used for epoll, which has crazy wakeups?
  *	@fasync_readers: reader side fasync
  *	@fasync_writers: writer side fasync
  *	@bufs: the circular array of pipe buffers
@@ -95,7 +95,9 @@ struct pipe_inode_info {
 	unsigned int files;
 	unsigned int r_counter;
 	unsigned int w_counter;
-	bool poll_usage;
+#ifdef CONFIG_EPOLL
+	bool epoll_usage;
+#endif
 #ifdef CONFIG_WATCH_QUEUE
 	bool note_loss;
 #endif
Signature · 1 line
2.52.0
#2 lore raw
Quoted 9 lines · Oleg Nesterov

pipe_poll() unconditionally sets poll_usage on the first call, forcing anon_pipe_write() to wake up readers on every write even if the pipe was not empty. But this is only needed for epoll's "nasty semantics"; poll() and select() users pay for it for no reason.

Rename it to epoll_usage, and only set it when the caller is actually using epoll on the read side of the pipe.

I am worried in that the poll_usage thing showed up in 2021 in commit 3b844826b6c6affa ("pipe: avoid unnecessary EPOLLET wakeups under normal loads"), fixing up "missing" wakeups after some rework. Even then it went ahead and as you noted it also did it for poll/select users. So by Hyrum's law there is plenty of potential for someone to depend on it.

However, I think this makes sense to try out and there is history of committing possibly breaking changes, so I think the idea is fine to go in.

I does rub me the wrong way that epoll lands in the same handler though, but that's not something I'm going to do anything about.

The routine pretends to operate in a lockless manner, possibly returning only a partial result. But it guarantees not blocking when it should not by queueing up the caller unconditionally, which later has to be undone. Have you considered patching that up? (as in, do the work. if there are no events, queue up and do the work again -- should reduce work if there was stuff already there)

Quoted 18 lines
---
 fs/pipe.c                 | 23 +++++++++++++++++++----
 include/linux/pipe_fs_i.h |  6 ++++--
 2 files changed, 23 insertions(+), 6 deletions(-)
fs/pipe.c +4 -0
index 32140cb00d7e..b45b4b8d0b0b 100644
@@ -357,6 +357,23 @@ static inline unsigned int pipe_update_tail(struct pipe_inode_info *pipe,
        return tail;
 }
 
+static void pipe_set_epoll_usage(struct file *filp, struct pipe_inode_info *pipe)
+{
+#ifdef CONFIG_EPOLL
+       if ((filp->f_mode & FMODE_READ) && filp->f_ep &&

fs/pipe.c static inline unsigned int pipe_update_tail(struct pipe_inode_info *pipe, · line 373

Quoted 10 lines
fs/pipe.c +10 -0
+           unlikely(!READ_ONCE(pipe->epoll_usage)))
+               WRITE_ONCE(pipe->epoll_usage, true);
+#endif
+}
+
+static bool pipe_get_epoll_usage(struct pipe_inode_info *pipe)
+{
+#ifdef CONFIG_EPOLL
+       return pipe->epoll_usage;
+#endif

If patching this up I think a comment explaining what's going on would be nice.

I think you could steal parts of 3a34b13a88caeb2800ab44a4918f230041b37dd9 for that purpose.

Quoted 16 lines
fs/pipe.c +4 -1
+       return false;
+}
+
 static ssize_t
 anon_pipe_read(struct kiocb *iocb, struct iov_iter *to)
 {
@@ -689,7 +706,7 @@ anon_pipe_write(struct kiocb *iocb, struct iov_iter *from)
         * Epoll nonsensically wants a wakeup whether the pipe
         * was already empty or not.
         */
-       if (was_empty || pipe->poll_usage)
+       if (was_empty || pipe_get_epoll_usage(pipe))
                wake_up_interruptible_sync_poll(&pipe->rd_wait, EPOLLIN | EPOLLRDNORM);
        kill_fasync(&pipe->fasync_readers, SIGIO, POLL_IN);
        if (wake_next_writer)

fs/pipe.c pipe_poll(struct file *filp, poll_table *wait) · line 784

Quoted 10 lines
fs/pipe.c +1 -3
@@ -761,9 +778,7 @@ pipe_poll(struct file *filp, poll_table *wait)
        union pipe_index idx;
 
        /* Epoll has some historical nasty semantics, this enables them */
-       if (unlikely(!READ_ONCE(pipe->poll_usage)))
-               WRITE_ONCE(pipe->poll_usage, true);
-
+       pipe_set_epoll_usage(filp, pipe);
        /*
         * Reading pipe state only -- no need for acquiring the semaphore.
         *

nit: I would add a new line between this and the following comment

#3 lore raw
In reply to Oleg Nesterov

OK, sashiko has some concerns

https://sashiko.dev/#/patchset/amIqmbbZx3NlzUsX%40redhat.com

As for READ_ONCE(), I agree, I just forgot to add them.

But so far I don't understand the note about io_uring... I'll try to check later.

Anyway, I'd like to know your opinion.

Quoted 18 lines · Oleg Nesterov

Breno, Christian, Mateusz,

To be honest, I am not sure this change makes a lot of sense. But somehow the extra wakeup in anon_pipe_write() looks annoying to me.

Can you take a look?

Plus I'd like to know what sashiko.dev thinks.

I have no idea how to test it wrt performance...

Oleg.
---

 fs/pipe.c                 | 23 +++++++++++++++++++----
 include/linux/pipe_fs_i.h |  6 ++++--
 2 files changed, 23 insertions(+), 6 deletions(-)
#4 lore raw
Quoted 4 lines · Oleg Nesterov

OK, sashiko has some concerns

https://sashiko.dev/#/patchset/amIqmbbZx3NlzUsX%40redhat.com

Let me quote:

Does skipping this wakeup for non-epoll consumers break io_uring?

Applications polling pipes via io_uring do not attach an eventpoll context, so pipe->epoll_usage will be false. If a writer writes to an empty pipe, io_uring receives the wakeup.

If the writer then writes a second chunk before the first is drained, anon_pipe_write() observes was_empty == false and pipe_get_epoll_usage() == false, skipping the waitqueue wakeup.

Could this cause io_uring to miss events and hang permanently, waiting for a CQE that will never be emitted for the new data?

and I am starting to think sashiko is right (damn as always ;) and this patch does affect/break io_uring.

Jens, could you confirm? If yes, we need to update the comments in pipe.c (I've attached 1/1 at the end, so that you can see what this patch does)

I know nothing about io_uring and io_uring/poll.c is not trivial to say at least ;) please check my understanding.

It seems that IORING_OP_POLL_ADD / IORING_POLL_ADD_MULTI is edge-triggered by default! Like EPOLL_CTL_ADD / EPOLLET.

This means that io_uring depends on the "nasty semantics" too, io_poll_wake() path should add the task work which calls io_req_post_cqe() every time the new data arrives, even if the pipe was not empty.

Strange...

And. io_poll_parse_events() doesn't set EPOLLET if IORING_POLL_ADD_LEVEL, but how is it possible to use IORING_POLL_ADD_LEVEL? io_poll_add_prep() only allows IORING_POLL_ADD_MULTI in flags? OK, I don't understand this code anyway...

Oleg.
---

Subject: [PATCH] pipe: only enable the extra wake_up(rd_wait) when epoll is actually used

pipe_poll() unconditionally sets poll_usage on the first call, forcing anon_pipe_write() to wake up readers on every write even if the pipe was not empty. But this is only needed for epoll's "nasty semantics"; poll() and select() users pay for it for no reason.

Rename it to epoll_usage, and only set it when the caller is actually using epoll on the read side of the pipe.

---
 fs/pipe.c                 | 23 +++++++++++++++++++----
 include/linux/pipe_fs_i.h |  6 ++++--
 2 files changed, 23 insertions(+), 6 deletions(-)
fs/pipe.c +19 -4
index 32140cb00d7e..b45b4b8d0b0b 100644
@@ -357,6 +357,23 @@ static inline unsigned int pipe_update_tail(struct pipe_inode_info *pipe,
 	return tail;
 }
 
+static void pipe_set_epoll_usage(struct file *filp, struct pipe_inode_info *pipe)
+{
+#ifdef CONFIG_EPOLL
+	if ((filp->f_mode & FMODE_READ) && filp->f_ep &&
+	    unlikely(!READ_ONCE(pipe->epoll_usage)))
+		WRITE_ONCE(pipe->epoll_usage, true);
+#endif
+}
+
+static bool pipe_get_epoll_usage(struct pipe_inode_info *pipe)
+{
+#ifdef CONFIG_EPOLL
+	return pipe->epoll_usage;
+#endif
+	return false;
+}
+
 static ssize_t
 anon_pipe_read(struct kiocb *iocb, struct iov_iter *to)
 {
@@ -689,7 +706,7 @@ anon_pipe_write(struct kiocb *iocb, struct iov_iter *from)
 	 * Epoll nonsensically wants a wakeup whether the pipe
 	 * was already empty or not.
 	 */
-	if (was_empty || pipe->poll_usage)
+	if (was_empty || pipe_get_epoll_usage(pipe))
 		wake_up_interruptible_sync_poll(&pipe->rd_wait, EPOLLIN | EPOLLRDNORM);
 	kill_fasync(&pipe->fasync_readers, SIGIO, POLL_IN);
 	if (wake_next_writer)
@@ -761,9 +778,7 @@ pipe_poll(struct file *filp, poll_table *wait)
 	union pipe_index idx;
 
 	/* Epoll has some historical nasty semantics, this enables them */
-	if (unlikely(!READ_ONCE(pipe->poll_usage)))
-		WRITE_ONCE(pipe->poll_usage, true);
-
+	pipe_set_epoll_usage(filp, pipe);
 	/*
 	 * Reading pipe state only -- no need for acquiring the semaphore.
 	 *
include/linux/pipe_fs_i.h +4 -2
index 7f6a92ac9704..d06da9bad3a8 100644
@@ -74,7 +74,7 @@ union pipe_index {
  *	@files: number of struct file referring this pipe (protected by ->i_lock)
  *	@r_counter: reader counter
  *	@w_counter: writer counter
- *	@poll_usage: is this pipe used for epoll, which has crazy wakeups?
+ *	@epoll_usage: is this pipe used for epoll, which has crazy wakeups?
  *	@fasync_readers: reader side fasync
  *	@fasync_writers: writer side fasync
  *	@bufs: the circular array of pipe buffers
@@ -95,7 +95,9 @@ struct pipe_inode_info {
 	unsigned int files;
 	unsigned int r_counter;
 	unsigned int w_counter;
-	bool poll_usage;
+#ifdef CONFIG_EPOLL
+	bool epoll_usage;
+#endif
 #ifdef CONFIG_WATCH_QUEUE
 	bool note_loss;
 #endif
Signature · 1 line
2.52.0
#5 lore raw
Quoted 31 lines · Oleg Nesterov
Quoted 4 lines · Oleg Nesterov

OK, sashiko has some concerns

https://sashiko.dev/#/patchset/amIqmbbZx3NlzUsX%40redhat.com

Let me quote:

Does skipping this wakeup for non-epoll consumers break io_uring?

Applications polling pipes via io_uring do not attach an eventpoll context, so pipe->epoll_usage will be false. If a writer writes to an empty pipe, io_uring receives the wakeup.

If the writer then writes a second chunk before the first is drained, anon_pipe_write() observes was_empty == false and pipe_get_epoll_usage() == false, skipping the waitqueue wakeup.

Could this cause io_uring to miss events and hang permanently, waiting for a CQE that will never be emitted for the new data?

and I am starting to think sashiko is right (damn as always ;) and this patch does affect/break io_uring.

Jens, could you confirm? If yes, we need to update the comments in pipe.c (I've attached 1/1 at the end, so that you can see what this patch does)

I know nothing about io_uring and io_uring/poll.c is not trivial to say at least ;) please check my understanding.

Quoted 7 lines

It seems that IORING_OP_POLL_ADD / IORING_POLL_ADD_MULTI is edge-triggered by default! Like EPOLL_CTL_ADD / EPOLLET.

This means that io_uring depends on the "nasty semantics" too, io_poll_wake() path should add the task work which calls io_req_post_cqe() every time the new data arrives, even if the pipe was not empty.

well io_uring could set the flag as well in that case? it does not inherently mean random select and poll consumers definitely need this.

#6 lore raw
Quoted 10 lines · Mateusz Guzik
Quoted 5 lines · Oleg Nesterov

This means that io_uring depends on the "nasty semantics" too, io_poll_wake() path should add the task work which calls io_req_post_cqe() every time the new data arrives, even if the pipe was not empty.

well io_uring could set the flag as well in that case? it does not inherently mean random select and poll consumers definitely need this.

How? I don't think io_uring can do this. It doesn't even know if the file is pipe or not. Only file_operations->poll() does know...

__Perhaps__ pipe_poll() could check something like wait->_key & EPOLLET to detect both epoll/io_uring... Not sure, probably not, I'll try to check later.

Oleg.

#7 lore raw
Quoted 9 lines · Oleg Nesterov
Quoted 8 lines · Mateusz Guzik
Quoted 5 lines · Oleg Nesterov

This means that io_uring depends on the "nasty semantics" too, io_poll_wake() path should add the task work which calls io_req_post_cqe() every time the new data arrives, even if the pipe was not empty.

Quoted 8 lines
Quoted 2 lines

well io_uring could set the flag as well in that case? it does not inherently mean random select and poll consumers definitely need this.

How? I don't think io_uring can do this. It doesn't even know if the file is pipe or not. Only file_operations->poll() does know...

__Perhaps__ pipe_poll() could check something like wait->_key & EPOLLET to detect both epoll/io_uring... Not sure, probably not, I'll try to check later.

I got the impression that pipe_poll() sees EPOLLET in wait->_key for io_uring the same as for EPOLLET epoll. Since d59bd748db0a ("io_uring/poll: disable level triggered poll").

io_poll_add_prep(), which is called before teh proper iouring command, rejects IORING_POLL_ADD_LEVEL, so io_poll_parse_events() always ORs in EPOLLET for IORING_OP_POLL_ADD.

The internal apoll path sets EPOLLET unconditionally too (io_arm_apoll()), so every io_uring poll arming carries it.

Maybe this would work on the set side?

        if ((filp->f_mode & FMODE_READ) && wait &&
            (wait->_key & EPOLLET) &&
            unlikely(!READ_ONCE(pipe->poll_usage)))
                WRITE_ONCE(pipe->poll_usage, true);

--breno

#8 lore raw
Quoted 14 lines · Breno Leitao
Quoted 3 lines · Oleg Nesterov

__Perhaps__ pipe_poll() could check something like wait->_key & EPOLLET to detect both epoll/io_uring... Not sure, probably not, I'll try to check later.

I got the impression that pipe_poll() sees EPOLLET in wait->_key for io_uring the same as for EPOLLET epoll. Since d59bd748db0a ("io_uring/poll: disable level triggered poll").

io_poll_add_prep(), which is called before teh proper iouring command, rejects IORING_POLL_ADD_LEVEL, so io_poll_parse_events() always ORs in EPOLLET for IORING_OP_POLL_ADD.

Quoted 7 lines

The internal apoll path sets EPOLLET unconditionally too (io_arm_apoll()), so every io_uring poll arming carries it.

Maybe this would work on the set side?

        if ((filp->f_mode & FMODE_READ) && wait &&
            (wait->_key & EPOLLET) &&

This is exactly what I meant and what I did. I am writing the changelog, will send the patch soon today. Both test-cases I wrote pass, same for tools/testing/selftests/filesystems/epoll/epoll_wakeup_test.c

Oleg.

#9 lore raw
In reply to Oleg Nesterov
Quoted 27 lines · Oleg Nesterov
Quoted 4 lines · Oleg Nesterov

OK, sashiko has some concerns

https://sashiko.dev/#/patchset/amIqmbbZx3NlzUsX%40redhat.com

Let me quote:

Does skipping this wakeup for non-epoll consumers break io_uring?

Applications polling pipes via io_uring do not attach an eventpoll context, so pipe->epoll_usage will be false. If a writer writes to an empty pipe, io_uring receives the wakeup.

If the writer then writes a second chunk before the first is drained, anon_pipe_write() observes was_empty == false and pipe_get_epoll_usage() == false, skipping the waitqueue wakeup.

Could this cause io_uring to miss events and hang permanently, waiting for a CQE that will never be emitted for the new data?

and I am starting to think sashiko is right (damn as always ;) and this patch does affect/break io_uring.

Jens, could you confirm? If yes, we need to update the comments in pipe.c (I've attached 1/1 at the end, so that you can see what this patch does)

...

Quoted 2 lines

It seems that IORING_OP_POLL_ADD / IORING_POLL_ADD_MULTI is edge-triggered by default! Like EPOLL_CTL_ADD / EPOLLET.

Yes, sashiko is right. With some help from AI I wrote the simple test-case

	#include <unistd.h>
	#include <sys/mman.h>
	#include <sys/epoll.h>
	#include <sys/syscall.h>
	#include <linux/io_uring.h>
	#include <assert.h>

	int main(void)
	{
		struct io_uring_params p = {};
		int fd, pfd[2];

		pipe(pfd);

		fd = syscall(SYS_io_uring_setup, 2, &p);
		assert(fd >= 0);

		void *ring = mmap(0, p.cq_off.cqes + p.cq_entries * sizeof(struct io_uring_cqe),
				  PROT_READ | PROT_WRITE, MAP_SHARED, fd, IORING_OFF_SQ_RING);
		assert(ring != MAP_FAILED);
		*(unsigned *)(ring + p.sq_off.array) = 0;
		*(unsigned *)(ring + p.sq_off.tail) = 1;

		struct io_uring_sqe *sqes = mmap(0, p.sq_entries * sizeof(*sqes),
				  PROT_READ | PROT_WRITE, MAP_SHARED, fd, IORING_OFF_SQES);
		assert(sqes != MAP_FAILED);
		sqes[0].opcode = IORING_OP_POLL_ADD;
		sqes[0].fd = pfd[0];
		sqes[0].len = IORING_POLL_ADD_MULTI;
		sqes[0].poll32_events = EPOLLIN;

		syscall(SYS_io_uring_enter, fd, 1, 0, 0, 0, 0);

		unsigned *cq_head = ring + p.cq_off.head;
		unsigned *cq_tail = ring + p.cq_off.tail;
		for (int i = 0; i < 2; ++i) {
			write(pfd[1], "", 1);
			syscall(SYS_io_uring_enter, fd, 0, 0, IORING_ENTER_GETEVENTS, 0, 0);
			assert(*cq_tail == ++*cq_head);
		}

		return 0;
	}

with this patch the 2nd assert(*cq_tail == ++*cq_head) fails.

And just for the record, another one for epoll

	#include <unistd.h>
	#include <sys/epoll.h>
	#include <assert.h>

	int main(void)
	{
		int pfd[2], efd;
		struct epoll_event evt = { .events = EPOLLIN | EPOLLET };

		pipe(pfd);
		efd = epoll_create1(0);
		epoll_ctl(efd, EPOLL_CTL_ADD, pfd[0], &evt);

		for (int i = 0; i < 2; ++i) {
			write(pfd[1], "", 1);
			assert(epoll_wait(efd, &evt, 1, 0) == 1);
		}

		return 0;
	}

I'll try to make V2 later.

Quoted 3 lines

And. io_poll_parse_events() doesn't set EPOLLET if IORING_POLL_ADD_LEVEL, but how is it possible to use IORING_POLL_ADD_LEVEL? io_poll_add_prep() only allows IORING_POLL_ADD_MULTI in flags? OK, I don't understand this code anyway...

OK, I see: d59bd748db0a9 ("io_uring/poll: disable level triggered poll")

Oleg.

#10 lore raw
Quoted 26 lines · Oleg Nesterov
Quoted 25 lines · Oleg Nesterov
Quoted 4 lines · Oleg Nesterov

OK, sashiko has some concerns

https://sashiko.dev/#/patchset/amIqmbbZx3NlzUsX%40redhat.com

Let me quote:

Does skipping this wakeup for non-epoll consumers break io_uring?

Applications polling pipes via io_uring do not attach an eventpoll context, so pipe->epoll_usage will be false. If a writer writes to an empty pipe, io_uring receives the wakeup.

If the writer then writes a second chunk before the first is drained, anon_pipe_write() observes was_empty == false and pipe_get_epoll_usage() == false, skipping the waitqueue wakeup.

Could this cause io_uring to miss events and hang permanently, waiting for a CQE that will never be emitted for the new data?

and I am starting to think sashiko is right (damn as always ;) and this patch does affect/break io_uring.

Quoted 7 lines
Quoted 2 lines

Jens, could you confirm? If yes, we need to update the comments in pipe.c (I've attached 1/1 at the end, so that you can see what this patch does)

...

Quoted 2 lines

It seems that IORING_OP_POLL_ADD / IORING_POLL_ADD_MULTI is edge-triggered by default! Like EPOLL_CTL_ADD / EPOLLET.

Yes, sashiko is right. With some help from AI I wrote the simple test-case

https://github.com/axboe/liburing/blob/master/man/io_uring_enter.2#L400

Seems like it's even documented, which is a shame because I deem IORING_OP_POLL_ADD + IORING_POLL_ADD_MULTI to be a half broken API that should be deprecated after investigating lxc problems with it some years back. Makes me wonder, likely it wouldn't even break anything if shifts from strict edge definition for pipes. It's a stream, and the user doesn't generally know the number of bytes sitting there.

Signature · 1 line
Pavel Begunkov
#11 lore raw
Quoted 12 lines · Pavel Begunkov
Quoted 9 lines · Oleg Nesterov
Quoted 3 lines · Oleg Nesterov

Jens, could you confirm? If yes, we need to update the comments in pipe.c (I've attached 1/1 at the end, so that you can see what this patch does)

...

Quoted 2 lines

It seems that IORING_OP_POLL_ADD / IORING_POLL_ADD_MULTI is edge-triggered by default! Like EPOLL_CTL_ADD / EPOLLET.

Yes, sashiko is right. With some help from AI I wrote the simple test-case

Quoted 8 lines

https://github.com/axboe/liburing/blob/master/man/io_uring_enter.2#L400

Seems like it's even documented, which is a shame because I deem IORING_OP_POLL_ADD + IORING_POLL_ADD_MULTI to be a half broken API that should be deprecated after investigating lxc problems with it some years back. Makes me wonder, likely it wouldn't even break anything if shifts from strict edge definition for pipes. It's a stream, and the user doesn't generally know the number of bytes sitting there.

Can't comment, as I said I know nothing about io_uring ;)

Can you look at

[RFC PATCH v2 1/1] pipe: only enable the extra wake_up(rd_wait) for edge-triggered consumers
https://lore.kernel.org/all/[email protected]/

?

Not that I think this patch can improve performance, but to me the "wait->_key & EPOLLET" check documents the purpose of ->poll_usage.

However, from v2 0/1:

	With or without this patch we need to update the comments to document
	that io_uring depends on ->pipe_usage too. And probably rename it to
	(say) ->et_poll.

Oleg.