Skip to contentSkip to conversation

media: v4l2-mem2mem: always call poll_wait() on queues

PATCH Superseded Alexandre Courbot 2 participants 14 messages Oct 22, 2020 – Nov 23, 2020 Media (V4L/DVB)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 media: v4l2-mem2mem: always call poll_wait() on queues

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

---
I seem to be hitting all the polling corner cases recently! ^_^; This time I was wondering why epoll_wait() never returned after I received the first resolution change event from the vicodec stateful decoder. This is why - please take a look!

 drivers/media/v4l2-core/v4l2-mem2mem.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/media/v4l2-core/v4l2-mem2mem.c +5 -3
index b221b4e438a1..65476ef2879f 100644
@@ -887,9 +887,6 @@ static __poll_t v4l2_m2m_poll_for_data(struct file *file,
 	src_q = v4l2_m2m_get_src_vq(m2m_ctx);
 	dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
 
-	poll_wait(file, &src_q->done_wq, wait);
-	poll_wait(file, &dst_q->done_wq, wait);
-
 	/*
 	 * There has to be at least one buffer queued on each queued_list, which
 	 * means either in driver already or waiting for driver to claim it
@@ -922,9 +919,14 @@ __poll_t v4l2_m2m_poll(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
 		       struct poll_table_struct *wait)
 {
 	struct video_device *vfd = video_devdata(file);
+	struct vb2_queue *src_q = v4l2_m2m_get_src_vq(m2m_ctx);
+	struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
 	__poll_t req_events = poll_requested_events(wait);
 	__poll_t rc = 0;
 
+	poll_wait(file, &src_q->done_wq, wait);
+	poll_wait(file, &dst_q->done_wq, wait);
+
 	if (req_events & (EPOLLOUT | EPOLLWRNORM | EPOLLIN | EPOLLRDNORM))
 		rc = v4l2_m2m_poll_for_data(file, m2m_ctx, wait);
 
Signature · 1 line
2.29.0
#1 lore raw
Quoted 44 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

---
I seem to be hitting all the polling corner cases recently! ^_^; This time I was wondering why epoll_wait() never returned after I received the first resolution change event from the vicodec stateful decoder. This is why - please take a look!

 drivers/media/v4l2-core/v4l2-mem2mem.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/media/v4l2-core/v4l2-mem2mem.c +0 -3
index b221b4e438a1..65476ef2879f 100644
@@ -887,9 +887,6 @@ static __poll_t v4l2_m2m_poll_for_data(struct file *file,
        src_q = v4l2_m2m_get_src_vq(m2m_ctx);
        dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
 
-       poll_wait(file, &src_q->done_wq, wait);
-       poll_wait(file, &dst_q->done_wq, wait);
-
        /*
         * There has to be at least one buffer queued on each queued_list, which
         * means either in driver already or waiting for driver to claim it

drivers/media/v4l2-core/v4l2-mem2mem.c __poll_t v4l2_m2m_poll(struct file *file, struct v4l2_m2m_ctx *m2m_ctx, · line 928

Quoted 10 lines
drivers/media/v4l2-core/v4l2-mem2mem.c +4 -0
@@ -922,9 +919,14 @@ __poll_t v4l2_m2m_poll(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
                       struct poll_table_struct *wait)
 {
        struct video_device *vfd = video_devdata(file);
+       struct vb2_queue *src_q = v4l2_m2m_get_src_vq(m2m_ctx);
+       struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
        __poll_t req_events = poll_requested_events(wait);
        __poll_t rc = 0;
 
+       poll_wait(file, &src_q->done_wq, wait);
+       poll_wait(file, &dst_q->done_wq, wait);

This should probably include a comment to not move this back to v4l2_m2m_poll_for_data(). I'll add one and send a v2 unless someone points out that the premise of this patch is a bad idea to begin with.

Quoted 7 lines
drivers/media/v4l2-core/v4l2-mem2mem.c +1 -1
+
        if (req_events & (EPOLLOUT | EPOLLWRNORM | EPOLLIN | EPOLLRDNORM))
                rc = v4l2_m2m_poll_for_data(file, m2m_ctx, wait);
 
--

2.29.0

#2 lore raw
In reply to Alexandre Courbot
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

Regards,

Hans

Quoted 41 lines

---
I seem to be hitting all the polling corner cases recently! ^_^; This time I was wondering why epoll_wait() never returned after I received the first resolution change event from the vicodec stateful decoder. This is why - please take a look!

 drivers/media/v4l2-core/v4l2-mem2mem.c | 8 +++++---
 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/media/v4l2-core/v4l2-mem2mem.c +5 -3
index b221b4e438a1..65476ef2879f 100644
@@ -887,9 +887,6 @@ static __poll_t v4l2_m2m_poll_for_data(struct file *file,
 	src_q = v4l2_m2m_get_src_vq(m2m_ctx);
 	dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
 
-	poll_wait(file, &src_q->done_wq, wait);
-	poll_wait(file, &dst_q->done_wq, wait);
-
 	/*
 	 * There has to be at least one buffer queued on each queued_list, which
 	 * means either in driver already or waiting for driver to claim it
@@ -922,9 +919,14 @@ __poll_t v4l2_m2m_poll(struct file *file, struct v4l2_m2m_ctx *m2m_ctx,
 		       struct poll_table_struct *wait)
 {
 	struct video_device *vfd = video_devdata(file);
+	struct vb2_queue *src_q = v4l2_m2m_get_src_vq(m2m_ctx);
+	struct vb2_queue *dst_q = v4l2_m2m_get_dst_vq(m2m_ctx);
 	__poll_t req_events = poll_requested_events(wait);
 	__poll_t rc = 0;
 
+	poll_wait(file, &src_q->done_wq, wait);
+	poll_wait(file, &dst_q->done_wq, wait);
+
 	if (req_events & (EPOLLOUT | EPOLLWRNORM | EPOLLIN | EPOLLRDNORM))
 		rc = v4l2_m2m_poll_for_data(file, m2m_ctx, wait);
 
#3 lore raw

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

Cheers,
Alex.

#4 lore raw
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

The epoll_wait() will fail when this issue hits, so that's a good place to add comments explaining this problem.

There is one other place where this needs to be tested: testEvents() in v4l2-test-controls.cpp: currently this only tests select(), but there should be a second epoll test here as well that just tests EPOLLPRI.

This would catch drivers that do not stream (i.e. no EPOLLIN/OUT) but that do have controls (so support EPOLLPRI).

Congratulation, you found a really nasty corner case! :-)

Regards,

Hans

#5 lore raw
Quoted 57 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

Quoted 7 lines

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

(quoted diff) +0 -0

b/utils/v4l2-compliance/v4l2-test-buffers.cpp
index 8000db23..b63326cd 100644

utils/v4l2-compliance/v4l2-test-buffers.cpp +0 -0
@@ -903,6 +903,10 @@ static int captureBufs(struct node *node, struct

node *node_m2m_cap, const cv4l_

                epollfd = epoll_create1(0);

                fail_on_test(epollfd < 0);

+
+ ev.events = 0;
+ fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_ADD,
node->g_fd(), &ev));
+

                if (node->is_m2m)
                        ev.events = EPOLLIN | EPOLLOUT | EPOLLPRI;
                else if (v4l_type_is_output(q.g_type()))
@@ -910,7 +914,7 @@ static int captureBufs(struct node *node, struct
node *node_m2m_cap, const cv4l_
                else
                        ev.events = EPOLLIN;
                ev.data.fd = node->g_fd();
-               fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_ADD,
node->g_fd(), &ev));
+               fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_MOD,
node->g_fd(), &ev));
        }

        if (pollmode)
Quoted 10 lines

The epoll_wait() will fail when this issue hits, so that's a good place to add comments explaining this problem.

There is one other place where this needs to be tested: testEvents() in v4l2-test-controls.cpp: currently this only tests select(), but there should be a second epoll test here as well that just tests EPOLLPRI.

This would catch drivers that do not stream (i.e. no EPOLLIN/OUT) but that do have controls (so support EPOLLPRI).

I'll take a look there as well, and think about a proper comment before sending a patch towards you.

Cheers,
Alex.

#6 lore raw
Quoted 61 lines · Alexandre Courbot
Quoted 60 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Quoted 7 lines
Quoted 4 lines

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

Does it also fail with vivid? I am curious to know whether this issue is m2m specific or a more general problem.

Regards,

Hans

Quoted 49 lines
utils/v4l2-compliance/v4l2-test-buffers.cpp +0 -0
index 8000db23..b63326cd 100644
@@ -903,6 +903,10 @@ static int captureBufs(struct node *node, struct

node *node_m2m_cap, const cv4l_

                epollfd = epoll_create1(0);

                fail_on_test(epollfd < 0);
utils/v4l2-compliance/v4l2-test-buffers.cpp +3 -0
+
+               ev.events = 0;
+               fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_ADD,

node->g_fd(), &ev));

utils/v4l2-compliance/v4l2-test-buffers.cpp +1 -0
+
                if (node->is_m2m)
                        ev.events = EPOLLIN | EPOLLOUT | EPOLLPRI;
                else if (v4l_type_is_output(q.g_type()))
@@ -910,7 +914,7 @@ static int captureBufs(struct node *node, struct

node *node_m2m_cap, const cv4l_

                else
                        ev.events = EPOLLIN;
                ev.data.fd = node->g_fd();
-               fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_ADD,
node->g_fd(), &ev));
+               fail_on_test(epoll_ctl(epollfd, EPOLL_CTL_MOD,
node->g_fd(), &ev));
        }

        if (pollmode)
Quoted 10 lines

The epoll_wait() will fail when this issue hits, so that's a good place to add comments explaining this problem.

There is one other place where this needs to be tested: testEvents() in v4l2-test-controls.cpp: currently this only tests select(), but there should be a second epoll test here as well that just tests EPOLLPRI.

This would catch drivers that do not stream (i.e. no EPOLLIN/OUT) but that do have controls (so support EPOLLPRI).

I'll take a look there as well, and think about a proper comment before sending a patch towards you.

Cheers,
Alex.

#7 lore raw
Quoted 68 lines · Hans Verkuil
Quoted 66 lines · Alexandre Courbot
Quoted 64 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Quoted 5 lines
Quoted 2 lines

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

Does it also fail with vivid? I am curious to know whether this issue is m2m specific or a more general problem.

It does fail actually! And that made me notice that vb2_poll() uses the same pattern as v4l2_m2m_poll() (probably because the latter is inspired by the former?) and needs to be fixed similarly. I will send another patch to fix vb2_poll() as well, thanks for pointing it out!

#8 lore raw
Quoted 72 lines · Alexandre Courbot
Quoted 71 lines · Hans Verkuil
Quoted 68 lines · Alexandre Courbot
Quoted 57 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

Quoted 7 lines

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

Quoted 7 lines
Quoted 2 lines

Does it also fail with vivid? I am curious to know whether this issue is m2m specific or a more general problem.

It does fail actually! And that made me notice that vb2_poll() uses the same pattern as v4l2_m2m_poll() (probably because the latter is inspired by the former?) and needs to be fixed similarly. I will send another patch to fix vb2_poll() as well, thanks for pointing it out!

I was afraid of that.

Testing epoll for control events would be interesting as well. The vivid radio device is an example of a device that has controls, but does not do streaming (so is not using vb2).

But from what I can see v4l2_ctrl_poll() does the right thing, so this should be fine.

Regards,

Hans

#9 lore raw
Quoted 82 lines · Hans Verkuil
Quoted 72 lines · Alexandre Courbot
Quoted 71 lines · Hans Verkuil
Quoted 68 lines · Alexandre Courbot
Quoted 57 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

Quoted 7 lines

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

Quoted 7 lines
Quoted 2 lines

Does it also fail with vivid? I am curious to know whether this issue is m2m specific or a more general problem.

It does fail actually! And that made me notice that vb2_poll() uses the same pattern as v4l2_m2m_poll() (probably because the latter is inspired by the former?) and needs to be fixed similarly. I will send another patch to fix vb2_poll() as well, thanks for pointing it out!

Quoted 8 lines

I was afraid of that.

Testing epoll for control events would be interesting as well. The vivid radio device is an example of a device that has controls, but does not do streaming (so is not using vb2).

But from what I can see v4l2_ctrl_poll() does the right thing, so this should be fine.

Indeed, it unconditionally calls poll_wait() with all the wait queues that may wake us up (that is, only one), so there is no problem there.

#10 lore raw
Quoted 90 lines · Alexandre Courbot
Quoted 88 lines · Hans Verkuil
Quoted 72 lines · Alexandre Courbot
Quoted 71 lines · Hans Verkuil
Quoted 68 lines · Alexandre Courbot
Quoted 57 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

Quoted 7 lines

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

Quoted 7 lines
Quoted 2 lines

Does it also fail with vivid? I am curious to know whether this issue is m2m specific or a more general problem.

It does fail actually! And that made me notice that vb2_poll() uses the same pattern as v4l2_m2m_poll() (probably because the latter is inspired by the former?) and needs to be fixed similarly. I will send another patch to fix vb2_poll() as well, thanks for pointing it out!

I was afraid of that.

Testing epoll for control events would be interesting as well. The vivid radio device is an example of a device that has controls, but does not do streaming (so is not using vb2).

Quoted 5 lines
Quoted 2 lines

But from what I can see v4l2_ctrl_poll() does the right thing, so this should be fine.

Indeed, it unconditionally calls poll_wait() with all the wait queues that may wake us up (that is, only one), so there is no problem there.

Sorry, I noticed that this patch was marked with "Changes Requested" in patchwork, but isn't it valid as-is? We need a similar change to VB2, but that should go as a separate patch IMHO. I'm fine with doing both in one go if you prefer that though.

#11 lore raw
Quoted 94 lines · Alexandre Courbot
Quoted 93 lines · Alexandre Courbot
Quoted 90 lines · Hans Verkuil
Quoted 72 lines · Alexandre Courbot
Quoted 71 lines · Hans Verkuil
Quoted 68 lines · Alexandre Courbot
Quoted 57 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

Quoted 7 lines

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

Quoted 7 lines
Quoted 2 lines

Does it also fail with vivid? I am curious to know whether this issue is m2m specific or a more general problem.

It does fail actually! And that made me notice that vb2_poll() uses the same pattern as v4l2_m2m_poll() (probably because the latter is inspired by the former?) and needs to be fixed similarly. I will send another patch to fix vb2_poll() as well, thanks for pointing it out!

I was afraid of that.

Testing epoll for control events would be interesting as well. The vivid radio device is an example of a device that has controls, but does not do streaming (so is not using vb2).

But from what I can see v4l2_ctrl_poll() does the right thing, so this should be fine.

Quoted 8 lines
Quoted 2 lines

Indeed, it unconditionally calls poll_wait() with all the wait queues that may wake us up (that is, only one), so there is no problem there.

Sorry, I noticed that this patch was marked with "Changes Requested" in patchwork, but isn't it valid as-is? We need a similar change to VB2, but that should go as a separate patch IMHO. I'm fine with doing both in one go if you prefer that though.

In at least one reply you mentioned that you wanted to add a comment (reply from 23 Oct). That's why I changed it to 'Changes Requested'.

Also, I prefer to fix both m2m and vb2 at the same time (separate patches, but part of the same patch series). And together with a separate patch improving v4l2-compliance.

Regards,

Hans

#12 lore raw
Quoted 99 lines · Hans Verkuil
Quoted 97 lines · Alexandre Courbot
Quoted 95 lines · Alexandre Courbot
Quoted 82 lines · Hans Verkuil
Quoted 72 lines · Alexandre Courbot
Quoted 71 lines · Hans Verkuil
Quoted 68 lines · Alexandre Courbot
Quoted 57 lines · Hans Verkuil
Quoted 46 lines · Alexandre Courbot

Hi Hans,

Quoted 20 lines · Hans Verkuil
Quoted 15 lines · Alexandre Courbot

do_poll()/do_select() seem to set the _qproc member of poll_table to NULL the first time they are called on a given table, making subsequent calls of poll_wait() on that table no-ops. This is a problem for mem2mem which calls poll_wait() on the V4L2 queues' waitqueues only when a queue-related event is requested, which may not necessarily be the case during the first poll.

For instance, a stateful decoder is typically only interested in EPOLLPRI events when it starts, and will switch to listening to both EPOLLPRI and EPOLLIN after receiving the initial resolution change event and configuring the CAPTURE queue. However by the time that switch happens and v4l2_m2m_poll_for_data() is called for the first time, poll_wait() has become a no-op and the V4L2 queues waitqueues thus cannot be registered.

Quoted 2 lines

Fix this by moving the registration to v4l2_m2m_poll() and do it whether or not one of the queue-related events are requested.

Quoted 6 lines

This looks good, but would it be possible to add a test for this to v4l2-compliance? (Look for POLL_MODE_EPOLL in v4l2-test-buffers.cpp)

If I understand this right, calling EPOLL_CTL_ADD for EPOLLPRI, then calling EPOLL_CTL_ADD for EPOLLIN/OUT would trigger this? Or does there have to be an epoll_wait call in between?

Even without an epoll_wait() in between the behavior is visible. v4l2_m2m_poll() will be called once during the initial EPOLL_CTL_ADD and this will trigger the bug.

Quoted 5 lines

Another reason for adding this test is that I wonder if regular capture or output V4L2 devices don't have the same issue.

It's a very subtle bug and so adding a test for this to v4l2-compliance would be very useful.

I fully agree, this is very counter-intuitive since what basically happens is that the kernel's poll_wait() function becomes a no-op after the poll() hook of a driver is called for the first time. There is no way one can expect this behavior just from browsing the code so this is likely to affect other drivers.

Quoted 5 lines

As for the test itself, we can easily reproduce the conditions for failure in v4l2-test-buffers.cpp's captureBufs() function, but doing so will make the streaming tests fail without being specific about the cause. Or maybe we should add another pollmode to specifically test epoll in this setup? Can I get your thoughts?

No, just keep it as part of the poll test. Just add comments at the place where it fails describing this error.

Quoted 7 lines

After all, it *is* a poll() bug, so it is only fair that it is tested as part of the epoll test.

Can you call EPOLL_CTL_ADD with ev.events set to 0? And then call it again with the actual value that you need? If that triggers this issue as well, then that is a nice test (but perhaps EPOLL_CTL_ADD won't call poll() if ev.events is 0, but perhaps EPOLLERR would work instead of 0).

Yup, actually the following is enough to make v4l2-compliance -s fail with vicodec:

Quoted 7 lines
Quoted 2 lines

Does it also fail with vivid? I am curious to know whether this issue is m2m specific or a more general problem.

It does fail actually! And that made me notice that vb2_poll() uses the same pattern as v4l2_m2m_poll() (probably because the latter is inspired by the former?) and needs to be fixed similarly. I will send another patch to fix vb2_poll() as well, thanks for pointing it out!

Quoted 8 lines

I was afraid of that.

Testing epoll for control events would be interesting as well. The vivid radio device is an example of a device that has controls, but does not do streaming (so is not using vb2).

But from what I can see v4l2_ctrl_poll() does the right thing, so this should be fine.

Indeed, it unconditionally calls poll_wait() with all the wait queues that may wake us up (that is, only one), so there is no problem there.

Quoted 8 lines
Quoted 5 lines

Sorry, I noticed that this patch was marked with "Changes Requested" in patchwork, but isn't it valid as-is? We need a similar change to VB2, but that should go as a separate patch IMHO. I'm fine with doing both in one go if you prefer that though.

In at least one reply you mentioned that you wanted to add a comment (reply from 23 Oct). That's why I changed it to 'Changes Requested'.

Ah, that's correct. Thanks for clarifying.

Quoted 3 lines

Also, I prefer to fix both m2m and vb2 at the same time (separate patches, but part of the same patch series). And together with a separate patch improving v4l2-compliance.

Ack - will submit a series and a separate patch for v4l-utils.

#13 lore raw
In reply to Hans Verkuil
Quoted 6 lines · Hans Verkuil

There is one other place where this needs to be tested: testEvents() in v4l2-test-controls.cpp: currently this only tests select(), but there should be a second epoll test here as well that just tests EPOLLPRI.

This would catch drivers that do not stream (i.e. no EPOLLIN/OUT) but that do have controls (so support EPOLLPRI).

IIUC this part should not require fixing - EPOLLPRI is behaving properly, the bug is only with EPOLLIN/EPOLLOUT.