VIDIOC_QBUF and VIDIOC_STREAMON order
Hello,
As far as I know V4L2 applications are allowed to call VIDIOC_STREAMON before queuing buffers with VIDIOC_QBUF.
This leads to situation that a H/W is attempted to be enabled by the driver when it has no any buffer ownership.
Effectively actual activation of the data pipeline has to be deferred until first buffer arrived in the driver. Which makes it difficult to signal any errors to user during enabling the data pipeline.
Is this allowed to force applications to queue some buffers before calling STREAMON, by returning an error in vidioc_streamon from the driver, when no buffers have been queued at this time?
I suppose this could render some applications to stop working if this kind of restriction is applied e.g. in camera capture driver.
What the applications really expect?
With the above I refer mostly to a snapshot mode where we have to be careful not to lose any frame, as there could be only one..
Please share you opinions.
Regards,
Signature · 2 lines
Sylwester Nawrocki Samsung Poland R&D Center
VIDIOC_STREAMON expects buffers to be queued before hardware part of image/video pipe is enabled. From my experience of V4L2 user space, I have always QBUFfed before invoking the STREAMON. Below is the API specification which also speaks something same:
http://linuxtv.org/downloads/v4l-dvb-apis/vidioc-streamon.html
I think its better to return EINVAL if there are no queued buffers when VIDIOC_STREAMON is invoked.
Regards,
Subash
Quoted 37 lines · Sylwester Nawrocki
Hello,
As far as I know V4L2 applications are allowed to call
VIDIOC_STREAMONbefore queuing buffers withVIDIOC_QBUF.This leads to situation that a H/W is attempted to be enabled by the driver when it has no any buffer ownership.
Effectively actual activation of the data pipeline has to be deferred until first buffer arrived in the driver. Which makes it difficult to signal any errors to user during enabling the data pipeline.
Is this allowed to force applications to queue some buffers before calling STREAMON, by returning an error in
vidioc_streamonfrom the driver, when no buffers have been queued at this time?I suppose this could render some applications to stop working if this kind of restriction is applied e.g. in camera capture driver.
What the applications really expect?
With the above I refer mostly to a snapshot mode where we have to be careful not to lose any frame, as there could be only one..
Please share you opinions.
Regards,
--
Sylwester Nawrocki
Samsung Poland R&D Center
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Hi,
Quoted 5 lines · Subash Patel
VIDIOC_STREAMONexpects buffers to be queued before hardware part of image/video pipe is enabled. From my experience of V4L2 user space, I have always QBUFfed before invoking the STREAMON. Below is the API specification which also speaks something same:
Not exactly. It says that the API requires buffers to be queued for output devices. It does not require any buffers to be queued for input devices. Sylwester is right here.
This feature of not having to queue input buffers before STREAMON introduces problems to driver implementations and I am personally not a big fan of it either. But I'm seeing some additional problems here. Suppose we forced QBUF to be done before STREAMON. This would work, but what happens next? What should happen when we want to DQBUF the last buffer? If the device couldn't start without any buffers queued, can it continue streaming with all of them dequeued? I would guess not. So we'd either have to deny DQBUF of the last buffer (which for me personally is almost unacceptable) or have the last DQBUF automatically cause a STREAMOFF. So, for the latter, should applications, after they get all the data they wanted, be made to always have one more buffer queued as a "throwaway" buffer? This is probably the only reasonable solution here, but the applications would have to keep count of their queued buffers and be aware of this. Also, there might still be situations where being able to STREAMON without buffers queued would be beneficial. For example, enabling the device might be a slow/expensive operation and we might prefer to keep it running even if we don't want any data at the moment. Even for faster devices, being able to keep them on and periodically take a snapshot would be faster without having to call STREAMON anyway...
Signature · 2 lines
Best regards, Pawel Osciak
Quoted 6 lines
Also, there might still be situations where being able to STREAMON without buffers queued would be beneficial. For example, enabling the device might be a slow/expensive operation and we might prefer to keep it running even if we don't want any data at the moment. Even for faster devices, being able to keep them on and periodically take a snapshot would be faster without having to call STREAMON anyway...
If we are speaking of devices like camera for phone/tabs, then this will have significant impact on the power consumption.
Quoted 7 lines
Suppose we forced QBUF to be done before STREAMON. This would work, but what happens next? What should happen when we want to DQBUF the last buffer? If the device couldn't start without any buffers queued, can it continue streaming with all of them dequeued? I would guess not. So we'd either have to deny DQBUF of the last buffer (which for me personally is almost unacceptable) or have the last DQBUF automatically cause a STREAMOFF.
I think it depends on hardware's behavior on how it behaves without any buffers queued. Some hardwares do notify the overflow state (interrupting for each frame recieved) if there was no buffer queued. This will ensure even the last frame is DQUEUEd.
Considering the scenario like preview/capture @ 30fps, if there is one frame loss too, it is acceptable. But that doesnt hold good for high-speed image capture.
Regards,
Subash
Quoted 38 lines · Pawel Osciak
Hi,
Quoted 5 lines · Subash Patel
VIDIOC_STREAMONexpects buffers to be queued before hardware part of image/video pipe is enabled. From my experience of V4L2 user space, I have always QBUFfed before invoking the STREAMON. Below is the API specification which also speaks something same:Not exactly. It says that the API requires buffers to be queued for output devices. It does not require any buffers to be queued for input devices. Sylwester is right here.
This feature of not having to queue input buffers before STREAMON introduces problems to driver implementations and I am personally not a big fan of it either. But I'm seeing some additional problems here. Suppose we forced QBUF to be done before STREAMON. This would work, but what happens next? What should happen when we want to DQBUF the last buffer? If the device couldn't start without any buffers queued, can it continue streaming with all of them dequeued? I would guess not. So we'd either have to deny DQBUF of the last buffer (which for me personally is almost unacceptable) or have the last DQBUF automatically cause a STREAMOFF. So, for the latter, should applications, after they get all the data they wanted, be made to always have one more buffer queued as a "throwaway" buffer? This is probably the only reasonable solution here, but the applications would have to keep count of their queued buffers and be aware of this. Also, there might still be situations where being able to STREAMON without buffers queued would be beneficial. For example, enabling the device might be a slow/expensive operation and we might prefer to keep it running even if we don't want any data at the moment. Even for faster devices, being able to keep them on and periodically take a snapshot would be faster without having to call STREAMON anyway...
--
Best regards,
Pawel Osciak
Quoted 25 lines · On Tuesday, March 15, 2011 04:21:05 Pawel Osciak wrote:
Hi,
Quoted 5 lines · Subash Patel
VIDIOC_STREAMONexpects buffers to be queued before hardware part of image/video pipe is enabled. From my experience of V4L2 user space, I have always QBUFfed before invoking the STREAMON. Below is the API specification which also speaks something same:Not exactly. It says that the API requires buffers to be queued for output devices. It does not require any buffers to be queued for input devices. Sylwester is right here.
This feature of not having to queue input buffers before STREAMON introduces problems to driver implementations and I am personally not a big fan of it either. But I'm seeing some additional problems here. Suppose we forced QBUF to be done before STREAMON. This would work, but what happens next? What should happen when we want to DQBUF the last buffer? If the device couldn't start without any buffers queued, can it continue streaming with all of them dequeued? I would guess not. So we'd either have to deny DQBUF of the last buffer (which for me personally is almost unacceptable) or have the last DQBUF automatically cause a STREAMOFF. So, for the latter, should applications, after they get all the data they wanted, be made to always have one more buffer queued as a "throwaway" buffer? This is
Quoted 8 lines
probably the only reasonable solution here, but the applications would have to keep count of their queued buffers and be aware of this. Also, there might still be situations where being able to STREAMON without buffers queued would be beneficial. For example, enabling the device might be a slow/expensive operation and we might prefer to keep it running even if we don't want any data at the moment. Even for faster devices, being able to keep them on and periodically take a snapshot would be faster without having to call STREAMON anyway...
The problem is that what is possible is highly hardware dependent. All video capture device I know of (composite in, HDMI in, etc) require that buffers are queued and they won't release that buffer to userspace until a new free buffer is available. They DMA continuously and stopping the DMA at the last buffer and restarting it when a new one appears tends to be too expensive and leads to additional loss of frames.
In part how this should act depends on the use-case: if you are streaming video, then requiring buffers to be present before STREAMON and holding on to a buffer if userspace can't keep up seems quite reasonable to me.
But for snapshot and codec type streams this behavior doesn't make sense. The main difference is that in this case the DMA is not driven by an external input, but by internal (userspace) demand.
Something for our meeting to discuss.
Regards,
Hans
Quoted 2 lines
Signature · 1 line
Hans Verkuil - video4linux developer - sponsored by Cisco
Hi Hans,
Quoted 32 lines · On Tuesday 15 March 2011 08:50:45 Hans Verkuil wrote:
Quoted 22 lines · On Tuesday, March 15, 2011 04:21:05 Pawel Osciak wrote:
Quoted 5 lines · Subash Patel
VIDIOC_STREAMONexpects buffers to be queued before hardware part of image/video pipe is enabled. From my experience of V4L2 user space, I have always QBUFfed before invoking the STREAMON. Below is the APIspecification which also speaks something same:
Not exactly. It says that the API requires buffers to be queued for output devices. It does not require any buffers to be queued for input devices. Sylwester is right here.
This feature of not having to queue input buffers before STREAMON introduces problems to driver implementations and I am personally not a big fan of it either. But I'm seeing some additional problems here. Suppose we forced QBUF to be done before STREAMON. This would work, but what happens next? What should happen when we want to DQBUF the last buffer? If the device couldn't start without any buffers queued, can it continue streaming with all of them dequeued? I would guess not. So we'd either have to deny DQBUF of the last buffer (which for me personally is almost unacceptable) or have the last DQBUF automatically cause a STREAMOFF. So, for the latter, should applications, after they get all the data they wanted, be made to always have one more buffer queued as a "throwaway" buffer? This is
Quoted 8 lines
probably the only reasonable solution here, but the applications would have to keep count of their queued buffers and be aware of this. Also, there might still be situations where being able to STREAMON without buffers queued would be beneficial. For example, enabling the device might be a slow/expensive operation and we might prefer to keep it running even if we don't want any data at the moment. Even for faster devices, being able to keep them on and periodically take a snapshot would be faster without having to call STREAMON anyway...
Quoted 4 lines
The problem is that what is possible is highly hardware dependent. All video capture device I know of (composite in, HDMI in, etc) require that buffers are queued and they won't release that buffer to userspace until a new free buffer is available.
That's funny, all video capture devices I know of behave the opposite way :-) They either pause the stream when they run out of buffers and resume it when a new buffer gets queued, or they throw away the data when intermediate buffers are used (such as with USB devices).
Quoted 13 lines
They DMA continuously and stopping the DMA
at the last buffer and restarting it when a new one appears tends to be too expensive and leads to additional loss of frames.In part how this should act depends on the use-case: if you are streaming video, then requiring buffers to be present before STREAMON and holding on to a buffer if userspace can't keep up seems quite reasonable to me.
But for snapshot and codec type streams this behavior doesn't make sense. The main difference is that in this case the DMA is not driven by an external input, but by internal (userspace) demand.
Something for our meeting to discuss.
Signature · 3 lines
Regards, Laurent Pinchart
Hi all,
Quoted 3 lines · Laurent Pinchart
Quoted 2 lines · On Tuesday 15 March 2011 08:50:45 Hans Verkuil wrote:
Quoted 1 line · On Tuesday, March 15, 2011 04:21:05 Pawel Osciak wrote:
On Mon, Mar 14, 2011 at 03:49, Subash Patel <[email protected]>
Quoted 5 lines · wrote:
Quoted 5 lines
Quoted 5 lines
Quoted 4 lines
VIDIOC_STREAMONexpects buffers to be queued before hardware part of image/video pipe is enabled. From my experience of V4L2 user space, I have always QBUFfed before invoking the STREAMON. Below is the API
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 1 line
specification which also speaks something same:
Not exactly. It says that the API requires buffers to be queued for output devices. It does not require any buffers to be queued for input devices. Sylwester is right here.
This feature of not having to queue input buffers before STREAMON introduces problems to driver implementations and I am personally not a big fan of it either. But I'm seeing some additional problems
here.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Suppose we forced QBUF to be done before STREAMON. This would work, but what happens next? What should happen when we want to DQBUF the last buffer? If the device couldn't start without any buffers queued, can it continue streaming with all of them dequeued? I would guess not. So we'd either have to deny DQBUF of the last buffer (which for me personally is almost unacceptable) or have the last DQBUF automatically cause a STREAMOFF. So, for the latter, should applications, after they get all the data they wanted, be made to always have one more buffer queued as a "throwaway" buffer? This is
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
probably the only reasonable solution here, but the applications would have to keep count of their queued buffers and be aware of this. Also, there might still be situations where being able to STREAMON without buffers queued would be beneficial. For example, enabling the device might be a slow/expensive operation and we might prefer to keep it running even if we don't want any data at the moment. Even for faster devices, being able to keep them on and periodically take a snapshot would be faster without having to call STREAMON
anyway...
Quoted 7 lines
Quoted 5 lines
The problem is that what is possible is highly hardware dependent. All video capture device I know of (composite in, HDMI in, etc) require that buffers are queued and they won't release that buffer to userspace until a new free buffer is available.
That's funny, all video capture devices I know of behave the opposite way
:-)
Quoted 1 line
They either pause the stream when they run out of buffers and resume it
when
Quoted 3 lines
a new buffer gets queued, or they throw away the data when intermediate buffers are used (such as with USB devices).
Laurent,
Exynos capture device is the same with your example.
It should pause the stream not to overwrite
when they run out of buffers and resume it when a new buffer gets queued.
Hans,
Do you mean that the buffer is overwritten without pause and resume until a new free buffer is coming ?
Quoted 10 lines
Quoted 7 lines
They DMA continuously and stopping the DMA at the last buffer and restarting it when a new one appears tends to be too expensive and leads to additional loss of frames.
In part how this should act depends on the use-case: if you are streaming video, then requiring buffers to be present before STREAMON and holding on to a buffer if userspace can't keep up seems quite
reasonable to me.
Quoted 2 lines
But for snapshot and codec type streams this behavior doesn't make
sense.
Quoted 11 lines
Quoted 4 lines
The main difference is that in this case the DMA is not driven by an external input, but by internal (userspace) demand.
Something for our meeting to discuss.
--
Regards,Laurent Pinchart
--
To unsubscribe from this list: send the line "unsubscribe linux-media" in
the
Quoted 2 lines
body of a message to [email protected] More majordomo info at
http://vger.kernel.org/majordomo-info.html
Best regards,
Jonghun Han
A message that is not in the archive (sent off-list or not yet received): [email protected]
Hi Subash,
thanks for your comments.
Quoted 7 lines · Subash Patel
VIDIOC_STREAMONexpects buffers to be queued before hardware part of image/video pipe is enabled. From my experience of V4L2 user space, I have always QBUFfed before invoking the STREAMON. Below is the API specification which also speaks something same:http://linuxtv.org/downloads/v4l-dvb-apis/vidioc-streamon.html
It seems pretty reasonable thing to do in the application, i.e. queue buffer(s) first. If the device needs buffers to work what STREAMON before QBUF would be supposed to mean? The application must be aware then that real start of data flow in the hardware pipeline is being done at the first QBUF. This is not obvious at all from the current specification.
Quoted 2 lines
I think its better to return EINVAL if there are no queued buffers when
VIDIOC_STREAMONis invoked.
Unfortunately, as Pawel pointed out the are are some further logic implications of such approach. Anyway we can handle both QBUF, STREAMON orders in the kernel, it just makes life a bit harder.
Adhering exactly to current videobuf2 rules it is not possible to return EINVAL if there is no buffers QBUFed when invoking STREAMON. And we are going to need to be able to return an error in buf_queue which could be then propagated up to the STREAMON caller. This has already been raised in "[RFC/PATCH 1/2] v4l: videobuf2: Handle buf_queue errors" from Laurent.
Regards,
Sylwester
Quoted 33 lines
Regards,
SubashOn Mon, Mar 14, 2011 at 3:44 PM, Sylwester Nawrocki <[email protected] <mailto:[email protected]>> wrote:
Hello,
As far as I know V4L2 applications are allowed to call
VIDIOC_STREAMONbefore queuing buffers withVIDIOC_QBUF.This leads to situation that a H/W is attempted to be enabled by the driver when it has no any buffer ownership.
Effectively actual activation of the data pipeline has to be deferred until first buffer arrived in the driver. Which makes it difficult to signal any errors to user during enabling the data pipeline.
Is this allowed to force applications to queue some buffers before calling STREAMON, by returning an error in
vidioc_streamonfrom the driver, when no buffers have been queued at this time?I suppose this could render some applications to stop working if this kind of restriction is applied e.g. in camera capture driver.
What the applications really expect?
With the above I refer mostly to a snapshot mode where we have to be careful not to lose any frame, as there could be only one..
Please share you opinions.