uvcvideo: extend UVC_QUIRK_FIX_BANDWIDTH to MJPEG streams
Subsystems (2)
PATCH 1/1 uvcvideo: extend UVC_QUIRK_FIX_BANDWIDTH to MJPEG streams
From: Pavel Rojtberg <[email protected]>
attaching two Logitech C615 webcams currently results in
VIDIOC_STREAMON: No space left on device
as the required bandwidth is not estimated correctly by the device. In fact it always requests 3060 bytes - no matter the format or resolution.
setting UVC_QUIRK_FIX_BANDWIDTH does not help either as it is only implemented for uncompressed streams.
This patch extends UVC_QUIRK_FIX_BANDWIDTH to MJPEG streams by making a (conservative) assumption of 4bpp for MJPEG streams.
As the actual compression ration is often closer to 1bpp this can be overridden
via the new mjpeg_bpp parameter.
Based on:
https://www.mail-archive.com/[email protected]/msg05724.html
--- drivers/media/usb/uvc/uvc_driver.c | 14 +++++++++++++- drivers/media/usb/uvc/uvc_video.c | 3 ++- 2 files changed, 15 insertions(+), 2 deletions(-)
drivers/media/usb/uvc/uvc_driver.c +13 -1
@@ -37,6 +37,7 @@ unsigned int uvc_no_drop_param;static unsigned int uvc_quirks_param =-1 ;unsigned int uvc_trace_param;unsigned int uvc_timeout_param =UVC_CTRL_STREAMING_TIMEOUT ; +static unsigned int uvc_mjpeg_bpp_param;/* ------------------------------------------------------------------------ * Video formats
@@ -463,7 +464,16 @@ static int uvc_parse_format(struct uvc_device *dev,strlcpy (format->name,"MJPEG" ,sizeof format->name); format->fcc =V4L2_PIX_FMT_MJPEG ; format->flags =UVC_FMT_FLAG_COMPRESSED ; - format->bpp =0 ; +if ((uvc_mjpeg_bpp_param >=1 ) && (uvc_mjpeg_bpp_param <=16 )) { + format->bpp = uvc_mjpeg_bpp_param; + }else { +/* conservative estimate. Actual values are around 1bpp. +* see e.g. +* https://developers.google.com/speed/webp/docs/webp_study +*/ + format->bpp =4 ; + } + ftype =UVC_VS_FRAME_MJPEG ;break ;
@@ -2274,6 +2284,8 @@ module_param_named(trace, uvc_trace_param, uint, S_IRUGO|S_IWUSR);MODULE_PARM_DESC (trace,"Trace level bitmask" );module_param_named (timeout, uvc_timeout_param, uint,S_IRUGO |S_IWUSR );MODULE_PARM_DESC (timeout,"Streaming control requests timeout" ); +module_param_named (mjpeg_bpp, uvc_mjpeg_bpp_param, uint,S_IRUGO |S_IWUSR ); +MODULE_PARM_DESC (mjpeg_bpp,"MJPEG bits per pixel for bandwidth quirk" );/* ------------------------------------------------------------------------ * Driver initialization and cleanup
drivers/media/usb/uvc/uvc_video.c +2 -1
@@ -127,7 +127,8 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming *stream,if ((ctrl->dwMaxPayloadTransferSize &0xffff0000 ) ==0xffff0000 ) ctrl->dwMaxPayloadTransferSize &= ~0xffff0000 ; -if (!(format->flags &UVC_FMT_FLAG_COMPRESSED ) && +if ((!(format->flags &UVC_FMT_FLAG_COMPRESSED ) || + (format->fcc ==V4L2_PIX_FMT_MJPEG )) && stream->dev->quirks &UVC_QUIRK_FIX_BANDWIDTH && stream->intf->num_altsetting >1 ) {u32 interval;
Signature · 1 line
2.7.4
Laurent Pinchart
maintainerHi Pavel,
Thank you for the patch.
Quoted 53 lines · On Monday, 4 September 2017 11:14:17 EEST Pavel Rojtberg wrote:
From: Pavel Rojtberg <[email protected]>
attaching two Logitech C615 webcams currently results in
VIDIOC_STREAMON: No space left on device
as the required bandwidth is not estimated correctly by the device. In fact it always requests 3060 bytes - no matter the format or resolution.setting
UVC_QUIRK_FIX_BANDWIDTHdoes not help either as it is only implemented for uncompressed streams.This patch extends
UVC_QUIRK_FIX_BANDWIDTHto MJPEG streams by making a (conservative) assumption of 4bpp for MJPEG streams.
As the actual compression ration is often closer to 1bpp this can be overridden via the newmjpeg_bppparameter.Based on:
https://www.mail-archive.com/[email protected]/msg05724.html--- drivers/media/usb/uvc/uvc_driver.c | 14 +++++++++++++- drivers/media/usb/uvc/uvc_video.c | 3 ++- 2 files changed, 15 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_driver.cindex 70842c5..f7b759e 100644drivers/media/usb/uvc/uvc_driver.c +11
-1@@ -37,6 +37,7 @@ unsigned int uvc_no_drop_param;static unsigned int uvc_quirks_param =-1 ;unsigned int uvc_trace_param;unsigned int uvc_timeout_param =UVC_CTRL_STREAMING_TIMEOUT ; +static unsigned int uvc_mjpeg_bpp_param;/* ------------------------------------------------------------------------ * Video formats @@ -463,7 +464,16 @@ static int uvc_parse_format(struct uvc_device *dev,strlcpy (format->name,"MJPEG" ,sizeof format->name); format->fcc =V4L2_PIX_FMT_MJPEG ; format->flags =UVC_FMT_FLAG_COMPRESSED ; - format->bpp =0 ; +if ((uvc_mjpeg_bpp_param >=1 ) && (uvc_mjpeg_bpp_param <=16 )) { + format->bpp = uvc_mjpeg_bpp_param; + }else { +/* conservative estimate. Actual values are around 1bpp. +* see e.g. +* https://developers.google.com/speed/webp/docs/webp_study +*/ + format->bpp =4 ; + } + ftype =UVC_VS_FRAME_MJPEG ;break ;
drivers/media/usb/uvc/uvc_driver.c module_param_named(trace, uvc_trace_param, uint, · line 2288
Quoted 5 lines
drivers/media/usb/uvc/uvc_driver.c +2
-0@@ -2274,6 +2284,8 @@ module_param_named(trace, uvc_trace_param, uint, S_IRUGO|S_IWUSR); MODULE_PARM_DESC(trace, "Trace level bitmask");module_param_named (timeout, uvc_timeout_param, uint,S_IRUGO |S_IWUSR );MODULE_PARM_DESC (timeout,"Streaming control requests timeout" ); +module_param_named (mjpeg_bpp, uvc_mjpeg_bpp_param, uint,S_IRUGO |S_IWUSR ); +MODULE_PARM_DESC (mjpeg_bpp,"MJPEG bits per pixel for bandwidth quirk" );
I'd rather avoid adding a new module parameter for this, it would be confusing for users. What is your use case to make the MJPEG average BPP configurable ? Can't we come up with a heuristic that would calculate it automatically ?
Quoted 16 lines
drivers/media/usb/uvc/uvc_driver.c +0
-0/* ------------------------------------------------------------------------ * Driver initialization and cleanup (quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_video.cindex fb86d6a..382a0be 100644drivers/media/usb/uvc/uvc_video.c +2
-1@@ -127,7 +127,8 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming *stream, if ((ctrl->dwMaxPayloadTransferSize & 0xffff0000) == 0xffff0000) ctrl->dwMaxPayloadTransferSize &= ~0xffff0000; -if (!(format->flags &UVC_FMT_FLAG_COMPRESSED ) && +if ((!(format->flags &UVC_FMT_FLAG_COMPRESSED ) || + (format->fcc ==V4L2_PIX_FMT_MJPEG )) && stream->dev->quirks &UVC_QUIRK_FIX_BANDWIDTH && stream->intf->num_altsetting >1 ) {u32 interval;
Signature · 3 lines
Regards, Laurent Pinchart
Hi Laurent,
2017-09-04 11:56 GMT+02:00 Laurent Pinchart <[email protected]>:
Quoted 57 lines
Hi Pavel,
Thank you for the patch.
Quoted 52 lines · On Monday, 4 September 2017 11:14:17 EEST Pavel Rojtberg wrote:
From: Pavel Rojtberg <[email protected]>
attaching two Logitech C615 webcams currently results in
VIDIOC_STREAMON: No space left on device
as the required bandwidth is not estimated correctly by the device. In fact it always requests 3060 bytes - no matter the format or resolution.setting
UVC_QUIRK_FIX_BANDWIDTHdoes not help either as it is only implemented for uncompressed streams.This patch extends
UVC_QUIRK_FIX_BANDWIDTHto MJPEG streams by making a (conservative) assumption of 4bpp for MJPEG streams.
As the actual compression ration is often closer to 1bpp this can be overridden via the newmjpeg_bppparameter.Based on:
https://www.mail-archive.com/[email protected]/msg05724.html--- drivers/media/usb/uvc/uvc_driver.c | 14 +++++++++++++- drivers/media/usb/uvc/uvc_video.c | 3 ++- 2 files changed, 15 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_driver.cindex 70842c5..f7b759e 100644drivers/media/usb/uvc/uvc_driver.c +11
-1@@ -37,6 +37,7 @@ unsigned int uvc_no_drop_param;static unsigned int uvc_quirks_param =-1 ;unsigned int uvc_trace_param;unsigned int uvc_timeout_param =UVC_CTRL_STREAMING_TIMEOUT ; +static unsigned int uvc_mjpeg_bpp_param;/* ------------------------------------------------------------------------ * Video formats @@ -463,7 +464,16 @@ static int uvc_parse_format(struct uvc_device *dev,strlcpy (format->name,"MJPEG" ,sizeof format->name); format->fcc =V4L2_PIX_FMT_MJPEG ; format->flags =UVC_FMT_FLAG_COMPRESSED ; - format->bpp =0 ; +if ((uvc_mjpeg_bpp_param >=1 ) && (uvc_mjpeg_bpp_param <=16 )) { + format->bpp = uvc_mjpeg_bpp_param; + }else { +/* conservative estimate. Actual values are around 1bpp. +* see e.g. +* https://developers.google.com/speed/webp/docs/webp_study +*/ + format->bpp =4 ; + } + ftype =UVC_VS_FRAME_MJPEG ;break ;
Quoted 10 lines
drivers/media/usb/uvc/uvc_driver.c module_param_named(trace, uvc_trace_param, uint, · line 2288
Quoted 6 lines
drivers/media/usb/uvc/uvc_driver.c +2
-0@@ -2274,6 +2284,8 @@ module_param_named(trace, uvc_trace_param, uint, S_IRUGO|S_IWUSR); MODULE_PARM_DESC(trace, "Trace level bitmask");module_param_named (timeout, uvc_timeout_param, uint,S_IRUGO |S_IWUSR );MODULE_PARM_DESC (timeout,"Streaming control requests timeout" ); +module_param_named (mjpeg_bpp, uvc_mjpeg_bpp_param, uint,S_IRUGO |S_IWUSR ); +MODULE_PARM_DESC (mjpeg_bpp,"MJPEG bits per pixel for bandwidth quirk" );I'd rather avoid adding a new module parameter for this, it would be confusing for users. What is your use case to make the MJPEG average BPP configurable ? Can't we come up with a heuristic that would calculate it automatically ?
this is the minimal expected JPEG compression ratio. It depends on the JPEG quality
chosen by the hardware (which we do not know) as well as the actual image (which we can not control either).
We could hardcode it to the average bpp which 1 - but this will break when somebody
films a high-frequency checkerborad. We could leave it at 4, but this would again
artificially limit the number of possible cameras when not filming a checkerboard.
4 would already be a noticable improvement over the current state though.
If you can come up with a working heuristic - I am open for that too.
Quoted 21 lines
Quoted 7 lines
drivers/media/usb/uvc/uvc_driver.c +0
-0/* ------------------------------------------------------------------------ * Driver initialization and cleanup (quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_video.cindex fb86d6a..382a0be 100644drivers/media/usb/uvc/uvc_video.c static void uvc_fixup_video_ctrl(struct uvc_streaming · line 134
Quoted 9 lines
drivers/media/usb/uvc/uvc_video.c +2
-1@@ -127,7 +127,8 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming *stream, if ((ctrl->dwMaxPayloadTransferSize & 0xffff0000) == 0xffff0000) ctrl->dwMaxPayloadTransferSize &= ~0xffff0000; -if (!(format->flags &UVC_FMT_FLAG_COMPRESSED ) && +if ((!(format->flags &UVC_FMT_FLAG_COMPRESSED ) || + (format->fcc ==V4L2_PIX_FMT_MJPEG )) && stream->dev->quirks &UVC_QUIRK_FIX_BANDWIDTH && stream->intf->num_altsetting >1 ) {u32 interval;--
Regards,
Quoted 2 lines
Laurent Pinchart
2017-09-04 11:56 GMT+02:00 Laurent Pinchart <[email protected]>:
Quoted 91 lines
Hi Pavel,
Thank you for the patch.
Quoted 53 lines · On Monday, 4 September 2017 11:14:17 EEST Pavel Rojtberg wrote:
From: Pavel Rojtberg <[email protected]>
attaching two Logitech C615 webcams currently results in
VIDIOC_STREAMON: No space left on device
as the required bandwidth is not estimated correctly by the device. In fact it always requests 3060 bytes - no matter the format or resolution.setting
UVC_QUIRK_FIX_BANDWIDTHdoes not help either as it is only implemented for uncompressed streams.This patch extends
UVC_QUIRK_FIX_BANDWIDTHto MJPEG streams by making a (conservative) assumption of 4bpp for MJPEG streams.
As the actual compression ration is often closer to 1bpp this can be overridden via the newmjpeg_bppparameter.Based on:
https://www.mail-archive.com/[email protected]/msg05724.html--- drivers/media/usb/uvc/uvc_driver.c | 14 +++++++++++++- drivers/media/usb/uvc/uvc_video.c | 3 ++- 2 files changed, 15 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_driver.cindex 70842c5..f7b759e 100644drivers/media/usb/uvc/uvc_driver.c +11
-1@@ -37,6 +37,7 @@ unsigned int uvc_no_drop_param;static unsigned int uvc_quirks_param =-1 ;unsigned int uvc_trace_param;unsigned int uvc_timeout_param =UVC_CTRL_STREAMING_TIMEOUT ; +static unsigned int uvc_mjpeg_bpp_param;/* ------------------------------------------------------------------------ * Video formats @@ -463,7 +464,16 @@ static int uvc_parse_format(struct uvc_device *dev,strlcpy (format->name,"MJPEG" ,sizeof format->name); format->fcc =V4L2_PIX_FMT_MJPEG ; format->flags =UVC_FMT_FLAG_COMPRESSED ; - format->bpp =0 ; +if ((uvc_mjpeg_bpp_param >=1 ) && (uvc_mjpeg_bpp_param <=16 )) { + format->bpp = uvc_mjpeg_bpp_param; + }else { +/* conservative estimate. Actual values are around 1bpp. +* see e.g. +* https://developers.google.com/speed/webp/docs/webp_study +*/ + format->bpp =4 ; + } + ftype =UVC_VS_FRAME_MJPEG ;break ;drivers/media/usb/uvc/uvc_driver.c module_param_named(trace, uvc_trace_param, uint, · line 2288
Quoted 5 lines
drivers/media/usb/uvc/uvc_driver.c +2
-0@@ -2274,6 +2284,8 @@ module_param_named(trace, uvc_trace_param, uint, S_IRUGO|S_IWUSR); MODULE_PARM_DESC(trace, "Trace level bitmask");module_param_named (timeout, uvc_timeout_param, uint,S_IRUGO |S_IWUSR );MODULE_PARM_DESC (timeout,"Streaming control requests timeout" ); +module_param_named (mjpeg_bpp, uvc_mjpeg_bpp_param, uint,S_IRUGO |S_IWUSR ); +MODULE_PARM_DESC (mjpeg_bpp,"MJPEG bits per pixel for bandwidth quirk" );I'd rather avoid adding a new module parameter for this, it would be confusing for users. What is your use case to make the MJPEG average BPP configurable ? Can't we come up with a heuristic that would calculate it automatically ?
Quoted 7 lines
drivers/media/usb/uvc/uvc_driver.c +0
-0/* ------------------------------------------------------------------------ * Driver initialization and cleanup (quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_video.cindex fb86d6a..382a0be 100644drivers/media/usb/uvc/uvc_video.c static void uvc_fixup_video_ctrl(struct uvc_streaming · line 134
Quoted 9 lines
drivers/media/usb/uvc/uvc_video.c +2
-1@@ -127,7 +127,8 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming *stream, if ((ctrl->dwMaxPayloadTransferSize & 0xffff0000) == 0xffff0000) ctrl->dwMaxPayloadTransferSize &= ~0xffff0000; -if (!(format->flags &UVC_FMT_FLAG_COMPRESSED ) && +if ((!(format->flags &UVC_FMT_FLAG_COMPRESSED ) || + (format->fcc ==V4L2_PIX_FMT_MJPEG )) && stream->dev->quirks &UVC_QUIRK_FIX_BANDWIDTH && stream->intf->num_altsetting >1 ) {u32 interval;--
Regards,Laurent Pinchart
Laurent Pinchart
maintainerHi Pavel,
Quoted 87 lines · On Tuesday, 5 September 2017 20:27:39 EEST Pavel Rojtberg wrote:
2017-09-04 11:56 GMT+02:00 Laurent Pinchart:
Quoted 68 lines · On Monday, 4 September 2017 11:14:17 EEST Pavel Rojtberg wrote:
Quoted 67 lines
From: Pavel Rojtberg <[email protected]>
attaching two Logitech C615 webcams currently results in
VIDIOC_STREAMON: No space left on deviceas the required bandwidth is not estimated correctly by the device. In fact it always requests 3060 bytes - no matter the format or resolution.
setting
UVC_QUIRK_FIX_BANDWIDTHdoes not help either as it is only implemented for uncompressed streams.This patch extends
UVC_QUIRK_FIX_BANDWIDTHto MJPEG streams by making a (conservative) assumption of 4bpp for MJPEG streams.
As the actual compression ration is often closer to 1bpp this can be overridden via the newmjpeg_bppparameter.Based on:
https://www.mail-archive.com/[email protected]/msg05724.ht
ml
---drivers/media/usb/uvc/uvc_driver.c | 14 +++++++++++++- drivers/media/usb/uvc/uvc_video.c | 3 ++- 2 files changed, 15 insertions(+), 2 deletions(-)(quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_driver.cindex 70842c5..f7b759e 100644drivers/media/usb/uvc/uvc_driver.c +5
-1@@ -37,6 +37,7 @@ unsigned int uvc_no_drop_param;static unsigned int uvc_quirks_param =-1 ;unsigned int uvc_trace_param;unsigned int uvc_timeout_param =UVC_CTRL_STREAMING_TIMEOUT ; +static unsigned int uvc_mjpeg_bpp_param;/* --------------------------------------------------------------------- * Video formats @@ -463,7 +464,16 @@ static int uvc_parse_format(struct uvc_device *dev,strlcpy (format->name,"MJPEG" ,sizeof format->name); format->fcc =V4L2_PIX_FMT_MJPEG ; format->flags =UVC_FMT_FLAG_COMPRESSED ; - format->bpp =0 ; +if ((uvc_mjpeg_bpp_param >=1 ) && (uvc_mjpeg_bpp_param <=16 )) { + format->bpp = uvc_mjpeg_bpp_param; + }else { +/* conservative estimate. Actual values are around 1bpp.
drivers/media/usb/uvc/uvc_driver.c +2
-0+* see e.g. +* https://developers.google.com/speed/webp/docs/webp_study
drivers/media/usb/uvc/uvc_driver.c +4
-0+*/ + format->bpp =4 ; + } + ftype =UVC_VS_FRAME_MJPEG ;break ;@@ -2274,6 +2284,8 @@ module_param_named(trace, uvc_trace_param, uint, S_IRUGO|S_IWUSR); MODULE_PARM_DESC(trace, "Trace level bitmask");module_param_named (timeout, uvc_timeout_param, uint,S_IRUGO |S_IWUSR );MODULE_PARM_DESC (timeout,"Streaming control requests timeout" );Quoted 8 lines
drivers/media/usb/uvc/uvc_driver.c module_param_named(trace, uvc_trace_param, uint, · line 2291
Quoted 3 lines
drivers/media/usb/uvc/uvc_driver.c +2
-0+module_param_named (mjpeg_bpp, uvc_mjpeg_bpp_param, uint,S_IRUGO |S_IWUSR ); +MODULE_PARM_DESC (mjpeg_bpp,"MJPEG bits per pixel for bandwidth quirk" );I'd rather avoid adding a new module parameter for this, it would be confusing for users. What is your use case to make the MJPEG average BPP configurable ? Can't we come up with a heuristic that would calculate it automatically ?
this is the minimal expected JPEG compression ratio. It depends on the JPEG quality chosen by the hardware (which we do not know) as well as the actual image (which we can not control either).
We could hardcode it to the average bpp which 1 - but this will break when somebody films a high-frequency checkerborad. We could leave it at 4, but this would again artificially limit the number of possible cameras when not filming a checkerboard.
4 would already be a noticable improvement over the current state though.
Quoted 1 line
If you can come up with a working heuristic - I am open for that too.
So it's basically an estimate of the MJPEG compression ratio, which depends on multiple parameters, some of which we could possibly control (such as the compression quality), and some of which we have no influence on (such as the scene in front of the camera).
What really bothers me is that if even we can't estimate the value, exposing it as a module parameter will only confuse most users. How about starting with a fixed conservative estimate (such as 4bpp) and see where that leads us ?
I would also not try to fake a bpp value in uvc_parse_format() for MJPEG formats, but instead adapt uvc_fixup_video_ctrl() with MJPEG-specific calculation. This would allow expressing a compression ratio in the code instead of an artificial bpp value, allowing more precision in the future if needed.
Quoted 17 lines
Quoted 17 lines
Quoted 17 lines
/* ---------------------------------------------------------------------
* Driver initialization and cleanup(quoted diff) +0
-0
b/drivers/media/usb/uvc/uvc_video.cindex fb86d6a..382a0be 100644drivers/media/usb/uvc/uvc_video.c +2
-1@@ -127,7 +127,8 @@ static void uvc_fixup_video_ctrl(struct uvc_streaming *stream, if ((ctrl->dwMaxPayloadTransferSize & 0xffff0000) == 0xffff0000) ctrl->dwMaxPayloadTransferSize &= ~0xffff0000; -if (!(format->flags &UVC_FMT_FLAG_COMPRESSED ) && +if ((!(format->flags &UVC_FMT_FLAG_COMPRESSED ) || + (format->fcc ==V4L2_PIX_FMT_MJPEG )) && stream->dev->quirks &UVC_QUIRK_FIX_BANDWIDTH && stream->intf->num_altsetting >1 ) {u32 interval;
Signature · 3 lines
Regards, Laurent Pinchart