media: venus: dec: Fix capture formats enumeration order
Subsystems (4)
PATCH 1/1 media: venus: dec: Fix capture formats enumeration order
Commit 9593126dae3e ("media: venus: Add a handling of QC08C compressed format") and commit cef92b14e653 ("media: venus: Add a handling of QC10C compressed format") added support for the QC08C and QC10C compressed formats respectively.
But these also caused a regression, because the new formats where added at the beginning of the vdec_formats[] array and the vdec_inst_init() function sets the default format output and capture using fixed indexes of that array:
static void vdec_inst_init(struct venus_inst *inst) {
...
inst->fmt_out = &vdec_formats[8]; inst->fmt_cap = &vdec_formats[0];
...
}
Since now V4L2_PIX_FMT_NV12 is not the first entry in the array anymore, the default capture format is not set to that as it was done before.
Both commits changed the first index to keep inst->fmt_out default format set to V4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out default format set to V4L2_PIX_FMT_NV12.
Rather than updating the index to the current V4L2_PIX_FMT_NV12 position, let's reorder the entries so that this format is the first entry again.
This would also make VIDIOC_ENUM_FMT report the V4L2_PIX_FMT_NV12 format with an index 0 as it did before the QC08C and QC10C formats were added.
---
drivers/media/platform/qcom/venus/vdec.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/media/platform/qcom/venus/vdec.c +4 -4
@@ -31,15 +31,15 @@*/ static const struct venus_format vdec_formats[] = { { - .pixfmt =V4L2_PIX_FMT_QC08C , + .pixfmt =V4L2_PIX_FMT_NV12 , .num_planes =1 , .type =V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE , }, { - .pixfmt =V4L2_PIX_FMT_QC10C , + .pixfmt =V4L2_PIX_FMT_QC08C , .num_planes =1 , .type =V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE , - },{ - .pixfmt =V4L2_PIX_FMT_NV12 , + }, { + .pixfmt =V4L2_PIX_FMT_QC10C , .num_planes =1 , .type =V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE , }, {
Signature · 1 line
2.39.1
Quoted 28 lines · Javier Martinez Canillas
Commit
9593126dae3e("media: venus: Add a handling of QC08C compressed format") and commitcef92b14e653("media: venus: Add a handling of QC10C compressed format") added support for the QC08C and QC10C compressed formats respectively.But these also caused a regression, because the new formats where added at the beginning of the vdec_formats[] array and the
vdec_inst_init()function sets the default format output and capture using fixed indexes of that array:static void
vdec_inst_init(struct venus_inst *inst){
...inst->fmt_out = &vdec_formats[8]; inst->fmt_cap = &vdec_formats[0];...
}Since now
V4L2_PIX_FMT_NV12is not the first entry in the array anymore, the default capture format is not set to that as it was done before.Both commits changed the first index to keep
inst->fmt_outdefault format set toV4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out default format set toV4L2_PIX_FMT_NV12.Rather than updating the index to the current
V4L2_PIX_FMT_NV12position, let's reorder the entries so that this format is the first entry again.
Quoted 6 lines
This would also make
VIDIOC_ENUM_FMTreport theV4L2_PIX_FMT_NV12format with an index 0 as it did before the QC08C and QC10C formats were added.Fixes 9593126dae3e ("media: venus: Add a handling of QC08C compressed format")Fixes cef92b14e653 ("media: venus: Add a handling of QC10C compressed format")Signed-off-by Javier Martinez Canillas <[email protected]>
I just came across this issue independently and can confirm this patch fixes the GStreamer V4L2 decoder on QRB5165.
Quoted 29 lines
---
drivers/media/platform/qcom/venus/vdec.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)drivers/media/platform/qcom/venus/vdec.c +4
-4@@ -31,15 +31,15 @@*/ static const struct venus_format vdec_formats[] = { { - .pixfmt =V4L2_PIX_FMT_QC08C , + .pixfmt =V4L2_PIX_FMT_NV12 , .num_planes =1 , .type =V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE , }, { - .pixfmt =V4L2_PIX_FMT_QC10C , + .pixfmt =V4L2_PIX_FMT_QC08C , .num_planes =1 , .type =V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE , - },{ - .pixfmt =V4L2_PIX_FMT_NV12 , + }, { + .pixfmt =V4L2_PIX_FMT_QC10C , .num_planes =1 , .type =V4L2_BUF_TYPE_VIDEO_CAPTURE_MPLANE , }, {
Jordan Crouse <[email protected]> writes:
Hello Jordan,
Quoted 32 lines · Javier Martinez Canillas
Quoted 31 lines
Commit
9593126dae3e("media: venus: Add a handling of QC08C compressed format") and commitcef92b14e653("media: venus: Add a handling of QC10C compressed format") added support for the QC08C and QC10C compressed formats respectively.But these also caused a regression, because the new formats where added at the beginning of the vdec_formats[] array and the
vdec_inst_init()function sets the default format output and capture using fixed indexes of that array:static void
vdec_inst_init(struct venus_inst *inst){
...inst->fmt_out = &vdec_formats[8]; inst->fmt_cap = &vdec_formats[0];...
}Since now
V4L2_PIX_FMT_NV12is not the first entry in the array anymore, the default capture format is not set to that as it was done before.Both commits changed the first index to keep
inst->fmt_outdefault format set toV4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out default format set toV4L2_PIX_FMT_NV12.Rather than updating the index to the current
V4L2_PIX_FMT_NV12position, let's reorder the entries so that this format is the first entry again.This would also make
VIDIOC_ENUM_FMTreport theV4L2_PIX_FMT_NV12format with an index 0 as it did before the QC08C and QC10C formats were added.
Quoted 9 lines
Quoted 3 lines
Fixes 9593126dae3e ("media: venus: Add a handling of QC08C compressed format")Fixes cef92b14e653 ("media: venus: Add a handling of QC10C compressed format")Signed-off-by Javier Martinez Canillas <[email protected]>I just came across this issue independently and can confirm this patch fixes the GStreamer V4L2 decoder on QRB5165.
Tested-by Jordan Crouse <[email protected]>
Thanks for testing it!
Stanimir, can we please get this for v6.3 as well?
Signature · 5 lines
Best regards, Javier Martinez Canillas Core Platforms Red Hat
Quoted 39 lines · Javier Martinez Canillas
Jordan Crouse <[email protected]> writes:
Hello Jordan,
Quoted 35 lines · Javier Martinez Canillas
Quoted 34 lines
Commit
9593126dae3e("media: venus: Add a handling of QC08C compressed format") and commitcef92b14e653("media: venus: Add a handling of QC10C compressed format") added support for the QC08C and QC10C compressed formats respectively.But these also caused a regression, because the new formats where added at the beginning of the vdec_formats[] array and the
vdec_inst_init()function sets the default format output and capture using fixed indexes of that array:static void
vdec_inst_init(struct venus_inst *inst){
...inst->fmt_out = &vdec_formats[8]; inst->fmt_cap = &vdec_formats[0];...
}Since now
V4L2_PIX_FMT_NV12is not the first entry in the array anymore, the default capture format is not set to that as it was done before.Both commits changed the first index to keep
inst->fmt_outdefault format set toV4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out default format set toV4L2_PIX_FMT_NV12.Rather than updating the index to the current
V4L2_PIX_FMT_NV12position, let's reorder the entries so that this format is the first entry again.This would also make
VIDIOC_ENUM_FMTreport theV4L2_PIX_FMT_NV12format with an index 0 as it did before the QC08C and QC10C formats were added.Fixes 9593126dae3e ("media: venus: Add a handling of QC08C compressed format")Fixes cef92b14e653 ("media: venus: Add a handling of QC10C compressed format")Signed-off-by Javier Martinez Canillas <[email protected]>
Quoted 8 lines
Quoted 5 lines
I just came across this issue independently and can confirm this patch fixes the GStreamer V4L2 decoder on QRB5165.
Tested-by Jordan Crouse <[email protected]>Thanks for testing it!
Stanimir, can we please get this for v6.3 as well?
Hi Javier, Jordan
Could you please explain what regression/issue you see with patch?
venus hardware supports QC08C which provides better performance hence driver is publishing it as preferred color format.
if client doesn't support this or want to use any other format, they can set the desired format with s_fmt.
Thanks,
Dikshita
Dikshita Agarwal <[email protected]> writes:
Hello Dikshita,
Quoted 51 lines · Javier Martinez Canillas
Quoted 39 lines
Jordan Crouse <[email protected]> writes:
Hello Jordan,
Quoted 35 lines · Javier Martinez Canillas
Quoted 34 lines
Commit
9593126dae3e("media: venus: Add a handling of QC08C compressed format") and commitcef92b14e653("media: venus: Add a handling of QC10C compressed format") added support for the QC08C and QC10C compressed formats respectively.But these also caused a regression, because the new formats where added at the beginning of the vdec_formats[] array and the
vdec_inst_init()function sets the default format output and capture using fixed indexes of that array:static void
vdec_inst_init(struct venus_inst *inst){
...inst->fmt_out = &vdec_formats[8]; inst->fmt_cap = &vdec_formats[0];...
}Since now
V4L2_PIX_FMT_NV12is not the first entry in the array anymore, the default capture format is not set to that as it was done before.Both commits changed the first index to keep
inst->fmt_outdefault format set toV4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out default format set toV4L2_PIX_FMT_NV12.Rather than updating the index to the current
V4L2_PIX_FMT_NV12position, let's reorder the entries so that this format is the first entry again.This would also make
VIDIOC_ENUM_FMTreport theV4L2_PIX_FMT_NV12format with an index 0 as it did before the QC08C and QC10C formats were added.Fixes 9593126dae3e ("media: venus: Add a handling of QC08C compressed format")Fixes cef92b14e653 ("media: venus: Add a handling of QC10C compressed format")Signed-off-by Javier Martinez Canillas <[email protected]>Quoted 8 lines
Quoted 5 lines
I just came across this issue independently and can confirm this patch fixes the GStreamer V4L2 decoder on QRB5165.
Tested-by Jordan Crouse <[email protected]>Thanks for testing it!
Stanimir, can we please get this for v6.3 as well?
Hi Javier, Jordan
Quoted 8 lines
Could you please explain what regression/issue you see with patch?
venus hardware supports QC08C which provides better performance hence driver is publishing it as preferred color format.
if client doesn't support this or want to use any other format, they can set the desired format with
s_fmt.
VIDIOC_S_FMT is currently broken for venus, at least on the HP X2 Chromebook and only the default works. I'm still investigating why vdec_s_fmt() is not working.
But basically, if VIDIOC_S_FMT is called for the capture queue, then later the VIDIOC_G_FMT ioctl fails with -EINVAL. This is due the following condition checked in vdec_check_src_change():
static int vdec_check_src_change(struct venus_inst *inst) {
...
if (inst->subscriptions & V4L2_EVENT_SOURCE_CHANGE && inst->codec_state == VENUS_DEC_STATE_INIT && !inst->reconfig) return -EINVAL;
...
}
But regardless, I think that it would be better for a driver to not change the order of advertised VIDIOC_ENUM_FMT pixel formats.
Because what happens now is that a decoding that was previously working by default is not working anymore due a combination of the default being changed and S_FMT not working as expected.
Signature · 5 lines
Best regards, Javier Martinez Canillas Core Platforms Red Hat
Enric Balletbo i Serra
Tested-byHi all,
Quoted 50 lines · Javier Martinez Canillas
Dikshita Agarwal <[email protected]> writes:
Hello Dikshita,
Quoted 45 lines · Javier Martinez Canillas
Quoted 44 lines
Jordan Crouse <[email protected]> writes:
Hello Jordan,
Quoted 40 lines · Javier Martinez Canillas
Quoted 28 lines
Commit
9593126dae3e("media: venus: Add a handling of QC08C compressed format") and commitcef92b14e653("media: venus: Add a handling of QC10C compressed format") added support for the QC08C and QC10C compressed formats respectively.But these also caused a regression, because the new formats where added at the beginning of the vdec_formats[] array and the
vdec_inst_init()function sets the default format output and capture using fixed indexes of that array:static void
vdec_inst_init(struct venus_inst *inst){
...inst->fmt_out = &vdec_formats[8]; inst->fmt_cap = &vdec_formats[0];...
}Since now
V4L2_PIX_FMT_NV12is not the first entry in the array anymore, the default capture format is not set to that as it was done before.Both commits changed the first index to keep
inst->fmt_outdefault format set toV4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out default format set toV4L2_PIX_FMT_NV12.Rather than updating the index to the current
V4L2_PIX_FMT_NV12position, let's reorder the entries so that this format is the first entry again.Quoted 6 lines
This would also make
VIDIOC_ENUM_FMTreport theV4L2_PIX_FMT_NV12format with an index 0 as it did before the QC08C and QC10C formats were added.Fixes 9593126dae3e ("media: venus: Add a handling of QC08C compressed format")Fixes cef92b14e653 ("media: venus: Add a handling of QC10C compressed format")Signed-off-by Javier Martinez Canillas <[email protected]>I just came across this issue independently and can confirm this patch fixes the GStreamer V4L2 decoder on QRB5165.
Tested-by Jordan Crouse <[email protected]>
This patch also fixes an issue running a V4L2 based decoder on Acer Chromebook Spin 513 which is very similar to the HP X2 Chromebook, not surprising as both platforms are basically the same, but anyway:
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Thanks for testing it!
Stanimir, can we please get this for v6.3 as well?
Hi Javier, Jordan
Quoted 8 lines
Quoted 8 lines
Could you please explain what regression/issue you see with patch?
venus hardware supports QC08C which provides better performance hence driver is publishing it as preferred color format.
if client doesn't support this or want to use any other format, they can set the desired format with
s_fmt.
I guess general clients are unlikely to support this format as it is an opaque intermediate format used by Qualcomm platforms, and the purpose of that format is to be used for other Qualcomm hardware blocks that know about this format. So I'd say that returning by default a more common format is more reliable. Using your argument if someone wants to use QC08C (because he knows it can use it) set with s_fmt will do the trick too.
In any case, the problem here seems to be that s_fmt is not working, so it would be nice to have a solution for that first and meanwhile do not change the old behaviour. Just my two cents.
Best regards,
Enric Balletbo
Quoted 33 lines
VIDIOC_S_FMTis currently broken for venus, at least on the HP X2 Chromebook and only the default works. I'm still investigating whyvdec_s_fmt()is not working.But basically, if
VIDIOC_S_FMTis called for the capture queue, then later theVIDIOC_G_FMTioctl fails with-EINVAL. This is due the following condition checked invdec_check_src_change():static int
vdec_check_src_change(struct venus_inst *inst){
...if (inst->subscriptions & V4L2_EVENT_SOURCE_CHANGE && inst->codec_state == VENUS_DEC_STATE_INIT && !inst->reconfig) return -EINVAL;...
}But regardless, I think that it would be better for a driver to not change the order of advertised
VIDIOC_ENUM_FMTpixel formats.Because what happens now is that a decoding that was previously working by default is not working anymore due a combination of the default being changed and
S_FMTnot working as expected.--
Best regards,Javier Martinez Canillas
Core Platforms
Red Hat
Quoted 85 lines · Enric Balletbo i Serra
Hi all,
Quoted 50 lines · Javier Martinez Canillas
Dikshita Agarwal <[email protected]> writes:
Hello Dikshita,
Quoted 45 lines · Javier Martinez Canillas
Quoted 44 lines
Jordan Crouse <[email protected]> writes:
Hello Jordan,
Quoted 40 lines · Javier Martinez Canillas
Quoted 28 lines
Commit
9593126dae3e("media: venus: Add a handling of QC08C compressed format") and commitcef92b14e653("media: venus: Add a handling of QC10C compressed format") added support for the QC08C and QC10C compressed formats respectively.But these also caused a regression, because the new formats where added at the beginning of the vdec_formats[] array and the
vdec_inst_init()function sets the default format output and capture using fixed indexes of that array:static void
vdec_inst_init(struct venus_inst *inst){
...inst->fmt_out = &vdec_formats[8]; inst->fmt_cap = &vdec_formats[0];...
}Since now
V4L2_PIX_FMT_NV12is not the first entry in the array anymore, the default capture format is not set to that as it was done before.Both commits changed the first index to keep
inst->fmt_outdefault format set toV4L2_PIX_FMT_H264, but did not update the latter to keep .fmt_out default format set toV4L2_PIX_FMT_NV12.Rather than updating the index to the current
V4L2_PIX_FMT_NV12position, let's reorder the entries so that this format is the first entry again.Quoted 6 lines
This would also make
VIDIOC_ENUM_FMTreport theV4L2_PIX_FMT_NV12format with an index 0 as it did before the QC08C and QC10C formats were added.Fixes 9593126dae3e ("media: venus: Add a handling of QC08C compressed format")Fixes cef92b14e653 ("media: venus: Add a handling of QC10C compressed format")Signed-off-by Javier Martinez Canillas <[email protected]>I just came across this issue independently and can confirm this patch fixes the GStreamer V4L2 decoder on QRB5165.
Tested-by Jordan Crouse <[email protected]>This patch also fixes an issue running a V4L2 based decoder on Acer Chromebook Spin 513 which is very similar to the HP X2 Chromebook, not surprising as both platforms are basically the same, but anyway:
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Thanks for testing it!
Stanimir, can we please get this for v6.3 as well?
Hi Javier, Jordan
Quoted 8 lines
Quoted 8 lines
Could you please explain what regression/issue you see with patch?
venus hardware supports QC08C which provides better performance hence driver is publishing it as preferred color format.
if client doesn't support this or want to use any other format, they can set the desired format with
s_fmt.I guess general clients are unlikely to support this format as it is an opaque intermediate format used by Qualcomm platforms, and the purpose of that format is to be used for other Qualcomm hardware blocks that know about this format. So I'd say that returning by default a more common format is more reliable. Using your argument if someone wants to use QC08C (because he knows it can use it) set with
s_fmtwill do the trick too.
Quoted 32 lines
In any case, the problem here seems to be that
s_fmtis not working, so it would be nice to have a solution for that first and meanwhile do not change the old behaviour. Just my two cents.Best regards,
Enric BalletboQuoted 25 lines
VIDIOC_S_FMTis currently broken for venus, at least on the HP X2 Chromebook and only the default works. I'm still investigating whyvdec_s_fmt()is not working.But basically, if
VIDIOC_S_FMTis called for the capture queue, then later theVIDIOC_G_FMTioctl fails with-EINVAL. This is due the following condition checked invdec_check_src_change():static int
vdec_check_src_change(struct venus_inst *inst){
...if (inst->subscriptions & V4L2_EVENT_SOURCE_CHANGE && inst->codec_state == VENUS_DEC_STATE_INIT && !inst->reconfig) return -EINVAL;...
}But regardless, I think that it would be better for a driver to not change the order of advertised
VIDIOC_ENUM_FMTpixel formats.Because what happens now is that a decoding that was previously working by default is not working anymore due a combination of the default being changed and
S_FMTnot working as expected.
For my part, I was using the gstreamer v4l2 decoder which for some reason tries to verify it can support whatever format it gets with G_FMT *before* trying a S_FMT. I can't confirm or deny if S_FMT currently works or not.
That said, I entirely agree with Javier. While it might be more bandwidth efficient, QC08C is a obscure format. It is far more likely that the average open source user would rather use a well known output format and, as has been mentioned, once S_FMT is fixed those in the know can use the other formats if they are working with other Qualcomm hardware blocks.
Jordan
Quoted 1 line · Jordan Crouse
On Tue, Mar 07, 2023 at 05:20:18PM +0100, Enric Balletbo i Serra wrote:
[...]
Quoted 12 lines
Quoted 7 lines
Quoted 7 lines
But regardless, I think that it would be better for a driver to not change the order of advertised
VIDIOC_ENUM_FMTpixel formats.Because what happens now is that a decoding that was previously working by default is not working anymore due a combination of the default being changed and
S_FMTnot working as expected.For my part, I was using the gstreamer v4l2 decoder which for some reason tries to verify it can support whatever format it gets with
G_FMT*before* trying aS_FMT. I can't confirm or deny ifS_FMTcurrently works or not.
Quoted 6 lines
That said, I entirely agree with Javier. While it might be more bandwidth efficient, QC08C is a obscure format. It is far more likely that the average open source user would rather use a well known output format and, as has been mentioned, once
S_FMTis fixed those in the know can use the other formats if they are working with other Qualcomm hardware blocks.
Agreed. The rule is that the kernel shouldn't regress user-space and the patches that changed the default format certainly did that. So from that point of view I think that this patch should land.
There's also Enric's point that NV12 is a more common format and supported by more user-space programs. That's why think that regardless of the S_FMT situation, makes sense to revert to the previous default behaviour.
Signature · 5 lines
Best regards, Javier Martinez Canillas Core Platforms Red Hat
Javier Martinez Canillas <[email protected]> writes:
Hello Stanimir and Dikshita,
Quoted 25 lines
Quoted 1 line · Jordan Crouse
On Tue, Mar 07, 2023 at 05:20:18PM +0100, Enric Balletbo i Serra wrote:
[...]
Quoted 12 lines
Quoted 7 lines
Quoted 7 lines
But regardless, I think that it would be better for a driver to not change the order of advertised
VIDIOC_ENUM_FMTpixel formats.Because what happens now is that a decoding that was previously working by default is not working anymore due a combination of the default being changed and
S_FMTnot working as expected.For my part, I was using the gstreamer v4l2 decoder which for some reason tries to verify it can support whatever format it gets with
G_FMT*before* trying aS_FMT. I can't confirm or deny ifS_FMTcurrently works or not.Quoted 6 lines
That said, I entirely agree with Javier. While it might be more bandwidth efficient, QC08C is a obscure format. It is far more likely that the average open source user would rather use a well known output format and, as has been mentioned, once
S_FMTis fixed those in the know can use the other formats if they are working with other Qualcomm hardware blocks.
Quoted 8 lines
Agreed. The rule is that the kernel shouldn't regress user-space and the patches that changed the default format certainly did that. So from that point of view I think that this patch should land.
There's also Enric's point that NV12 is a more common format and supported by more user-space programs. That's why think that regardless of the
S_FMTsituation, makes sense to revert to the previous default behaviour.
Any news on this patch? It would be great to fix this at least for v6.3.
Signature · 5 lines
Best regards, Javier Martinez Canillas Core Platforms Red Hat