media: hantro: Support H264 profile control
Subsystems (2)
PATCH 1/1 media: hantro: Support H264 profile control
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with the exception of the Extended profile.
Expose the V4L2_CID_MPEG_VIDEO_H264_PROFILE control, so that the applications can query the driver for the list of supported profiles.
--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)
drivers/staging/media/hantro/hantro_drv.c +10 -0
@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, { }, };
Signature · 1 line
2.24.0.432.g9d3f5f5b63-goog
Quoted 5 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with the exception of the Extended profile.
Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.
Thanks for this patch. Do you think we could also add the LEVEL control so the profile/level enumeration becomes complete ?
I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Quoted 27 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, { }, };
Quoted 8 lines · Nicolas Dufresne
Quoted 5 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with the exception of the Extended profile.
Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.
Quoted 8 lines
Thanks for this patch. Do you think we could also add the LEVEL control so the profile/level enumeration becomes complete ?
I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 28 lines
Quoted 27 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, { }, };
Quoted 23 lines · Tomasz Figa
Quoted 7 lines · Nicolas Dufresne
Quoted 5 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with the exception of the Extended profile.
Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 8 lines
Thanks for this patch. Do you think we could also add the LEVEL control so the profile/level enumeration becomes complete ?
I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, { }, };
Quoted 31 lines · Nicolas Dufresne
Quoted 23 lines · Tomasz Figa
Quoted 7 lines · Nicolas Dufresne
Quoted 5 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with the exception of the Extended profile.
Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 8 lines
Thanks for this patch. Do you think we could also add the LEVEL control so the profile/level enumeration becomes complete ?
I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 10 lines
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Agreed, one could potentially define it this way, but would it be really useful for the userspace and the users? I guess it could enable slightly faster fallback to software decoding in the extreme case of the hardware not supporting the level at all, but I suspect that the majority of cases would be the hardware just being unusably slow.
Also, as I mentioned before, we already return the range of supported resolutions, which in practice should map to the part of the level that may depend on hardware capabilities rather than performance, so exposing levels as well would add redundancy to the information exposed.
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Quoted 27 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, { }, };
Quoted 49 lines · Tomasz Figa
Quoted 31 lines · Nicolas Dufresne
Quoted 23 lines · Tomasz Figa
Quoted 7 lines · Nicolas Dufresne
Quoted 5 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with the exception of the Extended profile.
Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 8 lines
Thanks for this patch. Do you think we could also add the LEVEL control so the profile/level enumeration becomes complete ?
I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 10 lines
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Agreed, one could potentially define it this way, but would it be really useful for the userspace and the users? I guess it could enable slightly faster fallback to software decoding in the extreme case of the hardware not supporting the level at all, but I suspect that the majority of cases would be the hardware just being unusably slow.
Quoted 5 lines
Also, as I mentioned before, we already return the range of supported resolutions, which in practice should map to the part of the level that may depend on hardware capabilities rather than performance, so exposing levels as well would add redundancy to the information exposed.
There is a lot of discussion here, but it all revolves around a potential LEVEL control.
So I gather everyone is OK with merging this patch?
If not, let me know asap.
Regards,
Hans
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Quoted 28 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, { }, };
Ezequiel Garcia
Reviewed-byHello Hirokazu, Hans,
Quoted 55 lines · Hans Verkuil
Quoted 48 lines · Tomasz Figa
Quoted 30 lines · Nicolas Dufresne
Quoted 23 lines · Tomasz Figa
Quoted 7 lines · Nicolas Dufresne
Quoted 5 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with the exception of the Extended profile.
Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 8 lines
Thanks for this patch. Do you think we could also add the LEVEL control so the profile/level enumeration becomes complete ?
I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 10 lines
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Agreed, one could potentially define it this way, but would it be really useful for the userspace and the users? I guess it could enable slightly faster fallback to software decoding in the extreme case of the hardware not supporting the level at all, but I suspect that the majority of cases would be the hardware just being unusably slow.
Quoted 5 lines
Also, as I mentioned before, we already return the range of supported resolutions, which in practice should map to the part of the level that may depend on hardware capabilities rather than performance, so exposing levels as well would add redundancy to the information exposed.
Quoted 5 lines
There is a lot of discussion here, but it all revolves around a potential LEVEL control.
So I gather everyone is OK with merging this patch?
I'm fine with this.
[..]
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
drivers/staging/media/hantro/hantro_drv.c +0
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = {
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
drivers/staging/media/hantro/hantro_drv.c static const struct hantro_ctrl controls[] = { · line 362
Quoted 8 lines
drivers/staging/media/hantro/hantro_drv.c +5
-0.def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE,
I'd like to see the .def here ^^^ for consistency
with the other controls.
But I know this is my OCD speaking, so it's fine
as-is as well.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
drivers/staging/media/hantro/hantro_drv.c static const struct hantro_ctrl controls[] = { · line 366
Quoted 4 lines
drivers/staging/media/hantro/hantro_drv.c +4
-0+ .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN,
Thanks,
Eze
Quoted 60 lines · Hans Verkuil
Quoted 53 lines · Tomasz Figa
Quoted 34 lines · Nicolas Dufresne
Quoted 27 lines · Tomasz Figa
Quoted 9 lines · Nicolas Dufresne
Quoted 7 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with
the exception of the Extended profile.Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 9 lines
Thanks for this patch. Do you think we could also add the LEVEL control
so the profile/level enumeration becomes complete ?I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 10 lines
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Agreed, one could potentially define it this way, but would it be really useful for the userspace and the users? I guess it could enable slightly faster fallback to software decoding in the extreme case of the hardware not supporting the level at all, but I suspect that the majority of cases would be the hardware just being unusably slow.
Quoted 5 lines
Also, as I mentioned before, we already return the range of supported resolutions, which in practice should map to the part of the level that may depend on hardware capabilities rather than performance, so exposing levels as well would add redundancy to the information exposed.
Quoted 4 lines
There is a lot of discussion here, but it all revolves around a potential LEVEL control.
So I gather everyone is OK with merging this patch?
I'm ok with this. For me, the level reflects the real time performance capability as define in the spec, while the width/height constraints usually represent an addressing capabicity, which may or may not operate real-time.
Quoted 36 lines
If not, let me know asap.
Regards,
Hans
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, {},};
Quoted 62 lines · Nicolas Dufresne
Quoted 60 lines · Hans Verkuil
Quoted 58 lines · Tomasz Figa
Quoted 34 lines · Nicolas Dufresne
Quoted 27 lines · Tomasz Figa
Quoted 9 lines · Nicolas Dufresne
Quoted 7 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with
the exception of the Extended profile.Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 9 lines
Thanks for this patch. Do you think we could also add the LEVEL control
so the profile/level enumeration becomes complete ?I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 10 lines
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Agreed, one could potentially define it this way, but would it be really useful for the userspace and the users? I guess it could enable slightly faster fallback to software decoding in the extreme case of the hardware not supporting the level at all, but I suspect that the majority of cases would be the hardware just being unusably slow.
Also, as I mentioned before, we already return the range of supported resolutions, which in practice should map to the part of the level that may depend on hardware capabilities rather than performance, so exposing levels as well would add redundancy to the information exposed.
Quoted 9 lines
Quoted 4 lines
There is a lot of discussion here, but it all revolves around a potential LEVEL control.
So I gather everyone is OK with merging this patch?
I'm ok with this. For me, the level reflects the real time performance capability as define in the spec, while the width/height constraints usually represent an addressing capabicity, which may or may not operate real-time.
I'd like to see the level control documentation improved before we start adding it to the drivers. I'll be okay with that then as long as the values are exposed in a consistent and well defined way. :)
As for this patch, Hans, are you going to apply it?
Best regards,
Tomasz
Quoted 37 lines
Quoted 36 lines
If not, let me know asap.
Regards,
Hans
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, {},};
Quoted 73 lines · Tomasz Figa
Quoted 62 lines · Nicolas Dufresne
Quoted 60 lines · Hans Verkuil
Quoted 58 lines · Tomasz Figa
Quoted 34 lines · Nicolas Dufresne
Quoted 27 lines · Tomasz Figa
Quoted 9 lines · Nicolas Dufresne
Quoted 7 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with
the exception of the Extended profile.Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 9 lines
Thanks for this patch. Do you think we could also add the LEVEL control
so the profile/level enumeration becomes complete ?I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 10 lines
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Agreed, one could potentially define it this way, but would it be really useful for the userspace and the users? I guess it could enable slightly faster fallback to software decoding in the extreme case of the hardware not supporting the level at all, but I suspect that the majority of cases would be the hardware just being unusably slow.
Also, as I mentioned before, we already return the range of supported resolutions, which in practice should map to the part of the level that may depend on hardware capabilities rather than performance, so exposing levels as well would add redundancy to the information exposed.
Quoted 9 lines
Quoted 4 lines
There is a lot of discussion here, but it all revolves around a potential LEVEL control.
So I gather everyone is OK with merging this patch?
I'm ok with this. For me, the level reflects the real time performance capability as define in the spec, while the width/height constraints usually represent an addressing capabicity, which may or may not operate real-time.
Quoted 5 lines
I'd like to see the level control documentation improved before we start adding it to the drivers. I'll be okay with that then as long as the values are exposed in a consistent and well defined way. :)
As for this patch, Hans, are you going to apply it?
It's in a PR for 5.7. I had hoped it would go in for v5.6, but it was too late for that.
Regards,
Hans
Quoted 41 lines
Best regards,
TomaszQuoted 37 lines
Quoted 36 lines
If not, let me know asap.
Regards,
Hans
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, {},};
Quoted 75 lines · Hans Verkuil
Quoted 73 lines · Tomasz Figa
Quoted 71 lines · Nicolas Dufresne
Quoted 60 lines · Hans Verkuil
Quoted 53 lines · Tomasz Figa
Quoted 34 lines · Nicolas Dufresne
Quoted 27 lines · Tomasz Figa
Quoted 9 lines · Nicolas Dufresne
Quoted 7 lines · Hirokazu Honda
The Hantro G1 decoder supports H.264 profiles from Baseline to High, with
the exception of the Extended profile.Expose the
V4L2_CID_MPEG_VIDEO_H264_PROFILEcontrol, so that the applications can query the driver for the list of supported profiles.Quoted 9 lines
Thanks for this patch. Do you think we could also add the LEVEL control
so the profile/level enumeration becomes complete ?I'm thinking it would be nice if the v4l2 compliance test make sure that codecs do implement these controls (both stateful and stateless), it's essential for stack with software fallback, or multiple capable codec hardware but with different capabilities.
Level is a difficult story, because it also specifies the number of macroblocks per second, but for decoders like this the number of macroblocks per second it can handle depends on things the driver might be not aware of - clock frequencies, DDR throughput, system load, etc.
Quoted 5 lines
My take on this is that the decoder driver should advertise the highest resolution the decoder can handle due to hardware constraints. Performance related things depend on the integration details and should be managed elsewhere. For example Android and Chrome OS manage expected decoding performance in per-board configuration files.
Quoted 10 lines
When you read datasheet, the HW is always rated to maximum level (and it's a range) with the assumption of a single stream. It seems much easier to expose this as-is, statically then to start doing some math with data that isn't fully exposed to the user. This is about filtering of multiple CODEC instances, it does not need to be rocket science, specially that the amount of missing data is important (e.g. usage of tiles, compression, IPP all have an impact on the final performance). All we want to know in userspace is if this HW is even possibly capable of LEVEL X, and if not, we'll look for another one.
Agreed, one could potentially define it this way, but would it be really useful for the userspace and the users? I guess it could enable slightly faster fallback to software decoding in the extreme case of the hardware not supporting the level at all, but I suspect that the majority of cases would be the hardware just being unusably slow.
Quoted 5 lines
Also, as I mentioned before, we already return the range of supported resolutions, which in practice should map to the part of the level that may depend on hardware capabilities rather than performance, so exposing levels as well would add redundancy to the information exposed.
Quoted 4 lines
There is a lot of discussion here, but it all revolves around a potential LEVEL control.
So I gather everyone is OK with merging this patch?
I'm ok with this. For me, the level reflects the real time performance capability as define in the spec, while the width/height constraints usually represent an addressing capabicity, which may or may not operate real-time.
Quoted 8 lines
Quoted 5 lines
I'd like to see the level control documentation improved before we start adding it to the drivers. I'll be okay with that then as long as the values are exposed in a consistent and well defined way. :)
As for this patch, Hans, are you going to apply it?
It's in a PR for 5.7. I had hoped it would go in for v5.6, but it was too late for that.
Okay, thanks!
Quoted 47 lines
Regards,
Hans
Quoted 41 lines
Best regards,
TomaszQuoted 37 lines
Quoted 36 lines
If not, let me know asap.
Regards,
Hans
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Quoted 29 lines
Signed-off-by Hirokazu Honda <[email protected]>--- drivers/staging/media/hantro/hantro_drv.c | 10 ++++++++++ 1 file changed, 10 insertions(+)drivers/staging/media/hantro/hantro_drv.c +10
-0@@ -355,6 +355,16 @@ static const struct hantro_ctrl controls[] = { .def = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, .max = V4L2_MPEG_VIDEO_H264_START_CODE_ANNEX_B, }, + }, { + .codec = HANTRO_H264_DECODER, + .cfg = { + .id = V4L2_CID_MPEG_VIDEO_H264_PROFILE, + .min = V4L2_MPEG_VIDEO_H264_PROFILE_BASELINE, + .max = V4L2_MPEG_VIDEO_H264_PROFILE_HIGH, + .menu_skip_mask = + BIT(V4L2_MPEG_VIDEO_H264_PROFILE_EXTENDED), + .def = V4L2_MPEG_VIDEO_H264_PROFILE_MAIN, + } }, {},};