Skip to contentSkip to conversation

Enable support for AV1 stateful decoder

RFC PATCH In review DEEPA GUTHYAPPA MADIVALARA 2 participants 9 messages Sep 2, 2025 – Sep 10, 2025

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

cover Enable support for AV1 stateful decoder

Hi all,

This patch series adds initial support for the AV1 stateful decoder codecs in iris decoder. Also it adds support for AV1 stateful decoder in V4l2. The objective of this work is to extend the Iris decoder's capabilities to handle AV1 format codec streams, including necessary format handling and buffer management. I'm sharing this series as an RFC because conformance testing and gstreamer testing are still in progress. While initial functional tests show positive results, I would appreciate early feedback on the design, implementation, and fixes before moving to a formal submission. I plan to submit a formal patch series after completing all the compliance checks. Meanwhile, any feedback or suggestion to improve this work are very welcome and will be of great help.

Gstreamer testing:
Gstreamer MR for enabling AV1 stateful decoder:
https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1

Thanks to Nicolas Dufresne for proving the MR
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469

However, Gst testing with parsebin is not resolving to av1parser as
below:
Ex: With the following command parsebin is unable to resolve to
av1parser.
GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault
filesrc location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu ! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! video/x-raw,format=I420
! filesink location=gst_decoder_output.yuv

0:00:00.051674896 400 0xffff8c000b90 DEBUG
parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps video/x-h263, variant=(string)itu

The same test with the av1parse command parses correctly:
GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault
filesrc location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu ! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! video/x-raw,format=I420
! filesink location=/tmp/gst_decoder_output.yuv

Got EOS from element "pipeline0".
Execution ended after 0:00:01.599088176
Setting pipeline to NULL ...
0:00:03.580831249 1075 0x3354f960 DEBUG
av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop

Fluster testing:
As fluster.py is using parsebin for gstreamer, seeing the same issue as described above for the following testsuites. AV1-ARGON-PROFILE0-CORE-ANNEX-B AV1-ARGON-PROFILE0-NON-ANNEX-B AV1-ARGON-PROFILE0-NON-ANNEX-B

Test suite: AV1-TEST-VECTORS
The result of fluster test on SM8550:
134/242 testcases passed while testing AV1-TEST-VECTORS with GStreamer-AV1-V4L2-Gst1.0
unsupported content, bit depth: a000a (66 tests) Iris hardware decoder supports only 8bit NV12 av1-1-b10-00-quantizer-*

Unsupported resolution (36 tests).
Iris hardware decoder supports min resolution of 96x96 av1-1-b8-01-size-*

Unsupported colorformat (1 test)
av1-1-b8-24-monochrome

Crc mismatch: debug in progress (5tests)
av1-1-b8-03-sizeup
av1-1-b8-03-sizedown
av1-1-b8-16-intra_only-intrabc-extreme-dv
av1-1-b8-22-svc-L2T1
av1-1-b8-22-svc-L2T2

Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS
12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with GStreamer-AV1-V4L2-Gst1.0
Crc mismatch: debug in progress
av1-1-b8-03-sizeup.ivf

Unsupported test suites:
Iris Hardware Decoder supports only
PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN
and 8 bit, 420 only
AV1-ARGON-PROFILE1-CORE-ANNEX-B
AV1-ARGON-PROFILE1-NON-ANNEX-B
AV1-ARGON-PROFILE1-STRESS-ANNEX-B
AV1-ARGON-PROFILE2-CORE-ANNEX-B
AV1-ARGON-PROFILE2-NON-ANNEX-B
AV1-ARGON-PROFILE2-STRESS-ANNEX-B
CHROMIUM-10bit-AV1-TEST-VECTORS

Compliance test for iris_driver device /dev/video0:

Driver Info:

        Driver name      : iris_driver
        Card type        : Iris Decoder
        Bus info         : platform:aa00000.video-codec
        Driver version   : 6.16.0
        Capabilities     : 0x84204000

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful Decoder

Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 12 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test CREATE_BUFS maximum buffers: OK test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)

[65391.311675] qcom-iris aa00000.video-codec: invalid plane
[65395.340586] qcom-iris aa00000.video-codec: invalid plane
        test blocking wait: OK

Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0

V4l2-ctl Test verified for 2 streams as well.

Thanks,
Deepa

---
DEEPA GUTHYAPPA MADIVALARA (5):
media: uapi: videodev2: Add support for AV1 stateful decoder
media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
media: iris: Add support for AV1 format in iris decoder
media: iris: Add internal buffer calculation for AV1 decoder
media: iris: Define AV1-specific platform capabilities and properties

 drivers/media/platform/qcom/iris/iris_buffer.h     |   2 +
 drivers/media/platform/qcom/iris/iris_ctrls.c      |   8 +
 drivers/media/platform/qcom/iris/iris_hfi_common.h |   3 +
 .../platform/qcom/iris/iris_hfi_gen2_command.c     | 109 ++++++++-
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  10 +
 .../platform/qcom/iris/iris_hfi_gen2_response.c    |  22 ++
 drivers/media/platform/qcom/iris/iris_instance.h   |   1 +
 .../platform/qcom/iris/iris_platform_common.h      |  15 ++
 .../media/platform/qcom/iris/iris_platform_gen2.c  | 156 ++++++++++++-
 .../platform/qcom/iris/iris_platform_sm8250.c      |  17 ++
 drivers/media/platform/qcom/iris/iris_vdec.c       |  25 +-
 drivers/media/platform/qcom/iris/iris_vidc.c       |   1 +
 drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++-
 drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c               |   1 +
 include/uapi/linux/videodev2.h                     |   1 +
 16 files changed, 703 insertions(+), 28 deletions(-)

---
base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f
change-id: 20250821-rfc_split-c3ff834bb2c9
prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3
prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c
prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e
prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4
prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3
prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2
prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce
prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0
prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca
prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e
prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec
prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759
prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7
prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1
prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0
prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc
prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5
prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c
prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3
prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3
prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5
prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968
prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5
prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423
prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd
prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa
prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83

Best regards,

Signature · 1 line
Deepa Guthyappa Madivalara <[email protected]>
#1 lore raw

RFC PATCH 1/5 media: uapi: videodev2: Add support for AV1 stateful decoder

Introduce a new pixel format, V4L2_PIX_FMT_AV1, to the Video4Linux2(V4L2) API. This format is intended for AV1 bitstreams in stateful decoding/encoding workflows. The fourcc code 'AV10' is used to distinguish this format from the existing V4L2_PIX_FMT_AV1_FRAME, which is used for stateless AV1 decoder implementation.

---
 include/uapi/linux/videodev2.h | 1 +
 1 file changed, 1 insertion(+)
include/uapi/linux/videodev2.h +1 -0
index 3dd9fa45dde1066d52a68581625a39e7ec92c9b7..bff42a71c67b3f4b570dd6f3d250f1bb482ec8ae 100644
@@ -775,6 +775,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
 #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
 #define V4L2_PIX_FMT_AV1_FRAME v4l2_fourcc('A', 'V', '1', 'F') /* AV1 parsed frame */
+#define V4L2_PIX_FMT_AV1      v4l2_fourcc('A', 'V', '1', '0') /* AV1 (stateful) */
 #define V4L2_PIX_FMT_SPK      v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
 #define V4L2_PIX_FMT_RV30     v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
 #define V4L2_PIX_FMT_RV40     v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH RFC 1/5] media: uapi: videodev2: Add support for AV1 stateful decoder

Le mardi 02 septembre 2025 à 16:00 -0700, DEEPA GUTHYAPPA MADIVALARA a écrit :

Quoted 17 lines

Introduce a new pixel format, V4L2_PIX_FMT_AV1, to the Video4Linux2(V4L2) API. This format is intended for AV1 bitstreams in stateful decoding/encoding workflows. The fourcc code 'AV10' is used to distinguish this format from the existing V4L2_PIX_FMT_AV1_FRAME, which is used for stateless AV1 decoder implementation.

---
 include/uapi/linux/videodev2.h | 1 +

 1 file changed, 1 insertion(+)
include/uapi/linux/videodev2.h +0 -0
index 3dd9fa45dde1066d52a68581625a39e7ec92c9b7..bff42a71c67b3f4b570dd6f3d250f1bb482ec8ae 100644

include/uapi/linux/videodev2.h struct v4l2_pix_format { · line 778

Quoted 4 lines
include/uapi/linux/videodev2.h +1 -0
@@ -775,6 +775,7 @@ struct v4l2_pix_format {
  #define V4L2_PIX_FMT_H264_SLICE v4l2_fourcc('S', '2', '6', '4') /* H264 parsed slices */
  #define V4L2_PIX_FMT_HEVC_SLICE v4l2_fourcc('S', '2', '6', '5') /* HEVC parsed slices */
  #define V4L2_PIX_FMT_AV1_FRAME v4l2_fourcc('A', 'V', '1', 'F') /* AV1 parsed frame */
+#define V4L2_PIX_FMT_AV1      v4l2_fourcc('A', 'V', '1', '0') /* AV1 (stateful) */

You also need to update Documentation/userspace-api/media/v4l/pixfmt- compressed.rst. Mistakes were made in the past leading to great confusion, AV1 can be wrapped in different forms and with different alignments. Here's few question that should be answered in the RST documentation:

- Should we pass complete time units to the decoder ?
- If not
- How is the timestamp going to be interpreted and transferred ?
- Should userspace pass complete frame (with leading headers) ? or can it pass OBUs ?
- Is that format meant to be OBU streams or Annex B (not to confused with H.264 annex B, it groups the TUs for faster skipping) ? (I bet the first :-D)

cheers,
Nicolas

Quoted 3 lines
include/uapi/linux/videodev2.h +0 -0
  #define V4L2_PIX_FMT_SPK      v4l2_fourcc('S', 'P', 'K', '0') /* Sorenson Spark */
  #define V4L2_PIX_FMT_RV30     v4l2_fourcc('R', 'V', '3', '0') /* RealVideo 8 */
  #define V4L2_PIX_FMT_RV40     v4l2_fourcc('R', 'V', '4', '0') /* RealVideo 9 & 10 */
#3 lore raw

RFC PATCH 2/5 media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()

In reply to DEEPA GUTHYAPPA MADIVALARA

Add a descriptive string for the AV1 pixel format to v4l_fill_fmtdesc(), enabling proper reporting of AV1 support via VIDIOC_ENUM_FMT.

---
 drivers/media/v4l2-core/v4l2-ioctl.c | 1 +
 1 file changed, 1 insertion(+)
drivers/media/v4l2-core/v4l2-ioctl.c +1 -0
index 01cf52c3ea33e1a01e1b306036ba4e57ef5c95d0..d3ee7736b74b0f277d3208782e3ac3282eca1e6b 100644
@@ -1542,6 +1542,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_QC10C:	descr = "QCOM Compressed 10-bit Format"; break;
 		case V4L2_PIX_FMT_AJPG:		descr = "Aspeed JPEG"; break;
 		case V4L2_PIX_FMT_AV1_FRAME:	descr = "AV1 Frame"; break;
+		case V4L2_PIX_FMT_AV1:		descr = "AV1"; break;
 		case V4L2_PIX_FMT_MT2110T:	descr = "Mediatek 10bit Tile Mode"; break;
 		case V4L2_PIX_FMT_MT2110R:	descr = "Mediatek 10bit Raster Mode"; break;
 		case V4L2_PIX_FMT_HEXTILE:	descr = "Hextile Compressed Format"; break;
Signature · 1 line
2.34.1
#4 lore raw

RFC PATCH 3/5 media: iris: Add support for AV1 format in iris decoder

In reply to DEEPA GUTHYAPPA MADIVALARA

Extend iris decoder driver to support format V4L2_PIX_FMT_AV1. This change updates the format enumeration (VIDIOC_ENUM_FMT) and allows setting AV1 format via VIDIOC_S_FMT for gen2 and beyond. Gen1 iris hardware decoder does not support AV1 format.

---
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  1 +
 drivers/media/platform/qcom/iris/iris_instance.h   |  1 +
 .../platform/qcom/iris/iris_platform_common.h      |  2 ++
 .../media/platform/qcom/iris/iris_platform_gen2.c  | 23 ++++++++++++++++++++
 .../platform/qcom/iris/iris_platform_sm8250.c      | 17 +++++++++++++++
 drivers/media/platform/qcom/iris/iris_vdec.c       | 25 ++++++----------------
 6 files changed, 50 insertions(+), 19 deletions(-)
drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +1 -0
index aa1f795f5626c1f76a32dd650302633877ce67be..bbfe7a0851ea94fb7041a868b4df8b2ec63bf427 100644
@@ -138,6 +138,7 @@ enum hfi_codec_type {
 	HFI_CODEC_DECODE_HEVC			= 3,
 	HFI_CODEC_ENCODE_HEVC			= 4,
 	HFI_CODEC_DECODE_VP9			= 5,
+	HFI_CODEC_DECODE_AV1			= 7,
 };
 
 enum hfi_picture_type {
drivers/media/platform/qcom/iris/iris_instance.h +1 -0
index 5982d7adefeab80905478b32cddba7bd4651a691..f1883ffc138fd975fb76d4e45904ee04e196cd20 100644
@@ -19,6 +19,7 @@ enum iris_fmt_type {
 	IRIS_FMT_H264,
 	IRIS_FMT_HEVC,
 	IRIS_FMT_VP9,
+	IRIS_FMT_AV1,
 };
 
 struct iris_fmt {
drivers/media/platform/qcom/iris/iris_platform_common.h +2 -0
index 96fa7b1bb592441e85664da408ea4ba42c9a15b5..99057a624cb976af68403ef042173b5ebefde8af 100644
@@ -208,6 +208,8 @@ struct iris_platform_data {
 	u64 dma_mask;
 	const char *fwname;
 	u32 pas_id;
+	struct iris_fmt *inst_iris_fmts;
+	u32 inst_iris_fmts_size;
 	struct platform_inst_caps *inst_caps;
 	struct platform_inst_fw_cap *inst_fw_caps_dec;
 	u32 inst_fw_caps_dec_size;
drivers/media/platform/qcom/iris/iris_platform_gen2.c +23 -0
index cf4b92f534b272a0a1ac2a0e7bb9316501374332..fecf2e06f19723f30777c04bc4b155954c2b746d 100644
@@ -16,6 +16,25 @@
 #define VIDEO_ARCH_LX 1
 #define BITRATE_MAX				245000000
 
+static struct iris_fmt platform_fmts_sm8550_dec[] = {
+	[IRIS_FMT_H264] = {
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+	[IRIS_FMT_HEVC] = {
+		.pixfmt = V4L2_PIX_FMT_HEVC,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+	[IRIS_FMT_VP9] = {
+		.pixfmt = V4L2_PIX_FMT_VP9,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+	[IRIS_FMT_AV1] = {
+		.pixfmt = V4L2_PIX_FMT_AV1,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}
+};
+
 static struct platform_inst_fw_cap inst_fw_cap_sm8550_dec[] = {
 	{
 		.cap_id = PROFILE_H264,
@@ -756,6 +775,8 @@ struct iris_platform_data sm8550_data = {
 	.dma_mask = 0xe0000000 - 1,
 	.fwname = "qcom/vpu/vpu30_p4.mbn",
 	.pas_id = IRIS_PAS_ID,
+	.inst_iris_fmts = platform_fmts_sm8550_dec,
+	.inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
 	.inst_caps = &platform_inst_cap_sm8550,
 	.inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
 	.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
@@ -847,6 +868,8 @@ struct iris_platform_data sm8650_data = {
 	.dma_mask = 0xe0000000 - 1,
 	.fwname = "qcom/vpu/vpu33_p4.mbn",
 	.pas_id = IRIS_PAS_ID,
+	.inst_iris_fmts = platform_fmts_sm8550_dec,
+	.inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8550_dec),
 	.inst_caps = &platform_inst_cap_sm8550,
 	.inst_fw_caps_dec = inst_fw_cap_sm8550_dec,
 	.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8550_dec),
drivers/media/platform/qcom/iris/iris_platform_sm8250.c +17 -0
index 978d0130d43b5f6febb65430a9bbe3932e8f24df..947dd25a483a792681e34fbbf4396342db582203 100644
@@ -16,6 +16,21 @@
 #define BITRATE_PEAK_DEFAULT	(BITRATE_DEFAULT * 2)
 #define BITRATE_STEP		100
 
+static struct iris_fmt platform_fmts_sm8250_dec[] = {
+	[IRIS_FMT_H264] = {
+		.pixfmt = V4L2_PIX_FMT_H264,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+	[IRIS_FMT_HEVC] = {
+		.pixfmt = V4L2_PIX_FMT_HEVC,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	},
+	[IRIS_FMT_VP9] = {
+		.pixfmt = V4L2_PIX_FMT_VP9,
+		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
+	}
+};
+
 static struct platform_inst_fw_cap inst_fw_cap_sm8250_dec[] = {
 	{
 		.cap_id = PIPE,
@@ -335,6 +350,8 @@ struct iris_platform_data sm8250_data = {
 	.dma_mask = 0xe0000000 - 1,
 	.fwname = "qcom/vpu-1.0/venus.mbn",
 	.pas_id = IRIS_PAS_ID,
+	.inst_iris_fmts = platform_fmts_sm8250_dec,
+	.inst_iris_fmts_size = ARRAY_SIZE(platform_fmts_sm8250_dec),
 	.inst_caps = &platform_inst_cap_sm8250,
 	.inst_fw_caps_dec = inst_fw_cap_sm8250_dec,
 	.inst_fw_caps_dec_size = ARRAY_SIZE(inst_fw_cap_sm8250_dec),
drivers/media/platform/qcom/iris/iris_vdec.c +6 -19
index ae13c3e1b426bfd81a7b46dc6c3ff5eb5c4860cb..be8d2d48c82f385e4f46807f7e0dd52e469927cd 100644
@@ -67,26 +67,12 @@ void iris_vdec_inst_deinit(struct iris_inst *inst)
 	kfree(inst->fmt_src);
 }
 
-static const struct iris_fmt iris_vdec_formats[] = {
-	[IRIS_FMT_H264] = {
-		.pixfmt = V4L2_PIX_FMT_H264,
-		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-	},
-	[IRIS_FMT_HEVC] = {
-		.pixfmt = V4L2_PIX_FMT_HEVC,
-		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-	},
-	[IRIS_FMT_VP9] = {
-		.pixfmt = V4L2_PIX_FMT_VP9,
-		.type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE,
-	},
-};
-
 static const struct iris_fmt *
 find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
 {
-	unsigned int size = ARRAY_SIZE(iris_vdec_formats);
-	const struct iris_fmt *fmt = iris_vdec_formats;
+	unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
+	const struct iris_fmt *fmt =
+			inst->core->iris_platform_data->inst_iris_fmts;
 	unsigned int i;
 
 	for (i = 0; i < size; i++) {
@@ -103,8 +89,9 @@ find_format(struct iris_inst *inst, u32 pixfmt, u32 type)
 static const struct iris_fmt *
 find_format_by_index(struct iris_inst *inst, u32 index, u32 type)
 {
-	const struct iris_fmt *fmt = iris_vdec_formats;
-	unsigned int size = ARRAY_SIZE(iris_vdec_formats);
+	unsigned int size = inst->core->iris_platform_data->inst_iris_fmts_size;
+	const struct iris_fmt *fmt =
+			inst->core->iris_platform_data->inst_iris_fmts;
 
 	if (index >= size || fmt[index].type != type)
 		return NULL;
Signature · 1 line
2.34.1
#5 lore raw

RFC PATCH 4/5 media: iris: Add internal buffer calculation for AV1 decoder

In reply to DEEPA GUTHYAPPA MADIVALARA

Implement internal buffer count and size calculations for AV1 decoder.

---
 drivers/media/platform/qcom/iris/iris_buffer.h     |   2 +
 .../platform/qcom/iris/iris_hfi_gen2_command.c     | 109 ++++++++-
 .../platform/qcom/iris/iris_hfi_gen2_response.c    |   3 +
 .../media/platform/qcom/iris/iris_platform_gen2.c  |   1 +
 drivers/media/platform/qcom/iris/iris_vidc.c       |   1 +
 drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++-
 drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++
 7 files changed, 468 insertions(+), 8 deletions(-)
drivers/media/platform/qcom/iris/iris_buffer.h +2 -0
index 325d30fce5c99185b61ff989fbfd4de9a56762b2..75bb767761824c4c02e0df9b765896cc093be333 100644
@@ -27,6 +27,7 @@ struct iris_inst;
  * @BUF_SCRATCH_1: buffer to store decoding/encoding context data for HW
  * @BUF_SCRATCH_2: buffer to store encoding context data for HW
  * @BUF_VPSS: buffer to store VPSS context data for HW
+ * @BUF_PARTIAL: buffer for AV1 IBC data
  * @BUF_TYPE_MAX: max buffer types
  */
 enum iris_buffer_type {
@@ -42,6 +43,7 @@ enum iris_buffer_type {
 	BUF_SCRATCH_1,
 	BUF_SCRATCH_2,
 	BUF_VPSS,
+	BUF_PARTIAL,
 	BUF_TYPE_MAX,
 };
 
drivers/media/platform/qcom/iris/iris_hfi_gen2_command.c +105 -4

197 lines not on this page · raw message

drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +3 -0
index 2f1f118eae4f6462ab1aa1d16844b34e6e699f1e..943d3884248fafccc3b8e944e455c96124914353 100644
@@ -54,6 +54,8 @@ static u32 iris_hfi_gen2_buf_type_to_driver(struct iris_inst *inst,
 			return BUF_SCRATCH_2;
 	case HFI_BUFFER_PERSIST:
 		return BUF_PERSIST;
+	case HFI_BUFFER_PARTIAL_DATA:
+		return BUF_PARTIAL;
 	default:
 		return 0;
 	}
@@ -72,6 +74,7 @@ static bool iris_hfi_gen2_is_valid_hfi_buffer_type(u32 buffer_type)
 	case HFI_BUFFER_DPB:
 	case HFI_BUFFER_PERSIST:
 	case HFI_BUFFER_VPSS:
+	case HFI_BUFFER_PARTIAL_DATA:
 		return true;
 	default:
 		return false;
drivers/media/platform/qcom/iris/iris_platform_gen2.c +1 -0
index fecf2e06f19723f30777c04bc4b155954c2b746d..c0a03c03bb9349a0e551ce6dc72b6f8b23ff696d 100644
@@ -739,6 +739,7 @@ static const u32 sm8550_dec_ip_int_buf_tbl[] = {
 	BUF_COMV,
 	BUF_NON_COMV,
 	BUF_LINE,
+	BUF_PARTIAL,
 };
 
 static const u32 sm8550_dec_op_int_buf_tbl[] = {
drivers/media/platform/qcom/iris/iris_vidc.c +1 -0
index 798c3613e57eac1742633d61c1482229dbc32562..43a77687a668423a5547c7f159622249634264e0 100644
@@ -178,6 +178,7 @@ int iris_open(struct file *filp)
 	INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_1].list);
 	INIT_LIST_HEAD(&inst->buffers[BUF_SCRATCH_2].list);
 	INIT_LIST_HEAD(&inst->buffers[BUF_VPSS].list);
+	INIT_LIST_HEAD(&inst->buffers[BUF_PARTIAL].list);
 	init_completion(&inst->completion);
 	init_completion(&inst->flush_completion);
 
drivers/media/platform/qcom/iris/iris_vpu_buffer.c +251 -4

365 lines not on this page · raw message

drivers/media/platform/qcom/iris/iris_vpu_buffer.h +105 -0

123 lines not on this page · raw message

Signature · 1 line
2.34.1
#6 lore raw

RFC PATCH 5/5 media: iris: Define AV1-specific platform capabilities and properties

In reply to DEEPA GUTHYAPPA MADIVALARA

Defining platform specific capabilities specific to AV1 decoder. Set and subscribe to manadatory properties to firmware for AV1.

---
 drivers/media/platform/qcom/iris/iris_ctrls.c      |   8 ++
 drivers/media/platform/qcom/iris/iris_hfi_common.h |   3 +
 .../platform/qcom/iris/iris_hfi_gen2_defines.h     |   9 ++
 .../platform/qcom/iris/iris_hfi_gen2_response.c    |  19 +++
 .../platform/qcom/iris/iris_platform_common.h      |  13 ++
 .../media/platform/qcom/iris/iris_platform_gen2.c  | 132 ++++++++++++++++++++-
 6 files changed, 183 insertions(+), 1 deletion(-)
drivers/media/platform/qcom/iris/iris_ctrls.c +8 -0
index 754a5ad718bc37630bb861012301df7a2e7342a1..620c7e1bd273e25febd8ca70dd1dcfb0b862692b 100644
@@ -98,6 +98,10 @@ static enum platform_inst_fw_cap_type iris_get_cap_id(u32 id)
 		return B_FRAME_QP_H264;
 	case V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP:
 		return B_FRAME_QP_HEVC;
+	case V4L2_CID_MPEG_VIDEO_AV1_PROFILE:
+		return PROFILE_AV1;
+	case V4L2_CID_MPEG_VIDEO_AV1_LEVEL:
+		return LEVEL_AV1;
 	default:
 		return INST_FW_CAP_MAX;
 	}
@@ -185,6 +189,10 @@ static u32 iris_get_v4l2_id(enum platform_inst_fw_cap_type cap_id)
 		return V4L2_CID_MPEG_VIDEO_H264_B_FRAME_QP;
 	case B_FRAME_QP_HEVC:
 		return V4L2_CID_MPEG_VIDEO_HEVC_B_FRAME_QP;
+	case PROFILE_AV1:
+		return V4L2_CID_MPEG_VIDEO_AV1_PROFILE;
+	case LEVEL_AV1:
+		return V4L2_CID_MPEG_VIDEO_AV1_LEVEL;
 	default:
 		return 0;
 	}
drivers/media/platform/qcom/iris/iris_hfi_common.h +3 -0
index b51471fb32c70acee44c37f8e9dce0c6bc0b6ccc..3edb5ae582b49bea2e2408c4a5cfc0a742adc05f 100644
@@ -141,6 +141,9 @@ struct hfi_subscription_params {
 	u32	profile;
 	u32	level;
 	u32	tier;
+	u32	drap;
+	u32	film_grain;
+	u32	super_block;
 };
 
 u32 iris_hfi_get_v4l2_color_primaries(u32 hfi_primaries);
drivers/media/platform/qcom/iris/iris_hfi_gen2_defines.h +9 -0
index bbfe7a0851ea94fb7041a868b4df8b2ec63bf427..9cc0989c67d74a9e051725e9ee571a2ab9160519 100644
@@ -89,9 +89,18 @@ enum hfi_seq_header_mode {
 #define HFI_PROP_DEC_START_FROM_RAP_FRAME	0x03000169
 #define HFI_PROP_NO_OUTPUT			0x0300016a
 #define HFI_PROP_BUFFER_MARK			0x0300016c
+#define HFI_PROP_WORST_COMPRESSION_RATIO	0x03000174
+#define HFI_PROP_WORST_COMPLEXITY_FACTOR	0x03000175
 #define HFI_PROP_RAW_RESOLUTION		0x03000178
+#define HFI_PROP_AV1_FILM_GRAIN_PRESENT		0x03000180
+#define HFI_PROP_AV1_SUPER_BLOCK_ENABLED	0x03000181
+#define HFI_PROP_AV1_OP_POINT				0x03000182
+#define HFI_PROP_OPB_ENABLE                 0x03000184
+#define HFI_PROP_AV1_TILE_ROWS_COLUMNS      0x03000187
+#define HFI_PROP_AV1_DRAP_CONFIG            0x03000189
 #define HFI_PROP_TOTAL_PEAK_BITRATE		0x0300017C
 #define HFI_PROP_COMV_BUFFER_COUNT		0x03000193
+#define HFI_PROP_AV1_UNIFORM_TILE_SPACING	0x03000197
 #define HFI_PROP_END				0x03FFFFFF
 
 #define HFI_SESSION_ERROR_BEGIN			0x04000000
drivers/media/platform/qcom/iris/iris_hfi_gen2_response.c +19 -0
index 943d3884248fafccc3b8e944e455c96124914353..eb3373f0ad4a1b26fb847db02449ec8d8cb3bdbb 100644
@@ -599,6 +599,10 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
 		inst->fw_caps[PROFILE_H264].value = subsc_params.profile;
 		inst->fw_caps[LEVEL_H264].value = subsc_params.level;
 		break;
+	case V4L2_PIX_FMT_AV1:
+		inst->fw_caps[PROFILE_AV1].value = subsc_params.profile;
+		inst->fw_caps[LEVEL_AV1].value = subsc_params.level;
+		break;
 	}
 
 	inst->fw_caps[POC].value = subsc_params.pic_order_cnt;
@@ -611,6 +615,11 @@ static void iris_hfi_gen2_read_input_subcr_params(struct iris_inst *inst)
 		iris_inst_change_state(inst, IRIS_INST_ERROR);
 	}
 
+	if (inst->codec == V4L2_PIX_FMT_AV1) {
+		inst->fw_caps[FILM_GRAIN].value = subsc_params.film_grain;
+		inst->fw_caps[SUPER_BLOCK].value = subsc_params.super_block;
+	}
+
 	inst->fw_min_count = subsc_params.fw_min_count;
 	inst->buffers[BUF_OUTPUT].min_count = iris_vpu_buf_count(inst, BUF_OUTPUT);
 	inst->buffers[BUF_OUTPUT].size = pixmp_op->plane_fmt[0].sizeimage;
@@ -714,6 +723,12 @@ static int iris_hfi_gen2_handle_session_property(struct iris_inst *inst,
 	case HFI_PROP_NO_OUTPUT:
 		inst_hfi_gen2->hfi_frame_info.no_output = 1;
 		break;
+	case HFI_PROP_AV1_FILM_GRAIN_PRESENT:
+		inst_hfi_gen2->src_subcr_params.film_grain = pkt->payload[0];
+		break;
+	case HFI_PROP_AV1_SUPER_BLOCK_ENABLED:
+		inst_hfi_gen2->src_subcr_params.super_block = pkt->payload[0];
+		break;
 	case HFI_PROP_QUALITY_MODE:
 	case HFI_PROP_STAGE:
 	case HFI_PROP_PIPE:
@@ -844,6 +859,10 @@ static void iris_hfi_gen2_init_src_change_param(struct iris_inst *inst)
 		subsc_params->profile = inst->fw_caps[PROFILE_H264].value;
 		subsc_params->level = inst->fw_caps[LEVEL_H264].value;
 		break;
+	case V4L2_PIX_FMT_AV1:
+		subsc_params->profile = inst->fw_caps[PROFILE_AV1].value;
+		subsc_params->level = inst->fw_caps[LEVEL_AV1].value;
+		break;
 	}
 
 	subsc_params->pic_order_cnt = inst->fw_caps[POC].value;
drivers/media/platform/qcom/iris/iris_platform_common.h +13 -0
index 99057a624cb976af68403ef042173b5ebefde8af..e543a3cc142b15e50b12cfe672d00dd0acfdb4d1 100644
@@ -95,6 +95,13 @@ enum platform_inst_fw_cap_type {
 	LEVEL_H264,
 	LEVEL_HEVC,
 	LEVEL_VP9,
+	PROFILE_AV1,
+	LEVEL_AV1,
+	TIER_AV1,
+	DRAP,
+	FILM_GRAIN,
+	SUPER_BLOCK,
+	ENH_LAYER_COUNT,
 	INPUT_BUF_HOST_MAX_COUNT,
 	OUTPUT_BUF_HOST_MAX_COUNT,
 	STAGE,
@@ -231,8 +238,12 @@ struct iris_platform_data {
 	unsigned int dec_input_config_params_hevc_size;
 	const u32 *dec_input_config_params_vp9;
 	unsigned int dec_input_config_params_vp9_size;
+	const u32 *dec_input_config_params_av1;
+	unsigned int dec_input_config_params_av1_size;
 	const u32 *dec_output_config_params;
 	unsigned int dec_output_config_params_size;
+	const u32 *dec_output_config_params_av1;
+	unsigned int dec_output_config_params_av1_size;
 	const u32 *enc_input_config_params;
 	unsigned int enc_input_config_params_size;
 	const u32 *enc_output_config_params;
@@ -245,6 +256,8 @@ struct iris_platform_data {
 	unsigned int dec_output_prop_hevc_size;
 	const u32 *dec_output_prop_vp9;
 	unsigned int dec_output_prop_vp9_size;
+	const u32 *dec_output_prop_av1;
+	unsigned int dec_output_prop_av1_size;
 	const u32 *dec_ip_int_buf_tbl;
 	unsigned int dec_ip_int_buf_tbl_size;
 	const u32 *dec_op_int_buf_tbl;
drivers/media/platform/qcom/iris/iris_platform_gen2.c +131 -1

200 lines not on this page · raw message

Signature · 1 line
2.34.1
#7 lore raw
In reply to DEEPA GUTHYAPPA MADIVALARA

Le mardi 02 septembre 2025 à 16:00 -0700, DEEPA GUTHYAPPA MADIVALARA a écrit :

Quoted 23 lines

Hi all,

This patch series adds initial support for the AV1 stateful decoder codecs in iris decoder. Also it adds support for AV1 stateful decoder in V4l2. The objective of this work is to extend the Iris decoder's capabilities to handle AV1 format codec streams, including necessary format handling and buffer management. I'm sharing this series as an RFC because conformance testing and gstreamer testing are still in progress. While initial functional tests show positive results, I would appreciate early feedback on the design, implementation, and fixes before moving to a formal submission. I plan to submit a formal patch series after completing all the compliance checks. Meanwhile, any feedback or suggestion to improve this work are very welcome and will be of great help.

Gstreamer testing:
Gstreamer MR for enabling AV1 stateful decoder:
https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1

Thanks to Nicolas Dufresne for proving the MR
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469

Quoted 11 lines

However, Gst testing with parsebin is not resolving to av1parser as
below:
Ex:  With the following command parsebin is unable to resolve to
av1parser.
GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault
filesrc location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu ! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! video/x-raw,format=I420
! filesink location=gst_decoder_output.yuv

Be aware that Argon integration with fluster/GStreamer still needs some work, see issue tracker:

https://github.com/fluendo/fluster/issues/222

It is fine to proceed with testing the other (IVF or MKV based) test suites. Feel free to report all wrongly identified files onto that issue, this will be addressed in GStreamer (and eventually in FFMPEG too if need be).

Quoted 27 lines

0:00:00.051674896   400 0xffff8c000b90 DEBUG
parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps video/x-h263, variant=(string)itu

The same test with the av1parse command parses correctly:
GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault
filesrc location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu ! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! video/x-raw,format=I420
! filesink location=/tmp/gst_decoder_output.yuv

Got EOS from element "pipeline0".
Execution ended after 0:00:01.599088176
Setting pipeline to NULL ...
0:00:03.580831249  1075     0x3354f960 DEBUG av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop

Fluster testing:
As fluster.py is using parsebin for gstreamer, seeing the same issue as described above for the following testsuites. AV1-ARGON-PROFILE0-CORE-ANNEX-B AV1-ARGON-PROFILE0-NON-ANNEX-B AV1-ARGON-PROFILE0-NON-ANNEX-B

Quoted 7 lines

Test suite: AV1-TEST-VECTORS
The result of fluster test on SM8550:
134/242 testcases passed while testing AV1-TEST-VECTORS with GStreamer-AV1-V4L2-Gst1.0
unsupported content, bit depth: a000a (66 tests) Iris hardware decoder supports only 8bit NV12 av1-1-b10-00-quantizer-*

That is interesting, I believe there was no profile without 10bit, which would mean this is non-compliant hardware. Not a blocker, just a remark, and me being surprised 10bit isn't supported on modern codec in 2025.

Quoted 7 lines

Unsupported resolution (36 tests).
Iris hardware decoder supports min resolution of 96x96 av1-1-b8-01-size-*

Unsupported colorformat (1 test)
av1-1-b8-24-monochrome

Fair enough.

Quoted 4 lines

Crc mismatch: debug in progress (5tests)
av1-1-b8-03-sizeup
av1-1-b8-03-sizedown

These should be basic dynamic resolution changes cases, let me know what you find.

Quoted 1 line

av1-1-b8-16-intra_only-intrabc-extreme-dv

No idea about this one.

Quoted 2 lines

av1-1-b8-22-svc-L2T1
av1-1-b8-22-svc-L2T2

These two are spatial SVC. That means that some decode only frames are going to be smaller in dimensions. Only the last frame of a TU is displayed. Both do have the optional sequence header announcing the maximum resolution though. On stateless side, these usually works by simply decoding the hidden frames into the much larger buffers. Dealing with firmware can be harder, since firmware may silently drop the decode only frames, which leads to wrong timestamp matching, which could have side effect in frameworks.

On stateful decoding, SVC have never been done, some documentation will likely be needed. Strictly unsupported is the case the sequence header is not there, which results in resolution change on non-keyframe. We now have the ability to allocate frames of different dimension at run-time, and free old unused frames, but in stateful, decoders streamoff the capture queue, which currently results in lost of references.

Let us know your progress, inter-frame resolution change is not a strict requirement, it is quite rare in real life.

looking forward, there is nothing particularly alarming in this report. Thanks for the transparency.

Nicolas

Quoted 176 lines

Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS
12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with GStreamer-AV1-V4L2-Gst1.0
Crc mismatch: debug in progress
av1-1-b8-03-sizeup.ivf

Unsupported test suites:
Iris Hardware Decoder supports only
PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN
and 8 bit, 420 only
AV1-ARGON-PROFILE1-CORE-ANNEX-B
AV1-ARGON-PROFILE1-NON-ANNEX-B
AV1-ARGON-PROFILE1-STRESS-ANNEX-B
AV1-ARGON-PROFILE2-CORE-ANNEX-B
AV1-ARGON-PROFILE2-NON-ANNEX-B
AV1-ARGON-PROFILE2-STRESS-ANNEX-B
CHROMIUM-10bit-AV1-TEST-VECTORS

Compliance test for iris_driver device /dev/video0:

Driver Info:
        Driver name      : iris_driver
        Card type        : Iris Decoder
        Bus info         : platform:aa00000.video-codec Driver version   : 6.16.0
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar Streaming
                Extended Pix Format
        Detected Stateful Decoder

Required ioctls:
        test VIDIOC_QUERYCAP: OK
        test invalid ioctls: OK

Allow for multiple opens:
        test second /dev/video0 open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
        test VIDIOC_G/S_CTRL: OK
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 12 Private Controls: 0

Format ioctls:
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
        test Composing: OK
        test Scaling: OK (Not Supported)

Codec ioctls:
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test CREATE_BUFS maximum buffers: OK test VIDIOC_REMOVE_BUFS: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)

[65391.311675] qcom-iris aa00000.video-codec: invalid plane
[65395.340586] qcom-iris aa00000.video-codec: invalid plane

        test blocking wait: OK

Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0

V4l2-ctl Test verified for 2 streams as well.

Thanks,
Deepa

---
DEEPA GUTHYAPPA MADIVALARA (5):
      media: uapi: videodev2: Add support for AV1 stateful decoder
      media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
      media: iris: Add support for AV1 format in iris decoder
      media: iris: Add internal buffer calculation for AV1 decoder
      media: iris: Define AV1-specific platform capabilities and properties

 drivers/media/platform/qcom/iris/iris_buffer.h     |   2 + drivers/media/platform/qcom/iris/iris_ctrls.c      |   8 + drivers/media/platform/qcom/iris/iris_hfi_common.h |   3 + .../platform/qcom/iris/iris_hfi_gen2_command.c     | 109 ++++++++- .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  10 + .../platform/qcom/iris/iris_hfi_gen2_response.c    |  22 ++ drivers/media/platform/qcom/iris/iris_instance.h   |   1 + .../platform/qcom/iris/iris_platform_common.h      |  15 ++ .../media/platform/qcom/iris/iris_platform_gen2.c  | 156 ++++++++++++- .../platform/qcom/iris/iris_platform_sm8250.c      |  17 ++ drivers/media/platform/qcom/iris/iris_vdec.c       |  25 +- drivers/media/platform/qcom/iris/iris_vidc.c       |   1 + drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++- drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++ drivers/media/v4l2-core/v4l2-ioctl.c               |   1 + include/uapi/linux/videodev2.h                     |   1 +

 16 files changed, 703 insertions(+), 28 deletions(-)

---
base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f
change-id: 20250821-rfc_split-c3ff834bb2c9
prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3
prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c
prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e
prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4
prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3
prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2
prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce
prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0
prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca
prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e
prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec
prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759
prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7
prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1
prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0
prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc
prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5
prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c
prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3
prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3
prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5
prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968
prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5
prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423
prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd
prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa
prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83

Best regards,

#8 lore raw
Quoted 20 lines · Nicolas Dufresne

Le mardi 02 septembre 2025 à 16:00 -0700, DEEPA GUTHYAPPA MADIVALARA a écrit :

Quoted 19 lines

Hi all,

This patch series adds initial support for the AV1 stateful decoder codecs in iris decoder. Also it adds support for AV1 stateful decoder in V4l2. The objective of this work is to extend the Iris decoder's capabilities to handle AV1 format codec streams, including necessary format handling and buffer management. I'm sharing this series as an RFC because conformance testing and gstreamer testing are still in progress. While initial functional tests show positive results, I would appreciate early feedback on the design, implementation, and fixes before moving to a formal submission. I plan to submit a formal patch series after completing all the compliance checks. Meanwhile, any feedback or suggestion to improve this work are very welcome and will be of great help.

Gstreamer testing:
Gstreamer MR for enabling AV1 stateful decoder:
https://gitlab.freedesktop.org/dmadival/gstreamer/-/merge_requests/1

Hi Nicholas,

Thank you so much for a quick response.
Can you please provide feedback for the AV1 stateful decoder plugin - GST MR ?

Thanks to Nicolas Dufresne for proving the MR
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/9469

However, Gst testing with parsebin is not resolving to av1parser as
below:
Ex:  With the following command parsebin is unable to resolve to
av1parser.
GST_DEBUG=*:2,parsebin:6 gst-launch-1.0 --no-fault
filesrc location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu ! parsebin ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! video/x-raw,format=I420
! filesink location=gst_decoder_output.yuv

Quoted 8 lines

Be aware that Argon integration with fluster/GStreamer still needs some work, see issue tracker:

https://github.com/fluendo/fluster/issues/222

It is fine to proceed with testing the other (IVF or MKV based) test suites. Feel free to report all wrongly identified files onto that issue, this will be addressed in GStreamer (and eventually in FFMPEG too if need be).

Thanks for the above information. I will continue to work on ivf and mkv based test-suites. Is it okay to post the AV1 patch series even though Argon test-suite has failures?

Quoted 33 lines
Quoted 26 lines

0:00:00.051674896   400 0xffff8c000b90 DEBUG
parsebin gstparsebin.c:2439:type_found:<parsebin0> typefind found caps video/x-h263, variant=(string)itu

The same test with the av1parse command parses correctly:
GST_DEBUG=*:2,av1parse:6 gst-launch-1.0 --no-fault
filesrc location=/media/sd/fluster/fluster/resources/AV1-ARGON-PROFILE0-CORE-ANNEX-B/ argon_coveragetool_av1_base_and_extended_profiles_v2.1/profile0_core/streams/test10220.obu ! av1parse ! v4l2av1dec ! video/x-raw ! videoconvert dither=none ! video/x-raw,format=I420
! filesink location=/tmp/gst_decoder_output.yuv

Got EOS from element "pipeline0".
Execution ended after 0:00:01.599088176
Setting pipeline to NULL ...
0:00:03.580831249  1075     0x3354f960 DEBUG av1parse gstav1parse.c:435:gst_av1_parse_stop:<av1parse0> stop

Fluster testing:
As fluster.py is using parsebin for gstreamer, seeing the same issue as described above for the following testsuites. AV1-ARGON-PROFILE0-CORE-ANNEX-B AV1-ARGON-PROFILE0-NON-ANNEX-B AV1-ARGON-PROFILE0-NON-ANNEX-B

Quoted 7 lines

Test suite: AV1-TEST-VECTORS
The result of fluster test on SM8550:
134/242 testcases passed while testing AV1-TEST-VECTORS with GStreamer-AV1-V4L2-Gst1.0
unsupported content, bit depth: a000a (66 tests) Iris hardware decoder supports only 8bit NV12 av1-1-b10-00-quantizer-*

Quoted 3 lines

That is interesting, I believe there was no profile without 10bit, which would mean this is non-compliant hardware. Not a blocker, just a remark, and me being surprised 10bit isn't supported on modern codec in 2025.

Sorry, hardware does support 10bit. As we continue to add more features, we plan to include support for additional color formats as well.

Quoted 6 lines
Quoted 6 lines

Unsupported resolution (36 tests).
Iris hardware decoder supports min resolution of 96x96 av1-1-b8-01-size-*

Unsupported colorformat (1 test)
av1-1-b8-24-monochrome

Quoted 7 lines

Fair enough.

Quoted 3 lines

Crc mismatch: debug in progress (5tests)
av1-1-b8-03-sizeup
av1-1-b8-03-sizedown

These should be basic dynamic resolution changes cases, let me know what you find.

I have a corruption after second sequence change, which propagates further leading to hardware hang. I am working on fixing this.

Quoted 2 lines
Quoted 1 line

av1-1-b8-16-intra_only-intrabc-extreme-dv

No idea about this one.

This is passing actually. It is just an all intra bitstream.

Quoted 17 lines
Quoted 2 lines

av1-1-b8-22-svc-L2T1
av1-1-b8-22-svc-L2T2

These two are spatial SVC. That means that some decode only frames are going to be smaller in dimensions. Only the last frame of a TU is displayed. Both do have the optional sequence header announcing the maximum resolution though. On stateless side, these usually works by simply decoding the hidden frames into the much larger buffers. Dealing with firmware can be harder, since firmware may silently drop the decode only frames, which leads to wrong timestamp matching, which could have side effect in frameworks.

On stateful decoding, SVC have never been done, some documentation will likely be needed. Strictly unsupported is the case the sequence header is not there, which results in resolution change on non-keyframe. We now have the ability to allocate frames of different dimension at run-time, and free old unused frames, but in stateful, decoders streamoff the capture queue, which currently results in lost of references.

Quoted 7 lines

Let us know your progress, inter-frame resolution change is not a strict requirement, it is quite rare in real life.

looking forward, there is nothing particularly alarming in this report. Thanks for the transparency.

Nicolas

I did not have a chance into debug this yet. But these streams do have a resolution change on non-keyframe.

Quoted 5 lines
Quoted 5 lines

Testsuite: CHROMIUM-8bit-AV1-TEST-VECTORS
12/13 testcases passed while testing CHROMIUM-8bit-AV1-TEST-VECTORS with GStreamer-AV1-V4L2-Gst1.0
Crc mismatch: debug in progress
av1-1-b8-03-sizeup.ivf

Same corruption after second sequence change, which propagates further leading to hardware hang.

Quoted 169 lines
Quoted 169 lines

Unsupported test suites:
Iris Hardware Decoder supports only
PROFILE0/V4L2_MPEG_VIDEO_AV1_PROFILE_MAIN
and 8 bit, 420 only
AV1-ARGON-PROFILE1-CORE-ANNEX-B
AV1-ARGON-PROFILE1-NON-ANNEX-B
AV1-ARGON-PROFILE1-STRESS-ANNEX-B
AV1-ARGON-PROFILE2-CORE-ANNEX-B
AV1-ARGON-PROFILE2-NON-ANNEX-B
AV1-ARGON-PROFILE2-STRESS-ANNEX-B
CHROMIUM-10bit-AV1-TEST-VECTORS

Compliance test for iris_driver device /dev/video0:

Driver Info:
        Driver name      : iris_driver
        Card type        : Iris Decoder
        Bus info         : platform:aa00000.video-codec Driver version   : 6.16.0
        Capabilities     : 0x84204000
                Video Memory-to-Memory Multiplanar Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps      : 0x04204000
                Video Memory-to-Memory Multiplanar Streaming
                Extended Pix Format
        Detected Stateful Decoder

Required ioctls:
        test VIDIOC_QUERYCAP: OK
        test invalid ioctls: OK

Allow for multiple opens:
        test second /dev/video0 open: OK
        test VIDIOC_QUERYCAP: OK
        test VIDIOC_G/S_PRIORITY: OK
        test for unlimited opens: OK

Debug ioctls:
        test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
        test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
        test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
        Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
        test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
        Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
        test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
        test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
        test VIDIOC_G/S_CTRL: OK
        test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 12 Private Controls: 0

Format ioctls:
        test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
        test VIDIOC_TRY_FMT: OK
        test VIDIOC_S_FMT: OK
        test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
        test Composing: OK
        test Scaling: OK (Not Supported)

Codec ioctls:
        test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
        test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test CREATE_BUFS maximum buffers: OK test VIDIOC_REMOVE_BUFS: OK
        test VIDIOC_EXPBUF: OK
        test Requests: OK (Not Supported)

[65391.311675] qcom-iris aa00000.video-codec: invalid plane
[65395.340586] qcom-iris aa00000.video-codec: invalid plane
         test blocking wait: OK

Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0,
Warnings: 0

V4l2-ctl Test verified for 2 streams as well.

Thanks,
Deepa

---
DEEPA GUTHYAPPA MADIVALARA (5):
      media: uapi: videodev2: Add support for AV1 stateful decoder
      media: v4l2: Add description for V4L2_PIX_FMT_AV1 in v4l_fill_fmtdesc()
      media: iris: Add support for AV1 format in iris decoder
      media: iris: Add internal buffer calculation for AV1 decoder
      media: iris: Define AV1-specific platform capabilities and properties

  drivers/media/platform/qcom/iris/iris_buffer.h     |   2 +
  drivers/media/platform/qcom/iris/iris_ctrls.c      |   8 +
  drivers/media/platform/qcom/iris/iris_hfi_common.h |   3 +
  .../platform/qcom/iris/iris_hfi_gen2_command.c     | 109 ++++++++-
  .../platform/qcom/iris/iris_hfi_gen2_defines.h     |  10 +
  .../platform/qcom/iris/iris_hfi_gen2_response.c    |  22 ++
  drivers/media/platform/qcom/iris/iris_instance.h   |   1 +
  .../platform/qcom/iris/iris_platform_common.h      |  15 ++
  .../media/platform/qcom/iris/iris_platform_gen2.c  | 156 ++++++++++++-
  .../platform/qcom/iris/iris_platform_sm8250.c      |  17 ++
  drivers/media/platform/qcom/iris/iris_vdec.c       |  25 +-
  drivers/media/platform/qcom/iris/iris_vidc.c       |   1 +
  drivers/media/platform/qcom/iris/iris_vpu_buffer.c | 255 ++++++++++++++++++++-
  drivers/media/platform/qcom/iris/iris_vpu_buffer.h | 105 +++++++++
  drivers/media/v4l2-core/v4l2-ioctl.c               |   1 +
  include/uapi/linux/videodev2.h                     |   1 +
  16 files changed, 703 insertions(+), 28 deletions(-)

---
base-commit: 88a6b4187eacb700a678296afb0c610eb3781e2f
change-id: 20250821-rfc_split-c3ff834bb2c9
prerequisite-change-id: 20250704-iris-video-encoder-b193350b487a:v3
prerequisite-patch-id: 8a566690da276da34430c10dbc2fe64c1d623a9c
prerequisite-patch-id: 1430a33603b425d0b142aab98befcda771fb885e
prerequisite-patch-id: 32024cd49d2445ff396e31f40739b32597be59a4
prerequisite-patch-id: 65b569952650647174e8221dc7adde9b000a7ae3
prerequisite-patch-id: da128980fab8538bf668f19016c5121fb03759c2
prerequisite-patch-id: 079823dffbe8b89990797bf7f7640b754382d8ce
prerequisite-patch-id: 6ce10e03d7b3b96b2391e26cda703b650bde7cd0
prerequisite-patch-id: b5950670ac5068a0c5b26651ebd433f7d3bbe6ca
prerequisite-patch-id: 4de7a934f6bdfe28c84e461f70495925aa98365e
prerequisite-patch-id: 07682a6d2530b5796122bf8763f94b5bc92949ec
prerequisite-patch-id: 72b7eba20f1a222908d41323f28be3ba84106759
prerequisite-patch-id: fd9e2e1b157112c39c69486799493ee99e6033a7
prerequisite-patch-id: ae0ad8a04a04dd3434a092d4c2bb3f493417c6e1
prerequisite-patch-id: 52631eec348735d1dc5f5804b573e3cf942550a0
prerequisite-patch-id: 4109c59edb1b757162db46297914c8f7c14408dc
prerequisite-patch-id: fc0b713eb4822047e8172d11fd4cd5a097ef23a5
prerequisite-patch-id: 20ac8e7307f1f852b2a43268b2474178fbc0b94c
prerequisite-patch-id: e8419d716573beb64ad89968f0074d6bddfa86d3
prerequisite-patch-id: bdc72f5876ceb2e981d594c86a45cb21a6264af3
prerequisite-patch-id: fc5d26d01cab94d229a00eab819ae80196f3f5d5
prerequisite-patch-id: 32a9fe1371fffc9abd9a862b2814050a144d1968
prerequisite-patch-id: 25184583b5de886f78ee0444a4a59d5f3c271ce5
prerequisite-patch-id: df3376b9de27b23ae81a4c7b7a8fe4b429c32423
prerequisite-patch-id: e68fbf7c82567d2e9f3fdd0fdf2e2911329d5ccd
prerequisite-patch-id: 580fa40de01a81a8685e56420f562d299bfc60fa
prerequisite-patch-id: 89548da6690681854ee1de992a491bed73202b83

Best regards,