Skip to content

media: iris: enable VP8, MPEG2 and interlaced video support

Dmitry Baryshkov linux-arm-msm linux-media lkml 1 version In review

Get series with b4
b4 am [email protected] change-id 20260708-iris-vp8-8130cdae989d

Versions newest first

  1. v1 Latest9 patches 9 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/9media: iris: translate Gen1 profile/level to HFI enumerantsdrivers/media/platform/qcom/iris/iris_hfi_gen1_command.c drivers/media/platform/qcom/iris/iris_hfi_gen1_defines.h+144 -91 reply
2/9media: iris: account for UBWC extradata in the QC08C buffer sizedrivers/media/platform/qcom/iris/iris_buffer.c+2 -11 reply
3/9media: iris: add support for interlaced decoded contentdrivers/media/platform/qcom/iris/iris_ctrls.c drivers/media/platform/qcom/iris/iris_hfi_gen1_response.c drivers/media/platform/qcom/iris/iris_instance.h +3 more+23 -40 replies
4/9media: iris: split Gen2 firmware data for vpu2-generation SoCsdrivers/media/platform/qcom/iris/iris_hfi_gen2.c drivers/media/platform/qcom/iris/iris_platform_common.h drivers/media/platform/qcom/iris/iris_platform_vpu2.c+249 -11 reply
5/9media: iris: move the decode format list into the firmware datadrivers/media/platform/qcom/iris/iris_hfi_gen1.c drivers/media/platform/qcom/iris/iris_hfi_gen2.c drivers/media/platform/qcom/iris/iris_platform_common.h +4 more+47 -411 reply
6/9media: iris: add VP8 decode support on Gen1 firmwaredrivers/media/platform/qcom/iris/iris_ctrls.c drivers/media/platform/qcom/iris/iris_hfi_gen1.c drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +4 more+99 -21 reply
7/9media: iris: add MPEG2 decode support on Gen1 firmwaredrivers/media/platform/qcom/iris/iris_ctrls.c drivers/media/platform/qcom/iris/iris_hfi_gen1.c drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +4 more+56 -11 reply
8/9media: iris: move the encode format list into the firmware datadrivers/media/platform/qcom/iris/iris_hfi_gen1.c drivers/media/platform/qcom/iris/iris_hfi_gen2.c drivers/media/platform/qcom/iris/iris_platform_common.h +1 more+31 -140 replies
9/9media: iris: add VP8 encode support on Gen1 firmwaredrivers/media/platform/qcom/iris/iris_ctrls.c drivers/media/platform/qcom/iris/iris_hfi_gen1.c drivers/media/platform/qcom/iris/iris_hfi_gen1_command.c +1 more+41 -20 replies

Reviews trailers per patch

3 of 9 patches have a Reviewed-by or Acked-by.

PatchVishnu Reddy
1/9R-b
2/9
3/9
4/9R-b
5/9
6/9
7/9R-b
8/9
9/9

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH 0/9] media: iris: enable VP8, MPEG2 and interlaced video support

Bring back features supported by the Venus driver, but not supported by the Iris driver: VP8 (encoding, decoding), MPEG2 (decoding only) video support and also support for interlaced video streams.

On the SM8250 (RB5):

debian@debian:~$ v4l2-compliance -d /dev/video0
v4l2-compliance 1.30.1, 64 bits, 64-bit time_t

Compliance test for iris_driver device /dev/video0:

Driver Info:

        Driver name      : iris_driver
        Card type        : Iris Decoder
        Bus info         : platform:aa00000.video-codec
        Driver version   : 7.2.0
        Capabilities     : 0x84204000

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful Decoder

Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 6 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test CREATE_BUFS maximum buffers: OK test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported) test blocking wait: OK

Total for iris_driver device /dev/video0: 48, Succeeded: 48, Failed: 0, Warnings: 0

debian@debian:~$ v4l2-compliance  -d /dev/vid
v4l2-compliance 1.30.1, 64 bits, 64-bit time_t

Cannot open device /dev/vid, exiting.

debian@debian:~$ v4l2-compliance  -d /dev/video1
v4l2-compliance 1.30.1, 64 bits, 64-bit time_t

Compliance test for iris_driver device /dev/video1:

Driver Info:

        Driver name      : iris_driver
        Card type        : Iris Encoder
        Bus info         : platform:aa00000.video-codec
        Driver version   : 7.2.0
        Capabilities     : 0x84204000

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful Encoder

Required ioctls:
test VIDIOC_QUERYCAP: OK
test invalid ioctls: OK

Allow for multiple opens:
test second /dev/video1 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 34 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK
test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK
test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test CREATE_BUFS maximum buffers: OK test VIDIOC_REMOVE_BUFS: OK
test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported) test blocking wait: OK

Total for iris_driver device /dev/video1: 48, Succeeded: 48, Failed: 0, Warnings: 0

|Test|FFmpeg-VP8-v4l2m2m|GStreamer-VP8-V4L2|
|-|-|-|
|TOTAL|48/61|57/61|
|NOT SUPPORTED|0/61|0/61|
|FAIL/ERROR|13/61|4/61|
|TOTAL TIME|10.558s|9.834s|

Failing with the Gstreamer and FFMpeg:
- vp80-01-intra-1411
- vp80-02-inter-1412
- vp80-03-segmentation-1413
- vp80-05-sharpness-1443

Failing only with FFmpeg
- vp80-00-comprehensive-018
- vp80-01-intra-1416
- vp80-01-intra-1417
- vp80-03-segmentation-01
- vp80-03-segmentation-02
- vp80-03-segmentation-04
- vp80-03-segmentation-1425
- vp80-03-segmentation-1436
- vp80-05-sharpness-1439

|Test|FFmpeg-MPEG2_VIDEO-v4l2m2m|GStreamer-MPEG2_VIDEO-V4L2|
|-|-|-|
|TOTAL|36/43|30/43|
|NOT SUPPORTED|0/43|0/43|
|FAIL/ERROR|7/43|13/43|
|TOTAL TIME|71.857s|62.045s|

Failing with both Gstreamer and FFmpeg:
- att
- sony-ct1
- teracom_vlc4
- tcela-18
- ccm1
- ntr_skipped_v3

Failing with Gstreamer only:
- MEI
- MEI.stream16
- sony-ct4
- sony-ct2
- sony-ct3
- tcela-16
- tcela-19

Failing with FFmpeg only:
- test

The following H.264 tests now pass because of interlaced video being supported:
- cabac_mot_fld0_full
- cabac_mot_mbaff0_full
- cabac_mot_picaff0_full
- CAFI1_SVA_C
- CAMA1_Sony_C
- cama1_vtc_c
- cama2_vtc_b
- cama3_vtc_b
- CAMACI3_Sony_C (Gstreamer only)
- CAMASL3_Sony_B
- CAMP_MOT_MBAFF_L30
- CAMP_MOT_MBAFF_L31
- CANLMA2_Sony_C
- CANLMA3_Sony_C
- cavlc_mot_fld0_full_B
- cavlc_mot_mbaff0_full_B
- cavlc_mot_picaff0_full_B
- CVCANLMA2_Sony_C
- CVFI1_Sony_D
- CVFI1_SVA_C
- CVFI2_Sony_H
- CVFI2_SVA_C
- CVMA1_Sony_D
- CVMAPAQP3_Sony_E
- CVMAQP2_Sony_G
- CVMAQP3_Sony_D
- CVMP_MOT_FLD_L30_B
- CVMP_MOT_FRM_L31_B
- CVNLFI1_Sony_C
- CVNLFI2_Sony_H
- FI1_Sony_E
- MR6_BT_B
- MR7_BT_B
- MR8_BT_B
- MR9_BT_B
- Sharp_MP_Field_1_B
- Sharp_MP_Field_2_B
- Sharp_MP_Field_3_B
- Sharp_MP_PAFF_1r2
- Sharp_MP_PAFF_2r
- SVCHST-3-r1-L0
- HVLCFI0_Sony_B
- HVLCMFF0_Sony_B
- HVLCPFF0_Sony_B

Following tests had to be disabled because they cause firmware crashes or IOMMU faults (this behaviour matches Venus one):

- CAMA1_TOSHIBA_B
- CAMA3_Sand_E
- CAMANL1_TOSHIBA_B
- CAMANL2_TOSHIBA_B
- CAMANL3_Sand_E
- CAPAMA3_Sand_F
- CVMA1_TOSHIBA_B
- CVMANL1_TOSHIBA_B
- CVMANL2_TOSHIBA_B
- CVPA1_TOSHIBA_B
- FREXT01_JVC_D
- FREXT02_JVC_C
- FRExt2_Panasonic_C
- FRExt4_Panasonic_B
- HCAFF1_HHI_B
- HCAMFF1_HHI_B
- HPCAFLNL_BRCM_C
- HPCAFL_BRCM_C
- HPCAMAPALQ_BRCM_B
- HPCVFLNL_BRCM_A
- HPCVFL_BRCM_A
- brcm_freh5

The summary of test results can be seen at
https://github.com/linux-msm/fluster-tests/pull/2/. Test results for Kodiak (RB3 Gen2) mostly follow results on SM8250.

---
Dmitry Baryshkov (9):
media: iris: translate Gen1 profile/level to HFI enumerants
media: iris: account for UBWC extradata in the QC08C buffer size
media: iris: add support for interlaced decoded content
media: iris: split Gen2 firmware data for vpu2-generation SoCs
media: iris: move the decode format list into the firmware data
media: iris: add VP8 decode support on Gen1 firmware
media: iris: add MPEG2 decode support on Gen1 firmware
media: iris: move the encode format list into the firmware data
media: iris: add VP8 encode support on Gen1 firmware

 drivers/media/platform/qcom/iris/iris_buffer.c     |   3 +-
 drivers/media/platform/qcom/iris/iris_ctrls.c      |  33 ++-
 drivers/media/platform/qcom/iris/iris_hfi_gen1.c   |  82 ++++++
 .../platform/qcom/iris/iris_hfi_gen1_command.c     | 127 +++++++++-
 .../platform/qcom/iris/iris_hfi_gen1_defines.h     |  51 ++++
 .../platform/qcom/iris/iris_hfi_gen1_response.c    |   6 +-
 drivers/media/platform/qcom/iris/iris_hfi_gen2.c   | 279 +++++++++++++++++++++
 drivers/media/platform/qcom/iris/iris_instance.h   |   2 +
 .../platform/qcom/iris/iris_platform_common.h      |  18 +-
 .../media/platform/qcom/iris/iris_platform_vpu2.c  |  12 +-
 .../media/platform/qcom/iris/iris_platform_vpu3x.c |  17 --
 .../platform/qcom/iris/iris_platform_vpu_ar50lt.c  |   8 -
 drivers/media/platform/qcom/iris/iris_vb2.c        |   2 -
 drivers/media/platform/qcom/iris/iris_vdec.c       |  10 +-
 drivers/media/platform/qcom/iris/iris_venc.c       |  13 +-
 drivers/media/platform/qcom/iris/iris_vpu_buffer.c |  78 ++++++
 drivers/media/platform/qcom/iris/iris_vpu_buffer.h |   6 +
 17 files changed, 682 insertions(+), 65 deletions(-)

---
base-commit: b9810cd75b9fb56a3425d391cba3f608502bd474
change-id: 20260708-iris-vp8-8130cdae989d
prerequisite-change-id: 20260507-iris-ar50lt-06228469aa5b:v7
prerequisite-patch-id: 8b5a1be5778fb4c7feb069452ecb60e8b8dfc3f8
prerequisite-patch-id: e669517302297a7b315c9aa4f4189e4928091e0c
prerequisite-patch-id: 922bc325dbd9a499a3a209d2f8dbbce6a4166e60
prerequisite-patch-id: a5a49cda4e21d835558fa84f2f6bd25b8f7e72b4
prerequisite-patch-id: e62e8285bcbddade64c026379f23a8c5ae70fe52
prerequisite-patch-id: cac29b25348e2e791638a0286283109f2546f9a7
prerequisite-patch-id: 4dfaff2f9644d7a168b1b35fe947660b8f464af6
prerequisite-patch-id: a26fa49eefecf2b5ba6cf964be9327eb5ba5929a
prerequisite-patch-id: d97c79463fe1487fa7bc9eebd4cfb4610713f44a
prerequisite-patch-id: 76eafea8230160a3b5e6503c68a5c9c3c2d1497e
prerequisite-patch-id: ea9cd9a8e6561321cd9cf338a76da1382ff47e13
prerequisite-patch-id: 26e9083b709cc8f2f1184edf6b39fb89f365a9f1
prerequisite-patch-id: ce4576bd3d20880939a97a1cda9bb8678808ea65
prerequisite-patch-id: 8c94bec467b385c373bd98eb9ee5a5805fc4e6a6
prerequisite-patch-id: 6e258cd475b77927d6476704cbf91bf48a63aa0d
prerequisite-patch-id: 54c8855d0721ba8667d095e6ad6556c2021f89e9
prerequisite-patch-id: f5479d2ca6b850c0c750944a8e0b05a7df2cc5a8
prerequisite-patch-id: 96dfec1c2df69f748c1a4bd741628e946cb7bec1

Best regards,

Signature · 2 lines
With best wishes
Dmitry