Add Amlogic video decoder driver
Subsystems (6)
v7 cover Add Amlogic video decoder driver
Hi everyone,
[V7] The Driver was moved to staging until it can pass future specification & compliance tools.
[V6] Good news, the firmware situation is resolved. We have received a redistributable license from Amlogic and the firmwares have been merged in linux-firmware[5].
[V5] It's been a while since the last patch series regarding the Amlogic video decoder. Unfortunately, the firmware licensing situation is still on hold as we await communication from Amlogic. As such, we do not have the firmwares in linux-firmware yet.
I decided to push the V5 anyway, as changes were starting to pile up.
This patch series adds support for the Amlogic video decoder, as well as the corresponding dt bindings for GXBB/GXL/GXM chips.
It features decoding for the following formats:
- MPEG 1
- MPEG 2
The following formats will be added in future patches:
- MJPEG
- MPEG 4 (incl. Xvid, H.263)
- H.264
- HEVC (incl. 10-bit)
- VP9 (incl. 10-bit)
The code was made in such a way to allow easy inclusion of those formats in the future.
The decoder is single instance.
Files:
- vdec.c handles the V4L2 M2M logic
- esparser.c manages the hardware bitstream parser
- vdec_helpers.c provides helpers to DONE the dst buffers as well as various common code used by the codecs
- vdec_1.c manages the VDEC_1 block of the vdec IP
- vdec_ctrls.c handles the V4L2 controls exposed by the driver
- codec_mpeg12.c enables decoding for MPEG 1/2.
- vdec_platform.c links codec units with vdec units
(e.g vdec_1 with codec_mpeg12) and lists all the available src/dst formats and requirements (max width/height, etc.), per compatible chip.
It was tested primarily with ffmpeg's v4l2-m2m implementation. For instance:
$ ffmpeg -c:v mpeg2_v4l2m2m -i sample_mpeg2.mkv -f rawvideo out.nv12
The v4l2-compliance results are available below the patch diff. Tests start failing when v4l2-compliance tries to dequeue the V4L2_EVENT_SOURCE_CHANGE event, which is not supported for MPEG2 currently.
Changes since v6 [6]:
- Rebased against v5.2-rc2 (fixed build issue with missing interrupt include)
- Moved to staging - added TODO file
- Don't declare V4L2_CID_MIN_BUFFERS_FOR_CAPTURE as volatile. Use v4l2_ctrl_s_ctrl() instead.
- Fix compliance issues with V4L2_DEC_CMD_STOP, V4L2_DEC_CMD_START and vbuf->field not prepared to V4L2_FIELD_NONE.
- Fixed coccinelle issue (unneeded semicolon)
- Fixed smatch issue (PTR_ERR() could be called with 0)
Changes since v5 [4]:
- Rebased against media_tree (fixes conflict with platform/meson/Makefile following the meson CEC driver updates)
Changes since v4 [3]:
- Updated for 5.1:
- cropcap -> g_pixelaspect
- dma_zalloc_coherent -> dma_alloc_coherent
- enum_framesizes: changed from STEPWISE to CONTINUOUS, as suggested by Hans
- Reintroduced support for CREATE_BUFS. It was a driver bug where I wasn't accounting for the amount of buffers created this way (only buffers created via REQBUFS were).
- Added the patch introducing V4L2_FMT_FLAG_FIXED_RESOLUTION to the V4L2 framework, needed as MPEG2 on Amlogic hardware only supports a fixed resolution during a decode session.
- Added support for events and V4L2_EVENT_SOURCE_CHANGE, as it is now checked by v4l2-compliance for stateful decoders. I was planning on introducing this for codecs that actually use it (H264, HEVC..), but it doesn't hurt to have this in the initial commit.
- Added support for controls, mainly V4L2_CID_MIN_BUFFERS_FOR_CAPTURE
Changes since v3 [2]:
- strlcpy -> strscpy
- queue_setup: account for existing buffers when clamping *num_buffers
- removed support for CREATE_BUFS. This caused issues with gstreamer and allowed userspace to alloc more buffers than the decoder can handle in its fixed list. So for now we just disable it and only allow allocating via REQBUFS.
- rebased & tested with 4.20-rc1
Changes since v2 [1]:
- Override capture queue's min_buffers_needed in queue_setup
The HW needs the full buffer list to be available when doing start_streaming
- Fix the draining sequence
The blob that we write to the ESPARSER to trigger drain is codec-dependent. The one that was sent in v1 is specific to H.264 and isn't guaranteed to trigger drain for MPEG2. For the latter, a simple MPEG2 EOS code should be sent to the ESPARSER instead.
- Slight enhancements to the way we do vififo offset<=>timestamp matching
Changes since v1 [0]:
- use named interrupts in the bindings
- rewrite description in the bindings doc
- don't include the dts changes in the patch series
- fill the vb2 queues locks
- fill the video_device lock
- use helpers for wait_prepare and wait_finish vb2_ops
- remove unnecessary usleep in between esparser writes. Extensive testing of every codec on GXBB/GXL didn't reveal
any fails without it, so just remove it.
- compile v4l2_compliance inside the git repo
- Check for plane number/plane size to pass the latest v4l2-compliance test
- Moved the single instance check (returning -EBUSY) to start/stop streaming The check was previously in queue_setup but there was no great location to clear it except for .close().
- Slight rework of the way CAPTURE frames are timestamped for better accuracy
- Implement PAR reporting via VIDIOC_CROPCAP
[6] https://patchwork.kernel.org/cover/10943029/
[5] https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git/commit/?id=e04cc56d0e6b6ff05924ff88fdba1a438ee7d3c8
[4] https://patchwork.kernel.org/cover/10879959/
[3] https://patchwork.kernel.org/cover/10669875/
[2] https://lore.kernel.org/patchwork/cover/993093/
[1] https://patchwork.kernel.org/cover/10595803/
[0] https://patchwork.kernel.org/cover/10583391/
Maxime Jourdan (4):
dt-bindings: media: add Amlogic Video Decoder Bindings
media: videodev2: add V4L2_FMT_FLAG_FIXED_RESOLUTION
media: meson: add v4l2 m2m video decoder driver
MAINTAINERS: Add meson video decoder
.../bindings/media/amlogic,vdec.txt | 71 ++ .../media/uapi/v4l/vidioc-enum-fmt.rst | 6 + MAINTAINERS | 8 + drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/meson/vdec/Kconfig | 11 + drivers/staging/media/meson/vdec/Makefile | 8 + drivers/staging/media/meson/vdec/TODO | 8 + .../staging/media/meson/vdec/codec_mpeg12.c | 209 ++++ .../staging/media/meson/vdec/codec_mpeg12.h | 14 + drivers/staging/media/meson/vdec/dos_regs.h | 98 ++ drivers/staging/media/meson/vdec/esparser.c | 324 +++++ drivers/staging/media/meson/vdec/esparser.h | 32 + drivers/staging/media/meson/vdec/vdec.c | 1090 +++++++++++++++++ drivers/staging/media/meson/vdec/vdec.h | 269 ++++ drivers/staging/media/meson/vdec/vdec_1.c | 229 ++++ drivers/staging/media/meson/vdec/vdec_1.h | 14 + drivers/staging/media/meson/vdec/vdec_ctrls.c | 29 + drivers/staging/media/meson/vdec/vdec_ctrls.h | 14 + .../staging/media/meson/vdec/vdec_helpers.c | 449 +++++++ .../staging/media/meson/vdec/vdec_helpers.h | 83 ++ .../staging/media/meson/vdec/vdec_platform.c | 107 ++ .../staging/media/meson/vdec/vdec_platform.h | 30 + include/uapi/linux/videodev2.h | 5 +- 24 files changed, 3109 insertions(+), 2 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/amlogic,vdec.txt create mode 100644 drivers/staging/media/meson/vdec/Kconfig create mode 100644 drivers/staging/media/meson/vdec/Makefile create mode 100644 drivers/staging/media/meson/vdec/TODO create mode 100644 drivers/staging/media/meson/vdec/codec_mpeg12.c create mode 100644 drivers/staging/media/meson/vdec/codec_mpeg12.h create mode 100644 drivers/staging/media/meson/vdec/dos_regs.h create mode 100644 drivers/staging/media/meson/vdec/esparser.c create mode 100644 drivers/staging/media/meson/vdec/esparser.h create mode 100644 drivers/staging/media/meson/vdec/vdec.c create mode 100644 drivers/staging/media/meson/vdec/vdec.h create mode 100644 drivers/staging/media/meson/vdec/vdec_1.c create mode 100644 drivers/staging/media/meson/vdec/vdec_1.h create mode 100644 drivers/staging/media/meson/vdec/vdec_ctrls.c create mode 100644 drivers/staging/media/meson/vdec/vdec_ctrls.h create mode 100644 drivers/staging/media/meson/vdec/vdec_helpers.c create mode 100644 drivers/staging/media/meson/vdec/vdec_helpers.h create mode 100644 drivers/staging/media/meson/vdec/vdec_platform.c create mode 100644 drivers/staging/media/meson/vdec/vdec_platform.h
root@hardkernel-odroidc2:~# v4l2-compliance -s --stream-from=hst_2.mpg v4l2-compliance SHA: a162244d47d4bb01d0692da879dce5a070f118e7, 64 bits
Compliance test for meson-vdec device /dev/video0:
Driver Info:
Driver name : meson-vdec
Card type : Amlogic Video Decoder
Bus info : platform:meson-vdec
Driver version : 5.2.0
Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 2 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
test blocking wait: OK
fail: v4l2-test-buffers.cpp(1324): node->dqevent(ev)
test MMAP (select): FAIL
fail: v4l2-test-buffers.cpp(1324): node->dqevent(ev)
test MMAP (epoll): FAIL
test USERPTR (select): OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device
Total for meson-vdec device /dev/video0: 49, Succeeded: 47, Failed: 2, Warnings: 0
Signature · 1 line
2.21.0
PATCH v7 1/4 dt-bindings: media: add Amlogic Video Decoder Bindings
Add documentation for the meson vdec dts node.
--- .../bindings/media/amlogic,vdec.txt | 71 +++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/amlogic,vdec.txt
Documentation/devicetree/bindings/media/amlogic,vdec.txt new file +71 -0
@@ -0,0 +1,71 @@ +Amlogic Video Decoder +================================ + +The video decoding IP lies within the DOS memory region, +except for the hardware bitstream parser that makes use of an undocumented +region. + +It makes use of the following blocks: + +- ESPARSER is a bitstream parser that outputs to a VIFIFO. Further VDEC blocks +then feed from this VIFIFO. +- VDEC_1 can decode MPEG-1, MPEG-2, MPEG-4 part 2, MJPEG, H.263, H.264, VC-1. +- VDEC_HEVC can decode HEVC and VP9. + +Both VDEC_1 and VDEC_HEVC share the "vdec" IRQ and as such cannot run +concurrently. + +Device Tree Bindings: +--------------------- + +VDEC: Video Decoder +-------------------------- + +Required properties: +- compatible: value should be different for each SoC family as : + - GXBB (S905) : "amlogic,gxbb-vdec" + - GXL (S905X, S905D) : "amlogic,gxl-vdec" + - GXM (S912) : "amlogic,gxm-vdec" +- reg: base address and size of he following memory-mapped regions : + - dos + - esparser +- reg-names: should contain the names of the previous memory regions +- interrupts: should contain the following IRQs: + - vdec + - esparser +- interrupt-names: should contain the names of the previous interrupts +- amlogic,ao-sysctrl: should point to the AOBUS sysctrl node +- amlogic,canvas: should point to a canvas provider node +- clocks: should contain the following clocks : + - dos_parser + - dos + - vdec_1 + - vdec_hevc +- clock-names: should contain the names of the previous clocks +- resets: should contain the parser reset +- reset-names: should be "esparser" + +Example: + +vdec: video-decoder@c8820000 { + compatible = "amlogic,gxbb-vdec"; + reg = <0x0 0xc8820000 0x0 0x10000>, + <0x0 0xc110a580 0x0 0xe4>; + reg-names = "dos", "esparser"; + + interrupts = <GIC_SPI 44 IRQ_TYPE_EDGE_RISING>, + <GIC_SPI 32 IRQ_TYPE_EDGE_RISING>; + interrupt-names = "vdec", "esparser"; + + amlogic,ao-sysctrl = <&sysctrl_AO>; + amlogic,canvas = <&canvas>; + + clocks = <&clkc CLKID_DOS_PARSER>, + <&clkc CLKID_DOS>, + <&clkc CLKID_VDEC_1>, + <&clkc CLKID_VDEC_HEVC>; + clock-names = "dos_parser", "dos", "vdec_1", "vdec_hevc"; + + resets = <&reset RESET_PARSER>; + reset-names = "esparser"; +};
Signature · 1 line
2.21.0
PATCH v7 2/4 media: videodev2: add V4L2_FMT_FLAG_FIXED_RESOLUTION
In reply to Maxime JourdanWhen a v4l2 driver exposes V4L2_EVENT_SOURCE_CHANGE, some (usually OUTPUT) formats may not be able to trigger this event.
For instance, MPEG2 on Amlogic hardware does not support resolution switching on the fly, and a decode session must operate at a set resolution defined before the decoding start.
Add a enum_fmt format flag to tag those specific formats.
--- Documentation/media/uapi/v4l/vidioc-enum-fmt.rst | 6 ++++++ include/uapi/linux/videodev2.h | 5 +++-- 2 files changed, 9 insertions(+), 2 deletions(-)
Documentation/media/uapi/v4l/vidioc-enum-fmt.rst +6 -0
@@ -127,6 +127,12 @@ one until ``EINVAL`` is returned. - This format is not native to the device but emulated through software (usually libv4l2), where possible try to use a native format instead for better performance. + * - ``V4L2_FMT_FLAG_FIXED_RESOLUTION`` + - 0x0004 + - Dynamic resolution switching is not supported for this format, + even if the event ``V4L2_EVENT_SOURCE_CHANGE`` is supported by + the device. + Return Value
include/uapi/linux/videodev2.h +3 -2
@@ -768,8 +768,9 @@ struct v4l2_fmtdesc {__u32 reserved[4 ]; }; -#define V4L2_FMT_FLAG_COMPRESSED 0x0001 -#define V4L2_FMT_FLAG_EMULATED 0x0002 +#define V4L2_FMT_FLAG_COMPRESSED 0x0001 +#define V4L2_FMT_FLAG_EMULATED 0x0002 +#define V4L2_FMT_FLAG_FIXED_RESOLUTION 0x0004/* Frame Size and frame rate enumeration */ /*
Signature · 1 line
2.21.0
Re: [PATCH v7 2/4] media: videodev2: add V4L2_FMT_FLAG_FIXED_RESOLUTION
Hi Maxime,
I am wondering if this flag shouldn't be inverted: you set V4L2_FMT_FLAG_DYN_RESOLUTION if dynamic resolution is supported, otherwise it isn't.
Can all the existing mainlined codec drivers handle midstream resolution changes?
s5p-mfc, venus and mediatek can, but I see no SOURCE_CHANGE event in the coda drivers, so I suspect that that can't handle this.
Philipp, what is the status of the coda driver for dynamic resolution changes?
Another reason is that AFAIK all encoders are of the fixed resolution type.
And a final reason is that usually flags should indicate the presence of a feature, not the absence.
Regards,
Hans
Quoted 49 lines · Maxime Jourdan
When a v4l2 driver exposes
V4L2_EVENT_SOURCE_CHANGE, some (usually OUTPUT) formats may not be able to trigger this event.For instance, MPEG2 on Amlogic hardware does not support resolution switching on the fly, and a decode session must operate at a set resolution defined before the decoding start.
Add a
enum_fmtformat flag to tag those specific formats.Signed-off-by Maxime Jourdan <[email protected]>--- Documentation/media/uapi/v4l/vidioc-enum-fmt.rst | 6 ++++++ include/uapi/linux/videodev2.h | 5 +++-- 2 files changed, 9 insertions(+), 2 deletions(-)Documentation/media/uapi/v4l/vidioc-enum-fmt.rst +6
-0@@ -127,6 +127,12 @@ one until ``EINVAL`` is returned. - This format is not native to the device but emulated through software (usually libv4l2), where possible try to use a native format instead for better performance. + * - ``V4L2_FMT_FLAG_FIXED_RESOLUTION`` + - 0x0004 + - Dynamic resolution switching is not supported for this format, + even if the event ``V4L2_EVENT_SOURCE_CHANGE`` is supported by + the device. + Return Valueinclude/uapi/linux/videodev2.h +3
-2@@ -768,8 +768,9 @@ struct v4l2_fmtdesc {__u32 reserved[4 ]; }; -#define V4L2_FMT_FLAG_COMPRESSED 0x0001 -#define V4L2_FMT_FLAG_EMULATED 0x0002 +#define V4L2_FMT_FLAG_COMPRESSED 0x0001 +#define V4L2_FMT_FLAG_EMULATED 0x0002 +#define V4L2_FMT_FLAG_FIXED_RESOLUTION 0x0004/* Frame Size and frame rate enumeration */ /*
Re: [PATCH v7 2/4] media: videodev2: add V4L2_FMT_FLAG_FIXED_RESOLUTION
Quoted 6 lines · Hans Verkuil
Hi Maxime,
I am wondering if this flag shouldn't be inverted: you set
V4L2_FMT_FLAG_DYN_RESOLUTIONif dynamic resolution is supported, otherwise it isn't.
Quoted 8 lines
Can all the existing mainlined codec drivers handle midstream resolution changes?
s5p-mfc, venus and mediatek can, but I see no
SOURCE_CHANGEevent in the coda drivers, so I suspect that that can't handle this.Philipp, what is the status of the coda driver for dynamic resolution changes?
FTR, to my knowledge there is no dynamic resolution change support in the firmware, as there is no signal (interrupt nor picture run return value) to indicate that different headers were parsed.
I am planning to add the initial source change event required by the current decoder API documentation, but I am afraid there will be no support for source changes due to mid-stream resolution changes due to firmware limitations.
regards
Philipp
Re: [PATCH v7 2/4] media: videodev2: add V4L2_FMT_FLAG_FIXED_RESOLUTION
Quoted 16 lines · Philipp Zabel
Quoted 6 lines · Hans Verkuil
Hi Maxime,
I am wondering if this flag shouldn't be inverted: you set
V4L2_FMT_FLAG_DYN_RESOLUTIONif dynamic resolution is supported, otherwise it isn't.Quoted 8 lines
Can all the existing mainlined codec drivers handle midstream resolution changes?
s5p-mfc, venus and mediatek can, but I see no
SOURCE_CHANGEevent in the coda drivers, so I suspect that that can't handle this.Philipp, what is the status of the coda driver for dynamic resolution changes?
Quoted 8 lines
FTR, to my knowledge there is no dynamic resolution change support in the firmware, as there is no signal (interrupt nor picture run return value) to indicate that different headers were parsed.
I am planning to add the initial source change event required by the current decoder API documentation, but I am afraid there will be no support for source changes due to mid-stream resolution changes due to firmware limitations.
I'm far from familiar with this IP, but at least on CODA988, I can read from the manual that the workflow is to first guess the allocation, and if you guess it wrong, an error is returned. What seems to match the SOURCE_CHANGE event in that version would be in the picture status register, the bit 20, which is documented as triggered if the stream requires bigger buffers sizes, or more buffers. After fixing that, you should, if I read correctly, retry.
It does not notify if the buffers are too large, but you can detect, since there is register with the output stream information. This basically means that for V4L2 restriction, you'd have to bounce the buffers on frame size boundary or something like thisé
This workflow is very similar to how OMX works, but V4L2 is even less flexible on allocation vs format, forcing more re-allocation.
Quoted 3 lines
regards
Philipp
Re: [PATCH v7 2/4] media: videodev2: add V4L2_FMT_FLAG_FIXED_RESOLUTION
Quoted 34 lines · Nicolas Dufresne
Quoted 16 lines · Philipp Zabel
Quoted 6 lines · Hans Verkuil
Hi Maxime,
I am wondering if this flag shouldn't be inverted: you set
V4L2_FMT_FLAG_DYN_RESOLUTIONif dynamic resolution is supported, otherwise it isn't.Quoted 8 lines
Can all the existing mainlined codec drivers handle midstream resolution changes?
s5p-mfc, venus and mediatek can, but I see no
SOURCE_CHANGEevent in the coda drivers, so I suspect that that can't handle this.Philipp, what is the status of the coda driver for dynamic resolution changes?
Quoted 8 lines
FTR, to my knowledge there is no dynamic resolution change support in the firmware, as there is no signal (interrupt nor picture run return value) to indicate that different headers were parsed.
I am planning to add the initial source change event required by the current decoder API documentation, but I am afraid there will be no support for source changes due to mid-stream resolution changes due to firmware limitations.
I'm far from familiar with this IP, but at least on CODA988, I can read from the manual that the workflow is to first guess the allocation, and if you guess it wrong, an error is returned. What seems to match the
SOURCE_CHANGEevent in that version would be in the picture status register, the bit 20, which is documented as triggered if the stream requires bigger buffers sizes, or more buffers. After fixing that, you should, if I read correctly, retry.
Quoted 7 lines
It does not notify if the buffers are too large, but you can detect, since there is register with the output stream information. This basically means that for V4L2 restriction, you'd have to bounce the buffers on frame size boundary or something like thisé
This workflow is very similar to how OMX works, but V4L2 is even less flexible on allocation vs format, forcing more re-allocation.
Oh, actually, you already bounce the buffers through the color converter, so I guess this can all be hidden internally.
Quoted 3 lines
Quoted 3 lines
regards
Philipp
PATCH v7 3/4 media: meson: add v4l2 m2m video decoder driver
In reply to Maxime JourdanAmlogic SoCs feature a powerful video decoder unit able to decode many formats, with a performance of usually up to 4k60.
This is a driver for this IP that is based around the v4l2 m2m framework.
It features decoding for:
- MPEG 1
- MPEG 2
Supported SoCs are: GXBB (S905), GXL (S905X/W/D), GXM (S912)
There is also a hardware bitstream parser (ESPARSER) that is handled here.
--- drivers/staging/media/Kconfig | 2 + drivers/staging/media/Makefile | 1 + drivers/staging/media/meson/vdec/Kconfig | 11 + drivers/staging/media/meson/vdec/Makefile | 8 + drivers/staging/media/meson/vdec/TODO | 8 + .../staging/media/meson/vdec/codec_mpeg12.c | 209 ++++ .../staging/media/meson/vdec/codec_mpeg12.h | 14 + drivers/staging/media/meson/vdec/dos_regs.h | 98 ++ drivers/staging/media/meson/vdec/esparser.c | 324 +++++ drivers/staging/media/meson/vdec/esparser.h | 32 + drivers/staging/media/meson/vdec/vdec.c | 1090 +++++++++++++++++ drivers/staging/media/meson/vdec/vdec.h | 269 ++++ drivers/staging/media/meson/vdec/vdec_1.c | 229 ++++ drivers/staging/media/meson/vdec/vdec_1.h | 14 + drivers/staging/media/meson/vdec/vdec_ctrls.c | 29 + drivers/staging/media/meson/vdec/vdec_ctrls.h | 14 + .../staging/media/meson/vdec/vdec_helpers.c | 449 +++++++ .../staging/media/meson/vdec/vdec_helpers.h | 83 ++ .../staging/media/meson/vdec/vdec_platform.c | 107 ++ .../staging/media/meson/vdec/vdec_platform.h | 30 + 20 files changed, 3021 insertions(+) create mode 100644 drivers/staging/media/meson/vdec/Kconfig create mode 100644 drivers/staging/media/meson/vdec/Makefile create mode 100644 drivers/staging/media/meson/vdec/TODO create mode 100644 drivers/staging/media/meson/vdec/codec_mpeg12.c create mode 100644 drivers/staging/media/meson/vdec/codec_mpeg12.h create mode 100644 drivers/staging/media/meson/vdec/dos_regs.h create mode 100644 drivers/staging/media/meson/vdec/esparser.c create mode 100644 drivers/staging/media/meson/vdec/esparser.h create mode 100644 drivers/staging/media/meson/vdec/vdec.c create mode 100644 drivers/staging/media/meson/vdec/vdec.h create mode 100644 drivers/staging/media/meson/vdec/vdec_1.c create mode 100644 drivers/staging/media/meson/vdec/vdec_1.h create mode 100644 drivers/staging/media/meson/vdec/vdec_ctrls.c create mode 100644 drivers/staging/media/meson/vdec/vdec_ctrls.h create mode 100644 drivers/staging/media/meson/vdec/vdec_helpers.c create mode 100644 drivers/staging/media/meson/vdec/vdec_helpers.h create mode 100644 drivers/staging/media/meson/vdec/vdec_platform.c create mode 100644 drivers/staging/media/meson/vdec/vdec_platform.h
drivers/staging/media/Kconfig +2 -0
@@ -26,6 +26,8 @@ source "drivers/staging/media/davinci_vpfe/Kconfig" source "drivers/staging/media/imx/Kconfig" +source "drivers/staging/media/meson/vdec/Kconfig" + source "drivers/staging/media/omap4iss/Kconfig" source "drivers/staging/media/rockchip/vpu/Kconfig"
drivers/staging/media/Makefile +1 -0
@@ -2,6 +2,7 @@ obj-$(CONFIG_I2C_BCM2048) += bcm2048/ obj-$(CONFIG_VIDEO_IMX_MEDIA) += imx/ obj-$(CONFIG_VIDEO_DM365_VPFE) += davinci_vpfe/ +obj-$(CONFIG_VIDEO_MESON_VDEC) += meson/vdec/ obj-$(CONFIG_VIDEO_OMAP4) += omap4iss/ obj-$(CONFIG_VIDEO_SUNXI) += sunxi/ obj-$(CONFIG_TEGRA_VDE) += tegra-vde/
drivers/staging/media/meson/vdec/Kconfig new file +11 -0
@@ -0,0 +1,11 @@ +# SPDX-License-Identifier: GPL-2.0 + +config VIDEO_MESON_VDEC + tristate "Amlogic video decoder driver" + depends on VIDEO_DEV && VIDEO_V4L2 && HAS_DMA + depends on ARCH_MESON || COMPILE_TEST + select VIDEOBUF2_DMA_CONTIG + select V4L2_MEM2MEM_DEV + select MESON_CANVAS + help + Support for the video decoder found in gxbb/gxl/gxm chips.
drivers/staging/media/meson/vdec/Makefile new file +8 -0
@@ -0,0 +1,8 @@ +# SPDX-License-Identifier: GPL-2.0 +# Makefile for Amlogic meson video decoder driver + +meson-vdec-objs = esparser.o vdec.o vdec_ctrls.o vdec_helpers.o vdec_platform.o +meson-vdec-objs += vdec_1.o +meson-vdec-objs += codec_mpeg12.o + +obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
drivers/staging/media/meson/vdec/TODO new file +8 -0
@@ -0,0 +1,8 @@ +This driver is in staging until the V4L2 documentation about stateful video +decoders is finalized, as well as the corresponding compliance tests. + +It is at the moment not guaranteed to work properly with a userspace +stack that follows the latest version of the specification, especially +with compression standards like MPEG1/2 where the driver does not support +dynamic resolution switching, including the first one used to determine coded +resolution.
drivers/staging/media/meson/vdec/codec_mpeg12.c new file +209 -0
209 lines not on this page · raw message
drivers/staging/media/meson/vdec/codec_mpeg12.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* Copyright (C) 2018 BayLibre, SAS +* Author: Maxime Jourdan <[email protected]> +*/ + +#ifndef __MESON_VDEC_CODEC_MPEG12_H_ +#define __MESON_VDEC_CODEC_MPEG12_H_ + +#include "vdec.h" + +extern struct amvdec_codec_ops codec_mpeg12_ops; + +#endif
drivers/staging/media/meson/vdec/dos_regs.h new file +98 -0
98 lines not on this page · raw message
drivers/staging/media/meson/vdec/esparser.c new file +324 -0
324 lines not on this page · raw message
drivers/staging/media/meson/vdec/esparser.h new file +32 -0
@@ -0,0 +1,32 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* Copyright (C) 2018 BayLibre, SAS +* Author: Maxime Jourdan <[email protected]> +*/ + +#ifndef __MESON_VDEC_ESPARSER_H_ +#define __MESON_VDEC_ESPARSER_H_ + +#include <linux/platform_device.h> + +#include "vdec.h" + +int esparser_init (struct platform_device *pdev,struct amvdec_core *core); +int esparser_power_up (struct amvdec_session *sess); + +/** +* esparser_queue_eos() - write End Of Stream sequence to the ESPARSER +* +* @core vdec core struct +*/ +int esparser_queue_eos (struct amvdec_core *core,const u8 *data,u32 len); + +/** +* esparser_queue_all_src() - work handler that writes as many src buffers +* as possible to the ESPARSER +*/ +void esparser_queue_all_src (struct work_struct *work); + +#define ESPARSER_MIN_PACKET_SIZE SZ_4K + +#endif
drivers/staging/media/meson/vdec/vdec.c new file +1090 -0
1,090 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec.h new file +269 -0
269 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec_1.c new file +229 -0
229 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec_1.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* Copyright (C) 2018 BayLibre, SAS +* Author: Maxime Jourdan <[email protected]> +*/ + +#ifndef __MESON_VDEC_VDEC_1_H_ +#define __MESON_VDEC_VDEC_1_H_ + +#include "vdec.h" + +extern struct amvdec_ops vdec_1_ops; + +#endif
drivers/staging/media/meson/vdec/vdec_ctrls.c new file +29 -0
@@ -0,0 +1,29 @@ +// SPDX-License-Identifier: GPL-2.0+ +/* +* Copyright (C) 2018 BayLibre, SAS +* Author: Maxime Jourdan <[email protected]> +*/ + +#include "vdec_ctrls.h" + +int amvdec_init_ctrls (struct amvdec_session *sess) +{ +struct v4l2_ctrl_handler *ctrl_handler = &sess->ctrl_handler; +int ret; + + ret =v4l2_ctrl_handler_init (ctrl_handler,1 ); +if (ret) +return ret; + + sess->ctrl_min_buf_capture =v4l2_ctrl_new_std (ctrl_handler,NULL , +V4L2_CID_MIN_BUFFERS_FOR_CAPTURE ,1 ,32 ,1 ,1 ); + + ret = ctrl_handler->error; +if (ret) { +v4l2_ctrl_handler_free (ctrl_handler); +return ret; + } + +return 0 ; +} +EXPORT_SYMBOL_GPL (amvdec_init_ctrls);
drivers/staging/media/meson/vdec/vdec_ctrls.h new file +14 -0
@@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* Copyright (C) 2018 BayLibre, SAS +* Author: Maxime Jourdan <[email protected]> +*/ + +#ifndef __MESON_VDEC_CTRLS_H_ +#define __MESON_VDEC_CTRLS_H_ + +#include "vdec.h" + +int amvdec_init_ctrls (struct amvdec_session *sess); + +#endif
drivers/staging/media/meson/vdec/vdec_helpers.c new file +449 -0
449 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec_helpers.h new file +83 -0
83 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec_platform.c new file +107 -0
104 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec_platform.h new file +30 -0
Lines of this file are not shown. View the raw message.
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.21.0
Part of the diff is not shown here. View the raw message.
PATCH v7 4/4 MAINTAINERS: Add meson video decoder
In reply to Maxime JourdanAdd an entry for the meson video decoder for amlogic SoCs.
--- MAINTAINERS | 8 ++++++++ 1 file changed, 8 insertions(+)
MAINTAINERS +8 -0
@@ -10215,6 +10215,14 @@ S: Maintained F: drivers/mtd/nand/raw/meson_* F: Documentation/devicetree/bindings/mtd/amlogic,meson-nand.txt +MESON VIDEO DECODER DRIVER FOR AMLOGIC SOCS +M: Maxime Jourdan <[email protected]> +L: [email protected] +L: [email protected] +S: Supported +F: drivers/staging/media/meson/vdec/ +T: git git://linuxtv.org/media_tree.git + METHODE UDPU SUPPORT M: Vladimir Vid <[email protected]> S: Maintained
Signature · 1 line
2.21.0