media: meson: vdec: Add VP9 decoding support
Subsystems (4)
v5 cover media: meson: vdec: Add VP9 decoding support
Hello,
This patchset aims to bring VP9 decoding support to Amlogic GXL, G12A & SM1 platforms for the amlogic stateful video decoder driver.
With this, it passes v4l2-compliance with streaming on Amlogic G12A and Amlogic SM1 SoCs successfully using the stream at [1] with a fixed pyv4l2compliance script for VP9 at [2].
The original script kept the IVF headers in the stream, confusing the decoder. The fixed script only extracts the payload from the IVF container.
The decoder has been tested using the Google CTS TestVectorsIttiam VP9 yuv420 samples and the VP9 Decoder Performance streams at [5], decoding all streams from Profile 0 and 2 up to Level 4.1, with 10bit downsampling to 8bit.
This patchset depends on :
- H.264 and compliance v6 at [4]
Changes since v4 at [8]:
- Fixes checkpatch warning on patches 3 & 5
Changes since v3 at [7]:
- fixes necessary spare ref buffer handling in parser
- added a comment to indicate how it's handled
- fix VP9 on SM1, was working with G12A firmware, but needed some changed with SM1 specific firmware
- pushed (gxl) and switched to missing (sm1) vp9 firmwares to linux-firmware repo
Changes since v2 at [6]:
- Rebased on H.264 and compliance at [4]
Changes since v1 at [3]:
- Fixed compliance for delta frame resize, but proper ref keeping is broken
- Added warns for delta frame resize, to be fixed in a following patchset
- Added VP9 probabilities parsing and transformation support to decore the VP9 performance streams
- Fixed refs keeping, avoid deleting necessary refs for next frame
- Properly used the kernel clamp_val() macros
- Zeroed the workspace to avoid refs handling glitches
- Add lock around the flush & stop to avoid race between IRQ and drain/stop
[1] https://github.com/superna9999/pyv4l2compliance/raw/tests/output/Jellyfish_1080_10s_5MB.vp9.hdr
[2] https://github.com/superna9999/pyv4l2compliance
[3] https://lore.kernel.org/linux-media/[email protected]
[4] https://lore.kernel.org/linux-media/[email protected]
[5] https://www.webmproject.org/vp9/levels/
[6] https://lore.kernel.org/linux-media/[email protected]
[7] https://lore.kernel.org/linux-media/[email protected]
[8] https://lore.kernel.org/linux-media/[email protected]
The compliance log is:
# v4l2-compliance --stream-from-hdr Jellyfish_1080_10s_5MB.vp9.hdr -s 200 v4l2-compliance SHA: 7ead0e1856b89f2e19369af452bb03fd0cd16793, 64 bits
Compliance test for meson-vdec device /dev/video0:
Driver Info:
Driver name : meson-vdec Card type : Amlogic Video Decoder Bus info : platform:meson-vdec Driver version : 5.5.0 Capabilities : 0x84204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Detected Stateful Decoder
Required ioctls:
test VIDIOC_QUERYCAP: OK
Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK
Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)
Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0
Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0
Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)
Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 2 Private Controls: 0
Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK (Not Supported)
Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK
Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK
test Requests: OK (Not Supported)
Test input 0:
Streaming ioctls:
test read/write: OK (Not Supported)
test blocking wait: OK
Video Capture Multiplanar: Captured 198 buffers test MMAP (select): OK
Video Capture Multiplanar: Captured 198 buffers test MMAP (epoll): OK
test USERPTR (select): OK (Not Supported) test DMABUF: Cannot test, specify --expbuf-device
Total for meson-vdec device /dev/video0: 49, Succeeded: 49, Failed: 0, Warnings: 0
Maxime Jourdan (4):
media: meson: vdec: add helpers for lossless framebuffer compression
buffers
media: meson: vdec: add common HEVC decoder support
media: meson: vdec: add VP9 input support
media: meson: vdec: add VP9 decoder support
Neil Armstrong (1):
media: meson: vdec: align stride on 32 bytes
drivers/staging/media/meson/vdec/Makefile | 4 +- .../media/meson/vdec/codec_hevc_common.c | 284 +++ .../media/meson/vdec/codec_hevc_common.h | 77 + drivers/staging/media/meson/vdec/codec_vp9.c | 2142 +++++++++++++++++ drivers/staging/media/meson/vdec/codec_vp9.h | 13 + drivers/staging/media/meson/vdec/esparser.c | 150 +- drivers/staging/media/meson/vdec/hevc_regs.h | 218 ++ drivers/staging/media/meson/vdec/vdec.c | 15 +- .../staging/media/meson/vdec/vdec_helpers.c | 35 +- .../staging/media/meson/vdec/vdec_helpers.h | 4 + drivers/staging/media/meson/vdec/vdec_hevc.c | 230 ++ drivers/staging/media/meson/vdec/vdec_hevc.h | 13 + .../staging/media/meson/vdec/vdec_platform.c | 38 + 13 files changed, 3210 insertions(+), 13 deletions(-) create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.c create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.h create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.c create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.h create mode 100644 drivers/staging/media/meson/vdec/hevc_regs.h create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.c create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.h
base-commit: 1f79cb81ff15c036c996237aab684d806fa18c97
Signature · 7 lines
2.22.0 _______________________________________________ linux-amlogic mailing list [email protected] http://lists.infradead.org/mailman/listinfo/linux-amlogic
PATCH v5 1/5 media: meson: vdec: align stride on 32 bytes
The HEVC/VP9 aligns the plane stride on 32, so align the planes stride for all codecs to 32 to satisfy HEVC/VP9 decoding using the "HEVC" HW.
This fixes VP9 decoding of streams with following (not limited) widths:
- 264
-288
- 350
- 352
- 472
- 480
- 528
- 600
- 720
- 800
- 848
- 1440
--- drivers/staging/media/meson/vdec/vdec.c | 10 +++++----- drivers/staging/media/meson/vdec/vdec_helpers.c | 4 ++-- 2 files changed, 7 insertions(+), 7 deletions(-)
drivers/staging/media/meson/vdec/vdec.c +5 -5
@@ -528,20 +528,20 @@ vdec_try_fmt_common(struct amvdec_session *sess, u32 size,memset (pfmt[1 ].reserved,0 ,sizeof (pfmt[1 ].reserved));if (pixmp->pixelformat ==V4L2_PIX_FMT_NV12M ) { pfmt[0 ].sizeimage = output_size; - pfmt[0 ].bytesperline =ALIGN (pixmp->width,64 ); + pfmt[0 ].bytesperline =ALIGN (pixmp->width,32 ); pfmt[1 ].sizeimage = output_size /2 ; - pfmt[1 ].bytesperline =ALIGN (pixmp->width,64 ); + pfmt[1 ].bytesperline =ALIGN (pixmp->width,32 ); pixmp->num_planes =2 ; }else if (pixmp->pixelformat ==V4L2_PIX_FMT_YUV420M ) { pfmt[0 ].sizeimage = output_size; - pfmt[0 ].bytesperline =ALIGN (pixmp->width,64 ); + pfmt[0 ].bytesperline =ALIGN (pixmp->width,32 ); pfmt[1 ].sizeimage = output_size /4 ; - pfmt[1 ].bytesperline =ALIGN (pixmp->width,64 ) /2 ; + pfmt[1 ].bytesperline =ALIGN (pixmp->width,32 ) /2 ; pfmt[2 ].sizeimage = output_size /2 ; - pfmt[2 ].bytesperline =ALIGN (pixmp->width,64 ) /2 ; + pfmt[2 ].bytesperline =ALIGN (pixmp->width,32 ) /2 ; pixmp->num_planes =3 ; } }
drivers/staging/media/meson/vdec/vdec_helpers.c +2 -2
@@ -154,8 +154,8 @@ int amvdec_set_canvases(struct amvdec_session *sess, {struct v4l2_m2m_buffer *buf;u32 pixfmt = sess->pixfmt_cap; -u32 width =ALIGN (sess->width,64 ); -u32 height =ALIGN (sess->height,64 ); +u32 width =ALIGN (sess->width,32 ); +u32 height =ALIGN (sess->height,32 );u32 reg_cur = reg_base[0 ];u32 reg_num_cur =0 ;u32 reg_base_cur =0 ;
Signature · 1 line
2.22.0
PATCH v5 2/5 media: meson: vdec: add helpers for lossless framebuffer compression buffers
In reply to Neil ArmstrongFrom: Maxime Jourdan <[email protected]>
Add helpers to support the lossless framebuffer compression format that will be used in HEVC & VP9 decoders when decoding 10bit content for downsampling to 8bit NV12 and later proper compressed buffer support.
--- .../staging/media/meson/vdec/vdec_helpers.c | 27 +++++++++++++++++++ .../staging/media/meson/vdec/vdec_helpers.h | 4 +++ 2 files changed, 31 insertions(+)
drivers/staging/media/meson/vdec/vdec_helpers.c +27 -0
@@ -50,6 +50,33 @@ void amvdec_write_parser(struct amvdec_core *core, u32 reg, u32 val) }EXPORT_SYMBOL_GPL (amvdec_write_parser); +/* 4 KiB per 64x32 block */ +u32 amvdec_am21c_body_size (u32 width,u32 height) +{ +u32 width_64 =ALIGN (width,64 ) /64 ; +u32 height_32 =ALIGN (height,32 ) /32 ; + +return SZ_4K * width_64 * height_32; +} +EXPORT_SYMBOL_GPL (amvdec_am21c_body_size); + +/* 32 bytes per 128x64 block */ +u32 amvdec_am21c_head_size (u32 width,u32 height) +{ +u32 width_128 =ALIGN (width,128 ) /128 ; +u32 height_64 =ALIGN (height,64 ) /64 ; + +return 32 * width_128 * height_64; +} +EXPORT_SYMBOL_GPL (amvdec_am21c_head_size); + +u32 amvdec_am21c_size (u32 width,u32 height) +{ +return ALIGN (amvdec_am21c_body_size (width, height) + +amvdec_am21c_head_size (width, height),SZ_64K ); +} +EXPORT_SYMBOL_GPL (amvdec_am21c_size); +static int canvas_alloc (struct amvdec_session *sess,u8 *canvas_id) {int ret;
drivers/staging/media/meson/vdec/vdec_helpers.h +4 -0
@@ -27,6 +27,10 @@ void amvdec_clear_dos_bits(struct amvdec_core *core, u32 reg, u32 val);u32 amvdec_read_parser (struct amvdec_core *core,u32 reg);void amvdec_write_parser (struct amvdec_core *core,u32 reg,u32 val); +u32 amvdec_am21c_body_size (u32 width,u32 height); +u32 amvdec_am21c_head_size (u32 width,u32 height); +u32 amvdec_am21c_size (u32 width,u32 height); +/** * amvdec_dst_buf_done_idx() - Signal that a buffer is done decoding *
Signature · 1 line
2.22.0
PATCH v5 3/5 media: meson: vdec: add common HEVC decoder support
In reply to Neil ArmstrongFrom: Maxime Jourdan <[email protected]>
Add support for the HEVC & VP9 common decoder support, handling Amlogic GXBB, GXL, G12A and SM1 platforms.
This handles the "HEVC" hw decoder used for HEVC and VP9, and will be using in the new H264 multi-instance decoder for G12A & SM1 platforms.
--- drivers/staging/media/meson/vdec/Makefile | 4 +- .../media/meson/vdec/codec_hevc_common.c | 284 ++++++++++++++++++ .../media/meson/vdec/codec_hevc_common.h | 77 +++++ drivers/staging/media/meson/vdec/hevc_regs.h | 211 +++++++++++++ drivers/staging/media/meson/vdec/vdec_hevc.c | 230 ++++++++++++++ drivers/staging/media/meson/vdec/vdec_hevc.h | 13 + 6 files changed, 817 insertions(+), 2 deletions(-) create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.c create mode 100644 drivers/staging/media/meson/vdec/codec_hevc_common.h create mode 100644 drivers/staging/media/meson/vdec/hevc_regs.h create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.c create mode 100644 drivers/staging/media/meson/vdec/vdec_hevc.h
drivers/staging/media/meson/vdec/Makefile +2 -2
@@ -2,7 +2,7 @@ # Makefile for Amlogic meson video decoder driver meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o -meson-vdec-objs += vdec_1.o -meson-vdec-objs += codec_mpeg12.o codec_h264.o +meson-vdec-objs += vdec_1.o vdec_hevc.o +meson-vdec-objs += codec_mpeg12.o codec_h264.o codec_hevc_common.o obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
drivers/staging/media/meson/vdec/codec_hevc_common.c new file +284 -0
284 lines not on this page · raw message
drivers/staging/media/meson/vdec/codec_hevc_common.h new file +77 -0
77 lines not on this page · raw message
drivers/staging/media/meson/vdec/hevc_regs.h new file +211 -0
211 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec_hevc.c new file +230 -0
230 lines not on this page · raw message
drivers/staging/media/meson/vdec/vdec_hevc.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* Copyright (C) 2018 Maxime Jourdan <[email protected]> +*/ + +#ifndef __MESON_VDEC_VDEC_HEVC_H_ +#define __MESON_VDEC_VDEC_HEVC_H_ + +#include "vdec.h" + +extern struct amvdec_ops vdec_hevc_ops; + +#endif
Signature · 1 line
2.22.0
PATCH v5 4/5 media: meson: vdec: add VP9 input support
In reply to Neil ArmstrongFrom: Maxime Jourdan <[email protected]>
Amlogic VP9 decoder requires an additional 16-byte payload before every frame header.
The source buffer is updated in-place, then given to the Parser FIFO DMA.
The FIFO DMA copies the blocks into the 16MiB parser ring buffer, then parses and copies the slice into the decoder "workspace".
--- drivers/staging/media/meson/vdec/esparser.c | 150 +++++++++++++++++++- 1 file changed, 146 insertions(+), 4 deletions(-)
drivers/staging/media/meson/vdec/esparser.c +146 -4
@@ -52,6 +52,7 @@#define PARSER_VIDEO_HOLE 0x90#define SEARCH_PATTERN_LEN 512 +#define VP9_HEADER_SIZE 16static DECLARE_WAIT_QUEUE_HEAD (wq);static int search_done;
@@ -74,14 +75,121 @@ static irqreturn_t esparser_isr(int irq, void *dev)return IRQ_HANDLED ; } +/** +* VP9 frame headers need to be appended by a 16-byte long +* Amlogic custom header +*/ +static int vp9_update_header (struct amvdec_core *core,struct vb2_buffer *buf) +{ +u8 *dp; +u8 marker; +int dsize; +int num_frames, cur_frame; +int cur_mag, mag, mag_ptr; +int frame_size[8 ], tot_frame_size[8 ]; +int total_datasize =0 ; +int new_frame_size; +unsigned char *old_header =NULL ; + + dp = (uint8_t *)vb2_plane_vaddr (buf,0 ); + dsize =vb2_get_plane_payload (buf,0 ); + +if (dsize ==vb2_plane_size (buf,0 )) { +dev_warn (core->dev,"%s: unable to update header\n" , __func__); +return 0 ; + } + + marker = dp[dsize -1 ]; +if ((marker &0xe0 ) ==0xc0 ) { + num_frames = (marker &0x7 ) +1 ; + mag = ((marker >>3 ) &0x3 ) +1 ; + mag_ptr = dsize - mag * num_frames -2 ; +if (dp[mag_ptr] != marker) +return 0 ; + + mag_ptr++; +for (cur_frame =0 ; cur_frame < num_frames; cur_frame++) { + frame_size[cur_frame] =0 ; +for (cur_mag =0 ; cur_mag < mag; cur_mag++) { + frame_size[cur_frame] |= + (dp[mag_ptr] << (cur_mag *8 )); + mag_ptr++; + } +if (cur_frame ==0 ) + tot_frame_size[cur_frame] = + frame_size[cur_frame]; +else + tot_frame_size[cur_frame] = + tot_frame_size[cur_frame -1 ] + + frame_size[cur_frame]; + total_datasize += frame_size[cur_frame]; + } + }else { + num_frames =1 ; + frame_size[0 ] = dsize; + tot_frame_size[0 ] = dsize; + total_datasize = dsize; + } + + new_frame_size = total_datasize + num_frames *VP9_HEADER_SIZE ; + +if (new_frame_size >=vb2_plane_size (buf,0 )) { +dev_warn (core->dev,"%s: unable to update header\n" , __func__); +return 0 ; + } + +for (cur_frame = num_frames -1 ; cur_frame >=0 ; cur_frame--) { +int framesize = frame_size[cur_frame]; +int framesize_header = framesize +4 ; +int oldframeoff = tot_frame_size[cur_frame] - framesize; +int outheaderoff = oldframeoff + cur_frame *VP9_HEADER_SIZE ; +u8 *fdata = dp + outheaderoff; +u8 *old_framedata = dp + oldframeoff; + +memmove (fdata +VP9_HEADER_SIZE , old_framedata, framesize); + + fdata[0 ] = (framesize_header >>24 ) &0xff ; + fdata[1 ] = (framesize_header >>16 ) &0xff ; + fdata[2 ] = (framesize_header >>8 ) &0xff ; + fdata[3 ] = (framesize_header >>0 ) &0xff ; + fdata[4 ] = ((framesize_header >>24 ) &0xff ) ^0xff ; + fdata[5 ] = ((framesize_header >>16 ) &0xff ) ^0xff ; + fdata[6 ] = ((framesize_header >>8 ) &0xff ) ^0xff ; + fdata[7 ] = ((framesize_header >>0 ) &0xff ) ^0xff ; + fdata[8 ] =0 ; + fdata[9 ] =0 ; + fdata[10 ] =0 ; + fdata[11 ] =1 ; + fdata[12 ] ='A' ; + fdata[13 ] ='M' ; + fdata[14 ] ='L' ; + fdata[15 ] ='V' ; + +if (!old_header) { +/* nothing */ + }else if (old_header > fdata +16 + framesize) { +dev_dbg (core->dev,"%s: data has gaps, setting to 0\n" , + __func__); +memset (fdata +16 + framesize,0 , + (old_header - fdata +16 + framesize)); + }else if (old_header < fdata +16 + framesize) { +dev_err (core->dev,"%s: data overwritten\n" , __func__); + } + old_header = fdata; + } + +return new_frame_size; +} +/* Pad the packet to at least 4KiB bytes otherwise the VDEC unit won't trigger * ISRs. * Also append a start code 000001ff at the end to trigger * the ESPARSER interrupt. */ -static u32 esparser_pad_start_code (struct amvdec_core *core,struct vb2_buffer *vb) +static u32 esparser_pad_start_code (struct amvdec_core *core, +struct vb2_buffer *vb, +u32 payload_size) { -u32 payload_size =vb2_get_plane_payload (vb,0 );u32 pad_size =0 ;u8 *vaddr =vb2_plane_vaddr (vb,0 );
@@ -186,13 +294,35 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf)int ret;struct vb2_buffer *vb = &vbuf->vb2_buf;struct amvdec_core *core = sess->core; +struct amvdec_codec_ops *codec_ops = sess->fmt_out->codec_ops;u32 payload_size =vb2_get_plane_payload (vb,0 );dma_addr_t phy =vb2_dma_contig_plane_dma_addr (vb,0 ); +u32 num_dst_bufs =0 ;u32 offset;u32 pad_size; -if (esparser_vififo_get_free_space (sess) < payload_size) +/* +* When max ref frame is held by VP9, this should be -= 3 to prevent a +* shortage of CAPTURE buffers on the decoder side. +* For the future, a good enhancement of the way this is handled could +* be to notify new capture buffers to the decoding modules, so that +* they could pause when there is no capture buffer available and +* resume on this notification. +*/ +if (sess->fmt_out->pixfmt ==V4L2_PIX_FMT_VP9 ) { +if (codec_ops->num_pending_bufs) + num_dst_bufs = codec_ops->num_pending_bufs (sess); + + num_dst_bufs +=v4l2_m2m_num_dst_bufs_ready (sess->m2m_ctx); +if (sess->fmt_out->pixfmt ==V4L2_PIX_FMT_VP9 ) + num_dst_bufs -=3 ; + +if (esparser_vififo_get_free_space (sess) < payload_size || +atomic_read (&sess->esparser_queued_bufs) >= num_dst_bufs) +return -EAGAIN ; + }else if (esparser_vififo_get_free_space (sess) < payload_size) {return -EAGAIN ; + }v4l2_m2m_src_buf_remove_by_buf (sess->m2m_ctx, vbuf);
@@ -206,7 +336,19 @@ esparser_queue(struct amvdec_session *sess, struct vb2_v4l2_buffer *vbuf) vbuf->field =V4L2_FIELD_NONE ; vbuf->sequence = sess->sequence_out++; - pad_size =esparser_pad_start_code (core, vb); +if (sess->fmt_out->pixfmt ==V4L2_PIX_FMT_VP9 ) { + payload_size =vp9_update_header (core, vb); + +/* If unable to alter buffer to add headers */ +if (payload_size ==0 ) { +amvdec_remove_ts (sess, vb->timestamp); +v4l2_m2m_buf_done (vbuf,VB2_BUF_STATE_ERROR ); + +return 0 ; + } + } + + pad_size =esparser_pad_start_code (core, vb, payload_size); ret =esparser_write_data (core, phy, payload_size + pad_size);if (ret <=0 ) {
Signature · 1 line
2.22.0
PATCH v5 5/5 media: meson: vdec: add VP9 decoder support
In reply to Neil ArmstrongFrom: Maxime Jourdan <[email protected]>
This adds VP9 decoding for the Amlogic GXL, G12A & SM1 SoCs, using the commong "HEVC" HW decoder.
For G12A & SM1, it uses the IOMMU support from the firmware.
For 10bit decoding, the firmware can only decode in the proprietary Amlogic Framebuffer Compression format, but can output in 8bit NV12 buffer while writing the decoded frame.
--- drivers/staging/media/meson/vdec/Makefile | 2 +- drivers/staging/media/meson/vdec/codec_vp9.c | 2142 +++++++++++++++++ drivers/staging/media/meson/vdec/codec_vp9.h | 13 + drivers/staging/media/meson/vdec/hevc_regs.h | 7 + drivers/staging/media/meson/vdec/vdec.c | 5 + .../staging/media/meson/vdec/vdec_helpers.c | 4 + .../staging/media/meson/vdec/vdec_platform.c | 38 + 7 files changed, 2210 insertions(+), 1 deletion(-) create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.c create mode 100644 drivers/staging/media/meson/vdec/codec_vp9.h
drivers/staging/media/meson/vdec/Makefile +1 -1
@@ -3,6 +3,6 @@ meson-vdec-objs = esparser.o vdec.o vdec_helpers.o vdec_platform.o meson-vdec-objs += vdec_1.o vdec_hevc.o -meson-vdec-objs += codec_mpeg12.o codec_h264.o codec_hevc_common.o +meson-vdec-objs += codec_mpeg12.o codec_h264.o codec_hevc_common.o codec_vp9.o obj-$(CONFIG_VIDEO_MESON_VDEC) += meson-vdec.o
drivers/staging/media/meson/vdec/codec_vp9.c new file +2142 -0
2,142 lines not on this page · raw message
drivers/staging/media/meson/vdec/codec_vp9.h new file +13 -0
@@ -0,0 +1,13 @@ +/* SPDX-License-Identifier: GPL-2.0+ */ +/* +* Copyright (C) 2018 Maxime Jourdan <[email protected]> +*/ + +#ifndef __MESON_VDEC_CODEC_VP9_H_ +#define __MESON_VDEC_CODEC_VP9_H_ + +#include "vdec.h" + +extern struct amvdec_codec_ops codec_vp9_ops; + +#endif
drivers/staging/media/meson/vdec/hevc_regs.h +7 -0
@@ -122,6 +122,8 @@#define HEVC_MPRED_L0_REF00_POC 0xc880#define HEVC_MPRED_L1_REF00_POC 0xc8c0 +#define HEVC_MPRED_CTRL4 0xc930 +#define HEVC_MPRED_CUR_POC 0xc980#define HEVC_MPRED_COL_POC 0xc984#define HEVC_MPRED_MV_RD_END_ADDR 0xc988
@@ -140,6 +142,10 @@#define HEVCD_IPP_LINEBUFF_BASE 0xd024#define HEVCD_IPP_AXIIF_CONFIG 0xd02c +#define VP9D_MPP_REF_SCALE_ENBL 0xd104 +#define VP9D_MPP_REFINFO_TBL_ACCCONFIG 0xd108 +#define VP9D_MPP_REFINFO_DATA 0xd10c +#define HEVCD_MPP_ANC2AXI_TBL_CONF_ADDR 0xd180#define HEVCD_MPP_ANC2AXI_TBL_CMD_ADDR 0xd184#define HEVCD_MPP_ANC2AXI_TBL_DATA 0xd190
@@ -164,6 +170,7 @@#define HEVC_DBLK_CFG9 0xd424#define HEVC_DBLK_CFGA 0xd428#define HEVC_DBLK_STS0 0xd42c +#define HEVC_DBLK_CFGB 0xd42c#define HEVC_DBLK_STS1 0xd430#define HEVC_DBLK_CFGE 0xd438
drivers/staging/media/meson/vdec/vdec.c +5 -0
@@ -395,6 +395,7 @@ static void vdec_reset_bufs_recycle(struct amvdec_session *sess)static void vdec_stop_streaming (struct vb2_queue *q) {struct amvdec_session *sess =vb2_get_drv_priv (q); +struct amvdec_codec_ops *codec_ops = sess->fmt_out->codec_ops;struct amvdec_core *core = sess->core;struct vb2_v4l2_buffer *buf;
@@ -423,6 +424,10 @@ static void vdec_stop_streaming(struct vb2_queue *q) sess->streamon_out =0 ; }else { +/* Drain remaining refs if was still running */ +if (sess->status >=STATUS_RUNNING && codec_ops->drain) + codec_ops->drain (sess); +while ((buf =v4l2_m2m_dst_buf_remove (sess->m2m_ctx)))v4l2_m2m_buf_done (buf,VB2_BUF_STATE_ERROR );
drivers/staging/media/meson/vdec/vdec_helpers.c +4 -0
@@ -299,6 +299,10 @@ static void dst_buf_done(struct amvdec_session *sess, sess->sequence_cap - 1);v4l2_event_queue_fh (&sess->fh, &ev); vbuf->flags |=V4L2_BUF_FLAG_LAST ; + }else if (sess->status ==STATUS_NEEDS_RESUME ) { +/* Mark LAST for drained show frames during a source change */ + vbuf->flags |=V4L2_BUF_FLAG_LAST ; + sess->sequence_cap =0 ; }else if (sess->should_stop)dev_dbg (dev,"should_stop, %u bufs remain\n" ,atomic_read (&sess->esparser_queued_bufs));
drivers/staging/media/meson/vdec/vdec_platform.c +38 -0
@@ -8,8 +8,10 @@#include "vdec.h" #include "vdec_1.h" +#include "vdec_hevc.h" #include "codec_mpeg12.h" #include "codec_h264.h" +#include "codec_vp9.h" static const struct amvdec_format vdec_formats_gxbb[] = { {
@@ -51,6 +53,18 @@ static const struct amvdec_format vdec_formats_gxbb[] = {static const struct amvdec_format vdec_formats_gxl[] = { { + .pixfmt =V4L2_PIX_FMT_VP9 , + .min_buffers =16 , + .max_buffers =24 , + .max_width =3840 , + .max_height =2160 , + .vdec_ops = &vdec_hevc_ops, + .codec_ops = &codec_vp9_ops, + .firmware_path ="meson/vdec/gxl_vp9.bin" , + .pixfmts_cap = {V4L2_PIX_FMT_NV12M ,0 }, + .flags =V4L2_FMT_FLAG_COMPRESSED | +V4L2_FMT_FLAG_DYN_RESOLUTION , + }, { .pixfmt =V4L2_PIX_FMT_H264 , .min_buffers =2 , .max_buffers =24 ,
@@ -127,6 +141,18 @@ static const struct amvdec_format vdec_formats_gxm[] = {static const struct amvdec_format vdec_formats_g12a[] = { { + .pixfmt =V4L2_PIX_FMT_VP9 , + .min_buffers =16 , + .max_buffers =24 , + .max_width =3840 , + .max_height =2160 , + .vdec_ops = &vdec_hevc_ops, + .codec_ops = &codec_vp9_ops, + .firmware_path ="meson/vdec/g12a_vp9.bin" , + .pixfmts_cap = {V4L2_PIX_FMT_NV12M ,0 }, + .flags =V4L2_FMT_FLAG_COMPRESSED | +V4L2_FMT_FLAG_DYN_RESOLUTION , + }, { .pixfmt =V4L2_PIX_FMT_H264 , .min_buffers =2 , .max_buffers =24 ,
@@ -165,6 +191,18 @@ static const struct amvdec_format vdec_formats_g12a[] = {static const struct amvdec_format vdec_formats_sm1[] = { { + .pixfmt =V4L2_PIX_FMT_VP9 , + .min_buffers =16 , + .max_buffers =24 , + .max_width =3840 , + .max_height =2160 , + .vdec_ops = &vdec_hevc_ops, + .codec_ops = &codec_vp9_ops, + .firmware_path ="meson/vdec/sm1_vp9_mmu.bin" , + .pixfmts_cap = {V4L2_PIX_FMT_NV12M ,0 }, + .flags =V4L2_FMT_FLAG_COMPRESSED | +V4L2_FMT_FLAG_DYN_RESOLUTION , + }, { .pixfmt =V4L2_PIX_FMT_H264 , .min_buffers =2 , .max_buffers =24 ,
Signature · 1 line
2.22.0
Quoted 9 lines · Neil Armstrong
Hello,
This patchset aims to bring VP9 decoding support to Amlogic GXL, G12A & SM1 platforms for the amlogic stateful video decoder driver.
With this, it passes v4l2-compliance with streaming on Amlogic G12A and Amlogic SM1 SoCs successfully using the stream at [1] with a fixed pyv4l2compliance script for VP9 at [2].
Quoted 6 lines
The original script kept the IVF headers in the stream, confusing the decoder. The fixed script only extracts the payload from the IVF container.
The decoder has been tested using the Google CTS TestVectorsIttiam VP9 yuv420 samples and the VP9 Decoder Performance streams at [5], decoding all streams from Profile 0 and 2 up to Level 4.1, with 10bit downsampling to 8bit.
Tested on meson-sm1-sei610,
Kevin