support mt8188 h264 encoder
Subsystems (4)
v2 cover support mt8188 h264 encoder
mt8188 encoder hardware has a change compared to the previous chipsets, it supports 34-bit iova access. For 34-bit iova, the encoder driver needs to write all 34 bits to hardware, otherwise encoder will encode fail.
Patch 1 add the 34-bit iova supporting in driver.
Patch 2~3 add dt-bindings and driver compatible and private data. Patch 4~6 some modification for in driver.
---
compare with v1:
- split vpu_enc_encode function into 32bits and 34bits.
- some review comments modification.
---
Irui Wang (6):
media: mediatek: vcodec: Add encoder driver support for 34-bit iova
dt-bindings: media: mediatek: vcodec: Add encoder dt-bindings for
mt8188
media: mediatek: vcodec: Add mt8188 encoder driver
media: mediatek: vcodec: Remove encoder driver get IRQ resource
media: mediatek: vcodec: Fix bitstream crop information error
media: mediatek: vcodec: Use ctx vb2_queue mutex instead of device
mutex
.../media/mediatek,vcodec-encoder.yaml | 1 + .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 6 + .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 14 +- .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 22 +- .../mediatek/vcodec/venc/venc_h264_if.c | 200 +++++++++++++++--- .../platform/mediatek/vcodec/venc_ipi_msg.h | 24 +++ .../platform/mediatek/vcodec/venc_vpu_if.c | 76 ++++++- 7 files changed, 291 insertions(+), 52 deletions(-)
Signature · 1 line
2.18.0
PATCH v2 1/6 media: mediatek: vcodec: Add encoder driver support for 34-bit iova
Encoder driver got iova from IOMMU is 34-bit, for example:
Here is the sample code:
encoder input frame buffer dma address is:frm_buf =
vb2_dma_contig_plane_dma_addr(&vb2_v4l2_buffer->vb2_buf, 0);
the value of frm_buf is 0x1_ff30_0000.
encoder driver got the frm_buf and send the iova to SCP firmware through SCP IPI message, then write to encoder hardware in SCP. The iova is stored in IPI message as uint32_t data type, so the value will be truncated from *0x1_ff30_0000* to *0xff30_0000*, and then *0xff30_0000* will be written to encoder hardware, but IOMMU will help to add the high *0x1_* bit back, so IOMMU can translate the iova to PA correctly, encoder hardware can access the correct memory for encoding.
Another reason to do this is the encoder hardware can't access the 34-bit iova, IOMMU will help to add the remaining high bits of iova. But for mt8188, encoder hardware can access 34-bit iova directly, and encoder driver need write all 34 bits because IOMMU can't help driver do this if the hardware support access 34-bit iova.
For the reasons above, this patch is added to support transfer 34-bit iova between kernel and SCP encoder driver. Use uint64_t data type to store the iova, for compatibility with old chipsets, add some new struct definitions for 34-bit.
--- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 3 + .../mediatek/vcodec/venc/venc_h264_if.c | 200 +++++++++++++++--- .../platform/mediatek/vcodec/venc_ipi_msg.h | 24 +++ .../platform/mediatek/vcodec/venc_vpu_if.c | 76 ++++++- 4 files changed, 266 insertions(+), 37 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +3 -0
@@ -401,6 +401,7 @@ struct mtk_vcodec_dec_pdata {* @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats * @core_id: stand for h264 or vp8 encode index +* @uses_34bit: whether the encoder uses 34-bit iova */ struct mtk_vcodec_enc_pdata {bool uses_ext;
@@ -411,9 +412,11 @@ struct mtk_vcodec_enc_pdata {const struct mtk_video_fmt *output_formats;size_t num_output_formats;int core_id; +bool uses_34bit; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata->uses_ext) +#define MTK_ENC_IOVA_IS_34BIT (ctx) ((ctx)->dev->venc_pdata->uses_34bit)/** * struct mtk_vcodec_dev - driver data
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +167 -33
309 lines not on this page · raw message
drivers/media/platform/mediatek/vcodec/venc_ipi_msg.h +24 -0
@@ -100,6 +100,30 @@ struct venc_ap_ipi_msg_enc_ext {uint32_t data[32 ]; }; +/** +* struct venc_ap_ipi_msg_enc_ext_34 - AP to SCP extended enc cmd structure +* @msg_id: message id (AP_IPIMSG_XXX_ENC_ENCODE) +* @vpu_inst_addr: VPU encoder instance addr +* @bs_mode: bitstream mode for h264 +* @reserved: for struct padding +* @input_addr: input frame buffer 34 bit address +* @bs_addr: output bitstream buffer 34 bit address +* @bs_size: bitstream buffer size +* @data_item: number of items in the data array +* @data: data array to store the set parameters +*/ +struct venc_ap_ipi_msg_enc_ext_34 { +u32 msg_id; +u32 vpu_inst_addr; +u32 bs_mode; +u32 reserved; +u64 input_addr[3 ]; +u64 bs_addr; +u32 bs_size; +u32 data_item; +u32 data[32 ]; +}; +/** * struct venc_ap_ipi_msg_deinit - AP to VPU deinit cmd structure * @msg_id: message id (AP_IPIMSG_XXX_ENC_DEINIT)
drivers/media/platform/mediatek/vcodec/venc_vpu_if.c +72 -4
88 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v2 2/6 dt-bindings: media: mediatek: vcodec: Add encoder dt-bindings for mt8188
In reply to Irui WangAdd encoder dt-bindings for mt8188.
--- .../devicetree/bindings/media/mediatek,vcodec-encoder.yaml | 1 + 1 file changed, 1 insertion(+)
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml +1 -0
@@ -20,6 +20,7 @@ properties: - mediatek,mt8173-vcodec-enc-vp8 - mediatek,mt8173-vcodec-enc - mediatek,mt8183-vcodec-enc + - mediatek,mt8188-vcodec-enc - mediatek,mt8192-vcodec-enc - mediatek,mt8195-vcodec-enc
Signature · 1 line
2.18.0
Re: [PATCH v2, 2/6] dt-bindings: media: mediatek: vcodec: Add encoder dt-bindings for mt8188
Quoted 7 lines · Irui Wang
Add encoder dt-bindings for mt8188.
Signed-off-by Irui Wang <[email protected]>--- .../devicetree/bindings/media/mediatek,vcodec-encoder.yaml | 1 + 1 file changed, 1 insertion(+)
PATCH v2 3/6 media: mediatek: vcodec: Add mt8188 encoder driver
In reply to Irui WangAdd mt8188's compatible "mediatek,mt8188-vcodec-enc". Add mt8188's device private data "mt8188_pdata".
--- .../platform/mediatek/vcodec/mtk_vcodec_enc_drv.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +13 -0
@@ -406,6 +406,18 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = { .core_id = VENC_SYS, }; +static const struct mtk_vcodec_enc_pdata mt8188_pdata = { + .uses_ext = true, + .capture_formats = mtk_video_formats_capture_h264, + .num_capture_formats =ARRAY_SIZE (mtk_video_formats_capture_h264), + .output_formats = mtk_video_formats_output, + .num_output_formats =ARRAY_SIZE (mtk_video_formats_output), + .min_bitrate =64 , + .max_bitrate =50000000 , + .core_id =VENC_SYS , + .uses_34bit = true, +}; +static const struct mtk_vcodec_enc_pdata mt8192_pdata = { .uses_ext = true, .capture_formats = mtk_video_formats_capture_h264,
@@ -434,6 +446,7 @@ static const struct of_device_id mtk_vcodec_enc_match[] = { {.compatible = "mediatek,mt8173-vcodec-enc-vp8", .data = &mt8173_vp8_pdata}, {.compatible = "mediatek,mt8183-vcodec-enc", .data = &mt8183_pdata}, + {.compatible = "mediatek,mt8188-vcodec-enc", .data = &mt8188_pdata}, {.compatible = "mediatek,mt8192-vcodec-enc", .data = &mt8192_pdata}, {.compatible = "mediatek,mt8195-vcodec-enc", .data = &mt8195_pdata}, {},
Signature · 1 line
2.18.0
PATCH v2 4/6 media: mediatek: vcodec: Remove encoder driver get IRQ resource
In reply to Irui WangThe "platform_get_resource(pdev, IORESOURCE_IRQ, 0)" is no longer used after commit a1a2b7125e107("of/platform: Drop static setup of IRQ resource from DT core"), so just remove the function in encoder driver to avoid driver probe failed.
--- .../media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c | 8 -------- 1 file changed, 8 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0 -8
@@ -228,7 +228,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) {struct mtk_vcodec_dev *dev;struct video_device *vfd_enc; -struct resource *res;phandle rproc_phandle;enum mtk_vcodec_fw_type fw_type;int ret;
@@ -272,13 +271,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_res; } - res =platform_get_resource (pdev,IORESOURCE_IRQ ,0 ); -if (res ==NULL ) { -dev_err (&pdev->dev,"failed to get irq resource" ); - ret = -ENOENT ; - goto err_res; - } - dev->enc_irq =platform_get_irq (pdev,0 );irq_set_status_flags (dev->enc_irq,IRQ_NOAUTOEN ); ret = devm_request_irq(&pdev->dev, dev->enc_irq,
Signature · 1 line
2.18.0
Re: [PATCH v2, 4/6] media: mediatek: vcodec: Remove encoder driver get IRQ resource
Hi,
Quoted 25 lines · Irui Wang
The "
platform_get_resource(pdev, IORESOURCE_IRQ, 0)" is no longer used after commita1a2b7125e107("of/platform: Drop static setup of IRQ resource from DT core"), so just remove the function in encoder driver to avoid driver probe failed.Signed-off-by Irui Wang <[email protected]>--- .../media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c | 8 -------- 1 file changed, 8 deletions(-)drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-1@@ -228,7 +228,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) {struct mtk_vcodec_dev *dev;struct video_device *vfd_enc; -struct resource *res;phandle rproc_phandle;enum mtk_vcodec_fw_type fw_type;int ret;@@ -272,13 +271,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_res;
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c static int mtk_vcodec_probe(struct platform_device *pdev) · line 274
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-7} - res =platform_get_resource (pdev,IORESOURCE_IRQ ,0 ); -if (res ==NULL ) { -dev_err (&pdev->dev,"failed to get irq resource" ); - ret = -ENOENT ; - goto err_res; - } - dev->enc_irq =platform_get_irq (pdev,0 );
platform_get_irq() returns a negative number on failure. You should check for errors here as the previous check was removed.
The description of platform_get_irq() explicitly mentions:
Device drivers should check the return value for errors so as to not pass a negative integer value to the request_irq() APIs.
Hans has queued up this patch, so maybe you could provide a follow-up patch to fix this?
Regards
ChenYu
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-1irq_set_status_flags (dev->enc_irq,IRQ_NOAUTOEN ); ret = devm_request_irq(&pdev->dev, dev->enc_irq, --2.18.0
Re: [PATCH v2, 4/6] media: mediatek: vcodec: Remove encoder driver get IRQ resource
Dear Chen-Yu,
many thanks for your comments.
Quoted 48 lines · Chen-Yu Tsai
Hi,
Quoted 30 lines · Irui Wang
The "
platform_get_resource(pdev, IORESOURCE_IRQ, 0)" is no longer used after commita1a2b7125e107("of/platform: Drop static setup of IRQ resource from DT core"), so just remove the function in encoder driver to avoid driver probe failed.Signed-off-by Irui Wang <[email protected]>--- .../media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c | 8 ---- ---- 1 file changed, 8 deletions(-)diff --git
a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.cb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.cindexea667b867b56..6d8964fb4fa2100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-1@@ -228,7 +228,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) {struct mtk_vcodec_dev *dev;struct video_device *vfd_enc; -struct resource *res;phandle rproc_phandle;enum mtk_vcodec_fw_type fw_type;int ret;drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-0@@ -272,13 +271,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_res;drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c static int mtk_vcodec_probe(struct · line 275
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-7} - res =platform_get_resource (pdev,IORESOURCE_IRQ ,0 ); -if (res ==NULL ) { -dev_err (&pdev->dev,"failed to get irq resource" ); - ret = -ENOENT ; - goto err_res; - } - dev->enc_irq =platform_get_irq (pdev,0 );
platform_get_irq()returns a negative number on failure. You should check for errors here as the previous check was removed.
Quoted 8 lines
The description of
platform_get_irq()explicitly mentions:Device drivers should check the return value for errors so as to not pass a negative integer value to the
request_irq()APIs.Hans has queued up this patch, so maybe you could provide a follow-up patch to fix this?
Another same patch has been already merged into linux-next tree, so this current patch can be dropped.
Thanks
Best Regards
Quoted 10 lines
Regards
ChenYuQuoted 5 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-0irq_set_status_flags (dev->enc_irq,IRQ_NOAUTOEN );ret = devm_request_irq(&pdev->dev, dev->enc_irq,--
2.18.0
PATCH v2 5/6 media: mediatek: vcodec: Fix bitstream crop information error
In reply to Irui WangUsually, the real bitstream width and height will set to driver by vidioc_s_fmt, and vidioc_try_fmt() does align to get the buffer width and height, driver calculate the encoded bitstream crop information through them. The aligned resolution will be set as real resolution now if user didn't set crop info by V4L2_SEL_TGT_CROP, and the encoded bitstream may exist green line because of crop information error.
Fixs: 'b6c57d313f5f8 ("media: mtk-vcodec: venc: remove redundant code")'
--- drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c +4 -4
@@ -503,13 +503,13 @@ static int vidioc_venc_s_fmt_out(struct file *file, void *priv, f->fmt.pix.pixelformat = fmt->fourcc; } - ret =vidioc_try_fmt_out (ctx, f, fmt); + q_data->visible_width = f->fmt.pix_mp.width; + q_data->visible_height = f->fmt.pix_mp.height; + q_data->fmt = fmt; + ret =vidioc_try_fmt_out (ctx, f, q_data->fmt);if (ret)return ret; - q_data->fmt = fmt; - q_data->visible_width = f->fmt.pix_mp.width; - q_data->visible_height = f->fmt.pix_mp.height; q_data->coded_width = f->fmt.pix_mp.width; q_data->coded_height = f->fmt.pix_mp.height;
Signature · 1 line
2.18.0
PATCH v2 6/6 media: mediatek: vcodec: Use ctx vb2_queue mutex instead of device mutex
In reply to Irui WangThere is only one device mutex to lock vb2_queue when running multi-instance encoding, it can be set by each encoder context.
--- drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h | 3 +++ drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c | 6 +++--- drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c | 1 + 3 files changed, 7 insertions(+), 3 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +3 -0
@@ -324,6 +324,9 @@ struct mtk_vcodec_ctx {int hw_id;struct vdec_msg_queue msg_queue; + +/*q_mutex: vb2_queue mutex*/ +struct mutex q_mutex; };/*
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c +3 -3
@@ -1300,7 +1300,7 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx) {struct mtk_q_data *q_data; - ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex; + ctx->m2m_ctx->q_lock = &ctx->q_mutex; ctx->fh.m2m_ctx = ctx->m2m_ctx; ctx->fh.ctrl_handler = &ctx->ctrl_hdl;INIT_WORK (&ctx->encode_work, mtk_venc_worker);
@@ -1435,7 +1435,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, src_vq->ops = &mtk_venc_vb2_ops; src_vq->mem_ops = &vb2_dma_contig_memops; src_vq->timestamp_flags =V4L2_BUF_FLAG_TIMESTAMP_COPY ; - src_vq->lock = &ctx->dev->dev_mutex; + src_vq->lock = &ctx->q_mutex; src_vq->dev = &ctx->dev->plat_dev->dev; ret =vb2_queue_init (src_vq);
@@ -1449,7 +1449,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, dst_vq->ops = &mtk_venc_vb2_ops; dst_vq->mem_ops = &vb2_dma_contig_memops; dst_vq->timestamp_flags =V4L2_BUF_FLAG_TIMESTAMP_COPY ; - dst_vq->lock = &ctx->dev->dev_mutex; + dst_vq->lock = &ctx->q_mutex; dst_vq->dev = &ctx->dev->plat_dev->dev;return vb2_queue_init (dst_vq);
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +1 -0
@@ -130,6 +130,7 @@ static int fops_vcodec_open(struct file *file)INIT_LIST_HEAD (&ctx->list); ctx->dev = dev;init_waitqueue_head (&ctx->queue[0 ]); +mutex_init (&ctx->q_mutex); ctx->type =MTK_INST_ENCODER ; ret =mtk_vcodec_enc_ctrls_setup (ctx);
Signature · 1 line
2.18.0