Enable two H264 encoder cores on MT8195
Subsystems (4)
v2 cover Enable two H264 encoder cores on MT8195
MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps.
The two encoder core encoding process looks like this:
VENC Core0: frm#0....frm#2....frm#4....
VENC Core1: ..frm#1....frm#3....frm#5....This series of patches are used to enable the two H264 encoder cores, encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
...
To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.
There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance.
---
This series patches dependent on:
[1]: the latest linux stage tree: https://git.linuxtv.org/media_stage.git
mtk decoder patches
[2]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7105
[3]: https://patchwork.linuxtv.org/project/linux-media/list/?series=7131
new yaml included files
[4]:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641
[5]:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579
---
---
changes compared with v1:
- of_platform_populate was used in place of the component framework.
- new yaml file for venc cores.
- some modifications for patch v1's review comments.
---
Irui Wang (10):
media: mtk-vcodec: Use core type to indicate h264 and vp8 enc
media: mtk-vcodec: export encoder functions
dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for
mt8195
media: mtk-vcodec: Enable venc dual core usage
media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface
media: mtk-vcodec: Add venc power on/off interface
media: mtk-vcodec: Rewrite venc clock interface
media: mtk-vcodec: Add more extra processing for dual-core mode
media: mtk-vcodec: Add dual core mode encode process
media: mtk-vcodec: Done encode result to client
.../media/mediatek,vcodec-encoder-core.yaml | 214 +++++++++++++++++ drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 44 +++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 109 ++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 187 +++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 118 ++++++---- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 187 +++++++++++++-- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 216 +++++++++++++++--- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 +- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 18 files changed, 1097 insertions(+), 162 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h
Signature · 1 line
2.18.0
PATCH v2 1/10 media: mtk-vcodec: Use core type to indicate h264 and vp8 enc
use core type variables to indicate h264 and vp8 encoder
--- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 4 +-- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 30 ++++++++++--------- 2 files changed, 18 insertions(+), 16 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2 -2
@@ -419,7 +419,7 @@ struct mtk_vcodec_dec_pdata {* @num_capture_formats: number of entries in capture_formats * @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats -* @core_id: stand for h264 or vp8 encode index +* @core_type: stand for h264 or vp8 encode */ struct mtk_vcodec_enc_pdata {bool uses_ext;
@@ -429,7 +429,7 @@ struct mtk_vcodec_enc_pdata {size_t num_capture_formats;const struct mtk_video_fmt *output_formats;size_t num_output_formats; -int core_id; +int core_type; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata->uses_ext)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +16 -14
@@ -95,11 +95,13 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv) ctx = dev->curr_ctx;spin_unlock_irqrestore (&dev->irqlock, flags); -mtk_v4l2_debug (1 ,"id=%d coreid:%d" , ctx->id, dev->venc_pdata->core_id); - addr = dev->reg_base[dev->venc_pdata->core_id] + +mtk_v4l2_debug (1 ,"id: %d core type: %d" , + ctx->id, dev->venc_pdata->core_type); + + addr = dev->reg_base[dev->venc_pdata->core_type] +MTK_VENC_IRQ_ACK_OFFSET ; - ctx->irq_status =readl (dev->reg_base[dev->venc_pdata->core_id] + + ctx->irq_status =readl (dev->reg_base[dev->venc_pdata->core_type] + (MTK_VENC_IRQ_STATUS_OFFSET ));clean_irq_status (ctx->irq_status, addr);
@@ -266,10 +268,10 @@ static int mtk_vcodec_probe(struct platform_device *pdev)pm_runtime_enable (&pdev->dev); - dev->reg_base[dev->venc_pdata->core_id] = + dev->reg_base[dev->venc_pdata->core_type] =devm_platform_ioremap_resource (pdev,0 ); -if (IS_ERR (dev->reg_base[dev->venc_pdata->core_id])) { - ret =PTR_ERR (dev->reg_base[dev->venc_pdata->core_id]); +if (IS_ERR (dev->reg_base[dev->venc_pdata->core_type])) { + ret =PTR_ERR (dev->reg_base[dev->venc_pdata->core_type]); goto err_res; }
@@ -287,8 +289,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev) 0, pdev->name, dev);if (ret) {dev_err (&pdev->dev, -"Failed to install dev->enc_irq %d (%d) core_id (%d)" , - dev->enc_irq, ret, dev->venc_pdata->core_id); +"Failed to install dev->enc_irq %d (%d) core_type (%d)" , + dev->enc_irq, ret, dev->venc_pdata->core_type); ret = -EINVAL ; goto err_res; }
@@ -357,7 +359,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) }mtk_v4l2_debug (0 ,"encoder %d registered as /dev/video%d" , - dev->venc_pdata->core_id, vfd_enc->num); + dev->venc_pdata->core_type, vfd_enc->num);return 0 ;
@@ -384,7 +386,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_avc_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 60000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
@@ -394,7 +396,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 9000000, - .core_id = VENC_LT_SYS, + .core_type = VENC_LT_SYS, };static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
@@ -405,7 +407,7 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 40000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
@@ -416,7 +418,7 @@ static const struct mtk_vcodec_enc_pdata mt8192_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 100000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
@@ -427,7 +429,7 @@ static const struct mtk_vcodec_enc_pdata mt8195_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 100000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct of_device_id mtk_vcodec_enc_match[] = {
Signature · 1 line
2.18.0
PATCH v2 2/10 media: mtk-vcodec: export encoder functions
In reply to Irui Wangmtk vcodec is built as a module, export some functions to make them visible by other modules.
--- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 2 ++ drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 3 +++ 2 files changed, 5 insertions(+)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +2 -0
@@ -1432,6 +1432,7 @@ int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)mutex_unlock (&dev->enc_mutex);return 0 ; } +EXPORT_SYMBOL_GPL (mtk_venc_unlock);int mtk_venc_lock (struct mtk_vcodec_ctx *ctx) {
@@ -1440,6 +1441,7 @@ int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)mutex_lock (&dev->enc_mutex);return 0 ; } +EXPORT_SYMBOL_GPL (mtk_venc_lock);void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx) {
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +3 -0
@@ -85,6 +85,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)put_device (pm->larbvenc);return ret; } +EXPORT_SYMBOL_GPL (mtk_vcodec_init_enc_clk);void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm) {
@@ -111,6 +112,7 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)for (i -=1 ; i >=0 ; i--)clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_on);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm) {
@@ -121,3 +123,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)for (i = enc_clk->clk_num -1 ; i >=0 ; i--)clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_off);
Signature · 1 line
2.18.0
PATCH v2 3/10 dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Irui WangAdds encoder cores dt-bindings for mt8195
--- .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ 1 file changed, 214 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +214 -0
@@ -0,0 +1,214 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present in Mediatek + SoCs which supports high resolution encoding functionalities. Required + parent and child device node. + +properties: + compatible: + const: mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + maxItems: 1 + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps to memory. + + "#address-cells": + const: 1 + + "#size-cells": + const: 1 + + ranges: true + +# Required child node: +patternProperties: + "venc_core0@1a020000": + type: object + + properties: + compatible: + const: mediatek,mtk-venc-core0 + + reg: + maxItems: 1 + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + items: + - const: MT_CG_VENC0 + + assigned-clocks: + maxItems: 1 + + assigned-clock-parents: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + + "venc_core1@1b020000": + type: object + + properties: + compatible: + const: mediatek,mtk-venc-core1 + + reg: + maxItems: 1 + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + items: + - const: MT_CG_VENC1 + + assigned-clocks: + maxItems: 1 + + assigned-clock-parents: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + - ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <1>; + #size-cells = <1>; + ranges; + + venc_core0@1a020000 { + compatible = "mediatek,mtk-venc-core0"; + reg = <0x1a020000 0x10000>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "MT_CG_VENC0"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc_core1@1b020000 { + compatible = "mediatek,mtk-venc-core1"; + reg = <0x1b020000 0x10000>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "MT_CG_VENC1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + };
Signature · 1 line
2.18.0
PATCH v2 4/10 media: mtk-vcodec: Enable venc dual core usage
In reply to Irui WangAdds new venc core mode to indicate different venc hardware mode:VENC_SINGLE_CORE_MODE means only one core, the device has its own power/clk/irq, init_clk/request_irq helper can be used.
VENC_DUAL_CORE_MODE means more than one core inside, the core device can use the init_clk/request_irq helper to initialize their own power/clk/irq. And the main device doesn't need use these helper anymore.
MT8195 has two H264 venc cores, enable dual_core_mode for it.
--- drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 22 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 153 ++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 88 +++++----- 5 files changed, 266 insertions(+), 37 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h
drivers/media/platform/mtk-vcodec/Makefile +3 -1
@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk-vcodec-enc-core.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \
@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +22 -0
@@ -117,6 +117,23 @@ enum mtk_vdec_hw_count { MTK_VDEC_MAX_HW_COUNT, }; +/* +* enum mtk_venc_core_id -- encoder core id +*/ +enum mtk_venc_core_id { +MTK_VENC_CORE0 =0 , +MTK_VENC_CORE1 =1 , +MTK_VENC_CORE_MAX , +}; + +/** +* enmu mtk_venc_core_mode - Used to indicate different encode mode +*/ +enum mtk_venc_core_mode { +VENC_SINGLE_CORE_MODE =0 , +VENC_DUAL_CORE_MODE =1 , +}; +/* * struct mtk_video_fmt - Structure used to store information about pixelformats */
@@ -420,6 +437,7 @@ struct mtk_vcodec_dec_pdata {* @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats * @core_type: stand for h264 or vp8 encode +* @core_mode: indicate encode core mode */ struct mtk_vcodec_enc_pdata {bool uses_ext;
@@ -430,6 +448,7 @@ struct mtk_vcodec_enc_pdata {const struct mtk_video_fmt *output_formats;size_t num_output_formats;int core_type; +enum mtk_venc_core_mode core_mode; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata->uses_ext)
@@ -479,6 +498,7 @@ struct mtk_vcodec_enc_pdata {* @subdev_dev: subdev hardware device * @subdev_prob_done: check whether all used hw device is prob done * @subdev_bitmap: used to record hardware is ready or not +* @enc_core_dev: used to store venc core device */ struct mtk_vcodec_dev {struct v4l2_device v4l2_dev;
@@ -524,6 +544,8 @@ struct mtk_vcodec_dev {void *subdev_dev[MTK_VDEC_HW_MAX ];int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev);DECLARE_BITMAP (subdev_bitmap,MTK_VDEC_HW_MAX ); + +void *enc_core_dev[MTK_VENC_CORE_MAX ]; };static inline struct mtk_vcodec_ctx *fh_to_ctx (struct v4l2_fh *fh)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +153 -0
153 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#ifndef _MTK_VCODEC_ENC_CORE_H_ +#define _MTK_VCODEC_ENC_CORE_H_ + +#include <linux/platform_device.h> +#include "mtk_vcodec_drv.h" + +/* +* struct mtk_venc_core_dev - driver data +* @plat_dev: platform_device +* @main_dev: main device +* @pm: power management data +* @curr_ctx: the context that is waiting for venc hardware +* @reg_base: mapped address of venc registers +* @irq_status: venc core irq status +* @enc_irq: venc device irq +* @core id: for venc core id: core#0, core#1... +*/ +struct mtk_venc_core_dev { +struct platform_device *plat_dev; +struct mtk_vcodec_dev *main_dev; + +struct mtk_vcodec_pm pm; +struct mtk_vcodec_ctx *curr_ctx; + +void __iomem *reg_base; +unsigned int irq_status; +int enc_irq; +int core_id; +}; + +#endif /* _MTK_VCODEC_ENC_CORE_H_ */
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +52 -36
155 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v2 5/10 media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface
In reply to Irui WangThe args "struct mtk_vcodec_dev *" doesn't appropriate for init_clk functions because of sub-devices. For main device, it has no "pm/clk" properties in dtsi, sub-devices will init their own "pm/clk" instead. So rewrite the pm interface with args "platform_device *" and "mtk_vcodec_pm*".
--- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c | 9 +++++++++ drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 +- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 9 ++------- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 4 ++-- 4 files changed, 14 insertions(+), 10 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c +9 -0
@@ -13,6 +13,7 @@#include "mtk_vcodec_drv.h" #include "mtk_vcodec_enc.h" #include "mtk_vcodec_enc_core.h" +#include "mtk_vcodec_enc_pm.h" static const struct of_device_id mtk_venc_core_ids[] = { {
@@ -95,6 +96,13 @@ static int mtk_venc_core_probe(struct platform_device *pdev) core->plat_dev = pdev; + ret =mtk_vcodec_init_enc_clk (core->plat_dev, &core->pm); +if (ret <0 ) { +dev_err (&pdev->dev,"Failed to get venc core clock source!" ); +return ret; + } +pm_runtime_enable (&pdev->dev); + core->reg_base =devm_platform_ioremap_resource (pdev,0 );if (IS_ERR (core->reg_base)) {dev_err (&pdev->dev,"Failed to get reg base" );
@@ -137,6 +145,7 @@ static int mtk_venc_core_probe(struct platform_device *pdev)return 0 ; err: +pm_runtime_disable (core->pm.dev);return ret; }
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +1 -1
@@ -262,7 +262,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) core_type = dev->venc_pdata->core_type;if (dev->venc_pdata->core_mode ==VENC_SINGLE_CORE_MODE ) { - ret =mtk_vcodec_init_enc_clk (dev); + ret =mtk_vcodec_init_enc_clk (dev->plat_dev, &dev->pm);if (ret <0 ) {dev_err (&pdev->dev,"Failed to get mtk vcodec clock source!" );
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +2 -7
@@ -13,19 +13,15 @@#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_util.h" -int mtk_vcodec_init_enc_clk (struct mtk_vcodec_dev *mtkdev) +int mtk_vcodec_init_enc_clk (struct platform_device *pdev, +struct mtk_vcodec_pm *pm) {struct device_node *node; -struct platform_device *pdev; -struct mtk_vcodec_pm *pm;struct mtk_vcodec_clk *enc_clk;struct mtk_vcodec_clk_info *clk_info;int ret =0 , i =0 ;struct device *dev; - pdev = mtkdev->plat_dev; - pm = &mtkdev->pm; -memset (pm,0 ,sizeof (struct mtk_vcodec_pm )); pm->dev = &pdev->dev; dev = &pdev->dev; enc_clk = &pm->venc_clk;
@@ -42,7 +38,6 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)return -ENODEV ; } pm->larbvenc = &pdev->dev; - pdev = mtkdev->plat_dev; pm->dev = &pdev->dev; enc_clk->clk_num = of_property_count_strings(pdev->dev.of_node,
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +2 -2
@@ -9,8 +9,8 @@#include "mtk_vcodec_drv.h" -int mtk_vcodec_init_enc_clk (struct mtk_vcodec_dev *dev); - +int mtk_vcodec_init_enc_clk (struct platform_device *pdev, +struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm);
Signature · 1 line
2.18.0
PATCH v2 7/10 media: mtk-vcodec: Rewrite venc clock interface
In reply to Irui WangDual core mode need enable the specific core's clk, add an another param for clock_on/clock_off interface.
--- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 96 ++++++++++++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 4 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 10 +- 3 files changed, 86 insertions(+), 24 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +82 -14
@@ -83,17 +83,42 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev, }EXPORT_SYMBOL_GPL (mtk_vcodec_init_enc_clk); -void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_on (struct mtk_vcodec_dev *dev,int core_id) { -struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; +struct mtk_venc_core_dev *core; +struct mtk_vcodec_pm *enc_pm; +struct mtk_vcodec_clk *enc_clk; +struct clk *clk;int ret, i =0 ; -for (i =0 ; i < enc_clk->clk_num; i++) { - ret =clk_prepare_enable (enc_clk->clk_info[i].vcodec_clk); -if (ret) { -mtk_v4l2_err ("venc clk_prepare_enable %d %s fail %d" , i, - enc_clk->clk_info[i].clk_name, ret); - goto clkerr; +if (dev->venc_pdata->core_mode ==VENC_DUAL_CORE_MODE ) { + core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id]; + enc_pm = &core->pm; + enc_clk = &enc_pm->venc_clk; + +for (i =0 ; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret =clk_enable (clk); +if (ret) { +mtk_v4l2_err ("clk_enable %d %s fail %d" , i, + enc_clk->clk_info[i].clk_name, + ret); + goto core_clk_err; + } + } + }else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; + +for (i =0 ; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret =clk_prepare_enable (clk); +if (ret) { +mtk_v4l2_err ("clk_prepare %d %s fail %d" , + i, enc_clk->clk_info[i].clk_name, + ret); + goto clkerr; + } } }
@@ -104,27 +129,49 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) }return ; +core_clk_err: +for (i -=1 ; i >=0 ; i--) +clk_disable (enc_clk->clk_info[i].vcodec_clk); + +return ; + clkerr:for (i -=1 ; i >=0 ; i--)clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); }EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_on); -void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_off (struct mtk_vcodec_dev *dev,int core_id) { -struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; +struct mtk_venc_core_dev *core; +struct mtk_vcodec_pm *enc_pm; +struct mtk_vcodec_clk *enc_clk;int i =0 ;mtk_smi_larb_put (pm->larbvenc); -for (i = enc_clk->clk_num -1 ; i >=0 ; i--) -clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); + +if (dev->venc_pdata->core_mode ==VENC_DUAL_CORE_MODE ) { + core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id]; + enc_pm = &core->pm; + enc_clk = &enc_pm->venc_clk; + +for (i = enc_clk->clk_num -1 ; i >=0 ; i--) +clk_disable (enc_clk->clk_info[i].vcodec_clk); + }else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; + +for (i = enc_clk->clk_num -1 ; i >=0 ; i--) +clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); + } }EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_off);int mtk_venc_core_pw_on (struct mtk_vcodec_dev *dev) { -int i, ret; +int i, ret, j =0 ;struct mtk_venc_core_dev *core; +struct mtk_vcodec_clk *clk;/* power on all available venc cores */ for (i =0 ; i <MTK_VENC_CORE_MAX ; i++) {
@@ -137,12 +184,28 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev)mtk_v4l2_err ("power on core[%d] fail %d" , i, ret); goto pw_on_fail; } + + clk = &core->pm.venc_clk; +for (j =0 ; j < clk->clk_num; j++) { + ret =clk_prepare (clk->clk_info[j].vcodec_clk); +if (ret) { +mtk_v4l2_err ("prepare clk [%s] fail %d" , + clk->clk_info[j].clk_name, + ret); + goto pw_on_fail; + } + } }return 0 ; pw_on_fail:for (i -=1 ; i >=0 ; i--) { core = (struct mtk_venc_core_dev *)dev->enc_core_dev[i]; + + clk = &core->pm.venc_clk; +for (j -=1 ; j >=0 ; j--) +clk_unprepare (clk->clk_info[j].vcodec_clk); +pm_runtime_put_sync (&core->plat_dev->dev); }return ret;
@@ -150,8 +213,9 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev)int mtk_venc_core_pw_off (struct mtk_vcodec_dev *dev) { -int i, ret; +int i, ret, j;struct mtk_venc_core_dev *core; +struct mtk_vcodec_clk *clk;/* power off all available venc cores */ for (i =0 ; i <MTK_VENC_CORE_MAX ; i++) {
@@ -159,6 +223,10 @@ int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev)if (!core)return 0 ; + clk = &core->pm.venc_clk; +for (j = clk->clk_num -1 ; j >=0 ; j--) +clk_unprepare (clk->clk_info[j].vcodec_clk); + ret =pm_runtime_put_sync (&core->plat_dev->dev);if (ret <0 )mtk_v4l2_err ("power off core[%d] fail %d" , i, ret);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +2 -2
@@ -11,8 +11,8 @@int mtk_vcodec_init_enc_clk (struct platform_device *pdev,struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm); +void mtk_vcodec_enc_clock_on (struct mtk_vcodec_dev *dev,int core_id); +void mtk_vcodec_enc_clock_off (struct mtk_vcodec_dev *dev,int core_id);int mtk_venc_core_pw_on (struct mtk_vcodec_dev *dev);int mtk_venc_core_pw_off (struct mtk_vcodec_dev *dev);
drivers/media/platform/mtk-vcodec/venc_drv_if.c +2 -8
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) }mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->init (ctx); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx);return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,int ret =0 ;mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->set_param (ctx->drv_handle, type, in); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx);return ret;
@@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, ctx->dev->curr_ctx = ctx;spin_unlock_irqrestore (&ctx->dev->irqlock, flags); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); +mtk_vcodec_enc_clock_on (ctx->dev,0 ); ret = ctx->enc_if->encode (ctx->drv_handle, opt, frm_buf, bs_buf, result); -mtk_vcodec_enc_clock_off (&ctx->dev->pm); +mtk_vcodec_enc_clock_off (ctx->dev,0 );spin_lock_irqsave (&ctx->dev->irqlock, flags); ctx->dev->curr_ctx =NULL ;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)return 0 ;mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->deinit (ctx->drv_handle); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx); ctx->drv_handle =NULL ;
Signature · 1 line
2.18.0
PATCH v2 8/10 media: mtk-vcodec: Add more extra processing for dual-core mode
In reply to Irui WangDual core mode encoding need more venc working buffers, it will break the compatibility if we just add venc_vsi in AP-Kernel but not in firmware, so add more struct definition for it.
--- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 1 + .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 4 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 178 +++++++++++++++--- 4 files changed, 177 insertions(+), 25 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +1 -0
@@ -452,6 +452,7 @@ struct mtk_vcodec_enc_pdata { };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata->uses_ext) +#define MTK_ENC_CORE_MODE (ctx) ((ctx)->dev->venc_pdata->core_mode)/** * struct mtk_vcodec_dev - driver data
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c +19 -0
@@ -11,6 +11,7 @@#include "mtk_vcodec_dec_hw.h" #include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc_core.h" #include "mtk_vcodec_util.h" void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, }EXPORT_SYMBOL (mtk_vcodec_get_reg_addr); +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *ctx, +unsigned int core_id) +{ +struct mtk_venc_core_dev *core; + +if (core_id >=MTK_VENC_CORE_MAX ) { +mtk_v4l2_err ("Invalid core_id = %d" , core_id); +return NULL ; + } + + core = (struct mtk_venc_core_dev *)ctx->dev->enc_core_dev[core_id]; +if (!core) +return NULL ; + +return core->reg_base; +} +EXPORT_SYMBOL (mtk_venc_get_core_reg_addr); +int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem) {
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +4 -0
@@ -50,6 +50,10 @@ struct mtk_vcodec_dev;void __iomem *mtk_vcodec_get_reg_addr (struct mtk_vcodec_ctx *data,unsigned int reg_idx); + +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *data, +unsigned int core_id); +int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem); void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +153 -25
284 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v2 10/10 media: mtk-vcodec: Done encode result to client
In reply to Irui WangWhen enable dual core mode encoding, the wait IRQ done synchronous function should not be called, so the encode result can't return to client in device_run. device_run called to trigger encoding process, finish it, so another scheduled job can be ran. When encoding done, done the encode result to client in IRQ handler.
--- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 6 ++ .../platform/mtk-vcodec/mtk_vcodec_enc.c | 64 ++++++++++++++++++- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 3 + .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 27 +++++++- .../platform/mtk-vcodec/mtk_vcodec_util.h | 1 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 22 ++++++- .../media/platform/mtk-vcodec/venc_drv_if.h | 2 + 7 files changed, 120 insertions(+), 5 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +6 -0
@@ -311,6 +311,9 @@ struct vdec_pic_info {* @q_mutex: src & dst vb2_queue mutex * @enc_idx: used to record encoded frame count * @core_id: used to reoord used core +* @pfrm_buf: used to store current ctx's frame buffer +* @pbs_buf: used to store current ctx's bitstream buffer +* @hdr_size: used to store prepend header size */ struct mtk_vcodec_ctx {enum mtk_instance_type type;
@@ -363,6 +366,9 @@ struct mtk_vcodec_ctx {struct mutex q_mutex;int enc_idx;int core_id; +struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_CORE_MAX ]; +struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_CORE_MAX ]; +unsigned int hdr_size; };/*
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +62 -2
115 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +3 -0
@@ -46,5 +46,8 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx);int mtk_vcodec_enc_ctrls_setup (struct mtk_vcodec_ctx *ctx);void mtk_vcodec_enc_set_default_params (struct mtk_vcodec_ctx *ctx); +void mtk_venc_buf_done (struct mtk_vcodec_ctx *ctx,int core_id, +unsigned int bs_size,bool time_out,bool key_frame); +void mtk_venc_lock_all (struct mtk_vcodec_ctx *ctx);#endif /* _MTK_VCODEC_ENC_H_ */
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c +26 -1
@@ -14,6 +14,11 @@#include "mtk_vcodec_enc.h" #include "mtk_vcodec_enc_core.h" #include "mtk_vcodec_enc_pm.h" +#include "mtk_vcodec_intr.h" + +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 +#define VENC_PIC_FRM_TYPE 0x0010 +#define VENC_PIC_KEY_FRM 0x2static const struct of_device_id mtk_venc_core_ids[] = { {
@@ -47,6 +52,7 @@ static void clean_irq_status(unsigned int irq_status, void __iomem *addr)if (irq_status &MTK_VENC_IRQ_STATUS_FRM )writel (MTK_VENC_IRQ_STATUS_FRM , addr); + }static irqreturn_t mtk_enc_core_irq_handler (int irq,void *priv)
@@ -55,6 +61,9 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv)struct mtk_vcodec_ctx *ctx;unsigned long flags;void __iomem *addr; +unsigned int bs_size; +unsigned int frm_type; +bool is_key_frame =0 ;spin_lock_irqsave (&core->main_dev->irqlock, flags); ctx = core->curr_ctx;
@@ -66,9 +75,25 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv) addr = core->reg_base +MTK_VENC_IRQ_ACK_OFFSET ; ctx->irq_status =readl (core->reg_base +MTK_VENC_IRQ_STATUS_OFFSET ); + bs_size =readl (core->reg_base +VENC_PIC_BITSTREAM_BYTE_CNT ); + frm_type =readl (core->reg_base +VENC_PIC_FRM_TYPE );clean_irq_status (ctx->irq_status, addr); -wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); +if (ctx->irq_status &MTK_VENC_IRQ_STATUS_FRM ) { +if (ctx->hdr_size !=0 ) { + bs_size += ctx->hdr_size; + ctx->hdr_size =0 ; + } +if (frm_type &VENC_PIC_KEY_FRM ) + is_key_frame =1 ; + +mtk_venc_buf_done (ctx, core->core_id, bs_size,0 , is_key_frame); +mtk_vcodec_enc_clock_off (core->main_dev, core->core_id); +mtk_venc_unlock (ctx, core->core_id); + }else { +wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); + } +return IRQ_HANDLED ; }
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +1 -0
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {size_t size;void *va;dma_addr_t dma_addr; +void *buf; };struct mtk_vcodec_fb {
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +20 -2
@@ -558,6 +558,12 @@ static int h264_encode_frame(struct venc_h264_inst *inst,return ret; } +if (is_dual_core) { + ++inst->frm_cnt; +mtk_vcodec_debug (inst,"dual core : frm %d <-" , inst->frm_cnt); +return ret; + } + irq_status =h264_enc_wait_venc_done (inst);if (irq_status !=MTK_VENC_IRQ_STATUS_FRM ) {mtk_vcodec_err (inst,"irq_status=%d failed" , irq_status);
@@ -639,10 +645,12 @@ static int h264_enc_encode(void *handle,int ret =0 ;struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;struct mtk_vcodec_ctx *ctx = inst->ctx; +bool is_single_core = (MTK_ENC_CORE_MODE (ctx) ==VENC_SINGLE_CORE_MODE );mtk_vcodec_debug (inst,"opt %d ->" , opt); -enable_irq (ctx->dev->enc_irq); +if (is_single_core) +enable_irq (ctx->dev->enc_irq);switch (opt) {case VENC_START_OPT_ENCODE_SEQUENCE_HEADER : {
@@ -666,6 +674,13 @@ static int h264_enc_encode(void *handle,unsigned int bs_size_hdr;unsigned int bs_size_frm; +/* +* the frm_buf and bs_buf need to be recorded into ctx, +* when encoding done, the target buffer can be obtained +*/ + ctx->pfrm_buf[ctx->core_id] = frm_buf->src_addr; + ctx->pbs_buf[ctx->core_id] = bs_buf->buf; +if (!inst->prepend_hdr) { ret =h264_encode_frame (inst, frm_buf, bs_buf, &result->bs_size, ctx->core_id);
@@ -702,6 +717,8 @@ static int h264_enc_encode(void *handle, result->bs_size = hdr_sz + filler_sz + bs_size_frm; + ctx->hdr_size = hdr_sz + filler_sz; +mtk_vcodec_debug (inst,"hdr %d filler %d frame %d bs %d" , hdr_sz, filler_sz, bs_size_frm, result->bs_size);
@@ -718,8 +735,9 @@ static int h264_enc_encode(void *handle, } encode_err: +if (is_single_core) +disable_irq (ctx->dev->enc_irq); -disable_irq (ctx->dev->enc_irq);mtk_vcodec_debug (inst,"opt %d <-" , opt);return ret;
drivers/media/platform/mtk-vcodec/venc_drv_if.h +2 -0
@@ -108,9 +108,11 @@ struct venc_frame_info {/* * struct venc_frm_buf - frame buffer information used in venc_if_encode() * @fb_addr: plane frame buffer addresses +* @src_addr: vb2_v4l2_buffer src buffer address */ struct venc_frm_buf {struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES ]; +void *src_addr; };/*
Signature · 1 line
2.18.0
PATCH v2 9/10 media: mtk-vcodec: Add dual core mode encode process
In reply to Irui WangThe dual core mode encoding is tries to uses the two venc cores:
frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,
Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0.
--- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 11 ++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 23 +++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 4 +- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 7 +- .../platform/mtk-vcodec/venc/venc_h264_if.c | 16 ++-- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 73 ++++++++++++++----- .../media/platform/mtk-vcodec/venc_drv_if.h | 5 ++ .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 ++- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 10 files changed, 113 insertions(+), 42 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +10 -1
@@ -308,6 +308,9 @@ struct vdec_pic_info {* @max_width: hardware supported max width * @max_height: hardware supported max height * @msg_queue: msg queue used to store lat buffer information. +* @q_mutex: src & dst vb2_queue mutex +* @enc_idx: used to record encoded frame count +* @core_id: used to reoord used core */ struct mtk_vcodec_ctx {enum mtk_instance_type type;
@@ -356,6 +359,10 @@ struct mtk_vcodec_ctx {unsigned int max_width;unsigned int max_height;struct vdec_msg_queue msg_queue; + +struct mutex q_mutex; +int enc_idx; +int core_id; };/*
@@ -533,7 +540,9 @@ struct mtk_vcodec_dev {/* decoder hardware mutex lock */ struct mutex dec_mutex[MTK_VDEC_HW_MAX ]; -struct mutex enc_mutex; + +/* encoder core mutex lock */ +struct mutex enc_mutex[MTK_VENC_CORE_MAX ];struct mtk_vcodec_pm pm;unsigned int dec_capability;
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +13 -10
@@ -1198,14 +1198,17 @@ static void mtk_venc_worker(struct work_struct *work)vb2_set_plane_payload (&dst_buf->vb2_buf,0 ,0 );v4l2_m2m_buf_done (dst_buf,VB2_BUF_STATE_ERROR );mtk_v4l2_err ("venc_if_encode failed=%d" , ret); - }else { + }else if (ctx->dev->venc_pdata->core_mode ==VENC_SINGLE_CORE_MODE ) {v4l2_m2m_buf_done (src_buf,VB2_BUF_STATE_DONE ); -vb2_set_plane_payload (&dst_buf->vb2_buf,0 , enc_result.bs_size); +vb2_set_plane_payload (&dst_buf->vb2_buf,0 , + enc_result.bs_size);v4l2_m2m_buf_done (dst_buf,VB2_BUF_STATE_DONE );mtk_v4l2_debug (2 ,"venc_if_encode bs size=%d" , - enc_result.bs_size); + enc_result.bs_size); } + ctx->enc_idx++; +v4l2_m2m_job_finish (ctx->dev->m2m_dev_enc, ctx->m2m_ctx); mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
@@ -1259,7 +1262,7 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx) {struct mtk_q_data *q_data; - ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex; + ctx->m2m_ctx->q_lock = &ctx->q_mutex; ctx->fh.m2m_ctx = ctx->m2m_ctx; ctx->fh.ctrl_handler = &ctx->ctrl_hdl;INIT_WORK (&ctx->encode_work, mtk_venc_worker);
@@ -1391,7 +1394,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, src_vq->ops = &mtk_venc_vb2_ops; src_vq->mem_ops = &vb2_dma_contig_memops; src_vq->timestamp_flags =V4L2_BUF_FLAG_TIMESTAMP_COPY ; - src_vq->lock = &ctx->dev->dev_mutex; + src_vq->lock = &ctx->q_mutex; src_vq->dev = &ctx->dev->plat_dev->dev; ret =vb2_queue_init (src_vq);
@@ -1405,26 +1408,26 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, dst_vq->ops = &mtk_venc_vb2_ops; dst_vq->mem_ops = &vb2_dma_contig_memops; dst_vq->timestamp_flags =V4L2_BUF_FLAG_TIMESTAMP_COPY ; - dst_vq->lock = &ctx->dev->dev_mutex; + dst_vq->lock = &ctx->q_mutex; dst_vq->dev = &ctx->dev->plat_dev->dev;return vb2_queue_init (dst_vq); } -int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx) +int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx,int core_id) {struct mtk_vcodec_dev *dev = ctx->dev; -mutex_unlock (&dev->enc_mutex); +mutex_unlock (&dev->enc_mutex[core_id]);return 0 ; }EXPORT_SYMBOL_GPL (mtk_venc_unlock); -int mtk_venc_lock (struct mtk_vcodec_ctx *ctx) +int mtk_venc_lock (struct mtk_vcodec_ctx *ctx,int core_id) {struct mtk_vcodec_dev *dev = ctx->dev; -mutex_lock (&dev->enc_mutex); +mutex_lock (&dev->enc_mutex[core_id]);return 0 ; }EXPORT_SYMBOL_GPL (mtk_venc_lock);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +2 -2
@@ -39,8 +39,8 @@ struct mtk_video_enc_buf {extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;extern const struct v4l2_m2m_ops mtk_venc_m2m_ops; -int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx); -int mtk_venc_lock (struct mtk_vcodec_ctx *ctx); +int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx,int core_id); +int mtk_venc_lock (struct mtk_vcodec_ctx *ctx,int core_id);int mtk_vcodec_enc_queue_init (void *priv,struct vb2_queue *src_vq,struct vb2_queue *dst_vq);void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +5 -2
@@ -133,6 +133,7 @@ static int fops_vcodec_open(struct file *file)INIT_LIST_HEAD (&ctx->list); ctx->dev = dev;init_waitqueue_head (&ctx->queue[0 ]); +mutex_init (&ctx->q_mutex); ctx->type =MTK_INST_ENCODER ; ret =mtk_vcodec_enc_ctrls_setup (ctx);
@@ -240,7 +241,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)struct video_device *vfd_enc;phandle rproc_phandle;enum mtk_vcodec_fw_type fw_type; -int ret, core_type; +int ret, core_type, i; dev =devm_kzalloc (&pdev->dev,sizeof (*dev),GFP_KERNEL );if (!dev)
@@ -299,7 +300,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev) } } -mutex_init (&dev->enc_mutex); +for (i =0 ; i <MTK_VENC_CORE_MAX ; i++) +mutex_init (&dev->enc_mutex[i]); +mutex_init (&dev->dev_mutex);spin_lock_init (&dev->irqlock);
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +10 -6
@@ -448,7 +448,8 @@ static int h264_encode_sps(struct venc_h264_inst *inst,mtk_vcodec_debug_enter (inst); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_SPS ,NULL , bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_SPS , +NULL , bs_buf,NULL ,MTK_VENC_CORE0 );if (ret)return ret;
@@ -474,7 +475,8 @@ static int h264_encode_pps(struct venc_h264_inst *inst,mtk_vcodec_debug_enter (inst); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_PPS ,NULL , bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_PPS , +NULL , bs_buf,NULL ,MTK_VENC_CORE0 );if (ret)return ret;
@@ -516,7 +518,8 @@ static int h264_encode_header(struct venc_h264_inst *inst,static int h264_encode_frame (struct venc_h264_inst *inst,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -unsigned int *bs_size) +unsigned int *bs_size, +int core_id) {int ret =0 ;unsigned int irq_status;
@@ -536,7 +539,8 @@ static int h264_encode_frame(struct venc_h264_inst *inst,mtk_vcodec_debug (inst,"frm_count = %d,skip_frm_count =%d,frm_type=%d.\n" , frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_FRAME , frm_buf, bs_buf, &frame_info); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_FRAME , + frm_buf, bs_buf, &frame_info, core_id);if (ret)return ret;
@@ -664,7 +668,7 @@ static int h264_enc_encode(void *handle,if (!inst->prepend_hdr) { ret =h264_encode_frame (inst, frm_buf, bs_buf, - &result->bs_size); + &result->bs_size, ctx->core_id);if (ret) goto encode_err; result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -692,7 +696,7 @@ static int h264_enc_encode(void *handle, tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz); ret =h264_encode_frame (inst, frm_buf, &tmp_bs_buf, - &bs_size_frm); + &bs_size_frm, ctx->core_id);if (ret) goto encode_err;
drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c +2 -1
@@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,mtk_vcodec_debug (inst,"->frm_cnt=%d" , inst->frm_cnt); - ret =vpu_enc_encode (&inst->vpu_inst,0 , frm_buf, bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,0 , frm_buf, bs_buf, +NULL ,MTK_VENC_CORE0 );if (ret)return ret;
drivers/media/platform/mtk-vcodec/venc_drv_if.c +55 -18
114 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/venc_drv_if.h +5 -0
@@ -167,4 +167,9 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,struct mtk_vcodec_mem *bs_buf, struct venc_done_result *result); +void venc_encode_prepare (struct mtk_vcodec_ctx *ctx, +enum venc_start_opt opt); +void venc_encode_unprepare (struct mtk_vcodec_ctx *ctx, +enum venc_start_opt opt); +#endif /* _VENC_DRV_IF_H_ */
drivers/media/platform/mtk-vcodec/venc_vpu_if.c +9 -1
@@ -225,9 +225,11 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,int vpu_enc_encode (struct venc_vpu_inst *vpu,unsigned int bs_mode,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -struct venc_frame_info *frame_info) +struct venc_frame_info *frame_info, +int core_id) {const bool is_ext =MTK_ENC_CTX_IS_EXT (vpu->ctx); +const int core_mode =MTK_ENC_CORE_MODE (vpu->ctx);size_t msg_size = is_ext ?sizeof (struct venc_ap_ipi_msg_enc_ext ) :sizeof (struct venc_ap_ipi_msg_enc );
@@ -261,6 +263,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, out.data[1 ] = frame_info->skip_frm_count; out.data[2 ] = frame_info->frm_type; } + +if (core_mode ==VENC_DUAL_CORE_MODE ) { + out.data_item =4 ; + out.data[3 ] = core_id; + } +if (vpu_enc_send_msg (vpu, &out, msg_size)) {mtk_vcodec_err (vpu,"AP_IPIMSG_ENC_ENCODE %d fail" , bs_mode);
drivers/media/platform/mtk-vcodec/venc_vpu_if.h +2 -1
@@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,int vpu_enc_encode (struct venc_vpu_inst *vpu,unsigned int bs_mode,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -struct venc_frame_info *frame_info); +struct venc_frame_info *frame_info, +int core_id);int vpu_enc_deinit (struct venc_vpu_inst *vpu);#endif
Signature · 1 line
2.18.0
PATCH v2 6/10 media: mtk-vcodec: Add venc power on/off interface
In reply to Irui WangDual-core mode power on/off all venc available power, pm_runtime_xx helper is not appropriate called directly, add new power on/off interface for it.
--- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 20 +---- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 7 ++ .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 83 +++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 5 ++ 4 files changed, 98 insertions(+), 17 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +3 -17
@@ -13,6 +13,7 @@#include "mtk_vcodec_drv.h" #include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_intr.h" #include "mtk_vcodec_util.h" #include "venc_drv_if.h"
@@ -850,7 +851,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count) {struct mtk_vcodec_ctx *ctx =vb2_get_drv_priv (q);struct venc_enc_param param; -int ret, pm_ret; +int ret;int i;/* Once state turn into MTK_STATE_ABORT, we need stop_streaming
@@ -858,7 +859,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)*/ if ((ctx->state ==MTK_STATE_ABORT ) || (ctx->state ==MTK_STATE_FREE )) { ret = -EIO ; - goto err_start_stream; + goto err_set_param; }/* Do the initialization when both start_streaming have been called */
@@ -870,12 +871,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)return 0 ; } - ret =pm_runtime_resume_and_get (&ctx->dev->plat_dev->dev); -if (ret <0 ) { -mtk_v4l2_err ("pm_runtime_resume_and_get fail %d" , ret); - goto err_start_stream; - } -mtk_venc_set_param (ctx, ¶m); ret =venc_if_set_param (ctx,VENC_SET_PARAM_ENC , ¶m);if (ret) {
@@ -902,11 +897,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)return 0 ; err_set_param: - pm_ret =pm_runtime_put (&ctx->dev->plat_dev->dev); -if (pm_ret <0 ) -mtk_v4l2_err ("pm_runtime_put fail %d" , pm_ret); - -err_start_stream:for (i =0 ; i < q->num_buffers; ++i) {struct vb2_buffer *buf =vb2_get_buffer (q, i);
@@ -989,10 +979,6 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)if (ret)mtk_v4l2_err ("venc_if_deinit failed=%d" , ret); - ret =pm_runtime_put (&ctx->dev->plat_dev->dev); -if (ret <0 ) -mtk_v4l2_err ("pm_runtime_put fail %d" , ret); - ctx->state =MTK_STATE_FREE ; }
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +7 -0
@@ -174,6 +174,12 @@ static int fops_vcodec_open(struct file *file)mtk_v4l2_debug (0 ,"encoder capability %x" , dev->enc_capability); } + ret =mtk_vcodec_enc_pw_on (dev); +if (ret <0 ) { +mtk_v4l2_err ("encoder power on failed %d" , ret); + goto err_load_fw; + } +mtk_v4l2_debug (2 ,"Create instance [%d]@%p m2m_ctx=%p " , ctx->id, ctx, ctx->m2m_ctx);
@@ -208,6 +214,7 @@ static int fops_vcodec_release(struct file *file)v4l2_m2m_ctx_release (ctx->m2m_ctx);mtk_vcodec_enc_release (ctx); +mtk_vcodec_enc_pw_off (dev);v4l2_fh_del (&ctx->fh);v4l2_fh_exit (&ctx->fh);v4l2_ctrl_handler_free (&ctx->ctrl_hdl);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +83 -0
92 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +5 -0
@@ -14,4 +14,9 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm); +int mtk_venc_core_pw_on (struct mtk_vcodec_dev *dev); +int mtk_venc_core_pw_off (struct mtk_vcodec_dev *dev); +int mtk_vcodec_enc_pw_on (struct mtk_vcodec_dev *dev); +int mtk_vcodec_enc_pw_off (struct mtk_vcodec_dev *dev); +#endif /* _MTK_VCODEC_ENC_PM_H_ */
Signature · 1 line
2.18.0
Rob Herring
maintainerRe: [PATCH v2, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Irui WangQuoted 8 lines · Irui Wang
Adds encoder cores dt-bindings for mt8195
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ 1 file changed, 214 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
My bot found errors running 'make DT_CHECKER_FLAGS=-m dt_binding_check' on your patch (DT_CHECKER_FLAGS is new in v5.13):
yamllint warnings/errors:
dtschema/dtc warnings/errors:Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.example.dts:20:18: fatal error: dt-bindings/memory/mt8195-memory-port.h: No such file or directory
20 | #include <dt-bindings/memory/mt8195-memory-port.h>
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.
make[1]: *** [scripts/Makefile.lib:373: Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.example.dt.yaml] Error 1 make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:1413: dt_binding_check] Error 2
doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/patch/1580741
This check can fail if there are any dependencies. The base for a patch series is generally the most recent rc1.
If you already ran 'make dt_binding_check' and didn't see the above error(s), then make sure 'yamllint' is installed and dt-schema is up to date:
pip3 install dtschema --upgrade
Please check and re-submit.
Rob Herring
maintainerRe: [PATCH v2, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Irui WangQuoted 28 lines · Irui Wang
Adds encoder cores dt-bindings for mt8195
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ 1 file changed, 214 insertions(+) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +14
-0@@ -0,0 +1,214 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present in Mediatek
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 24
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ SoCs which supports high resolution encoding functionalities. Required + parent and child device node. + +properties: + compatible: + const: mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + maxItems: 1
'phandle' is already 1 item. Drop.
Quoted 17 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +17
-0+ description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps to memory. + + "#address-cells":
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 51
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ const: 1 + + "#size-cells": + const: 1 + + ranges: true + +# Required child node: +patternProperties: + "venc_core0@1a020000":
Address should generally not be defined in the node name schema:
'^venc-core0@'
Though I think you should also drop the '0' here. The unit-address is enough to distinguish each instance. Then the schemas for each child node can be combined.
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 56
Quoted 5 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +5
-0+ type: object + + properties: + compatible: + const: mediatek,mtk-venc-core0
Is the programming model for each core the same, but just different codecs implemented? I'd just add a property to indicate which codec if that's not discoverable.
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ + reg: + maxItems: 1 + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml.
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 76
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + items: + - const: MT_CG_VENC0
The name is supposed to be local to the instance reflecting what the clock drives rather than a top-level or clock controller name. Lowercase is also the norm. Given there's only 1 clock, I'd just drop the name.
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 82
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +6
-0+ + assigned-clocks: + maxItems: 1 + + assigned-clock-parents: + maxItems: 1
These are always allowed and shouldn't be required.
Quoted 136 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +132
-0+ + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + + "venc_core1@1b020000": + type: object + + properties: + compatible: + const: mediatek,mtk-venc-core1 + + reg: + maxItems: 1 + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + items: + - const: MT_CG_VENC1 + + assigned-clocks: + maxItems: 1 + + assigned-clock-parents: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + - ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <1>; + #size-cells = <1>; + ranges; + + venc_core0@1a020000 { + compatible = "mediatek,mtk-venc-core0"; + reg = <0x1a020000 0x10000>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "MT_CG_VENC0"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc_core1@1b020000 { + compatible = "mediatek,mtk-venc-core1"; + reg = <0x1b020000 0x10000>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "MT_CG_VENC1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + };Signature · 1 line
2.18.0
Re: [PATCH v2, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Rob HerringDear Rob,
Many thanks for your comments.
Quoted 40 lines · Rob Herring
Quoted 11 lines · Irui Wang
Adds encoder cores dt-bindings for mt8195
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ 1 file changed, 214 insertions(+) create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamlMy bot found errors running 'make DT_CHECKER_FLAGS=-m
dt_binding_check'
on your patch (DT_CHECKER_FLAGSis new in v5.13):yamllint warnings/errors:
dtschema/dtc warnings/errors:
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.example.dts:20:18: fatal error: dt-bindings/memory/mt8195-memory-port.h: No such file or directory
20 | #include <dt-bindings/memory/mt8195-memory-port.h>| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~compilation terminated.
make[1]: *** [scripts/Makefile.lib:373:Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.example.dt.yaml] Error 1
make[1]: *** Waiting for unfinished jobs....
make: *** [Makefile:1413:dt_binding_check] Error 2doc reference errors (make refcheckdocs):
See https://patchwork.ozlabs.org/patch/1580741
This check can fail if there are any dependencies. The base for a patch
series is generally the most recent rc1.
Quoted 8 lines
If you already ran 'make
dt_binding_check' and didn't see the above error(s), then make sure 'yamllint' is installed and dt-schema is up to
date:pip3 install dtschema --upgrade
Please check and re-submit.
This new encoder yaml includes mt8195 iommu and power&clock node, so it depends on patches:
iommu:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641
power:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579
which are not accept yet.
But IOMMU and power already send new version series patches, I will rebase this series in next version.
Thanks
BRs
Quoted 1 line
Re: [PATCH v2, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Rob HerringDear Rob,
Many thanks for your review comments.
Quoted 45 lines · Rob Herring
Quoted 44 lines · Irui Wang
Adds encoder cores dt-bindings for mt8195
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ 1 file changed, 214 insertions(+) create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamldiff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-
core.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
new file mode 100644
index 000000000000..d1e7bfa50bceDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-0
encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0@@ -0,0 +1,214 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present inMediatek
+ SoCs which supports high resolution encoding functionalities. RequiredDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ parent and child device node. + +properties:
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ compatible: + const: mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + maxItems: 1'phandle' is already 1 item. Drop.
Do you mean delete "maxItems: 1"? we will delete in next version.
Quoted 33 lines
Quoted 20 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +9
-0+ description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps tomemory.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ + "#address-cells":Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10
-0+ const: 1 + + "#size-cells": + const: 1 + + ranges: true + +# Required child node: +patternProperties: + "venc_core0@1a020000":Address should generally not be defined in the node name schema:
Quoted 6 lines
'^venc-core0@'
Though I think you should also drop the '0' here. The unit-address is
enough to distinguish each instance. Then the schemas for each child node can be combined.
Thanks, we will rename it in next version.
Quoted 7 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0+ type: object + + properties: + compatible: + const: mediatek,mtk-venc-core0
Quoted 5 lines
Is the programming model for each core the same, but just different codecs implemented? I'd just add a property to indicate which codec if
that's not discoverable.
Core-0 and Core-1 are two different encoder hardware, they have their own hardware, used irq/clock/power...So, we need add two model nodes, a "property" may not distinguish the two encoder hardware.
Quoted 18 lines
Quoted 18 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +9
-0+ + reg: + maxItems: 1 + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 +
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 3 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ clock-names: + items: + - const: MT_CG_VENC0The name is supposed to be local to the instance reflecting what the clock drives rather than a top-level or clock controller name. Lowercase
is also the norm. Given there's only 1 clock, I'd just drop the name.
Thanks, we will rename it in next version, but we need parse clock info by "clock-names"(of_property_count_strings), so maybe we can't drop it.
Quoted 9 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 7 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ + assigned-clocks: + maxItems: 1 + + assigned-clock-parents: + maxItems: 1These are always allowed and shouldn't be required.
Do you mean we can not write this two properties here? we will try to delete them in next version.
Thanks
BRs
Quoted 140 lines
Quoted 140 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +32
-0+ + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + + "venc_core1@1b020000": + type: object + + properties: + compatible: + const: mediatek,mtk-venc-core1 + + reg: + maxItems: 1 + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +77
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + items: + - const: MT_CG_VENC1 + + assigned-clocks: + maxItems: 1 + + assigned-clock-parents: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + - ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <1>; + #size-cells = <1>; + ranges; + + venc_core0@1a020000 { + compatible = "mediatek,mtk-venc-core0"; + reg = <0x1a020000 0x10000>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "MT_CG_VENC0"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen
CLK_TOP_UNIVPLL_D4>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +20
-0+ power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc_core1@1b020000 { + compatible = "mediatek,mtk-venc-core1"; + reg = <0x1b020000 0x10000>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "MT_CG_VENC1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen
CLK_TOP_UNIVPLL_D4>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + };Signature · 1 line
2.18.0
Dear all maintainers,
Gently ping.
Could you help to review this series of patches? I would be very grateful for any of your comments.
Thanks
Best Regards
Quoted 40 lines · Irui Wang
MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve
4K@60fps.
The two encoder core encoding process looks like this:VENC Core0: frm#0....frm#2....frm#4.... VENC Core1: ..frm#1....frm#3....frm#5....This series of patches are used to enable the two H264 encoder cores, encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 -->device_run(triger encoder)--> wait encoder IRQ -->
encoding done with result -->job_finishV4l2_VIDIOC_QBUF#1 -->device_run(triger encoder)--> wait encoder IRQ -->
encoding done with result -->job_finish
...To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 -->device_run(triger encoder)-->job_finish..V4l2_VIDIOC_QBUF#1 -->device_run(triger encoder)-->job_finish(venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 -->device_run(triger encoder)-->job_finish.There is no "wait encoder IRQ" synchronous call during frame-racing mode
encoding process, it can full use the two encoder cores to achieve higher
performance.---
This series patches dependent on:
[1]: the latest linux stage tree:
https://git.linuxtv.org/media_stage.git
Quoted 9 lines
mtk decoder patches
[2]:
https://patchwork.linuxtv.org/project/linux-media/list/?series=7105
[3]:
https://patchwork.linuxtv.org/project/linux-media/list/?series=7131new yaml included files
[4]:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=551641
Quoted 2 lines
[5]:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579
Quoted 51 lines
---
---
changes compared with v1:
-of_platform_populatewas used in place of the component framework.
- new yaml file for venc cores.
- some modifications for patch v1's review comments.
---Irui Wang (10):
media: mtk-vcodec: Use core type to indicate h264 and vp8 enc
media: mtk-vcodec: export encoder functions
dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings formt8195media: mtk-vcodec: Enable venc dual core usage
media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface
media: mtk-vcodec: Add venc power on/off interface
media: mtk-vcodec: Rewrite venc clock interface
media: mtk-vcodec: Add more extra processing for dual-core mode
media: mtk-vcodec: Add dual core mode encode process
media: mtk-vcodec: Done encode result to client.../media/mediatek,vcodec-encoder-core.yaml | 214 +++++++++++++++++ drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 44 +++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 109 ++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 187 +++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 118 ++++++---- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 187 +++++++++++++-- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 216 +++++++++++++++- -- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + .../media/platform/mtk-vcodec/venc_vpu_if.c | 10 +- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 18 files changed, 1097 insertions(+), 162 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- core.yaml create mode 100644 drivers/media/platform/mtk- vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-
vcodec/mtk_vcodec_enc_core.h
AngeloGioacchino Del Regno
maintainerRe: [PATCH v2, 04/10] media: mtk-vcodec: Enable venc dual core usage
In reply to Irui WangIl 17/01/22 13:06, Irui Wang ha scritto:
Quoted 48 lines
Adds new venc core mode to indicate different venc hardware mode:
VENC_SINGLE_CORE_MODEmeans only one core, the device has its own power/clk/irq, init_clk/request_irq helper can be used.
VENC_DUAL_CORE_MODEmeans more than one core inside, the core device can use the init_clk/request_irq helper to initialize their own power/clk/irq. And the main device doesn't need use these helper anymore.MT8195 has two H264 venc cores, enable
dual_core_modefor it.Signed-off-by Irui Wang <[email protected]>--- drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 22 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 153 ++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 88 +++++----- 5 files changed, 266 insertions(+), 37 deletions(-)create mode 100644
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.ccreate mode 100644drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.hdrivers/media/platform/mtk-vcodec/Makefile +3
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk-vcodec-enc-core.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +0
-0
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h enum mtk_vdec_hw_count { · line 125
Quoted 9 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +6
-0@@ -117,6 +117,23 @@ enum mtk_vdec_hw_count { MTK_VDEC_MAX_HW_COUNT, }; +/* +* enum mtk_venc_core_id -- encoder core id +*/ +enum mtk_venc_core_id { +MTK_VENC_CORE0 =0 , +MTK_VENC_CORE1 =1 ,
You don't have to say "= 1" for core1, just...
MTK_VENC_CORE0 = 0, MTK_VENC_CORE1,
...is fine, and better.
Quoted 70 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +16
-0+ MTK_VENC_CORE_MAX, +}; + +/** +* enmu mtk_venc_core_mode - Used to indicate different encode mode +*/ +enum mtk_venc_core_mode { +VENC_SINGLE_CORE_MODE =0 , +VENC_DUAL_CORE_MODE =1 , +}; +/* * struct mtk_video_fmt - Structure used to store information about pixelformats */ @@ -420,6 +437,7 @@ struct mtk_vcodec_dec_pdata {* @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats * @core_type: stand for h264 or vp8 encode +* @core_mode: indicate encode core mode */ struct mtk_vcodec_enc_pdata {bool uses_ext;@@ -430,6 +448,7 @@ struct mtk_vcodec_enc_pdata {const struct mtk_video_fmt *output_formats;size_t num_output_formats;int core_type; +enum mtk_venc_core_mode core_mode; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata->uses_ext)@@ -479,6 +498,7 @@ struct mtk_vcodec_enc_pdata {* @subdev_dev: subdev hardware device * @subdev_prob_done: check whether all used hw device is prob done * @subdev_bitmap: used to record hardware is ready or not +* @enc_core_dev: used to store venc core device */ struct mtk_vcodec_dev {struct v4l2_device v4l2_dev;@@ -524,6 +544,8 @@ struct mtk_vcodec_dev {void *subdev_dev[MTK_VDEC_HW_MAX ];int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev);DECLARE_BITMAP (subdev_bitmap,MTK_VDEC_HW_MAX ); + +void *enc_core_dev[MTK_VENC_CORE_MAX ]; };static inline struct mtk_vcodec_ctx *fh_to_ctx (struct v4l2_fh *fh)drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +17
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> + +#include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_core.h" + +static const struct of_device_id mtk_venc_core_ids[] = {
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 27
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+ { + .compatible = "mediatek,mtk-venc-core0", + .data = (void *)MTK_VENC_CORE0, + }, + { + .compatible = "mediatek,mtk-venc-core1", + .data = (void *)MTK_VENC_CORE1, + }, + {}, +};
Hello Irui,
You don't need a different compatible for the different cores, as in the declaration, there's nothing special that differentiates them that much.
I understand that there may be a need to differentiate the core number, as in, CORE0 always has to be the leader, while CORE1 would be the follower, but this is not a good reason to give them a different compatible string.
I want to make you aware that Kyrie Wu did the same thing as you did here and in my review on his patch I was able to give an extensive example of how this should look; the exactly same logic would apply to this patch.
Please have a look here: https://patchwork.kernel.org/comment/24726607/
P.S.: In short, you should have only one "mediatek,mtk-venc-hw" compatible
used for probing both cores.
Quoted 47 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +47
-0+MODULE_DEVICE_TABLE (of, mtk_venc_core_ids); + +static void clean_irq_status (unsigned int irq_status,void __iomem *addr) +{ +if (irq_status &MTK_VENC_IRQ_STATUS_PAUSE ) +writel (MTK_VENC_IRQ_STATUS_PAUSE , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_SWITCH ) +writel (MTK_VENC_IRQ_STATUS_SWITCH , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_DRAM ) +writel (MTK_VENC_IRQ_STATUS_DRAM , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_SPS ) +writel (MTK_VENC_IRQ_STATUS_SPS , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_PPS ) +writel (MTK_VENC_IRQ_STATUS_PPS , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_FRM ) +writel (MTK_VENC_IRQ_STATUS_FRM , addr); +} + +static irqreturn_t mtk_enc_core_irq_handler (int irq,void *priv) +{ +struct mtk_venc_core_dev *core = priv; +struct mtk_vcodec_ctx *ctx; +unsigned long flags; +void __iomem *addr; + +spin_lock_irqsave (&core->main_dev->irqlock, flags); + ctx = core->curr_ctx; +spin_unlock_irqrestore (&core->main_dev->irqlock, flags); +if (!ctx) +return IRQ_HANDLED ; + +mtk_v4l2_debug (1 ,"id=%d core :%d" , ctx->id, core->core_id); + + addr = core->reg_base +MTK_VENC_IRQ_ACK_OFFSET ; + ctx->irq_status =readl (core->reg_base +MTK_VENC_IRQ_STATUS_OFFSET ); +clean_irq_status (ctx->irq_status, addr); + +wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); +return IRQ_HANDLED ; +} + +static int mtk_venc_core_probe (struct platform_device *pdev)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 84
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+{ +struct device *dev = &pdev->dev; +struct mtk_venc_core_dev *core; +struct mtk_vcodec_dev *main_dev; +int ret; + +if (!dev->parent) { +dev_err (dev,"No parent for venc core device\n" ); +return -ENODEV ; + }
Please, use dev_err_probe(), here and everywhere else.
Quoted 8 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +8
-0+ + main_dev =dev_get_drvdata (dev->parent); +if (!main_dev) { +dev_err (dev,"Failed to get parent driver data" ); +return -EINVAL ; + } + + core =devm_kzalloc (&pdev->dev,sizeof (*core),GFP_KERNEL );
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 102
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+if (!core) +return -ENOMEM ; + + core->plat_dev = pdev; + + core->reg_base =devm_platform_ioremap_resource (pdev,0 ); +if (IS_ERR (core->reg_base)) { +dev_err (&pdev->dev,"Failed to get reg base" ); + ret =PTR_ERR (core->reg_base); + goto err;
You don't need the "err" label. Also, you can return dev_err_probe here too. Example:
return dev_err_probe(&pdev->dev, PTR_ERR(core->reg_base), "Failed to get reg base\n");
Quoted 11 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +11
-0+ } + + core->enc_irq =platform_get_irq (pdev,0 ); +if (core->enc_irq <0 ) { +dev_err (&pdev->dev,"Failed to get irq resource" ); + ret = core->enc_irq; + goto err; + } + + ret = devm_request_irq(&pdev->dev, core->enc_irq, + mtk_enc_core_irq_handler, 0,
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 123
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+ pdev->name, core); +if (ret) { +dev_err (&pdev->dev, +"Failed to install core->enc_irq %d (%d)" , + core->enc_irq, ret); + ret = -EINVAL ; + goto err; + } + + core->core_id =
core_id would be 0 if "mediatek,hw-leader", N if it's not a mediatek,hw-leader.
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 135
Quoted 12 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +12
-0+ (enum mtk_venc_core_id )of_device_get_match_data (&pdev->dev); +if (core->core_id <0 || core->core_id >=MTK_VENC_CORE_MAX ) { + ret = -EINVAL ; + goto err; + } + + main_dev->enc_core_dev[core->core_id] = core; + core->main_dev = main_dev; + +platform_set_drvdata (pdev, core); + +dev_info (dev,"Venc core :%d probe done\n" , core->core_id);
Please change this to dev_dbg().
Quoted 26 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +18
-0+ +return 0 ; + +err: +return ret; +} + +static struct platform_driver mtk_venc_core_driver = { + .probe = mtk_venc_core_probe, + .driver = { + .name ="mtk-venc-core" , + .of_match_table = mtk_venc_core_ids, + }, +}; +module_platform_driver (mtk_venc_core_driver); + +MODULE_LICENSE ("GPL v2" ); +MODULE_DESCRIPTION ("Mediatek video encoder core driver" );drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +2
-0@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/*
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h line 12
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +10
-0+* Copyright (c) 2021 MediaTek Inc. +*/ + +#ifndef _MTK_VCODEC_ENC_CORE_H_ +#define _MTK_VCODEC_ENC_CORE_H_ + +#include <linux/platform_device.h> +#include "mtk_vcodec_drv.h" + +/*
I love when developers try to write kerneldoc, this is something that should really be done *always*, so thank you for this!!!!
....but you missed something here: for this to be parsed by kerneldoc, you need to do:
/**
^^^ this one :))
* struct mtk_venc_core_dev - driver data
Quoted 14 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +14
-0+* struct mtk_venc_core_dev - driver data +* @plat_dev: platform_device +* @main_dev: main device +* @pm: power management data +* @curr_ctx: the context that is waiting for venc hardware +* @reg_base: mapped address of venc registers +* @irq_status: venc core irq status +* @enc_irq: venc device irq +* @core id: for venc core id: core#0, core#1... +*/ +struct mtk_venc_core_dev { +struct platform_device *plat_dev; +struct mtk_vcodec_dev *main_dev; +
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h line 36
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +10
-0+struct mtk_vcodec_pm pm; +struct mtk_vcodec_ctx *curr_ctx; + +void __iomem *reg_base; +unsigned int irq_status; +int enc_irq; + int core_id; +}; + +#endif /* _MTK_VCODEC_ENC_CORE_H_ */
Thanks,
Angelo
AngeloGioacchino Del Regno
maintainerRe: [PATCH v2, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Irui WangIl 17/01/22 13:06, Irui Wang ha scritto:
Quoted 55 lines
Adds encoder cores dt-bindings for mt8195
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ 1 file changed, 214 insertions(+)create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +41
-0@@ -0,0 +1,214 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present in Mediatek + SoCs which supports high resolution encoding functionalities. Required + parent and child device node. + +properties: + compatible: + const: mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + maxItems: 1 + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps to memory. + + "#address-cells":
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 51
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ const: 1 + + "#size-cells": + const: 1 + + ranges: true + +# Required child node: +patternProperties: + "venc_core0@1a020000":
This should be a pattern, venc-coreN where N is a number between 0 and 9, and any iostart should be allowed, not just 1a020000; doing it that way allows you to do a spec for all the core subnodes in one.
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +6
-0+ type: object + + properties: + compatible: + const: mediatek,mtk-venc-core0 +
..snip..
Quoted 16 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +16
-0+ +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + - ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h>
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 173
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ #include <dt-bindings/power/mt8195-power.h> + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <1>; + #size-cells = <1>; + ranges;
What about doing instead...
#address-cells = <2>;
#size-cells = <2>;
(or fix your dma-ranges property)
and...
ranges = <0 0 0 0x1a020000 0 0x1020000>;
venc-core0@0 {
compatible = "mediatek,mtk-venc-hw";
reg = <0 0 0 0x10000>;
mediatek,hw-leader;
..... other properties .....
};
venc-core1@1000000 {
compatible = "mediatek,mtk-venc-hw";
reg = <0 0x1000000 0 0x10000>;
..... other properties .....
}Regards,
Angelo
Quoted 41 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +41
-0+ + venc_core0@1a020000 { + compatible = "mediatek,mtk-venc-core0"; + reg = <0x1a020000 0x10000>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "MT_CG_VENC0"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc_core1@1b020000 { + compatible = "mediatek,mtk-venc-core1"; + reg = <0x1b020000 0x10000>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "MT_CG_VENC1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + };
Re: [PATCH v2, 04/10] media: mtk-vcodec: Enable venc dual core usage
In reply to AngeloGioacchino Del RegnoHello, Angelo,
Many thanks for your review.
Quoted 68 lines · AngeloGioacchino Del Regno
Il 17/01/22 13:06, Irui Wang ha scritto:
Quoted 55 lines
Adds new venc core mode to indicate different venc hardware mode:
VENC_SINGLE_CORE_MODEmeans only one core, the device has its own power/clk/irq, init_clk/request_irq helper can be used.
VENC_DUAL_CORE_MODEmeans more than one core inside, the core device
can use the init_clk/request_irq helper to initialize their own power/clk/irq. And the main device doesn't need use these helper anymore.MT8195 has two H264 venc cores, enable
dual_core_modefor it.Signed-off-by Irui Wang <[email protected]>--- drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 22 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 153 ++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 88 +++++----- 5 files changed, 266 insertions(+), 37 deletions(-)create mode 100644
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
create mode 100644drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.hdrivers/media/platform/mtk-vcodec/Makefile +3
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk-vcodec-enc-core.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +0
-0drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h enum mtk_vdec_hw_count { · line 125
Quoted 9 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +6
-0@@ -117,6 +117,23 @@ enum mtk_vdec_hw_count { MTK_VDEC_MAX_HW_COUNT, }; +/* +* enum mtk_venc_core_id -- encoder core id +*/ +enum mtk_venc_core_id { +MTK_VENC_CORE0 =0 , +MTK_VENC_CORE1 =1 ,You don't have to say "= 1" for core1, just...
Quoted 4 lines
MTK_VENC_CORE0 = 0, MTK_VENC_CORE1,...is fine, and better.
I will fix it.
Quoted 110 lines
Quoted 78 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +5
-0+ MTK_VENC_CORE_MAX, +}; + +/** +* enmu mtk_venc_core_mode - Used to indicate different encode mode
+ */drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +5
-0+enum mtk_venc_core_mode { +VENC_SINGLE_CORE_MODE =0 , +VENC_DUAL_CORE_MODE =1 , +}; +/* * struct mtk_video_fmt - Structure used to store information about pixelformats
*/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2
-0@@ -420,6 +437,7 @@ struct mtk_vcodec_dec_pdata {* @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats * @core_type: stand for h264 or vp8 encode +* @core_mode: indicate encode core mode */ struct mtk_vcodec_enc_pdata {bool uses_ext;@@ -430,6 +448,7 @@ struct mtk_vcodec_enc_pdata {const struct mtk_video_fmt *output_formats;size_t num_output_formats;int core_type; +enum mtk_venc_core_mode core_mode; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata-Quoted 1 line
uses_ext)drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +0
-0@@ -479,6 +498,7 @@ struct mtk_vcodec_enc_pdata {* @subdev_dev: subdev hardware device * @subdev_prob_done: check whether all used hw device is prob done
* @subdev_bitmap: used to record hardware is ready or not
+ * @enc_core_dev: used to store venc core device */
structmtk_vcodec_dev{
structv4l2_devicev4l2_dev;drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2
-0@@ -524,6 +544,8 @@ struct mtk_vcodec_dev {void *subdev_dev[MTK_VDEC_HW_MAX ];int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev);DECLARE_BITMAP (subdev_bitmap,MTK_VDEC_HW_MAX ); + +void *enc_core_dev[MTK_VENC_CORE_MAX ]; }; static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh*fh)
(quoted diff) +0
-0
vcodec/mtk_vcodec_enc_core.cb/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
new file mode 100644
index 000000000000..d84914f615a5drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +17
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> + +#include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_core.h" + +static const struct of_device_id mtk_venc_core_ids[] = {drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 27
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+ { + .compatible = "mediatek,mtk-venc-core0", + .data = (void *)MTK_VENC_CORE0, + }, + { + .compatible = "mediatek,mtk-venc-core1", + .data = (void *)MTK_VENC_CORE1, + }, + {}, +};Hello Irui,
You don't need a different compatible for the different cores, as in the
declaration, there's nothing special that differentiates them that much.I understand that there may be a need to differentiate the core number, as
in, CORE0 always has to be the leader, while CORE1 would be the follower,
but this is not a good reason to give them a different compatible string.I want to make you aware that Kyrie Wu did the same thing as you did here
and in my review on his patch I was able to give an extensive example of
how this should look; the exactly same logic would apply to this patch.
Quoted 6 lines
Please have a look here:
https://patchwork.kernel.org/comment/24726607/
P.S.: In short, you should have only one "mediatek,mtk-venc-hw" compatible
used for probing both cores.
thanks for your suggestions, with your example, venc can be rewritten like this:
venc {
compatible = "mediatek,mt8195-vcodec-enc";
..... other properties .....
venc_core0 {
compatible = "mediatek,mtk-venc-hw";
mediatek,hw-leader;//mediatek,venc-core0;
..... other properties .....
};
venc_core1 {
compatible = "mediatek,mtk-venc-hw";
//mediatek,venc-core1;
..... other properties .....
};
};I will rewrite this code if it matches your suggestions.
Quoted 54 lines
Quoted 54 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +3
-0+MODULE_DEVICE_TABLE (of, mtk_venc_core_ids); + +static void clean_irq_status(unsigned int irq_status, void __iomem*addr)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +37
-0+{ +if (irq_status &MTK_VENC_IRQ_STATUS_PAUSE ) +writel (MTK_VENC_IRQ_STATUS_PAUSE , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_SWITCH ) +writel (MTK_VENC_IRQ_STATUS_SWITCH , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_DRAM ) +writel (MTK_VENC_IRQ_STATUS_DRAM , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_SPS ) +writel (MTK_VENC_IRQ_STATUS_SPS , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_PPS ) +writel (MTK_VENC_IRQ_STATUS_PPS , addr); + +if (irq_status &MTK_VENC_IRQ_STATUS_FRM ) +writel (MTK_VENC_IRQ_STATUS_FRM , addr); +} + +static irqreturn_t mtk_enc_core_irq_handler (int irq,void *priv) +{ +struct mtk_venc_core_dev *core = priv; +struct mtk_vcodec_ctx *ctx; +unsigned long flags; +void __iomem *addr; + +spin_lock_irqsave (&core->main_dev->irqlock, flags); + ctx = core->curr_ctx; +spin_unlock_irqrestore (&core->main_dev->irqlock, flags); +if (!ctx) +return IRQ_HANDLED ; + +mtk_v4l2_debug (1 ,"id=%d core :%d" , ctx->id, core->core_id); + + addr = core->reg_base +MTK_VENC_IRQ_ACK_OFFSET ; + ctx->irq_status = readl(core->reg_base +
MTK_VENC_IRQ_STATUS_OFFSET);drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +11
-0+clean_irq_status (ctx->irq_status, addr); + +wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); +return IRQ_HANDLED ; +} + +static int mtk_venc_core_probe (struct platform_device *pdev) +{ +struct device *dev = &pdev->dev; +struct mtk_venc_core_dev *core; +struct mtk_vcodec_dev *main_dev;
Quoted 8 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 84
Quoted 6 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +6
-0+int ret; + +if (!dev->parent) { +dev_err (dev,"No parent for venc core device\n" ); +return -ENODEV ; + }Please, use
dev_err_probe(), here and everywhere else.
I will fix it
Quoted 24 lines
Quoted 9 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +8
-0+ + main_dev =dev_get_drvdata (dev->parent); +if (!main_dev) { +dev_err (dev,"Failed to get parent driver data" ); +return -EINVAL ; + } + + core =devm_kzalloc (&pdev->dev,sizeof (*core),GFP_KERNEL );drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 102
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+if (!core) +return -ENOMEM ; + + core->plat_dev = pdev; + + core->reg_base =devm_platform_ioremap_resource (pdev,0 ); +if (IS_ERR (core->reg_base)) { +dev_err (&pdev->dev,"Failed to get reg base" ); + ret =PTR_ERR (core->reg_base); + goto err;You don't need the "err" label. Also, you can return
dev_err_probehere too.
Example:
Quoted 3 lines
return dev_err_probe(&pdev->dev, PTR_ERR(core-Quoted 1 line
reg_base),"Failed to get reg base\n");
I will fix it
Quoted 28 lines
Quoted 12 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +11
-0+ } + + core->enc_irq =platform_get_irq (pdev,0 ); +if (core->enc_irq <0 ) { +dev_err (&pdev->dev,"Failed to get irq resource" ); + ret = core->enc_irq; + goto err; + } + + ret = devm_request_irq(&pdev->dev, core->enc_irq, + mtk_enc_core_irq_handler, 0,drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 123
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+ pdev->name, core); +if (ret) { +dev_err (&pdev->dev, +"Failed to install core->enc_irq %d (%d)" , + core->enc_irq, ret); + ret = -EINVAL ; + goto err; + } + + core->core_id =
core_idwould be 0 if "mediatek,hw-leader", N if it's not a mediatek,hw-leader.Quoted 2 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +1
-0+ (enum mtk_venc_core_id)of_device_get_match_data(&pdev-Quoted 1 line
dev);
Quoted 13 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 135
Quoted 11 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +11
-0+if (core->core_id <0 || core->core_id >=MTK_VENC_CORE_MAX ) { + ret = -EINVAL ; + goto err; + } + + main_dev->enc_core_dev[core->core_id] = core; + core->main_dev = main_dev; + +platform_set_drvdata (pdev, core); + +dev_info (dev,"Venc core :%d probe done\n" , core->core_id);Please change this to
dev_dbg().
I will fix it
Quoted 47 lines
Quoted 29 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +18
-0+ +return 0 ; + +err: +return ret; +} + +static struct platform_driver mtk_venc_core_driver = { + .probe = mtk_venc_core_probe, + .driver = { + .name ="mtk-venc-core" , + .of_match_table = mtk_venc_core_ids, + }, +}; +module_platform_driver (mtk_venc_core_driver); + +MODULE_LICENSE ("GPL v2" ); +MODULE_DESCRIPTION ("Mediatek video encoder core driver" );(quoted diff) +0
-0
vcodec/mtk_vcodec_enc_core.hb/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h
new file mode 100644
index 000000000000..856681989869drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +2
-0@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h line 12
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +10
-0+* Copyright (c) 2021 MediaTek Inc. +*/ + +#ifndef _MTK_VCODEC_ENC_CORE_H_ +#define _MTK_VCODEC_ENC_CORE_H_ + +#include <linux/platform_device.h> +#include "mtk_vcodec_drv.h" + +/* I love when developers try to write kerneldoc, this is something that should really be done *always*, so thank you for this!!!!
....but you missed something here: for this to be parsed by kerneldoc,
you need to do:
Quoted 6 lines
/**
^^^ this one :))* struct
mtk_venc_core_dev- driver data
I will fix it
Quoted 21 lines
Quoted 21 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +20
-0+* struct mtk_venc_core_dev - driver data +* @plat_dev: platform_device +* @main_dev: main device +* @pm: power management data +* @curr_ctx: the context that is waiting for venc hardware +* @reg_base: mapped address of venc registers +* @irq_status: venc core irq status +* @enc_irq: venc device irq +* @core id: for venc core id: core#0, core#1... +*/ +struct mtk_venc_core_dev { +struct platform_device *plat_dev; +struct mtk_vcodec_dev *main_dev; + +struct mtk_vcodec_pm pm; +struct mtk_vcodec_ctx *curr_ctx; + +void __iomem *reg_base; +unsigned int irq_status; +int enc_irq;
Quoted 8 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h line 36
Quoted 4 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +4
-0+ int core_id; +}; + +#endif /* _MTK_VCODEC_ENC_CORE_H_ */ Thanks,
Angelo
Thanks
Best Regards
Re: [PATCH v2, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to AngeloGioacchino Del RegnoHello, Angelo,
Many thanks for your review.
Quoted 78 lines · AngeloGioacchino Del Regno
Il 17/01/22 13:06, Irui Wang ha scritto:
Quoted 77 lines
Adds encoder cores dt-bindings for mt8195
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 214 ++++++++++++++++++ 1 file changed, 214 insertions(+)create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamldiff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-
core.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
new file mode 100644
index 000000000000..d1e7bfa50bceDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-0
encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0@@ -0,0 +1,214 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present inMediatek
+ SoCs which supports high resolution encoding functionalities. RequiredDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +18
-0+ parent and child device node. + +properties: + compatible: + const: mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + maxItems: 1 + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps tomemory.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10
-0+ + "#address-cells": + const: 1 + + "#size-cells": + const: 1 + + ranges: true + +# Required child node:
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 2 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+patternProperties: + "venc_core0@1a020000":This should be a pattern, venc-coreN where N is a number between 0 and 9,
and any iostart should be allowed, not just1a020000; doing it that way
allows you to do a spec for all the core subnodes in one.
I will fix it.
Quoted 40 lines
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ type: object + + properties: + compatible: + const: mediatek,mtk-venc-core0 +..snip..
Quoted 16 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +16
-0+ +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + - ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h>Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10
-0+ #include <dt-bindings/power/mt8195-power.h> + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <1>; + #size-cells = <1>; + ranges;What about doing instead...
Quoted 3 lines
#address-cells = <2>;
#size-cells = <2>;
(or fix your dma-ranges property)
cells=2 may cause some unexpected errors, I will check it again.
Quoted 17 lines
and...
ranges = <0 0 0 0x1a020000 0 0x1020000>; venc-core0@0 { compatible = "mediatek,mtk-venc-hw"; reg = <0 0 0 0x10000>; mediatek,hw-leader; ..... other properties ..... }; venc-core1@1000000 { compatible = "mediatek,mtk-venc-hw"; reg = <0 0x1000000 0 0x10000>; ..... other properties ..... }
Quoted 2 lines
Regards,
Angelo
I will rewrite the examples.
Thanks
Best Regards
Quoted 46 lines
Quoted 44 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +18
-0+ + venc_core0@1a020000 { + compatible = "mediatek,mtk-venc-core0"; + reg = <0x1a020000 0x10000>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "MT_CG_VENC0"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen
CLK_TOP_UNIVPLL_D4>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +20
-0+ power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc_core1@1b020000 { + compatible = "mediatek,mtk-venc-core1"; + reg = <0x1b020000 0x10000>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "MT_CG_VENC1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen
CLK_TOP_UNIVPLL_D4>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + };
AngeloGioacchino Del Regno
maintainerRe: [PATCH v2, 04/10] media: mtk-vcodec: Enable venc dual core usage
In reply to Irui WangIl 04/03/22 03:12, Irui Wang ha scritto:
Quoted 215 lines
Hello, Angelo,
Many thanks for your review.
Quoted 68 lines · AngeloGioacchino Del Regno
Il 17/01/22 13:06, Irui Wang ha scritto:
Quoted 55 lines
Adds new venc core mode to indicate different venc hardware mode:
VENC_SINGLE_CORE_MODEmeans only one core, the device has its own power/clk/irq, init_clk/request_irq helper can be used.
VENC_DUAL_CORE_MODEmeans more than one core inside, the core device
can use the init_clk/request_irq helper to initialize their own power/clk/irq. And the main device doesn't need use these helper anymore.MT8195 has two H264 venc cores, enable
dual_core_modefor it.Signed-off-by Irui Wang <[email protected]>--- drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 22 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 153 ++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 88 +++++----- 5 files changed, 266 insertions(+), 37 deletions(-)create mode 100644
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
create mode 100644drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.hdrivers/media/platform/mtk-vcodec/Makefile +3
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk-vcodec-enc-core.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +0
-0drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h enum mtk_vdec_hw_count { · line 125
Quoted 9 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +6
-0@@ -117,6 +117,23 @@ enum mtk_vdec_hw_count { MTK_VDEC_MAX_HW_COUNT, }; +/* +* enum mtk_venc_core_id -- encoder core id +*/ +enum mtk_venc_core_id { +MTK_VENC_CORE0 =0 , +MTK_VENC_CORE1 =1 ,You don't have to say "= 1" for core1, just...
Quoted 4 lines
MTK_VENC_CORE0 = 0, MTK_VENC_CORE1,...is fine, and better.
I will fix it.
Quoted 110 lines
Quoted 78 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +5
-0+ MTK_VENC_CORE_MAX, +}; + +/** +* enmu mtk_venc_core_mode - Used to indicate different encode mode
+ */drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +5
-0+enum mtk_venc_core_mode { +VENC_SINGLE_CORE_MODE =0 , +VENC_DUAL_CORE_MODE =1 , +}; +/* * struct mtk_video_fmt - Structure used to store information about pixelformats
*/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2
-0@@ -420,6 +437,7 @@ struct mtk_vcodec_dec_pdata {* @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats * @core_type: stand for h264 or vp8 encode +* @core_mode: indicate encode core mode */ struct mtk_vcodec_enc_pdata {bool uses_ext;@@ -430,6 +448,7 @@ struct mtk_vcodec_enc_pdata {const struct mtk_video_fmt *output_formats;size_t num_output_formats;int core_type; +enum mtk_venc_core_mode core_mode; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata-Quoted 1 line
uses_ext)drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +0
-0@@ -479,6 +498,7 @@ struct mtk_vcodec_enc_pdata {* @subdev_dev: subdev hardware device * @subdev_prob_done: check whether all used hw device is prob done
* @subdev_bitmap: used to record hardware is ready or not
+ * @enc_core_dev: used to store venc core device*/ struct mtk_vcodec_dev { struct v4l2_device v4l2_dev;drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2
-0@@ -524,6 +544,8 @@ struct mtk_vcodec_dev {void *subdev_dev[MTK_VDEC_HW_MAX ];int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev);DECLARE_BITMAP (subdev_bitmap,MTK_VDEC_HW_MAX ); + +void *enc_core_dev[MTK_VENC_CORE_MAX ]; }; static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh*fh)
(quoted diff) +0
-0
vcodec/mtk_vcodec_enc_core.cb/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
new file mode 100644
index 000000000000..d84914f615a5drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +17
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> + +#include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_core.h" + +static const struct of_device_id mtk_venc_core_ids[] = {drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 27
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+ { + .compatible = "mediatek,mtk-venc-core0", + .data = (void *)MTK_VENC_CORE0, + }, + { + .compatible = "mediatek,mtk-venc-core1", + .data = (void *)MTK_VENC_CORE1, + }, + {}, +};Hello Irui,
You don't need a different compatible for the different cores, as in the
declaration, there's nothing special that differentiates them that much.I understand that there may be a need to differentiate the core number, as
in, CORE0 always has to be the leader, while CORE1 would be the follower,
but this is not a good reason to give them a different compatible string.I want to make you aware that Kyrie Wu did the same thing as you did here
and in my review on his patch I was able to give an extensive example of
how this should look; the exactly same logic would apply to this patch.Quoted 6 lines
Please have a look here:
https://patchwork.kernel.org/comment/24726607/
P.S.: In short, you should have only one "mediatek,mtk-venc-hw" compatible
used for probing both cores.thanks for your suggestions, with your example, venc can be rewritten like this:
venc { compatible = "mediatek,mt8195-vcodec-enc"; ..... other properties ..... venc_core0 { compatible = "mediatek,mtk-venc-hw"; mediatek,hw-leader;//mediatek,venc-core0; ..... other properties ..... }; venc_core1 { compatible = "mediatek,mtk-venc-hw"; //mediatek,venc-core1; ..... other properties ..... }; };
Quoted 1 line
I will rewrite this code if it matches your suggestions.
Yes, exactly. Just one nit, please don't use underscores.
venc_core0: venc-hw@(addr)
this is fine ^
venc_core0: venc_hw@(addr)
this is NOT ok ^^
By the way, one (or more than one) of the commits in this series is not working correctly, giving a kernel panic on dma mem alloc.
Looking forward to see the new version!
Regards,
Angelo
Re: [PATCH v2, 04/10] media: mtk-vcodec: Enable venc dual core usage
Hello, Angelo,
thanks for your review comments. I will update my new device tree, example inline:
Quoted 241 lines · AngeloGioacchino Del Regno
Il 04/03/22 03:12, Irui Wang ha scritto:
Quoted 230 lines
Hello, Angelo,
Many thanks for your review.
Quoted 72 lines · AngeloGioacchino Del Regno
Il 17/01/22 13:06, Irui Wang ha scritto:
Quoted 59 lines
Adds new venc core mode to indicate different venc hardware mode:
VENC_SINGLE_CORE_MODEmeans only one core, the device has its own
power/clk/irq, init_clk/request_irq helper can be used.
VENC_DUAL_CORE_MODEmeans more than one core inside, the core device
can use the init_clk/request_irq helper to initialize their own power/clk/irq. And the main device doesn't need use these helper
anymore.MT8195 has two H264 venc cores, enable
dual_core_modefor it.Signed-off-by Irui Wang <[email protected]>--- drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 22 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 153 ++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 88 +++++--- -- 5 files changed, 266 insertions(+), 37 deletions(-)create mode 100644
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
create mode 100644drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.hdrivers/media/platform/mtk-vcodec/Makefile +3
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk-vcodec-enc-core.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +0
-0drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h enum mtk_vdec_hw_count { · line 125
Quoted 9 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +6
-0@@ -117,6 +117,23 @@ enum mtk_vdec_hw_count { MTK_VDEC_MAX_HW_COUNT, }; +/* +* enum mtk_venc_core_id -- encoder core id +*/ +enum mtk_venc_core_id { +MTK_VENC_CORE0 =0 , +MTK_VENC_CORE1 =1 ,You don't have to say "= 1" for core1, just...
Quoted 4 lines
MTK_VENC_CORE0 = 0, MTK_VENC_CORE1,...is fine, and better.
I will fix it.
Quoted 119 lines
Quoted 83 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +5
-0+ MTK_VENC_CORE_MAX, +}; + +/** +* enmu mtk_venc_core_mode - Used to indicate different encode mode
+ */drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +5
-0+enum mtk_venc_core_mode { +VENC_SINGLE_CORE_MODE =0 , +VENC_DUAL_CORE_MODE =1 , +}; +/* * struct mtk_video_fmt - Structure used to store information
about pixelformats*/drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2
-0@@ -420,6 +437,7 @@ struct mtk_vcodec_dec_pdata {* @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats * @core_type: stand for h264 or vp8 encode +* @core_mode: indicate encode core mode */ struct mtk_vcodec_enc_pdata {bool uses_ext;@@ -430,6 +448,7 @@ struct mtk_vcodec_enc_pdata {const struct mtk_video_fmt *output_formats;size_t num_output_formats;int core_type; +enum mtk_venc_core_mode core_mode; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata-Quoted 1 line
uses_ext)drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +0
-0@@ -479,6 +498,7 @@ struct mtk_vcodec_enc_pdata {* @subdev_dev: subdev hardware device * @subdev_prob_done: check whether all used hw device is prob
done
* @subdev_bitmap: used to record hardware is ready or not
+ * @enc_core_dev: used to store venc core device*/ struct mtk_vcodec_dev { struct v4l2_device v4l2_dev;drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2
-0@@ -524,6 +544,8 @@ struct mtk_vcodec_dev {void *subdev_dev[MTK_VDEC_HW_MAX ];int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev);DECLARE_BITMAP (subdev_bitmap,MTK_VDEC_HW_MAX ); + +void *enc_core_dev[MTK_VENC_CORE_MAX ]; };static inline struct
mtk_vcodec_ctx*fh_to_ctx(structv4l2_fh
*fh)(quoted diff) +0
-0
vcodec/mtk_vcodec_enc_core.cb/drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
new file mode 100644
index 000000000000..d84914f615a5drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +17
-0@@ -0,0 +1,153 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> + +#include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_core.h" + +static const struct of_device_id mtk_venc_core_ids[] = {drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c line 27
Quoted 10 lines
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +10
-0+ { + .compatible = "mediatek,mtk-venc-core0", + .data = (void *)MTK_VENC_CORE0, + }, + { + .compatible = "mediatek,mtk-venc-core1", + .data = (void *)MTK_VENC_CORE1, + }, + {}, +};Hello Irui,
You don't need a different compatible for the different cores, as in
the
declaration, there's nothing special that differentiates them that
much.I understand that there may be a need to differentiate the core number, as
in, CORE0 always has to be the leader, while CORE1 would be the follower,
but this is not a good reason to give them a different compatible string.I want to make you aware that Kyrie Wu did the same thing as you did
here
and in my review on his patch I was able to give an extensive example
of
how this should look; the exactly same logic would apply to this patch.Quoted 6 lines
Please have a look here:
https://patchwork.kernel.org/comment/24726607/
P.S.: In short, you should have only one "mediatek,mtk-venc-hw" compatible
used for probing both cores.thanks for your suggestions, with your example, venc can be rewritten
like this:venc { compatible = "mediatek,mt8195-vcodec-enc"; ..... other properties ..... venc_core0 { compatible = "mediatek,mtk-venc-hw"; mediatek,hw-leader;//mediatek,venc-core0; ..... other properties ..... }; venc_core1 { compatible = "mediatek,mtk-venc-hw"; //mediatek,venc-core1; ..... other properties ..... }; };Quoted 1 line
I will rewrite this code if it matches your suggestions.
Yes, exactly. Just one nit, please don't use underscores.
venc_core0: venc-hw@(addr)
this is fine ^
venc_core0: venc_hw@(addr)
this is NOT ok ^^
Quoted 7 lines
By the way, one (or more than one) of the commits in this series is not working correctly, giving a kernel panic on dma mem alloc.
Looking forward to see the new version!
Regards,
Angelo
We add a new property "mediatek,core-id" to indicite current encoder core id.
if we enable the two encoder cores, it will need more dma mem buffers for encoding, in patch-08 add a new venc-vsi to store these encoding working buffers, both kernel and user side need update the venc-vsi, otherwise, we will get a panic on mem alloc.
There are still some encoding issues after apply the device tree, we still working on it, and we will send a new version patches once wesolve them.
Thanks
Best Regards
example dtsi
soc {
#address-cells = <2>;
#size-cells = <2>;
venc {
compatible = "mediatek,mt8195-vcodec-enc";
mediatek,scp = <&scp>;
iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>;
#address-cells = <2>;
#size-cells = <2>;
ranges;
dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>;
venc-core@1a020000 {
compatible = "mediatek,mtk-venc-core";
reg = <0 0x1a020000 0 0x10000>;
mediatek,core-id = <0>;
iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>,
<&iommu_vdo M4U_PORT_L19_VENC_REC>,
<&iommu_vdo M4U_PORT_L19_VENC_BSDMA>,
<&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>,
<&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>,
<&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>,
<&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>,
<&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>,
<&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>;
interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>;
clocks = <&vencsys CLK_VENC_VENC>;
clock-names = "clk_venc";
assigned-clocks = <&topckgen CLK_TOP_VENC>;
assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
power-domains = <&spm MT8195_POWER_DOMAIN_VENC>;
};
venc-core@1b020000 {
compatible = "mediatek,mtk-venc-core";
reg = <0 0x1b020000 0 0x10000>;
mediatek,core-id = <1>;
iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>,
<&iommu_vpp M4U_PORT_L20_VENC_REC>,
<&iommu_vpp M4U_PORT_L20_VENC_BSDMA>,
<&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>,
<&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>,
<&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>,
<&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>,
<&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>,
<&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>;
interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>;
clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>;
clock-names = "clk_venc_core1";
assigned-clocks = <&topckgen CLK_TOP_VENC>;
assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>;
};
};
};