Enable two H264 encoder core on MT8195
Subsystems (4)
v3 cover Enable two H264 encoder core on MT8195
MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps.
The two encoder core encoding process looks like this:
VENC Core0: frm#0....frm#2....frm#4....
VENC Core1: ..frm#1....frm#3....frm#5....This series of patches are used to enable the two H264 encoder cores, encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
...
To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.
There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance.
---
This series patches dependent on:media_stage tree:
[1]
https://git.linuxtv.org/media_stage.git/commit/?id=b3627647f9ea7473d10fb08a95fd7c4133a17ca4
mt8192 decoder:
[2] https://patchwork.kernel.org/project/linux-mediatek/list/?series=616991
patch3 new venc dt-bindings included files
[3] MM IOMMU binding:
https://patchwork.kernel.org/project/linux-mediatek/patch/[email protected]/
[4] MT8195 power domain:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579
changes compared with v2:
- update venc core dt-bindings, add two new properties for current usage.
- parse venc multi_core mode from device tree.
- rebase to the newer linux media stage.
changes compared with v1:
- of_platform_populate was used in place of the component framework.
- new yaml file for venc cores.
- some modifications for patch v1's review comments.
---
Irui Wang (10):
media: mtk-vcodec: Use core type to indicate h264 and vp8 enc
media: mtk-vcodec: export encoder functions
dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for
mt8195
media: mtk-vcodec: Enable venc dual core usage
media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface
media: mtk-vcodec: Add venc power on/off interface
media: mtk-vcodec: Rewrite venc clock interface
media: mtk-vcodec: Add more extra processing for venc_multi_core mode
media: mtk-vcodec: Add venc_multi_core mode encode process
media: mtk-vcodec: Done encode result to client
.../media/mediatek,vcodec-encoder-core.yaml | 181 +++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 36 ++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 113 +++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 7 +- .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 169 ++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 107 +++++---- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 183 +++++++++++++-- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 11 +- .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 5 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 212 +++++++++++++++--- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 79 +++++-- .../media/platform/mtk-vcodec/venc_drv_if.h | 7 + .../media/platform/mtk-vcodec/venc_vpu_if.c | 11 +- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 19 files changed, 1026 insertions(+), 161 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h
Signature · 1 line
2.18.0
PATCH v3 1/10 media: mtk-vcodec: Use core type to indicate h264 and vp8 enc
use core type variables to indicate h264 and vp8 encoder
--- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 4 +-- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 30 ++++++++++--------- 2 files changed, 18 insertions(+), 16 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2 -2
@@ -416,7 +416,7 @@ struct mtk_vcodec_dec_pdata {* @num_capture_formats: number of entries in capture_formats * @output_formats: array of supported output formats * @num_output_formats: number of entries in output_formats -* @core_id: stand for h264 or vp8 encode index +* @core_type: stand for h264 or vp8 encode */ struct mtk_vcodec_enc_pdata {bool uses_ext;
@@ -426,7 +426,7 @@ struct mtk_vcodec_enc_pdata {size_t num_capture_formats;const struct mtk_video_fmt *output_formats;size_t num_output_formats; -int core_id; +int core_type; };#define MTK_ENC_CTX_IS_EXT (ctx) ((ctx)->dev->venc_pdata->uses_ext)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +16 -14
@@ -94,11 +94,13 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv) ctx = dev->curr_ctx;spin_unlock_irqrestore (&dev->irqlock, flags); -mtk_v4l2_debug (1 ,"id=%d coreid:%d" , ctx->id, dev->venc_pdata->core_id); - addr = dev->reg_base[dev->venc_pdata->core_id] + +mtk_v4l2_debug (1 ,"id: %d core type: %d" , + ctx->id, dev->venc_pdata->core_type); + + addr = dev->reg_base[dev->venc_pdata->core_type] +MTK_VENC_IRQ_ACK_OFFSET ; - ctx->irq_status =readl (dev->reg_base[dev->venc_pdata->core_id] + + ctx->irq_status =readl (dev->reg_base[dev->venc_pdata->core_type] + (MTK_VENC_IRQ_STATUS_OFFSET ));clean_irq_status (ctx->irq_status, addr);
@@ -265,10 +267,10 @@ static int mtk_vcodec_probe(struct platform_device *pdev)pm_runtime_enable (&pdev->dev); - dev->reg_base[dev->venc_pdata->core_id] = + dev->reg_base[dev->venc_pdata->core_type] =devm_platform_ioremap_resource (pdev,0 ); -if (IS_ERR (dev->reg_base[dev->venc_pdata->core_id])) { - ret =PTR_ERR (dev->reg_base[dev->venc_pdata->core_id]); +if (IS_ERR (dev->reg_base[dev->venc_pdata->core_type])) { + ret =PTR_ERR (dev->reg_base[dev->venc_pdata->core_type]); goto err_res; }
@@ -286,8 +288,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev) 0, pdev->name, dev);if (ret) {dev_err (&pdev->dev, -"Failed to install dev->enc_irq %d (%d) core_id (%d)" , - dev->enc_irq, ret, dev->venc_pdata->core_id); +"Failed to install dev->enc_irq %d (%d) core_type (%d)" , + dev->enc_irq, ret, dev->venc_pdata->core_type); ret = -EINVAL ; goto err_res; }
@@ -356,7 +358,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) }mtk_v4l2_debug (0 ,"encoder %d registered as /dev/video%d" , - dev->venc_pdata->core_id, vfd_enc->num); + dev->venc_pdata->core_type, vfd_enc->num);return 0 ;
@@ -382,7 +384,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_avc_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 60000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
@@ -392,7 +394,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 9000000, - .core_id = VENC_LT_SYS, + .core_type = VENC_LT_SYS, };static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
@@ -403,7 +405,7 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 40000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
@@ -414,7 +416,7 @@ static const struct mtk_vcodec_enc_pdata mt8192_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 100000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
@@ -425,7 +427,7 @@ static const struct mtk_vcodec_enc_pdata mt8195_pdata = { .num_output_formats = ARRAY_SIZE(mtk_video_formats_output), .min_bitrate = 64, .max_bitrate = 100000000, - .core_id = VENC_SYS, + .core_type = VENC_SYS, };static const struct of_device_id mtk_vcodec_enc_match[] = {
Signature · 1 line
2.18.0
PATCH v3 2/10 media: mtk-vcodec: export encoder functions
In reply to Irui Wangmtk vcodec is built as a module, export some functions to make them visible by other modules.
--- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c | 2 ++ drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 3 +++ 2 files changed, 5 insertions(+)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +2 -0
@@ -1431,6 +1431,7 @@ int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)mutex_unlock (&dev->enc_mutex);return 0 ; } +EXPORT_SYMBOL_GPL (mtk_venc_unlock);int mtk_venc_lock (struct mtk_vcodec_ctx *ctx) {
@@ -1439,6 +1440,7 @@ int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)mutex_lock (&dev->enc_mutex);return 0 ; } +EXPORT_SYMBOL_GPL (mtk_venc_lock);void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx) {
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +3 -0
@@ -58,6 +58,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)return 0 ; } +EXPORT_SYMBOL_GPL (mtk_vcodec_init_enc_clk);void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm) {
@@ -79,6 +80,7 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)for (i -=1 ; i >=0 ; i--)clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_on);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm) {
@@ -88,3 +90,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)for (i = enc_clk->clk_num -1 ; i >=0 ; i--)clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_off);
Signature · 1 line
2.18.0
PATCH v3 3/10 dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Irui WangAdds encoder cores dt-bindings for mt8195.
--- .../media/mediatek,vcodec-encoder-core.yaml | 181 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 181 insertions(+), 1 deletion(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +181 -0
@@ -0,0 +1,181 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present in Mediatek + SoCs which supports high resolution encoding functionalities. Required + parent and child device node. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + mediatek,venc-multi-core: + type: boolean + description: | + Indicates whether the encoder has multiple cores or not. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps to memory. + + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be added as subnodes to + the main venc node. + + properties: + compatible: + items: + - const: mediatek,mtk-venc-core + + reg: + maxItems: 1 + + mediatek,core-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - mediatek,core-id + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + soc { + #address-cells = <2>; + #size-cells = <2>; + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + mediatek,venc-multi-core; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + + venc-core@1a020000 { + compatible = "mediatek,mtk-venc-core"; + reg = <0 0x1a020000 0 0x10000>; + mediatek,core-id = <0>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "clk_venc"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc-core@1b020000 { + compatible = "mediatek,mtk-venc-core"; + reg = <0 0x1b020000 0 0x10000>; + mediatek,core-id = <1>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "clk_venc_core1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + }; + };
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml +0 -1
@@ -21,7 +21,6 @@ properties: - mediatek,mt8173-vcodec-enc - mediatek,mt8183-vcodec-enc - mediatek,mt8192-vcodec-enc - - mediatek,mt8195-vcodec-enc reg: maxItems: 1
Signature · 1 line
2.18.0
Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
Hi Irui,
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 8
Quoted 7 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +7
-0+ +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: Mediatek Video Encoder Accelerator With Multi Core
s/Mediatek/MediaTek/
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 14
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +6
-0+ +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present in Mediatek
s/Mediatek/MediaTek/
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 17
Quoted 3 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +3
-0+ SoCs which supports high resolution encoding functionalities. Required + parent and child device node. +
Thanks,
Miles
Rob Herring
maintainerRe: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
In reply to Irui WangQuoted 1 line · Irui Wang
Adds encoder cores dt-bindings for mt8195.
Why?
Quoted 37 lines
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 181 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 181 insertions(+), 1 deletion(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +23
-0@@ -0,0 +1,181 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present in Mediatek + SoCs which supports high resolution encoding functionalities. Required + parent and child device node. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc +
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 33
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + mediatek,venc-multi-core: + type: boolean + description: | + Indicates whether the encoder has multiple cores or not.
Isn't this implied by how many child nodes you have?
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 37
Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +4
-0+ + iommus: + minItems: 1 + maxItems: 32
You really have up to 32 IOMMUs?
Quoted 27 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +27
-0+ description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps to memory. + + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be added as subnodes to + the main venc node. + + properties: + compatible:
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 74
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ items: + - const: mediatek,mtk-venc-core + + reg: + maxItems: 1 + + mediatek,core-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id.
What is this for and what does its value correspond to in the h/w. We generally don't do made up indices in DT.
Quoted 123 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +107
-0+ + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - mediatek,core-id + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + soc { + #address-cells = <2>; + #size-cells = <2>; + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + mediatek,venc-multi-core; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + + venc-core@1a020000 { + compatible = "mediatek,mtk-venc-core"; + reg = <0 0x1a020000 0 0x10000>; + mediatek,core-id = <0>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "clk_venc"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc-core@1b020000 { + compatible = "mediatek,mtk-venc-core"; + reg = <0 0x1b020000 0 0x10000>; + mediatek,core-id = <1>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "clk_venc_core1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + }; + };Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml +0
-1@@ -21,7 +21,6 @@ properties: - mediatek,mt8173-vcodec-enc - mediatek,mt8183-vcodec-enc - mediatek,mt8192-vcodec-enc - - mediatek,mt8195-vcodec-enc reg: maxItems: 1Signature · 1 line
2.18.0
Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
Dear Rob,
Thanks for your review and comments.
Quoted 4 lines · Rob Herring
Quoted 1 line · Irui Wang
Adds encoder cores dt-bindings for mt8195.
Why?
mt8195 has two h264 encoder cores and we want to enable both of them. the original encoder bindings file is for single core which may can not handle it, so we new a core dt-bindings. We will think about it again about this dt-bindings.
Quoted 54 lines
Quoted 54 lines
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 181 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 181 insertions(+), 1 deletion(-) create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamldiff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-
core.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
new file mode 100644
index 000000000000..fcfb48900c76Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-0
encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0@@ -0,0 +1,181 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present inMediatek
+ SoCs which supports high resolution encoding functionalities. RequiredDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +12
-0+ parent and child device node. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), using
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ the remoteproc & rpmsg framework. + + mediatek,venc-multi-core: + type: boolean + description: | + Indicates whether the encoder has multiple cores or not.Isn't this implied by how many child nodes you have?
It implies that we have child nodes, distinguish with single core,
Quoted 7 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 5 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +4
-0+ + iommus: + minItems: 1 + maxItems: 32You really have up to 32 IOMMUs?
Currently, totally 27 IOMMUs and used 9 of them.
Quoted 37 lines
Quoted 37 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps tomemory.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +15
-0+ + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be added assubnodes to
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10
-0+ the main venc node. + + properties: + compatible: + items: + - const: mediatek,mtk-venc-core + + reg: + maxItems: 1 +
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +4
-0+ mediatek,core-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id.What is this for and what does its value correspond to in the h/w. We
generally don't do made up indices in DT.
It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?
Quoted 132 lines
Quoted 132 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +69
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - mediatek,core-id + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - mediatek,scp + - iommus + - dma-ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + soc { + #address-cells = <2>; + #size-cells = <2>; + + venc { + compatible = "mediatek,mt8195-vcodec-enc"; + mediatek,scp = <&scp>; + mediatek,venc-multi-core; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + + venc-core@1a020000 { + compatible = "mediatek,mtk-venc-core"; + reg = <0 0x1a020000 0 0x10000>; + mediatek,core-id = <0>; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo
M4U_PORT_L19_VENC_CUR_CHROMA>,Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo
M4U_PORT_L19_VENC_REF_CHROMA>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0+ interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "clk_venc"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen
CLK_TOP_UNIVPLL_D4>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +14
-0+ power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + }; + + venc-core@1b020000 { + compatible = "mediatek,mtk-venc-core"; + reg = <0 0x1b020000 0 0x10000>; + mediatek,core-id = <1>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp
M4U_PORT_L20_VENC_CUR_CHROMA>,Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp
M4U_PORT_L20_VENC_REF_CHROMA>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0+ interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "clk_venc_core1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen
CLK_TOP_UNIVPLL_D4>;
+ power-domains = <&spmMT8195_POWER_DOMAIN_VENC_CORE1>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ }; + }; + };diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-
encoder.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
indexe7b65a91c92c..0530a694bcbe100644
--- a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-1@@ -21,7 +21,6 @@ properties: - mediatek,mt8173-vcodec-enc - mediatek,mt8183-vcodec-enc - mediatek,mt8192-vcodec-enc - - mediatek,mt8195-vcodec-enc
Quoted 6 lines
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-0reg: maxItems: 1Signature · 1 line
2.18.0
Thanks
Best Regards
Rob Herring
maintainerRe: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
Quoted 75 lines · Irui Wang
Dear Rob,
Thanks for your review and comments.
Quoted 4 lines · Rob Herring
Quoted 1 line · Irui Wang
Adds encoder cores dt-bindings for mt8195.
Why?
mt8195 has two h264 encoder cores and we want to enable both of them. the original encoder bindings file is for single core which may can not handle it, so we new a core dt-bindings. We will think about it again about this dt-bindings.
Quoted 54 lines
Quoted 54 lines
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 181 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 181 insertions(+), 1 deletion(-) create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamldiff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-
core.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
new file mode 100644
index 000000000000..fcfb48900c76Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-0
encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0@@ -0,0 +1,181 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware present inMediatek
+ SoCs which supports high resolution encoding functionalities. RequiredDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +12
-0+ parent and child device node. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), usingQuoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ the remoteproc & rpmsg framework. + + mediatek,venc-multi-core: + type: boolean + description: | + Indicates whether the encoder has multiple cores or not.Isn't this implied by how many child nodes you have?
Quoted 1 line
It implies that we have child nodes, distinguish with single core,
Just check if you have a child node or not.
Quoted 51 lines
Quoted 5 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 3 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ iommus: + minItems: 1 + maxItems: 32You really have up to 32 IOMMUs?
Currently, totally 27 IOMMUs and used 9 of them.
Quoted 37 lines
Quoted 37 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps tomemory.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +15
-0+ + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be added assubnodes to
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10
-0+ the main venc node. + + properties: + compatible: + items: + - const: mediatek,mtk-venc-core + + reg: + maxItems: 1 +Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +4
-0+ mediatek,core-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id.What is this for and what does its value correspond to in the h/w. We
generally don't do made up indices in DT.
Quoted 4 lines
It's for encoder core id, core@1a020000 must be core-0, core@1b020000 must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?
I still don't understand why it is needed. What is 'core-0'?
Is there some functional difference between the cores? If so, describe that difference.
Rob
Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
Dear Rob,
Many thanks for your attention.
Quoted 42 lines · Rob Herring
Quoted 41 lines · Irui Wang
Dear Rob,
Thanks for your review and comments.
Quoted 4 lines · Rob Herring
Quoted 1 line · Irui Wang
Adds encoder cores dt-bindings for mt8195.
Why?
mt8195 has two h264 encoder cores and we want to enable both of them.
the original encoder bindings file is for single core which may can not
handle it, so we new a core dt-bindings. We will think about it again
about this dt-bindings.Quoted 24 lines
Quoted 24 lines
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 181 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 181 insertions(+), 1 deletion(-) create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamldiff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder-
core.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder-core.yaml
new file mode 100644
index 000000000000..fcfb48900c76Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-0
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- line 5
Quoted 8 lines
encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0@@ -0,0 +1,181 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "
http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
Quoted 36 lines
Quoted 36 lines
Quoted 35 lines
Quoted 23 lines
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ +title: Mediatek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + Mediatek Video Encode is the video encode hardware presentin
Mediatek
+ SoCs which supports high resolution encoding
functionalities.
RequiredDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ parent and child device node. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc +Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10
-0+ mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + mediatek,venc-multi-core: + type: boolean + description: | + Indicates whether the encoder has multiple cores or not.Isn't this implied by how many child nodes you have?
Quoted 4 lines
Quoted 2 lines
It implies that we have child nodes, distinguish with single core,
Just check if you have a child node or not.
Yes.
Quoted 63 lines
Quoted 57 lines
Quoted 7 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 5 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ iommus: + minItems: 1 + maxItems: 32You really have up to 32 IOMMUs?
Currently, totally 27 IOMMUs and used 9 of them.
Quoted 39 lines
Quoted 39 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps tomemory.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +15
-0+ + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be addedas
subnodes toDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +11
-0+ the main venc node. + + properties: + compatible: + items: + - const: mediatek,mtk-venc-core + + reg: + maxItems: 1 + + mediatek,core-id:Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 3 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id.What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.Quoted 5 lines
It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?
Quoted 7 lines
I still don't understand why it is needed. What is 'core-0'?
Is there some functional difference between the cores? If so, describe
that difference.Rob
They are two different pieces of hardware, it's our encoder hardware design. There are two encoder hardware cores inside MT8195, named core0 and core1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1".
So the two encoder cores have their own base, IRQ, clock, power, etc. Each core can encode independently, moreover, they can work together for higher performance.
We will describe more details in YAML about it if it's OK for you.
Thanks
Best Regards.
Rob Herring
maintainerRe: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
Quoted 13 lines · Irui Wang
Dear Rob,
Many thanks for your attention.
Quoted 8 lines · Rob Herring
Quoted 7 lines · Irui Wang
Dear Rob,
Thanks for your review and comments.
Quoted 2 lines · Rob Herring
Quoted 1 line · Irui Wang
Adds encoder cores dt-bindings for mt8195.
[...]
Quoted 24 lines
Quoted 16 lines
Quoted 10 lines
Quoted 9 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 74
Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +4
-0+ mediatek,core-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id.What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.Quoted 5 lines
It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?Quoted 7 lines
I still don't understand why it is needed. What is 'core-0'?
Is there some functional difference between the cores? If so, describe
that difference.Rob
Quoted 11 lines
They are two different pieces of hardware, it's our encoder hardware design. There are two encoder hardware cores inside MT8195, named core0 and
core1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1".
So the two encoder cores have their own base, IRQ, clock, power, etc. Each core can encode independently, moreover, they can work together for higher performance.
We will describe more details in YAML about it if it's OK for you.
All the resources you list are in the child nodes, so you don't need 0 and 1 numbering for those.
Looking at the driver patches, the only thing I see distinguishing core numbers is this:
"frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,
Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0."
Is this a requirement in the h/w or just what the driver picked? IOW, could frame#0 use core#1?
Rob
Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
Dear Rob,
A sample encoder hardware block diagram attached.
Quoted 66 lines · Rob Herring
Quoted 13 lines · Irui Wang
Dear Rob,
Many thanks for your attention.
Quoted 8 lines · Rob Herring
Quoted 7 lines · Irui Wang
Dear Rob,
Thanks for your review and comments.
Quoted 2 lines · Rob Herring
Quoted 1 line · Irui Wang
Adds encoder cores dt-bindings for mt8195.
[...]
Quoted 26 lines
Quoted 18 lines
Quoted 10 lines
Quoted 9 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 74
Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +4
-0+ mediatek,core-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id.What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.Quoted 7 lines
It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can
get core-id in drivers. If it can't ref "uint32" types yaml, would
you mind giving some more suggestions ?Quoted 7 lines
I still don't understand why it is needed. What is 'core-0'?
Is there some functional difference between the cores? If so, describe
that difference.Rob
Quoted 15 lines
They are two different pieces of hardware, it's our encoder hardware
design. There are two encoder hardware cores inside MT8195, named core0
andcore1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1". So the two encoder cores have their own base, IRQ, clock, power, etc.
Each core can encode independently, moreover, they can work together
for higher performance.
We will describe more details in YAML about it if it's OK for you.All the resources you list are in the child nodes, so you don't need 0
and 1 numbering for those.Looking at the driver patches, the only thing I see distinguishing core numbers is this:
Quoted 8 lines
"frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,
Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0."
Is this a requirement in the h/w or just what the driver picked? IOW,
could frame#0 use core#1?
No, it's a requirement in the h/w, driver trigger core start encoding is in order.
About the encoder hardware block diagram, please check below:
--------------------------------------------------------------
Input Buffer: 0 1 2 3 4 5 6
| | | | | | |
v | v | v | v
+-------+ | +-------+ | +-------+ | +-------+
| core0 | | | core0 | | | core0 | | | core0 |
+-------+ | +-------+ | +-------+ | +-------+
| | | | | | |
| v | v | v |
| +-------+ | +-------+ | +-------+ |
| | core1 | | | core1 | | | core1 | |
| +-------+ | +-------+ | +-------+ |
| | | | | | |
v v v v v v v <parent>
--------------------------------------------------------------
core || index <child>
\/
+-----------------------------------------------+
| core0/core1 |
| enable/disable power/clk/irq |
+-----------------------------------------------+--------------------------------------------------------------
As above , there are parent and child devices, child mean each venc core, the child device controls the information of each core independent which inlcude power/clk/irq.
When start encoding, input buffer 0 will be encoded by core0, and input buffer 1 can be encoded by core1 even if buffer 0 has not been encoded done yet, after buffer 0 encoded done, buffer 2 will be encoded by core0, and buffer 1 encoded done by core1. These two encoder cores will encode each input in this overlapping manner.
We need manage each child device in parent device by core-id property. And we also need record current encoding input buffer, encode done output buffers and which one core is in used through core-id, because the two cores are encoding at the same time under one parent driver.
Quoted 2 lines
Rob
Thanks
Best Regards
Rob Herring
maintainerRe: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195
Quoted 112 lines · Irui Wang
Dear Rob,
A sample encoder hardware block diagram attached.
Quoted 66 lines · Rob Herring
Quoted 13 lines · Irui Wang
Dear Rob,
Many thanks for your attention.
Quoted 8 lines · Rob Herring
Quoted 7 lines · Irui Wang
Dear Rob,
Thanks for your review and comments.
Quoted 2 lines · Rob Herring
Quoted 1 line · Irui Wang
Adds encoder cores dt-bindings for mt8195.
[...]
Quoted 26 lines
Quoted 18 lines
Quoted 10 lines
Quoted 9 lines
Review
Quoted 4 lines
(quoted diff) +4
-0+ mediatek,core-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id.What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.Quoted 7 lines
It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can
get core-id in drivers. If it can't ref "uint32" types yaml, would
you mind giving some more suggestions ?Quoted 7 lines
I still don't understand why it is needed. What is 'core-0'?
Is there some functional difference between the cores? If so, describe
that difference.Rob
Quoted 15 lines
They are two different pieces of hardware, it's our encoder hardware
design. There are two encoder hardware cores inside MT8195, named core0
andcore1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1". So the two encoder cores have their own base, IRQ, clock, power, etc.
Each core can encode independently, moreover, they can work together
for higher performance.
We will describe more details in YAML about it if it's OK for you.All the resources you list are in the child nodes, so you don't need 0
and 1 numbering for those.Looking at the driver patches, the only thing I see distinguishing core numbers is this:
Quoted 8 lines
"frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,
Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0."
Is this a requirement in the h/w or just what the driver picked? IOW,
could frame#0 use core#1?No, it's a requirement in the h/w, driver trigger core start encoding is in order.
About the encoder hardware block diagram, please check below:
--------------------------------------------------------------Input Buffer: 0 1 2 3 4 5 6 | | | | | | | v | v | v | v +-------+ | +-------+ | +-------+ | +-------+ | core0 | | | core0 | | | core0 | | | core0 | +-------+ | +-------+ | +-------+ | +-------+ | | | | | | | | v | v | v | | +-------+ | +-------+ | +-------+ | | | core1 | | | core1 | | | core1 | | | +-------+ | +-------+ | +-------+ | | | | | | | | v v v v v v v <parent> -------------------------------------------------------------- core || index <child> \/ +-----------------------------------------------+ | core0/core1 | | enable/disable power/clk/irq | +-----------------------------------------------+--------------------------------------------------------------
As above , there are parent and child devices, child mean each venc core, the child device controls the information of each core independent which inlcude power/clk/irq.
When start encoding, input buffer 0 will be encoded by core0, and input buffer 1 can be encoded by core1 even if buffer 0 has not been encoded done yet, after buffer 0 encoded done, buffer 2 will be encoded by core0, and buffer 1 encoded done by core1. These two encoder cores will encode each input in this overlapping manner.
Quoted 4 lines
We need manage each child device in parent device by core-id property. And we also need record current encoding input buffer, encode done output buffers and which one core is in used through core-id, because the two cores are encoding at the same time under one parent driver.
Okay, please summarize the h/w requirement in the property description.
You might put the above diagram in the top description too.
Rob
PATCH v3 4/10 media: mtk-vcodec: Enable venc dual core usage
In reply to Irui WangAdds new property to indicate whether the encoder has multiple cores.
Use of_platform_populate to probe each venc cores, the core device can use the init_clk/request_irq helper to initialize their own power/clk/irq, and the main device doesn't need use these helper anymore.
--- drivers/media/platform/mtk-vcodec/Makefile | 4 +- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 14 ++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 138 ++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_core.h | 36 +++++ .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 77 +++++----- 5 files changed, 235 insertions(+), 34 deletions(-) create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h
drivers/media/platform/mtk-vcodec/Makefile +3 -1
@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk-vcodec-enc-core.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \
@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +14 -0
@@ -117,6 +117,15 @@ enum mtk_vdec_hw_count { MTK_VDEC_MAX_HW_COUNT, }; +/* +* enum mtk_venc_core_id -- encoder core id +*/ +enum mtk_venc_core_id { +MTK_VENC_CORE0 =0 , +MTK_VENC_CORE1 , +MTK_VENC_CORE_MAX , +}; +/* * struct mtk_video_fmt - Structure used to store information about pixelformats */
@@ -476,6 +485,8 @@ struct mtk_vcodec_enc_pdata {* @subdev_dev: subdev hardware device * @subdev_prob_done: check whether all used hw device is prob done * @subdev_bitmap: used to record hardware is ready or not +* @venc_multi_core: indicates whether the encoder has multiple cores or not +* @enc_core_dev: used to store venc core device */ struct mtk_vcodec_dev {struct v4l2_device v4l2_dev;
@@ -521,6 +532,9 @@ struct mtk_vcodec_dev {void *subdev_dev[MTK_VDEC_HW_MAX ];int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev);DECLARE_BITMAP (subdev_bitmap,MTK_VDEC_HW_MAX ); + +bool venc_multi_core; +void *enc_core_dev[MTK_VENC_CORE_MAX ]; };static inline struct mtk_vcodec_ctx *fh_to_ctx (struct v4l2_fh *fh)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +138 -0
138 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#ifndef _MTK_VCODEC_ENC_CORE_H_ +#define _MTK_VCODEC_ENC_CORE_H_ + +#include <linux/platform_device.h> +#include "mtk_vcodec_drv.h" + +/** +* struct mtk_venc_core_dev - driver data +* @plat_dev: platform_device +* @main_dev: main device +* @pm: power management data +* @curr_ctx: the context that is waiting for venc hardware +* @reg_base: mapped address of venc registers +* @irq_status: venc core irq status +* @enc_irq: venc device irq +* @core id: for venc core id: core#0, core#1... +*/ +struct mtk_venc_core_dev { +struct platform_device *plat_dev; +struct mtk_vcodec_dev *main_dev; + +struct mtk_vcodec_pm pm; +struct mtk_vcodec_ctx *curr_ctx; + +void __iomem *reg_base; +unsigned int irq_status; +int enc_irq; +int core_id; +}; + +#endif /* _MTK_VCODEC_ENC_CORE_H_ */
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +44 -33
114 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v3 5/10 media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface
In reply to Irui WangThe args "struct mtk_vcodec_dev *" doesn't appropriate for init_clk functions because of sub-devices. For main device, it has no "pm/clk" properties in dtsi, sub-devices will init their own "pm/clk" instead. So rewrite the pm interface with args "platform_device *" and "mtk_vcodec_pm*".
--- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c | 8 ++++++++ drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 2 +- drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 8 ++------ drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 4 ++-- 4 files changed, 13 insertions(+), 9 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c +8 -0
@@ -13,6 +13,7 @@#include "mtk_vcodec_drv.h" #include "mtk_vcodec_enc.h" #include "mtk_vcodec_enc_core.h" +#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_intr.h" static const struct of_device_id mtk_venc_core_ids[] = {
@@ -117,6 +118,13 @@ static int mtk_venc_core_probe(struct platform_device *pdev)return dev_err_probe (dev, -EINVAL ,"Invalid core id %d\n" , core->core_id); + ret =mtk_vcodec_init_enc_clk (core->plat_dev, &core->pm); +if (ret <0 ) +return dev_err_probe (dev, ret, +"Failed to get venc core clock source!" ); + +pm_runtime_enable (&pdev->dev); + main_dev->enc_core_dev[core->core_id] = core; core->main_dev = main_dev;
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +1 -1
@@ -272,7 +272,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev) }if (!dev->venc_multi_core) { - ret =mtk_vcodec_init_enc_clk (dev); + ret =mtk_vcodec_init_enc_clk (dev->plat_dev, &dev->pm);if (ret <0 ) {dev_err (&pdev->dev,"Failed to get mtk vcodec clock source!" );
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +2 -6
@@ -12,17 +12,13 @@#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_util.h" -int mtk_vcodec_init_enc_clk (struct mtk_vcodec_dev *mtkdev) +int mtk_vcodec_init_enc_clk (struct platform_device *pdev, +struct mtk_vcodec_pm *pm) { -struct platform_device *pdev; -struct mtk_vcodec_pm *pm;struct mtk_vcodec_clk *enc_clk;struct mtk_vcodec_clk_info *clk_info;int ret, i; - pdev = mtkdev->plat_dev; - pm = &mtkdev->pm; -memset (pm,0 ,sizeof (struct mtk_vcodec_pm )); pm->dev = &pdev->dev; enc_clk = &pm->venc_clk;
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +2 -2
@@ -9,8 +9,8 @@#include "mtk_vcodec_drv.h" -int mtk_vcodec_init_enc_clk (struct mtk_vcodec_dev *dev); - +int mtk_vcodec_init_enc_clk (struct platform_device *pdev, +struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm);
Signature · 1 line
2.18.0
PATCH v3 6/10 media: mtk-vcodec: Add venc power on/off interface
In reply to Irui Wangvenc_multi_core mode power on/off all venc available power, pm_runtime_xx helper is not appropriate called directly, add new power on/off interface for it.
--- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 20 +---- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 7 ++ .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 83 +++++++++++++++++++ .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 5 ++ 4 files changed, 98 insertions(+), 17 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +3 -17
@@ -12,6 +12,7 @@#include "mtk_vcodec_drv.h" #include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_intr.h" #include "mtk_vcodec_util.h" #include "venc_drv_if.h"
@@ -849,7 +850,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count) {struct mtk_vcodec_ctx *ctx =vb2_get_drv_priv (q);struct venc_enc_param param; -int ret, pm_ret; +int ret;int i;/* Once state turn into MTK_STATE_ABORT, we need stop_streaming
@@ -857,7 +858,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)*/ if ((ctx->state ==MTK_STATE_ABORT ) || (ctx->state ==MTK_STATE_FREE )) { ret = -EIO ; - goto err_start_stream; + goto err_set_param; }/* Do the initialization when both start_streaming have been called */
@@ -869,12 +870,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)return 0 ; } - ret =pm_runtime_resume_and_get (&ctx->dev->plat_dev->dev); -if (ret <0 ) { -mtk_v4l2_err ("pm_runtime_resume_and_get fail %d" , ret); - goto err_start_stream; - } -mtk_venc_set_param (ctx, ¶m); ret =venc_if_set_param (ctx,VENC_SET_PARAM_ENC , ¶m);if (ret) {
@@ -901,11 +896,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)return 0 ; err_set_param: - pm_ret =pm_runtime_put (&ctx->dev->plat_dev->dev); -if (pm_ret <0 ) -mtk_v4l2_err ("pm_runtime_put fail %d" , pm_ret); - -err_start_stream:for (i =0 ; i < q->num_buffers; ++i) {struct vb2_buffer *buf =vb2_get_buffer (q, i);
@@ -988,10 +978,6 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)if (ret)mtk_v4l2_err ("venc_if_deinit failed=%d" , ret); - ret =pm_runtime_put (&ctx->dev->plat_dev->dev); -if (ret <0 ) -mtk_v4l2_err ("pm_runtime_put fail %d" , ret); - ctx->state =MTK_STATE_FREE ; }
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +7 -0
@@ -173,6 +173,12 @@ static int fops_vcodec_open(struct file *file)mtk_v4l2_debug (0 ,"encoder capability %x" , dev->enc_capability); } + ret =mtk_vcodec_enc_pw_on (dev); +if (ret <0 ) { +mtk_v4l2_err ("encoder power on failed %d" , ret); + goto err_load_fw; + } +mtk_v4l2_debug (2 ,"Create instance [%d]@%p m2m_ctx=%p " , ctx->id, ctx, ctx->m2m_ctx);
@@ -207,6 +213,7 @@ static int fops_vcodec_release(struct file *file)v4l2_m2m_ctx_release (ctx->m2m_ctx);mtk_vcodec_enc_release (ctx); +mtk_vcodec_enc_pw_off (dev);v4l2_fh_del (&ctx->fh);v4l2_fh_exit (&ctx->fh);v4l2_ctrl_handler_free (&ctx->ctrl_hdl);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +83 -0
92 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +5 -0
@@ -14,4 +14,9 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm); +int mtk_venc_core_pw_on (struct mtk_vcodec_dev *dev); +int mtk_venc_core_pw_off (struct mtk_vcodec_dev *dev); +int mtk_vcodec_enc_pw_on (struct mtk_vcodec_dev *dev); +int mtk_vcodec_enc_pw_off (struct mtk_vcodec_dev *dev); +#endif /* _MTK_VCODEC_ENC_PM_H_ */
Signature · 1 line
2.18.0
PATCH v3 7/10 media: mtk-vcodec: Rewrite venc clock interface
In reply to Irui Wangvenc_multi_core mode need enable the specific core's clk, add an another param for clock_on/clock_off interface.
--- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 95 ++++++++++++++++--- .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h | 4 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 10 +- 3 files changed, 84 insertions(+), 25 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +80 -15
@@ -57,42 +57,86 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev, }EXPORT_SYMBOL_GPL (mtk_vcodec_init_enc_clk); -void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_on (struct mtk_vcodec_dev *dev,int core_id) { -struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; +struct mtk_venc_core_dev *core; +struct mtk_vcodec_pm *enc_pm; +struct mtk_vcodec_clk *enc_clk; +struct clk *clk;int ret, i =0 ; -for (i =0 ; i < enc_clk->clk_num; i++) { - ret =clk_prepare_enable (enc_clk->clk_info[i].vcodec_clk); -if (ret) { -mtk_v4l2_err ("venc clk_prepare_enable %d %s fail %d" , i, - enc_clk->clk_info[i].clk_name, ret); - goto clkerr; +if (dev->venc_multi_core) { + core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id]; + enc_pm = &core->pm; + enc_clk = &enc_pm->venc_clk; + +for (i =0 ; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret =clk_enable (clk); +if (ret) { +mtk_v4l2_err ("clk_enable %d %s fail %d" , i, + enc_clk->clk_info[i].clk_name, + ret); + goto core_clk_err; + } + } + }else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; + +for (i =0 ; i < enc_clk->clk_num; i++) { + clk = enc_clk->clk_info[i].vcodec_clk; + ret =clk_prepare_enable (clk); +if (ret) { +mtk_v4l2_err ("clk_prepare %d %s fail %d" , + i, enc_clk->clk_info[i].clk_name, + ret); + goto clkerr; + } } }return ; +core_clk_err: +for (i -=1 ; i >=0 ; i--) +clk_disable (enc_clk->clk_info[i].vcodec_clk); clkerr:for (i -=1 ; i >=0 ; i--)clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); }EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_on); -void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_off (struct mtk_vcodec_dev *dev,int core_id) { -struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; +struct mtk_venc_core_dev *core; +struct mtk_vcodec_pm *enc_pm; +struct mtk_vcodec_clk *enc_clk;int i =0 ; -for (i = enc_clk->clk_num -1 ; i >=0 ; i--) -clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); +if (dev->venc_multi_core) { + core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id]; + enc_pm = &core->pm; + enc_clk = &enc_pm->venc_clk; + +for (i = enc_clk->clk_num -1 ; i >=0 ; i--) +clk_disable (enc_clk->clk_info[i].vcodec_clk); + }else { + enc_pm = &dev->pm; + enc_clk = &enc_pm->venc_clk; + +for (i = enc_clk->clk_num -1 ; i >=0 ; i--) +clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); + } + }EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_off);int mtk_venc_core_pw_on (struct mtk_vcodec_dev *dev) { -int i, ret; +int i, ret, j;struct mtk_venc_core_dev *core; +struct mtk_vcodec_clk *clk;/* power on all available venc cores */ for (i =0 ; i <MTK_VENC_CORE_MAX ; i++) {
@@ -102,15 +146,31 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev) ret =pm_runtime_resume_and_get (&core->plat_dev->dev);if (ret <0 ) { -mtk_v4l2_err ("power on core[%d] fail %d" , i, ret); +mtk_v4l2_err ("power on core[%d] fail" , i); goto pw_on_fail; } + + clk = &core->pm.venc_clk; +for (j =0 ; j < clk->clk_num; j++) { + ret =clk_prepare (clk->clk_info[j].vcodec_clk); +if (ret) { +mtk_v4l2_err ("prepare clk [%s] fail %d" , + clk->clk_info[j].clk_name, + ret); + goto pw_on_fail; + } + } }return 0 ; pw_on_fail:for (i -=1 ; i >=0 ; i--) { core = (struct mtk_venc_core_dev *)dev->enc_core_dev[i]; + + clk = &core->pm.venc_clk; +for (j -=1 ; j >=0 ; j--) +clk_unprepare (clk->clk_info[j].vcodec_clk); +pm_runtime_put_sync (&core->plat_dev->dev); }return ret;
@@ -118,8 +178,9 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev)int mtk_venc_core_pw_off (struct mtk_vcodec_dev *dev) { -int i, ret; +int i, ret, j;struct mtk_venc_core_dev *core; +struct mtk_vcodec_clk *clk;/* power off all available venc cores */ for (i =0 ; i <MTK_VENC_CORE_MAX ; i++) {
@@ -127,6 +188,10 @@ int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev)if (!core)return 0 ; + clk = &core->pm.venc_clk; +for (j = clk->clk_num -1 ; j >=0 ; j--) +clk_unprepare (clk->clk_info[j].vcodec_clk); + ret =pm_runtime_put_sync (&core->plat_dev->dev);if (ret <0 )mtk_v4l2_err ("power off core[%d] fail %d" , i, ret);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +2 -2
@@ -11,8 +11,8 @@int mtk_vcodec_init_enc_clk (struct platform_device *pdev,struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm); +void mtk_vcodec_enc_clock_on (struct mtk_vcodec_dev *dev,int core_id); +void mtk_vcodec_enc_clock_off (struct mtk_vcodec_dev *dev,int core_id);int mtk_venc_core_pw_on (struct mtk_vcodec_dev *dev);int mtk_venc_core_pw_off (struct mtk_vcodec_dev *dev);
drivers/media/platform/mtk-vcodec/venc_drv_if.c +2 -8
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) }mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->init (ctx); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx);return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,int ret =0 ;mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->set_param (ctx->drv_handle, type, in); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx);return ret;
@@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, ctx->dev->curr_ctx = ctx;spin_unlock_irqrestore (&ctx->dev->irqlock, flags); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); +mtk_vcodec_enc_clock_on (ctx->dev,0 ); ret = ctx->enc_if->encode (ctx->drv_handle, opt, frm_buf, bs_buf, result); -mtk_vcodec_enc_clock_off (&ctx->dev->pm); +mtk_vcodec_enc_clock_off (ctx->dev,0 );spin_lock_irqsave (&ctx->dev->irqlock, flags); ctx->dev->curr_ctx =NULL ;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)return 0 ;mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->deinit (ctx->drv_handle); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx); ctx->drv_handle =NULL ;
Signature · 1 line
2.18.0
PATCH v3 8/10 media: mtk-vcodec: Add more extra processing for venc_multi_core mode
In reply to Irui Wangvenc_multi_core mode encoding need more venc working buffers, it will break the compatibility if we just add venc_vsi in AP-Kernel but not in firmware, so add more struct definition for it.
--- .../platform/mtk-vcodec/mtk_vcodec_util.c | 19 ++ .../platform/mtk-vcodec/mtk_vcodec_util.h | 4 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 174 +++++++++++++++--- 3 files changed, 172 insertions(+), 25 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c +19 -0
@@ -11,6 +11,7 @@#include "mtk_vcodec_dec_hw.h" #include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc_core.h" #include "mtk_vcodec_util.h" void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, }EXPORT_SYMBOL (mtk_vcodec_get_reg_addr); +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *ctx, +unsigned int core_id) +{ +struct mtk_venc_core_dev *core; + +if (core_id >=MTK_VENC_CORE_MAX ) { +mtk_v4l2_err ("Invalid core_id = %d" , core_id); +return NULL ; + } + + core = (struct mtk_venc_core_dev *)ctx->dev->enc_core_dev[core_id]; +if (!core) +return NULL ; + +return core->reg_base; +} +EXPORT_SYMBOL (mtk_venc_get_core_reg_addr); +int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem) {
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +4 -0
@@ -50,6 +50,10 @@ struct mtk_vcodec_dev;void __iomem *mtk_vcodec_get_reg_addr (struct mtk_vcodec_ctx *data,unsigned int reg_idx); + +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *data, +unsigned int core_id); +int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem); void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +149 -25
280 lines not on this page · raw message
Signature · 1 line
2.18.0
PATCH v3 9/10 media: mtk-vcodec: Add venc_multi_core mode encode process
In reply to Irui WangThe venc_multi_core mode encoding is tries to uses all available cores. MT8195 has two venc cores:
frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,
Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0.
--- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 12 ++- .../platform/mtk-vcodec/mtk_vcodec_enc.c | 23 +++--- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 4 +- .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c | 7 +- .../platform/mtk-vcodec/venc/venc_h264_if.c | 16 ++-- .../platform/mtk-vcodec/venc/venc_vp8_if.c | 3 +- .../media/platform/mtk-vcodec/venc_drv_if.c | 73 ++++++++++++++----- .../media/platform/mtk-vcodec/venc_drv_if.h | 5 ++ .../media/platform/mtk-vcodec/venc_vpu_if.c | 11 ++- .../media/platform/mtk-vcodec/venc_vpu_if.h | 3 +- 10 files changed, 115 insertions(+), 42 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +11 -1
@@ -297,6 +297,9 @@ struct vdec_pic_info {* @max_width: hardware supported max width * @max_height: hardware supported max height * @msg_queue: msg queue used to store lat buffer information. +* @q_mutex: src & dst vb2_queue mutex +* @enc_idx: used to record encoded frame count +* @core_id: used to record used core */ struct mtk_vcodec_ctx {enum mtk_instance_type type;
@@ -345,6 +348,11 @@ struct mtk_vcodec_ctx {unsigned int max_width;unsigned int max_height;struct vdec_msg_queue msg_queue; + +/* src & dst vb2_queue mutex */ +struct mutex q_mutex; +int enc_idx; +int core_id; };/*
@@ -520,7 +528,9 @@ struct mtk_vcodec_dev {/* decoder hardware mutex lock */ struct mutex dec_mutex[MTK_VDEC_HW_MAX ]; -struct mutex enc_mutex; + +/* encoder core mutex lock */ +struct mutex enc_mutex[MTK_VENC_CORE_MAX ];struct mtk_vcodec_pm pm;unsigned int dec_capability;
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +13 -10
@@ -1197,14 +1197,17 @@ static void mtk_venc_worker(struct work_struct *work)vb2_set_plane_payload (&dst_buf->vb2_buf,0 ,0 );v4l2_m2m_buf_done (dst_buf,VB2_BUF_STATE_ERROR );mtk_v4l2_err ("venc_if_encode failed=%d" , ret); - }else { + }else if (!ctx->dev->venc_multi_core) {v4l2_m2m_buf_done (src_buf,VB2_BUF_STATE_DONE ); -vb2_set_plane_payload (&dst_buf->vb2_buf,0 , enc_result.bs_size); +vb2_set_plane_payload (&dst_buf->vb2_buf,0 , + enc_result.bs_size);v4l2_m2m_buf_done (dst_buf,VB2_BUF_STATE_DONE );mtk_v4l2_debug (2 ,"venc_if_encode bs size=%d" , - enc_result.bs_size); + enc_result.bs_size); } + ctx->enc_idx++; +v4l2_m2m_job_finish (ctx->dev->m2m_dev_enc, ctx->m2m_ctx); mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
@@ -1258,7 +1261,7 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx) {struct mtk_q_data *q_data; - ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex; + ctx->m2m_ctx->q_lock = &ctx->q_mutex; ctx->fh.m2m_ctx = ctx->m2m_ctx; ctx->fh.ctrl_handler = &ctx->ctrl_hdl;INIT_WORK (&ctx->encode_work, mtk_venc_worker);
@@ -1390,7 +1393,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, src_vq->ops = &mtk_venc_vb2_ops; src_vq->mem_ops = &vb2_dma_contig_memops; src_vq->timestamp_flags =V4L2_BUF_FLAG_TIMESTAMP_COPY ; - src_vq->lock = &ctx->dev->dev_mutex; + src_vq->lock = &ctx->q_mutex; src_vq->dev = &ctx->dev->plat_dev->dev; ret =vb2_queue_init (src_vq);
@@ -1404,26 +1407,26 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq, dst_vq->ops = &mtk_venc_vb2_ops; dst_vq->mem_ops = &vb2_dma_contig_memops; dst_vq->timestamp_flags =V4L2_BUF_FLAG_TIMESTAMP_COPY ; - dst_vq->lock = &ctx->dev->dev_mutex; + dst_vq->lock = &ctx->q_mutex; dst_vq->dev = &ctx->dev->plat_dev->dev;return vb2_queue_init (dst_vq); } -int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx) +int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx,int core_id) {struct mtk_vcodec_dev *dev = ctx->dev; -mutex_unlock (&dev->enc_mutex); +mutex_unlock (&dev->enc_mutex[core_id]);return 0 ; }EXPORT_SYMBOL_GPL (mtk_venc_unlock); -int mtk_venc_lock (struct mtk_vcodec_ctx *ctx) +int mtk_venc_lock (struct mtk_vcodec_ctx *ctx,int core_id) {struct mtk_vcodec_dev *dev = ctx->dev; -mutex_lock (&dev->enc_mutex); +mutex_lock (&dev->enc_mutex[core_id]);return 0 ; }EXPORT_SYMBOL_GPL (mtk_venc_lock);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +2 -2
@@ -39,8 +39,8 @@ struct mtk_video_enc_buf {extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;extern const struct v4l2_m2m_ops mtk_venc_m2m_ops; -int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx); -int mtk_venc_lock (struct mtk_vcodec_ctx *ctx); +int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx,int core_id); +int mtk_venc_lock (struct mtk_vcodec_ctx *ctx,int core_id);int mtk_vcodec_enc_queue_init (void *priv,struct vb2_queue *src_vq,struct vb2_queue *dst_vq);void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +5 -2
@@ -132,6 +132,7 @@ static int fops_vcodec_open(struct file *file)INIT_LIST_HEAD (&ctx->list); ctx->dev = dev;init_waitqueue_head (&ctx->queue[0 ]); +mutex_init (&ctx->q_mutex); ctx->type =MTK_INST_ENCODER ; ret =mtk_vcodec_enc_ctrls_setup (ctx);
@@ -239,7 +240,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)struct video_device *vfd_enc;phandle rproc_phandle;enum mtk_vcodec_fw_type fw_type; -int ret, core_type; +int ret, core_type, i; dev =devm_kzalloc (&pdev->dev,sizeof (*dev),GFP_KERNEL );if (!dev)
@@ -309,7 +310,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev) } } -mutex_init (&dev->enc_mutex); +for (i =0 ; i <MTK_VENC_CORE_MAX ; i++) +mutex_init (&dev->enc_mutex[i]); +mutex_init (&dev->dev_mutex);spin_lock_init (&dev->irqlock);
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +10 -6
@@ -446,7 +446,8 @@ static int h264_encode_sps(struct venc_h264_inst *inst,mtk_vcodec_debug_enter (inst); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_SPS ,NULL , bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_SPS , +NULL , bs_buf,NULL ,MTK_VENC_CORE0 );if (ret)return ret;
@@ -472,7 +473,8 @@ static int h264_encode_pps(struct venc_h264_inst *inst,mtk_vcodec_debug_enter (inst); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_PPS ,NULL , bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_PPS , +NULL , bs_buf,NULL ,MTK_VENC_CORE0 );if (ret)return ret;
@@ -514,7 +516,8 @@ static int h264_encode_header(struct venc_h264_inst *inst,static int h264_encode_frame (struct venc_h264_inst *inst,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -unsigned int *bs_size) +unsigned int *bs_size, +int core_id) {int ret =0 ;unsigned int irq_status;
@@ -533,7 +536,8 @@ static int h264_encode_frame(struct venc_h264_inst *inst,mtk_vcodec_debug (inst,"frm_count = %d,skip_frm_count =%d,frm_type=%d.\n" , frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_FRAME , frm_buf, bs_buf, &frame_info); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_FRAME , + frm_buf, bs_buf, &frame_info, core_id);if (ret)return ret;
@@ -660,7 +664,7 @@ static int h264_enc_encode(void *handle,if (!inst->prepend_hdr) { ret =h264_encode_frame (inst, frm_buf, bs_buf, - &result->bs_size); + &result->bs_size, ctx->core_id);if (ret) goto encode_err; result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -688,7 +692,7 @@ static int h264_enc_encode(void *handle, tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz); ret =h264_encode_frame (inst, frm_buf, &tmp_bs_buf, - &bs_size_frm); + &bs_size_frm, ctx->core_id);if (ret) goto encode_err;
drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c +2 -1
@@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,mtk_vcodec_debug (inst,"->frm_cnt=%d" , inst->frm_cnt); - ret =vpu_enc_encode (&inst->vpu_inst,0 , frm_buf, bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,0 , frm_buf, bs_buf, +NULL ,MTK_VENC_CORE0 );if (ret)return ret;
drivers/media/platform/mtk-vcodec/venc_drv_if.c +55 -18
114 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/venc_drv_if.h +5 -0
@@ -167,4 +167,9 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,struct mtk_vcodec_mem *bs_buf, struct venc_done_result *result); +void venc_encode_prepare (struct mtk_vcodec_ctx *ctx, +enum venc_start_opt opt); +void venc_encode_unprepare (struct mtk_vcodec_ctx *ctx, +enum venc_start_opt opt); +#endif /* _VENC_DRV_IF_H_ */
drivers/media/platform/mtk-vcodec/venc_vpu_if.c +10 -1
@@ -225,9 +225,12 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,int vpu_enc_encode (struct venc_vpu_inst *vpu,unsigned int bs_mode,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -struct venc_frame_info *frame_info) +struct venc_frame_info *frame_info, +int core_id) {const bool is_ext =MTK_ENC_CTX_IS_EXT (vpu->ctx); +const bool is_multi_core = vpu->ctx->dev->venc_multi_core; +size_t msg_size = is_ext ?sizeof (struct venc_ap_ipi_msg_enc_ext ) :sizeof (struct venc_ap_ipi_msg_enc );
@@ -261,6 +264,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, out.data[1 ] = frame_info->skip_frm_count; out.data[2 ] = frame_info->frm_type; } + +if (is_multi_core) { + out.data_item =4 ; + out.data[3 ] = core_id; + } +if (vpu_enc_send_msg (vpu, &out, msg_size)) {mtk_vcodec_err (vpu,"AP_IPIMSG_ENC_ENCODE %d fail" , bs_mode);
drivers/media/platform/mtk-vcodec/venc_vpu_if.h +2 -1
@@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,int vpu_enc_encode (struct venc_vpu_inst *vpu,unsigned int bs_mode,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -struct venc_frame_info *frame_info); +struct venc_frame_info *frame_info, +int core_id);int vpu_enc_deinit (struct venc_vpu_inst *vpu);#endif
Signature · 1 line
2.18.0
PATCH v3 10/10 media: mtk-vcodec: Done encode result to client
In reply to Irui WangWhen enable venc_multi_core mode encoding, the wait IRQ done synchronous function should not be called, so the encode result can't return to client in device_run. device_run called to trigger encoding process, finish it, so another scheduled job can be ran. When encoding done, done the encode result to client in IRQ handler.
--- .../platform/mtk-vcodec/mtk_vcodec_drv.h | 6 ++ .../platform/mtk-vcodec/mtk_vcodec_enc.c | 68 +++++++++++++++++-- .../platform/mtk-vcodec/mtk_vcodec_enc.h | 3 + .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 25 ++++++- .../platform/mtk-vcodec/mtk_vcodec_util.h | 1 + .../platform/mtk-vcodec/venc/venc_h264_if.c | 22 +++++- .../media/platform/mtk-vcodec/venc_drv_if.h | 2 + 7 files changed, 120 insertions(+), 7 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +6 -0
@@ -300,6 +300,9 @@ struct vdec_pic_info {* @q_mutex: src & dst vb2_queue mutex * @enc_idx: used to record encoded frame count * @core_id: used to record used core +* @pfrm_buf: used to store current ctx's frame buffer +* @pbs_buf: used to store current ctx's bitstream buffer +* @hdr_size: used to store prepend header size */ struct mtk_vcodec_ctx {enum mtk_instance_type type;
@@ -353,6 +356,9 @@ struct mtk_vcodec_ctx {struct mutex q_mutex;int enc_idx;int core_id; +struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_CORE_MAX ]; +struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_CORE_MAX ]; +unsigned int hdr_size; };/*
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +64 -4
130 lines not on this page · raw message
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +3 -0
@@ -46,5 +46,8 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx);int mtk_vcodec_enc_ctrls_setup (struct mtk_vcodec_ctx *ctx);void mtk_vcodec_enc_set_default_params (struct mtk_vcodec_ctx *ctx); +void mtk_venc_buf_done (struct mtk_vcodec_ctx *ctx,int core_id, +unsigned int bs_size,bool time_out,bool key_frame); +void mtk_venc_lock_all (struct mtk_vcodec_ctx *ctx);#endif /* _MTK_VCODEC_ENC_H_ */
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c +24 -1
@@ -16,6 +16,10 @@#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_intr.h" +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 +#define VENC_PIC_FRM_TYPE 0x0010 +#define VENC_PIC_KEY_FRM 0x2 +static const struct of_device_id mtk_venc_core_ids[] = { { .compatible ="mediatek,mtk-venc-core" ,
@@ -51,6 +55,9 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv)struct mtk_vcodec_ctx *ctx;unsigned long flags;void __iomem *addr; +unsigned int bs_size; +unsigned int frm_type; +bool is_key_frame =0 ;spin_lock_irqsave (&core->main_dev->irqlock, flags); ctx = core->curr_ctx;
@@ -62,9 +69,25 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv) addr = core->reg_base +MTK_VENC_IRQ_ACK_OFFSET ; ctx->irq_status =readl (core->reg_base +MTK_VENC_IRQ_STATUS_OFFSET ); + bs_size =readl (core->reg_base +VENC_PIC_BITSTREAM_BYTE_CNT ); + frm_type =readl (core->reg_base +VENC_PIC_FRM_TYPE );clean_irq_status (ctx->irq_status, addr); -wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); +if (ctx->irq_status &MTK_VENC_IRQ_STATUS_FRM ) { +if (ctx->hdr_size !=0 ) { + bs_size += ctx->hdr_size; + ctx->hdr_size =0 ; + } +if (frm_type &VENC_PIC_KEY_FRM ) + is_key_frame =1 ; + +mtk_venc_buf_done (ctx, core->core_id, bs_size,0 , is_key_frame); +mtk_vcodec_enc_clock_off (core->main_dev, core->core_id); +mtk_venc_unlock (ctx, core->core_id); + }else { +wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); + } +return IRQ_HANDLED ; }
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +1 -0
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {size_t size;void *va;dma_addr_t dma_addr; +void *buf; };struct mtk_vcodec_fb {
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +20 -2
@@ -555,6 +555,12 @@ static int h264_encode_frame(struct venc_h264_inst *inst,return ret; } +if (ctx->dev->venc_multi_core) { + ++inst->frm_cnt; +mtk_vcodec_debug (inst,"multi core : frm %d <-" , inst->frm_cnt); +return ret; + } + irq_status =h264_enc_wait_venc_done (inst);if (irq_status !=MTK_VENC_IRQ_STATUS_FRM ) {mtk_vcodec_err (inst,"irq_status=%d failed" , irq_status);
@@ -635,10 +641,12 @@ static int h264_enc_encode(void *handle,int ret =0 ;struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;struct mtk_vcodec_ctx *ctx = inst->ctx; +bool is_single_core = !ctx->dev->venc_multi_core;mtk_vcodec_debug (inst,"opt %d ->" , opt); -enable_irq (ctx->dev->enc_irq); +if (is_single_core) +enable_irq (ctx->dev->enc_irq);switch (opt) {case VENC_START_OPT_ENCODE_SEQUENCE_HEADER : {
@@ -662,6 +670,13 @@ static int h264_enc_encode(void *handle,unsigned int bs_size_hdr;unsigned int bs_size_frm; +/* +* the frm_buf and bs_buf need to be recorded into ctx, +* when encoding done, the target buffer can be obtained +*/ + ctx->pfrm_buf[ctx->core_id] = frm_buf->src_addr; + ctx->pbs_buf[ctx->core_id] = bs_buf->buf; +if (!inst->prepend_hdr) { ret =h264_encode_frame (inst, frm_buf, bs_buf, &result->bs_size, ctx->core_id);
@@ -698,6 +713,8 @@ static int h264_enc_encode(void *handle, result->bs_size = hdr_sz + filler_sz + bs_size_frm; + ctx->hdr_size = hdr_sz + filler_sz; +mtk_vcodec_debug (inst,"hdr %d filler %d frame %d bs %d" , hdr_sz, filler_sz, bs_size_frm, result->bs_size);
@@ -714,8 +731,9 @@ static int h264_enc_encode(void *handle, } encode_err: +if (is_single_core) +disable_irq (ctx->dev->enc_irq); -disable_irq (ctx->dev->enc_irq);mtk_vcodec_debug (inst,"opt %d <-" , opt);return ret;
drivers/media/platform/mtk-vcodec/venc_drv_if.h +2 -0
@@ -108,9 +108,11 @@ struct venc_frame_info {/* * struct venc_frm_buf - frame buffer information used in venc_if_encode() * @fb_addr: plane frame buffer addresses +* @src_addr: vb2_v4l2_buffer src buffer address */ struct venc_frm_buf {struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES ]; +void *src_addr; };/*
Signature · 1 line
2.18.0
AngeloGioacchino Del Regno
maintainerIl 17/03/22 09:22, Irui Wang ha scritto:
Quoted 25 lines
MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps.
The two encoder core encoding process looks like this:VENC Core0: frm#0....frm#2....frm#4.... VENC Core1: ..frm#1....frm#3....frm#5....This series of patches are used to enable the two H264 encoder cores, encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 -->device_run(triger encoder)--> wait encoder IRQ --> encoding done with result -->job_finish
V4l2_VIDIOC_QBUF#1 -->device_run(triger encoder)--> wait encoder IRQ --> encoding done with result -->job_finish
...To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 -->device_run(triger encoder)-->job_finish..V4l2_VIDIOC_QBUF#1 -->device_run(triger encoder)-->job_finish(venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 -->device_run(triger encoder)-->job_finish.
Quoted 4 lines
There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance.
Hello Irui,
like v1 and v2, also this version of the series will crash the kernel when trying to encode.
Before pushing the next series, you shall ensure that it is not crashing the entire kernel, hence, please test your code with gstreamer.
Please use the following pipeline to test:
gst-launch-1.0 videotestsrc num-buffers=100 ! v4l2h264enc ! queue ! h264parse ! qtmux ! filesink location=test.mov
Regards,
Angelo
Dear Angelo,
Thanks for your comments.
Quoted 40 lines · AngeloGioacchino Del Regno
Il 17/03/22 09:22, Irui Wang ha scritto:
Quoted 32 lines
MT8195 has two H264 encoder cores, they have their own power- domains,
clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve
4K@60fps.
The two encoder core encoding process looks like this:VENC Core0: frm#0....frm#2....frm#4.... VENC Core1: ..frm#1....frm#3....frm#5....This series of patches are used to enable the two H264 encoder cores,
encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 -->device_run(triger encoder)--> wait encoder IRQ -->
encoding done with result -->job_finishV4l2_VIDIOC_QBUF#1 -->device_run(triger encoder)--> wait encoder IRQ -->
encoding done with result -->job_finish
...To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 -->device_run(triger encoder)-->job_finish..V4l2_VIDIOC_QBUF#1 -->device_run(triger encoder)-->job_finish(venc core0 may encode done here, done the encoding result to client)
V4L2_VIDIOC_QBUF#2 -->device_run(triger encoder)-->job_finish.Quoted 6 lines
There is no "wait encoder IRQ" synchronous call during frame-racing mode
encoding process, it can full use the two encoder cores to achieve higher
performance.
Quoted 4 lines
Hello Irui,
like v1 and v2, also this version of the series will crash the kernel when
trying to encode.
As mentioned before, dual core encoding need more encoding working buffers in patch-8:
media: mtk-vcodec: Add more extra processing for venc_multi_core mode a new structure, struct venc_multi_core_vsi defined for it,
encoding processing like this:
kernel --> ipi message --> scp firmware(userspace), vsi is a share memory to transfer data between kernel and user, so, the definition of vsi must be the same in kernel and userspace. otherwise, errors will be occurred, for example, kernel crash.
When we enable dual core encoding in this patch, the venc_multi_core_vsi used in kernel, but scp(userspace) still use the older one vsi(struct venc_h264_vsi), the crash happened.
Maybe we would find out how to resolve the limitations, many thanks for your review.
Quoted 8 lines
Before pushing the next series, you shall ensure that it is not crashing
the entire kernel, hence, please test your code with gstreamer.Please use the following pipeline to test:
Quoted 7 lines
gst-launch-1.0 videotestsrc num-buffers=100 ! v4l2h264enc ! queue ! h264parse !
qtmux ! filesink location=test.movRegards,
Angelo
Thanks
Best Regards