Skip to contentSkip to conversation

Enable two H264 encoder core on MT8195

PATCH In review Irui Wang 4 participants 21 messages Mar 17, 2022 – Mar 31, 2022

Subsystems (4)
lore
Share & tools
lore

Irui Wang

author
#0 lore raw

v3 cover Enable two H264 encoder core on MT8195

MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps.
The two encoder core encoding process looks like this:

    VENC Core0: frm#0....frm#2....frm#4....
    VENC Core1: ..frm#1....frm#3....frm#5....

This series of patches are used to enable the two H264 encoder cores, encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
...

To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.

There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance.

---
This series patches dependent on:
media_stage tree:
[1]
https://git.linuxtv.org/media_stage.git/commit/?id=b3627647f9ea7473d10fb08a95fd7c4133a17ca4

mt8192 decoder:
[2] https://patchwork.kernel.org/project/linux-mediatek/list/?series=616991

patch3 new venc dt-bindings included files
[3] MM IOMMU binding:
https://patchwork.kernel.org/project/linux-mediatek/patch/[email protected]/

[4] MT8195 power domain:
https://patchwork.kernel.org/project/linux-mediatek/list/?series=580579

changes compared with v2:
- update venc core dt-bindings, add two new properties for current usage.
- parse venc multi_core mode from device tree.
- rebase to the newer linux media stage.

changes compared with v1:
- of_platform_populate was used in place of the component framework.
- new yaml file for venc cores.
- some modifications for patch v1's review comments.
---

Irui Wang (10):
media: mtk-vcodec: Use core type to indicate h264 and vp8 enc
media: mtk-vcodec: export encoder functions
dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for

    mt8195

media: mtk-vcodec: Enable venc dual core usage
media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface
media: mtk-vcodec: Add venc power on/off interface
media: mtk-vcodec: Rewrite venc clock interface
media: mtk-vcodec: Add more extra processing for venc_multi_core mode
media: mtk-vcodec: Add venc_multi_core mode encode process
media: mtk-vcodec: Done encode result to client

 .../media/mediatek,vcodec-encoder-core.yaml   | 181 +++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  36 ++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 113 +++++++---
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |   7 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 169 ++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_core.h |  36 +++
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 107 +++++----
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 183 +++++++++++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  11 +-
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 ++
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   5 +
 .../platform/mtk-vcodec/venc/venc_h264_if.c   | 212 +++++++++++++++---
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |   3 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   |  79 +++++--
 .../media/platform/mtk-vcodec/venc_drv_if.h   |   7 +
 .../media/platform/mtk-vcodec/venc_vpu_if.c   |  11 +-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |   3 +-
 19 files changed, 1026 insertions(+), 161 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h
Signature · 1 line
2.18.0
author
#1 lore raw

PATCH v3 1/10 media: mtk-vcodec: Use core type to indicate h264 and vp8 enc

use core type variables to indicate h264 and vp8 encoder

---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  4 +--
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 30 ++++++++++---------
 2 files changed, 18 insertions(+), 16 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +2 -2
index ea58f11e7659..3affb2d4a4ec 100644
@@ -416,7 +416,7 @@ struct mtk_vcodec_dec_pdata {
  * @num_capture_formats: number of entries in capture_formats
  * @output_formats: array of supported output formats
  * @num_output_formats: number of entries in output_formats
- * @core_id: stand for h264 or vp8 encode index
+ * @core_type: stand for h264 or vp8 encode
  */
 struct mtk_vcodec_enc_pdata {
 	bool uses_ext;
@@ -426,7 +426,7 @@ struct mtk_vcodec_enc_pdata {
 	size_t num_capture_formats;
 	const struct mtk_video_fmt *output_formats;
 	size_t num_output_formats;
-	int core_id;
+	int core_type;
 };
 
 #define MTK_ENC_CTX_IS_EXT(ctx) ((ctx)->dev->venc_pdata->uses_ext)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +16 -14
index 95e8c29ccc65..f12f887e4bdc 100644
@@ -94,11 +94,13 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)
 	ctx = dev->curr_ctx;
 	spin_unlock_irqrestore(&dev->irqlock, flags);
 
-	mtk_v4l2_debug(1, "id=%d coreid:%d", ctx->id, dev->venc_pdata->core_id);
-	addr = dev->reg_base[dev->venc_pdata->core_id] +
+	mtk_v4l2_debug(1, "id: %d core type: %d",
+		       ctx->id, dev->venc_pdata->core_type);
+
+	addr = dev->reg_base[dev->venc_pdata->core_type] +
 				MTK_VENC_IRQ_ACK_OFFSET;
 
-	ctx->irq_status = readl(dev->reg_base[dev->venc_pdata->core_id] +
+	ctx->irq_status = readl(dev->reg_base[dev->venc_pdata->core_type] +
 				(MTK_VENC_IRQ_STATUS_OFFSET));
 
 	clean_irq_status(ctx->irq_status, addr);
@@ -265,10 +267,10 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 
 	pm_runtime_enable(&pdev->dev);
 
-	dev->reg_base[dev->venc_pdata->core_id] =
+	dev->reg_base[dev->venc_pdata->core_type] =
 		devm_platform_ioremap_resource(pdev, 0);
-	if (IS_ERR(dev->reg_base[dev->venc_pdata->core_id])) {
-		ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_id]);
+	if (IS_ERR(dev->reg_base[dev->venc_pdata->core_type])) {
+		ret = PTR_ERR(dev->reg_base[dev->venc_pdata->core_type]);
 		goto err_res;
 	}
 
@@ -286,8 +288,8 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 			       0, pdev->name, dev);
 	if (ret) {
 		dev_err(&pdev->dev,
-			"Failed to install dev->enc_irq %d (%d) core_id (%d)",
-			dev->enc_irq, ret, dev->venc_pdata->core_id);
+			"Failed to install dev->enc_irq %d (%d) core_type (%d)",
+			dev->enc_irq, ret, dev->venc_pdata->core_type);
 		ret = -EINVAL;
 		goto err_res;
 	}
@@ -356,7 +358,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	}
 
 	mtk_v4l2_debug(0, "encoder %d registered as /dev/video%d",
-		       dev->venc_pdata->core_id, vfd_enc->num);
+		       dev->venc_pdata->core_type, vfd_enc->num);
 
 	return 0;
 
@@ -382,7 +384,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_avc_pdata = {
 	.num_output_formats = ARRAY_SIZE(mtk_video_formats_output),
 	.min_bitrate = 64,
 	.max_bitrate = 60000000,
-	.core_id = VENC_SYS,
+	.core_type = VENC_SYS,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
@@ -392,7 +394,7 @@ static const struct mtk_vcodec_enc_pdata mt8173_vp8_pdata = {
 	.num_output_formats = ARRAY_SIZE(mtk_video_formats_output),
 	.min_bitrate = 64,
 	.max_bitrate = 9000000,
-	.core_id = VENC_LT_SYS,
+	.core_type = VENC_LT_SYS,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
@@ -403,7 +405,7 @@ static const struct mtk_vcodec_enc_pdata mt8183_pdata = {
 	.num_output_formats = ARRAY_SIZE(mtk_video_formats_output),
 	.min_bitrate = 64,
 	.max_bitrate = 40000000,
-	.core_id = VENC_SYS,
+	.core_type = VENC_SYS,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
@@ -414,7 +416,7 @@ static const struct mtk_vcodec_enc_pdata mt8192_pdata = {
 	.num_output_formats = ARRAY_SIZE(mtk_video_formats_output),
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
-	.core_id = VENC_SYS,
+	.core_type = VENC_SYS,
 };
 
 static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
@@ -425,7 +427,7 @@ static const struct mtk_vcodec_enc_pdata mt8195_pdata = {
 	.num_output_formats = ARRAY_SIZE(mtk_video_formats_output),
 	.min_bitrate = 64,
 	.max_bitrate = 100000000,
-	.core_id = VENC_SYS,
+	.core_type = VENC_SYS,
 };
 
 static const struct of_device_id mtk_vcodec_enc_match[] = {
Signature · 1 line
2.18.0
author
#2 lore raw

PATCH v3 2/10 media: mtk-vcodec: export encoder functions

In reply to Irui Wang

mtk vcodec is built as a module, export some functions to make them visible by other modules.

---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c    | 2 ++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c | 3 +++
 2 files changed, 5 insertions(+)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +2 -0
index c21367038c34..0d1b065048a9 100644
@@ -1431,6 +1431,7 @@ int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
 	mutex_unlock(&dev->enc_mutex);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_venc_unlock);
 
 int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
 {
@@ -1439,6 +1440,7 @@ int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
 	mutex_lock(&dev->enc_mutex);
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_venc_lock);
 
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx)
 {
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +3 -0
index 7055954eb2af..74495edab558 100644
@@ -58,6 +58,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)
 
 	return 0;
 }
+EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 {
@@ -79,6 +80,7 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
 	for (i -= 1; i >= 0; i--)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
+EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_on);
 
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
 {
@@ -88,3 +90,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
 	for (i = enc_clk->clk_num - 1; i >= 0; i--)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
+EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_off);
Signature · 1 line
2.18.0
author
#3 lore raw

PATCH v3 3/10 dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

In reply to Irui Wang

Adds encoder cores dt-bindings for mt8195.

---
 .../media/mediatek,vcodec-encoder-core.yaml   | 181 ++++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 2 files changed, 181 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +181 -0
new file mode 100644
index 000000000000..fcfb48900c76
@@ -0,0 +1,181 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Mediatek Video Encoder Accelerator With Multi Core
+
+maintainers:
+  - Irui Wang <[email protected]>
+
+description: |
+  Mediatek Video Encode is the video encode hardware present in Mediatek
+  SoCs which supports high resolution encoding functionalities. Required
+  parent and child device node.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - mediatek,mt8195-vcodec-enc
+
+  mediatek,scp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      The node of system control processor (SCP), using
+      the remoteproc & rpmsg framework.
+
+  mediatek,venc-multi-core:
+    type: boolean
+    description: |
+      Indicates whether the encoder has multiple cores or not.
+
+  iommus:
+    minItems: 1
+    maxItems: 32
+    description: |
+      List of the hardware port in respective IOMMU block for current Socs.
+      Refer to bindings/iommu/mediatek,iommu.yaml.
+
+  dma-ranges:
+    maxItems: 1
+    description: |
+      Describes the physical address space of IOMMU maps to memory.
+
+  "#address-cells":
+    const: 2
+
+  "#size-cells":
+    const: 2
+
+  ranges: true
+
+# Required child node:
+patternProperties:
+  "^venc-core@[0-9a-f]+$":
+    type: object
+    description: |
+      The video encoder core device node which should be added as subnodes to
+      the main venc node.
+
+    properties:
+      compatible:
+        items:
+         - const: mediatek,mtk-venc-core
+
+      reg:
+        maxItems: 1
+
+      mediatek,core-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.
+
+      iommus:
+        minItems: 1
+        maxItems: 32
+        description: |
+          List of the hardware port in respective IOMMU block for current Socs.
+          Refer to bindings/iommu/mediatek,iommu.yaml.
+
+      interrupts:
+        maxItems: 1
+
+      clocks:
+        maxItems: 1
+
+      clock-names:
+        maxItems: 1
+
+      power-domains:
+        maxItems: 1
+
+    required:
+      - compatible
+      - reg
+      - mediatek,core-id
+      - iommus
+      - interrupts
+      - clocks
+      - clock-names
+      - assigned-clocks
+      - assigned-clock-parents
+      - power-domains
+
+    additionalProperties: false
+
+required:
+  - compatible
+  - mediatek,scp
+  - iommus
+  - dma-ranges
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/memory/mt8195-memory-port.h>
+    #include <dt-bindings/interrupt-controller/irq.h>
+    #include <dt-bindings/clock/mt8195-clk.h>
+    #include <dt-bindings/power/mt8195-power.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        venc {
+            compatible = "mediatek,mt8195-vcodec-enc";
+            mediatek,scp = <&scp>;
+            mediatek,venc-multi-core;
+            iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>;
+            #address-cells = <2>;
+            #size-cells = <2>;
+            ranges;
+            dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>;
+
+            venc-core@1a020000 {
+                compatible = "mediatek,mtk-venc-core";
+                reg = <0 0x1a020000 0 0x10000>;
+                mediatek,core-id = <0>;
+                iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_REC>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>;
+                interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys CLK_VENC_VENC>;
+                clock-names = "clk_venc";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
+                power-domains = <&spm MT8195_POWER_DOMAIN_VENC>;
+            };
+
+            venc-core@1b020000 {
+                compatible = "mediatek,mtk-venc-core";
+                reg = <0 0x1b020000 0 0x10000>;
+                mediatek,core-id = <1>;
+                iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REC>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>;
+                interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>;
+                clock-names = "clk_venc_core1";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
+                power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>;
+            };
+        };
+    };
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml +0 -1
index e7b65a91c92c..0530a694bcbe 100644
@@ -21,7 +21,6 @@ properties:
       - mediatek,mt8173-vcodec-enc
       - mediatek,mt8183-vcodec-enc
       - mediatek,mt8192-vcodec-enc
-      - mediatek,mt8195-vcodec-enc
 
   reg:
     maxItems: 1
Signature · 1 line
2.18.0
#4 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

Hi Irui,

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 8

Quoted 7 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +7 -0
+
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Mediatek Video Encoder Accelerator With Multi Core

s/Mediatek/MediaTek/

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 14

Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +6 -0
+
+maintainers:
+  - Irui Wang <[email protected]>
+
+description: |
+  Mediatek Video Encode is the video encode hardware present in Mediatek

s/Mediatek/MediaTek/

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 17

Quoted 3 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +3 -0
+  SoCs which supports high resolution encoding functionalities. Required
+  parent and child device node.
+

Thanks,
Miles

maintainer
#5 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

In reply to Irui Wang
Quoted 1 line · Irui Wang

Adds encoder cores dt-bindings for mt8195.

Why?

Quoted 37 lines
---
 .../media/mediatek,vcodec-encoder-core.yaml   | 181 ++++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 2 files changed, 181 insertions(+), 1 deletion(-)
 create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +23 -0
new file mode 100644
index 000000000000..fcfb48900c76
@@ -0,0 +1,181 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"
+
+title: Mediatek Video Encoder Accelerator With Multi Core
+
+maintainers:
+  - Irui Wang <[email protected]>
+
+description: |
+  Mediatek Video Encode is the video encode hardware present in Mediatek
+  SoCs which supports high resolution encoding functionalities. Required
+  parent and child device node.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - mediatek,mt8195-vcodec-enc
+

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 33

Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10 -0
+  mediatek,scp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      The node of system control processor (SCP), using
+      the remoteproc & rpmsg framework.
+
+  mediatek,venc-multi-core:
+    type: boolean
+    description: |
+      Indicates whether the encoder has multiple cores or not.

Isn't this implied by how many child nodes you have?

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 37

Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +4 -0
+
+  iommus:
+    minItems: 1
+    maxItems: 32

You really have up to 32 IOMMUs?

Quoted 27 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +27 -0
+    description: |
+      List of the hardware port in respective IOMMU block for current Socs.
+      Refer to bindings/iommu/mediatek,iommu.yaml.
+
+  dma-ranges:
+    maxItems: 1
+    description: |
+      Describes the physical address space of IOMMU maps to memory.
+
+  "#address-cells":
+    const: 2
+
+  "#size-cells":
+    const: 2
+
+  ranges: true
+
+# Required child node:
+patternProperties:
+  "^venc-core@[0-9a-f]+$":
+    type: object
+    description: |
+      The video encoder core device node which should be added as subnodes to
+      the main venc node.
+
+    properties:
+      compatible:

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 74

Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10 -0
+        items:
+         - const: mediatek,mtk-venc-core
+
+      reg:
+        maxItems: 1
+
+      mediatek,core-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.

What is this for and what does its value correspond to in the h/w. We generally don't do made up indices in DT.

Quoted 123 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +107 -0
+
+      iommus:
+        minItems: 1
+        maxItems: 32
+        description: |
+          List of the hardware port in respective IOMMU block for current Socs.
+          Refer to bindings/iommu/mediatek,iommu.yaml.
+
+      interrupts:
+        maxItems: 1
+
+      clocks:
+        maxItems: 1
+
+      clock-names:
+        maxItems: 1
+
+      power-domains:
+        maxItems: 1
+
+    required:
+      - compatible
+      - reg
+      - mediatek,core-id
+      - iommus
+      - interrupts
+      - clocks
+      - clock-names
+      - assigned-clocks
+      - assigned-clock-parents
+      - power-domains
+
+    additionalProperties: false
+
+required:
+  - compatible
+  - mediatek,scp
+  - iommus
+  - dma-ranges
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/memory/mt8195-memory-port.h>
+    #include <dt-bindings/interrupt-controller/irq.h>
+    #include <dt-bindings/clock/mt8195-clk.h>
+    #include <dt-bindings/power/mt8195-power.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        venc {
+            compatible = "mediatek,mt8195-vcodec-enc";
+            mediatek,scp = <&scp>;
+            mediatek,venc-multi-core;
+            iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>;
+            #address-cells = <2>;
+            #size-cells = <2>;
+            ranges;
+            dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>;
+
+            venc-core@1a020000 {
+                compatible = "mediatek,mtk-venc-core";
+                reg = <0 0x1a020000 0 0x10000>;
+                mediatek,core-id = <0>;
+                iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_REC>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>;
+                interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys CLK_VENC_VENC>;
+                clock-names = "clk_venc";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
+                power-domains = <&spm MT8195_POWER_DOMAIN_VENC>;
+            };
+
+            venc-core@1b020000 {
+                compatible = "mediatek,mtk-venc-core";
+                reg = <0 0x1b020000 0 0x10000>;
+                mediatek,core-id = <1>;
+                iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REC>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>;
+                interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>;
+                clock-names = "clk_venc_core1";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>;
+                power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>;
+            };
+        };
+    };
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml +0 -1
index e7b65a91c92c..0530a694bcbe 100644
@@ -21,7 +21,6 @@ properties:
       - mediatek,mt8173-vcodec-enc
       - mediatek,mt8183-vcodec-enc
       - mediatek,mt8192-vcodec-enc
-      - mediatek,mt8195-vcodec-enc
 
   reg:
     maxItems: 1
Signature · 1 line
2.18.0
author
#6 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

Dear Rob,

Thanks for your review and comments.

Quoted 4 lines · Rob Herring
Quoted 1 line · Irui Wang

Adds encoder cores dt-bindings for mt8195.

Why?

mt8195 has two h264 encoder cores and we want to enable both of them. the original encoder bindings file is for single core which may can not handle it, so we new a core dt-bindings. We will think about it again about this dt-bindings.

Quoted 54 lines
Quoted 54 lines
---
 .../media/mediatek,vcodec-encoder-core.yaml   | 181
++++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 2 files changed, 181 insertions(+), 1 deletion(-)
 create mode 100644

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- core.yaml

diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- core.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- core.yaml
new file mode 100644
index 000000000000..fcfb48900c76

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0 -0

encoder-core.yaml

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5 -0
@@ -0,0 +1,181 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "

http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8 -0
+
+title: Mediatek Video Encoder Accelerator With Multi Core
+
+maintainers:
+  - Irui Wang <[email protected]>
+
+description: |
+  Mediatek Video Encode is the video encode hardware present in

Mediatek
+ SoCs which supports high resolution encoding functionalities. Required

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +12 -0
+  parent and child device node.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - mediatek,mt8195-vcodec-enc
+
+  mediatek,scp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      The node of system control processor (SCP), using
Quoted 8 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6 -0
+      the remoteproc & rpmsg framework.
+
+  mediatek,venc-multi-core:
+    type: boolean
+    description: |
+      Indicates whether the encoder has multiple cores or not.

Isn't this implied by how many child nodes you have?

It implies that we have child nodes, distinguish with single core,

Quoted 7 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 5 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +4 -0
+
+  iommus:
+    minItems: 1
+    maxItems: 32

You really have up to 32 IOMMUs?

Currently, totally 27 IOMMUs and used 9 of them.

Quoted 37 lines
Quoted 37 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2 -0
+    description: |
+      List of the hardware port in respective IOMMU block for

current Socs.

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6 -0
+      Refer to bindings/iommu/mediatek,iommu.yaml.
+
+  dma-ranges:
+    maxItems: 1
+    description: |
+      Describes the physical address space of IOMMU maps to

memory.

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +15 -0
+
+  "#address-cells":
+    const: 2
+
+  "#size-cells":
+    const: 2
+
+  ranges: true
+
+# Required child node:
+patternProperties:
+  "^venc-core@[0-9a-f]+$":
+    type: object
+    description: |
+      The video encoder core device node which should be added as

subnodes to

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10 -0
+      the main venc node.
+
+    properties:
+      compatible:
+        items:
+         - const: mediatek,mtk-venc-core
+
+      reg:
+        maxItems: 1
+
Quoted 8 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +4 -0
+      mediatek,core-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.

What is this for and what does its value correspond to in the h/w. We
generally don't do made up indices in DT.

It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?

Quoted 132 lines
Quoted 132 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6 -0
+
+      iommus:
+        minItems: 1
+        maxItems: 32
+        description: |
+          List of the hardware port in respective IOMMU block for

current Socs.

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +69 -0
+          Refer to bindings/iommu/mediatek,iommu.yaml.
+
+      interrupts:
+        maxItems: 1
+
+      clocks:
+        maxItems: 1
+
+      clock-names:
+        maxItems: 1
+
+      power-domains:
+        maxItems: 1
+
+    required:
+      - compatible
+      - reg
+      - mediatek,core-id
+      - iommus
+      - interrupts
+      - clocks
+      - clock-names
+      - assigned-clocks
+      - assigned-clock-parents
+      - power-domains
+
+    additionalProperties: false
+
+required:
+  - compatible
+  - mediatek,scp
+  - iommus
+  - dma-ranges
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/memory/mt8195-memory-port.h>
+    #include <dt-bindings/interrupt-controller/irq.h>
+    #include <dt-bindings/clock/mt8195-clk.h>
+    #include <dt-bindings/power/mt8195-power.h>
+
+    soc {
+        #address-cells = <2>;
+        #size-cells = <2>;
+
+        venc {
+            compatible = "mediatek,mt8195-vcodec-enc";
+            mediatek,scp = <&scp>;
+            mediatek,venc-multi-core;
+            iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>;
+            #address-cells = <2>;
+            #size-cells = <2>;
+            ranges;
+            dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>;
+
+            venc-core@1a020000 {
+                compatible = "mediatek,mtk-venc-core";
+                reg = <0 0x1a020000 0 0x10000>;
+                mediatek,core-id = <0>;
+                iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_REC>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>,
+                         <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>,
+                         <&iommu_vdo

M4U_PORT_L19_VENC_CUR_CHROMA>,

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2 -0
+                         <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>,
+                         <&iommu_vdo

M4U_PORT_L19_VENC_REF_CHROMA>;

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5 -0
+                interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys CLK_VENC_VENC>;
+                clock-names = "clk_venc";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen

CLK_TOP_UNIVPLL_D4>;

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +14 -0
+                power-domains = <&spm MT8195_POWER_DOMAIN_VENC>;
+            };
+
+            venc-core@1b020000 {
+                compatible = "mediatek,mtk-venc-core";
+                reg = <0 0x1b020000 0 0x10000>;
+                mediatek,core-id = <1>;
+                iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_REC>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>,
+                         <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>,
+                         <&iommu_vpp

M4U_PORT_L20_VENC_CUR_CHROMA>,

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2 -0
+                         <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>,
+                         <&iommu_vpp

M4U_PORT_L20_VENC_REF_CHROMA>;

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5 -0
+                interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>;
+                clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>;
+                clock-names = "clk_venc_core1";
+                assigned-clocks = <&topckgen CLK_TOP_VENC>;
+                assigned-clock-parents = <&topckgen

CLK_TOP_UNIVPLL_D4>;
+ power-domains = <&spm
MT8195_POWER_DOMAIN_VENC_CORE1>;

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3 -0
+            };
+        };
+    };

diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder.yaml
index e7b65a91c92c..0530a694bcbe 100644
--- a/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder.yaml

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0 -1
@@ -21,7 +21,6 @@ properties:
       - mediatek,mt8173-vcodec-enc
       - mediatek,mt8183-vcodec-enc
       - mediatek,mt8192-vcodec-enc
-      - mediatek,mt8195-vcodec-enc
 
Quoted 6 lines
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0 -0
   reg:
     maxItems: 1
Signature · 1 line
2.18.0

Thanks
Best Regards

maintainer
#7 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

Quoted 75 lines · Irui Wang

Dear Rob,

Thanks for your review and comments.

Quoted 4 lines · Rob Herring
Quoted 1 line · Irui Wang

Adds encoder cores dt-bindings for mt8195.

Why?

mt8195 has two h264 encoder cores and we want to enable both of them. the original encoder bindings file is for single core which may can not handle it, so we new a core dt-bindings. We will think about it again about this dt-bindings.

Quoted 54 lines
Quoted 54 lines
---
 .../media/mediatek,vcodec-encoder-core.yaml   | 181
++++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 2 files changed, 181 insertions(+), 1 deletion(-)
 create mode 100644

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- core.yaml

diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- core.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder- core.yaml
new file mode 100644
index 000000000000..fcfb48900c76

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0 -0

encoder-core.yaml

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5 -0
@@ -0,0 +1,181 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "

http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8 -0
+
+title: Mediatek Video Encoder Accelerator With Multi Core
+
+maintainers:
+  - Irui Wang <[email protected]>
+
+description: |
+  Mediatek Video Encode is the video encode hardware present in

Mediatek
+ SoCs which supports high resolution encoding functionalities. Required

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +12 -0
+  parent and child device node.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - mediatek,mt8195-vcodec-enc
+
+  mediatek,scp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      The node of system control processor (SCP), using
Quoted 8 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6 -0
+      the remoteproc & rpmsg framework.
+
+  mediatek,venc-multi-core:
+    type: boolean
+    description: |
+      Indicates whether the encoder has multiple cores or not.

Isn't this implied by how many child nodes you have?

Quoted 1 line

It implies that we have child nodes, distinguish with single core,

Just check if you have a child node or not.

Quoted 51 lines
Quoted 5 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 3 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3 -0
+  iommus:
+    minItems: 1
+    maxItems: 32

You really have up to 32 IOMMUs?

Currently, totally 27 IOMMUs and used 9 of them.

Quoted 37 lines
Quoted 37 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2 -0
+    description: |
+      List of the hardware port in respective IOMMU block for

current Socs.

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6 -0
+      Refer to bindings/iommu/mediatek,iommu.yaml.
+
+  dma-ranges:
+    maxItems: 1
+    description: |
+      Describes the physical address space of IOMMU maps to

memory.

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +15 -0
+
+  "#address-cells":
+    const: 2
+
+  "#size-cells":
+    const: 2
+
+  ranges: true
+
+# Required child node:
+patternProperties:
+  "^venc-core@[0-9a-f]+$":
+    type: object
+    description: |
+      The video encoder core device node which should be added as

subnodes to

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10 -0
+      the main venc node.
+
+    properties:
+      compatible:
+        items:
+         - const: mediatek,mtk-venc-core
+
+      reg:
+        maxItems: 1
+
Quoted 8 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +4 -0
+      mediatek,core-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.

What is this for and what does its value correspond to in the h/w. We
generally don't do made up indices in DT.

Quoted 4 lines

It's for encoder core id, core@1a020000 must be core-0, core@1b020000 must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?

I still don't understand why it is needed. What is 'core-0'?

Is there some functional difference between the cores? If so, describe that difference.

Rob

author
#8 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

Dear Rob,

Many thanks for your attention.

Quoted 42 lines · Rob Herring
Quoted 41 lines · Irui Wang

Dear Rob,

Thanks for your review and comments.

Quoted 4 lines · Rob Herring
Quoted 1 line · Irui Wang

Adds encoder cores dt-bindings for mt8195.

Why?

mt8195 has two h264 encoder cores and we want to enable both of them.
the original encoder bindings file is for single core which may can not
handle it, so we new a core dt-bindings. We will think about it again
about this dt-bindings.

Quoted 24 lines
Quoted 24 lines
---
 .../media/mediatek,vcodec-encoder-core.yaml   | 181
++++++++++++++++++
 .../media/mediatek,vcodec-encoder.yaml        |   1 -
 2 files changed, 181 insertions(+), 1 deletion(-)
 create mode 100644

Documentation/devicetree/bindings/media/mediatek,vcodec- encoder-
core.yaml

diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder-
core.yaml b/Documentation/devicetree/bindings/media/mediatek,vcodec- encoder-
core.yaml
new file mode 100644
index 000000000000..fcfb48900c76

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0 -0
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

Documentation/devicetree/bindings/media/mediatek,vcodec- line 5

Quoted 8 lines

encoder-core.yaml

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5 -0
@@ -0,0 +1,181 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: "

http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#

Quoted 36 lines
Quoted 36 lines
Quoted 35 lines
Quoted 23 lines

"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8 -0
+
+title: Mediatek Video Encoder Accelerator With Multi Core
+
+maintainers:
+  - Irui Wang <[email protected]>
+
+description: |
+  Mediatek Video Encode is the video encode hardware present

in
Mediatek
+ SoCs which supports high resolution encoding
functionalities.
Required

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8 -0
+  parent and child device node.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - mediatek,mt8195-vcodec-enc
+

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +10 -0
+  mediatek,scp:
+    $ref: /schemas/types.yaml#/definitions/phandle
+    description: |
+      The node of system control processor (SCP), using
+      the remoteproc & rpmsg framework.
+
+  mediatek,venc-multi-core:
+    type: boolean
+    description: |
+      Indicates whether the encoder has multiple cores or not.

Isn't this implied by how many child nodes you have?

Quoted 4 lines
Quoted 2 lines

It implies that we have child nodes, distinguish with single core,

Just check if you have a child node or not.

Yes.

Quoted 63 lines
Quoted 57 lines
Quoted 7 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 5 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3 -0
+  iommus:
+    minItems: 1
+    maxItems: 32

You really have up to 32 IOMMUs?

Currently, totally 27 IOMMUs and used 9 of them.

Quoted 39 lines
Quoted 39 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2 -0
+    description: |
+      List of the hardware port in respective IOMMU block for

current Socs.

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6 -0
+      Refer to bindings/iommu/mediatek,iommu.yaml.
+
+  dma-ranges:
+    maxItems: 1
+    description: |
+      Describes the physical address space of IOMMU maps to

memory.

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +15 -0
+
+  "#address-cells":
+    const: 2
+
+  "#size-cells":
+    const: 2
+
+  ranges: true
+
+# Required child node:
+patternProperties:
+  "^venc-core@[0-9a-f]+$":
+    type: object
+    description: |
+      The video encoder core device node which should be added

as
subnodes to

Documentation/devicetree/bindings/media/mediatek,vcodec- new file +11 -0
+      the main venc node.
+
+    properties:
+      compatible:
+        items:
+         - const: mediatek,mtk-venc-core
+
+      reg:
+        maxItems: 1
+
+      mediatek,core-id:
Quoted 8 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-

Quoted 3 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3 -0
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.

What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.

Quoted 5 lines

It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?

Quoted 7 lines

I still don't understand why it is needed. What is 'core-0'?

Is there some functional difference between the cores? If so, describe
that difference.

Rob

They are two different pieces of hardware, it's our encoder hardware design. There are two encoder hardware cores inside MT8195, named core0 and core1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1".
So the two encoder cores have their own base, IRQ, clock, power, etc. Each core can encode independently, moreover, they can work together for higher performance.
We will describe more details in YAML about it if it's OK for you.

Thanks
Best Regards.

maintainer
#9 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

Quoted 13 lines · Irui Wang

Dear Rob,

Many thanks for your attention.

Quoted 8 lines · Rob Herring
Quoted 7 lines · Irui Wang

Dear Rob,

Thanks for your review and comments.

Quoted 2 lines · Rob Herring
Quoted 1 line · Irui Wang

Adds encoder cores dt-bindings for mt8195.

[...]

Quoted 24 lines
Quoted 16 lines
Quoted 10 lines
Quoted 9 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 74

Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +4 -0
+      mediatek,core-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.

What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.

Quoted 5 lines

It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can get core-id in drivers. If it can't ref "uint32" types yaml, would you mind giving some more suggestions ?

Quoted 7 lines

I still don't understand why it is needed. What is 'core-0'?

Is there some functional difference between the cores? If so, describe
that difference.

Rob

Quoted 11 lines

They are two different pieces of hardware, it's our encoder hardware design. There are two encoder hardware cores inside MT8195, named core0 and core1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1".
So the two encoder cores have their own base, IRQ, clock, power, etc. Each core can encode independently, moreover, they can work together for higher performance.
We will describe more details in YAML about it if it's OK for you.

All the resources you list are in the child nodes, so you don't need 0 and 1 numbering for those.

Looking at the driver patches, the only thing I see distinguishing core numbers is this:

"frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,

Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0."

Is this a requirement in the h/w or just what the driver picked? IOW, could frame#0 use core#1?

Rob

author
#10 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

Dear Rob,

A sample encoder hardware block diagram attached.

Quoted 66 lines · Rob Herring
Quoted 13 lines · Irui Wang

Dear Rob,

Many thanks for your attention.

Quoted 8 lines · Rob Herring
Quoted 7 lines · Irui Wang

Dear Rob,

Thanks for your review and comments.

Quoted 2 lines · Rob Herring
Quoted 1 line · Irui Wang

Adds encoder cores dt-bindings for mt8195.

[...]

Quoted 26 lines
Quoted 18 lines
Quoted 10 lines
Quoted 9 lines

Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 74

Quoted 4 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +4 -0
+      mediatek,core-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.

What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.

Quoted 7 lines

It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can
get core-id in drivers. If it can't ref "uint32" types yaml, would
you mind giving some more suggestions ?

Quoted 7 lines

I still don't understand why it is needed. What is 'core-0'?

Is there some functional difference between the cores? If so, describe
that difference.

Rob

Quoted 15 lines

They are two different pieces of hardware, it's our encoder hardware
design. There are two encoder hardware cores inside MT8195, named core0
and core1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1". So the two encoder cores have their own base, IRQ, clock, power, etc.
Each core can encode independently, moreover, they can work together
for higher performance.
We will describe more details in YAML about it if it's OK for you.

All the resources you list are in the child nodes, so you don't need 0
and 1 numbering for those.

Looking at the driver patches, the only thing I see distinguishing core numbers is this:

Quoted 8 lines

"frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,

Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0."

Is this a requirement in the h/w or just what the driver picked? IOW,
could frame#0 use core#1?

No, it's a requirement in the h/w, driver trigger core start encoding is in order.
About the encoder hardware block diagram, please check below:
--------------------------------------------------------------

Input Buffer: 0     1     2     3     4     5     6
              |     |     |     |     |     |     |
              v     |     v     |     v     |     v
          +-------+ | +-------+ | +-------+ | +-------+
          | core0 | | | core0 | | | core0 | | | core0 |
          +-------+ | +-------+ | +-------+ | +-------+
              |     |     |     |     |     |     |
              |     v     |     v     |     v     |
              | +-------+ | +-------+ | +-------+ |
              | | core1 | | | core1 | | | core1 | |
              | +-------+ | +-------+ | +-------+ |
              |     |     |     |     |     |     |
              v     v     v     v     v     v     v    <parent>
--------------------------------------------------------------
                        core || index                  <child>
                             \/
       +-----------------------------------------------+
       |                  core0/core1                  |
       |          enable/disable power/clk/irq         |
       +-----------------------------------------------+

--------------------------------------------------------------
As above , there are parent and child devices, child mean each venc core, the child device controls the information of each core independent which inlcude power/clk/irq.
When start encoding, input buffer 0 will be encoded by core0, and input buffer 1 can be encoded by core1 even if buffer 0 has not been encoded done yet, after buffer 0 encoded done, buffer 2 will be encoded by core0, and buffer 1 encoded done by core1. These two encoder cores will encode each input in this overlapping manner.

We need manage each child device in parent device by core-id property. And we also need record current encoding input buffer, encode done output buffers and which one core is in used through core-id, because the two cores are encoding at the same time under one parent driver.

Quoted 2 lines

Rob

Thanks
Best Regards

maintainer
#11 lore raw

Re: [PATCH v3, 03/10] dt-bindings: media: mtk-vcodec: Adds encoder cores dt-bindings for mt8195

Quoted 112 lines · Irui Wang

Dear Rob,

A sample encoder hardware block diagram attached.

Quoted 66 lines · Rob Herring
Quoted 13 lines · Irui Wang

Dear Rob,

Many thanks for your attention.

Quoted 8 lines · Rob Herring
Quoted 7 lines · Irui Wang

Dear Rob,

Thanks for your review and comments.

Quoted 2 lines · Rob Herring
Quoted 1 line · Irui Wang

Adds encoder cores dt-bindings for mt8195.

[...]

Quoted 26 lines
Quoted 18 lines
Quoted 10 lines
Quoted 9 lines

Review

Quoted 4 lines
(quoted diff) +4 -0
+      mediatek,core-id:
+        $ref: /schemas/types.yaml#/definitions/uint32
+        description: |
+          Current encoder core id.

What is this for and what does its value correspond to in the h/w.
We
generally don't do made up indices in DT.

Quoted 7 lines

It's for encoder core id, core@1a020000 must be core-0, core@1b020000
must be core-1, we add this property in each child node, so we can
get core-id in drivers. If it can't ref "uint32" types yaml, would
you mind giving some more suggestions ?

Quoted 7 lines

I still don't understand why it is needed. What is 'core-0'?

Is there some functional difference between the cores? If so, describe
that difference.

Rob

Quoted 15 lines

They are two different pieces of hardware, it's our encoder hardware
design. There are two encoder hardware cores inside MT8195, named core0
and core1(we can rename it, but core id should be declared), for core0, its module base address is 0x1A02_0000, uses IOMMU "vdo0_iommu" and power domain "POWER_DOMAIN_VENC",
for core1, its module base address is 0x1B02_0000, uses IOMMU "vpp_iommu" and power domain "POWER_DOMAIN_VENC_CORE1". So the two encoder cores have their own base, IRQ, clock, power, etc.
Each core can encode independently, moreover, they can work together
for higher performance.
We will describe more details in YAML about it if it's OK for you.

All the resources you list are in the child nodes, so you don't need 0
and 1 numbering for those.

Looking at the driver patches, the only thing I see distinguishing core numbers is this:

Quoted 8 lines

"frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,

Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0."

Is this a requirement in the h/w or just what the driver picked? IOW,
could frame#0 use core#1?

No, it's a requirement in the h/w, driver trigger core start encoding is in order.
About the encoder hardware block diagram, please check below:
--------------------------------------------------------------

Input Buffer: 0     1     2     3     4     5     6
              |     |     |     |     |     |     |
              v     |     v     |     v     |     v
          +-------+ | +-------+ | +-------+ | +-------+
          | core0 | | | core0 | | | core0 | | | core0 |
          +-------+ | +-------+ | +-------+ | +-------+
              |     |     |     |     |     |     |
              |     v     |     v     |     v     |
              | +-------+ | +-------+ | +-------+ |
              | | core1 | | | core1 | | | core1 | |
              | +-------+ | +-------+ | +-------+ |
              |     |     |     |     |     |     |
              v     v     v     v     v     v     v    <parent>
--------------------------------------------------------------
                        core || index                  <child>
                             \/
       +-----------------------------------------------+
       |                  core0/core1                  |
       |          enable/disable power/clk/irq         |
       +-----------------------------------------------+

--------------------------------------------------------------
As above , there are parent and child devices, child mean each venc core, the child device controls the information of each core independent which inlcude power/clk/irq.
When start encoding, input buffer 0 will be encoded by core0, and input buffer 1 can be encoded by core1 even if buffer 0 has not been encoded done yet, after buffer 0 encoded done, buffer 2 will be encoded by core0, and buffer 1 encoded done by core1. These two encoder cores will encode each input in this overlapping manner.

Quoted 4 lines

We need manage each child device in parent device by core-id property. And we also need record current encoding input buffer, encode done output buffers and which one core is in used through core-id, because the two cores are encoding at the same time under one parent driver.

Okay, please summarize the h/w requirement in the property description.

You might put the above diagram in the top description too.

Rob

author
#12 lore raw

PATCH v3 4/10 media: mtk-vcodec: Enable venc dual core usage

In reply to Irui Wang

Adds new property to indicate whether the encoder has multiple cores.

Use of_platform_populate to probe each venc cores, the core device can use the init_clk/request_irq helper to initialize their own power/clk/irq, and the main device doesn't need use these helper anymore.

---
 drivers/media/platform/mtk-vcodec/Makefile    |   4 +-
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  14 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 138 ++++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_core.h |  36 +++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  77 +++++-----
 5 files changed, 235 insertions(+), 34 deletions(-)
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c
 create mode 100644 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h
drivers/media/platform/mtk-vcodec/Makefile +3 -1
index 93e7a343b5b0..c472b221bd6b 100644
@@ -3,7 +3,8 @@
 obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \
 				       mtk-vcodec-enc.o \
 				       mtk-vcodec-common.o \
-				       mtk-vcodec-dec-hw.o
+				       mtk-vcodec-dec-hw.o \
+				       mtk-vcodec-enc-core.o
 
 mtk-vcodec-dec-y := vdec/vdec_h264_if.o \
 		vdec/vdec_vp8_if.o \
@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \
 		venc_drv_if.o \
 		venc_vpu_if.o \
 
+mtk-vcodec-enc-core-y := mtk_vcodec_enc_core.o
 
 mtk-vcodec-common-y := mtk_vcodec_intr.o \
 		mtk_vcodec_util.o \
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +14 -0
index 3affb2d4a4ec..43e1349d6a54 100644
@@ -117,6 +117,15 @@ enum mtk_vdec_hw_count {
 	MTK_VDEC_MAX_HW_COUNT,
 };
 
+/*
+ * enum mtk_venc_core_id -- encoder core id
+ */
+enum mtk_venc_core_id {
+	MTK_VENC_CORE0 = 0,
+	MTK_VENC_CORE1,
+	MTK_VENC_CORE_MAX,
+};
+
 /*
  * struct mtk_video_fmt - Structure used to store information about pixelformats
  */
@@ -476,6 +485,8 @@ struct mtk_vcodec_enc_pdata {
  * @subdev_dev: subdev hardware device
  * @subdev_prob_done: check whether all used hw device is prob done
  * @subdev_bitmap: used to record hardware is ready or not
+ * @venc_multi_core: indicates whether the encoder has multiple cores or not
+ * @enc_core_dev: used to store venc core device
  */
 struct mtk_vcodec_dev {
 	struct v4l2_device v4l2_dev;
@@ -521,6 +532,9 @@ struct mtk_vcodec_dev {
 	void *subdev_dev[MTK_VDEC_HW_MAX];
 	int (*subdev_prob_done)(struct mtk_vcodec_dev *vdec_dev);
 	DECLARE_BITMAP(subdev_bitmap, MTK_VDEC_HW_MAX);
+
+	bool venc_multi_core;
+	void *enc_core_dev[MTK_VENC_CORE_MAX];
 };
 
 static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh *fh)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c new file +138 -0

138 lines not on this page · raw message

drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.h new file +36 -0
new file mode 100644
index 000000000000..9c0f661509df
@@ -0,0 +1,36 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Copyright (c) 2021 MediaTek Inc.
+ */
+
+#ifndef _MTK_VCODEC_ENC_CORE_H_
+#define _MTK_VCODEC_ENC_CORE_H_
+
+#include <linux/platform_device.h>
+#include "mtk_vcodec_drv.h"
+
+/**
+ * struct mtk_venc_core_dev - driver data
+ * @plat_dev: platform_device
+ * @main_dev: main device
+ * @pm: power management data
+ * @curr_ctx: the context that is waiting for venc hardware
+ * @reg_base: mapped address of venc registers
+ * @irq_status: venc core irq status
+ * @enc_irq: venc device irq
+ * @core id: for venc core id: core#0, core#1...
+ */
+struct mtk_venc_core_dev {
+	struct platform_device *plat_dev;
+	struct mtk_vcodec_dev *main_dev;
+
+	struct mtk_vcodec_pm pm;
+	struct mtk_vcodec_ctx *curr_ctx;
+
+	void __iomem *reg_base;
+	unsigned int irq_status;
+	int enc_irq;
+	int core_id;
+};
+
+#endif /* _MTK_VCODEC_ENC_CORE_H_ */
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +44 -33

114 lines not on this page · raw message

Signature · 1 line
2.18.0
author
#13 lore raw

PATCH v3 5/10 media: mtk-vcodec: mtk-vcodec: Rewrite venc power manage interface

In reply to Irui Wang

The args "struct mtk_vcodec_dev *" doesn't appropriate for init_clk functions because of sub-devices. For main device, it has no "pm/clk" properties in dtsi, sub-devices will init their own "pm/clk" instead. So rewrite the pm interface with args "platform_device *" and "mtk_vcodec_pm*".

---
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c | 8 ++++++++
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c  | 2 +-
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 8 ++------
 drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h   | 4 ++--
 4 files changed, 13 insertions(+), 9 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c +8 -0
index 9d3ea5a8afe4..7e376413beaa 100644
@@ -13,6 +13,7 @@
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_enc.h"
 #include "mtk_vcodec_enc_core.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 
 static const struct of_device_id mtk_venc_core_ids[] = {
@@ -117,6 +118,13 @@ static int mtk_venc_core_probe(struct platform_device *pdev)
 		return dev_err_probe(dev, -EINVAL,
 				     "Invalid core id %d\n", core->core_id);
 
+	ret = mtk_vcodec_init_enc_clk(core->plat_dev, &core->pm);
+	if (ret < 0)
+		return dev_err_probe(dev, ret,
+				     "Failed to get venc core clock source!");
+
+	pm_runtime_enable(&pdev->dev);
+
 	main_dev->enc_core_dev[core->core_id] = core;
 	core->main_dev = main_dev;
 
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +1 -1
index d540d52191d9..1b1c17b03ef5 100644
@@ -272,7 +272,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	}
 
 	if (!dev->venc_multi_core) {
-		ret = mtk_vcodec_init_enc_clk(dev);
+		ret = mtk_vcodec_init_enc_clk(dev->plat_dev, &dev->pm);
 		if (ret < 0) {
 			dev_err(&pdev->dev,
 				"Failed to get mtk vcodec clock source!");
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +2 -6
index 74495edab558..1e814fc3172f 100644
@@ -12,17 +12,13 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_util.h"
 
-int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)
+int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
+			    struct mtk_vcodec_pm *pm)
 {
-	struct platform_device *pdev;
-	struct mtk_vcodec_pm *pm;
 	struct mtk_vcodec_clk *enc_clk;
 	struct mtk_vcodec_clk_info *clk_info;
 	int ret, i;
 
-	pdev = mtkdev->plat_dev;
-	pm = &mtkdev->pm;
-	memset(pm, 0, sizeof(struct mtk_vcodec_pm));
 	pm->dev = &pdev->dev;
 	enc_clk = &pm->venc_clk;
 
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +2 -2
index bc455cefc0cd..97a394c68f4e 100644
@@ -9,8 +9,8 @@
 
 #include "mtk_vcodec_drv.h"
 
-int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *dev);
-
+int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
+			    struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
 
Signature · 1 line
2.18.0
author
#14 lore raw

PATCH v3 6/10 media: mtk-vcodec: Add venc power on/off interface

In reply to Irui Wang

venc_multi_core mode power on/off all venc available power, pm_runtime_xx helper is not appropriate called directly, add new power on/off interface for it.

---
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 20 +----
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  7 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 83 +++++++++++++++++++
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  5 ++
 4 files changed, 98 insertions(+), 17 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +3 -17
index 0d1b065048a9..b0bc71f00930 100644
@@ -12,6 +12,7 @@
 
 #include "mtk_vcodec_drv.h"
 #include "mtk_vcodec_enc.h"
+#include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 #include "mtk_vcodec_util.h"
 #include "venc_drv_if.h"
@@ -849,7 +850,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 {
 	struct mtk_vcodec_ctx *ctx = vb2_get_drv_priv(q);
 	struct venc_enc_param param;
-	int ret, pm_ret;
+	int ret;
 	int i;
 
 	/* Once state turn into MTK_STATE_ABORT, we need stop_streaming
@@ -857,7 +858,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	  */
 	if ((ctx->state == MTK_STATE_ABORT) || (ctx->state == MTK_STATE_FREE)) {
 		ret = -EIO;
-		goto err_start_stream;
+		goto err_set_param;
 	}
 
 	/* Do the initialization when both start_streaming have been called */
@@ -869,12 +870,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 			return 0;
 	}
 
-	ret = pm_runtime_resume_and_get(&ctx->dev->plat_dev->dev);
-	if (ret < 0) {
-		mtk_v4l2_err("pm_runtime_resume_and_get fail %d", ret);
-		goto err_start_stream;
-	}
-
 	mtk_venc_set_param(ctx, &param);
 	ret = venc_if_set_param(ctx, VENC_SET_PARAM_ENC, &param);
 	if (ret) {
@@ -901,11 +896,6 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)
 	return 0;
 
 err_set_param:
-	pm_ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (pm_ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", pm_ret);
-
-err_start_stream:
 	for (i = 0; i < q->num_buffers; ++i) {
 		struct vb2_buffer *buf = vb2_get_buffer(q, i);
 
@@ -988,10 +978,6 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)
 	if (ret)
 		mtk_v4l2_err("venc_if_deinit failed=%d", ret);
 
-	ret = pm_runtime_put(&ctx->dev->plat_dev->dev);
-	if (ret < 0)
-		mtk_v4l2_err("pm_runtime_put fail %d", ret);
-
 	ctx->state = MTK_STATE_FREE;
 }
 
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +7 -0
index 1b1c17b03ef5..d2937e62df29 100644
@@ -173,6 +173,12 @@ static int fops_vcodec_open(struct file *file)
 		mtk_v4l2_debug(0, "encoder capability %x", dev->enc_capability);
 	}
 
+	ret = mtk_vcodec_enc_pw_on(dev);
+	if (ret < 0) {
+		mtk_v4l2_err("encoder power on failed %d", ret);
+		goto err_load_fw;
+	}
+
 	mtk_v4l2_debug(2, "Create instance [%d]@%p m2m_ctx=%p ",
 			ctx->id, ctx, ctx->m2m_ctx);
 
@@ -207,6 +213,7 @@ static int fops_vcodec_release(struct file *file)
 
 	v4l2_m2m_ctx_release(ctx->m2m_ctx);
 	mtk_vcodec_enc_release(ctx);
+	mtk_vcodec_enc_pw_off(dev);
 	v4l2_fh_del(&ctx->fh);
 	v4l2_fh_exit(&ctx->fh);
 	v4l2_ctrl_handler_free(&ctx->ctrl_hdl);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +83 -0

92 lines not on this page · raw message

drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +5 -0
index 97a394c68f4e..99b6b6e29e35 100644
@@ -14,4 +14,9 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
 void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
 
+int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev);
+int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_enc_pw_on(struct mtk_vcodec_dev *dev);
+int mtk_vcodec_enc_pw_off(struct mtk_vcodec_dev *dev);
+
 #endif /* _MTK_VCODEC_ENC_PM_H_ */
Signature · 1 line
2.18.0
author
#15 lore raw

PATCH v3 7/10 media: mtk-vcodec: Rewrite venc clock interface

In reply to Irui Wang

venc_multi_core mode need enable the specific core's clk, add an another param for clock_on/clock_off interface.

---
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.c   | 95 ++++++++++++++++---
 .../platform/mtk-vcodec/mtk_vcodec_enc_pm.h   |  4 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   | 10 +-
 3 files changed, 84 insertions(+), 25 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.c +80 -15
index 22b2eb867998..cdf7083c9335 100644
@@ -57,42 +57,86 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);
 
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
+	struct mtk_venc_core_dev *core;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
+	struct clk              *clk;
 	int ret, i = 0;
 
-	for (i = 0; i < enc_clk->clk_num; i++) {
-		ret = clk_prepare_enable(enc_clk->clk_info[i].vcodec_clk);
-		if (ret) {
-			mtk_v4l2_err("venc clk_prepare_enable %d %s fail %d", i,
-				enc_clk->clk_info[i].clk_name, ret);
-			goto clkerr;
+	if (dev->venc_multi_core) {
+		core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id];
+		enc_pm = &core->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_enable %d %s fail %d", i,
+					     enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto core_clk_err;
+			}
+		}
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = 0; i < enc_clk->clk_num; i++) {
+			clk = enc_clk->clk_info[i].vcodec_clk;
+			ret = clk_prepare_enable(clk);
+			if (ret) {
+				mtk_v4l2_err("clk_prepare %d %s fail %d",
+					     i, enc_clk->clk_info[i].clk_name,
+					     ret);
+				goto clkerr;
+			}
 		}
 	}
 
 	return;
 
+core_clk_err:
+	for (i -= 1; i >= 0; i--)
+		clk_disable(enc_clk->clk_info[i].vcodec_clk);
 clkerr:
 	for (i -= 1; i >= 0; i--)
 		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_on);
 
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm)
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id)
 {
-	struct mtk_vcodec_clk *enc_clk = &pm->venc_clk;
+	struct mtk_venc_core_dev *core;
+	struct mtk_vcodec_pm *enc_pm;
+	struct mtk_vcodec_clk *enc_clk;
 	int i = 0;
 
-	for (i = enc_clk->clk_num - 1; i >= 0; i--)
-		clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	if (dev->venc_multi_core) {
+		core = (struct mtk_venc_core_dev *)dev->enc_core_dev[core_id];
+		enc_pm = &core->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable(enc_clk->clk_info[i].vcodec_clk);
+	} else {
+		enc_pm = &dev->pm;
+		enc_clk = &enc_pm->venc_clk;
+
+		for (i = enc_clk->clk_num - 1; i >= 0; i--)
+			clk_disable_unprepare(enc_clk->clk_info[i].vcodec_clk);
+	}
+
 }
 EXPORT_SYMBOL_GPL(mtk_vcodec_enc_clock_off);
 
 int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev)
 {
-	int i, ret;
+	int i, ret, j;
 	struct mtk_venc_core_dev *core;
+	struct mtk_vcodec_clk *clk;
 
 	/* power on all available venc cores */
 	for (i = 0; i < MTK_VENC_CORE_MAX; i++) {
@@ -102,15 +146,31 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev)
 
 		ret = pm_runtime_resume_and_get(&core->plat_dev->dev);
 		if (ret < 0) {
-			mtk_v4l2_err("power on core[%d] fail %d", i, ret);
+			mtk_v4l2_err("power on core[%d] fail", i);
 			goto pw_on_fail;
 		}
+
+		clk = &core->pm.venc_clk;
+		for (j = 0; j < clk->clk_num; j++) {
+			ret = clk_prepare(clk->clk_info[j].vcodec_clk);
+			if (ret) {
+				mtk_v4l2_err("prepare clk [%s] fail %d",
+					     clk->clk_info[j].clk_name,
+					     ret);
+				goto pw_on_fail;
+			}
+		}
 	}
 	return 0;
 
 pw_on_fail:
 	for (i -= 1; i >= 0; i--) {
 		core = (struct mtk_venc_core_dev *)dev->enc_core_dev[i];
+
+		clk = &core->pm.venc_clk;
+		for (j -= 1; j >= 0; j--)
+			clk_unprepare(clk->clk_info[j].vcodec_clk);
+
 		pm_runtime_put_sync(&core->plat_dev->dev);
 	}
 	return ret;
@@ -118,8 +178,9 @@ int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev)
 
 int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev)
 {
-	int i, ret;
+	int i, ret, j;
 	struct mtk_venc_core_dev *core;
+	struct mtk_vcodec_clk *clk;
 
 	/* power off all available venc cores */
 	for (i = 0; i < MTK_VENC_CORE_MAX; i++) {
@@ -127,6 +188,10 @@ int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev)
 		if (!core)
 			return 0;
 
+		clk = &core->pm.venc_clk;
+		for (j = clk->clk_num - 1; j >= 0; j--)
+			clk_unprepare(clk->clk_info[j].vcodec_clk);
+
 		ret = pm_runtime_put_sync(&core->plat_dev->dev);
 		if (ret < 0)
 			mtk_v4l2_err("power off core[%d] fail %d", i, ret);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_pm.h +2 -2
index 99b6b6e29e35..5113ed8a869e 100644
@@ -11,8 +11,8 @@
 
 int mtk_vcodec_init_enc_clk(struct platform_device *pdev,
 			    struct mtk_vcodec_pm *pm);
-void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm);
-void mtk_vcodec_enc_clock_off(struct mtk_vcodec_pm *pm);
+void mtk_vcodec_enc_clock_on(struct mtk_vcodec_dev *dev, int core_id);
+void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int core_id);
 
 int mtk_venc_core_pw_on(struct mtk_vcodec_dev *dev);
 int mtk_venc_core_pw_off(struct mtk_vcodec_dev *dev);
drivers/media/platform/mtk-vcodec/venc_drv_if.c +2 -8
index ce0bce811615..6cbdb7e30bb3 100644
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc)
 	}
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->init(ctx);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,
 	int ret = 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->set_param(ctx->drv_handle, type, in);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	return ret;
@@ -68,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 	ctx->dev->curr_ctx = ctx;
 	spin_unlock_irqrestore(&ctx->dev->irqlock, flags);
 
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_on(ctx->dev, 0);
 	ret = ctx->enc_if->encode(ctx->drv_handle, opt, frm_buf,
 				  bs_buf, result);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
+	mtk_vcodec_enc_clock_off(ctx->dev, 0);
 
 	spin_lock_irqsave(&ctx->dev->irqlock, flags);
 	ctx->dev->curr_ctx = NULL;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)
 		return 0;
 
 	mtk_venc_lock(ctx);
-	mtk_vcodec_enc_clock_on(&ctx->dev->pm);
 	ret = ctx->enc_if->deinit(ctx->drv_handle);
-	mtk_vcodec_enc_clock_off(&ctx->dev->pm);
 	mtk_venc_unlock(ctx);
 
 	ctx->drv_handle = NULL;
Signature · 1 line
2.18.0
author
#16 lore raw

PATCH v3 8/10 media: mtk-vcodec: Add more extra processing for venc_multi_core mode

In reply to Irui Wang

venc_multi_core mode encoding need more venc working buffers, it will break the compatibility if we just add venc_vsi in AP-Kernel but not in firmware, so add more struct definition for it.

---
 .../platform/mtk-vcodec/mtk_vcodec_util.c     |  19 ++
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |   4 +
 .../platform/mtk-vcodec/venc/venc_h264_if.c   | 174 +++++++++++++++---
 3 files changed, 172 insertions(+), 25 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.c +19 -0
index ace78c4b5b9e..059f665afa96 100644
@@ -11,6 +11,7 @@
 
 #include "mtk_vcodec_dec_hw.h"
 #include "mtk_vcodec_drv.h"
+#include "mtk_vcodec_enc_core.h"
 #include "mtk_vcodec_util.h"
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 }
 EXPORT_SYMBOL(mtk_vcodec_get_reg_addr);
 
+void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *ctx,
+					 unsigned int core_id)
+{
+	struct mtk_venc_core_dev *core;
+
+	if (core_id >= MTK_VENC_CORE_MAX) {
+		mtk_v4l2_err("Invalid core_id = %d", core_id);
+		return NULL;
+	}
+
+	core = (struct mtk_venc_core_dev *)ctx->dev->enc_core_dev[core_id];
+	if (!core)
+		return NULL;
+
+	return core->reg_base;
+}
+EXPORT_SYMBOL(mtk_venc_get_core_reg_addr);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 			struct mtk_vcodec_mem *mem)
 {
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +4 -0
index 71956627a0e2..de9e18688842 100644
@@ -50,6 +50,10 @@ struct mtk_vcodec_dev;
 
 void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
 				unsigned int reg_idx);
+
+void __iomem *mtk_venc_get_core_reg_addr(struct mtk_vcodec_ctx *data,
+					 unsigned int core_id);
+
 int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,
 				struct mtk_vcodec_mem *mem);
 void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +149 -25

280 lines not on this page · raw message

Signature · 1 line
2.18.0
author
#17 lore raw

PATCH v3 9/10 media: mtk-vcodec: Add venc_multi_core mode encode process

In reply to Irui Wang

The venc_multi_core mode encoding is tries to uses all available cores. MT8195 has two venc cores:
frame#0 uses core#0, frame#1 uses core#1, frame#2 uses core#0...,

Lock the device and enable the clock by used core, for sequence header encoding, it always uses core#0.

---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      | 12 ++-
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 23 +++---
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  4 +-
 .../platform/mtk-vcodec/mtk_vcodec_enc_drv.c  |  7 +-
 .../platform/mtk-vcodec/venc/venc_h264_if.c   | 16 ++--
 .../platform/mtk-vcodec/venc/venc_vp8_if.c    |  3 +-
 .../media/platform/mtk-vcodec/venc_drv_if.c   | 73 ++++++++++++++-----
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  5 ++
 .../media/platform/mtk-vcodec/venc_vpu_if.c   | 11 ++-
 .../media/platform/mtk-vcodec/venc_vpu_if.h   |  3 +-
 10 files changed, 115 insertions(+), 42 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +11 -1
index 43e1349d6a54..889ce61f0fc8 100644
@@ -297,6 +297,9 @@ struct vdec_pic_info {
  * @max_width: hardware supported max width
  * @max_height: hardware supported max height
  * @msg_queue: msg queue used to store lat buffer information.
+ * @q_mutex: src & dst vb2_queue mutex
+ * @enc_idx: used to record encoded frame count
+ * @core_id: used to record used core
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -345,6 +348,11 @@ struct mtk_vcodec_ctx {
 	unsigned int max_width;
 	unsigned int max_height;
 	struct vdec_msg_queue msg_queue;
+
+	/* src & dst vb2_queue mutex */
+	struct mutex q_mutex;
+	int enc_idx;
+	int core_id;
 };
 
 /*
@@ -520,7 +528,9 @@ struct mtk_vcodec_dev {
 
 	/* decoder hardware mutex lock */
 	struct mutex dec_mutex[MTK_VDEC_HW_MAX];
-	struct mutex enc_mutex;
+
+	/* encoder core mutex lock */
+	struct mutex enc_mutex[MTK_VENC_CORE_MAX];
 
 	struct mtk_vcodec_pm pm;
 	unsigned int dec_capability;
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +13 -10
index b0bc71f00930..2753f004f123 100644
@@ -1197,14 +1197,17 @@ static void mtk_venc_worker(struct work_struct *work)
 		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, 0);
 		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_ERROR);
 		mtk_v4l2_err("venc_if_encode failed=%d", ret);
-	} else {
+	} else if (!ctx->dev->venc_multi_core) {
 		v4l2_m2m_buf_done(src_buf, VB2_BUF_STATE_DONE);
-		vb2_set_plane_payload(&dst_buf->vb2_buf, 0, enc_result.bs_size);
+		vb2_set_plane_payload(&dst_buf->vb2_buf, 0,
+				      enc_result.bs_size);
 		v4l2_m2m_buf_done(dst_buf, VB2_BUF_STATE_DONE);
 		mtk_v4l2_debug(2, "venc_if_encode bs size=%d",
-				 enc_result.bs_size);
+			       enc_result.bs_size);
 	}
 
+	ctx->enc_idx++;
+
 	v4l2_m2m_job_finish(ctx->dev->m2m_dev_enc, ctx->m2m_ctx);
 
 	mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
@@ -1258,7 +1261,7 @@ void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx)
 {
 	struct mtk_q_data *q_data;
 
-	ctx->m2m_ctx->q_lock = &ctx->dev->dev_mutex;
+	ctx->m2m_ctx->q_lock = &ctx->q_mutex;
 	ctx->fh.m2m_ctx = ctx->m2m_ctx;
 	ctx->fh.ctrl_handler = &ctx->ctrl_hdl;
 	INIT_WORK(&ctx->encode_work, mtk_venc_worker);
@@ -1390,7 +1393,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	src_vq->ops		= &mtk_venc_vb2_ops;
 	src_vq->mem_ops		= &vb2_dma_contig_memops;
 	src_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
-	src_vq->lock		= &ctx->dev->dev_mutex;
+	src_vq->lock		= &ctx->q_mutex;
 	src_vq->dev		= &ctx->dev->plat_dev->dev;
 
 	ret = vb2_queue_init(src_vq);
@@ -1404,26 +1407,26 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 	dst_vq->ops		= &mtk_venc_vb2_ops;
 	dst_vq->mem_ops		= &vb2_dma_contig_memops;
 	dst_vq->timestamp_flags = V4L2_BUF_FLAG_TIMESTAMP_COPY;
-	dst_vq->lock		= &ctx->dev->dev_mutex;
+	dst_vq->lock		= &ctx->q_mutex;
 	dst_vq->dev		= &ctx->dev->plat_dev->dev;
 
 	return vb2_queue_init(dst_vq);
 }
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_unlock(&dev->enc_mutex);
+	mutex_unlock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(mtk_venc_unlock);
 
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx)
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id)
 {
 	struct mtk_vcodec_dev *dev = ctx->dev;
 
-	mutex_lock(&dev->enc_mutex);
+	mutex_lock(&dev->enc_mutex[core_id]);
 	return 0;
 }
 EXPORT_SYMBOL_GPL(mtk_venc_lock);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +2 -2
index 513ee7993e34..434d91d36158 100644
@@ -39,8 +39,8 @@ struct mtk_video_enc_buf {
 extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;
 extern const struct v4l2_m2m_ops mtk_venc_m2m_ops;
 
-int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx);
-int mtk_venc_lock(struct mtk_vcodec_ctx *ctx);
+int mtk_venc_unlock(struct mtk_vcodec_ctx *ctx, int core_id);
+int mtk_venc_lock(struct mtk_vcodec_ctx *ctx, int core_id);
 int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 			      struct vb2_queue *dst_vq);
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_drv.c +5 -2
index d2937e62df29..c89d3a20eb42 100644
@@ -132,6 +132,7 @@ static int fops_vcodec_open(struct file *file)
 	INIT_LIST_HEAD(&ctx->list);
 	ctx->dev = dev;
 	init_waitqueue_head(&ctx->queue[0]);
+	mutex_init(&ctx->q_mutex);
 
 	ctx->type = MTK_INST_ENCODER;
 	ret = mtk_vcodec_enc_ctrls_setup(ctx);
@@ -239,7 +240,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 	struct video_device *vfd_enc;
 	phandle rproc_phandle;
 	enum mtk_vcodec_fw_type fw_type;
-	int ret, core_type;
+	int ret, core_type, i;
 
 	dev = devm_kzalloc(&pdev->dev, sizeof(*dev), GFP_KERNEL);
 	if (!dev)
@@ -309,7 +310,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev)
 		}
 	}
 
-	mutex_init(&dev->enc_mutex);
+	for (i = 0; i < MTK_VENC_CORE_MAX; i++)
+		mutex_init(&dev->enc_mutex[i]);
+
 	mutex_init(&dev->dev_mutex);
 	spin_lock_init(&dev->irqlock);
 
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +10 -6
index aee31cb836c6..4bfaaf2ec91c 100644
@@ -446,7 +446,8 @@ static int h264_encode_sps(struct venc_h264_inst *inst,
 
 	mtk_vcodec_debug_enter(inst);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS, NULL, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_SPS,
+			     NULL, bs_buf, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -472,7 +473,8 @@ static int h264_encode_pps(struct venc_h264_inst *inst,
 
 	mtk_vcodec_debug_enter(inst);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS, NULL, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_PPS,
+			     NULL, bs_buf, NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
@@ -514,7 +516,8 @@ static int h264_encode_header(struct venc_h264_inst *inst,
 static int h264_encode_frame(struct venc_h264_inst *inst,
 			     struct venc_frm_buf *frm_buf,
 			     struct mtk_vcodec_mem *bs_buf,
-			     unsigned int *bs_size)
+			     unsigned int *bs_size,
+			     int core_id)
 {
 	int ret = 0;
 	unsigned int irq_status;
@@ -533,7 +536,8 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 	mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",
 			 frame_info.frm_count, frame_info.skip_frm_count,
 			 frame_info.frm_type);
-	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME, frm_buf, bs_buf, &frame_info);
+	ret = vpu_enc_encode(&inst->vpu_inst, H264_BS_MODE_FRAME,
+			     frm_buf, bs_buf, &frame_info, core_id);
 	if (ret)
 		return ret;
 
@@ -660,7 +664,7 @@ static int h264_enc_encode(void *handle,
 
 		if (!inst->prepend_hdr) {
 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
-						&result->bs_size);
+						&result->bs_size, ctx->core_id);
 			if (ret)
 				goto encode_err;
 			result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -688,7 +692,7 @@ static int h264_enc_encode(void *handle,
 		tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz);
 
 		ret = h264_encode_frame(inst, frm_buf, &tmp_bs_buf,
-					&bs_size_frm);
+					&bs_size_frm, ctx->core_id);
 		if (ret)
 			goto encode_err;
 
drivers/media/platform/mtk-vcodec/venc/venc_vp8_if.c +2 -1
index 56ce58f761f1..3b3471d90e21 100644
@@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,
 
 	mtk_vcodec_debug(inst, "->frm_cnt=%d", inst->frm_cnt);
 
-	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf, NULL);
+	ret = vpu_enc_encode(&inst->vpu_inst, 0, frm_buf, bs_buf,
+			     NULL, MTK_VENC_CORE0);
 	if (ret)
 		return ret;
 
drivers/media/platform/mtk-vcodec/venc_drv_if.c +55 -18

114 lines not on this page · raw message

drivers/media/platform/mtk-vcodec/venc_drv_if.h +5 -0
index 0b04a1020873..15e9a2ab9cda 100644
@@ -167,4 +167,9 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,
 		   struct mtk_vcodec_mem *bs_buf,
 		   struct venc_done_result *result);
 
+void venc_encode_prepare(struct mtk_vcodec_ctx *ctx,
+			 enum venc_start_opt opt);
+void venc_encode_unprepare(struct mtk_vcodec_ctx *ctx,
+			   enum venc_start_opt opt);
+
 #endif /* _VENC_DRV_IF_H_ */
drivers/media/platform/mtk-vcodec/venc_vpu_if.c +10 -1
index d3570c4c177d..0a672ada8928 100644
@@ -225,9 +225,12 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   struct venc_frame_info *frame_info)
+		   struct venc_frame_info *frame_info,
+		   int core_id)
 {
 	const bool is_ext = MTK_ENC_CTX_IS_EXT(vpu->ctx);
+	const bool is_multi_core = vpu->ctx->dev->venc_multi_core;
+
 	size_t msg_size = is_ext ?
 		sizeof(struct venc_ap_ipi_msg_enc_ext) :
 		sizeof(struct venc_ap_ipi_msg_enc);
@@ -261,6 +264,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		out.data[1] = frame_info->skip_frm_count;
 		out.data[2] = frame_info->frm_type;
 	}
+
+	if (is_multi_core) {
+		out.data_item = 4;
+		out.data[3] = core_id;
+	}
+
 	if (vpu_enc_send_msg(vpu, &out, msg_size)) {
 		mtk_vcodec_err(vpu, "AP_IPIMSG_ENC_ENCODE %d fail",
 			       bs_mode);
drivers/media/platform/mtk-vcodec/venc_vpu_if.h +2 -1
index f83bc1b3f2bf..a8055eb62f7c 100644
@@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,
 int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode,
 		   struct venc_frm_buf *frm_buf,
 		   struct mtk_vcodec_mem *bs_buf,
-		   struct venc_frame_info *frame_info);
+		   struct venc_frame_info *frame_info,
+		   int core_id);
 int vpu_enc_deinit(struct venc_vpu_inst *vpu);
 
 #endif
Signature · 1 line
2.18.0
author
#18 lore raw

PATCH v3 10/10 media: mtk-vcodec: Done encode result to client

In reply to Irui Wang

When enable venc_multi_core mode encoding, the wait IRQ done synchronous function should not be called, so the encode result can't return to client in device_run. device_run called to trigger encoding process, finish it, so another scheduled job can be ran. When encoding done, done the encode result to client in IRQ handler.

---
 .../platform/mtk-vcodec/mtk_vcodec_drv.h      |  6 ++
 .../platform/mtk-vcodec/mtk_vcodec_enc.c      | 68 +++++++++++++++++--
 .../platform/mtk-vcodec/mtk_vcodec_enc.h      |  3 +
 .../platform/mtk-vcodec/mtk_vcodec_enc_core.c | 25 ++++++-
 .../platform/mtk-vcodec/mtk_vcodec_util.h     |  1 +
 .../platform/mtk-vcodec/venc/venc_h264_if.c   | 22 +++++-
 .../media/platform/mtk-vcodec/venc_drv_if.h   |  2 +
 7 files changed, 120 insertions(+), 7 deletions(-)
drivers/media/platform/mtk-vcodec/mtk_vcodec_drv.h +6 -0
index 889ce61f0fc8..0317707b2ed3 100644
@@ -300,6 +300,9 @@ struct vdec_pic_info {
  * @q_mutex: src & dst vb2_queue mutex
  * @enc_idx: used to record encoded frame count
  * @core_id: used to record used core
+ * @pfrm_buf: used to store current ctx's frame buffer
+ * @pbs_buf: used to store current ctx's bitstream buffer
+ * @hdr_size: used to store prepend header size
  */
 struct mtk_vcodec_ctx {
 	enum mtk_instance_type type;
@@ -353,6 +356,9 @@ struct mtk_vcodec_ctx {
 	struct mutex q_mutex;
 	int enc_idx;
 	int core_id;
+	struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_CORE_MAX];
+	struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_CORE_MAX];
+	unsigned int hdr_size;
 };
 
 /*
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.c +64 -4

130 lines not on this page · raw message

drivers/media/platform/mtk-vcodec/mtk_vcodec_enc.h +3 -0
index 434d91d36158..1e22c77a2760 100644
@@ -46,5 +46,8 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,
 void mtk_vcodec_enc_release(struct mtk_vcodec_ctx *ctx);
 int mtk_vcodec_enc_ctrls_setup(struct mtk_vcodec_ctx *ctx);
 void mtk_vcodec_enc_set_default_params(struct mtk_vcodec_ctx *ctx);
+void mtk_venc_buf_done(struct mtk_vcodec_ctx *ctx, int core_id,
+		       unsigned int bs_size, bool time_out, bool key_frame);
+void mtk_venc_lock_all(struct mtk_vcodec_ctx *ctx);
 
 #endif /* _MTK_VCODEC_ENC_H_ */
drivers/media/platform/mtk-vcodec/mtk_vcodec_enc_core.c +24 -1
index 7e376413beaa..c295b5d0b877 100644
@@ -16,6 +16,10 @@
 #include "mtk_vcodec_enc_pm.h"
 #include "mtk_vcodec_intr.h"
 
+#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098
+#define VENC_PIC_FRM_TYPE      0x0010
+#define VENC_PIC_KEY_FRM       0x2
+
 static const struct of_device_id mtk_venc_core_ids[] = {
 	{
 		.compatible = "mediatek,mtk-venc-core",
@@ -51,6 +55,9 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv)
 	struct mtk_vcodec_ctx *ctx;
 	unsigned long flags;
 	void __iomem *addr;
+	unsigned int bs_size;
+	unsigned int frm_type;
+	bool is_key_frame = 0;
 
 	spin_lock_irqsave(&core->main_dev->irqlock, flags);
 	ctx = core->curr_ctx;
@@ -62,9 +69,25 @@ static irqreturn_t mtk_enc_core_irq_handler(int irq, void *priv)
 
 	addr = core->reg_base + MTK_VENC_IRQ_ACK_OFFSET;
 	ctx->irq_status = readl(core->reg_base + MTK_VENC_IRQ_STATUS_OFFSET);
+	bs_size = readl(core->reg_base + VENC_PIC_BITSTREAM_BYTE_CNT);
+	frm_type = readl(core->reg_base + VENC_PIC_FRM_TYPE);
 	clean_irq_status(ctx->irq_status, addr);
 
-	wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+	if (ctx->irq_status & MTK_VENC_IRQ_STATUS_FRM) {
+		if (ctx->hdr_size != 0) {
+			bs_size += ctx->hdr_size;
+			ctx->hdr_size = 0;
+		}
+		if (frm_type & VENC_PIC_KEY_FRM)
+			is_key_frame = 1;
+
+		mtk_venc_buf_done(ctx, core->core_id, bs_size, 0, is_key_frame);
+		mtk_vcodec_enc_clock_off(core->main_dev, core->core_id);
+		mtk_venc_unlock(ctx, core->core_id);
+	} else {
+		wake_up_ctx(ctx, MTK_INST_IRQ_RECEIVED, 0);
+	}
+
 	return IRQ_HANDLED;
 }
 
drivers/media/platform/mtk-vcodec/mtk_vcodec_util.h +1 -0
index de9e18688842..4436d8bc60a0 100644
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {
 	size_t size;
 	void *va;
 	dma_addr_t dma_addr;
+	void *buf;
 };
 
 struct mtk_vcodec_fb {
drivers/media/platform/mtk-vcodec/venc/venc_h264_if.c +20 -2
index 4bfaaf2ec91c..b54c3c7ccab8 100644
@@ -555,6 +555,12 @@ static int h264_encode_frame(struct venc_h264_inst *inst,
 		return ret;
 	}
 
+	if (ctx->dev->venc_multi_core) {
+		++inst->frm_cnt;
+		mtk_vcodec_debug(inst, "multi core : frm %d <-", inst->frm_cnt);
+		return ret;
+	}
+
 	irq_status = h264_enc_wait_venc_done(inst);
 	if (irq_status != MTK_VENC_IRQ_STATUS_FRM) {
 		mtk_vcodec_err(inst, "irq_status=%d failed", irq_status);
@@ -635,10 +641,12 @@ static int h264_enc_encode(void *handle,
 	int ret = 0;
 	struct venc_h264_inst *inst = (struct venc_h264_inst *)handle;
 	struct mtk_vcodec_ctx *ctx = inst->ctx;
+	bool is_single_core = !ctx->dev->venc_multi_core;
 
 	mtk_vcodec_debug(inst, "opt %d ->", opt);
 
-	enable_irq(ctx->dev->enc_irq);
+	if (is_single_core)
+		enable_irq(ctx->dev->enc_irq);
 
 	switch (opt) {
 	case VENC_START_OPT_ENCODE_SEQUENCE_HEADER: {
@@ -662,6 +670,13 @@ static int h264_enc_encode(void *handle,
 		unsigned int bs_size_hdr;
 		unsigned int bs_size_frm;
 
+		/*
+		 * the frm_buf and bs_buf need to be recorded into ctx,
+		 * when encoding done, the target buffer can be obtained
+		 */
+		ctx->pfrm_buf[ctx->core_id] = frm_buf->src_addr;
+		ctx->pbs_buf[ctx->core_id] = bs_buf->buf;
+
 		if (!inst->prepend_hdr) {
 			ret = h264_encode_frame(inst, frm_buf, bs_buf,
 						&result->bs_size, ctx->core_id);
@@ -698,6 +713,8 @@ static int h264_enc_encode(void *handle,
 
 		result->bs_size = hdr_sz + filler_sz + bs_size_frm;
 
+		ctx->hdr_size = hdr_sz + filler_sz;
+
 		mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d",
 				 hdr_sz, filler_sz, bs_size_frm,
 				 result->bs_size);
@@ -714,8 +731,9 @@ static int h264_enc_encode(void *handle,
 	}
 
 encode_err:
+	if (is_single_core)
+		disable_irq(ctx->dev->enc_irq);
 
-	disable_irq(ctx->dev->enc_irq);
 	mtk_vcodec_debug(inst, "opt %d <-", opt);
 
 	return ret;
drivers/media/platform/mtk-vcodec/venc_drv_if.h +2 -0
index 15e9a2ab9cda..3ad31abb7290 100644
@@ -108,9 +108,11 @@ struct venc_frame_info {
 /*
  * struct venc_frm_buf - frame buffer information used in venc_if_encode()
  * @fb_addr: plane frame buffer addresses
+ * @src_addr: vb2_v4l2_buffer src buffer address
  */
 struct venc_frm_buf {
 	struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES];
+	void *src_addr;
 };
 
 /*
Signature · 1 line
2.18.0
#19 lore raw
In reply to Irui Wang

Il 17/03/22 09:22, Irui Wang ha scritto:

Quoted 25 lines

MT8195 has two H264 encoder cores, they have their own power-domains, clocks, interrupts, register base. The two H264 encoder cores can work together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve 4K@60fps.
The two encoder core encoding process looks like this:

     VENC Core0: frm#0....frm#2....frm#4....
     VENC Core1: ..frm#1....frm#3....frm#5....

This series of patches are used to enable the two H264 encoder cores, encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ --> encoding done with result --> job_finish
...

To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.

Quoted 4 lines

There is no "wait encoder IRQ" synchronous call during frame-racing mode encoding process, it can full use the two encoder cores to achieve higher performance.

Hello Irui,
like v1 and v2, also this version of the series will crash the kernel when trying to encode.

Before pushing the next series, you shall ensure that it is not crashing the entire kernel, hence, please test your code with gstreamer.

Please use the following pipeline to test:

gst-launch-1.0 videotestsrc num-buffers=100 ! v4l2h264enc ! queue ! h264parse ! qtmux ! filesink location=test.mov

Regards,
Angelo

author
#20 lore raw

Dear Angelo,

Thanks for your comments.

Quoted 40 lines · AngeloGioacchino Del Regno

Il 17/03/22 09:22, Irui Wang ha scritto:

Quoted 32 lines

MT8195 has two H264 encoder cores, they have their own power- domains,
clocks, interrupts, register base. The two H264 encoder cores can work
together to achieve higher performance, it's a core mode called frame-racing, one core has 4K@30fps performance, two cores can achieve
4K@60fps.
The two encoder core encoding process looks like this:

     VENC Core0: frm#0....frm#2....frm#4....
     VENC Core1: ..frm#1....frm#3....frm#5....

This series of patches are used to enable the two H264 encoder cores,
encoding process will be changed:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
encoding done with result --> job_finish V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
encoding done with result --> job_finish
...

To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client)
V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.

Quoted 6 lines

There is no "wait encoder IRQ" synchronous call during frame-racing mode
encoding process, it can full use the two encoder cores to achieve higher
performance.

Quoted 4 lines

Hello Irui,
like v1 and v2, also this version of the series will crash the kernel when
trying to encode.

As mentioned before, dual core encoding need more encoding working buffers in patch-8:
media: mtk-vcodec: Add more extra processing for venc_multi_core mode a new structure, struct venc_multi_core_vsi defined for it,

encoding processing like this:
kernel --> ipi message --> scp firmware(userspace), vsi is a share memory to transfer data between kernel and user, so, the definition of vsi must be the same in kernel and userspace. otherwise, errors will be occurred, for example, kernel crash.

When we enable dual core encoding in this patch, the venc_multi_core_vsi used in kernel, but scp(userspace) still use the older one vsi(struct venc_h264_vsi), the crash happened.

Maybe we would find out how to resolve the limitations, many thanks for your review.

Quoted 8 lines

Before pushing the next series, you shall ensure that it is not crashing
the entire kernel, hence, please test your code with gstreamer.

Please use the following pipeline to test:

Quoted 7 lines

gst-launch-1.0 videotestsrc num-buffers=100 ! v4l2h264enc ! queue ! h264parse !
qtmux ! filesink location=test.mov

Regards,
Angelo

Thanks
Best Regards