Support H264 multi-core encoder on MT8195
Subsystems (4)
v4 cover Support H264 multi-core encoder on MT8195
MT8195 has two H264 encoder hardware, named core0 and core1, this two cores can encode two input frames separately at the same time to achieve higher performance.
This series of patches are used to enable the two H264 encoder cores, the difference between encoding process before and after enable two cores is just like as below:
As-Is: Synchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> wait encoder IRQ -->
encoding done with result --> job_finish
V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> wait encoder IRQ -->
encoding done with result --> job_finish
...
To-Be: Asynchronous
V4L2_VIDIOC_QBUF#0 --> device_run(triger encoder) --> job_finish ..V4l2_VIDIOC_QBUF#1 --> device_run(triger encoder) --> job_finish (venc core0 may encode done here, done the encoding result to client) V4L2_VIDIOC_QBUF#2 --> device_run(triger encoder) --> job_finish.
---
changes compared with v3:
- rebase to the newer linux media stage.
- add a capability to indicate scp firmware support multi-core.
- probe core0 as main device, core1 as sub-device.
changes compared with v2:
- update venc core dt-bindings, add two new properties for current usage.
- parse venc multi_core mode from device tree.
- rebase to the newer linux media stage.
changes compared with v1:
- of_platform_populate was used in place of the component framework.
- new yaml file for venc cores.
- some modifications for patch v1's review comments.
---
Irui Wang (8):
dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings
for mt8195
media: mediatek: vcodec: Enable venc dual core usage
media: mediatek: vcodec: Refactor venc power manage function
media: mediatek: vcodec: Add more extra processing for multi-core
encoding
media: mediatek: vcodec: Add venc power on/off function
media: mediatek: vcodec: Refactor encoder clock on/off function
media: mediatek: vcodec: Add multi-core encoding process
media: mediatek: vcodec: Return encoding result in asynchronous mode
.../media/mediatek,vcodec-encoder-core.yaml | 225 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - .../media/platform/mediatek/vcodec/Makefile | 4 +- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 28 ++- .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 113 ++++++--- .../platform/mediatek/vcodec/mtk_vcodec_enc.h | 11 +- .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 48 +++- .../mediatek/vcodec/mtk_vcodec_enc_hw.c | 159 +++++++++++++ .../mediatek/vcodec/mtk_vcodec_enc_hw.h | 34 +++ .../mediatek/vcodec/mtk_vcodec_enc_pm.c | 177 ++++++++++++-- .../mediatek/vcodec/mtk_vcodec_enc_pm.h | 9 +- .../mediatek/vcodec/mtk_vcodec_util.c | 19 ++ .../mediatek/vcodec/mtk_vcodec_util.h | 3 + .../mediatek/vcodec/venc/venc_h264_if.c | 207 +++++++++++++--- .../mediatek/vcodec/venc/venc_vp8_if.c | 3 +- .../platform/mediatek/vcodec/venc_drv_if.c | 75 ++++-- .../platform/mediatek/vcodec/venc_drv_if.h | 6 + .../platform/mediatek/vcodec/venc_vpu_if.c | 9 +- .../platform/mediatek/vcodec/venc_vpu_if.h | 3 +- 19 files changed, 1015 insertions(+), 119 deletions(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
Signature · 1 line
2.18.0
PATCH v4 1/8 dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings for mt8195
mt8195 has two H264 encoder hardware, which are named core0 and core1. The two encoder cores are independent, we can just enable one core to do encoding or enable both of them to achieve higher performance. We pick core0 as main device and core1 as its subdevice, it just a way to to manage the two encoder hardware, because they are two equal encoder hardware with the same function.
--- .../media/mediatek,vcodec-encoder-core.yaml | 225 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 225 insertions(+), 1 deletion(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +225 -0
@@ -0,0 +1,225 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + MediaTek Video Encoder is the video encoder hardware present in MediaTek + SoCs which supports high resolution encoding functionalities. To meet higher + encoder performance, there will be one or more encoder hardware inside SoC, + which named core0, core1, etc.. For example, mt8195 has two encoder hardware, + the two encoder cores block diagram, can check below. + -------------------------------------------------------------- + Input frame 0 1 2 3 4 5 6 + | | | | | | | + v | v | v | v + +-------+ | +-------+ | +-------+ | +-------+ + | core0 | | | core0 | | | core0 | | | core0 | + +-------+ | +-------+ | +-------+ | +-------+ + | | | | | | | + | v | v | v | + | +-------+ | +-------+ | +-------+ | + | | core1 | | | core1 | | | core1 | | + | +-------+ | +-------+ | +-------+ | + | | | | | | | + v v v v v v v <parent> + -------------------------------------------------------------- + core || index <child> + \/ + +--------------------------------------------------+ + | core0/core1 | + | enable/disable power/clk/irq | + +--------------------------------------------------+ + -------------------------------------------------------------- + As above, there are two cores child devices, they are two encoder hardware + which can encode input frames in order. When start encoding, input frame 0 + will be encoded by core0, and input frame 1 can be encoded by core1 even if + frame 0 has not been encoded done yet, after frame 0 encoded done, frame 2 + will be encoded by core0, even input frames are encoded by core0 and odd + input frames are encoded by core1, these two encoder cores encode ench input + frames in this overlapping manner. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc + + reg: + maxItems: 1 + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + mediatek,venc-multi-core: + type: boolean + description: | + Indicates whether the encoder has multiple cores or not. We use this + to probe additional encoder cores device. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps to memory. + + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be added as subnodes to + the main venc node, it represents a encoder hardware. + + properties: + compatible: + items: + - const: mediatek,mtk-venc-hw + + reg: + maxItems: 1 + + mediatek,hw-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id. We use it to pick which one encoder core + will be used to encoding current input frame. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - mediatek,hw-id + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - reg + - mediatek,scp + - iommus + - interrupts + - clocks + - clock-names + - dma-ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + soc { + #address-cells = <2>; + #size-cells = <2>; + + venc: venc@1a020000 { + compatible = "mediatek,mt8195-vcodec-enc"; + reg = <0 0x1a020000 0 0x10000>; + mediatek,scp = <&scp>; + mediatek,venc-multi-core; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "clk_venc"; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + + venc-core@1b020000 { + compatible = "mediatek,mtk-venc-hw"; + reg = <0 0x1b020000 0 0x10000>; + mediatek,hw-id = <1>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "clk_venc_core1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + }; + };
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml +0 -1
@@ -21,7 +21,6 @@ properties: - mediatek,mt8173-vcodec-enc - mediatek,mt8183-vcodec-enc - mediatek,mt8192-vcodec-enc - - mediatek,mt8195-vcodec-enc reg: maxItems: 1
Signature · 1 line
2.18.0
Rob Herring
maintainerRe: [PATCH v4, 1/8] dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings for mt8195
Quoted 6 lines · Irui Wang
mt8195 has two H264 encoder hardware, which are named core0 and core1. The two encoder cores are independent, we can just enable one core to do encoding or enable both of them to achieve higher performance. We pick core0 as main device and core1 as its subdevice, it just a way to to manage the two encoder hardware, because they are two equal encoder hardware with the same function.
If the h/w is symmetrical, why did you do this?
Quoted 68 lines
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 225 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 225 insertions(+), 1 deletion(-) create mode 100644 Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +54
-0@@ -0,0 +1,225 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#" +$schema: "http://devicetree.org/meta-schemas/core.yaml#" + +title: MediaTek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + MediaTek Video Encoder is the video encoder hardware present in MediaTek + SoCs which supports high resolution encoding functionalities. To meet higher + encoder performance, there will be one or more encoder hardware inside SoC, + which named core0, core1, etc.. For example, mt8195 has two encoder hardware, + the two encoder cores block diagram, can check below. + -------------------------------------------------------------- + Input frame 0 1 2 3 4 5 6 + | | | | | | | + v | v | v | v + +-------+ | +-------+ | +-------+ | +-------+ + | core0 | | | core0 | | | core0 | | | core0 | + +-------+ | +-------+ | +-------+ | +-------+ + | | | | | | | + | v | v | v | + | +-------+ | +-------+ | +-------+ | + | | core1 | | | core1 | | | core1 | | + | +-------+ | +-------+ | +-------+ | + | | | | | | | + v v v v v v v <parent> + -------------------------------------------------------------- + core || index <child> + \/ + +--------------------------------------------------+ + | core0/core1 | + | enable/disable power/clk/irq | + +--------------------------------------------------+ + -------------------------------------------------------------- + As above, there are two cores child devices, they are two encoder hardware + which can encode input frames in order. When start encoding, input frame 0 + will be encoded by core0, and input frame 1 can be encoded by core1 even if + frame 0 has not been encoded done yet, after frame 0 encoded done, frame 2 + will be encoded by core0, even input frames are encoded by core0 and odd + input frames are encoded by core1, these two encoder cores encode ench input + frames in this overlapping manner. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc +
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 64
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ reg: + maxItems: 1 + + mediatek,scp: + $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + mediatek,venc-multi-core:
Can't you detect this with presence of child nodes?
Quoted 52 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +52
-0+ type: boolean + description: | + Indicates whether the encoder has multiple cores or not. We use this + to probe additional encoder cores device. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps to memory. + + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be added as subnodes to + the main venc node, it represents a encoder hardware. + + properties: + compatible: + items: + - const: mediatek,mtk-venc-hw + + reg: + maxItems: 1
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml line 126
Quoted 10 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +10
-0+ + mediatek,hw-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id. We use it to pick which one encoder core + will be used to encoding current input frame. + + iommus: + minItems: 1 + maxItems: 32
It looks like there are 9 iommus. How does this vary on an mt8195 SoC?
Quoted 115 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml new file +99
-0+ description: | + List of the hardware port in respective IOMMU block for current Socs. + Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - mediatek,hw-id + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - reg + - mediatek,scp + - iommus + - interrupts + - clocks + - clock-names + - dma-ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + soc { + #address-cells = <2>; + #size-cells = <2>; + + venc: venc@1a020000 { + compatible = "mediatek,mt8195-vcodec-enc"; + reg = <0 0x1a020000 0 0x10000>; + mediatek,scp = <&scp>; + mediatek,venc-multi-core; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "clk_venc"; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + + venc-core@1b020000 { + compatible = "mediatek,mtk-venc-hw"; + reg = <0 0x1b020000 0 0x10000>; + mediatek,hw-id = <1>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_CHROMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp M4U_PORT_L20_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "clk_venc_core1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen CLK_TOP_UNIVPLL_D4>; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC_CORE1>; + }; + }; + };Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml +0
-1@@ -21,7 +21,6 @@ properties: - mediatek,mt8173-vcodec-enc - mediatek,mt8183-vcodec-enc - mediatek,mt8192-vcodec-enc - - mediatek,mt8195-vcodec-enc reg: maxItems: 1Signature · 1 line
2.18.0
Re: [PATCH v4, 1/8] dt-bindings: media: mediatek: vcodec: Adds encoder cores dt-bindings for mt8195
Dear Rob,
Thank you for reviewing.
Quoted 13 lines · Rob Herring
Quoted 11 lines · Irui Wang
mt8195 has two H264 encoder hardware, which are named core0 and core1.
The two encoder cores are independent, we can just enable one core to
do encoding or enable both of them to achieve higher performance. We
pick core0 as main device and core1 as its subdevice, it just a way to
to manage the two encoder hardware, because they are two equal encoder
hardware with the same function.
Quoted 1 line
If the h/w is symmetrical, why did you do this?
We want to register the two cores into one V4L2 device, and use core0 by default, as for core1, we can enable it when the firmware supports. I think it should be one way to manage the two encoder hardware, and we can still use core0 normally event if the firmware doesn't support multi-core encode.
Quoted 93 lines
Quoted 93 lines
Signed-off-by Irui Wang <[email protected]>--- .../media/mediatek,vcodec-encoder-core.yaml | 225 ++++++++++++++++++ .../media/mediatek,vcodec-encoder.yaml | 1 - 2 files changed, 225 insertions(+), 1 deletion(-) create mode 100644
Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yamldiff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-
core.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder-core.yaml
new file mode 100644
index 000000000000..afd7d645aa80Documentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-0
encoder-core.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0@@ -0,0 +1,225 @@ +# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause) + +%YAML 1.2 +--- +$id: "http://devicetree.org/schemas/media/mediatek,vcodec-encoder-core.yaml#
"
+$schema: "http://devicetree.org/meta-schemas/core.yaml#"Documentation/devicetree/bindings/media/mediatek,vcodec- new file +8
-0+ +title: MediaTek Video Encoder Accelerator With Multi Core + +maintainers: + - Irui Wang <[email protected]> + +description: | + MediaTek Video Encoder is the video encoder hardware present inMediaTek
+ SoCs which supports high resolution encoding functionalities. To meet higher
+ encoder performance, there will be one or more encoder hardware inside SoC,
+ which named core0, core1, etc.. For example, mt8195 has two encoder hardware,Documentation/devicetree/bindings/media/mediatek,vcodec- new file +24
-0+ the two encoder cores block diagram, can check below. + -------------------------------------------------------------- + Input frame 0 1 2 3 4 5 6 + | | | | | | | + v | v | v | v + +-------+ | +-------+ | +-------+ | +-------+ + | core0 | | | core0 | | | core0 | | | core0 | + +-------+ | +-------+ | +-------+ | +-------+ + | | | | | | | + | v | v | v | + | +-------+ | +-------+ | +-------+ | + | | core1 | | | core1 | | | core1 | | + | +-------+ | +-------+ | +-------+ | + | | | | | | | + v v v v v v v <parent> + -------------------------------------------------------------- + core || index <child> + \/ + +--------------------------------------------------+ + | core0/core1 | + | enable/disable power/clk/irq | + +--------------------------------------------------+ + -------------------------------------------------------------- + As above, there are two cores child devices, they are twoencoder hardware
+ which can encode input frames in order. When start encoding, input frame 0
+ will be encoded by core0, and input frame 1 can be encoded by core1 even if
+ frame 0 has not been encoded done yet, after frame 0 encoded done, frame 2
+ will be encoded by core0, even input frames are encoded by core0 and odd
+ input frames are encoded by core1, these two encoder cores encode ench inputDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +12
-0+ frames in this overlapping manner. + +properties: + compatible: + items: + - enum: + - mediatek,mt8195-vcodec-enc + + reg: + maxItems: 1 + + mediatek,scp:
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 6 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +6
-0+ $ref: /schemas/types.yaml#/definitions/phandle + description: | + The node of system control processor (SCP), using + the remoteproc & rpmsg framework. + + mediatek,venc-multi-core:Can't you detect this with presence of child nodes?
We plan to delete the property in next version.
Quoted 63 lines
Quoted 63 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ type: boolean + description: | + Indicates whether the encoder has multiple cores or not. Weuse this
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +7
-0+ to probe additional encoder cores device. + + iommus: + minItems: 1 + maxItems: 32 + description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +18
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + dma-ranges: + maxItems: 1 + description: | + Describes the physical address space of IOMMU maps tomemory.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +15
-0+ + "#address-cells": + const: 2 + + "#size-cells": + const: 2 + + ranges: true + +# Required child node: +patternProperties: + "^venc-core@[0-9a-f]+$": + type: object + description: | + The video encoder core device node which should be added assubnodes to
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +14
-0+ the main venc node, it represents a encoder hardware. + + properties: + compatible: + items: + - const: mediatek,mtk-venc-hw + + reg: + maxItems: 1 + + mediatek,hw-id: + $ref: /schemas/types.yaml#/definitions/uint32 + description: | + Current encoder core id. We use it to pick which oneencoder core
Quoted 8 lines
Documentation/devicetree/bindings/media/mediatek,vcodec-
Quoted 5 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0+ will be used to encoding current input frame. + + iommus: + minItems: 1 + maxItems: 32It looks like there are 9 iommus. How does this vary on an mt8195 SoC?
Currently, mt8195 defines total 27 iommus for Larb19, and now we can use 9 iommus of them.
Thanks
Best Regards
Quoted 127 lines
Quoted 127 lines
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ description: | + List of the hardware port in respective IOMMU block forcurrent Socs.
Documentation/devicetree/bindings/media/mediatek,vcodec- new file +86
-0+ Refer to bindings/iommu/mediatek,iommu.yaml. + + interrupts: + maxItems: 1 + + clocks: + maxItems: 1 + + clock-names: + maxItems: 1 + + power-domains: + maxItems: 1 + + required: + - compatible + - reg + - mediatek,hw-id + - iommus + - interrupts + - clocks + - clock-names + - assigned-clocks + - assigned-clock-parents + - power-domains + + additionalProperties: false + +required: + - compatible + - reg + - mediatek,scp + - iommus + - interrupts + - clocks + - clock-names + - dma-ranges + +additionalProperties: false + +examples: + - | + #include <dt-bindings/interrupt-controller/arm-gic.h> + #include <dt-bindings/memory/mt8195-memory-port.h> + #include <dt-bindings/interrupt-controller/irq.h> + #include <dt-bindings/clock/mt8195-clk.h> + #include <dt-bindings/power/mt8195-power.h> + + soc { + #address-cells = <2>; + #size-cells = <2>; + + venc: venc@1a020000 { + compatible = "mediatek,mt8195-vcodec-enc"; + reg = <0 0x1a020000 0 0x10000>; + mediatek,scp = <&scp>; + mediatek,venc-multi-core; + iommus = <&iommu_vdo M4U_PORT_L19_VENC_RCPU>, + <&iommu_vdo M4U_PORT_L19_VENC_REC>, + <&iommu_vdo M4U_PORT_L19_VENC_BSDMA>, + <&iommu_vdo M4U_PORT_L19_VENC_SV_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_RD_COMV>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_CUR_CHROMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_LUMA>, + <&iommu_vdo M4U_PORT_L19_VENC_REF_CHROMA>; + interrupts = <GIC_SPI 341 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys CLK_VENC_VENC>; + clock-names = "clk_venc"; + power-domains = <&spm MT8195_POWER_DOMAIN_VENC>; + dma-ranges = <0x1 0x0 0x0 0x40000000 0x0 0xfff00000>; + #address-cells = <2>; + #size-cells = <2>; + ranges; + + venc-core@1b020000 { + compatible = "mediatek,mtk-venc-hw"; + reg = <0 0x1b020000 0 0x10000>; + mediatek,hw-id = <1>; + iommus = <&iommu_vpp M4U_PORT_L20_VENC_RCPU>, + <&iommu_vpp M4U_PORT_L20_VENC_REC>, + <&iommu_vpp M4U_PORT_L20_VENC_BSDMA>, + <&iommu_vpp M4U_PORT_L20_VENC_SV_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_RD_COMV>, + <&iommu_vpp M4U_PORT_L20_VENC_CUR_LUMA>, + <&iommu_vpp
M4U_PORT_L20_VENC_CUR_CHROMA>,Documentation/devicetree/bindings/media/mediatek,vcodec- new file +2
-0+ <&iommu_vpp M4U_PORT_L20_VENC_REF_LUMA>, + <&iommu_vpp
M4U_PORT_L20_VENC_REF_CHROMA>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +5
-0+ interrupts = <GIC_SPI 346 IRQ_TYPE_LEVEL_HIGH 0>; + clocks = <&vencsys_core1 CLK_VENC_CORE1_VENC>; + clock-names = "clk_venc_core1"; + assigned-clocks = <&topckgen CLK_TOP_VENC>; + assigned-clock-parents = <&topckgen
CLK_TOP_UNIVPLL_D4>;
+ power-domains = <&spmMT8195_POWER_DOMAIN_VENC_CORE1>;Documentation/devicetree/bindings/media/mediatek,vcodec- new file +3
-0+ }; + }; + };diff --git a/Documentation/devicetree/bindings/media/mediatek,vcodec-
encoder.yamlb/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
indexd36fcca04cbc..11682659c4c4100644
--- a/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yaml
+++ b/Documentation/devicetree/bindings/media/mediatek,vcodec-encoder.yamlDocumentation/devicetree/bindings/media/mediatek,vcodec- new file +0
-1@@ -21,7 +21,6 @@ properties: - mediatek,mt8173-vcodec-enc - mediatek,mt8183-vcodec-enc - mediatek,mt8192-vcodec-enc - - mediatek,mt8195-vcodec-enc reg: maxItems: 1Signature · 1 line
2.18.0
PATCH v4 2/8 media: mediatek: vcodec: Enable venc dual core usage
In reply to Irui WangAdds new property to indicate whether the encoder has multiple cores. Use of_platform_populate to probe each venc cores, the core device can use the init_clk/request_irq helper to initialize their own power/clk/irq.
--- .../media/platform/mediatek/vcodec/Makefile | 4 +- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 12 ++ .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 10 ++ .../mediatek/vcodec/mtk_vcodec_enc_hw.c | 139 ++++++++++++++++++ .../mediatek/vcodec/mtk_vcodec_enc_hw.h | 36 +++++ 5 files changed, 200 insertions(+), 1 deletion(-) create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c create mode 100644 drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h
drivers/media/platform/mediatek/vcodec/Makefile +3 -1
@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk_vcodec_enc_hw.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \
@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-hw-y := mtk_vcodec_enc_hw.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \
drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +12 -0
@@ -97,6 +97,15 @@ enum mtk_fmt_type { MTK_FMT_FRAME = 2, }; +/* +* enum mtk_venc_hw_id -- encoder hardware id +*/ +enum mtk_venc_hw_id { +MTK_VENC_CORE_0 =0 , +MTK_VENC_CORE_1 , +MTK_VENC_HW_MAX , +}; +/* * enum mtk_vdec_hw_id - Hardware index used to separate * different hardware
@@ -484,6 +493,7 @@ struct mtk_vcodec_enc_pdata {* @dec_active_cnt: used to mark whether need to record register value * @vdec_racing_info: record register value * @dec_racing_info_mutex: mutex lock used for inner racing mode +* @enc_hw_dev: used to store venc core device */ struct mtk_vcodec_dev {struct v4l2_device v4l2_dev;
@@ -534,6 +544,8 @@ struct mtk_vcodec_dev {u32 vdec_racing_info[132 ];/* Protects access to vdec_racing_info data */ struct mutex dec_racing_info_mutex; + +void *enc_hw_dev[MTK_VENC_HW_MAX ]; };static inline struct mtk_vcodec_ctx *fh_to_ctx (struct v4l2_fh *fh)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +10 -0
@@ -263,6 +263,16 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_enc_pm; } +if (of_property_read_bool (pdev->dev.of_node, +"mediatek,venc-multi-core" )) { + ret =of_platform_populate (pdev->dev.of_node, +NULL ,NULL , &pdev->dev); +if (ret) { +mtk_v4l2_err ("Venc core device populate failed" ); + goto err_enc_pm; + } + } +pm_runtime_enable (&pdev->dev); dev->reg_base[dev->venc_pdata->core_id] =
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +139 -0
139 lines not on this page · raw message
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h new file +36 -0
@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#ifndef _MTK_VCODEC_ENC_HW_H_ +#define _MTK_VCODEC_ENC_HW_H_ + +#include <linux/platform_device.h> +#include "mtk_vcodec_drv.h" + +/** +* struct mtk_venc_hw_dev - driver data +* @plat_dev: platform_device +* @main_dev: main device +* @pm: power management data +* @curr_ctx: the context that is waiting for venc hardware +* @reg_base: mapped address of venc registers +* @irq_status: venc hardware irq status +* @enc_irq: venc device irq +* @hw_id: for venc hardware id: core#0, core#1... +*/ +struct mtk_venc_hw_dev { +struct platform_device *plat_dev; +struct mtk_vcodec_dev *main_dev; + +struct mtk_vcodec_pm pm; +struct mtk_vcodec_ctx *curr_ctx; + +void __iomem *reg_base; +unsigned int irq_status; +int enc_irq; +int hw_id; +}; + +#endif /* _MTK_VCODEC_ENC_HW_H_ */
Signature · 1 line
2.18.0
AngeloGioacchino Del Regno
maintainerRe: [PATCH v4, 2/8] media: mediatek: vcodec: Enable venc dual core usage
Il 24/06/22 10:23, Irui Wang ha scritto:
Quoted 79 lines
Adds new property to indicate whether the encoder has multiple cores. Use
of_platform_populateto probe each venc cores, the core device can use the init_clk/request_irq helper to initialize their own power/clk/irq.Signed-off-by Irui Wang <[email protected]>--- .../media/platform/mediatek/vcodec/Makefile | 4 +- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 12 ++ .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 10 ++ .../mediatek/vcodec/mtk_vcodec_enc_hw.c | 139 ++++++++++++++++++ .../mediatek/vcodec/mtk_vcodec_enc_hw.h | 36 +++++ 5 files changed, 200 insertions(+), 1 deletion(-)create mode 100644
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.ccreate mode 100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.hdrivers/media/platform/mediatek/vcodec/Makefile +3
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk_vcodec_enc_hw.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-hw-y := mtk_vcodec_enc_hw.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +12
-0@@ -97,6 +97,15 @@ enum mtk_fmt_type { MTK_FMT_FRAME = 2, }; +/* +* enum mtk_venc_hw_id -- encoder hardware id +*/ +enum mtk_venc_hw_id { +MTK_VENC_CORE_0 =0 , +MTK_VENC_CORE_1 , +MTK_VENC_HW_MAX , +}; +/* * enum mtk_vdec_hw_id - Hardware index used to separate * different hardware @@ -484,6 +493,7 @@ struct mtk_vcodec_enc_pdata {* @dec_active_cnt: used to mark whether need to record register value * @vdec_racing_info: record register value * @dec_racing_info_mutex: mutex lock used for inner racing mode +* @enc_hw_dev: used to store venc core device */ struct mtk_vcodec_dev {struct v4l2_device v4l2_dev;@@ -534,6 +544,8 @@ struct mtk_vcodec_dev {u32 vdec_racing_info[132 ];/* Protects access to vdec_racing_info data */ struct mutex dec_racing_info_mutex; + +void *enc_hw_dev[MTK_VENC_HW_MAX ]; };static inline struct mtk_vcodec_ctx *fh_to_ctx (struct v4l2_fh *fh)drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-0
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c static int mtk_vcodec_probe(struct platform_device *pdev) · line 267
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +2
-0@@ -263,6 +263,16 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_enc_pm; } +if (of_property_read_bool (pdev->dev.of_node, +"mediatek,venc-multi-core" )) {
You don't need this property here: just call of_platform_populate() unconditionally. If there's no child node, this function will do nothing so this conditional is useless and can be avoided.
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c static int mtk_vcodec_probe(struct platform_device *pdev) · line 271
Quoted 4 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +4
-0+ ret =of_platform_populate (pdev->dev.of_node, +NULL ,NULL , &pdev->dev); +if (ret) { +mtk_v4l2_err ("Venc core device populate failed" );
What about "Failed to populate children devices" ?
Quoted 24 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +4
-0+ goto err_enc_pm; + } + } +pm_runtime_enable (&pdev->dev); dev->reg_base[dev->venc_pdata->core_id] =drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +11
-0@@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h>
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c line 21
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +10
-0+ +#include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_hw.h" +#include "mtk_vcodec_intr.h" + +static const struct of_device_id mtk_venc_hw_ids[] = { + { + .compatible ="mediatek,mtk-venc-hw" , + },
Please compress this in one line.
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c line 22
Quoted 1 line
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +1
-0+ {},
Usually, we say that this is a sentinel.
{ .compatible = "mediatek,mtk-venc-hw" },
{ /* sentinel */ },Quoted 3 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +3
-0+}; +MODULE_DEVICE_TABLE (of, mtk_venc_hw_ids); +
..snip..
Quoted 37 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +37
-0+ +static int mtk_venc_hw_probe (struct platform_device *pdev) +{ +struct device *dev = &pdev->dev; +struct mtk_venc_hw_dev *sub_core; +struct mtk_vcodec_dev *main_dev; +int ret; + +if (!dev->parent) +return dev_err_probe (dev, -ENODEV , +"No parent for venc core device\n" ); + + main_dev =dev_get_drvdata (dev->parent); +if (!main_dev) +return dev_err_probe (dev, -EINVAL , +"Failed to get parent driver data\n" ); + + sub_core =devm_kzalloc (&pdev->dev,sizeof (*sub_core),GFP_KERNEL ); +if (!sub_core) +return dev_err_probe (dev, -ENOMEM , +"Failed to get alloc core data\n" ); + + sub_core->plat_dev = pdev; + +platform_set_drvdata (pdev, sub_core); + + sub_core->reg_base =devm_platform_ioremap_resource (pdev,0 ); +if (IS_ERR (sub_core->reg_base)) +return dev_err_probe (dev,PTR_ERR (sub_core->reg_base), +"Failed to get reg base\n" ); + + sub_core->enc_irq =platform_get_irq (pdev,0 ); +if (sub_core->enc_irq <0 ) +return dev_err_probe (dev, -EINVAL , +"Failed to get irq resource\n" ); + + ret = devm_request_irq(dev, sub_core->enc_irq,
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c line 115
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +10
-0+ mtk_enc_hw_irq_handler,0 , + pdev->name, sub_core); +if (ret) +return dev_err_probe (dev, -EINVAL , +"Failed to install sub_core->enc_irq %d\n" , + sub_core->enc_irq); + +of_property_read_u32 (dev->of_node,"mediatek,hw-id" , + &sub_core->hw_id); +
I'd do it like this, instead:
ret = of_property_read_u32(dev->of_node, "mediatek,hw-id", &sub_core->hw_id);
if (ret || sub_core->hw_id >= MTK_VENC_HW_MAX)
return dev_err_probe(dev, (ret ? ret : -EINVAL),
"Cannot parse hardware id");P.S.: you're reading an unsigned value from devicetree, this cannot ever be less
than zero!
Quoted 53 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +24
-0+if (sub_core->hw_id <0 || sub_core->hw_id >=MTK_VENC_HW_MAX ) +return dev_err_probe (dev, -EINVAL , +"Invalid hardware id %d\n" , + sub_core->hw_id); + + main_dev->enc_hw_dev[sub_core->hw_id] = sub_core; + sub_core->main_dev = main_dev; + +dev_dbg (dev,"Venc core :%d probe done\n" , sub_core->hw_id); + +return 0 ; +} + +static struct platform_driver mtk_venc_core_driver = { + .probe = mtk_venc_hw_probe, + .driver = { + .name ="mtk-venc-core" , + .of_match_table = mtk_venc_hw_ids, + }, +}; +module_platform_driver (mtk_venc_core_driver); + +MODULE_LICENSE ("GPL" ); +MODULE_DESCRIPTION ("MediaTek video encoder core driver" );drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h new file +23
-0@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#ifndef _MTK_VCODEC_ENC_HW_H_ +#define _MTK_VCODEC_ENC_HW_H_ + +#include <linux/platform_device.h> +#include "mtk_vcodec_drv.h" + +/** +* struct mtk_venc_hw_dev - driver data +* @plat_dev: platform_device +* @main_dev: main device +* @pm: power management data +* @curr_ctx: the context that is waiting for venc hardware +* @reg_base: mapped address of venc registers +* @irq_status: venc hardware irq status +* @enc_irq: venc device irq +* @hw_id: for venc hardware id: core#0, core#1... +*/ +struct mtk_venc_hw_dev {
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h line 33
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h new file +10
-0+struct platform_device *plat_dev; +struct mtk_vcodec_dev *main_dev; + +struct mtk_vcodec_pm pm; +struct mtk_vcodec_ctx *curr_ctx; + +void __iomem *reg_base; +unsigned int irq_status; +int enc_irq; +int hw_id;
For consistency, this should be enum mtk_venc_hw_id hw_id;
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h line 36
Quoted 3 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h new file +3
-0+}; + +#endif /* _MTK_VCODEC_ENC_HW_H_ */
Cheers,
Angelo
Re: [PATCH v4, 2/8] media: mediatek: vcodec: Enable venc dual core usage
Dear Angelo,
Thanks for the comments.
Quoted 97 lines · AngeloGioacchino Del Regno
Il 24/06/22 10:23, Irui Wang ha scritto:
Quoted 96 lines
Adds new property to indicate whether the encoder has multiple cores.
Useof_platform_populateto probe each venc cores, the core device can
use the init_clk/request_irq helper to initialize their own power/clk/irq.Signed-off-by Irui Wang <[email protected]>--- .../media/platform/mediatek/vcodec/Makefile | 4 +- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 12 ++ .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 10 ++ .../mediatek/vcodec/mtk_vcodec_enc_hw.c | 139 ++++++++++++++++++ .../mediatek/vcodec/mtk_vcodec_enc_hw.h | 36 +++++ 5 files changed, 200 insertions(+), 1 deletion(-)create mode 100644
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c
create mode 100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.hdrivers/media/platform/mediatek/vcodec/Makefile +3
-1@@ -3,7 +3,8 @@ obj-$(CONFIG_VIDEO_MEDIATEK_VCODEC) += mtk-vcodec-dec.o \ mtk-vcodec-enc.o \ mtk-vcodec-common.o \ - mtk-vcodec-dec-hw.o + mtk-vcodec-dec-hw.o \ + mtk_vcodec_enc_hw.o mtk-vcodec-dec-y := vdec/vdec_h264_if.o \ vdec/vdec_vp8_if.o \@@ -32,6 +33,7 @@ mtk-vcodec-enc-y := venc/venc_vp8_if.o \ venc_drv_if.o \ venc_vpu_if.o \ +mtk-vcodec-enc-hw-y := mtk_vcodec_enc_hw.o mtk-vcodec-common-y := mtk_vcodec_intr.o \ mtk_vcodec_util.o \diff --git
a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.hb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.hindexdc6aada882d9..8919bdf2eef5100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +9
-0@@ -97,6 +97,15 @@ enum mtk_fmt_type { MTK_FMT_FRAME = 2, }; +/* +* enum mtk_venc_hw_id -- encoder hardware id +*/ +enum mtk_venc_hw_id { +MTK_VENC_CORE_0 =0 , +MTK_VENC_CORE_1 , +MTK_VENC_HW_MAX , +}; +/* * enum mtk_vdec_hw_id - Hardware index used to separate * different hardware @@ -484,6 +493,7 @@ struct mtk_vcodec_enc_pdata {* @dec_active_cnt: used to mark whether need to record register value
* @vdec_racing_info: record register value
* @dec_racing_info_mutex: mutex lock used for inner racing mode
+ * @enc_hw_dev: used to store venc core device
*/
structmtk_vcodec_dev{
structv4l2_devicev4l2_dev;drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +2
-0@@ -534,6 +544,8 @@ struct mtk_vcodec_dev {u32 vdec_racing_info[132 ];/* Protects access to vdec_racing_info data */ struct mutex dec_racing_info_mutex; + +void *enc_hw_dev[MTK_VENC_HW_MAX ]; }; static inline struct mtk_vcodec_ctx *fh_to_ctx(struct v4l2_fh*fh)
diff --gita/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.cb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.cindex95e8c29ccc65..65a8251a5a68100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +0
-0@@ -263,6 +263,16 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_enc_pm; }
Quoted 7 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c static int mtk_vcodec_probe(struct platform_device *pdev) · line 267
Quoted 2 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +2
-0+if (of_property_read_bool (pdev->dev.of_node, +"mediatek,venc-multi-core" )) {You don't need this property here: just call
of_platform_populate()unconditionally. If there's no child node, this function will do nothing
so this conditional is useless and can be avoided.
I will update YAML and fix it in next version.
Quoted 8 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c static int mtk_vcodec_probe(struct platform_device *pdev)
Quoted 6 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +4
-0+ ret =of_platform_populate (pdev->dev.of_node, +NULL ,NULL , &pdev->dev); +if (ret) { +mtk_v4l2_err ("Venc core device populate failed");What about "Failed to populate children devices" ?
fix it in next version.
Quoted 40 lines
Quoted 27 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +4
-0+ goto err_enc_pm; + } + } +pm_runtime_enable (&pdev->dev);dev->reg_base[dev->venc_pdata->core_id] =
diff --gita/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.cb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.cnew file mode 100644
index 000000000000..02582cce4863drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +11
-0@@ -0,0 +1,139 @@ +// SPDX-License-Identifier: GPL-2.0 +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#include <linux/interrupt.h> +#include <linux/irq.h> +#include <linux/module.h> +#include <linux/of_platform.h> +#include <linux/pm_runtime.h> +#include <linux/slab.h> drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c line 21
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +10
-0+ +#include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_hw.h" +#include "mtk_vcodec_intr.h" + +static const struct of_device_id mtk_venc_hw_ids[] = { + { + .compatible ="mediatek,mtk-venc-hw" , + },Please compress this in one line.
Quoted 6 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c line 22
Quoted 1 line
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +1
-0+ {},Usually, we say that this is a sentinel.
{ .compatible = "mediatek,mtk-venc-hw" }, { /* sentinel */ },
fix it in next version.
Quoted 68 lines
Quoted 4 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +3
-0+}; +MODULE_DEVICE_TABLE (of, mtk_venc_hw_ids); +..snip..
Quoted 47 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +11
-0+ +static int mtk_venc_hw_probe (struct platform_device *pdev) +{ +struct device *dev = &pdev->dev; +struct mtk_venc_hw_dev *sub_core; +struct mtk_vcodec_dev *main_dev; +int ret; + +if (!dev->parent) + return dev_err_probe(dev, -ENODEV, + "No parent for venc coredevice\n");
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +5
-0+ + main_dev =dev_get_drvdata (dev->parent); +if (!main_dev) + return dev_err_probe(dev, -EINVAL, + "Failed to get parent driverdata\n");
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +2
-0+ + sub_core = devm_kzalloc(&pdev->dev, sizeof(*sub_core),
GFP_KERNEL);drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +3
-0+if (!sub_core) + return dev_err_probe(dev, -ENOMEM, + "Failed to get alloc coredata\n");
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +21
-0+ + sub_core->plat_dev = pdev; + +platform_set_drvdata (pdev, sub_core); + + sub_core->reg_base =devm_platform_ioremap_resource (pdev,0 ); +if (IS_ERR (sub_core->reg_base)) +return dev_err_probe (dev,PTR_ERR (sub_core->reg_base), +"Failed to get reg base\n" ); + + sub_core->enc_irq =platform_get_irq (pdev,0 ); +if (sub_core->enc_irq <0 ) +return dev_err_probe (dev, -EINVAL , +"Failed to get irq resource\n" ); + + ret =devm_request_irq (dev, sub_core->enc_irq, + mtk_enc_hw_irq_handler,0 , + pdev->name, sub_core); +if (ret) + return dev_err_probe(dev, -EINVAL, + "Failed to install sub_core-Quoted 1 line
enc_irq%d\n",drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c line 115
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +5
-0+ sub_core->enc_irq); + +of_property_read_u32 (dev->of_node,"mediatek,hw-id" , + &sub_core->hw_id); +I'd do it like this, instead:
ret = of_property_read_u32(dev->of_node, "mediatek,hw-id", &sub_core->hw_id); if (ret || sub_core->hw_id >= MTK_VENC_HW_MAX) return dev_err_probe(dev, (ret ? ret : -EINVAL), "Cannot parse hardware id");
Quoted 3 lines
P.S.: you're reading an unsigned value from devicetree, this cannot ever be less
than zero!
fix it in next version.
Quoted 60 lines
Quoted 60 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c new file +24
-0+if (sub_core->hw_id <0 || sub_core->hw_id >=MTK_VENC_HW_MAX ) +return dev_err_probe (dev, -EINVAL , +"Invalid hardware id %d\n" , + sub_core->hw_id); + + main_dev->enc_hw_dev[sub_core->hw_id] = sub_core; + sub_core->main_dev = main_dev; + +dev_dbg (dev,"Venc core :%d probe done\n" , sub_core->hw_id); + +return 0 ; +} + +static struct platform_driver mtk_venc_core_driver = { + .probe = mtk_venc_hw_probe, + .driver = { + .name ="mtk-venc-core" , + .of_match_table = mtk_venc_hw_ids, + }, +}; +module_platform_driver (mtk_venc_core_driver); + +MODULE_LICENSE ("GPL" ); +MODULE_DESCRIPTION ("MediaTek video encoder core driver" );diff --git
a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.hb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.hnew file mode 100644
index 000000000000..0ff544c20eb9drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h new file +27
-0@@ -0,0 +1,36 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Copyright (c) 2021 MediaTek Inc. +*/ + +#ifndef _MTK_VCODEC_ENC_HW_H_ +#define _MTK_VCODEC_ENC_HW_H_ + +#include <linux/platform_device.h> +#include "mtk_vcodec_drv.h" + +/** + * struct mtk_venc_hw_dev - driver data + * @plat_dev: platform_device + * @main_dev: main device + * @pm: power management data + * @curr_ctx: the context that is waiting for venc hardware + * @reg_base: mapped address of venc registers + * @irq_status: venc hardware irq status + * @enc_irq: venc device irq + * @hw_id: for venc hardware id: core#0, core#1... + */ +struct mtk_venc_hw_dev { + struct platform_device *plat_dev; + struct mtk_vcodec_dev *main_dev; + + struct mtk_vcodec_pm pm;
Quoted 8 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h line 33
Quoted 6 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h new file +6
-0+struct mtk_vcodec_ctx *curr_ctx; + +void __iomem *reg_base; +unsigned int irq_status; +int enc_irq; +int hw_id;For consistency, this should be
enum mtk_venc_hw_id hw_id;
fix it in next version.
Quoted 7 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h line 36
Quoted 4 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h new file +3
-0+}; + +#endif /* _MTK_VCODEC_ENC_HW_H_ */ Cheers,
Angelo
Thanks
Best Regards
PATCH v4 3/8 media: mediatek: vcodec: Refactor venc power manage function
In reply to Irui WangThe args "struct mtk_vcodec_dev *" doesn't appropriate for init_clk functions because of sub-devices, sub-devices will init their own "pm/clk" instead, so refactor the pm function with args "platform_device *" and "mtk_vcodec_pm*".
--- .../media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c | 2 +- .../media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c | 8 ++++++++ .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c | 9 +++------ .../media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h | 3 ++- 4 files changed, 14 insertions(+), 8 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +1 -1
@@ -257,7 +257,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)return PTR_ERR (dev->fw_handler); dev->venc_pdata =of_device_get_match_data (&pdev->dev); - ret =mtk_vcodec_init_enc_clk (dev); + ret =mtk_vcodec_init_enc_clk (dev->plat_dev, &dev->pm);if (ret <0 ) {dev_err (&pdev->dev,"Failed to get mtk vcodec clock source!" ); goto err_enc_pm;
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c +8 -0
@@ -13,6 +13,7 @@#include "mtk_vcodec_drv.h" #include "mtk_vcodec_enc.h" #include "mtk_vcodec_enc_hw.h" +#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_intr.h" static const struct of_device_id mtk_venc_hw_ids[] = {
@@ -118,6 +119,13 @@ static int mtk_venc_hw_probe(struct platform_device *pdev)"Invalid hardware id %d\n" , sub_core->hw_id); + ret =mtk_vcodec_init_enc_clk (sub_core->plat_dev, &sub_core->pm); +if (ret <0 ) +return dev_err_probe (dev, ret, +"Failed to get venc core clock source!" ); + +pm_runtime_enable (&pdev->dev); + main_dev->enc_hw_dev[sub_core->hw_id] = sub_core; sub_core->main_dev = main_dev;
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c +3 -6
@@ -12,17 +12,13 @@#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_util.h" -int mtk_vcodec_init_enc_clk (struct mtk_vcodec_dev *mtkdev) +int mtk_vcodec_init_enc_clk (struct platform_device *pdev, +struct mtk_vcodec_pm *pm) { -struct platform_device *pdev; -struct mtk_vcodec_pm *pm;struct mtk_vcodec_clk *enc_clk;struct mtk_vcodec_clk_info *clk_info;int ret, i; - pdev = mtkdev->plat_dev; - pm = &mtkdev->pm; -memset (pm,0 ,sizeof (struct mtk_vcodec_pm )); pm->dev = &pdev->dev; enc_clk = &pm->venc_clk;
@@ -58,6 +54,7 @@ int mtk_vcodec_init_enc_clk(struct mtk_vcodec_dev *mtkdev)return 0 ; } +EXPORT_SYMBOL_GPL (mtk_vcodec_init_enc_clk);void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm) {
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h +2 -1
@@ -9,7 +9,8 @@#include "mtk_vcodec_drv.h" -int mtk_vcodec_init_enc_clk (struct mtk_vcodec_dev *dev); +int mtk_vcodec_init_enc_clk (struct platform_device *pdev, +struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm);
Signature · 1 line
2.18.0
PATCH v4 4/8 media: mediatek: vcodec: Add more extra processing for multi-core encoding
In reply to Irui WangAdd a bit for indicating support multi-core encoding, because multi-core encoding need more working buffers for encoder hardware. The working buffers are allocated from kernel side then pass to scp firmware side through shared memory, the struct definition must be kept align between kernel and scp firmware side. New another shared memory struct for multi-core encoding.
--- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 2 + .../mediatek/vcodec/mtk_vcodec_util.c | 19 ++ .../mediatek/vcodec/mtk_vcodec_util.h | 2 + .../mediatek/vcodec/venc/venc_h264_if.c | 171 +++++++++++++++--- 4 files changed, 170 insertions(+), 24 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +2 -0
@@ -29,6 +29,8 @@#define WAIT_INTR_TIMEOUT_MS 1000#define IS_VDEC_LAT_ARCH (hw_arch) ((hw_arch) >= MTK_VDEC_LAT_SINGLE_CORE)#define IS_VDEC_INNER_RACING (capability) ((capability) & MTK_VCODEC_INNER_RACING) +#define MTK_VENC_MULTICORE_ENABLE BIT(1) +#define IS_VENC_MULTICORE (capability) ((capability) & MTK_VENC_MULTICORE_ENABLE)/* * enum mtk_hw_reg_idx - MTK hw register base index
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +19 -0
@@ -11,6 +11,7 @@#include "mtk_vcodec_dec_hw.h" #include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc_hw.h" #include "mtk_vcodec_util.h" void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, }EXPORT_SYMBOL (mtk_vcodec_get_reg_addr); +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *ctx, +unsigned int hw_id) +{ +struct mtk_venc_hw_dev *sub_core; + +if (hw_id >=MTK_VENC_HW_MAX ) { +mtk_v4l2_err ("Invalid hw_id = %d" , hw_id); +return NULL ; + } + + sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[hw_id]; +if (!sub_core) +return NULL ; + +return sub_core->reg_base; +} +EXPORT_SYMBOL (mtk_venc_get_core_reg_addr); +int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem) {
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h +2 -0
@@ -50,6 +50,8 @@ struct mtk_vcodec_dev;void __iomem *mtk_vcodec_get_reg_addr (struct mtk_vcodec_ctx *data,unsigned int reg_idx); +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *data, +unsigned int hw_id);int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem); void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +147 -24
278 lines not on this page · raw message
Signature · 1 line
2.18.0
AngeloGioacchino Del Regno
maintainerRe: [PATCH v4, 4/8] media: mediatek: vcodec: Add more extra processing for multi-core encoding
Il 24/06/22 10:23, Irui Wang ha scritto:
Quoted 41 lines
Add a bit for indicating support multi-core encoding, because multi-core encoding need more working buffers for encoder hardware. The working buffers are allocated from kernel side then pass to scp firmware side through shared memory, the struct definition must be kept align between kernel and scp firmware side. New another shared memory struct for multi-core encoding.
Signed-off-by Irui Wang <[email protected]>--- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 2 + .../mediatek/vcodec/mtk_vcodec_util.c | 19 ++ .../mediatek/vcodec/mtk_vcodec_util.h | 2 + .../mediatek/vcodec/venc/venc_h264_if.c | 171 +++++++++++++++--- 4 files changed, 170 insertions(+), 24 deletions(-)drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +2
-0@@ -29,6 +29,8 @@#define WAIT_INTR_TIMEOUT_MS 1000#define IS_VDEC_LAT_ARCH (hw_arch) ((hw_arch) >= MTK_VDEC_LAT_SINGLE_CORE)#define IS_VDEC_INNER_RACING (capability) ((capability) & MTK_VCODEC_INNER_RACING) +#define MTK_VENC_MULTICORE_ENABLE BIT(1) +#define IS_VENC_MULTICORE (capability) ((capability) & MTK_VENC_MULTICORE_ENABLE)/* * enum mtk_hw_reg_idx - MTK hw register base index drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +1
-0@@ -11,6 +11,7 @@#include "mtk_vcodec_dec_hw.h" #include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc_hw.h" #include "mtk_vcodec_util.h" void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data,
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, · line 31
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +2
-0@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx *data, }EXPORT_SYMBOL (mtk_vcodec_get_reg_addr); +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *ctx, +unsigned int hw_id)
This is enum mtk_venc_hw_id.....
Quoted 36 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +16
-0+{ +struct mtk_venc_hw_dev *sub_core; + +if (hw_id >=MTK_VENC_HW_MAX ) { +mtk_v4l2_err ("Invalid hw_id = %d" , hw_id); +return NULL ; + } + + sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[hw_id]; +if (!sub_core) +return NULL ; + +return sub_core->reg_base; +} +EXPORT_SYMBOL (mtk_venc_get_core_reg_addr); +int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem) {drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h +2
-0@@ -50,6 +50,8 @@ struct mtk_vcodec_dev;void __iomem *mtk_vcodec_get_reg_addr (struct mtk_vcodec_ctx *data,unsigned int reg_idx); +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *data, +unsigned int hw_id);int mtk_vcodec_mem_alloc (struct mtk_vcodec_ctx *data,struct mtk_vcodec_mem *mem); void mtk_vcodec_mem_free(struct mtk_vcodec_ctx *data,drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +0
-0
..snip..
Quoted 12 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +2
-2@@ -143,8 +184,8 @@ struct venc_h264_vsi {* @ctx: context for v4l2 layer integration */ struct venc_h264_inst { -void __iomem *hw_base; -struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX ]; +void __iomem *hw_base[MTK_VENC_HW_MAX ]; +struct mtk_vcodec_mem work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX ];struct mtk_vcodec_mem pps_buf;bool work_buf_allocated;unsigned int frm_cnt;
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c struct venc_h264_inst { · line 196
Quoted 4 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +1
-0@@ -152,12 +193,13 @@ struct venc_h264_inst {unsigned int prepend_hdr;struct venc_vpu_inst vpu_inst;struct venc_h264_vsi *vsi; +struct venc_multi_core_vsi *core_vsi;
You're adding a new struct member without adding it to the documentation, please add it there too.
Quoted 11 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +1
-1struct mtk_vcodec_ctx *ctx; };static inline u32 h264_read_reg (struct venc_h264_inst *inst,u32 addr) { -return readl (inst->hw_base + addr); +return readl (inst->hw_base[MTK_VENC_CORE_0 ] + addr); } static unsigned int h264_get_profile(struct venc_h264_inst *inst,
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static unsigned int h264_get_level(struct venc_h264_inst *inst, · line 274
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +2
-0@@ -228,13 +270,20 @@ static unsigned int h264_get_level(struct venc_h264_inst *inst,static void h264_enc_free_work_buf (struct venc_h264_inst *inst) {int i; +struct mtk_vcodec_ctx *ctx = inst->ctx; +int max_work_buf;
int i, max_work_buf;
Quoted 17 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +6
-1mtk_vcodec_debug_enter (inst); +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) + max_work_buf =VENC_MULTI_CORE_WORK_BUF_MAX ; +else + max_work_buf =VENC_H264_VPU_WORK_BUF_MAX ; +/* Except the SKIP_FRAME buffers, * other buffers need to be freed by AP. */ -for (i =0 ; i <VENC_H264_VPU_WORK_BUF_MAX ; i++) { +for (i =0 ; i < max_work_buf; i++) {if (i !=VENC_H264_VPU_WORK_BUF_SKIP_FRAME )mtk_vcodec_mem_free (inst->ctx, &inst->work_bufs[i]); }
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst) · line 302
Quoted 7 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +3
-1@@ -248,11 +297,21 @@ static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst) {int i;int ret =0 ; -struct venc_h264_vpu_buf *wb = inst->vsi->work_bufs; +struct mtk_vcodec_ctx *ctx = inst->ctx; +struct venc_h264_vpu_buf *wb; +int max_work_buf;
While at it, can you please fix the order of these declarations?
struct mtk_vcodec_ctx *ctx = inst->ctx; struct venc_h264_vpu_buf *wb; int i, max_work_buf; int ret = 0;
Quoted 20 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +9
-1mtk_vcodec_debug_enter (inst); -for (i =0 ; i <VENC_H264_VPU_WORK_BUF_MAX ; i++) { +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) { + wb = inst->core_vsi->work_bufs; + max_work_buf =VENC_MULTI_CORE_WORK_BUF_MAX ; + }else { + wb = inst->vsi->work_bufs; + max_work_buf =VENC_H264_VPU_WORK_BUF_MAX ; + } + +for (i =0 ; i < max_work_buf; i++) {/* * This 'wb' structure is set by VPU side and shared to AP for * buffer allocation and IO virtual addr mapping. For most of @@ -358,6 +417,26 @@ static int h264_frame_type(struct venc_h264_inst *inst)return VENC_H264_P_FRM ;/* Note: B frames are not supported */ } }
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static int h264_frame_type(struct venc_h264_inst *inst) · line 429
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +10
-0+ +static int h264_core_frame_type (struct venc_h264_inst *inst) +{ +struct venc_multi_core_vsi *vsi = inst->core_vsi; + +if ((vsi->config.gop_size !=0 && + (inst->frm_cnt % vsi->config.gop_size) ==0 ) || + (inst->frm_cnt ==0 && vsi->config.gop_size ==0 )) { +/* IDR frame */ +return VENC_H264_IDR_FRM ;
You can get to a maximum of 100 columns and, thinking about readability...:
if ((vsi->config.gop_size != 0 && (inst->frm_cnt % vsi->config.gop_size) == 0) ||
(inst->frm_cnt == 0 && vsi->config.gop_size == 0)) {this gets to 90 columns, so it's acceptable (and more readable).
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static int h264_frame_type(struct venc_h264_inst *inst) · line 432
Quoted 3 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +3
-0+ }else if ((vsi->config.intra_period !=0 && + (inst->frm_cnt % vsi->config.intra_period) ==0 ) || + (inst->frm_cnt ==0 && vsi->config.intra_period ==0 )) {
..but, unfortunately, this doesn't, so we have to live with it.
Quoted 29 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +12
-1+/* I frame */ +return VENC_H264_I_FRM ; + }else { +return VENC_H264_P_FRM ;/* Note: B frames are not supported */ + } +} +static int h264_encode_sps (struct venc_h264_inst *inst,struct mtk_vcodec_mem *bs_buf,unsigned int *bs_size)@@ -440,12 +519,16 @@ static int h264_encode_frame(struct venc_h264_inst *inst,int ret =0 ;unsigned int irq_status;struct venc_frame_info frame_info; +struct mtk_vcodec_ctx *ctx = inst->ctx;mtk_vcodec_debug_enter (inst);mtk_vcodec_debug (inst,"frm_cnt = %d\n " , inst->frm_cnt); frame_info.frm_count = inst->frm_cnt; frame_info.skip_frm_count = inst->skip_frm_cnt; - frame_info.frm_type =h264_frame_type (inst); +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) + frame_info.frm_type =h264_core_frame_type (inst); +else + frame_info.frm_type =h264_frame_type (inst);mtk_vcodec_debug (inst,"frm_count = %d,skip_frm_count =%d,frm_type=%d.\n" , frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type);
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static void h264_encode_filler(struct venc_h264_inst *inst, void *buf, · line 587
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +1
-1@@ -501,7 +584,7 @@ static void h264_encode_filler(struct venc_h264_inst *inst, void *buf,static int h264_enc_init (struct mtk_vcodec_ctx *ctx) {const bool is_ext =MTK_ENC_CTX_IS_EXT (ctx); -int ret =0 ; +int ret, i;
Can you also move this after inst please?
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static void h264_encode_filler(struct venc_h264_inst *inst, void *buf, · line 589
Quoted 2 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +0
-0struct venc_h264_inst *inst;
Cheers,
Angelo
Re: [PATCH v4, 4/8] media: mediatek: vcodec: Add more extra processing for multi-core encoding
Dear Angelo,
Many thanks for your reviewing and comments. I will fix them in next version.
Quoted 59 lines · AngeloGioacchino Del Regno
Il 24/06/22 10:23, Irui Wang ha scritto:
Quoted 58 lines
Add a bit for indicating support multi-core encoding, because multi-core
encoding need more working buffers for encoder hardware. The working
buffers are allocated from kernel side then pass to scp firmware side
through shared memory, the struct definition must be kept align between
kernel and scp firmware side. New another shared memory struct for multi-core encoding.Signed-off-by Irui Wang <[email protected]>--- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 2 + .../mediatek/vcodec/mtk_vcodec_util.c | 19 ++ .../mediatek/vcodec/mtk_vcodec_util.h | 2 + .../mediatek/vcodec/venc/venc_h264_if.c | 171 +++++++++++++++--- 4 files changed, 170 insertions(+), 24 deletions(-)diff --git
a/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.hb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.hindex8919bdf2eef5..3a291cac6b5e100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +2
-0@@ -29,6 +29,8 @@#define WAIT_INTR_TIMEOUT_MS 1000#define IS_VDEC_LAT_ARCH (hw_arch) ((hw_arch) >= MTK_VDEC_LAT_SINGLE_CORE)#define IS_VDEC_INNER_RACING (capability) ((capability) & MTK_VCODEC_INNER_RACING) +#define MTK_VENC_MULTICORE_ENABLE BIT(1) +#define IS_VENC_MULTICORE (capability) ((capability) & MTK_VENC_MULTICORE_ENABLE)/*
* enummtk_hw_reg_idx- MTK hw register base index
diff --gita/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.cb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.cindexace78c4b5b9e..a723243626c0100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +1
-0@@ -11,6 +11,7 @@#include "mtk_vcodec_dec_hw.h" #include "mtk_vcodec_drv.h" +#include "mtk_vcodec_enc_hw.h" #include "mtk_vcodec_util.h" void __iomem *mtk_vcodec_get_reg_addr(struct mtk_vcodec_ctx*data,
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +0
-0@@ -26,6 +27,24 @@ void __iomem *mtk_vcodec_get_reg_addr(structmtk_vcodec_ctx *data, }EXPORT_SYMBOL (mtk_vcodec_get_reg_addr );
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c void __iomem *mtk_vcodec_get_reg_addr(struct · line 33
Quoted 3 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +2
-0+void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *ctx, +unsigned int hw_id)This is enum
mtk_venc_hw_id.....
fix in next version.
Quoted 59 lines
Quoted 44 lines
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.c +16
-0+{ +struct mtk_venc_hw_dev *sub_core; + +if (hw_id >=MTK_VENC_HW_MAX ) { +mtk_v4l2_err ("Invalid hw_id = %d" , hw_id); +return NULL ; + } + + sub_core = (struct mtk_venc_hw_dev *)ctx->dev- >enc_hw_dev[hw_id]; +if (!sub_core) +return NULL ; + +return sub_core->reg_base; +} +EXPORT_SYMBOL (mtk_venc_get_core_reg_addr); +int mtk_vcodec_mem_alloc(struct
mtk_vcodec_ctx*data,
structmtk_vcodec_mem*mem)
{
diff --gita/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.hb/drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.hindex71956627a0e2..a74c98aa355b100644drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h +2
-0@@ -50,6 +50,8 @@ struct mtk_vcodec_dev;void __iomem *mtk_vcodec_get_reg_addr (struct mtk_vcodec_ctx *data,unsigned int reg_idx); +void __iomem *mtk_venc_get_core_reg_addr (struct mtk_vcodec_ctx *data, +unsigned int hw_id); int mtk_vcodec_mem_alloc(struct mtk_vcodec_ctx *data,struct
mtk_vcodec_mem*mem);
void mtk_vcodec_mem_free(structmtk_vcodec_ctx*data,
diff --gita/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.cb/drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.cindex4d9b8798dffe..888cee39d324100644drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +0
-0..snip..
Quoted 12 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +2
-2@@ -143,8 +184,8 @@ struct venc_h264_vsi {* @ctx: context for v4l2 layer integration */ struct venc_h264_inst { -void __iomem *hw_base; -struct mtk_vcodec_mem work_bufs[VENC_H264_VPU_WORK_BUF_MAX ]; +void __iomem *hw_base[MTK_VENC_HW_MAX ]; +struct mtk_vcodec_mem work_bufs[VENC_MULTI_CORE_WORK_BUF_MAX ];struct mtk_vcodec_mem pps_buf;bool work_buf_allocated;unsigned int frm_cnt;@@ -152,12 +193,13 @@ struct venc_h264_inst {
Quoted 8 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c struct venc_h264_inst { · line 196
Quoted 4 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +1
-0unsigned int prepend_hdr;struct venc_vpu_inst vpu_inst;struct venc_h264_vsi *vsi; +struct venc_multi_core_vsi *core_vsi;You're adding a new struct member without adding it to the documentation,
please add it there too.
Add it in next version.
Quoted 51 lines
Quoted 13 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +1
-1struct mtk_vcodec_ctx *ctx; };static inline u32 h264_read_reg (struct venc_h264_inst *inst,u32 addr) { -return readl (inst->hw_base + addr); +return readl (inst->hw_base[MTK_VENC_CORE_0 ] + addr); }static unsigned int h264_get_profile(struct
venc_h264_inst*inst,drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static unsigned int h264_get_level(struct · line 275
Quoted 6 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +2
-0@@ -228,13 +270,20 @@ static unsigned int h264_get_level(structvenc_h264_inst *inst, static void h264_enc_free_work_buf(struct venc_h264_inst *inst) { int i; +struct mtk_vcodec_ctx *ctx = inst->ctx; +int max_work_buf;int i,
max_work_buf;Quoted 18 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +6
-1mtk_vcodec_debug_enter (inst); +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) + max_work_buf =VENC_MULTI_CORE_WORK_BUF_MAX ; +else + max_work_buf =VENC_H264_VPU_WORK_BUF_MAX ; +/* Except the SKIP_FRAME buffers, * other buffers need to be freed by AP. */ -for (i =0 ; i <VENC_H264_VPU_WORK_BUF_MAX ; i++) { +for (i =0 ; i < max_work_buf; i++) {if (i !=VENC_H264_VPU_WORK_BUF_SKIP_FRAME )mtk_vcodec_mem_free (inst->ctx, &inst- >work_bufs[i]);}drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static int h264_enc_alloc_work_buf(struct · line 303
Quoted 8 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +3
-1@@ -248,11 +297,21 @@ static int h264_enc_alloc_work_buf(struct venc_h264_inst *inst) {int i;int ret =0 ; -struct venc_h264_vpu_buf *wb = inst->vsi->work_bufs; +struct mtk_vcodec_ctx *ctx = inst->ctx; +struct venc_h264_vpu_buf *wb; +int max_work_buf;While at it, can you please fix the order of these declarations?
Quoted 5 lines
struct mtk_vcodec_ctx *ctx = inst->ctx; struct venc_h264_vpu_buf *wb; int i, max_work_buf; int ret = 0;
fix it in next version.
Quoted 45 lines
Quoted 24 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +9
-1mtk_vcodec_debug_enter (inst); -for (i =0 ; i <VENC_H264_VPU_WORK_BUF_MAX ; i++) { +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) { + wb = inst->core_vsi->work_bufs; + max_work_buf =VENC_MULTI_CORE_WORK_BUF_MAX ; + }else { + wb = inst->vsi->work_bufs; + max_work_buf =VENC_H264_VPU_WORK_BUF_MAX ; + } + +for (i =0 ; i < max_work_buf; i++) {/* * This 'wb' structure is set by VPU side and shared to AP for
* buffer allocation and IO virtual addr mapping. For
most ofdrivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +0
-0@@ -358,6 +417,26 @@ static int h264_frame_type(struct venc_h264_inst *inst)return VENC_H264_P_FRM ;/* Note: B frames are not supported */
}
}drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static int h264_frame_type(struct venc_h264_inst *inst) · line 429
Quoted 10 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +10
-0+ +static int h264_core_frame_type (struct venc_h264_inst *inst) +{ +struct venc_multi_core_vsi *vsi = inst->core_vsi; + +if ((vsi->config.gop_size !=0 && + (inst->frm_cnt % vsi->config.gop_size) ==0 ) || + (inst->frm_cnt ==0 && vsi->config.gop_size ==0 )) { +/* IDR frame */ +return VENC_H264_IDR_FRM ;You can get to a maximum of 100 columns and, thinking about readability...:
if ((
vsi->config.gop_size!= 0 && (inst->frm_cnt% vsi-Quoted 1 line
config.gop_size) == 0) ||
(inst->frm_cnt == 0 && vsi->config.gop_size == 0)) {this gets to 90 columns, so it's acceptable (and more readable).
Quoted 6 lines
Quoted 4 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +3
-0+ } else if ((vsi->config.intra_period != 0 && + (inst->frm_cnt % vsi->config.intra_period) == 0) || + (inst->frm_cnt == 0 && vsi->config.intra_period ==0)) {
..but, unfortunately, this doesn't, so we have to live with it.
thanks, I will check it.
Quoted 35 lines
Quoted 35 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +4
-0+/* I frame */ +return VENC_H264_I_FRM ; + }else { +return VENC_H264_P_FRM ;/* Note: B frames are not supported */
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +8
-1+ } +} +static int h264_encode_sps (struct venc_h264_inst *inst,struct mtk_vcodec_mem *bs_buf,unsigned int *bs_size)@@ -440,12 +519,16 @@ static int h264_encode_frame(structvenc_h264_inst *inst, int ret = 0;unsigned int irq_status;struct venc_frame_info frame_info; +struct mtk_vcodec_ctx *ctx = inst->ctx;mtk_vcodec_debug_enter (inst);mtk_vcodec_debug (inst,"frm_cnt = %d\n " , inst->frm_cnt); frame_info.frm_count = inst->frm_cnt; frame_info.skip_frm_count = inst->skip_frm_cnt; - frame_info.frm_type =h264_frame_type (inst); +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) + frame_info.frm_type =h264_core_frame_type (inst); +else + frame_info.frm_type =h264_frame_type (inst); mtk_vcodec_debug(inst, "frm_count = %d,skip_frm_count =%d,frm_type=%d.\n",frame_info.frm_count,
frame_info.skip_frm_count,
frame_info.frm_type);drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +0
-0@@ -501,7 +584,7 @@ static void h264_encode_filler(struct
Quoted 9 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static void h264_encode_filler(struct venc_h264_inst *inst, void *buf, · line 587
Quoted 6 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +1
-1venc_h264_inst *inst, void *buf, static int h264_enc_init(struct mtk_vcodec_ctx *ctx) {const bool is_ext =MTK_ENC_CTX_IS_EXT (ctx); -int ret =0 ; +int ret, i;Can you also move this after
instplease?
fix it in next version.
Quoted 5 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c static void h264_encode_filler(struct venc_h264_inst *inst, void *buf, · line 589
Quoted 2 lines
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +0
-0struct venc_h264_inst *inst;Cheers,
Angelo
Thanks
Best Regards
PATCH v4 5/8 media: mediatek: vcodec: Add venc power on/off function
In reply to Irui Wangwhen enable multi-core encoding, all available encoder cores' power need to be on/off, add new functions for encoder cores' power management.
--- .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 30 +++---- .../mediatek/vcodec/mtk_vcodec_enc_pm.c | 84 +++++++++++++++++++ .../mediatek/vcodec/mtk_vcodec_enc_pm.h | 2 + .../platform/mediatek/vcodec/venc_drv_if.c | 6 -- 4 files changed, 98 insertions(+), 24 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c +12 -18
@@ -8,10 +8,10 @@#include <media/v4l2-event.h> #include <media/v4l2-mem2mem.h> #include <media/videobuf2-dma-contig.h> -#include <linux/pm_runtime.h> #include "mtk_vcodec_drv.h" #include "mtk_vcodec_enc.h" +#include "mtk_vcodec_enc_pm.h" #include "mtk_vcodec_intr.h" #include "mtk_vcodec_util.h" #include "venc_drv_if.h"
@@ -849,7 +849,7 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count) {struct mtk_vcodec_ctx *ctx =vb2_get_drv_priv (q);struct venc_enc_param param; -int ret, pm_ret; +int ret;int i;/* Once state turn into MTK_STATE_ABORT, we need stop_streaming
@@ -869,18 +869,12 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)return 0 ; } - ret =pm_runtime_resume_and_get (&ctx->dev->plat_dev->dev); -if (ret <0 ) { -mtk_v4l2_err ("pm_runtime_resume_and_get fail %d" , ret); - goto err_start_stream; - } -mtk_venc_set_param (ctx, ¶m); ret =venc_if_set_param (ctx,VENC_SET_PARAM_ENC , ¶m);if (ret) {mtk_v4l2_err ("venc_if_set_param failed=%d" , ret); ctx->state =MTK_STATE_ABORT ; - goto err_set_param; + goto err_start_stream; } ctx->param_change =MTK_ENCODE_PARAM_NONE ;
@@ -893,17 +887,19 @@ static int vb2ops_venc_start_streaming(struct vb2_queue *q, unsigned int count)if (ret) {mtk_v4l2_err ("venc_if_set_param failed=%d" , ret); ctx->state =MTK_STATE_ABORT ; - goto err_set_param; + goto err_start_stream; } ctx->state =MTK_STATE_HEADER ; } -return 0 ; + ret =mtk_vcodec_enc_power_on (ctx); +if (ret) { +mtk_v4l2_err ("mtk_vcodec_enc_power_on failed=%d" , ret); + ctx->state =MTK_STATE_ABORT ; + goto err_start_stream; + } -err_set_param: - pm_ret =pm_runtime_put (&ctx->dev->plat_dev->dev); -if (pm_ret <0 ) -mtk_v4l2_err ("pm_runtime_put fail %d" , pm_ret); +return 0 ; err_start_stream:for (i =0 ; i < q->num_buffers; ++i) {
@@ -988,9 +984,7 @@ static void vb2ops_venc_stop_streaming(struct vb2_queue *q)if (ret)mtk_v4l2_err ("venc_if_deinit failed=%d" , ret); - ret =pm_runtime_put (&ctx->dev->plat_dev->dev); -if (ret <0 ) -mtk_v4l2_err ("pm_runtime_put fail %d" , ret); +mtk_vcodec_enc_power_off (ctx); ctx->state =MTK_STATE_FREE ; }
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c +84 -0
96 lines not on this page · raw message
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h +2 -0
@@ -12,6 +12,8 @@int mtk_vcodec_init_enc_clk (struct platform_device *pdev,struct mtk_vcodec_pm *pm); +int mtk_vcodec_enc_power_on (struct mtk_vcodec_ctx *ctx); +void mtk_vcodec_enc_power_off (struct mtk_vcodec_ctx *ctx);void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm);void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm);
drivers/media/platform/mediatek/vcodec/venc_drv_if.c +0 -6
@@ -32,9 +32,7 @@ int venc_if_init(struct mtk_vcodec_ctx *ctx, unsigned int fourcc) }mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->init (ctx); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx);return ret;
@@ -46,9 +44,7 @@ int venc_if_set_param(struct mtk_vcodec_ctx *ctx,int ret =0 ;mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->set_param (ctx->drv_handle, type, in); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx);return ret;
@@ -89,9 +85,7 @@ int venc_if_deinit(struct mtk_vcodec_ctx *ctx)return 0 ;mtk_venc_lock (ctx); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); ret = ctx->enc_if->deinit (ctx->drv_handle); -mtk_vcodec_enc_clock_off (&ctx->dev->pm);mtk_venc_unlock (ctx); ctx->drv_handle =NULL ;
Signature · 1 line
2.18.0
PATCH v4 6/8 media: mediatek: vcodec: Refactor encoder clock on/off function
In reply to Irui Wangwhen enable multi-core encoding, encoder cores use their own clock, refactor clock management functions with used encoder hardware id.
--- .../mediatek/vcodec/mtk_vcodec_enc_pm.c | 95 ++++++++++++++++--- .../mediatek/vcodec/mtk_vcodec_enc_pm.h | 4 +- .../platform/mediatek/vcodec/venc_drv_if.c | 4 +- 3 files changed, 84 insertions(+), 19 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c +80 -15
@@ -60,7 +60,9 @@ EXPORT_SYMBOL_GPL(mtk_vcodec_init_enc_clk);static int mtk_enc_core_power_on (struct mtk_vcodec_ctx *ctx) {struct mtk_venc_hw_dev *sub_core; +struct mtk_vcodec_clk *clk;int ret, i; +int j =0 ;/* multi-core encoding need power on all available cores */ for (i =0 ; i <MTK_VENC_HW_MAX ; i++) {
@@ -73,12 +75,27 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)mtk_v4l2_err ("power on sub_core[%d] fail %d" , i, ret); goto pm_on_fail; } + + clk = &sub_core->pm.venc_clk; +for (j =0 ; j < clk->clk_num; j++) { + ret =clk_prepare (clk->clk_info[j].vcodec_clk); +if (ret) { +mtk_v4l2_err ("prepare clk [%s] fail %d" , + clk->clk_info[j].clk_name, ret); + goto pm_on_fail; + } + } }return ret; pm_on_fail:for (i -=1 ; i >=0 ; i--) { sub_core = (struct mtk_venc_hw_dev *)ctx->dev->enc_hw_dev[i]; + + clk = &sub_core->pm.venc_clk; +for (j -=1 ; j >=0 ; j--) +clk_unprepare (clk->clk_info[j].vcodec_clk); +pm_runtime_put_sync (&sub_core->plat_dev->dev); }return ret;
@@ -87,7 +104,9 @@ static int mtk_enc_core_power_on(struct mtk_vcodec_ctx *ctx)int mtk_vcodec_enc_power_on (struct mtk_vcodec_ctx *ctx) {struct mtk_vcodec_pm *pm = &ctx->dev->pm; +struct mtk_vcodec_clk *clk;int ret; +int i =0 ; ret =pm_runtime_resume_and_get (pm->dev);if (ret) {
@@ -95,6 +114,16 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)return ret; } + clk = &pm->venc_clk; +for (i =0 ; i < clk->clk_num; i++) { + ret =clk_prepare (clk->clk_info[i].vcodec_clk); +if (ret) { +mtk_v4l2_err ("prepare clk [%s] fail %d" , + clk->clk_info[i].clk_name, ret); + goto clk_error; + } + } +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) { ret =mtk_enc_core_power_on (ctx);if (ret) {
@@ -104,6 +133,9 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx) }return ret; +clk_error: +for (i -=1 ; i >=0 ; i--) +clk_unprepare (clk->clk_info[i].vcodec_clk); core_error:pm_runtime_put_sync (pm->dev);return ret;
@@ -112,7 +144,8 @@ int mtk_vcodec_enc_power_on(struct mtk_vcodec_ctx *ctx)static void mtk_enc_core_power_off (struct mtk_vcodec_ctx *ctx) {struct mtk_venc_hw_dev *sub_core; -int ret, i; +struct mtk_vcodec_clk *clk; +int i, ret, j;/* multi-core encoding need power off all available cores */ for (i =0 ; i <MTK_VENC_HW_MAX ; i++) {
@@ -120,6 +153,10 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)if (!sub_core)continue ; + clk = &sub_core->pm.venc_clk; +for (j = clk->clk_num -1 ; j >=0 ; j--) +clk_unprepare (clk->clk_info[j].vcodec_clk); + ret =pm_runtime_put_sync (&sub_core->plat_dev->dev);if (ret)mtk_v4l2_err ("power off sub_core[%d] fail %d" , i, ret);
@@ -129,27 +166,44 @@ static void mtk_enc_core_power_off(struct mtk_vcodec_ctx *ctx)void mtk_vcodec_enc_power_off (struct mtk_vcodec_ctx *ctx) {struct mtk_vcodec_pm *pm = &ctx->dev->pm; -int ret; - - ret =pm_runtime_put_sync (pm->dev); -if (ret) -mtk_v4l2_err ("pm_runtime_put_sync fail %d" , ret); +struct mtk_vcodec_clk *clk; +int ret, i;if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) {mtk_enc_core_power_off (ctx); } + + clk = &pm->venc_clk; +for (i = clk->clk_num -1 ; i >=0 ; i--) +clk_unprepare (clk->clk_info[i].vcodec_clk); + + ret =pm_runtime_put_sync (pm->dev); +if (ret) +mtk_v4l2_err ("pm_runtime_put_sync fail %d" , ret); } -void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_on (struct mtk_vcodec_dev *dev,int hw_id) { -struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; +struct mtk_venc_hw_dev *sub_core; +struct mtk_vcodec_clk *enc_clk; +int ret, i =0 ; +if (hw_id ==MTK_VENC_CORE_0 ) { + enc_clk = &dev->pm.venc_clk; + }else if (hw_id ==MTK_VENC_CORE_1 ) { + sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id]; + enc_clk = &sub_core->pm.venc_clk; + }else { +mtk_v4l2_err ("invalid hw id : %d" , hw_id); +return ; + } +for (i =0 ; i < enc_clk->clk_num; i++) { - ret =clk_prepare_enable (enc_clk->clk_info[i].vcodec_clk); + ret =clk_enable (enc_clk->clk_info[i].vcodec_clk);if (ret) { -mtk_v4l2_err ("venc clk_prepare_enable %d %s fail %d" , i, - enc_clk->clk_info[i].clk_name, ret); +mtk_v4l2_err ("venc clk_enable %d %s fail %d" , i, + enc_clk->clk_info[i].clk_name, ret); goto clkerr; } }
@@ -158,14 +212,25 @@ void mtk_vcodec_enc_clock_on(struct mtk_vcodec_pm *pm) clkerr:for (i -=1 ; i >=0 ; i--) -clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); +clk_disable (enc_clk->clk_info[i].vcodec_clk); } -void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm) +void mtk_vcodec_enc_clock_off (struct mtk_vcodec_dev *dev,int hw_id) { -struct mtk_vcodec_clk *enc_clk = &pm->venc_clk; +struct mtk_venc_hw_dev *sub_core; +struct mtk_vcodec_clk *enc_clk;int i =0 ; +if (hw_id ==MTK_VENC_CORE_0 ) { + enc_clk = &dev->pm.venc_clk; + }else if (hw_id ==MTK_VENC_CORE_1 ) { + sub_core = (struct mtk_venc_hw_dev *)dev->enc_hw_dev[hw_id]; + enc_clk = &sub_core->pm.venc_clk; + }else { +mtk_v4l2_err ("invalid hw id : %d" , hw_id); +return ; + } +for (i = enc_clk->clk_num -1 ; i >=0 ; i--) -clk_disable_unprepare (enc_clk->clk_info[i].vcodec_clk); +clk_disable (enc_clk->clk_info[i].vcodec_clk); }
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.h +2 -2
@@ -14,7 +14,7 @@ int mtk_vcodec_init_enc_clk(struct platform_device *pdev,int mtk_vcodec_enc_power_on (struct mtk_vcodec_ctx *ctx);void mtk_vcodec_enc_power_off (struct mtk_vcodec_ctx *ctx); -void mtk_vcodec_enc_clock_on (struct mtk_vcodec_pm *pm); -void mtk_vcodec_enc_clock_off (struct mtk_vcodec_pm *pm); +void mtk_vcodec_enc_clock_on (struct mtk_vcodec_dev *dev,int hw_id); +void mtk_vcodec_enc_clock_off (struct mtk_vcodec_dev *dev,int hw_id);#endif /* _MTK_VCODEC_ENC_PM_H_ */
drivers/media/platform/mediatek/vcodec/venc_drv_if.c +2 -2
@@ -64,10 +64,10 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx, ctx->dev->curr_ctx = ctx;spin_unlock_irqrestore (&ctx->dev->irqlock, flags); -mtk_vcodec_enc_clock_on (&ctx->dev->pm); +mtk_vcodec_enc_clock_on (ctx->dev,0 ); ret = ctx->enc_if->encode (ctx->drv_handle, opt, frm_buf, bs_buf, result); -mtk_vcodec_enc_clock_off (&ctx->dev->pm); +mtk_vcodec_enc_clock_off (ctx->dev,0 );spin_lock_irqsave (&ctx->dev->irqlock, flags); ctx->dev->curr_ctx =NULL ;
Signature · 1 line
2.18.0
PATCH v4 7/8 media: mediatek: vcodec: Add multi-core encoding process
In reply to Irui WangMT8195 has two encoder hardware cores, when enable multi-core encoding, it is possiable to trying to use the two cores. According to hardware requirements, input frame 0 uses core0, frame 1 uses core1, then frame 2 uses core0, lock the device and enable clock by used core, and for sequence header encoding, it always use core0.
--- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 8 ++- .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 10 +-- .../platform/mediatek/vcodec/mtk_vcodec_enc.h | 4 +- .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 8 ++- .../mediatek/vcodec/mtk_vcodec_enc_hw.c | 7 +- .../mediatek/vcodec/mtk_vcodec_enc_hw.h | 2 - .../mediatek/vcodec/venc/venc_h264_if.c | 16 +++-- .../mediatek/vcodec/venc/venc_vp8_if.c | 3 +- .../platform/mediatek/vcodec/venc_drv_if.c | 69 ++++++++++++++----- .../platform/mediatek/vcodec/venc_drv_if.h | 4 ++ .../platform/mediatek/vcodec/venc_vpu_if.c | 9 ++- .../platform/mediatek/vcodec/venc_vpu_if.h | 3 +- 12 files changed, 100 insertions(+), 43 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +7 -1
@@ -301,6 +301,7 @@ struct vdec_pic_info {* @max_width: hardware supported max width * @max_height: hardware supported max height * @msg_queue: msg queue used to store lat buffer information. +* @encoded_frame_cnt: number of encoded frames */ struct mtk_vcodec_ctx {enum mtk_instance_type type;
@@ -349,6 +350,8 @@ struct mtk_vcodec_ctx {unsigned int max_width;unsigned int max_height;struct vdec_msg_queue msg_queue; + +int encoded_frame_cnt; };/*
@@ -496,6 +499,7 @@ struct mtk_vcodec_enc_pdata {* @vdec_racing_info: record register value * @dec_racing_info_mutex: mutex lock used for inner racing mode * @enc_hw_dev: used to store venc core device +* @curr_enc_ctx: used to store current used enc context */ struct mtk_vcodec_dev {struct v4l2_device v4l2_dev;
@@ -529,7 +533,8 @@ struct mtk_vcodec_dev {/* decoder hardware mutex lock */ struct mutex dec_mutex[MTK_VDEC_HW_MAX ]; -struct mutex enc_mutex; +/* encoder hardware mutex lock */ +struct mutex enc_mutex[MTK_VENC_HW_MAX ];struct mtk_vcodec_pm pm;unsigned int dec_capability;
@@ -548,6 +553,7 @@ struct mtk_vcodec_dev {struct mutex dec_racing_info_mutex;void *enc_hw_dev[MTK_VENC_HW_MAX ]; +struct mtk_vcodec_ctx *curr_enc_ctx[MTK_VENC_HW_MAX ]; };static inline struct mtk_vcodec_ctx *fh_to_ctx (struct v4l2_fh *fh)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c +6 -4
@@ -1213,6 +1213,8 @@ static void mtk_venc_worker(struct work_struct *work) enc_result.bs_size); } + ctx->encoded_frame_cnt++; +v4l2_m2m_job_finish (ctx->dev->m2m_dev_enc, ctx->m2m_ctx); mtk_v4l2_debug(1, "<=== src_buf[%d] dst_buf[%d] venc_if_encode ret=%d Size=%u===>",
@@ -1418,19 +1420,19 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,return vb2_queue_init (dst_vq); } -int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx) +int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx,int hw_id) {struct mtk_vcodec_dev *dev = ctx->dev; -mutex_unlock (&dev->enc_mutex); +mutex_unlock (&dev->enc_mutex[hw_id]);return 0 ; } -int mtk_venc_lock (struct mtk_vcodec_ctx *ctx) +int mtk_venc_lock (struct mtk_vcodec_ctx *ctx,int hw_id) {struct mtk_vcodec_dev *dev = ctx->dev; -mutex_lock (&dev->enc_mutex); +mutex_lock (&dev->enc_mutex[hw_id]);return 0 ; }
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h +2 -2
@@ -39,8 +39,8 @@ struct mtk_video_enc_buf {extern const struct v4l2_ioctl_ops mtk_venc_ioctl_ops;extern const struct v4l2_m2m_ops mtk_venc_m2m_ops; -int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx); -int mtk_venc_lock (struct mtk_vcodec_ctx *ctx); +int mtk_venc_unlock (struct mtk_vcodec_ctx *ctx,int hw_id); +int mtk_venc_lock (struct mtk_vcodec_ctx *ctx,int hw_id);int mtk_vcodec_enc_queue_init (void *priv,struct vb2_queue *src_vq,struct vb2_queue *dst_vq);void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx);
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +5 -3
@@ -91,7 +91,7 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)void __iomem *addr;spin_lock_irqsave (&dev->irqlock, flags); - ctx = dev->curr_ctx; + ctx = dev->curr_enc_ctx[MTK_VENC_CORE_0 ];spin_unlock_irqrestore (&dev->irqlock, flags);mtk_v4l2_debug (1 ,"id=%d coreid:%d" , ctx->id, dev->venc_pdata->core_id);
@@ -231,7 +231,7 @@ static int mtk_vcodec_probe(struct platform_device *pdev)struct resource *res;phandle rproc_phandle;enum mtk_vcodec_fw_type fw_type; -int ret; +int ret, i; dev =devm_kzalloc (&pdev->dev,sizeof (*dev),GFP_KERNEL );if (!dev)
@@ -302,7 +302,9 @@ static int mtk_vcodec_probe(struct platform_device *pdev) goto err_res; } -mutex_init (&dev->enc_mutex); +for (i =0 ; i <MTK_VENC_HW_MAX ; i++) +mutex_init (&dev->enc_mutex[i]); +mutex_init (&dev->dev_mutex);spin_lock_init (&dev->irqlock);
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c +4 -3
@@ -48,13 +48,14 @@ static void clean_hw_irq_status(unsigned int irq_status, void __iomem *addr)static irqreturn_t mtk_enc_hw_irq_handler (int irq,void *priv) {struct mtk_venc_hw_dev *dev = priv; +struct mtk_vcodec_dev *main_dev = dev->main_dev;struct mtk_vcodec_ctx *ctx;unsigned long flags;void __iomem *addr; -spin_lock_irqsave (&dev->main_dev->irqlock, flags); - ctx = dev->curr_ctx; -spin_unlock_irqrestore (&dev->main_dev->irqlock, flags); +spin_lock_irqsave (&main_dev->irqlock, flags); + ctx = main_dev->curr_enc_ctx[dev->hw_id]; +spin_unlock_irqrestore (&main_dev->irqlock, flags);if (!ctx)return IRQ_HANDLED ;
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.h +0 -2
@@ -14,7 +14,6 @@* @plat_dev: platform_device * @main_dev: main device * @pm: power management data -* @curr_ctx: the context that is waiting for venc hardware * @reg_base: mapped address of venc registers * @irq_status: venc hardware irq status * @enc_irq: venc device irq
@@ -25,7 +24,6 @@ struct mtk_venc_hw_dev {struct mtk_vcodec_dev *main_dev;struct mtk_vcodec_pm pm; -struct mtk_vcodec_ctx *curr_ctx;void __iomem *reg_base;unsigned int irq_status;
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +10 -6
@@ -446,7 +446,8 @@ static int h264_encode_sps(struct venc_h264_inst *inst,mtk_vcodec_debug_enter (inst); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_SPS ,NULL , bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_SPS ,NULL , + bs_buf,NULL ,MTK_VENC_CORE_0 );if (ret)return ret;
@@ -472,7 +473,8 @@ static int h264_encode_pps(struct venc_h264_inst *inst,mtk_vcodec_debug_enter (inst); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_PPS ,NULL , bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_PPS ,NULL , + bs_buf,NULL ,MTK_VENC_CORE_0 );if (ret)return ret;
@@ -514,7 +516,8 @@ static int h264_encode_header(struct venc_h264_inst *inst,static int h264_encode_frame (struct venc_h264_inst *inst,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -unsigned int *bs_size) +unsigned int *bs_size, +int hw_id) {int ret =0 ;unsigned int irq_status;
@@ -532,7 +535,8 @@ static int h264_encode_frame(struct venc_h264_inst *inst,mtk_vcodec_debug (inst,"frm_count = %d,skip_frm_count =%d,frm_type=%d.\n" , frame_info.frm_count, frame_info.skip_frm_count, frame_info.frm_type); - ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_FRAME , frm_buf, bs_buf, &frame_info); + ret =vpu_enc_encode (&inst->vpu_inst,H264_BS_MODE_FRAME , frm_buf, + bs_buf, &frame_info, hw_id);if (ret)return ret;
@@ -660,7 +664,7 @@ static int h264_enc_encode(void *handle,if (!inst->prepend_hdr) { ret =h264_encode_frame (inst, frm_buf, bs_buf, - &result->bs_size); + &result->bs_size, ctx->hw_id);if (ret) goto encode_err; result->is_key_frm = inst->vpu_inst.is_key_frm;
@@ -688,7 +692,7 @@ static int h264_enc_encode(void *handle, tmp_bs_buf.size = bs_buf->size - (hdr_sz + filler_sz); ret =h264_encode_frame (inst, frm_buf, &tmp_bs_buf, - &bs_size_frm); + &bs_size_frm, ctx->hw_id);if (ret) goto encode_err;
drivers/media/platform/mediatek/vcodec/venc/venc_vp8_if.c +2 -1
@@ -302,7 +302,8 @@ static int vp8_enc_encode_frame(struct venc_vp8_inst *inst,mtk_vcodec_debug (inst,"->frm_cnt=%d" , inst->frm_cnt); - ret =vpu_enc_encode (&inst->vpu_inst,0 , frm_buf, bs_buf,NULL ); + ret =vpu_enc_encode (&inst->vpu_inst,0 , frm_buf, + bs_buf,NULL ,MTK_VENC_CORE_0 );if (ret)return ret;
drivers/media/platform/mediatek/vcodec/venc_drv_if.c +50 -19
109 lines not on this page · raw message
drivers/media/platform/mediatek/vcodec/venc_drv_if.h +4 -0
@@ -167,4 +167,8 @@ int venc_if_encode(struct mtk_vcodec_ctx *ctx,struct mtk_vcodec_mem *bs_buf, struct venc_done_result *result); +void venc_encode_prepare (struct mtk_vcodec_ctx *ctx, +enum venc_start_opt opt); +void venc_encode_unprepare (struct mtk_vcodec_ctx *ctx, +enum venc_start_opt opt,int ret);#endif /* _VENC_DRV_IF_H_ */
drivers/media/platform/mediatek/vcodec/venc_vpu_if.c +8 -1
@@ -225,7 +225,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,int vpu_enc_encode (struct venc_vpu_inst *vpu,unsigned int bs_mode,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -struct venc_frame_info *frame_info) +struct venc_frame_info *frame_info, +int hw_id) {const bool is_ext =MTK_ENC_CTX_IS_EXT (vpu->ctx); size_t msg_size = is_ext ?
@@ -261,6 +262,12 @@ int vpu_enc_encode(struct venc_vpu_inst *vpu, unsigned int bs_mode, out.data[1 ] = frame_info->skip_frm_count; out.data[2 ] = frame_info->frm_type; } + +if (IS_VENC_MULTICORE (vpu->ctx->dev->enc_capability)) { + out.data_item =4 ; + out.data[3 ] = hw_id; + } +if (vpu_enc_send_msg (vpu, &out, msg_size)) {mtk_vcodec_err (vpu,"AP_IPIMSG_ENC_ENCODE %d fail" , bs_mode);
drivers/media/platform/mediatek/vcodec/venc_vpu_if.h +2 -1
@@ -45,7 +45,8 @@ int vpu_enc_set_param(struct venc_vpu_inst *vpu,int vpu_enc_encode (struct venc_vpu_inst *vpu,unsigned int bs_mode,struct venc_frm_buf *frm_buf,struct mtk_vcodec_mem *bs_buf, -struct venc_frame_info *frame_info); +struct venc_frame_info *frame_info, +int hw_id);int vpu_enc_deinit (struct venc_vpu_inst *vpu);#endif
Signature · 1 line
2.18.0
PATCH v4 8/8 media: mediatek: vcodec: Return encoding result in asynchronous mode
In reply to Irui Wangwhen enable multi-core encoding, the wait IRQ done synchronous function should not be called, so the encoding result can't return to client in device_run. Move the buffer done function in IRQ handler.
--- .../platform/mediatek/vcodec/mtk_vcodec_drv.h | 6 ++ .../platform/mediatek/vcodec/mtk_vcodec_enc.c | 73 +++++++++++++++++-- .../platform/mediatek/vcodec/mtk_vcodec_enc.h | 7 +- .../mediatek/vcodec/mtk_vcodec_enc_drv.c | 28 ++++++- .../mediatek/vcodec/mtk_vcodec_enc_hw.c | 13 +++- .../mediatek/vcodec/mtk_vcodec_enc_pm.c | 1 + .../mediatek/vcodec/mtk_vcodec_util.h | 1 + .../mediatek/vcodec/venc/venc_h264_if.c | 20 +++-- .../platform/mediatek/vcodec/venc_drv_if.h | 2 + 9 files changed, 138 insertions(+), 13 deletions(-)
drivers/media/platform/mediatek/vcodec/mtk_vcodec_drv.h +6 -0
@@ -302,6 +302,9 @@ struct vdec_pic_info {* @max_height: hardware supported max height * @msg_queue: msg queue used to store lat buffer information. * @encoded_frame_cnt: number of encoded frames +* @pfrm_buf: used to store current ctx's frame buffer +* @pbs_buf: used to store current ctx's bitstream buffer +* @hdr_size: used to store prepend header size */ struct mtk_vcodec_ctx {enum mtk_instance_type type;
@@ -352,6 +355,9 @@ struct mtk_vcodec_ctx {struct vdec_msg_queue msg_queue;int encoded_frame_cnt; +struct vb2_v4l2_buffer *pfrm_buf[MTK_VENC_HW_MAX ]; +struct vb2_v4l2_buffer *pbs_buf[MTK_VENC_HW_MAX ]; +unsigned int hdr_size; };/*
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.c +68 -5
118 lines not on this page · raw message
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc.h +6 -1
@@ -20,6 +20,9 @@#define MTK_VENC_IRQ_STATUS_OFFSET 0x05C#define MTK_VENC_IRQ_ACK_OFFSET 0x060 +#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098 +#define VENC_PIC_FRM_TYPE 0x0010 +#define VENC_PIC_KEY_FRM 0x2/** * struct mtk_video_enc_buf - Private data related to each VB2 buffer.
@@ -46,5 +49,7 @@ int mtk_vcodec_enc_queue_init(void *priv, struct vb2_queue *src_vq,void mtk_vcodec_enc_release (struct mtk_vcodec_ctx *ctx);int mtk_vcodec_enc_ctrls_setup (struct mtk_vcodec_ctx *ctx);void mtk_vcodec_enc_set_default_params (struct mtk_vcodec_ctx *ctx); - +void mtk_venc_buf_done (struct mtk_vcodec_ctx *ctx,int hw_id, +unsigned int bs_size,bool time_out,bool key_frame); +void mtk_venc_lock_all (struct mtk_vcodec_ctx *ctx);#endif /* _MTK_VCODEC_ENC_H_ */
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_drv.c +27 -1
@@ -89,6 +89,9 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv)struct mtk_vcodec_ctx *ctx;unsigned long flags;void __iomem *addr; +unsigned int bs_size; +unsigned int frm_type; +bool is_key_frame =0 ;spin_lock_irqsave (&dev->irqlock, flags); ctx = dev->curr_enc_ctx[MTK_VENC_CORE_0 ];
@@ -101,8 +104,32 @@ static irqreturn_t mtk_vcodec_enc_irq_handler(int irq, void *priv) ctx->irq_status =readl (dev->reg_base[dev->venc_pdata->core_id] + (MTK_VENC_IRQ_STATUS_OFFSET )); + bs_size =readl (dev->reg_base[dev->venc_pdata->core_id] + + (VENC_PIC_BITSTREAM_BYTE_CNT )); + frm_type =readl (dev->reg_base[dev->venc_pdata->core_id] + + (VENC_PIC_FRM_TYPE )); +clean_irq_status (ctx->irq_status, addr); +if (IS_VENC_MULTICORE (dev->enc_capability)) { +if (ctx->irq_status &MTK_VENC_IRQ_STATUS_FRM ) { +if (ctx->hdr_size !=0 ) { + bs_size += ctx->hdr_size; + ctx->hdr_size =0 ; + } + +if (frm_type &VENC_PIC_KEY_FRM ) + is_key_frame =1 ; + +mtk_venc_buf_done (ctx,0 , bs_size,0 , is_key_frame); +mtk_vcodec_enc_clock_off (dev,0 ); +mtk_venc_unlock (ctx,0 ); + }else { +wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); + } +return IRQ_HANDLED ; + } +wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 );return IRQ_HANDLED ; }
@@ -290,7 +317,6 @@ static int mtk_vcodec_probe(struct platform_device *pdev) } dev->enc_irq =platform_get_irq (pdev,0 ); -irq_set_status_flags (dev->enc_irq,IRQ_NOAUTOEN ); ret =devm_request_irq (&pdev->dev, dev->enc_irq, mtk_vcodec_enc_irq_handler,0 , pdev->name, dev);
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_hw.c +12 -1
@@ -52,6 +52,9 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv)struct mtk_vcodec_ctx *ctx;unsigned long flags;void __iomem *addr; +unsigned int bs_size; +unsigned int frm_type; +bool is_key_frame =0 ;spin_lock_irqsave (&main_dev->irqlock, flags); ctx = main_dev->curr_enc_ctx[dev->hw_id];
@@ -63,9 +66,17 @@ static irqreturn_t mtk_enc_hw_irq_handler(int irq, void *priv) addr = dev->reg_base +MTK_VENC_IRQ_ACK_OFFSET ; ctx->irq_status =readl (dev->reg_base +MTK_VENC_IRQ_STATUS_OFFSET ); + bs_size =readl (dev->reg_base +VENC_PIC_BITSTREAM_BYTE_CNT ); + frm_type =readl (dev->reg_base +VENC_PIC_FRM_TYPE );clean_hw_irq_status (ctx->irq_status, addr); -wake_up_ctx (ctx,MTK_INST_IRQ_RECEIVED ,0 ); +if (frm_type &VENC_PIC_KEY_FRM ) + is_key_frame =1 ; + +mtk_venc_buf_done (ctx, dev->hw_id, bs_size,0 , is_key_frame); +mtk_vcodec_enc_clock_off (main_dev, dev->hw_id); +mtk_venc_unlock (ctx, dev->hw_id); +return IRQ_HANDLED ; }
drivers/media/platform/mediatek/vcodec/mtk_vcodec_enc_pm.c +1 -0
@@ -234,3 +234,4 @@ void mtk_vcodec_enc_clock_off(struct mtk_vcodec_dev *dev, int hw_id)for (i = enc_clk->clk_num -1 ; i >=0 ; i--)clk_disable (enc_clk->clk_info[i].vcodec_clk); } +EXPORT_SYMBOL_GPL (mtk_vcodec_enc_clock_off);
drivers/media/platform/mediatek/vcodec/mtk_vcodec_util.h +1 -0
@@ -15,6 +15,7 @@ struct mtk_vcodec_mem {size_t size;void *va;dma_addr_t dma_addr; +void *buf; };struct mtk_vcodec_fb {
drivers/media/platform/mediatek/vcodec/venc/venc_h264_if.c +15 -5
@@ -22,7 +22,6 @@static const char h264_filler_marker[] = {0x0 ,0x0 ,0x0 ,0x1 ,0xc };#define H264_FILLER_MARKER_SIZE ARRAY_SIZE(h264_filler_marker) -#define VENC_PIC_BITSTREAM_BYTE_CNT 0x0098/* * enum venc_h264_frame_type - h264 encoder output bitstream frame type
@@ -554,6 +553,11 @@ static int h264_encode_frame(struct venc_h264_inst *inst,return ret; } +if (IS_VENC_MULTICORE (ctx->dev->enc_capability)) { + ++inst->frm_cnt; +return ret; + } + irq_status =h264_enc_wait_venc_done (inst);if (irq_status !=MTK_VENC_IRQ_STATUS_FRM ) {mtk_vcodec_err (inst,"irq_status=%d failed" , irq_status);
@@ -638,8 +642,6 @@ static int h264_enc_encode(void *handle,mtk_vcodec_debug (inst,"opt %d ->" , opt); -enable_irq (ctx->dev->enc_irq); -switch (opt) {case VENC_START_OPT_ENCODE_SEQUENCE_HEADER : {unsigned int bs_size_hdr;
@@ -662,6 +664,13 @@ static int h264_enc_encode(void *handle,unsigned int bs_size_hdr;unsigned int bs_size_frm; +/* +* the frm_buf and bs_buf need to recorded into current ctx, +* when encoding done, the target buffers can be got from ctx. +*/ + ctx->pfrm_buf[ctx->hw_id] = frm_buf->frm_addr; + ctx->pbs_buf[ctx->hw_id] = bs_buf->buf; +if (!inst->prepend_hdr) { ret =h264_encode_frame (inst, frm_buf, bs_buf, &result->bs_size, ctx->hw_id);
@@ -696,7 +705,9 @@ static int h264_enc_encode(void *handle,if (ret) goto encode_err; - result->bs_size = hdr_sz + filler_sz + bs_size_frm; + ctx->hdr_size = hdr_sz + filler_sz; + + result->bs_size = ctx->hdr_size + bs_size_frm; mtk_vcodec_debug(inst, "hdr %d filler %d frame %d bs %d", hdr_sz, filler_sz, bs_size_frm,
@@ -715,7 +726,6 @@ static int h264_enc_encode(void *handle, encode_err: -disable_irq (ctx->dev->enc_irq);mtk_vcodec_debug (inst,"opt %d <-" , opt);return ret;
drivers/media/platform/mediatek/vcodec/venc_drv_if.h +2 -0
@@ -108,9 +108,11 @@ struct venc_frame_info {/* * struct venc_frm_buf - frame buffer information used in venc_if_encode() * @fb_addr: plane frame buffer addresses +* @frm_addr: current v4l2 buffer address */ struct venc_frm_buf {struct mtk_vcodec_fb fb_addr[MTK_VCODEC_MAX_PLANES ]; +void *frm_addr; };/*
Signature · 1 line
2.18.0