Skip to contentSkip to conversation

Add Rockchip VPU JPEG encoder

PATCH Superseded Ezequiel Garcia 1 participant 4 messages Aug 31, 2018 1 message not in the archive

Subsystems (3)
Device tree bindings Media (V4L/DVB) Arm Rockchip

v4 of 11 Superseded Latest is v11 →

lore
Share & tools
lore

Ezequiel Garcia

author
#0 lore raw

v4 cover Add Rockchip VPU JPEG encoder

This series adds support for JPEG encoding via the VPU block present in Rockchip platforms. Currently, support for RK3288 and RK3399 is included.

This v4 patchset changes quantization luma and chroma array controls, and instead introduces a compound control, V4L2_CID_JPEG_QUANTIZATION for userspace to specify the luma and chroma quantization tables, via a struct.

Rest of the changes are the same as v3.

This series does not depend on the Request API, since there are some encoder uses that doesn't require it. So, if there are no more comments, I'd like to merge this sooner than later.

Compliance
==========

(Same results as v3)

v4l2-compliance SHA: d0f4ea7ddab6d0244c4fe1e960bb2aaeefb911b9, 64 bits

Compliance test for device /dev/video0:

Driver Info:

	Driver name      : rockchip-vpu
	Card type        : rockchip,rk3399-vpu-enc
	Bus info         : platform: rockchip-vpu
	Driver version   : 4.18.0
	Capabilities     : 0x84204000

Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Device Capabilities
Device Caps : 0x04204000
Video Memory-to-Memory Multiplanar Streaming
Extended Pix Format
Media Driver Info:

	Driver name      : rockchip-vpu
	Model            : rockchip-vpu
	Serial           : 
	Bus info         : 
	Media version    : 4.18.0
	Hardware revision: 0x00000000 (0)
	Driver version   : 4.18.0
Interface Info:
	ID               : 0x0300000c
	Type             : V4L Video
Entity Info:
	ID               : 0x00000001 (1)
	Name             : rockchip,rk3399-vpu-enc-source
	Function         : V4L2 I/O
	Pad 0x01000002   : Source
	  Link 0x02000008: to remote pad 0x1000005 of entity 'rockchip,rk3399-vpu-enc-proc': Data, Enabled, Immutable

Required ioctls:
test MC information (see 'Media Driver Info' above): OK test VIDIOC_QUERYCAP: OK

Allow for multiple opens:
test second /dev/video0 open: OK
test VIDIOC_QUERYCAP: OK
test VIDIOC_G/S_PRIORITY: OK
test for unlimited opens: OK

Debug ioctls:
test VIDIOC_DBG_G/S_REGISTER: OK (Not Supported) test VIDIOC_LOG_STATUS: OK (Not Supported)

Input ioctls:
test VIDIOC_G/S_TUNER/ENUM_FREQ_BANDS: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_S_HW_FREQ_SEEK: OK (Not Supported) test VIDIOC_ENUMAUDIO: OK (Not Supported)
test VIDIOC_G/S/ENUMINPUT: OK (Not Supported) test VIDIOC_G/S_AUDIO: OK (Not Supported)
Inputs: 0 Audio Inputs: 0 Tuners: 0

Output ioctls:
test VIDIOC_G/S_MODULATOR: OK (Not Supported) test VIDIOC_G/S_FREQUENCY: OK (Not Supported) test VIDIOC_ENUMAUDOUT: OK (Not Supported) test VIDIOC_G/S/ENUMOUTPUT: OK (Not Supported) test VIDIOC_G/S_AUDOUT: OK (Not Supported)
Outputs: 0 Audio Outputs: 0 Modulators: 0

Input/Output configuration ioctls:
test VIDIOC_ENUM/G/S/QUERY_STD: OK (Not Supported) test VIDIOC_ENUM/G/S/QUERY_DV_TIMINGS: OK (Not Supported) test VIDIOC_DV_TIMINGS_CAP: OK (Not Supported) test VIDIOC_G/S_EDID: OK (Not Supported)

Control ioctls:
test VIDIOC_QUERY_EXT_CTRL/QUERYMENU: OK test VIDIOC_QUERYCTRL: OK
test VIDIOC_G/S_CTRL: OK
test VIDIOC_G/S/TRY_EXT_CTRLS: OK test VIDIOC_(UN)SUBSCRIBE_EVENT/DQEVENT: OK test VIDIOC_G/S_JPEGCOMP: OK (Not Supported) Standard Controls: 2 Private Controls: 0

Format ioctls:
test VIDIOC_ENUM_FMT/FRAMESIZES/FRAMEINTERVALS: OK test VIDIOC_G/S_PARM: OK (Not Supported) test VIDIOC_G_FBUF: OK (Not Supported) test VIDIOC_G_FMT: OK
test VIDIOC_TRY_FMT: OK
test VIDIOC_S_FMT: OK
test VIDIOC_G_SLICED_VBI_CAP: OK (Not Supported) test Cropping: OK (Not Supported)
test Composing: OK (Not Supported)
test Scaling: OK

Codec ioctls:
test VIDIOC_(TRY_)ENCODER_CMD: OK (Not Supported) test VIDIOC_G_ENC_INDEX: OK (Not Supported) test VIDIOC_(TRY_)DECODER_CMD: OK (Not Supported)

Buffer ioctls:
test VIDIOC_REQBUFS/CREATE_BUFS/QUERYBUF: OK test VIDIOC_EXPBUF: OK

Test input 0:

Streaming ioctls:
test read/write: OK (Not Supported)
test blocking wait: OK
test MMAP: OK
test USERPTR: OK (Not Supported)
test DMABUF: Cannot test, specify --expbuf-device

Total: 48, Succeeded: 48, Failed: 0, Warnings: 0

v4:
- Change luma and chroma array controls, with a compound control, as suggested by Paul.
v3:
- Refactor driver to allow a more elegant integration with other codec modes (h264 decoding, jpeg decoding, etc). Each variant can now be encoders, decoders or both.
- Register driver in the media controller framework,
in preparation for the Request API.
- Set values for JPEG quantization controls in the core, as suggested by Tomasz and Hans.
- Move pm_runtime_get/put to run/done, reducing power consumption. This was possible thanks to Miouyouyou, who pointed out the power domains missing [1].
- Use bulk clock API for simpler code.
v2:
- Add devicetree binding documentation and devicetree changes
- Add documentation to added pixel format and controls
- Address Hans' review comments
- Get rid of unused running_ctx field
- Fix wrong planar pixel format depths
- Other minor changes for v4l2-compliance
- Drop .crop support, we will add support for the
selector API later, if needed.

[1] https://github.com/Miouyouyou/RockMyy/blob/master/patches/kernel/v4.18/DTS/0026-ARM-DTSI-rk3288-Set-the-VPU-MMU-power-domains.patch

Ezequiel Garcia (4):
dt-bindings: Document the Rockchip VPU bindings
ARM: dts: rockchip: add VPU device node for RK3288
arm64: dts: rockchip: add VPU device node for RK3399
media: add Rockchip VPU JPEG encoder driver

Shunqian Zheng (2):
media: Add JPEG_RAW format
media: Add controls for JPEG quantization tables

 .../bindings/media/rockchip-vpu.txt           |  30 +
 .../media/uapi/v4l/extended-controls.rst      |  23 +
 .../media/uapi/v4l/pixfmt-compressed.rst      |   9 +
 .../media/videodev2.h.rst.exceptions          |   1 +
 MAINTAINERS                                   |   7 +
 arch/arm/boot/dts/rk3288.dtsi                 |  14 +-
 arch/arm64/boot/dts/rockchip/rk3399.dtsi      |  14 +-
 drivers/media/platform/Kconfig                |  13 +
 drivers/media/platform/Makefile               |   1 +
 drivers/media/platform/rockchip/vpu/Makefile  |   9 +
 .../platform/rockchip/vpu/rk3288_vpu_hw.c     | 123 ++++
 .../rockchip/vpu/rk3288_vpu_hw_jpege.c        | 126 ++++
 .../platform/rockchip/vpu/rk3288_vpu_regs.h   | 442 +++++++++++++
 .../platform/rockchip/vpu/rk3399_vpu_hw.c     | 124 ++++
 .../rockchip/vpu/rk3399_vpu_hw_jpege.c        | 154 +++++
 .../platform/rockchip/vpu/rk3399_vpu_regs.h   | 601 +++++++++++++++++
 .../platform/rockchip/vpu/rockchip_vpu.h      | 362 +++++++++++
 .../rockchip/vpu/rockchip_vpu_common.h        |  37 ++
 .../platform/rockchip/vpu/rockchip_vpu_drv.c  | 545 ++++++++++++++++
 .../platform/rockchip/vpu/rockchip_vpu_enc.c  | 607 ++++++++++++++++++
 .../platform/rockchip/vpu/rockchip_vpu_hw.h   |  65 ++
 drivers/media/v4l2-core/v4l2-ctrls.c          |  10 +
 drivers/media/v4l2-core/v4l2-ioctl.c          |   1 +
 include/uapi/linux/v4l2-controls.h            |   5 +
 include/uapi/linux/videodev2.h                |   2 +
 25 files changed, 3323 insertions(+), 2 deletions(-)
 create mode 100644 Documentation/devicetree/bindings/media/rockchip-vpu.txt
 create mode 100644 drivers/media/platform/rockchip/vpu/Makefile
 create mode 100644 drivers/media/platform/rockchip/vpu/rk3288_vpu_hw.c
 create mode 100644 drivers/media/platform/rockchip/vpu/rk3288_vpu_hw_jpege.c
 create mode 100644 drivers/media/platform/rockchip/vpu/rk3288_vpu_regs.h
 create mode 100644 drivers/media/platform/rockchip/vpu/rk3399_vpu_hw.c
 create mode 100644 drivers/media/platform/rockchip/vpu/rk3399_vpu_hw_jpege.c
 create mode 100644 drivers/media/platform/rockchip/vpu/rk3399_vpu_regs.h
 create mode 100644 drivers/media/platform/rockchip/vpu/rockchip_vpu.h
 create mode 100644 drivers/media/platform/rockchip/vpu/rockchip_vpu_common.h
 create mode 100644 drivers/media/platform/rockchip/vpu/rockchip_vpu_drv.c
 create mode 100644 drivers/media/platform/rockchip/vpu/rockchip_vpu_enc.c
 create mode 100644 drivers/media/platform/rockchip/vpu/rockchip_vpu_hw.h
Signature · 1 line
2.18.0

A message that is not in the archive (sent off-list or not yet received): 20180831154113.18872-1-ezequiel-ZGY8ohtN/[email protected]

author
#2 lore raw

PATCH v4 1/6 dt-bindings: Document the Rockchip VPU bindings

Add devicetree binding documentation for Rockchip Video Processing Unit IP.

Reviewed-by Rob Herring <[email protected]>
Signed-off-by Ezequiel Garcia <ezequiel-ZGY8ohtN/[email protected]>
---
 .../bindings/media/rockchip-vpu.txt           | 30 +++++++++++++++++++
 1 file changed, 30 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/rockchip-vpu.txt
Documentation/devicetree/bindings/media/rockchip-vpu.txt new file +30 -0
new file mode 100644
index 000000000000..5e0d421301ca
@@ -0,0 +1,30 @@
+device-tree bindings for rockchip VPU codec
+
+Rockchip (Video Processing Unit) present in various Rockchip platforms,
+such as RK3288 and RK3399.
+
+Required properties:
+- compatible: value should be one of the following
+		"rockchip,rk3288-vpu";
+		"rockchip,rk3399-vpu";
+- interrupts: encoding and decoding interrupt specifiers
+- interrupt-names: should be "vepu" and "vdpu"
+- clocks: phandle to VPU aclk, hclk clocks
+- clock-names: should be "aclk" and "hclk"
+- power-domains: phandle to power domain node
+- iommus: phandle to a iommu node
+
+Example:
+SoC-specific DT entry:
+	vpu: video-codec@ff9a0000 {
+		compatible = "rockchip,rk3288-vpu";
+		reg = <0x0 0xff9a0000 0x0 0x800>;
+		interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>,
+			     <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>;
+		interrupt-names = "vepu", "vdpu";
+		clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
+		clock-names = "aclk", "hclk";
+		power-domains = <&power RK3288_PD_VIDEO>;
+		iommus = <&vpu_mmu>;
+	};
+
Signature · 1 line
2.18.0
author
#3 lore raw

PATCH v4 2/6 ARM: dts: rockchip: add VPU device node for RK3288

In reply to a message not in the archive

Add the Video Processing Unit node for RK3288 SoC.

Fix the VPU IOMMU node, which was disabled and lacking its power domain property.

Signed-off-by Ezequiel Garcia <ezequiel-ZGY8ohtN/[email protected]>
---
 arch/arm/boot/dts/rk3288.dtsi | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
arch/arm/boot/dts/rk3288.dtsi +13 -1
index d7e49d29ace5..093f941ae273 100644
@@ -1209,6 +1209,18 @@
 		};
 	};
 
+	vpu: video-codec@ff9a0000 {
+		compatible = "rockchip,rk3288-vpu";
+		reg = <0x0 0xff9a0000 0x0 0x800>;
+		interrupts = <GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>,
+			     <GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>;
+		interrupt-names = "vepu", "vdpu";
+		clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
+		clock-names = "aclk", "hclk";
+		power-domains = <&power RK3288_PD_VIDEO>;
+		iommus = <&vpu_mmu>;
+	};
+
 	vpu_mmu: iommu@ff9a0800 {
 		compatible = "rockchip,iommu";
 		reg = <0x0 0xff9a0800 0x0 0x100>;

@@ -1216,8 +1228,8 @@
 		interrupt-names = "vpu_mmu";
 		clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
 		clock-names = "aclk", "iface";
+		power-domains = <&power RK3288_PD_VIDEO>;
 		#iommu-cells = <0>;
-		status = "disabled";
 	};
 
 	hevc_mmu: iommu@ff9c0440 {
Signature · 1 line
2.18.0
author
#4 lore raw

PATCH v4 3/6 arm64: dts: rockchip: add VPU device node for RK3399

In reply to a message not in the archive

Add the Video Processing Unit node for the RK3399 SoC.

Also, fix the VPU IOMMU node, which was disabled and lacking its power domain property.

Signed-off-by Ezequiel Garcia <ezequiel-ZGY8ohtN/[email protected]>
---
 arch/arm64/boot/dts/rockchip/rk3399.dtsi | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)
arch/arm64/boot/dts/rockchip/rk3399.dtsi +13 -1
index 6ba438427515..5dd840b3deb1 100644
@@ -1231,6 +1231,18 @@
 		status = "disabled";
 	};
 
+	vpu: video-codec@ff650000 {
+		compatible = "rockchip,rk3399-vpu";
+		reg = <0x0 0xff650000 0x0 0x800>;
+		interrupts = <GIC_SPI 114 IRQ_TYPE_LEVEL_HIGH 0>,
+			     <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH 0>;
+		interrupt-names = "vepu", "vdpu";
+		clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
+		clock-names = "aclk", "hclk";
+		power-domains = <&power RK3399_PD_VCODEC>;
+		iommus = <&vpu_mmu>;
+	};
+
 	vpu_mmu: iommu@ff650800 {
 		compatible = "rockchip,iommu";
 		reg = <0x0 0xff650800 0x0 0x40>;

@@ -1238,8 +1250,8 @@
 		interrupt-names = "vpu_mmu";
 		clocks = <&cru ACLK_VCODEC>, <&cru HCLK_VCODEC>;
 		clock-names = "aclk", "iface";
+		power-domains = <&power RK3399_PD_VCODEC>;
 		#iommu-cells = <0>;
-		status = "disabled";
 	};
 
 	vdec_mmu: iommu@ff660480 {
Signature · 1 line
2.18.0