Skip to contentSkip to conversation

Support Nuvoton NPCM Video Capture/Encode Engine

PATCH In review Marvin Lin 2 participants 10 messages Oct 7, 2022 – Oct 11, 2022

Subsystems (4)
lore
Share & tools
lore

Marvin Lin

author
#0 lore raw

cover Support Nuvoton NPCM Video Capture/Encode Engine

This patch series add DTS node, dt-bindings document and drivers for Video Capture/Differentiation Engine (VCD) and Encoding Compression Engine (ECE) present on Nuvoton NPCM SoCs.

VCD can capture/differentiate video data from digital or analog sources, then the ECE will compress the data into HEXTILE format.

HEXTILE compressed format is defined in Remote Framebuffer Protocol (RFC 6143) and is used by VNC features, so we also add a patch to support it.

Marvin Lin (5):
arm64: dts: nuvoton: Add node for NPCM Video Capture/Encode Engine
dt-bindings: media: Add dt-bindings for NPCM Video Capture/Encode
Engine
dt-bindings: arm/npcm: Add dt-bindings for Graphics Core Information
media: Add HEXTILE compressed format
drivers: media: platform: Add NPCM Video Capture/Encode Engine driver

 .../bindings/arm/npcm/nuvoton,gfxi.yaml       |   41 +
 .../bindings/media/nuvoton,npcm-video.yaml    |   87 +
 .../media/v4l/pixfmt-reserved.rst             |    8 +
 MAINTAINERS                                   |    1 +
 .../dts/nuvoton/nuvoton-common-npcm8xx.dtsi   |   19 +
 .../boot/dts/nuvoton/nuvoton-npcm845-evb.dts  |   18 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/nuvoton/Kconfig        |    9 +
 drivers/media/platform/nuvoton/Makefile       |    2 +
 drivers/media/platform/nuvoton/npcm-video.c   | 2095 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |    1 +
 include/uapi/linux/videodev2.h                |    1 +
 13 files changed, 2284 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/npcm/nuvoton,gfxi.yaml
 create mode 100644 Documentation/devicetree/bindings/media/nuvoton,npcm-video.yaml
 create mode 100644 drivers/media/platform/nuvoton/Kconfig
 create mode 100644 drivers/media/platform/nuvoton/Makefile
 create mode 100644 drivers/media/platform/nuvoton/npcm-video.c
Signature · 1 line
2.34.1
author
#1 lore raw

PATCH 1/5 arm64: dts: nuvoton: Add node for NPCM Video Capture/Encode Engine

Add node for Video Capture/Differentiation Engine (VCD) and Encoding Compression Engine (ECE) present on Nuvoton NPCM SoCs.

---
 .../dts/nuvoton/nuvoton-common-npcm8xx.dtsi   | 19 +++++++++++++++++++
 .../boot/dts/nuvoton/nuvoton-npcm845-evb.dts  | 18 ++++++++++++++++++
 2 files changed, 37 insertions(+)
arch/arm64/boot/dts/nuvoton/nuvoton-common-npcm8xx.dtsi +19 -0
index 28ba3b4633bb..925e3c42278c 100644
@@ -152,6 +152,19 @@ pcimbox: pcimbox@f0848000 {
 			interrupts = <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
 		};
 
+		video: video@f0810000 {
+			compatible = "nuvoton,npcm845-video";
+			reg = <0x0 0xf0810000 0x0 0x10000>, <0x0 0xf0820000 0x0 0x2000>;
+			reg-names = "vcd", "ece";
+			interrupts = <GIC_SPI 22 IRQ_TYPE_LEVEL_HIGH>;
+			resets = <&rstc NPCM8XX_RESET_IPSRST2 NPCM8XX_RESET_VCD>,
+				 <&rstc NPCM8XX_RESET_IPSRST2 NPCM8XX_RESET_ECE>;
+			reset-names = "vcd", "ece";
+			gcr-syscon = <&gcr>;
+			gfxi-syscon = <&gfxi>;
+			status = "disabled";
+		};
+
 		apb {
 			#address-cells = <1>;
 			#size-cells = <1>;
@@ -782,6 +795,12 @@ i2c26: i2c@fff0a000 {
 				nuvoton,sys-mgr = <&gcr>;
 				status = "disabled";
 			};
+
+			gfxi: gfxi@e000 {
+				compatible = "nuvoton,npcm845-gfxi", "syscon",
+					     "simple-mfd";
+				reg = <0xe000 0x100>;
+			};
 		};
 	};
 
arch/arm64/boot/dts/nuvoton/nuvoton-npcm845-evb.dts +18 -0
index 577e031e11e4..4dbf65ab9dd2 100644
@@ -65,6 +65,19 @@ chosen {
 	memory {
 		reg = <0x0 0x0 0x0 0x40000000>;
 	};
+
+	reserved-memory {
+		#address-cells = <2>;
+		#size-cells = <2>;
+		ranges;
+
+		video_memory: framebuffer@0x33000000 {
+			compatible = "shared-dma-pool";
+			reg = <0x0 0x33000000 0x0 0x01800000>;
+			reusable;
+			linux,cma-default;
+		};
+	};
 };
 
 &serial0 {
@@ -493,3 +506,8 @@ &pin241_slew
 			&pin242_slew
 			&pin243_slew>;
 };
+
+&video {
+	status = "okay";
+	memory-region = <&video_memory>;
+};
Signature · 1 line
2.34.1
author
#2 lore raw

PATCH 2/5 dt-bindings: media: Add dt-bindings for NPCM Video Capture/Encode Engine

In reply to Marvin Lin

Add dt-bindings document for NPCM Video Capture/Encode Engine.

---
 .../bindings/media/nuvoton,npcm-video.yaml    | 87 +++++++++++++++++++
 1 file changed, 87 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/media/nuvoton,npcm-video.yaml
Documentation/devicetree/bindings/media/nuvoton,npcm-video.yaml new file +87 -0
new file mode 100644
index 000000000000..b5be7ef09038
@@ -0,0 +1,87 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/media/nuvoton,npcm-video.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Nuvoton NPCM Video Capture/Encode Engine Device Tree Bindings
+
+maintainers:
+  - Joseph Liu <[email protected]>
+  - Marvin Lin <[email protected]>
+
+description: |
+  Video Capture/Differentiation Engine (VCD) and Encoding Compression Engine
+  (ECE) present on Nuvoton NPCM SoCs.
+
+properties:
+  compatible:
+    enum:
+      - nuvoton,npcm750-video
+      - nuvoton,npcm845-video
+
+  reg:
+    items:
+      - description: VCD registers
+      - description: ECE registers
+
+  reg-names:
+    items:
+      - const: vcd
+      - const: ece
+
+  interrupts:
+    maxItems: 1
+
+  resets:
+    items:
+      - description: VCD reset control
+      - description: ECE reset control
+
+  reset-names:
+    items:
+      - const: vcd
+      - const: ece
+
+  nuvoton,syscon-gcr:
+    $ref: /schemas/types.yaml#definitions/phandle
+    description: Phandle to the Global Control Register DT node
+
+  nuvoton,syscon-gfxi:
+    $ref: /schemas/types.yaml#definitions/phandle
+    description: Phandle to the Graphics Core Information DT node
+
+  memory-region:
+    description:
+      CMA pool to use for buffers allocation instead of the default CMA pool.
+
+required:
+  - compatible
+  - reg
+  - reg-names
+  - interrupts
+  - resets
+  - reset-names
+  - nuvoton,syscon-gcr
+  - nuvoton,syscon-gfxi
+
+additionalProperties: false
+
+examples:
+  - |
+    #include <dt-bindings/interrupt-controller/arm-gic.h>
+    #include <dt-bindings/reset/nuvoton,npcm7xx-reset.h>
+
+    video: video@f0810000 {
+        compatible = "nuvoton,npcm750-video";
+        reg = <0xf0810000 0x10000>,
+              <0xf0820000 0x2000>;
+        reg-names = "vcd", "ece";
+        interrupts = <GIC_SPI 22 IRQ_TYPE_LEVEL_HIGH>;
+        resets = <&rstc NPCM7XX_RESET_IPSRST2 NPCM7XX_RESET_VCD>,
+                 <&rstc NPCM7XX_RESET_IPSRST2 NPCM7XX_RESET_ECE>;
+        reset-names = "vcd", "ece";
+        nuvoton,syscon-gcr = <&gcr>;
+        nuvoton,syscon-gfxi = <&gfxi>;
+    };
Signature · 1 line
2.34.1
author
#3 lore raw

PATCH 3/5 dt-bindings: arm/npcm: Add dt-bindings for Graphics Core Information

In reply to Marvin Lin

Add dt-bindings document for Graphics Core Information (GFXI) node. It is necessary for the NPCM Video Capture/Encode Engine driver to retrieve Graphics core information.

---
 .../bindings/arm/npcm/nuvoton,gfxi.yaml       | 41 +++++++++++++++++++
 1 file changed, 41 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/npcm/nuvoton,gfxi.yaml
Documentation/devicetree/bindings/arm/npcm/nuvoton,gfxi.yaml new file +41 -0
new file mode 100644
index 000000000000..5b785bda5739
@@ -0,0 +1,41 @@
+# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
+
+%YAML 1.2
+---
+$id: http://devicetree.org/schemas/arm/npcm/nuvoton,gfxi.yaml#
+$schema: http://devicetree.org/meta-schemas/core.yaml#
+
+title: Graphics Core Information block in Nuvoton SoCs
+
+maintainers:
+  - Joseph Liu <[email protected]>
+  - Marvin Lin <[email protected]>
+
+description:
+  The Graphics Core Information (GFXI) are a block of registers in Nuvoton SoCs
+  that analyzes Graphics core behavior and provides inforomation in registers.
+
+properties:
+  compatible:
+    items:
+      - enum:
+          - nuvoton,npcm750-gfxi
+          - nuvoton,npcm845-gfxi
+      - const: syscon
+      - const: simple-mfd
+
+  reg:
+    maxItems: 1
+
+required:
+  - compatible
+  - reg
+
+additionalProperties: false
+
+examples:
+  - |
+    gfxi: gfxi@e000 {
+      compatible = "nuvoton,npcm750-gfxi", "syscon", "simple-mfd";
+      reg = <0xe000 0x100>;
+    };
Signature · 1 line
2.34.1
author
#4 lore raw

PATCH 4/5 media: Add HEXTILE compressed format

In reply to Marvin Lin

Add HEXTILE compressed format. This format is defined in Remote Framebuffer Protocol (RFC 6143) and is used by Encoding Compression Engine present on Nuvoton NPCM SoCs.

---
 Documentation/userspace-api/media/v4l/pixfmt-reserved.rst | 8 ++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c                      | 1 +
 include/uapi/linux/videodev2.h                            | 1 +
 3 files changed, 10 insertions(+)
Documentation/userspace-api/media/v4l/pixfmt-reserved.rst +8 -0
index 0b879c0da713..719e2c4f0256 100644
@@ -261,6 +261,14 @@ please make a proposal on the linux-media mailing list.
 	and 16-aligned resolutions for the chrominance plane (with 2x2
 	subsampling).
 
+    * .. _V4L2-PIX-FMT-HEXTILE:
+
+      - ``V4L2_PIX_FMT_HEXTILE``
+      - 'HXTL'
+      - Compressed format used by Nuvoton NPCM video driver. This format is
+        defined in Remote Framebuffer Protocol (RFC 6143, chapter 7.7.4 Hextile
+        Encoding).
+
 .. raw:: latex
 
     \normalsize
drivers/media/v4l2-core/v4l2-ioctl.c +1 -0
index 7c596a85f34f..7536bc21c76d 100644
@@ -1436,6 +1436,7 @@ static void v4l_fill_fmtdesc(struct v4l2_fmtdesc *fmt)
 		case V4L2_PIX_FMT_S5C_UYVY_JPG:	descr = "S5C73MX interleaved UYVY/JPEG"; break;
 		case V4L2_PIX_FMT_MT21C:	descr = "Mediatek Compressed Format"; break;
 		case V4L2_PIX_FMT_SUNXI_TILED_NV12: descr = "Sunxi Tiled NV12 Format"; break;
+		case V4L2_PIX_FMT_HEXTILE:	descr = "Hextile Compressed Format"; break;
 		default:
 			if (fmt->description[0])
 				return;
include/uapi/linux/videodev2.h +1 -0
index 9260791b8438..6fcd1a61914a 100644
@@ -737,6 +737,7 @@ struct v4l2_pix_format {
 #define V4L2_PIX_FMT_SUNXI_TILED_NV12 v4l2_fourcc('S', 'T', '1', '2') /* Sunxi Tiled NV12 Format */
 #define V4L2_PIX_FMT_CNF4     v4l2_fourcc('C', 'N', 'F', '4') /* Intel 4-bit packed depth confidence information */
 #define V4L2_PIX_FMT_HI240    v4l2_fourcc('H', 'I', '2', '4') /* BTTV 8-bit dithered RGB */
+#define V4L2_PIX_FMT_HEXTILE  v4l2_fourcc('H', 'X', 'T', 'L') /* Hextile compression */
 
 /* 10bit raw bayer packed, 32 bytes for every 25 pixels, last LSB 6 bits unused */
 #define V4L2_PIX_FMT_IPU3_SBGGR10	v4l2_fourcc('i', 'p', '3', 'b') /* IPU3 packed 10-bit BGGR bayer */
Signature · 1 line
2.34.1
author
#5 lore raw

PATCH 5/5 drivers: media: platform: Add NPCM Video Capture/Encode Engine driver

In reply to Marvin Lin

Add driver for Video Capture/Differentiation Engine (VCD) and Encoding Compression Engine (ECE) present on Nuvoton NPCM SoCs. The VCD can capture and differentiate video data from digital or analog sources, then the ECE will compress the data into HEXTILE format. This driver implements V4L2 interfaces to support OpenBMC KVM feature, also tested with VNC Viewer and openbmc/obmc-ikvm to verify the video output result.

---
 MAINTAINERS                                 |    1 +
 drivers/media/platform/Kconfig              |    1 +
 drivers/media/platform/Makefile             |    1 +
 drivers/media/platform/nuvoton/Kconfig      |    9 +
 drivers/media/platform/nuvoton/Makefile     |    2 +
 drivers/media/platform/nuvoton/npcm-video.c | 2095 +++++++++++++++++++
 6 files changed, 2109 insertions(+)
 create mode 100644 drivers/media/platform/nuvoton/Kconfig
 create mode 100644 drivers/media/platform/nuvoton/Makefile
 create mode 100644 drivers/media/platform/nuvoton/npcm-video.c
MAINTAINERS +1 -0
index 66804a23cbf3..74d04beef20a 100644
@@ -2303,6 +2303,7 @@ F:	arch/arm/mach-npcm/
 F:	arch/arm64/boot/dts/nuvoton/
 F:	drivers/*/*npcm*
 F:	drivers/*/*/*npcm*
+F:	drivers/media/platform/nuvoton/
 F:	include/dt-bindings/clock/nuvoton,npcm7xx-clock.h
 F:	include/dt-bindings/clock/nuvoton,npcm8xx-clk.h
 
drivers/media/platform/Kconfig +1 -0
index 80321e03809a..d1e5e64784d8 100644
@@ -171,6 +171,7 @@ source "drivers/media/platform/xilinx/Kconfig"
 source "drivers/media/platform/rcar-vin/Kconfig"
 source "drivers/media/platform/atmel/Kconfig"
 source "drivers/media/platform/sunxi/Kconfig"
+source "drivers/media/platform/nuvoton/Kconfig"
 
 config VIDEO_TI_CAL
 	tristate "TI CAL (Camera Adaptation Layer) driver"
drivers/media/platform/Makefile +1 -0
index 73ce083c2fc6..693827b8b50f 100644
@@ -86,3 +86,4 @@ obj-$(CONFIG_VIDEO_QCOM_VENUS)		+= qcom/venus/
 obj-y					+= sunxi/
 
 obj-$(CONFIG_VIDEO_MESON_GE2D)		+= meson/ge2d/
+obj-y					+= nuvoton/
drivers/media/platform/nuvoton/Kconfig new file +9 -0
new file mode 100644
index 000000000000..bd3fb0e0ff25
@@ -0,0 +1,9 @@
+# SPDX-License-Identifier: GPL-2.0-only
+config VIDEO_NPCM_VCD_ECE
+	tristate "Nuvoton NPCM Video Capture/Encode Engine driver"
+	depends on V4L_PLATFORM_DRIVERS
+	depends on VIDEO_V4L2
+	select VIDEOBUF2_DMA_CONTIG
+	help
+	  Support for the Video Capture/Differentiation Engine (VCD) and
+	  Encoding Compression Engine (ECE) present on Nuvoton NPCM SoCs.
drivers/media/platform/nuvoton/Makefile new file +2 -0
new file mode 100644
index 000000000000..74a4e3fc8555
@@ -0,0 +1,2 @@
+# SPDX-License-Identifier: GPL-2.0-only
+obj-$(CONFIG_VIDEO_NPCM_VCD_ECE) += npcm-video.o
drivers/media/platform/nuvoton/npcm-video.c new file +2095 -0

2,095 lines not on this page · raw message

Signature · 1 line
2.34.1
#6 lore raw
In reply to Marvin Lin

Hello Marvin,

What is this patchset's relation to the previous patchset "Support Nuvoton NPCM Video Capture/Encode Engine"[1], which went up to v5? Why is this patchset at v1 again?

Confused,
Jonathan

[1]: https://lore.kernel.org/lkml/[email protected]/

Quoted 41 lines · Marvin Lin

This patch series add DTS node, dt-bindings document and drivers for Video Capture/Differentiation Engine (VCD) and Encoding Compression Engine (ECE) present on Nuvoton NPCM SoCs.

VCD can capture/differentiate video data from digital or analog sources, then the ECE will compress the data into HEXTILE format.

HEXTILE compressed format is defined in Remote Framebuffer Protocol (RFC 6143) and is used by VNC features, so we also add a patch to support it.

Marvin Lin (5):
arm64: dts: nuvoton: Add node for NPCM Video Capture/Encode Engine
dt-bindings: media: Add dt-bindings for NPCM Video Capture/Encode
Engine
dt-bindings: arm/npcm: Add dt-bindings for Graphics Core Information
media: Add HEXTILE compressed format
drivers: media: platform: Add NPCM Video Capture/Encode Engine driver

 .../bindings/arm/npcm/nuvoton,gfxi.yaml       |   41 +
 .../bindings/media/nuvoton,npcm-video.yaml    |   87 +
 .../media/v4l/pixfmt-reserved.rst             |    8 +
 MAINTAINERS                                   |    1 +
 .../dts/nuvoton/nuvoton-common-npcm8xx.dtsi   |   19 +
 .../boot/dts/nuvoton/nuvoton-npcm845-evb.dts  |   18 +
 drivers/media/platform/Kconfig                |    1 +
 drivers/media/platform/Makefile               |    1 +
 drivers/media/platform/nuvoton/Kconfig        |    9 +
 drivers/media/platform/nuvoton/Makefile       |    2 +
 drivers/media/platform/nuvoton/npcm-video.c   | 2095 +++++++++++++++++
 drivers/media/v4l2-core/v4l2-ioctl.c          |    1 +
 include/uapi/linux/videodev2.h                |    1 +
 13 files changed, 2284 insertions(+)
 create mode 100644 Documentation/devicetree/bindings/arm/npcm/nuvoton,gfxi.yaml
 create mode 100644 Documentation/devicetree/bindings/media/nuvoton,npcm-video.yaml
 create mode 100644 drivers/media/platform/nuvoton/Kconfig
 create mode 100644 drivers/media/platform/nuvoton/Makefile
 create mode 100644 drivers/media/platform/nuvoton/npcm-video.c
Signature · 1 line
2.34.1
#8 lore raw
Quoted 6 lines · Kun-Fa Lin

Hi Jonathan,

Previous patchset was aimed for torvalds/linux and we would also like to upstream to openbmc/llinux.
So this new v1 patchset is specifically aimed for openbmc/linux branch dev-6.0.
https://github.com/openbmc/linux/tree/dev-6.0

Ah, I missed that, thank you for the explanation!

To avoid such confusion, I'd suggest perhaps indicating the branch in the subject line, e.g. using [PATCH openbmc-6.0 v1 0/5] or similar instead of [PATCH v1 0/5]. "git format-patch" can do this with the --subject-prefix option.

Best regards,
Jonathan

author
#9 lore raw

Jonathan Neuschäfer <[email protected]> 於 2022年10月11日 週二 凌晨1:31寫道:

Quoted 6 lines

Ah, I missed that, thank you for the explanation!

To avoid such confusion, I'd suggest perhaps indicating the branch in the subject line, e.g. using [PATCH openbmc-6.0 v1 0/5] or similar instead of [PATCH v1 0/5]. "git format-patch" can do this with the --subject-prefix option.

Got it, sorry for the confusion and thanks for the suggestion!

Regards,
Marvin